Deep Ensembles (Lakshminarayanan et al., 2017) is a technique where multiple
deep learning models are independently trained from different initializations,
and their predictions are combined to make a final prediction. This approach
exploits the idea that different models will make different types of errors,
which can be reduced by averaging model predictions, leading to better overall
performance and more robust uncertainty estimates (Fort et al., 2019).
Formally, given $M$ models in the ensemble, each with parameters
$\bm{\theta}_{m},m\in\\{1,\dots,M\\}$, the ensemble prediction for an input
data point $\bm{x}$ is given by:

$\hat{y}=\frac{1}{M}\sum_{m=1}^{M}\hat{y}_{m}=\frac{1}{M}\sum_{m=1}^{M}f(\bm{x};\bm{\theta}_{m})$
(10)

where $f$ represents the model architecture, and $\hat{y}_{m}$ is the post-
activation result of the $m$-th model. We set $M=3$ for QM8, QM9 and MUV to
reduce computational consumption, and $M=10$ for other scaffold split
datasets. For random split, we uniformly use $M=3$, as mentioned in appendix
A.

For regression tasks, Lakshminarayanan et al. (2017) aggregate the variances
of different network predictions through parameterizing a Gaussian mixture
model. In contrast, we take a simpler approach by computing the mean of the
variances as the final output variance.

Table 7:  Computation resources required by different uncertainty estimation
methods. We assume that we have already trained a deterministic backbone model
for property prediction, and would like to build up a UQ method on top of it.

UQ Method | Training Starting Checkpoint |  Additional Cost(a)
---|---|---
Deterministic | - | $0$
Temperature | from fine-tuned backbone | $(T_{\text{infer}}+T_{\text{train-FFN}})\times M_{\text{train-extra}}$
Focal Loss | from scratch | $T_{\text{train}}\times M_{\text{train}}$
MC Dropout | no training | $T_{\text{infer}}\times M_{\text{infer}}$
SWAG | from fine-tuned backbone | $T_{\text{train}}\times M_{\text{train-extra}}+T_{\text{infer}}\times M_{\text{infer}}$
BBP | from scratch | $T_{\text{train}}\times M_{\text{train}}+T_{\text{infer}}\times M_{\text{infer}}$
SGLD | from scratch | $T_{\text{train}}\times(M_{\text{train}}+M_{\text{train-extra}})+T_{\text{infer}}\times M_{\text{infer}}$
Ensembles | from scratch | $T_{\text{train}}\times M_{\text{train}}\times(N_{\text{ensembles}}-1)$

  * (a)

$T_{\text{train}}$ and $T_{\text{infer}}$ are the time for one epoch of
training and inference of the backbone model, respectively. In general,
$T_{\text{train}}\gg T_{\text{infer}}$. $M_{\text{train}}$, $M_{\text{train-
extra}}$, and $M_{\text{infer}}$ are the number of training epochs, additional
training epochs, and inference epochs, respectively (§ C.1). In general,
$M_{\text{train}}\gg M_{\text{train-extra}}$. Different backbones and UQ
methods have different $T$s and $M$s, but we use the same symbols nonetheless
for simplicity. The result is a rough estimation without considering the
additional inference time or the early stopping if a model is retrained.

###  C.2 Resource Analysis

Table 7 summarizes the additional training cost to apply each UQ method to a
backbone model already fine-tuned for property prediction. From the table, we
can see that post-hoc calibration and MC Dropout are the most efficient
methods, while Deep Ensembles is undoubtedly the most expensive one, even
though it performs the best most of the time. Several works aim to reduce the
computational cost (Wen et al., 2020; Li et al., 2023b), but we do not
consider them in MUBen and leave them to future works.

##  Appendix D Metrics

In this section, we elaborate on the metrics utilized in our research for both
property prediction and uncertainty estimation.

###  D.1 Property Prediction Metrics

##### ROC-AUC

The receiver operating characteristic area under the curve (ROC-AUC) is widely
used for binary classification tasks. The ROC curve plots the true positive
rate (TPR), or recall, against the false positive rate (FPR) at various
decision thresholds $t\in(0,1)$. Given a set of true positives (TP), true
negatives (TN), false positives (FP), and false negatives (FN), the TPR and
FPR are computed as $\text{TPR}=\frac{\text{TP}}{\text{TP}+\text{FN}}$ and
$\text{FPR}=\frac{\text{FP}}{\text{FP}+\text{TN}}$ respectively. The AUC
signifies the likelihood of a randomly selected positive instance being ranked
above a randomly chosen negative instance. This integral under the ROC curve
is calculated as

$\text{ROC-AUC}=\int_{0}^{1}\text{TPR}(t)\frac{d}{dt}\text{FPR}(t)dt$ (11)

and can be approximated using numerical methods.999We use scikit-learn’s
roc_auc_score function in our implementation.

##### RMSE

For regression tasks, the root mean square error (RMSE) quantifies the average
discrepancy between predicted values $\hat{y}_{n}\in$ and actual values
$y_{n}\in$ for $N$ data points, given by:

$\text{RMSE}=\sqrt{\frac{1}{N}\sum_{n=1}^{N}(\hat{y}_{n}-y_{n})^{2}}.$ (12)

##### MAE

The mean absolute error (MAE) is another regression metric, measuring the
average absolute deviation between predicted and actual values. It is
calculated as:

$\text{MAE}=\frac{1}{N}\sum_{i=1}^{N}|\hat{y}_{n}-y_{n}|.$ (13)

###  D.2 Uncertainty Quantification Metrics

##### NLL

Negative Log-Likelihood (NLL) quantifies the mean deviation between predicted
and actual values in logarithmic space and is commonly used to evaluate UQ
performance (Gal & Ghahramani, 2016; Lakshminarayanan et al., 2017;
Gawlikowski et al., 2021).

For binary classification with Sigmoid output activation, NLL is given by:

$\text{NLL}=-\frac{1}{N}\sum_{n=1}^{N}\left[y_{n}\log\hat{p}_{n}+(1-y_{n})\log(1-\hat{p}_{n})\right],$
(14)

where $y_{n}\in\\{0,1\\}$ is the true label and
$\hat{p}_{n}\in(0,1)=p_{\bm{\theta}}(\hat{y}_{n}\mid\bm{x}_{n})$ is the
predicted probability.

For regression, the Gaussian NLL is calculated as:

$\text{NLL}=-\frac{1}{N}\sum_{n=1}^{N}\log\mathcal{N}(y_{n};\hat{y}_{n},\hat{\sigma}_{n})=\frac{1}{N}\sum_{n=1}^{N}\frac{1}{2}\left[\log(2\pi\hat{\sigma}_{n})+\frac{(y_{n}-\hat{y}_{n})^{2}}{\hat{\sigma}_{n}}\right],$
(15)

where $\hat{y}_{n}\in$ is the predicted mean and $\hat{\sigma}_{n}\in_{+}$ is
the predicted variance regularized by the SoftPlus activation.

##### Brier Score

In classification, the Brier Score (BS) is a proper scoring rule for measuring
the accuracy of predicted probabilities. Similar to mean square error (MSE) in
regression, it measures the mean squared difference between predicted
probabilities and actual binary outcomes. For binary classification, the Brier
Score is calculated as

$\text{Brier Score}=\frac{1}{N}\sum_{i=1}^{N}(\hat{p}_{n}-y_{n})^{2},$ (16)

where $y_{n}\in\\{0,1\\}$ and $\hat{p}_{n}\in(0,1)$.

The Brier score provides an insightful decomposition as
$\text{BS}=\text{Uncertainty}-\text{Resolution}+\text{Reliability}$ (Murphy,
1973). “Uncertainty” corresponds to the inherent variability over labels;
“Resolution” quantifies the deviation of individual predictions from the
average; and “Reliability” gauges the extent to which predicted probabilities
align with the actual probabilities.

##### ECE

Expected Calibration Error (ECE) measures the average difference between
predicted probabilities and their corresponding empirical probabilities, whose
calculation is presented in § 3 and rewritten here specifically for the binary
classification case.

Binary ECE puts the predicted probabilities $\\{\hat{p}_{n}\\}_{n=1}^{N}$ into
$S$ (which is set to $S=15$ in our experiments) equal-width bins
$\mathbb{B}_{s}=\\{n\in\\{1,\dots,N\\}\mid\hat{p}_{n}\in(\rho_{s},\rho_{s+1}]\\}$
with $s\in\\{1,\dots,S\\}$. For binary classification, ECE is calculated as

$\begin{gathered}\text{ECE}=\sum_{s=1}^{S}\frac{|\mathbb{B}_{s}|}{N}\left|\text{acc}(\mathbb{B}_{s})-\text{conf}(\mathbb{B}_{s})\right|;\\\
\text{acc}(\mathbb{B}_{s})=\frac{1}{|\mathbb{B}_{s}|}\sum_{n\in\mathbb{B}_{s}}y_{n};\quad\quad\text{conf}(\mathbb{B}_{s})=\frac{1}{|\mathbb{B}_{s}|}\sum_{n\in\mathbb{B}_{s}}\hat{p}_{n}.\\\
\end{gathered}$ (17)

Here, $\text{acc}(\cdot)$ is the predicion accuracy and $\text{conf}(\cdot)$
is the confidence level within each bin. Notice that $|\cdot|$ represents the
size when used with a set, and the absolute value when its input is a real
number.

##### Regression CE

Regression calibration is less studied than classification calibration, and
Regression Calibration Error (CE), proposed by Kuleshov et al. (2018), is
adopted in only a few works (Song et al., 2019; Kamarthi et al., 2021).
Similar to ECE, CE measures the average difference between the observed
confidence level and the expected confidence level within each bin.
Specifically, a confidence level of $\alpha$ is measured as the fraction of
data points whose true value falls within $\frac{1-\alpha}{2}$th quantile and
$(1-\frac{\alpha}{2})$th quantile of the predicted distribution. For example,
$90\%$ confidence level contains data points with true values falling within
the $5\%$th quantile and $95\%$th quantile of the predicted distribution. With
this established, we can calculate the confidence levels with the predicted
quantile function, in our case, the cumulative distribution function (CDF) of
the predicted Gaussian. Specifically,

$\text{CE}=\frac{1}{S}\sum_{s=1}^{S}\left(\rho_{s}-\frac{1}{N}\left|\left\\{n\in\\{1,\dots,N\\}\mid\Phi(y_{n};\hat{y}_{n},\hat{\sigma}_{n})\leq\rho_{s}\right\\}\right|\right)^{2}$
(18)

where $\rho_{s}=\frac{s}{S}$ is the expected quantile, $\Phi$ is the Gaussian
CDF:

$\begin{gathered}\Phi(y_{n};\hat{y}_{n},\hat{\sigma}_{n})=\frac{1}{2}\left(1+\text{erf}\left(\frac{y_{n}-\hat{y}_{n}}{\sqrt{2}\hat{\sigma}_{n}}\right)\right);\\\
\text{erf}(z)=\frac{2}{\sqrt{\pi}}\int_{0}^{z}e^{-t^{2}}dt.\end{gathered}$
(19)

$\hat{p}_{s}=\frac{1}{N}\left|\left\\{n\in\\{1,\dots,N\\}\mid\Phi(y_{n};\hat{y}_{n},\hat{\sigma}_{n})\leq\rho_{s}\right\\}\right|$
is the empirical frequency of the predicted values falling into the
$\rho_{s}$th quantile. This is slightly different from the confidence level-
based calculation, where

$\text{CE}=\frac{2}{S}\sum_{s=1}^{\frac{S}{2}}\left(\left(\rho_{S-s}-\rho_{s}\right)-\left(\hat{p}_{S-s}-\hat{p}_{s}\right)\right)^{2},$
(20)

but it reveals the same trend. We use $S=20$ in our experiments.

##  Appendix E Additional Results

In this section, we supplement the discussions from § 5 with further results
and analyses. Given the extensive volume of outcomes from our experiments, we
have consolidated them into tables at the end of this article. For enhanced
accessibility and offline analysis, we have also made these results available
as CSV files in our GitHub repository.101010https://github.com/Yinghao-
Li/MUBen/tree/main/reports.

###  E.1 More Visualization of Uncertainty Estimation Performance

Table 8 displays the average rankings of UQ methods, complemented by Figure 7
which offers a visual representation. As elaborated in § 5, both Deep
Ensembles and MC Dropout generally enhance prediction and uncertainty
estimation across tasks. However, the performance boost is more noticeable
with Deep Ensembles compared to MC Dropout, while DC Dropout is much cheaper
to apply. For classification tasks, Temperature Scaling offers moderate
efficacy. Given the minimal computational cost of implementing Temperature
Scaling, it emerges as a favorable option for calibration enhancement when
necessary. In regression tasks, both BBP and SGLD stand out as reliable
choices irrespective of the backbone model, particularly for uncertainty
estimation. Nonetheless, they mandate a full re-training. Conversely, SWAG and
Focal Loss are underwhelming and warrant reconsideration in selection
processes while dealing with molecular property prediction.

###  E.2 Comprehensive Fine-Tuning Results

Tables 10–17 illustrate scores associated with each backbone-UQ pairing across
the scaffold split classification datasets, while Tables 18–23 detail the
scores regarding regression metrics. Scores are represented in a “mean
(standard deviation)” structure, where both mean and standard deviation values
are derived from $3$ experiment runs conducted with random seeds $0$, $1$, and
$2$. As Deep Ensembles aggregates network predictions prior to metric
calculation, standard deviation values are not generated and metrics are
calculated only once.

Challenges are encountered during training on the notably imbalanced HIV and
MUV dataset, depicted in Table 4. SGLD at times proves unsuccessful due to
weak directional cues from the true labels, overwhelmed by the random noise
embedded in the Langevin dynamics. With TorchMD-NET backbone, SGLD even fails
training and constantly produces NaN values regardless of the hyperparameters
we choose. Conversely, the application of Focal Loss demonstrates effective
performance on the MUV dataset, aligning with assertions in Lin et al. (2017).
ECE also may fail to compute on these datasets as predicted probabilities
$\hat{p}$ are excessively minimal, leading to unpopulated bins and subsequent
“division by zero” errors.

In terms of the majority of datasets, the performance trends closely adhere to
our previous discussions. However, an anomaly is observed with the BBBP and
ClinTox datasets, where ChemBERTa produces higher performance than the more
powerful models, deviating from prior findings (Ahmad et al., 2022; Zhou et
al., 2023). Conceptually, the ChemBERTa checkpoint we use is solely pre-
trained on the MLM objective, which is unrelated to downstream tasks and
should not include label information. Consequently, we hypothesize that either
the SMILES strings are potentially more expressive than other descriptors with
respect to the physiological properties given that the ChemBERTa’s
architectures is not considered as more expressive than GROVER or Uni-Mol.

Table 8:  Averaged ranks of UQ methods across all scaffold-split datasets and backbone models. Lower value indicates better resuls. Temperature Scaling and Focal Loss are not applicable to regression tasks.  | Classification | Regression
---|---|---
ROC-AUC | ECE | NLL | BS | RMSE | MAE | NLL | CE
Deterministic | 16.12 | 15.41 | 17.03 | 16.53 | 12.75 | 12.21 | 16.12 | 15.58
Temperature | 16.31 | 12.06 | 13.97 | 14.38 | - | - | - | -
Focal Loss | 17.12 | 22.84 | 20.41 | 21.97 | - | - | - | -
MC Dropout | 15.78 | 13.53 | 14.28 | 13.62 | 10.71 | 10.83 | 14.75 | 15.0
SWAG | 15.47 | 17.94 | 19.62 | 17.69 | 14.0 | 13.92 | 18.58 | 17.21
BBP | 18.97 | 20.28 | 17.0 | 18.88 | 14.04 | 14.42 | 8.29 | 7.67
SGLD | 20.72 | 20.66 | 20.59 | 19.22 | 14.25 | 14.75 | 5.87 | 5.54
Ensembles | 11.5 | 9.28 | 9.09 | 9.72 | 9.25 | 8.88 | 11.38 | 14.0

(a) Classification MRR$\uparrow$ (b) Regression MRR$\uparrow$

Figure 7:  MRR of the UQ methods for each metric, each is macro-averaged from
the reciprocal ranks of the results of all corresponding backbone models on
all datasets.

###  E.3 Results on Random Split Datasets

Tables 24 to 31 illustrate the efficacy of primary backbone models across
datasets with random divisions. Mirroring the discussion in § E.1, Table 9 and
Figure 8 depict the average (reciprocal) rankings of UQ methods. For the most
part, the comparative performance among the backbone models and UQ methods is
consistent with those observed in scaffold split datasets. Nonetheless,
certain exceptions emerge, especially concerning BBP, SGLD, and SWAG. On
randomly split classification datasets, both BBP and SGLD continue to
underperform, while a marked decline is noted in their regression NLL. This
phenomenon might be attributed to the balanced nature of random split
datasets, intensifying the predictive indications from actual labels. Given
this environment, the noise from Langevin dynamics or Gaussian samples, which
usually aids generalization, might become detrimental. This revelation
highlights the capability of traditional Bayesian methods in aptly navigating
OOD data (i.e., addressing epistemic uncertainty), mainly through curbing
overfitting. However, they may struggle when dealing with in-distribution
attributes (i.e., addressing aleatoric uncertainty) having relatively stable
labels in the context of molecular property prediction. On the other hand,
SWAG, another approximation of the Bayesian network that is almost useless on
OOD data, delivers a much more impressive performance on random split
datasets. Deep Ensembles, MC Dropout, and Temperature Scaling still emerge as
the most reliable UQ methods, although Deep Ensembles no longer dominates the
benchmark in the in-distribution case.

Upon comparing Figure 9 with Figure 2, it’s evident that the relative
positions of the backbone models remain largely unchanged for regression
datasets across both splitting methods; however, this consistency falters for
classification datasets. Notably, Uni-Mol displays a marked performance
decline in the randomly split datasets, positioning it below the more
straightforward and cost-efficient backbones, ChemBERTa and GROVER. While
definitive explanations for this observed behavior remain elusive, we
hypothesize that the molecular descriptor or the pre-training data inherent to
ChemBERTa and GROVER could render them better equipped to adapt to
Physiological and Biophysical features. Such potential advantages might be
obscured by their model architectures’ suboptimal generalization capabilities
when scaffold splitting is employed.

Table 9:  Averaged ranks of UQ methods across all randomly split datasets. Lower rank indicates better performance.  | Classification | Regression
---|---|---
ROC-AUC | ECE | NLL | BS | RMSE | MAE | NLL | CE
Deterministic | 16.44 | 16.31 | 17.06 | 16.5 | 13.25 | 12.69 | 12.62 | 15.88
Temperature | 16.88 | 14.0 | 13.06 | 14.5 | - | - | - | -
Focal Loss | 17.31 | 28.75 | 23.62 | 26.56 | - | - | - | -
MC Dropout | 14.88 | 9.56 | 12.88 | 13.75 | 10.81 | 10.94 | 10.75 | 15.56
SWAG | 13.81 | 15.0 | 15.06 | 14.0 | 10.44 | 10.31 | 11.19 | 16.44
BBP | 19.94 | 16.12 | 18.12 | 17.31 | 16.56 | 17.5 | 15.94 | 5.44
SGLD | 18.44 | 18.81 | 19.0 | 17.56 | 14.12 | 14.12 | 14.19 | 6.69
Ensembles | 14.31 | 13.44 | 13.19 | 11.81 | 9.81 | 9.44 | 10.31 | 15.0

(a) Classification MRR$\uparrow$ (b) Regression MRR$\uparrow$

Figure 8:  MRR of the UQ methods for each metric, each is macro-averaged from
the reciprocal ranks of the results of all corresponding backbone models on
all randomly split datasets.

(a) Classification MRR$\uparrow$ (b) Regression MRR$\uparrow$

Figure 9:  MRR of the UQ methods for each backbone model, each is macro-
averaged from the reciprocal ranks of the results of all corresponding UQ
methods on all randomly split datasets.

###  E.4 Results with Frozen Backbone Models

Analysis of Tables 32 through 39 reveals that when the network parameters of
the backbone models are fixed, all models underperform significantly. Among
the three primary backbone models, Uni-Mol experiences the most drastic
decline, emerging as the least effective in the majority of evaluations. These
findings highlight that these backbone models are not inherently optimized for
mere feature extraction. The fine-tuning process is pivotal for ensuring
optimal model performance, irrespective of the use of UQ methods.

(a) DNN RMSE $\downarrow$

(b) DNN MAE $\downarrow$

(c) DNN NLL $\downarrow$

(d) DNN CE $\downarrow$

(e) ChemBERTa RMSE $\downarrow$

(f) ChemBERTa MAE $\downarrow$

(g) ChemBERTa NLL $\downarrow$

(h) ChemBERTa CE $\downarrow$

(i) GROVER RMSE $\downarrow$

(j) GROVER MAE $\downarrow$

(k) GROVER NLL $\downarrow$

(l) GROVER CE $\downarrow$

(m) Uni-Mol RMSE $\downarrow$

(n) Uni-Mol MAE $\downarrow$

(o) Uni-Mol NLL $\downarrow$

(p) Uni-Mol CE $\downarrow$

Figure 10:  Performance of backbone and UQ methods according to the similarity between test data points and training scaffolds in the QM9 dataset.  Table 10: Test results on BACE in the format of “metric mean $\pm$ standard deviation”. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.8185 $\pm$ 0.0164 |  0.2827 $\pm$ 0.0155 |  0.8946 $\pm$ 0.0797 |  0.2810 $\pm$ 0.0125
Temperature |  0.8185 $\pm$ 0.0164 |  0.2537 $\pm$ 0.0144 |  0.7412 $\pm$ 0.0472 |  0.2553 $\pm$ 0.0111
Focal Loss |  0.8190 $\pm$ 0.0112 |  0.2543 $\pm$ 0.0011 |  0.6908 $\pm$ 0.0099 |  0.2488 $\pm$ 0.0027
MC Dropout |  0.8168 $\pm$ 0.0121 |  0.2648 $\pm$ 0.0323 |  0.7667 $\pm$ 0.0446 |  0.2589 $\pm$ 0.0157
SWAG |  0.8173 $\pm$ 0.0119 |  0.2728 $\pm$ 0.0068 |  0.8730 $\pm$ 0.0643 |  0.2730 $\pm$ 0.0085
BBP |  0.8288 $\pm$ 0.0039 |  0.2711 $\pm$ 0.0193 |  0.7943 $\pm$ 0.0235 |  0.2535 $\pm$ 0.0075
SGLD |  0.8181 $\pm$ 0.0160 |  0.2713 $\pm$ 0.0024 |  0.9023 $\pm$ 0.0416 |  0.2734 $\pm$ 0.0073
Ensembles |  0.8207 $\pm$ - |  0.2550 $\pm$ - |  0.7985 $\pm$ - |  0.2557 $\pm$ -
ChemBERTa | Deterministic |  0.7223 $\pm$ 0.0089 |  0.2407 $\pm$ 0.0135 |  0.7567 $\pm$ 0.0203 |  0.2720 $\pm$ 0.0081
Temperature |  0.7223 $\pm$ 0.0089 | 0.1758 $\pm$ 0.0059 |  0.6937 $\pm$ 0.0118 |  0.2498 $\pm$ 0.0052
Focal Loss |  0.6985 $\pm$ 0.0062 |  0.2140 $\pm$ 0.0070 |  0.7322 $\pm$ 0.0051 |  0.2679 $\pm$ 0.0023
MC Dropout |  0.7613 $\pm$ 0.0216 |  0.2391 $\pm$ 0.0262 |  0.7464 $\pm$ 0.0257 |  0.2673 $\pm$ 0.0070
SWAG |  0.7580 $\pm$ 0.0139 |  0.2539 $\pm$ 0.0091 |  0.7707 $\pm$ 0.0374 |  0.2654 $\pm$ 0.0139
BBP |  0.7372 $\pm$ 0.0233 |  0.2767 $\pm$ 0.0032 |  0.8325 $\pm$ 0.0190 |  0.2878 $\pm$ 0.0046
SGLD |  0.7775 $\pm$ 0.0205 |  0.2492 $\pm$ 0.0217 |  0.7922 $\pm$ 0.0283 |  0.2588 $\pm$ 0.0069
Ensembles |  0.7350 $\pm$ - |  0.2352 $\pm$ - |  0.7457 $\pm$ - |  0.2683 $\pm$ -
GROVER | Deterministic |  0.8308 $\pm$ 0.0012 |  0.2304 $\pm$ 0.0658 |  0.6921 $\pm$ 0.1114 |  0.2342 $\pm$ 0.0386
Temperature |  0.8305 $\pm$ 0.0017 |  0.2083 $\pm$ 0.0521 | 0.6251 $\pm$ 0.0671 |  0.2184 $\pm$ 0.0283
Focal Loss |  0.8509 $\pm$ 0.0072 |  0.2334 $\pm$ 0.0302 |  0.6397 $\pm$ 0.0300 |  0.2281 $\pm$ 0.0141
MC Dropout |  0.8278 $\pm$ 0.0011 |  0.2366 $\pm$ 0.0629 |  0.6899 $\pm$ 0.1094 |  0.2368 $\pm$ 0.0394
SWAG |  0.8604 $\pm$ 0.0048 |  0.2239 $\pm$ 0.0016 |  0.6656 $\pm$ 0.0169 |  0.2148 $\pm$ 0.0030
BBP |  0.8322 $\pm$ 0.0099 |  0.2570 $\pm$ 0.0331 |  0.7001 $\pm$ 0.0583 |  0.2450 $\pm$ 0.0228
SGLD |  0.8494 $\pm$ 0.0028 |  0.2281 $\pm$ 0.0066 |  0.6769 $\pm$ 0.0144 |  0.2204 $\pm$ 0.0038
Ensembles |  0.8388 $\pm$ - |  0.2288 $\pm$ - |  0.6571 $\pm$ - |  0.2252 $\pm$ -
Uni-Mol | Deterministic |  0.8080 $\pm$ 0.0266 |  0.2419 $\pm$ 0.0409 |  0.7928 $\pm$ 0.1639 |  0.2517 $\pm$ 0.0335
Temperature |  0.8087 $\pm$ 0.0277 |  0.2125 $\pm$ 0.0259 |  0.6782 $\pm$ 0.0662 |  0.2352 $\pm$ 0.0212
Focal Loss |  0.8208 $\pm$ 0.0259 |  0.2445 $\pm$ 0.0287 |  0.6721 $\pm$ 0.0482 |  0.2406 $\pm$ 0.0198
MC Dropout |  0.8268 $\pm$ 0.0269 |  0.2483 $\pm$ 0.0333 |  0.8065 $\pm$ 0.1383 |  0.2512 $\pm$ 0.0331
SWAG |  0.8193 $\pm$ 0.0106 |  0.2677 $\pm$ 0.0075 |  0.9415 $\pm$ 0.0623 |  0.2625 $\pm$ 0.0058
BBP |  0.8386 $\pm$ 0.0135 |  0.2673 $\pm$ 0.0473 |  0.8158 $\pm$ 0.1584 |  0.2588 $\pm$ 0.0375
SGLD | 0.8699 $\pm$ 0.0118 |  0.2437 $\pm$ 0.0131 |  0.7354 $\pm$ 0.0502 |  0.2244 $\pm$ 0.0150
Ensembles |  0.8505 $\pm$ - |  0.2176 $\pm$ - |  0.6301 $\pm$ - |  0.2184 $\pm$ -
TorchMD-NET | Deterministic |  0.7945 $\pm$ 0.0232 |  0.2193 $\pm$ 0.0374 |  1.1663 $\pm$ 0.6182 |  0.2272 $\pm$ 0.0201
Temperature |  0.7945 $\pm$ 0.0231 |  0.1808 $\pm$ 0.0425 |  0.7151 $\pm$ 0.1720 | 0.2085 $\pm$ 0.0187
Focal Loss |  0.7602 $\pm$ 0.0089 |  0.2141 $\pm$ 0.0249 |  0.7328 $\pm$ 0.0376 |  0.2420 $\pm$ 0.0200
MC Dropout |  0.8018 $\pm$ 0.0260 |  0.2017 $\pm$ 0.0297 |  0.8454 $\pm$ 0.2767 |  0.2178 $\pm$ 0.0199
SWAG |  0.7457 $\pm$ 0.0208 |  0.3287 $\pm$ 0.0164 |  1.9159 $\pm$ 0.3026 |  0.3172 $\pm$ 0.0122
BBP |  0.7638 $\pm$ 0.0374 |  0.3130 $\pm$ 0.0457 |  2.8823 $\pm$ 1.2929 |  0.3035 $\pm$ 0.0425
SGLD |  0.7787 $\pm$ 0.0173 |  0.2793 $\pm$ 0.0199 |  1.1648 $\pm$ 0.2135 |  0.2731 $\pm$ 0.0226
Ensembles |  0.8169 $\pm$ - |  0.2220 $\pm$ - |  0.6703 $\pm$ - |  0.2197 $\pm$ -
GIN | Deterministic |  0.7312 $\pm$ 0.0152 |  0.4024 $\pm$ 0.0362 |  1.5262 $\pm$ 0.4443 |  0.3946 $\pm$ 0.0340
Temperature |  0.7312 $\pm$ 0.0152 |  0.3494 $\pm$ 0.0392 |  1.0850 $\pm$ 0.1950 |  0.3482 $\pm$ 0.0285
Focal Loss |  0.7272 $\pm$ 0.0374 |  0.3034 $\pm$ 0.1213 |  1.2375 $\pm$ 0.6434 |  0.3264 $\pm$ 0.0917
MC Dropout |  0.7312 $\pm$ 0.0152 |  0.4024 $\pm$ 0.0362 |  1.5262 $\pm$ 0.4443 |  0.3946 $\pm$ 0.0340
SWAG |  0.5554 $\pm$ 0.0075 |  0.1929 $\pm$ 0.0326 |  0.7817 $\pm$ 0.0442 |  0.2839 $\pm$ 0.0156
BBP |  0.6775 $\pm$ 0.0533 |  0.2567 $\pm$ 0.0644 |  1.3036 $\pm$ 0.7463 |  0.2842 $\pm$ 0.0546
SGLD |  0.6682 $\pm$ 0.0302 |  0.2271 $\pm$ 0.1028 |  0.8664 $\pm$ 0.2399 |  0.2887 $\pm$ 0.0569
Ensembles |  0.7767 $\pm$ - |  0.2051 $\pm$ - |  0.7157 $\pm$ - |  0.2469 $\pm$ -
Table 11: Test results on BBBP in the format of “metric mean $\pm$ standard deviation”. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.6721 $\pm$ 0.0129 |  0.3292 $\pm$ 0.0517 |  3.8147 $\pm$ 4.0782 |  0.3423 $\pm$ 0.0378
Temperature |  0.6719 $\pm$ 0.0129 |  0.2890 $\pm$ 0.0742 |  2.1338 $\pm$ 1.8963 |  0.3186 $\pm$ 0.0489
Focal Loss |  0.6693 $\pm$ 0.0047 |  0.1689 $\pm$ 0.1056 |  0.8925 $\pm$ 0.3424 |  0.2623 $\pm$ 0.0470
MC Dropout |  0.6669 $\pm$ 0.0151 |  0.3287 $\pm$ 0.0402 |  2.1699 $\pm$ 1.7478 |  0.3391 $\pm$ 0.0296
SWAG |  0.6777 $\pm$ 0.0136 |  0.3020 $\pm$ 0.0694 |  4.8415 $\pm$ 5.5407 |  0.3268 $\pm$ 0.0465
BBP |  0.6856 $\pm$ 0.0075 |  0.3392 $\pm$ 0.0506 |  1.8126 $\pm$ 0.6030 |  0.3463 $\pm$ 0.0357
SGLD |  0.6922 $\pm$ 0.0125 |  0.3134 $\pm$ 0.0487 |  1.7569 $\pm$ 1.0592 |  0.3247 $\pm$ 0.0396
Ensembles |  0.7018 $\pm$ - |  0.2615 $\pm$ - |  0.9043 $\pm$ - |  0.2938 $\pm$ -
ChemBERTa | Deterministic |  0.7407 $\pm$ 0.0101 |  0.2259 $\pm$ 0.0206 |  1.0306 $\pm$ 0.1002 |  0.2559 $\pm$ 0.0090
Temperature |  0.7407 $\pm$ 0.0101 |  0.2107 $\pm$ 0.0196 |  0.8539 $\pm$ 0.0266 |  0.2453 $\pm$ 0.0064
Focal Loss |  0.7410 $\pm$ 0.0122 |  0.1639 $\pm$ 0.0118 |  0.6995 $\pm$ 0.0246 | 0.2313 $\pm$ 0.0087
MC Dropout | 0.7451 $\pm$ 0.0067 |  0.2082 $\pm$ 0.0035 |  0.8814 $\pm$ 0.0292 |  0.2418 $\pm$ 0.0032
SWAG |  0.7408 $\pm$ 0.0112 |  0.2373 $\pm$ 0.0202 |  1.1435 $\pm$ 0.0810 |  0.2629 $\pm$ 0.0109
BBP |  0.7245 $\pm$ 0.0112 |  0.2415 $\pm$ 0.0293 |  0.9653 $\pm$ 0.1062 |  0.2652 $\pm$ 0.0120
SGLD |  0.7359 $\pm$ 0.0104 |  0.2398 $\pm$ 0.0256 |  1.0692 $\pm$ 0.1940 |  0.2580 $\pm$ 0.0056
Ensembles |  0.7399 $\pm$ - |  0.2290 $\pm$ - |  0.9672 $\pm$ - |  0.2506 $\pm$ -
GROVER | Deterministic |  0.7075 $\pm$ 0.0061 |  0.2902 $\pm$ 0.0111 |  0.9457 $\pm$ 0.0234 |  0.2998 $\pm$ 0.0060
Temperature |  0.7072 $\pm$ 0.0059 |  0.2685 $\pm$ 0.0147 |  0.8607 $\pm$ 0.0258 |  0.2865 $\pm$ 0.0069
Focal Loss |  0.7146 $\pm$ 0.0059 | 0.1340 $\pm$ 0.0177 |  0.6505 $\pm$ 0.0070 |  0.2284 $\pm$ 0.0040
MC Dropout |  0.7066 $\pm$ 0.0071 |  0.2901 $\pm$ 0.0147 |  0.9267 $\pm$ 0.0285 |  0.2996 $\pm$ 0.0071
SWAG |  0.7160 $\pm$ 0.0032 |  0.3314 $\pm$ 0.0032 |  1.1105 $\pm$ 0.0142 |  0.3251 $\pm$ 0.0025
BBP |  0.6968 $\pm$ 0.0152 |  0.2735 $\pm$ 0.0209 |  0.9146 $\pm$ 0.0326 |  0.2986 $\pm$ 0.0055
SGLD |  0.7266 $\pm$ 0.0037 |  0.3127 $\pm$ 0.0065 |  1.0187 $\pm$ 0.0202 |  0.3149 $\pm$ 0.0019
Ensembles |  0.7083 $\pm$ - |  0.2890 $\pm$ - |  0.9338 $\pm$ - |  0.3029 $\pm$ -
Uni-Mol | Deterministic |  0.7103 $\pm$ 0.0075 |  0.3311 $\pm$ 0.0506 |  1.4209 $\pm$ 0.3000 |  0.3354 $\pm$ 0.0397
Temperature |  0.7093 $\pm$ 0.0083 |  0.2922 $\pm$ 0.0418 |  1.0023 $\pm$ 0.1064 |  0.3084 $\pm$ 0.0315
Focal Loss |  0.7102 $\pm$ 0.0045 |  0.2031 $\pm$ 0.0816 |  0.7646 $\pm$ 0.0923 |  0.2632 $\pm$ 0.0275
MC Dropout |  0.7029 $\pm$ 0.0142 |  0.3328 $\pm$ 0.0483 |  1.4086 $\pm$ 0.2828 |  0.3382 $\pm$ 0.0372
SWAG |  0.7247 $\pm$ 0.0062 |  0.3547 $\pm$ 0.0136 |  1.8530 $\pm$ 0.0824 |  0.3456 $\pm$ 0.0078
BBP |  0.6902 $\pm$ 0.0286 |  0.3420 $\pm$ 0.0318 |  1.3548 $\pm$ 0.0713 |  0.3463 $\pm$ 0.0275
SGLD |  0.7147 $\pm$ 0.0118 |  0.3336 $\pm$ 0.0127 |  1.4174 $\pm$ 0.0475 |  0.3359 $\pm$ 0.0059
Ensembles |  0.7253 $\pm$ - |  0.3035 $\pm$ - |  1.0889 $\pm$ - |  0.3035 $\pm$ -
TorchMD-NET | Deterministic |  0.6753 $\pm$ 0.0060 |  0.3207 $\pm$ 0.0249 |  1.7352 $\pm$ 0.2714 |  0.3299 $\pm$ 0.0218
Temperature |  0.6753 $\pm$ 0.0060 |  0.2739 $\pm$ 0.0251 |  1.1382 $\pm$ 0.1017 |  0.2995 $\pm$ 0.0188
Focal Loss |  0.7060 $\pm$ 0.0206 |  0.1910 $\pm$ 0.0200 |  0.8688 $\pm$ 0.0498 |  0.2487 $\pm$ 0.0119
MC Dropout |  0.6767 $\pm$ 0.0057 |  0.3098 $\pm$ 0.0236 |  1.4864 $\pm$ 0.1962 |  0.3190 $\pm$ 0.0177
SWAG |  0.6720 $\pm$ 0.0131 |  0.3601 $\pm$ 0.0121 |  3.1976 $\pm$ 0.5027 |  0.3632 $\pm$ 0.0090
BBP |  0.6895 $\pm$ 0.0166 |  0.3019 $\pm$ 0.0598 |  3.1956 $\pm$ 1.7451 |  0.3159 $\pm$ 0.0343
SGLD |  0.6708 $\pm$ 0.0058 |  0.3629 $\pm$ 0.0083 |  2.3208 $\pm$ 0.5515 |  0.3617 $\pm$ 0.0068
Ensembles |  0.6844 $\pm$ - |  0.2700 $\pm$ - |  1.1018 $\pm$ - |  0.2934 $\pm$ -
GIN | Deterministic |  0.6282 $\pm$ 0.0025 |  0.2277 $\pm$ 0.0302 |  1.0566 $\pm$ 0.0130 |  0.2920 $\pm$ 0.0078
Temperature |  0.6281 $\pm$ 0.0025 |  0.1987 $\pm$ 0.0164 |  0.9092 $\pm$ 0.0166 |  0.2778 $\pm$ 0.0051
Focal Loss |  0.6234 $\pm$ 0.0081 |  0.1550 $\pm$ 0.0116 | 0.7581 $\pm$ 0.0143 |  0.2611 $\pm$ 0.0043
MC Dropout |  0.6282 $\pm$ 0.0025 |  0.2277 $\pm$ 0.0302 |  1.0566 $\pm$ 0.0130 |  0.2920 $\pm$ 0.0078
SWAG |  0.6067 $\pm$ 0.0131 |  0.3162 $\pm$ 0.0021 |  1.2223 $\pm$ 0.0235 |  0.3353 $\pm$ 0.0056
BBP |  0.6100 $\pm$ 0.0095 |  0.2738 $\pm$ 0.0056 |  1.1617 $\pm$ 0.0529 |  0.3119 $\pm$ 0.0058
SGLD |  0.5577 $\pm$ 0.0497 |  0.3493 $\pm$ 0.0217 |  1.4064 $\pm$ 0.1006 |  0.3681 $\pm$ 0.0218
Ensembles |  0.6298 $\pm$ - |  0.2112 $\pm$ - |  0.9696 $\pm$ - |  0.2802 $\pm$ -
Table 12: Test results on ClinTox in the format of “metric mean $\pm$ standard deviation”. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.8301 $\pm$ 0.0226 |  0.0637 $\pm$ 0.0126 |  0.2719 $\pm$ 0.0481 |  0.0597 $\pm$ 0.0052
Temperature |  0.8300 $\pm$ 0.0224 |  0.0547 $\pm$ 0.0109 |  0.2371 $\pm$ 0.0059 |  0.0572 $\pm$ 0.0039
Focal Loss |  0.8507 $\pm$ 0.0165 |  0.0812 $\pm$ 0.0368 |  0.2284 $\pm$ 0.0305 |  0.0639 $\pm$ 0.0114
MC Dropout |  0.8194 $\pm$ 0.0217 |  0.0445 $\pm$ 0.0055 |  0.2149 $\pm$ 0.0307 |  0.0534 $\pm$ 0.0005
SWAG |  0.8290 $\pm$ 0.0163 |  0.0590 $\pm$ 0.0117 |  0.2964 $\pm$ 0.0617 |  0.0591 $\pm$ 0.0054
BBP |  0.7541 $\pm$ 0.0102 |  0.0574 $\pm$ 0.0121 |  0.2439 $\pm$ 0.0276 |  0.0609 $\pm$ 0.0046
SGLD |  0.8335 $\pm$ 0.0109 |  0.0567 $\pm$ 0.0125 |  0.2769 $\pm$ 0.0452 |  0.0553 $\pm$ 0.0063
Ensembles |  0.8154 $\pm$ - |  0.0283 $\pm$ - |  0.1860 $\pm$ - |  0.0484 $\pm$ -
ChemBERTa | Deterministic |  0.9856 $\pm$ 0.0028 |  0.0215 $\pm$ 0.0035 |  0.1206 $\pm$ 0.0318 |  0.0225 $\pm$ 0.0016
Temperature |  0.9856 $\pm$ 0.0028 |  0.0199 $\pm$ 0.0026 |  0.0964 $\pm$ 0.0147 |  0.0222 $\pm$ 0.0013
Focal Loss |  0.9832 $\pm$ 0.0027 |  0.0897 $\pm$ 0.0378 |  0.1367 $\pm$ 0.0350 |  0.0273 $\pm$ 0.0045
MC Dropout |  0.9848 $\pm$ 0.0021 |  0.0212 $\pm$ 0.0019 |  0.1083 $\pm$ 0.0247 |  0.0213 $\pm$ 0.0021
SWAG |  0.9850 $\pm$ 0.0031 |  0.0222 $\pm$ 0.0025 |  0.1327 $\pm$ 0.0317 |  0.0232 $\pm$ 0.0010
BBP |  0.9816 $\pm$ 0.0079 |  0.0325 $\pm$ 0.0212 |  0.0972 $\pm$ 0.0156 |  0.0226 $\pm$ 0.0024
SGLD | 0.9869 $\pm$ 0.0012 |  0.0183 $\pm$ 0.0006 |  0.0893 $\pm$ 0.0050 |  0.0214 $\pm$ 0.0005
Ensembles |  0.9812 $\pm$ - | 0.0157 $\pm$ - | 0.0810 $\pm$ - | 0.0208 $\pm$ -
GROVER | Deterministic |  0.9398 $\pm$ 0.0152 |  0.0635 $\pm$ 0.0114 |  0.1471 $\pm$ 0.0106 |  0.0367 $\pm$ 0.0031
Temperature |  0.9409 $\pm$ 0.0130 |  0.0449 $\pm$ 0.0055 |  0.1260 $\pm$ 0.0045 |  0.0333 $\pm$ 0.0034
Focal Loss |  0.9377 $\pm$ 0.0042 |  0.2158 $\pm$ 0.0034 |  0.2963 $\pm$ 0.0018 |  0.0731 $\pm$ 0.0011
MC Dropout |  0.9347 $\pm$ 0.0152 |  0.0706 $\pm$ 0.0122 |  0.1475 $\pm$ 0.0121 |  0.0366 $\pm$ 0.0036
SWAG |  0.9623 $\pm$ 0.0060 |  0.0460 $\pm$ 0.0117 |  0.1119 $\pm$ 0.0046 |  0.0280 $\pm$ 0.0022
BBP |  0.9085 $\pm$ 0.0109 |  0.0940 $\pm$ 0.0104 |  0.1976 $\pm$ 0.0136 |  0.0488 $\pm$ 0.0033
SGLD |  0.9507 $\pm$ 0.0058 |  0.0595 $\pm$ 0.0065 |  0.1287 $\pm$ 0.0063 |  0.0306 $\pm$ 0.0011
Ensembles |  0.9429 $\pm$ - |  0.0743 $\pm$ - |  0.1453 $\pm$ - |  0.0344 $\pm$ -
Uni-Mol | Deterministic |  0.8814 $\pm$ 0.0379 |  0.0522 $\pm$ 0.0120 |  0.1822 $\pm$ 0.0381 |  0.0484 $\pm$ 0.0124
Temperature |  0.8830 $\pm$ 0.0375 |  0.0417 $\pm$ 0.0158 |  0.1754 $\pm$ 0.0358 |  0.0477 $\pm$ 0.0121
Focal Loss |  0.8836 $\pm$ 0.0202 |  0.1651 $\pm$ 0.0556 |  0.2972 $\pm$ 0.0587 |  0.0827 $\pm$ 0.0207
MC Dropout |  0.8745 $\pm$ 0.0438 |  0.0560 $\pm$ 0.0131 |  0.1889 $\pm$ 0.0406 |  0.0521 $\pm$ 0.0138
SWAG |  0.9055 $\pm$ 0.0027 |  0.0311 $\pm$ 0.0034 |  0.1450 $\pm$ 0.0035 |  0.0337 $\pm$ 0.0009
BBP |  0.9257 $\pm$ 0.0287 |  0.0379 $\pm$ 0.0026 |  0.1468 $\pm$ 0.0047 |  0.0398 $\pm$ 0.0032
SGLD |  0.8598 $\pm$ 0.0209 |  0.0327 $\pm$ 0.0029 |  0.1619 $\pm$ 0.0059 |  0.0389 $\pm$ 0.0014
Ensembles |  0.9088 $\pm$ - |  0.0389 $\pm$ - |  0.1476 $\pm$ - |  0.0386 $\pm$ -
TorchMD-NET | Deterministic |  0.8938 $\pm$ 0.0178 |  0.0567 $\pm$ 0.0058 |  0.1905 $\pm$ 0.0083 |  0.0470 $\pm$ 0.0014
Temperature |  0.8938 $\pm$ 0.0178 |  0.0608 $\pm$ 0.0100 |  0.1869 $\pm$ 0.0076 |  0.0487 $\pm$ 0.0016
Focal Loss |  0.8952 $\pm$ 0.0195 |  0.1260 $\pm$ 0.0242 |  0.2582 $\pm$ 0.0218 |  0.0683 $\pm$ 0.0050
MC Dropout |  0.8898 $\pm$ 0.0192 |  0.0505 $\pm$ 0.0029 |  0.1711 $\pm$ 0.0054 |  0.0460 $\pm$ 0.0013
SWAG |  0.8995 $\pm$ 0.0185 |  0.0517 $\pm$ 0.0082 |  0.1969 $\pm$ 0.0065 |  0.0486 $\pm$ 0.0027
BBP |  0.8801 $\pm$ 0.0278 |  0.0667 $\pm$ 0.0139 |  0.2483 $\pm$ 0.0463 |  0.0610 $\pm$ 0.0077
SGLD |  0.8855 $\pm$ 0.0177 |  0.0475 $\pm$ 0.0045 |  0.1776 $\pm$ 0.0123 |  0.0475 $\pm$ 0.0045
Ensembles |  0.8944 $\pm$ - |  0.0569 $\pm$ - |  0.1886 $\pm$ - |  0.0485 $\pm$ -
GIN | Deterministic |  0.6965 $\pm$ 0.0184 |  0.1042 $\pm$ 0.0000 |  0.3398 $\pm$ 0.0260 |  0.0798 $\pm$ 0.0044
Temperature |  0.6965 $\pm$ 0.0184 |  0.0876 $\pm$ 0.0000 |  0.3595 $\pm$ 0.0346 |  0.0789 $\pm$ 0.0044
Focal Loss |  0.6777 $\pm$ 0.0168 |  0.2690 $\pm$ 0.0232 |  0.4610 $\pm$ 0.0314 |  0.1448 $\pm$ 0.0148
MC Dropout |  0.6965 $\pm$ 0.0184 |  0.1042 $\pm$ 0.0000 |  0.3398 $\pm$ 0.0260 |  0.0798 $\pm$ 0.0044
SWAG |  0.6552 $\pm$ 0.0197 |  0.1000 $\pm$ 0.0000 |  0.4573 $\pm$ 0.1485 |  0.0799 $\pm$ 0.0031
BBP |  0.6122 $\pm$ 0.0078 |  \- $\pm$ - |  0.5270 $\pm$ 0.1577 |  0.0888 $\pm$ 0.0131
SGLD |  0.6539 $\pm$ 0.0312 |  0.1256 $\pm$ 0.0000 |  0.4745 $\pm$ 0.1417 |  0.0827 $\pm$ 0.0046
Ensembles |  0.6872 $\pm$ - |  0.1170 $\pm$ - |  0.3222 $\pm$ - |  0.0868 $\pm$ -
Table 13: Test results on Tox21 in the format of “metric mean $\pm$ standard deviation”. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.7386 $\pm$ 0.0061 |  0.0417 $\pm$ 0.0028 |  0.2771 $\pm$ 0.0024 |  0.0779 $\pm$ 0.0013
Temperature |  0.7386 $\pm$ 0.0061 |  0.0342 $\pm$ 0.0027 |  0.2723 $\pm$ 0.0028 |  0.0773 $\pm$ 0.0013
Focal Loss |  0.7374 $\pm$ 0.0038 |  0.1058 $\pm$ 0.0060 |  0.3161 $\pm$ 0.0078 |  0.0871 $\pm$ 0.0024
MC Dropout |  0.7376 $\pm$ 0.0039 |  0.0356 $\pm$ 0.0016 |  0.2727 $\pm$ 0.0016 |  0.0763 $\pm$ 0.0013
SWAG |  0.7364 $\pm$ 0.0045 |  0.0438 $\pm$ 0.0010 |  0.2793 $\pm$ 0.0037 |  0.0790 $\pm$ 0.0017
BBP |  0.7243 $\pm$ 0.0036 |  0.0422 $\pm$ 0.0019 |  0.2847 $\pm$ 0.0010 |  0.0814 $\pm$ 0.0002
SGLD |  0.7257 $\pm$ 0.0018 |  0.1192 $\pm$ 0.0332 |  0.3455 $\pm$ 0.0315 |  0.0978 $\pm$ 0.0100
Ensembles |  0.7540 $\pm$ - |  0.0344 $\pm$ - |  0.2648 $\pm$ - |  0.0746 $\pm$ -
ChemBERTa | Deterministic |  0.7542 $\pm$ 0.0009 |  0.0571 $\pm$ 0.0020 |  0.2962 $\pm$ 0.0067 |  0.0812 $\pm$ 0.0009
Temperature |  0.7542 $\pm$ 0.0009 |  0.0424 $\pm$ 0.0040 |  0.2744 $\pm$ 0.0027 |  0.0792 $\pm$ 0.0011
Focal Loss |  0.7523 $\pm$ 0.0022 |  0.0969 $\pm$ 0.0012 |  0.3052 $\pm$ 0.0018 |  0.0845 $\pm$ 0.0004
MC Dropout |  0.7641 $\pm$ 0.0032 |  0.0423 $\pm$ 0.0023 |  0.2697 $\pm$ 0.0032 |  0.0744 $\pm$ 0.0005
SWAG |  0.7538 $\pm$ 0.0021 |  0.0592 $\pm$ 0.0022 |  0.3008 $\pm$ 0.0079 |  0.0818 $\pm$ 0.0011
BBP |  0.7433 $\pm$ 0.0055 |  0.0459 $\pm$ 0.0028 |  0.2765 $\pm$ 0.0035 |  0.0780 $\pm$ 0.0014
SGLD |  0.7475 $\pm$ 0.0036 |  0.0504 $\pm$ 0.0038 |  0.2784 $\pm$ 0.0030 |  0.0795 $\pm$ 0.0014
Ensembles |  0.7681 $\pm$ - |  0.0440 $\pm$ - |  0.2679 $\pm$ - |  0.0750 $\pm$ -
GROVER | Deterministic |  0.7808 $\pm$ 0.0017 |  0.0358 $\pm$ 0.0038 |  0.2473 $\pm$ 0.0022 |  0.0694 $\pm$ 0.0005
Temperature |  0.7810 $\pm$ 0.0016 |  0.0291 $\pm$ 0.0016 |  0.2439 $\pm$ 0.0008 |  0.0686 $\pm$ 0.0003
Focal Loss |  0.7779 $\pm$ 0.0028 |  0.1148 $\pm$ 0.0102 |  0.3052 $\pm$ 0.0094 |  0.0811 $\pm$ 0.0028
MC Dropout |  0.7817 $\pm$ 0.0018 |  0.0346 $\pm$ 0.0015 |  0.2455 $\pm$ 0.0016 |  0.0689 $\pm$ 0.0005
SWAG |  0.7837 $\pm$ 0.0017 |  0.0359 $\pm$ 0.0009 |  0.2482 $\pm$ 0.0009 |  0.0689 $\pm$ 0.0001
BBP |  0.7697 $\pm$ 0.0031 |  0.0438 $\pm$ 0.0017 |  0.2552 $\pm$ 0.0016 |  0.0711 $\pm$ 0.0003
SGLD |  0.7635 $\pm$ 0.0009 |  0.0402 $\pm$ 0.0024 |  0.2558 $\pm$ 0.0010 |  0.0716 $\pm$ 0.0002
Ensembles |  0.7876 $\pm$ - | 0.0316 $\pm$ - |  0.2411 $\pm$ - |  0.0675 $\pm$ -
Uni-Mol | Deterministic |  0.7895 $\pm$ 0.0017 |  0.0454 $\pm$ 0.0056 |  0.2601 $\pm$ 0.0072 |  0.0716 $\pm$ 0.0022
Temperature |  0.7896 $\pm$ 0.0016 |  0.0346 $\pm$ 0.0027 |  0.2483 $\pm$ 0.0035 |  0.0704 $\pm$ 0.0014
Focal Loss |  0.7904 $\pm$ 0.0040 |  0.0972 $\pm$ 0.0045 |  0.2899 $\pm$ 0.0009 |  0.0785 $\pm$ 0.0001
MC Dropout |  0.7891 $\pm$ 0.0021 |  0.0480 $\pm$ 0.0066 |  0.2628 $\pm$ 0.0065 |  0.0726 $\pm$ 0.0021
SWAG |  0.7842 $\pm$ 0.0048 |  0.0593 $\pm$ 0.0025 |  0.2994 $\pm$ 0.0054 |  0.0728 $\pm$ 0.0012
BBP |  0.7932 $\pm$ 0.0043 |  0.0396 $\pm$ 0.0048 |  0.2520 $\pm$ 0.0039 |  0.0703 $\pm$ 0.0010
SGLD |  0.7887 $\pm$ 0.0059 |  0.0433 $\pm$ 0.0023 |  0.2569 $\pm$ 0.0055 |  0.0684 $\pm$ 0.0014
Ensembles | 0.8052 $\pm$ - |  0.0332 $\pm$ - | 0.2389 $\pm$ - | 0.0662 $\pm$ -
TorchMD-NET | Deterministic |  0.7342 $\pm$ 0.0143 |  0.0738 $\pm$ 0.0146 |  0.4523 $\pm$ 0.1083 |  0.0857 $\pm$ 0.0044
Temperature |  0.7342 $\pm$ 0.0143 |  0.0515 $\pm$ 0.0047 |  0.2944 $\pm$ 0.0115 |  0.0801 $\pm$ 0.0014
Focal Loss |  0.7403 $\pm$ 0.0032 |  0.0639 $\pm$ 0.0044 |  0.3102 $\pm$ 0.0140 |  0.0808 $\pm$ 0.0027
MC Dropout |  0.7369 $\pm$ 0.0144 |  0.0685 $\pm$ 0.0131 |  0.3913 $\pm$ 0.0759 |  0.0837 $\pm$ 0.0033
SWAG |  0.7347 $\pm$ 0.0093 |  0.0740 $\pm$ 0.0040 |  0.4567 $\pm$ 0.0576 |  0.0832 $\pm$ 0.0018
BBP |  0.7459 $\pm$ 0.0067 |  0.0808 $\pm$ 0.0075 |  0.5355 $\pm$ 0.1830 |  0.0888 $\pm$ 0.0016
SGLD |  0.7387 $\pm$ 0.0051 |  0.0427 $\pm$ 0.0019 |  0.2761 $\pm$ 0.0029 |  0.0757 $\pm$ 0.0004
Ensembles |  0.7793 $\pm$ - |  0.0409 $\pm$ - |  0.2614 $\pm$ - |  0.0708 $\pm$ -
GIN | Deterministic |  0.6706 $\pm$ 0.0032 |  0.0789 $\pm$ 0.0172 |  0.3573 $\pm$ 0.0052 |  0.0916 $\pm$ 0.0070
Temperature |  0.6706 $\pm$ 0.0032 |  0.0914 $\pm$ 0.0158 |  0.3393 $\pm$ 0.0150 |  0.0941 $\pm$ 0.0068
Focal Loss |  0.6571 $\pm$ 0.0061 |  0.1040 $\pm$ 0.0073 |  0.3361 $\pm$ 0.0036 |  0.0930 $\pm$ 0.0014
MC Dropout |  0.6706 $\pm$ 0.0032 |  0.0789 $\pm$ 0.0172 |  0.3573 $\pm$ 0.0052 |  0.0916 $\pm$ 0.0070
SWAG |  0.6619 $\pm$ 0.0103 |  0.0671 $\pm$ 0.0032 |  0.3536 $\pm$ 0.0029 |  0.0867 $\pm$ 0.0026
BBP |  0.6425 $\pm$ 0.0034 |  0.0851 $\pm$ 0.0084 |  0.3632 $\pm$ 0.0039 |  0.0949 $\pm$ 0.0033
SGLD |  0.6190 $\pm$ 0.0048 |  0.0967 $\pm$ 0.0135 |  0.3817 $\pm$ 0.0275 |  0.0995 $\pm$ 0.0056
Ensembles |  0.6829 $\pm$ - |  0.0634 $\pm$ - |  0.3268 $\pm$ - |  0.0840 $\pm$ -
Table 14: Test results on ToxCast in the format of “metric mean $\pm$ standard deviation”. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.6222 $\pm$ 0.0042 |  0.1168 $\pm$ 0.0096 |  0.4436 $\pm$ 0.0146 |  0.1397 $\pm$ 0.0021
Temperature |  0.6220 $\pm$ 0.0046 |  0.1114 $\pm$ 0.0050 |  0.4882 $\pm$ 0.0170 |  0.1398 $\pm$ 0.0015
Focal Loss |  0.6289 $\pm$ 0.0077 |  0.1264 $\pm$ 0.0034 |  0.4389 $\pm$ 0.0084 |  0.1396 $\pm$ 0.0024
MC Dropout |  0.6248 $\pm$ 0.0030 |  0.1093 $\pm$ 0.0075 |  0.4319 $\pm$ 0.0096 |  0.1358 $\pm$ 0.0015
SWAG |  0.6207 $\pm$ 0.0073 |  0.1175 $\pm$ 0.0098 |  0.4440 $\pm$ 0.0150 |  0.1400 $\pm$ 0.0023
BBP |  0.6020 $\pm$ 0.0028 |  0.1443 $\pm$ 0.0060 |  0.4673 $\pm$ 0.0059 |  0.1510 $\pm$ 0.0020
SGLD |  0.5319 $\pm$ 0.0040 |  0.3054 $\pm$ 0.0034 |  0.6685 $\pm$ 0.0035 |  0.2378 $\pm$ 0.0017
Ensembles |  0.6486 $\pm$ - | 0.0900 $\pm$ - |  0.4008 $\pm$ - |  0.1292 $\pm$ -
ChemBERTa | Deterministic |  0.6554 $\pm$ 0.0037 |  0.1209 $\pm$ 0.0026 |  0.4313 $\pm$ 0.0054 |  0.1330 $\pm$ 0.0008
Temperature |  0.6540 $\pm$ 0.0042 |  0.1067 $\pm$ 0.0019 |  0.4817 $\pm$ 0.0261 |  0.1313 $\pm$ 0.0004
Focal Loss |  0.6442 $\pm$ 0.0170 |  0.1197 $\pm$ 0.0037 |  0.4243 $\pm$ 0.0153 |  0.1346 $\pm$ 0.0051
MC Dropout |  0.6624 $\pm$ 0.0024 |  0.1069 $\pm$ 0.0036 |  0.4070 $\pm$ 0.0052 |  0.1276 $\pm$ 0.0015
SWAG |  0.6556 $\pm$ 0.0033 |  0.1202 $\pm$ 0.0036 |  0.4305 $\pm$ 0.0064 |  0.1327 $\pm$ 0.0010
BBP |  0.5814 $\pm$ 0.0075 |  0.1276 $\pm$ 0.0015 |  0.4545 $\pm$ 0.0011 |  0.1469 $\pm$ 0.0003
SGLD |  0.5436 $\pm$ 0.0056 |  0.2238 $\pm$ 0.0019 |  0.5602 $\pm$ 0.0022 |  0.1881 $\pm$ 0.0011
Ensembles |  0.6733 $\pm$ - |  0.1037 $\pm$ - |  0.3986 $\pm$ - |  0.1258 $\pm$ -
GROVER | Deterministic |  0.6587 $\pm$ 0.0018 |  0.1043 $\pm$ 0.0006 |  0.4091 $\pm$ 0.0023 |  0.1298 $\pm$ 0.0004
Temperature |  0.6496 $\pm$ 0.0023 |  0.1424 $\pm$ 0.0026 |  0.4612 $\pm$ 0.0045 |  0.1424 $\pm$ 0.0013
Focal Loss |  0.6359 $\pm$ 0.0026 |  0.1221 $\pm$ 0.0005 |  0.4365 $\pm$ 0.0012 |  0.1383 $\pm$ 0.0004
MC Dropout |  0.6615 $\pm$ 0.0014 |  0.1009 $\pm$ 0.0010 |  0.4042 $\pm$ 0.0022 |  0.1288 $\pm$ 0.0004
SWAG |  0.6603 $\pm$ 0.0020 |  0.1060 $\pm$ 0.0007 |  0.4114 $\pm$ 0.0015 |  0.1301 $\pm$ 0.0001
BBP |  0.5995 $\pm$ 0.0026 |  0.1731 $\pm$ 0.0041 |  0.5090 $\pm$ 0.0040 |  0.1660 $\pm$ 0.0015
SGLD |  0.5542 $\pm$ 0.0069 |  0.2712 $\pm$ 0.0008 |  0.6194 $\pm$ 0.0006 |  0.2139 $\pm$ 0.0003
Ensembles |  0.6646 $\pm$ - |  0.1034 $\pm$ - |  0.4061 $\pm$ - |  0.1290 $\pm$ -
Uni-Mol | Deterministic |  0.6734 $\pm$ 0.0101 |  0.1020 $\pm$ 0.0023 |  0.3983 $\pm$ 0.0075 |  0.1274 $\pm$ 0.0025
Temperature | 0.7028 $\pm$ 0.0101 |  0.1456 $\pm$ 0.0045 |  0.4566 $\pm$ 0.0104 |  0.1355 $\pm$ 0.0029
Focal Loss |  0.6934 $\pm$ 0.0035 |  0.1227 $\pm$ 0.0011 |  0.4079 $\pm$ 0.0016 |  0.1284 $\pm$ 0.0003
MC Dropout |  0.6833 $\pm$ 0.0117 |  0.1074 $\pm$ 0.0090 |  0.4015 $\pm$ 0.0112 |  0.1274 $\pm$ 0.0025
SWAG |  0.6870 $\pm$ 0.0075 |  0.1085 $\pm$ 0.0034 |  0.4005 $\pm$ 0.0055 |  0.1271 $\pm$ 0.0019
BBP |  0.6273 $\pm$ 0.0066 |  0.1296 $\pm$ 0.0065 |  0.4522 $\pm$ 0.0068 |  0.1456 $\pm$ 0.0021
SGLD |  0.5700 $\pm$ 0.0005 |  0.1953 $\pm$ 0.0033 |  0.5207 $\pm$ 0.0029 |  0.1717 $\pm$ 0.0012
Ensembles |  0.6841 $\pm$ - |  0.0953 $\pm$ - | 0.3877 $\pm$ - | 0.1247 $\pm$ -
TorchMD-NET | Deterministic |  0.6361 $\pm$ 0.0001 |  0.1734 $\pm$ 0.0053 |  0.4870 $\pm$ 0.0164 |  0.1486 $\pm$ 0.0027
Temperature |  0.6384 $\pm$ 0.0020 |  \- $\pm$ - |  0.6053 $\pm$ 0.0984 |  0.1466 $\pm$ 0.0009
Focal Loss |  0.6443 $\pm$ 0.0073 |  0.1543 $\pm$ 0.0032 |  0.4337 $\pm$ 0.0011 |  0.1395 $\pm$ 0.0006
MC Dropout |  0.6339 $\pm$ 0.0042 |  0.1707 $\pm$ 0.0048 |  0.4778 $\pm$ 0.0141 |  0.1477 $\pm$ 0.0027
SWAG |  0.6457 $\pm$ 0.0064 |  0.1648 $\pm$ 0.0012 |  0.4617 $\pm$ 0.0025 |  0.1446 $\pm$ 0.0004
BBP |  0.6103 $\pm$ 0.0182 |  0.1471 $\pm$ 0.0049 |  0.4539 $\pm$ 0.0171 |  0.1434 $\pm$ 0.0026
SGLD |  0.5224 $\pm$ 0.0048 |  0.2976 $\pm$ 0.0020 |  0.6678 $\pm$ 0.0020 |  0.2374 $\pm$ 0.0010
Ensembles |  0.6540 $\pm$ - |  0.1546 $\pm$ - |  0.4424 $\pm$ - |  0.1396 $\pm$ -
GIN | Deterministic |  0.5667 $\pm$ 0.0032 |  0.1443 $\pm$ 0.0018 |  0.5142 $\pm$ 0.0048 |  0.1506 $\pm$ 0.0015
Temperature |  0.5669 $\pm$ 0.0028 |  0.1402 $\pm$ 0.0013 |  0.5333 $\pm$ 0.0192 |  0.1476 $\pm$ 0.0014
Focal Loss |  0.5483 $\pm$ 0.0028 |  0.1446 $\pm$ 0.0027 |  0.4615 $\pm$ 0.0009 |  0.1468 $\pm$ 0.0006
MC Dropout |  0.5667 $\pm$ 0.0032 |  0.1443 $\pm$ 0.0018 |  0.5142 $\pm$ 0.0048 |  0.1506 $\pm$ 0.0015
SWAG |  0.5633 $\pm$ 0.0059 |  0.1418 $\pm$ 0.0018 |  0.5125 $\pm$ 0.0052 |  0.1495 $\pm$ 0.0014
BBP |  0.5096 $\pm$ 0.0059 |  0.2243 $\pm$ 0.0007 |  0.6405 $\pm$ 0.0128 |  0.1935 $\pm$ 0.0014
SGLD |  0.4872 $\pm$ 0.0050 |  0.3166 $\pm$ 0.0019 |  0.8083 $\pm$ 0.0245 |  0.2668 $\pm$ 0.0047
Ensembles |  0.5752 $\pm$ - |  0.1381 $\pm$ - |  0.4835 $\pm$ - |  0.1477 $\pm$ -
Table 15: Test results on SIDER in the format of “metric mean $\pm$ standard deviation”. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.5981 $\pm$ 0.0152 |  0.1188 $\pm$ 0.0344 |  0.5462 $\pm$ 0.0469 |  0.1760 $\pm$ 0.0124
Temperature |  0.5981 $\pm$ 0.0152 |  0.1033 $\pm$ 0.0271 |  0.5186 $\pm$ 0.0270 |  0.1709 $\pm$ 0.0094
Focal Loss |  0.5864 $\pm$ 0.0071 |  0.1663 $\pm$ 0.0217 |  0.5605 $\pm$ 0.0193 |  0.1893 $\pm$ 0.0075
MC Dropout |  0.5977 $\pm$ 0.0164 |  0.1071 $\pm$ 0.0271 |  0.5245 $\pm$ 0.0304 |  0.1705 $\pm$ 0.0084
SWAG |  0.5988 $\pm$ 0.0210 |  0.1206 $\pm$ 0.0333 |  0.5477 $\pm$ 0.0491 |  0.1764 $\pm$ 0.0125
BBP |  0.5738 $\pm$ 0.0272 |  0.1215 $\pm$ 0.0046 |  0.5241 $\pm$ 0.0077 |  0.1733 $\pm$ 0.0027
SGLD |  0.5895 $\pm$ 0.0087 |  0.1559 $\pm$ 0.0119 |  0.5653 $\pm$ 0.0119 |  0.1900 $\pm$ 0.0051
Ensembles |  0.6158 $\pm$ - |  0.0954 $\pm$ - |  0.4950 $\pm$ - |  0.1631 $\pm$ -
ChemBERTa | Deterministic |  0.6146 $\pm$ 0.0078 |  0.1483 $\pm$ 0.0102 |  0.5740 $\pm$ 0.0217 |  0.1804 $\pm$ 0.0055
Temperature |  0.6146 $\pm$ 0.0078 |  0.1275 $\pm$ 0.0104 |  0.5256 $\pm$ 0.0138 |  0.1721 $\pm$ 0.0043
Focal Loss |  0.6119 $\pm$ 0.0105 |  0.1498 $\pm$ 0.0136 |  0.5346 $\pm$ 0.0129 |  0.1777 $\pm$ 0.0046
MC Dropout |  0.6166 $\pm$ 0.0022 |  0.1136 $\pm$ 0.0143 |  0.5191 $\pm$ 0.0131 |  0.1672 $\pm$ 0.0037
SWAG |  0.6149 $\pm$ 0.0095 |  0.1527 $\pm$ 0.0090 |  0.5836 $\pm$ 0.0230 |  0.1825 $\pm$ 0.0056
BBP |  0.6067 $\pm$ 0.0108 |  0.1184 $\pm$ 0.0018 |  0.5127 $\pm$ 0.0043 |  0.1686 $\pm$ 0.0013
SGLD |  0.6127 $\pm$ 0.0110 |  0.1299 $\pm$ 0.0118 |  0.5255 $\pm$ 0.0157 |  0.1725 $\pm$ 0.0053
Ensembles |  0.6193 $\pm$ - |  0.1305 $\pm$ - |  0.5396 $\pm$ - |  0.1718 $\pm$ -
GROVER | Deterministic |  0.6213 $\pm$ 0.0027 |  0.1034 $\pm$ 0.0061 |  0.5023 $\pm$ 0.0071 |  0.1637 $\pm$ 0.0021
Temperature |  0.6209 $\pm$ 0.0030 |  0.0939 $\pm$ 0.0040 |  0.4935 $\pm$ 0.0046 |  0.1618 $\pm$ 0.0018
Focal Loss |  0.6059 $\pm$ 0.0165 |  0.1817 $\pm$ 0.0019 |  0.5666 $\pm$ 0.0027 |  0.1903 $\pm$ 0.0011
MC Dropout |  0.6248 $\pm$ 0.0060 |  0.0969 $\pm$ 0.0035 |  0.4942 $\pm$ 0.0057 |  0.1616 $\pm$ 0.0018
SWAG |  0.6133 $\pm$ 0.0024 |  0.1183 $\pm$ 0.0016 |  0.5280 $\pm$ 0.0066 |  0.1702 $\pm$ 0.0019
BBP |  0.5926 $\pm$ 0.0052 |  0.1347 $\pm$ 0.0014 |  0.5367 $\pm$ 0.0023 |  0.1762 $\pm$ 0.0009
SGLD |  0.5989 $\pm$ 0.0129 |  0.1012 $\pm$ 0.0040 |  0.5017 $\pm$ 0.0051 |  0.1637 $\pm$ 0.0019
Ensembles |  0.6237 $\pm$ - |  0.0878 $\pm$ - |  0.4917 $\pm$ - |  0.1604 $\pm$ -
Uni-Mol | Deterministic |  0.6214 $\pm$ 0.0073 |  0.1054 $\pm$ 0.0130 |  0.5047 $\pm$ 0.0184 |  0.1661 $\pm$ 0.0065
Temperature |  0.6210 $\pm$ 0.0075 |  0.0950 $\pm$ 0.0074 |  0.4937 $\pm$ 0.0101 |  0.1625 $\pm$ 0.0040
Focal Loss |  0.6200 $\pm$ 0.0118 |  0.1660 $\pm$ 0.0096 |  0.5471 $\pm$ 0.0090 |  0.1834 $\pm$ 0.0038
MC Dropout |  0.6125 $\pm$ 0.0060 |  0.1269 $\pm$ 0.0178 |  0.5324 $\pm$ 0.0270 |  0.1739 $\pm$ 0.0071
SWAG |  0.6246 $\pm$ 0.0090 |  0.1327 $\pm$ 0.0084 |  0.5397 $\pm$ 0.0125 |  0.1753 $\pm$ 0.0033
BBP |  0.6301 $\pm$ 0.0260 |  0.0991 $\pm$ 0.0130 |  0.4928 $\pm$ 0.0119 |  0.1612 $\pm$ 0.0049
SGLD |  0.6323 $\pm$ 0.0108 |  0.1286 $\pm$ 0.0093 |  0.5408 $\pm$ 0.0137 |  0.1725 $\pm$ 0.0030
Ensembles | 0.6378 $\pm$ - | 0.0872 $\pm$ - | 0.4836 $\pm$ - |  0.1585 $\pm$ -
TorchMD-NET | Deterministic |  0.5881 $\pm$ 0.0060 |  0.1305 $\pm$ 0.0239 |  0.5988 $\pm$ 0.0541 |  0.1789 $\pm$ 0.0072
Temperature |  0.5881 $\pm$ 0.0060 |  0.1142 $\pm$ 0.0109 |  0.5380 $\pm$ 0.0147 |  0.1721 $\pm$ 0.0038
Focal Loss |  0.5909 $\pm$ 0.0048 |  0.1515 $\pm$ 0.0034 |  0.5470 $\pm$ 0.0058 |  0.1848 $\pm$ 0.0031
MC Dropout |  0.5862 $\pm$ 0.0059 |  0.1230 $\pm$ 0.0205 |  0.5551 $\pm$ 0.0327 |  0.1756 $\pm$ 0.0061
SWAG |  0.6081 $\pm$ 0.0085 |  0.1281 $\pm$ 0.0152 |  0.5596 $\pm$ 0.0326 |  0.1751 $\pm$ 0.0059
BBP |  0.6072 $\pm$ 0.0150 |  0.1293 $\pm$ 0.0005 |  0.5619 $\pm$ 0.0059 |  0.1753 $\pm$ 0.0008
SGLD |  0.6026 $\pm$ 0.0089 |  0.1143 $\pm$ 0.0076 |  0.5162 $\pm$ 0.0054 |  0.1693 $\pm$ 0.0017
Ensembles |  0.6329 $\pm$ - |  0.1093 $\pm$ - |  0.5175 $\pm$ - |  0.1660 $\pm$ -
GIN | Deterministic |  0.6159 $\pm$ 0.0016 |  0.0900 $\pm$ 0.0020 |  0.5385 $\pm$ 0.0084 |  0.1627 $\pm$ 0.0009
Temperature |  0.6159 $\pm$ 0.0017 |  0.0916 $\pm$ 0.0031 |  0.5219 $\pm$ 0.0125 |  0.1625 $\pm$ 0.0009
Focal Loss |  0.5892 $\pm$ 0.0046 |  0.1863 $\pm$ 0.0011 |  0.5764 $\pm$ 0.0034 |  0.1953 $\pm$ 0.0013
MC Dropout |  0.6159 $\pm$ 0.0016 |  0.0900 $\pm$ 0.0020 |  0.5385 $\pm$ 0.0084 |  0.1627 $\pm$ 0.0009
SWAG |  0.6054 $\pm$ 0.0034 |  0.0959 $\pm$ 0.0042 |  0.5520 $\pm$ 0.0021 |  0.1642 $\pm$ 0.0011
BBP |  0.5499 $\pm$ 0.0132 |  0.1477 $\pm$ 0.0071 |  0.5666 $\pm$ 0.0079 |  0.1876 $\pm$ 0.0037
SGLD |  0.5792 $\pm$ 0.0074 |  0.1121 $\pm$ 0.0028 |  0.6447 $\pm$ 0.0421 |  0.1723 $\pm$ 0.0018
Ensembles |  0.6232 $\pm$ - |  0.0874 $\pm$ - |  0.5238 $\pm$ - | 0.1584 $\pm$ -
Table 16: Test results on HIV in the format of “metric mean $\pm$ standard deviation”. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.7345 $\pm$ 0.0065 |  0.0077 $\pm$ 0.0010 |  0.1236 $\pm$ 0.0016 |  0.0276 $\pm$ 0.0006
Temperature |  0.7345 $\pm$ 0.0065 |  0.0136 $\pm$ 0.0005 |  0.1277 $\pm$ 0.0021 |  0.0277 $\pm$ 0.0007
Focal Loss |  0.7338 $\pm$ 0.0078 |  0.1205 $\pm$ 0.0179 |  0.2059 $\pm$ 0.0179 |  0.0429 $\pm$ 0.0042
MC Dropout |  0.7365 $\pm$ 0.0088 |  0.0066 $\pm$ 0.0010 |  0.1219 $\pm$ 0.0016 |  0.0270 $\pm$ 0.0004
SWAG |  0.7439 $\pm$ 0.0014 |  0.0105 $\pm$ 0.0025 |  0.1240 $\pm$ 0.0013 |  0.0279 $\pm$ 0.0005
BBP |  0.7541 $\pm$ 0.0120 |  0.0181 $\pm$ 0.0065 |  0.1235 $\pm$ 0.0036 |  0.0277 $\pm$ 0.0011
SGLD |  0.7414 $\pm$ 0.0019 |  0.0160 $\pm$ 0.0048 |  0.1313 $\pm$ 0.0061 |  0.0281 $\pm$ 0.0007
Ensembles |  0.7613 $\pm$ - |  0.0074 $\pm$ - |  0.1160 $\pm$ - |  0.0260 $\pm$ -
ChemBERTa | Deterministic |  0.7681 $\pm$ 0.0061 |  0.0214 $\pm$ 0.0033 |  0.1247 $\pm$ 0.0058 |  0.0292 $\pm$ 0.0019
Temperature |  0.7681 $\pm$ 0.0061 |  0.0221 $\pm$ 0.0039 |  0.1259 $\pm$ 0.0067 |  0.0289 $\pm$ 0.0018
Focal Loss |  0.7509 $\pm$ 0.0106 |  0.0651 $\pm$ 0.0351 |  0.1599 $\pm$ 0.0205 |  0.0343 $\pm$ 0.0025
MC Dropout |  0.7717 $\pm$ 0.0042 |  0.0079 $\pm$ 0.0014 |  0.1105 $\pm$ 0.0011 |  0.0247 $\pm$ 0.0005
SWAG |  0.7474 $\pm$ 0.0123 |  0.0263 $\pm$ 0.0045 |  0.1384 $\pm$ 0.0063 |  0.0319 $\pm$ 0.0017
BBP |  0.7672 $\pm$ 0.0031 |  0.0217 $\pm$ 0.0043 |  0.1241 $\pm$ 0.0071 |  0.0289 $\pm$ 0.0027
SGLD |  0.7224 $\pm$ 0.0095 |  0.0374 $\pm$ 0.0011 |  0.1602 $\pm$ 0.0106 |  0.0376 $\pm$ 0.0002
Ensembles |  0.7885 $\pm$ - |  0.0205 $\pm$ - |  0.1161 $\pm$ - |  0.0267 $\pm$ -
GROVER | Deterministic |  0.7979 $\pm$ 0.0045 |  0.0078 $\pm$ 0.0032 |  0.1133 $\pm$ 0.0019 |  0.0263 $\pm$ 0.0005
Temperature |  0.7977 $\pm$ 0.0046 |  0.0120 $\pm$ 0.0011 |  0.1154 $\pm$ 0.0022 |  0.0265 $\pm$ 0.0004
Focal Loss |  0.7746 $\pm$ 0.0113 |  0.1286 $\pm$ 0.0223 |  0.2156 $\pm$ 0.0259 |  0.0444 $\pm$ 0.0066
MC Dropout |  0.8002 $\pm$ 0.0018 |  0.0077 $\pm$ 0.0028 |  0.1126 $\pm$ 0.0006 |  0.0261 $\pm$ 0.0001
SWAG |  0.8021 $\pm$ 0.0033 |  0.0090 $\pm$ 0.0019 |  0.1118 $\pm$ 0.0018 |  0.0258 $\pm$ 0.0005
BBP |  0.7928 $\pm$ 0.0116 |  0.0162 $\pm$ 0.0062 |  0.1151 $\pm$ 0.0034 |  0.0258 $\pm$ 0.0002
SGLD |  0.7420 $\pm$ 0.0131 |  0.0200 $\pm$ 0.0012 |  0.1231 $\pm$ 0.0001 |  0.0272 $\pm$ 0.0000
Ensembles | 0.8103 $\pm$ - | 0.0035 $\pm$ - | 0.1081 $\pm$ - | 0.0251 $\pm$ -
Uni-Mol | Deterministic |  0.7913 $\pm$ 0.0042 |  0.0189 $\pm$ 0.0063 |  0.1335 $\pm$ 0.0225 |  0.0273 $\pm$ 0.0014
Temperature |  0.7915 $\pm$ 0.0041 |  0.0151 $\pm$ 0.0017 |  0.1190 $\pm$ 0.0040 |  0.0269 $\pm$ 0.0009
Focal Loss |  0.7785 $\pm$ 0.0073 |  0.0713 $\pm$ 0.0251 |  0.1561 $\pm$ 0.0205 |  0.0316 $\pm$ 0.0032
MC Dropout |  0.7858 $\pm$ 0.0044 |  0.0229 $\pm$ 0.0091 |  0.1557 $\pm$ 0.0339 |  0.0294 $\pm$ 0.0033
SWAG |  0.7786 $\pm$ 0.0061 |  0.0264 $\pm$ 0.0015 |  0.1676 $\pm$ 0.0160 |  0.0286 $\pm$ 0.0011
BBP |  0.7781 $\pm$ 0.0128 |  0.0191 $\pm$ 0.0071 |  0.1428 $\pm$ 0.0327 |  0.0284 $\pm$ 0.0016
SGLD |  0.7777 $\pm$ 0.0121 |  0.0205 $\pm$ 0.0006 |  0.1373 $\pm$ 0.0087 |  0.0263 $\pm$ 0.0002
Ensembles |  0.8020 $\pm$ - |  0.0181 $\pm$ - |  0.1181 $\pm$ - |  0.0265 $\pm$ -
TorchMD-NET | Deterministic |  0.7221 $\pm$ 0.0133 |  0.0270 $\pm$ 0.0031 |  0.2340 $\pm$ 0.0668 |  0.0315 $\pm$ 0.0007
Temperature |  0.7221 $\pm$ 0.0133 |  0.0266 $\pm$ 0.0025 |  0.1532 $\pm$ 0.0024 |  0.0317 $\pm$ 0.0008
Focal Loss |  0.7350 $\pm$ 0.0170 |  0.0673 $\pm$ 0.0312 |  0.1585 $\pm$ 0.0212 |  0.0332 $\pm$ 0.0036
MC Dropout |  0.7225 $\pm$ 0.0098 |  0.0265 $\pm$ 0.0036 |  0.2191 $\pm$ 0.0590 |  0.0313 $\pm$ 0.0009
SWAG |  0.7555 $\pm$ 0.0132 |  0.0253 $\pm$ 0.0007 |  0.2110 $\pm$ 0.0153 |  0.0288 $\pm$ 0.0006
BBP |  0.7521 $\pm$ 0.0078 |  0.0195 $\pm$ 0.0114 |  0.1384 $\pm$ 0.0251 |  0.0290 $\pm$ 0.0036
SGLD |  \- $\pm$ - |  \- $\pm$ - |  \- $\pm$ - |  \- $\pm$ -
Ensembles |  0.7370 $\pm$ - |  0.0218 $\pm$ - |  0.1537 $\pm$ - |  0.0284 $\pm$ -
GIN | Deterministic |  0.6893 $\pm$ 0.0214 |  \- $\pm$ - |  0.1715 $\pm$ 0.0040 |  0.0396 $\pm$ 0.0021
Temperature |  0.6893 $\pm$ 0.0214 |  \- $\pm$ - |  0.1708 $\pm$ 0.0032 |  0.0378 $\pm$ 0.0024
Focal Loss |  0.6703 $\pm$ 0.0204 |  \- $\pm$ - |  0.2283 $\pm$ 0.0151 |  0.0515 $\pm$ 0.0046
MC Dropout |  0.6893 $\pm$ 0.0214 |  \- $\pm$ - |  0.1715 $\pm$ 0.0040 |  0.0396 $\pm$ 0.0021
SWAG |  0.6863 $\pm$ 0.0305 |  \- $\pm$ - |  0.1496 $\pm$ 0.0091 |  0.0318 $\pm$ 0.0007
BBP |  0.6947 $\pm$ 0.0106 |  \- $\pm$ - |  0.1625 $\pm$ 0.0055 |  0.0360 $\pm$ 0.0021
SGLD |  0.6845 $\pm$ 0.0063 |  \- $\pm$ - |  0.1954 $\pm$ 0.0322 |  0.0362 $\pm$ 0.0026
Ensembles |  0.7209 $\pm$ - |  \- $\pm$ - |  0.1492 $\pm$ - |  0.0336 $\pm$ -
Table 17: Test results on MUV in the format of “metric mean $\pm$ standard deviation”. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.6029 $\pm$ 0.0576 |  0.0015 $\pm$ 0.0003 |  0.0326 $\pm$ 0.0196 |  0.0026 $\pm$ 0.0001
Temperature |  0.6059 $\pm$ 0.0538 |  \- $\pm$ - |  0.0321 $\pm$ 0.0145 |  0.0027 $\pm$ 0.0003
Focal Loss |  0.6261 $\pm$ 0.0322 |  0.0780 $\pm$ 0.0033 |  0.0720 $\pm$ 0.0262 |  0.0076 $\pm$ 0.0032
MC Dropout |  0.6282 $\pm$ 0.0530 |  0.0015 $\pm$ 0.0003 |  0.0256 $\pm$ 0.0114 | 0.0025 $\pm$ 0.0000
SWAG |  0.5830 $\pm$ 0.0392 |  0.0025 $\pm$ 0.0006 |  0.0393 $\pm$ 0.0254 |  0.0027 $\pm$ 0.0001
BBP |  0.5974 $\pm$ 0.0287 |  \- $\pm$ - |  0.0251 $\pm$ 0.0068 |  0.0027 $\pm$ 0.0001
SGLD |  0.4955 $\pm$ 0.0509 |  0.2380 $\pm$ 0.0156 |  0.2798 $\pm$ 0.0201 |  0.0610 $\pm$ 0.0071
Ensembles |  0.6736 $\pm$ - | 0.0013 $\pm$ - |  0.0175 $\pm$ - | 0.0025 $\pm$ -
ChemBERTa | Deterministic |  0.6695 $\pm$ 0.0343 |  0.0030 $\pm$ 0.0001 |  0.0241 $\pm$ 0.0011 |  0.0029 $\pm$ 0.0000
Temperature |  0.6695 $\pm$ 0.0343 |  0.0026 $\pm$ 0.0003 |  0.0196 $\pm$ 0.0010 |  0.0029 $\pm$ 0.0000
Focal Loss |  0.7150 $\pm$ 0.0038 |  0.0208 $\pm$ 0.0119 |  0.0333 $\pm$ 0.0100 |  0.0040 $\pm$ 0.0007
MC Dropout |  0.7168 $\pm$ 0.0100 |  0.0026 $\pm$ 0.0003 |  0.0230 $\pm$ 0.0027 |  0.0026 $\pm$ 0.0001
SWAG |  0.6672 $\pm$ 0.0342 |  0.0031 $\pm$ 0.0000 |  0.0261 $\pm$ 0.0015 |  0.0029 $\pm$ 0.0000
BBP |  0.5970 $\pm$ 0.0622 |  0.0026 $\pm$ 0.0010 |  0.0198 $\pm$ 0.0018 |  0.0030 $\pm$ 0.0004
SGLD |  0.5030 $\pm$ 0.0346 |  0.0074 $\pm$ 0.0003 |  0.0217 $\pm$ 0.0002 |  0.0026 $\pm$ 0.0000
Ensembles |  0.7224 $\pm$ - |  0.0025 $\pm$ - |  0.0197 $\pm$ - |  0.0027 $\pm$ -
GROVER | Deterministic |  0.7157 $\pm$ 0.0330 |  0.0016 $\pm$ 0.0004 |  0.0169 $\pm$ 0.0004 | 0.0025 $\pm$ 0.0000
Temperature |  0.7155 $\pm$ 0.0331 |  0.0014 $\pm$ 0.0001 |  0.0187 $\pm$ 0.0004 | 0.0025 $\pm$ 0.0000
Focal Loss |  0.6825 $\pm$ 0.0127 |  0.0869 $\pm$ 0.0033 |  0.0995 $\pm$ 0.0036 |  0.0101 $\pm$ 0.0006
MC Dropout |  0.7163 $\pm$ 0.0363 |  0.0016 $\pm$ 0.0004 |  0.0170 $\pm$ 0.0004 | 0.0025 $\pm$ 0.0000
SWAG |  0.7281 $\pm$ 0.0226 |  0.0014 $\pm$ 0.0003 | 0.0162 $\pm$ 0.0002 |  0.0025 $\pm$ 0.0000
BBP |  0.7490 $\pm$ 0.0068 |  0.0052 $\pm$ 0.0015 |  0.0192 $\pm$ 0.0011 | 0.0025 $\pm$ 0.0000
SGLD |  0.5532 $\pm$ 0.0255 |  0.0375 $\pm$ 0.0009 |  0.0488 $\pm$ 0.0008 |  0.0039 $\pm$ 0.0001
Ensembles |  0.7268 $\pm$ - |  0.0014 $\pm$ - |  0.0166 $\pm$ - | 0.0025 $\pm$ -
Uni-Mol | Deterministic |  0.7718 $\pm$ 0.0200 |  0.0024 $\pm$ 0.0002 |  0.0225 $\pm$ 0.0027 | 0.0025 $\pm$ 0.0000
Temperature |  0.7625 $\pm$ 0.0098 |  0.0017 $\pm$ 0.0000 |  0.0233 $\pm$ 0.0039 | 0.0025 $\pm$ 0.0001
Focal Loss | 0.8239 $\pm$ 0.0140 |  0.0250 $\pm$ 0.0126 |  0.0354 $\pm$ 0.0116 |  0.0036 $\pm$ 0.0006
MC Dropout |  0.7781 $\pm$ 0.0201 |  0.0026 $\pm$ 0.0001 |  0.0253 $\pm$ 0.0017 | 0.0025 $\pm$ 0.0000
SWAG |  0.7429 $\pm$ 0.0442 |  0.0028 $\pm$ 0.0001 |  0.0276 $\pm$ 0.0016 |  0.0027 $\pm$ 0.0001
BBP |  0.7394 $\pm$ 0.0140 |  0.0016 $\pm$ 0.0002 |  0.0169 $\pm$ 0.0002 | 0.0025 $\pm$ 0.0000
SGLD |  0.5901 $\pm$ 0.0200 |  0.0014 $\pm$ 0.0000 |  0.0178 $\pm$ 0.0000 | 0.0025 $\pm$ 0.0000
Ensembles |  0.7904 $\pm$ - |  0.0023 $\pm$ - |  0.0195 $\pm$ - | 0.0025 $\pm$ -
TorchMD-NET | Deterministic |  0.7043 $\pm$ 0.0027 |  \- $\pm$ - |  0.0553 $\pm$ 0.0092 |  0.0027 $\pm$ 0.0001
Temperature |  0.7073 $\pm$ 0.0016 |  \- $\pm$ - |  0.0318 $\pm$ 0.0059 |  0.0027 $\pm$ 0.0001
Focal Loss |  0.7227 $\pm$ 0.0117 |  \- $\pm$ - |  0.0225 $\pm$ 0.0034 |  0.0028 $\pm$ 0.0002
MC Dropout |  0.7003 $\pm$ 0.0057 |  \- $\pm$ - |  0.0455 $\pm$ 0.0074 |  0.0026 $\pm$ 0.0001
SWAG |  0.7285 $\pm$ 0.0094 |  \- $\pm$ - |  0.0457 $\pm$ 0.0011 |  0.0029 $\pm$ 0.0001
BBP |  0.7330 $\pm$ 0.0044 |  \- $\pm$ - |  0.0407 $\pm$ 0.0070 |  0.0029 $\pm$ 0.0002
SGLD |  \- $\pm$ - |  \- $\pm$ - |  \- $\pm$ - |  \- $\pm$ -
Ensembles |  0.7045 $\pm$ - |  \- $\pm$ - |  0.0437 $\pm$ - | 0.0025 $\pm$ -
GIN | Deterministic |  0.6996 $\pm$ 0.0094 |  0.0024 $\pm$ 0.0006 |  0.0224 $\pm$ 0.0034 |  0.0028 $\pm$ 0.0001
Temperature |  0.6885 $\pm$ 0.0072 |  \- $\pm$ - |  0.0284 $\pm$ 0.0037 |  0.0028 $\pm$ 0.0002
Focal Loss |  0.6964 $\pm$ 0.0121 |  0.0146 $\pm$ 0.0053 |  0.0280 $\pm$ 0.0042 |  0.0037 $\pm$ 0.0004
MC Dropout |  0.6996 $\pm$ 0.0094 |  0.0024 $\pm$ 0.0006 |  0.0224 $\pm$ 0.0034 |  0.0028 $\pm$ 0.0001
SWAG |  0.6898 $\pm$ 0.0132 |  0.0021 $\pm$ 0.0003 |  0.0226 $\pm$ 0.0030 |  0.0026 $\pm$ 0.0000
BBP |  0.6425 $\pm$ 0.0114 |  0.0022 $\pm$ 0.0002 |  0.0229 $\pm$ 0.0009 |  0.0027 $\pm$ 0.0001
SGLD |  0.5553 $\pm$ 0.0110 |  0.0063 $\pm$ 0.0007 |  0.0400 $\pm$ 0.0058 |  0.0029 $\pm$ 0.0001
Ensembles |  0.7155 $\pm$ - |  0.0021 $\pm$ - |  0.0191 $\pm$ - |  0.0026 $\pm$ -
Table 18: Test results on ESOL in the format of “metric mean $\pm$ standard deviation”. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.9048 $\pm$ 0.0289 |  0.6747 $\pm$ 0.0059 |  1.0033 $\pm$ 0.3765 |  0.0465 $\pm$ 0.0052
MC Dropout |  0.9129 $\pm$ 0.0212 |  0.6853 $\pm$ 0.0049 |  1.0246 $\pm$ 0.3698 |  0.0495 $\pm$ 0.0051
SWAG |  0.9165 $\pm$ 0.0244 |  0.6827 $\pm$ 0.0074 |  0.9461 $\pm$ 0.3428 |  0.0459 $\pm$ 0.0048
BBP |  0.9557 $\pm$ 0.0298 |  0.7070 $\pm$ 0.0237 |  0.4306 $\pm$ 0.0325 |  0.0325 $\pm$ 0.0038
SGLD |  0.9630 $\pm$ 0.0425 |  0.7175 $\pm$ 0.0302 |  0.5101 $\pm$ 0.0380 |  0.0162 $\pm$ 0.0017
Ensembles |  0.8690 $\pm$ - |  0.6455 $\pm$ - |  0.5674 $\pm$ - |  0.0453 $\pm$ -
ChemBERTa | Deterministic |  1.0044 $\pm$ 0.0358 |  0.7989 $\pm$ 0.0216 |  2.2803 $\pm$ 0.1218 |  0.0500 $\pm$ 0.0007
MC Dropout |  0.9497 $\pm$ 0.0271 |  0.7620 $\pm$ 0.0176 |  1.4258 $\pm$ 0.1905 |  0.0481 $\pm$ 0.0023
SWAG |  0.9966 $\pm$ 0.0334 |  0.7940 $\pm$ 0.0206 |  2.5815 $\pm$ 0.2470 |  0.0513 $\pm$ 0.0013
BBP |  1.0125 $\pm$ 0.0113 |  0.8139 $\pm$ 0.0110 |  0.5990 $\pm$ 0.0499 |  0.0337 $\pm$ 0.0013
SGLD |  0.9978 $\pm$ 0.0340 |  0.8083 $\pm$ 0.0225 |  1.2727 $\pm$ 0.0711 |  0.0464 $\pm$ 0.0013
Ensembles |  0.9752 $\pm$ - |  0.7763 $\pm$ - |  1.4981 $\pm$ - |  0.0470 $\pm$ -
GROVER | Deterministic |  0.9355 $\pm$ 0.0252 |  0.7396 $\pm$ 0.0193 |  0.9888 $\pm$ 0.0395 |  0.0451 $\pm$ 0.0006
MC Dropout |  0.9437 $\pm$ 0.0246 |  0.7491 $\pm$ 0.0216 |  1.0065 $\pm$ 0.0351 |  0.0449 $\pm$ 0.0007
SWAG |  0.9443 $\pm$ 0.0219 |  0.7420 $\pm$ 0.0189 |  1.2996 $\pm$ 0.1883 |  0.0470 $\pm$ 0.0011
BBP |  0.9892 $\pm$ 0.0430 |  0.7821 $\pm$ 0.0327 |  0.5470 $\pm$ 0.0378 |  0.0153 $\pm$ 0.0028
SGLD |  0.9955 $\pm$ 0.0104 |  0.7942 $\pm$ 0.0153 |  0.4786 $\pm$ 0.0140 |  0.0243 $\pm$ 0.0008
Ensembles |  0.9110 $\pm$ - |  0.7209 $\pm$ - |  0.5854 $\pm$ - |  0.0394 $\pm$ -
Uni-Mol | Deterministic |  0.8530 $\pm$ 0.0224 |  0.6809 $\pm$ 0.0164 |  1.0739 $\pm$ 0.1136 |  0.0527 $\pm$ 0.0004
MC Dropout |  0.8367 $\pm$ 0.0237 |  0.6648 $\pm$ 0.0209 |  1.1098 $\pm$ 0.2625 |  0.0527 $\pm$ 0.0012
SWAG |  0.8644 $\pm$ 0.0097 |  0.6950 $\pm$ 0.0042 |  1.3868 $\pm$ 0.1522 |  0.0540 $\pm$ 0.0003
BBP |  0.8441 $\pm$ 0.0332 |  0.6719 $\pm$ 0.0279 |  0.3151 $\pm$ 0.0292 |  0.0329 $\pm$ 0.0015
SGLD | 0.8037 $\pm$ 0.0121 | 0.6444 $\pm$ 0.0112 | 0.2685 $\pm$ 0.0187 |  0.0343 $\pm$ 0.0018
Ensembles |  0.8181 $\pm$ - |  0.6516 $\pm$ - |  0.8372 $\pm$ - |  0.0512 $\pm$ -
TorchMD-NET | Deterministic |  0.8772 $\pm$ 0.0638 |  0.6688 $\pm$ 0.0517 |  0.6817 $\pm$ 0.1947 |  0.0409 $\pm$ 0.0059
MC Dropout |  0.8720 $\pm$ 0.0645 |  0.6660 $\pm$ 0.0510 |  0.6166 $\pm$ 0.1780 |  0.0402 $\pm$ 0.0063
SWAG |  0.8569 $\pm$ 0.0581 |  0.6642 $\pm$ 0.0401 |  0.4144 $\pm$ 0.1136 |  0.0372 $\pm$ 0.0053
BBP |  2.4684 $\pm$ 0.5232 |  1.9253 $\pm$ 0.3748 |  1.5283 $\pm$ 0.0753 |  0.0191 $\pm$ 0.0173
SGLD |  0.9691 $\pm$ 0.0579 |  0.7843 $\pm$ 0.0439 |  0.5284 $\pm$ 0.0243 |  0.0143 $\pm$ 0.0016
Ensembles |  1.0512 $\pm$ - |  0.8394 $\pm$ - |  0.9756 $\pm$ - | 0.0005 $\pm$ -
GIN | Deterministic |  1.5513 $\pm$ 0.0639 |  1.2268 $\pm$ 0.0618 |  1.0705 $\pm$ 0.1566 |  0.0122 $\pm$ 0.0072
MC Dropout |  1.5513 $\pm$ 0.0639 |  1.2268 $\pm$ 0.0618 |  1.0705 $\pm$ 0.1566 |  0.0122 $\pm$ 0.0072
SWAG |  1.8390 $\pm$ 0.2814 |  1.4650 $\pm$ 0.2128 |  1.4527 $\pm$ 0.2933 |  0.0166 $\pm$ 0.0089
BBP |  1.9791 $\pm$ 0.0920 |  1.5041 $\pm$ 0.0290 |  1.4189 $\pm$ 0.2794 |  0.0048 $\pm$ 0.0013
SGLD |  3.3157 $\pm$ 2.1193 |  2.7240 $\pm$ 1.8086 |  1.6595 $\pm$ 0.4615 |  0.0424 $\pm$ 0.0276
Ensembles |  1.3952 $\pm$ - |  1.0846 $\pm$ - |  0.9598 $\pm$ - |  0.0025 $\pm$ -
Table 19: Test results on FreeSolv in the format of “metric mean $\pm$ standard deviation”. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
DNN-rdkit | Deterministic |  2.2403 $\pm$ 0.1366 |  1.6768 $\pm$ 0.1007 |  1.3370 $\pm$ 0.1120 |  0.0272 $\pm$ 0.0026
MC Dropout |  2.2316 $\pm$ 0.1070 |  1.6469 $\pm$ 0.1122 |  1.3357 $\pm$ 0.1257 |  0.0247 $\pm$ 0.0015
SWAG |  2.4516 $\pm$ 0.1188 |  1.7853 $\pm$ 0.1478 |  1.8461 $\pm$ 0.2920 |  0.0358 $\pm$ 0.0043
BBP |  2.8456 $\pm$ 0.1416 |  2.1049 $\pm$ 0.1037 |  1.5285 $\pm$ 0.0469 |  0.0173 $\pm$ 0.0056
SGLD |  2.5503 $\pm$ 0.1010 |  1.8549 $\pm$ 0.1092 |  1.4260 $\pm$ 0.0095 |  0.0137 $\pm$ 0.0033
Ensembles |  1.9627 $\pm$ - |  1.4359 $\pm$ - |  1.0149 $\pm$ - |  0.0240 $\pm$ -
ChemBERTa | Deterministic |  2.5267 $\pm$ 0.0864 |  1.7760 $\pm$ 0.0318 |  1.7859 $\pm$ 0.1916 |  0.0364 $\pm$ 0.0034
MC Dropout |  2.5042 $\pm$ 0.0981 |  1.7804 $\pm$ 0.0456 |  1.7810 $\pm$ 0.1526 |  0.0359 $\pm$ 0.0033
SWAG |  2.6269 $\pm$ 0.1627 |  1.8562 $\pm$ 0.1055 |  2.2239 $\pm$ 0.2879 |  0.0426 $\pm$ 0.0003
BBP |  2.6340 $\pm$ 0.1368 |  1.9072 $\pm$ 0.1600 |  1.6143 $\pm$ 0.1930 |  0.0322 $\pm$ 0.0066
SGLD |  2.6564 $\pm$ 0.0357 |  1.8633 $\pm$ 0.0344 |  1.7627 $\pm$ 0.0809 |  0.0347 $\pm$ 0.0053
Ensembles |  2.3727 $\pm$ - |  1.6904 $\pm$ - |  1.5142 $\pm$ - |  0.0341 $\pm$ -
GROVER | Deterministic |  2.1352 $\pm$ 0.0193 |  1.4770 $\pm$ 0.0303 |  2.8244 $\pm$ 0.8307 |  0.0428 $\pm$ 0.0034
MC Dropout |  2.0981 $\pm$ 0.0069 |  1.4660 $\pm$ 0.0231 |  2.6575 $\pm$ 0.7926 |  0.0415 $\pm$ 0.0040
SWAG |  2.2321 $\pm$ 0.0902 |  1.5041 $\pm$ 0.0460 |  3.3021 $\pm$ 0.0530 |  0.0469 $\pm$ 0.0008
BBP |  2.6133 $\pm$ 0.0662 |  1.8288 $\pm$ 0.0843 |  1.4699 $\pm$ 0.0598 |  0.0126 $\pm$ 0.0014
SGLD |  2.1117 $\pm$ 0.0985 |  1.3784 $\pm$ 0.0459 |  1.2791 $\pm$ 0.1411 |  0.0238 $\pm$ 0.0020
Ensembles |  2.0238 $\pm$ - |  1.3539 $\pm$ - |  1.9474 $\pm$ - |  0.0393 $\pm$ -
Uni-Mol | Deterministic |  1.7871 $\pm$ 0.1265 |  1.2906 $\pm$ 0.0483 |  2.5450 $\pm$ 1.2435 |  0.0521 $\pm$ 0.0018
MC Dropout |  1.7585 $\pm$ 0.1322 |  1.2652 $\pm$ 0.0559 |  2.3425 $\pm$ 1.2327 |  0.0515 $\pm$ 0.0018
SWAG |  1.8524 $\pm$ 0.1449 |  1.3481 $\pm$ 0.1063 |  2.9865 $\pm$ 1.1583 |  0.0545 $\pm$ 0.0013
BBP |  1.7464 $\pm$ 0.0246 |  1.2424 $\pm$ 0.0049 | 0.9940 $\pm$ 0.0065 |  0.0289 $\pm$ 0.0027
SGLD |  1.7462 $\pm$ 0.0908 |  1.2963 $\pm$ 0.0610 |  1.0358 $\pm$ 0.0881 |  0.0296 $\pm$ 0.0048
Ensembles | 1.6716 $\pm$ - | 1.1780 $\pm$ - |  1.4306 $\pm$ - |  0.0504 $\pm$ -
TorchMD-NET | Deterministic |  2.6436 $\pm$ 0.0969 |  1.9379 $\pm$ 0.1236 |  1.9251 $\pm$ 0.2705 |  0.0219 $\pm$ 0.0060
MC Dropout |  2.6360 $\pm$ 0.0986 |  1.9475 $\pm$ 0.1212 |  1.8397 $\pm$ 0.2050 |  0.0212 $\pm$ 0.0052
SWAG |  2.6997 $\pm$ 0.1009 |  1.8466 $\pm$ 0.0863 |  2.5681 $\pm$ 0.4816 |  0.0399 $\pm$ 0.0026
BBP |  3.6709 $\pm$ 0.7927 |  2.7858 $\pm$ 0.6062 |  1.8484 $\pm$ 0.2682 |  0.0227 $\pm$ 0.0055
SGLD |  2.7769 $\pm$ 0.1054 |  1.9696 $\pm$ 0.1725 |  1.5583 $\pm$ 0.1031 |  0.0155 $\pm$ 0.0025
Ensembles |  2.9347 $\pm$ - |  2.0826 $\pm$ - |  1.6610 $\pm$ - |  0.0138 $\pm$ -
GIN | Deterministic |  2.2009 $\pm$ 0.2750 |  1.7292 $\pm$ 0.2873 |  1.3182 $\pm$ 0.0981 |  0.0142 $\pm$ 0.0039
MC Dropout |  2.2009 $\pm$ 0.2750 |  1.7292 $\pm$ 0.2873 |  1.3182 $\pm$ 0.0981 |  0.0142 $\pm$ 0.0039
SWAG |  2.3232 $\pm$ 0.0818 |  1.8154 $\pm$ 0.1109 |  1.5179 $\pm$ 0.1044 |  0.0198 $\pm$ 0.0106
BBP |  3.1290 $\pm$ 0.6024 |  2.4897 $\pm$ 0.5111 |  1.8594 $\pm$ 0.2348 |  0.0092 $\pm$ 0.0048
SGLD |  2.7563 $\pm$ 0.6927 |  2.2125 $\pm$ 0.6421 |  1.6046 $\pm$ 0.2001 |  0.0116 $\pm$ 0.0093
Ensembles |  2.0088 $\pm$ - |  1.5676 $\pm$ - |  1.5171 $\pm$ - | 0.0005 $\pm$ -
Table 20: Test results on Lipophilicity in the format of “metric mean $\pm$ standard deviation”. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.7575 $\pm$ 0.0116 |  0.5793 $\pm$ 0.0138 |  0.6154 $\pm$ 0.2016 |  0.0293 $\pm$ 0.0030
MC Dropout |  0.7559 $\pm$ 0.0110 |  0.5773 $\pm$ 0.0132 |  0.9071 $\pm$ 0.3721 |  0.0341 $\pm$ 0.0052
SWAG |  0.7572 $\pm$ 0.0101 |  0.5823 $\pm$ 0.0129 |  0.7191 $\pm$ 0.2267 |  0.0308 $\pm$ 0.0027
BBP |  0.7730 $\pm$ 0.0253 |  0.5938 $\pm$ 0.0245 |  0.7578 $\pm$ 0.1627 |  0.0305 $\pm$ 0.0046
SGLD |  0.7468 $\pm$ 0.0018 |  0.5743 $\pm$ 0.0016 |  0.2152 $\pm$ 0.0097 |  0.0090 $\pm$ 0.0009
Ensembles |  0.7172 $\pm$ - |  0.5490 $\pm$ - |  0.6165 $\pm$ - |  0.0322 $\pm$ -
ChemBERTa | Deterministic |  0.7553 $\pm$ 0.0074 |  0.5910 $\pm$ 0.0065 |  1.2368 $\pm$ 0.3994 |  0.0362 $\pm$ 0.0032
MC Dropout |  0.7142 $\pm$ 0.0063 |  0.5601 $\pm$ 0.0029 |  0.8178 $\pm$ 0.2701 |  0.0349 $\pm$ 0.0033
SWAG |  0.7672 $\pm$ 0.0101 |  0.5992 $\pm$ 0.0080 |  1.5809 $\pm$ 0.4168 |  0.0395 $\pm$ 0.0021
BBP |  0.7542 $\pm$ 0.0050 |  0.5869 $\pm$ 0.0087 |  0.4419 $\pm$ 0.0775 |  0.0279 $\pm$ 0.0023
SGLD |  0.7622 $\pm$ 0.0045 |  0.5982 $\pm$ 0.0070 |  0.8719 $\pm$ 0.1359 |  0.0355 $\pm$ 0.0020
Ensembles |  0.7367 $\pm$ - |  0.5763 $\pm$ - |  0.9756 $\pm$ - |  0.0360 $\pm$ -
GROVER | Deterministic |  0.6316 $\pm$ 0.0066 |  0.4747 $\pm$ 0.0040 |  2.1512 $\pm$ 0.2203 |  0.0478 $\pm$ 0.0009
MC Dropout |  0.6293 $\pm$ 0.0067 |  0.4740 $\pm$ 0.0028 |  2.0526 $\pm$ 0.2139 |  0.0476 $\pm$ 0.0009
SWAG |  0.6317 $\pm$ 0.0075 |  0.4750 $\pm$ 0.0041 |  2.3980 $\pm$ 0.3597 |  0.0485 $\pm$ 0.0007
BBP |  0.6481 $\pm$ 0.0063 |  0.5058 $\pm$ 0.0041 |  0.0789 $\pm$ 0.0206 |  0.0196 $\pm$ 0.0024
SGLD |  0.6360 $\pm$ 0.0041 |  0.4984 $\pm$ 0.0017 |  0.0544 $\pm$ 0.0199 |  0.0215 $\pm$ 0.0010
Ensembles |  0.6250 $\pm$ - |  0.4693 $\pm$ - |  1.6046 $\pm$ - |  0.0460 $\pm$ -
Uni-Mol | Deterministic |  0.6079 $\pm$ 0.0032 |  0.4509 $\pm$ 0.0044 |  0.8975 $\pm$ 0.5565 |  0.0425 $\pm$ 0.0061
MC Dropout |  0.5983 $\pm$ 0.0099 |  0.4438 $\pm$ 0.0117 |  1.3663 $\pm$ 1.2187 |  0.0440 $\pm$ 0.0084
SWAG |  0.6026 $\pm$ 0.0004 |  0.4476 $\pm$ 0.0022 |  1.0101 $\pm$ 0.1639 |  0.0453 $\pm$ 0.0012
BBP |  0.6044 $\pm$ 0.0016 |  0.4469 $\pm$ 0.0025 | 0.0679 $\pm$ 0.0218 |  0.0306 $\pm$ 0.0010
SGLD |  0.6040 $\pm$ 0.0031 |  0.4554 $\pm$ 0.0042 |  0.1565 $\pm$ 0.0573 |  0.0329 $\pm$ 0.0017
Ensembles | 0.5809 $\pm$ - | 0.4266 $\pm$ - |  0.6450 $\pm$ - |  0.0438 $\pm$ -
TorchMD-NET | Deterministic |  0.8235 $\pm$ 0.2218 |  0.6462 $\pm$ 0.1853 |  4.3336 $\pm$ 2.6567 |  0.0447 $\pm$ 0.0127
MC Dropout |  0.8887 $\pm$ 0.3142 |  0.7001 $\pm$ 0.2635 |  3.9796 $\pm$ 2.2502 |  0.0439 $\pm$ 0.0130
SWAG |  0.6819 $\pm$ 0.0271 |  0.5288 $\pm$ 0.0225 |  2.2934 $\pm$ 1.5338 |  0.0394 $\pm$ 0.0144
BBP |  2.8400 $\pm$ 2.2383 |  2.5087 $\pm$ 2.1497 |  1.6533 $\pm$ 0.9853 |  0.1008 $\pm$ 0.1219
SGLD |  0.8608 $\pm$ 0.0679 |  0.7002 $\pm$ 0.0502 |  0.9818 $\pm$ 0.4045 |  0.0183 $\pm$ 0.0128
Ensembles |  1.0313 $\pm$ - |  0.8196 $\pm$ - |  0.8619 $\pm$ - |  0.0195 $\pm$ -
GIN | Deterministic |  0.9286 $\pm$ 0.1113 |  0.7455 $\pm$ 0.1056 |  0.4542 $\pm$ 0.1096 |  0.0048 $\pm$ 0.0055
MC Dropout |  0.9286 $\pm$ 0.1113 |  0.7455 $\pm$ 0.1056 |  0.4542 $\pm$ 0.1096 |  0.0048 $\pm$ 0.0055
SWAG |  1.1105 $\pm$ 0.2448 |  0.9010 $\pm$ 0.2179 |  0.6805 $\pm$ 0.2447 |  0.0147 $\pm$ 0.0090
BBP |  1.0863 $\pm$ 0.0704 |  0.8606 $\pm$ 0.0492 |  0.6095 $\pm$ 0.0841 | 0.0018 $\pm$ 0.0010
SGLD |  1.3804 $\pm$ 0.4327 |  1.1187 $\pm$ 0.3763 |  0.9884 $\pm$ 0.4025 |  0.0229 $\pm$ 0.0137
Ensembles |  0.8071 $\pm$ - |  0.6515 $\pm$ - |  0.3241 $\pm$ - |  0.0020 $\pm$ -
Table 21: Test results on QM7 in the format of “metric mean $\pm$ standard deviation”. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
DNN-rdkit | Deterministic |  142.9279 $\pm$ 2.9477 |  95.7200 $\pm$ 3.2512 |  10.2976 $\pm$ 1.0848 |  0.0393 $\pm$ 0.0036
MC Dropout |  138.6817 $\pm$ 2.5873 |  90.7039 $\pm$ 3.1531 |  7.4627 $\pm$ 0.4138 |  0.0339 $\pm$ 0.0020
SWAG |  143.6922 $\pm$ 1.9420 |  97.2590 $\pm$ 1.9750 |  11.3649 $\pm$ 1.4063 |  0.0418 $\pm$ 0.0041
BBP |  141.5333 $\pm$ 4.8179 |  99.0430 $\pm$ 3.9287 |  6.8897 $\pm$ 0.8805 |  0.0330 $\pm$ 0.0064
SGLD |  139.8857 $\pm$ 5.4764 |  99.4873 $\pm$ 2.0159 |  5.4071 $\pm$ 0.0338 |  0.0098 $\pm$ 0.0019
Ensembles |  128.7182 $\pm$ - |  84.7269 $\pm$ - |  5.4568 $\pm$ - |  0.0281 $\pm$ -
ChemBERTa | Deterministic |  114.1954 $\pm$ 1.9473 |  66.6504 $\pm$ 3.6637 |  6.9619 $\pm$ 1.4000 |  0.0251 $\pm$ 0.0059
MC Dropout |  112.4931 $\pm$ 1.2013 |  64.7828 $\pm$ 3.4349 |  6.1942 $\pm$ 0.7998 |  0.0309 $\pm$ 0.0061
SWAG |  114.2023 $\pm$ 1.5302 |  68.0992 $\pm$ 3.5122 |  6.9684 $\pm$ 1.3236 |  0.0236 $\pm$ 0.0052
BBP |  116.2333 $\pm$ 1.8548 |  69.9783 $\pm$ 0.3732 |  5.5954 $\pm$ 0.2659 |  0.0262 $\pm$ 0.0034
SGLD |  116.9667 $\pm$ 0.3664 |  74.6714 $\pm$ 1.3652 |  5.0906 $\pm$ 0.0281 |  0.0147 $\pm$ 0.0019
Ensembles |  111.2764 $\pm$ - |  63.8752 $\pm$ - |  5.3896 $\pm$ - |  0.0224 $\pm$ -
GROVER | Deterministic |  139.5969 $\pm$ 3.1922 |  98.6424 $\pm$ 4.8571 |  5.4936 $\pm$ 0.0870 |  0.0121 $\pm$ 0.0012
MC Dropout |  136.1780 $\pm$ 2.9204 |  95.4940 $\pm$ 4.4117 |  5.4272 $\pm$ 0.0697 |  0.0107 $\pm$ 0.0014
SWAG |  142.6412 $\pm$ 1.3203 |  107.1310 $\pm$ 1.6574 |  5.7041 $\pm$ 0.1256 |  0.0153 $\pm$ 0.0018
BBP |  136.8576 $\pm$ 3.2036 |  99.8460 $\pm$ 2.2180 |  5.4555 $\pm$ 0.0135 | 0.0035 $\pm$ 0.0007
SGLD |  159.7489 $\pm$ 2.9839 |  126.6406 $\pm$ 2.8867 |  5.5587 $\pm$ 0.0308 |  0.0072 $\pm$ 0.0009
Ensembles |  131.9411 $\pm$ - |  89.4504 $\pm$ - |  5.3241 $\pm$ - |  0.0090 $\pm$ -
Uni-Mol | Deterministic |  103.9043 $\pm$ 1.3754 |  56.6287 $\pm$ 2.3360 |  4.6198 $\pm$ 0.2317 |  0.0338 $\pm$ 0.0065
MC Dropout |  103.5853 $\pm$ 1.3201 |  54.8976 $\pm$ 1.3061 |  4.5158 $\pm$ 0.2036 |  0.0386 $\pm$ 0.0019
SWAG |  107.8128 $\pm$ 0.1969 |  63.3542 $\pm$ 2.2114 |  5.5588 $\pm$ 0.6515 |  0.0412 $\pm$ 0.0026
BBP |  101.5479 $\pm$ 1.1382 |  51.9550 $\pm$ 0.7905 |  4.5153 $\pm$ 0.0535 |  0.0263 $\pm$ 0.0022
SGLD |  105.1573 $\pm$ 2.6264 |  57.0601 $\pm$ 4.5219 |  4.5642 $\pm$ 0.0453 |  0.0290 $\pm$ 0.0014
Ensembles | 101.1805 $\pm$ - |  52.0409 $\pm$ - | 4.3732 $\pm$ - |  0.0360 $\pm$ -
TorchMD-NET | Deterministic |  104.1467 $\pm$ 0.3144 |  49.6743 $\pm$ 0.7443 |  11.6606 $\pm$ 0.7198 |  0.0619 $\pm$ 0.0004
MC Dropout |  104.1265 $\pm$ 0.3415 |  49.8274 $\pm$ 0.5136 |  10.8641 $\pm$ 0.6233 |  0.0609 $\pm$ 0.0003
SWAG |  103.6753 $\pm$ 0.6387 |  50.1259 $\pm$ 1.4952 |  7.4456 $\pm$ 0.2718 |  0.0580 $\pm$ 0.0005
BBP |  104.4771 $\pm$ 0.4558 |  51.7268 $\pm$ 0.6870 |  9.5890 $\pm$ 0.4097 |  0.0568 $\pm$ 0.0002
SGLD |  112.7406 $\pm$ 3.9319 |  69.4583 $\pm$ 2.8874 |  5.1299 $\pm$ 0.0314 |  0.0212 $\pm$ 0.0022
Ensembles |  102.0535 $\pm$ - | 48.2234 $\pm$ - |  10.6292 $\pm$ - |  0.0618 $\pm$ -
GIN | Deterministic |  122.4303 $\pm$ 0.0690 |  76.2333 $\pm$ 1.0882 |  5.2490 $\pm$ 0.0500 |  0.0136 $\pm$ 0.0003
MC Dropout |  122.4303 $\pm$ 0.0690 |  76.2333 $\pm$ 1.0882 |  5.2490 $\pm$ 0.0500 |  0.0136 $\pm$ 0.0003
SWAG |  125.4810 $\pm$ 0.9210 |  83.3207 $\pm$ 1.3398 |  5.3423 $\pm$ 0.0619 |  0.0129 $\pm$ 0.0001
BBP |  125.2453 $\pm$ 0.6326 |  85.2772 $\pm$ 1.4557 |  5.3348 $\pm$ 0.0059 |  0.0077 $\pm$ 0.0014
SGLD |  132.1431 $\pm$ 6.9275 |  97.6270 $\pm$ 11.0966 |  5.3910 $\pm$ 0.0487 |  0.0126 $\pm$ 0.0011
Ensembles |  118.0550 $\pm$ - |  71.2196 $\pm$ - |  5.1922 $\pm$ - |  0.0105 $\pm$ -
Table 22: Test results on QM8 in the format of “metric mean $\pm$ standard deviation”. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.0356 $\pm$ 0.0004 |  0.0209 $\pm$ 0.0001 |  -0.7811 $\pm$ 2.1740 |  0.0282 $\pm$ 0.0008
MC Dropout |  0.0352 $\pm$ 0.0002 |  0.0209 $\pm$ 0.0002 |  -2.7716 $\pm$ 0.1922 |  0.0267 $\pm$ 0.0001
SWAG |  0.0360 $\pm$ 0.0002 |  0.0212 $\pm$ 0.0001 |  -0.7143 $\pm$ 1.5808 |  0.0289 $\pm$ 0.0005
BBP |  0.0370 $\pm$ 0.0002 |  0.0214 $\pm$ 0.0001 |  -0.9807 $\pm$ 0.5485 |  0.0269 $\pm$ 0.0004
SGLD |  0.0446 $\pm$ 0.0008 |  0.0294 $\pm$ 0.0005 |  -2.7053 $\pm$ 0.0184 |  0.0121 $\pm$ 0.0001
Ensembles |  0.0345 $\pm$ - |  0.0202 $\pm$ - |  -3.0118 $\pm$ - |  0.0279 $\pm$ -
ChemBERTa | Deterministic |  0.0377 $\pm$ 0.0001 |  0.0208 $\pm$ 0.0000 |  3.8214 $\pm$ 1.1599 |  0.0356 $\pm$ 0.0001
MC Dropout |  0.0370 $\pm$ 0.0002 |  0.0204 $\pm$ 0.0001 |  -0.6884 $\pm$ 0.1974 |  0.0332 $\pm$ 0.0001
SWAG |  0.0379 $\pm$ 0.0002 |  0.0210 $\pm$ 0.0000 |  5.4548 $\pm$ 1.9400 |  0.0360 $\pm$ 0.0002
BBP |  0.0358 $\pm$ 0.0003 |  0.0206 $\pm$ 0.0001 |  -1.5642 $\pm$ 0.0700 |  0.0316 $\pm$ 0.0005
SGLD |  0.0362 $\pm$ 0.0002 |  0.0212 $\pm$ 0.0004 |  -3.1380 $\pm$ 0.0611 |  0.0169 $\pm$ 0.0007
Ensembles |  0.0373 $\pm$ - |  0.0205 $\pm$ - |  0.9180 $\pm$ - |  0.0349 $\pm$ -
GROVER | Deterministic |  0.0319 $\pm$ 0.0007 |  0.0174 $\pm$ 0.0003 |  0.0201 $\pm$ 1.4388 |  0.0386 $\pm$ 0.0020
MC Dropout |  0.0318 $\pm$ 0.0006 |  0.0174 $\pm$ 0.0003 |  -0.8810 $\pm$ 1.1057 |  0.0381 $\pm$ 0.0021
SWAG |  0.0323 $\pm$ 0.0004 |  0.0176 $\pm$ 0.0003 |  0.6655 $\pm$ 0.9582 |  0.0409 $\pm$ 0.0013
BBP |  0.0324 $\pm$ 0.0004 |  0.0183 $\pm$ 0.0003 |  -3.3037 $\pm$ 0.0260 |  0.0199 $\pm$ 0.0005
SGLD |  0.0340 $\pm$ 0.0007 |  0.0205 $\pm$ 0.0007 |  -3.1826 $\pm$ 0.0378 |  0.0157 $\pm$ 0.0006
Ensembles |  0.0315 $\pm$ - |  0.0171 $\pm$ - |  -1.1036 $\pm$ - |  0.0383 $\pm$ -
Uni-Mol | Deterministic |  0.0334 $\pm$ 0.0006 |  0.0172 $\pm$ 0.0000 |  2.7759 $\pm$ 2.9972 |  0.0432 $\pm$ 0.0020
MC Dropout |  0.0331 $\pm$ 0.0006 |  0.0171 $\pm$ 0.0001 |  0.8170 $\pm$ 2.8226 |  0.0412 $\pm$ 0.0023
SWAG |  0.0338 $\pm$ 0.0004 |  0.0173 $\pm$ 0.0001 |  3.5568 $\pm$ 2.5309 |  0.0440 $\pm$ 0.0013
BBP |  0.0309 $\pm$ 0.0007 |  0.0164 $\pm$ 0.0002 |  -2.4581 $\pm$ 0.4546 |  0.0316 $\pm$ 0.0023
SGLD | 0.0300 $\pm$ 0.0003 | 0.0163 $\pm$ 0.0001 |  -3.2399 $\pm$ 0.0908 |  0.0210 $\pm$ 0.0004
Ensembles |  0.0331 $\pm$ - |  0.0169 $\pm$ - |  -0.0541 $\pm$ - |  0.0421 $\pm$ -
TorchMD-NET | Deterministic |  0.0324 $\pm$ 0.0006 |  0.0178 $\pm$ 0.0006 |  1.4661 $\pm$ 2.0676 |  0.0439 $\pm$ 0.0045
MC Dropout |  0.0324 $\pm$ 0.0007 |  0.0179 $\pm$ 0.0006 |  0.2637 $\pm$ 1.5600 |  0.0419 $\pm$ 0.0042
SWAG |  0.0319 $\pm$ 0.0005 |  0.0177 $\pm$ 0.0005 |  -1.1581 $\pm$ 0.9675 |  0.0403 $\pm$ 0.0019
BBP |  0.0332 $\pm$ 0.0002 |  0.0176 $\pm$ 0.0001 |  4.4554 $\pm$ 0.7641 |  0.0469 $\pm$ 0.0007
SGLD |  0.0354 $\pm$ 0.0003 |  0.0217 $\pm$ 0.0001 |  -2.9735 $\pm$ 0.0091 | 0.0091 $\pm$ 0.0003
Ensembles |  0.0315 $\pm$ - |  0.0173 $\pm$ - |  -1.9286 $\pm$ - |  0.0419 $\pm$ -
GIN | Deterministic |  0.0344 $\pm$ 0.0004 |  0.0206 $\pm$ 0.0003 |  -3.3711 $\pm$ 0.0959 |  0.0271 $\pm$ 0.0008
MC Dropout |  0.0344 $\pm$ 0.0004 |  0.0206 $\pm$ 0.0003 |  -3.3711 $\pm$ 0.0959 |  0.0271 $\pm$ 0.0008
SWAG |  0.0352 $\pm$ 0.0005 |  0.0215 $\pm$ 0.0004 |  -3.3091 $\pm$ 0.0569 |  0.0277 $\pm$ 0.0003
BBP |  0.0352 $\pm$ 0.0003 |  0.0220 $\pm$ 0.0001 |  -3.2994 $\pm$ 0.0296 |  0.0238 $\pm$ 0.0007
SGLD |  0.0428 $\pm$ 0.0011 |  0.0290 $\pm$ 0.0009 |  -2.8866 $\pm$ 0.0191 |  0.0189 $\pm$ 0.0016
Ensembles |  0.0341 $\pm$ - |  0.0205 $\pm$ - | -3.4906 $\pm$ - |  0.0271 $\pm$ -
Table 23: Test results on QM9 in the format of “metric mean $\pm$ standard deviation”. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.0151 $\pm$ 0.0001 |  0.0101 $\pm$ 0.0001 |  -3.3798 $\pm$ 0.0592 |  0.0442 $\pm$ 0.0003
MC Dropout |  0.0148 $\pm$ 0.0001 |  0.0100 $\pm$ 0.0001 |  -3.5269 $\pm$ 0.0258 |  0.0433 $\pm$ 0.0003
SWAG |  0.0152 $\pm$ 0.0002 |  0.0102 $\pm$ 0.0001 |  -3.2845 $\pm$ 0.0814 |  0.0449 $\pm$ 0.0003
BBP |  0.0153 $\pm$ 0.0002 |  0.0103 $\pm$ 0.0001 |  -3.3470 $\pm$ 0.0280 |  0.0445 $\pm$ 0.0004
SGLD |  0.0196 $\pm$ 0.0003 |  0.0144 $\pm$ 0.0002 |  -3.3352 $\pm$ 0.0052 |  0.0070 $\pm$ 0.0003
Ensembles |  0.0143 $\pm$ - |  0.0096 $\pm$ - |  -3.6023 $\pm$ - |  0.0436 $\pm$ -
ChemBERTa | Deterministic |  0.0146 $\pm$ 0.0004 |  0.0092 $\pm$ 0.0002 |  -2.4104 $\pm$ 0.0698 |  0.0547 $\pm$ 0.0002
MC Dropout |  0.0141 $\pm$ 0.0002 |  0.0088 $\pm$ 0.0001 |  -3.1501 $\pm$ 0.1094 |  0.0513 $\pm$ 0.0005
SWAG |  0.0148 $\pm$ 0.0003 |  0.0092 $\pm$ 0.0002 |  -2.1703 $\pm$ 0.0205 |  0.0553 $\pm$ 0.0001
BBP |  0.0144 $\pm$ 0.0001 |  0.0093 $\pm$ 0.0001 |  -2.5932 $\pm$ 0.0665 |  0.0540 $\pm$ 0.0002
SGLD |  0.0153 $\pm$ 0.0001 |  0.0101 $\pm$ 0.0002 |  -3.7586 $\pm$ 0.0106 |  0.0338 $\pm$ 0.0007
Ensembles |  0.0140 $\pm$ - |  0.0087 $\pm$ - |  -2.8766 $\pm$ - |  0.0543 $\pm$ -
GROVER | Deterministic |  0.0115 $\pm$ 0.0000 |  0.0068 $\pm$ 0.0000 |  -0.7874 $\pm$ 0.5323 |  0.0621 $\pm$ 0.0008
MC Dropout |  0.0114 $\pm$ 0.0000 |  0.0068 $\pm$ 0.0000 |  -1.1002 $\pm$ 0.5003 |  0.0616 $\pm$ 0.0008
SWAG |  0.0116 $\pm$ 0.0000 |  0.0068 $\pm$ 0.0000 |  -0.4778 $\pm$ 0.1923 |  0.0625 $\pm$ 0.0002
BBP |  0.0118 $\pm$ 0.0000 |  0.0070 $\pm$ 0.0000 |  -1.8855 $\pm$ 0.1189 |  0.0591 $\pm$ 0.0004
SGLD |  0.0136 $\pm$ 0.0006 |  0.0088 $\pm$ 0.0003 |  -3.7855 $\pm$ 0.0411 |  0.0291 $\pm$ 0.0017
Ensembles |  0.0114 $\pm$ - |  0.0067 $\pm$ - |  -1.0286 $\pm$ - |  0.0620 $\pm$ -
Uni-Mol | Deterministic |  0.0096 $\pm$ 0.0001 |  0.0054 $\pm$ 0.0001 |  0.0143 $\pm$ 0.4195 |  0.0664 $\pm$ 0.0003
MC Dropout |  0.0096 $\pm$ 0.0001 |  0.0054 $\pm$ 0.0001 |  -0.2513 $\pm$ 0.3961 |  0.0661 $\pm$ 0.0003
SWAG |  0.0097 $\pm$ 0.0000 |  0.0054 $\pm$ 0.0000 |  -0.4624 $\pm$ 0.0619 |  0.0660 $\pm$ 0.0001
BBP |  0.0095 $\pm$ 0.0002 |  0.0054 $\pm$ 0.0000 |  -2.9595 $\pm$ 0.1250 |  0.0618 $\pm$ 0.0003
SGLD |  0.0095 $\pm$ 0.0001 |  0.0055 $\pm$ 0.0001 | -4.2093 $\pm$ 0.0049 |  0.0459 $\pm$ 0.0002
Ensembles |  0.0095 $\pm$ - |  0.0053 $\pm$ - |  -0.3192 $\pm$ - |  0.0663 $\pm$ -
TorchMD-NET | Deterministic |  0.0087 $\pm$ 0.0001 |  0.0048 $\pm$ 0.0000 |  3.6334 $\pm$ 1.0190 |  0.0690 $\pm$ 0.0003
MC Dropout |  0.0087 $\pm$ 0.0001 |  0.0048 $\pm$ 0.0000 |  3.1525 $\pm$ 1.0291 |  0.0687 $\pm$ 0.0003
SWAG |  0.0087 $\pm$ 0.0000 |  0.0048 $\pm$ 0.0000 |  -0.7161 $\pm$ 0.1242 |  0.0670 $\pm$ 0.0001
BBP | 0.0083 $\pm$ 0.0002 | 0.0046 $\pm$ 0.0001 |  1.7970 $\pm$ 1.8061 |  0.0685 $\pm$ 0.0010
SGLD |  0.0113 $\pm$ 0.0006 |  0.0075 $\pm$ 0.0004 |  -3.6560 $\pm$ 0.0469 | 0.0132 $\pm$ 0.0013
Ensembles |  0.0086 $\pm$ - | 0.0046 $\pm$ - |  2.2619 $\pm$ - |  0.0687 $\pm$ -
GIN | Deterministic |  0.0132 $\pm$ 0.0001 |  0.0084 $\pm$ 0.0001 |  -3.4286 $\pm$ 0.0389 |  0.0497 $\pm$ 0.0003
MC Dropout |  0.0132 $\pm$ 0.0001 |  0.0084 $\pm$ 0.0001 |  -3.4286 $\pm$ 0.0389 |  0.0497 $\pm$ 0.0003
SWAG |  0.0136 $\pm$ 0.0000 |  0.0088 $\pm$ 0.0001 |  -3.3355 $\pm$ 0.0589 |  0.0495 $\pm$ 0.0005
BBP |  0.0131 $\pm$ 0.0000 |  0.0084 $\pm$ 0.0000 |  -3.3469 $\pm$ 0.0650 |  0.0499 $\pm$ 0.0003
SGLD |  0.0161 $\pm$ 0.0003 |  0.0115 $\pm$ 0.0004 |  -3.5925 $\pm$ 0.0198 |  0.0211 $\pm$ 0.0031
Ensembles |  0.0130 $\pm$ - |  0.0081 $\pm$ - |  -3.5207 $\pm$ - |  0.0500 $\pm$ -
Table 24: Test results on the randomly split BACE dataset. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.8850 $\pm$ 0.0220 |  0.1123 $\pm$ 0.0233 |  0.4820 $\pm$ 0.0723 |  0.1380 $\pm$ 0.0266
Temperature |  0.8850 $\pm$ 0.0220 |  0.1022 $\pm$ 0.0019 |  0.4475 $\pm$ 0.0624 |  0.1359 $\pm$ 0.0231
Focal Loss |  0.8884 $\pm$ 0.0167 |  0.1351 $\pm$ 0.0228 |  0.4623 $\pm$ 0.0239 |  0.1471 $\pm$ 0.0072
MC Dropout |  0.8910 $\pm$ 0.0245 |  0.0969 $\pm$ 0.0134 |  0.4328 $\pm$ 0.0594 |  0.1344 $\pm$ 0.0215
SWAG |  0.8898 $\pm$ 0.0220 |  0.1019 $\pm$ 0.0070 |  0.4346 $\pm$ 0.0525 |  0.1346 $\pm$ 0.0211
BBP |  0.8784 $\pm$ 0.0133 |  0.1101 $\pm$ 0.0256 |  0.4954 $\pm$ 0.0755 |  0.1414 $\pm$ 0.0156
SGLD |  0.8830 $\pm$ 0.0214 |  0.1119 $\pm$ 0.0294 |  0.4706 $\pm$ 0.0793 |  0.1360 $\pm$ 0.0244
Ensembles |  0.8872 $\pm$ 0.0223 |  0.0972 $\pm$ 0.0101 |  0.4572 $\pm$ 0.0712 |  0.1308 $\pm$ 0.0252
ChemBERTa | Deterministic |  0.8835 $\pm$ 0.0044 |  0.1240 $\pm$ 0.0226 |  0.5182 $\pm$ 0.0812 |  0.1492 $\pm$ 0.0072
Temperature |  0.8835 $\pm$ 0.0044 |  0.1077 $\pm$ 0.0201 |  0.4566 $\pm$ 0.0274 |  0.1427 $\pm$ 0.0029
Focal Loss |  0.8884 $\pm$ 0.0096 |  0.1371 $\pm$ 0.0357 |  0.4490 $\pm$ 0.0328 |  0.1437 $\pm$ 0.0110
MC Dropout |  0.8825 $\pm$ 0.0124 |  0.0978 $\pm$ 0.0099 |  0.4544 $\pm$ 0.0082 |  0.1447 $\pm$ 0.0073
SWAG |  0.8868 $\pm$ 0.0059 |  0.1248 $\pm$ 0.0265 |  0.5286 $\pm$ 0.0964 |  0.1479 $\pm$ 0.0103
BBP |  0.8918 $\pm$ 0.0057 |  0.1230 $\pm$ 0.0362 |  0.5192 $\pm$ 0.0661 |  0.1468 $\pm$ 0.0117
SGLD |  0.8846 $\pm$ 0.0092 |  0.1362 $\pm$ 0.0046 |  0.5406 $\pm$ 0.0774 |  0.1438 $\pm$ 0.0073
Ensembles |  0.8959 $\pm$ 0.0058 |  0.1219 $\pm$ 0.0141 |  0.4715 $\pm$ 0.0441 |  0.1370 $\pm$ 0.0071
GROVER | Deterministic |  0.9067 $\pm$ 0.0148 |  0.1003 $\pm$ 0.0139 |  0.4230 $\pm$ 0.0372 |  0.1254 $\pm$ 0.0114
Temperature |  0.9067 $\pm$ 0.0148 |  0.0884 $\pm$ 0.0130 |  0.4049 $\pm$ 0.0351 |  0.1237 $\pm$ 0.0113
Focal Loss |  0.9084 $\pm$ 0.0154 |  0.1600 $\pm$ 0.0069 |  0.4651 $\pm$ 0.0323 |  0.1492 $\pm$ 0.0137
MC Dropout |  0.9078 $\pm$ 0.0168 | 0.0802 $\pm$ 0.0069 |  0.4073 $\pm$ 0.0394 |  0.1238 $\pm$ 0.0119
SWAG | 0.9112 $\pm$ 0.0141 |  0.0887 $\pm$ 0.0260 |  0.3916 $\pm$ 0.0351 |  0.1203 $\pm$ 0.0112
BBP |  0.9089 $\pm$ 0.0065 |  0.0807 $\pm$ 0.0112 |  0.3946 $\pm$ 0.0178 |  0.1209 $\pm$ 0.0103
SGLD |  0.9066 $\pm$ 0.0145 |  0.0852 $\pm$ 0.0101 |  0.4029 $\pm$ 0.0313 |  0.1231 $\pm$ 0.0097
Ensembles |  0.9046 $\pm$ 0.0133 |  0.0915 $\pm$ 0.0054 |  0.4207 $\pm$ 0.0289 |  0.1248 $\pm$ 0.0104
Uni-Mol | Deterministic |  0.9056 $\pm$ 0.0204 |  0.0894 $\pm$ 0.0129 |  0.4081 $\pm$ 0.0529 |  0.1257 $\pm$ 0.0191
Temperature |  0.9059 $\pm$ 0.0205 |  0.0854 $\pm$ 0.0082 |  0.4060 $\pm$ 0.0491 |  0.1261 $\pm$ 0.0180
Focal Loss |  0.8939 $\pm$ 0.0206 |  0.1634 $\pm$ 0.0148 |  0.5027 $\pm$ 0.0139 |  0.1655 $\pm$ 0.0065
MC Dropout |  0.9061 $\pm$ 0.0199 |  0.0864 $\pm$ 0.0086 |  0.4069 $\pm$ 0.0514 |  0.1256 $\pm$ 0.0188
SWAG |  0.9105 $\pm$ 0.0142 |  0.0988 $\pm$ 0.0122 | 0.3822 $\pm$ 0.0340 | 0.1179 $\pm$ 0.0131
BBP |  0.8990 $\pm$ 0.0154 |  0.0867 $\pm$ 0.0206 |  0.4303 $\pm$ 0.0513 |  0.1309 $\pm$ 0.0184
SGLD |  0.9080 $\pm$ 0.0104 |  0.0834 $\pm$ 0.0139 |  0.3949 $\pm$ 0.0317 |  0.1223 $\pm$ 0.0138
Ensembles |  0.9023 $\pm$ 0.0189 |  0.1020 $\pm$ 0.0140 |  0.4040 $\pm$ 0.0491 |  0.1272 $\pm$ 0.0220
Table 25: Test results on the randomly split BBBP dataset. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.9215 $\pm$ 0.0254 |  0.0838 $\pm$ 0.0047 |  0.3722 $\pm$ 0.0275 |  0.0918 $\pm$ 0.0052
Temperature |  0.9215 $\pm$ 0.0254 |  0.0711 $\pm$ 0.0035 |  0.3131 $\pm$ 0.0141 |  0.0878 $\pm$ 0.0063
Focal Loss |  0.9180 $\pm$ 0.0142 |  0.1214 $\pm$ 0.0528 |  0.3506 $\pm$ 0.0599 |  0.1038 $\pm$ 0.0188
MC Dropout |  0.9226 $\pm$ 0.0244 |  0.0683 $\pm$ 0.0007 |  0.3156 $\pm$ 0.0151 |  0.0864 $\pm$ 0.0079
SWAG |  0.9205 $\pm$ 0.0234 |  0.0816 $\pm$ 0.0065 |  0.3358 $\pm$ 0.0247 |  0.0876 $\pm$ 0.0081
BBP |  0.9238 $\pm$ 0.0149 |  0.0846 $\pm$ 0.0188 |  0.3752 $\pm$ 0.0675 |  0.0932 $\pm$ 0.0113
SGLD |  0.9172 $\pm$ 0.0284 |  0.0865 $\pm$ 0.0125 |  0.3930 $\pm$ 0.0735 |  0.0905 $\pm$ 0.0072
Ensembles |  0.9226 $\pm$ 0.0088 |  0.0700 $\pm$ 0.0120 |  0.3324 $\pm$ 0.0220 |  0.0856 $\pm$ 0.0066
ChemBERTa | Deterministic |  0.9605 $\pm$ 0.0034 |  0.0686 $\pm$ 0.0066 |  0.2494 $\pm$ 0.0089 |  0.0731 $\pm$ 0.0030
Temperature |  0.9605 $\pm$ 0.0034 |  0.0591 $\pm$ 0.0013 | 0.2275 $\pm$ 0.0111 | 0.0697 $\pm$ 0.0040
Focal Loss |  0.9587 $\pm$ 0.0027 |  0.0816 $\pm$ 0.0262 |  0.2645 $\pm$ 0.0437 |  0.0795 $\pm$ 0.0133
MC Dropout | 0.9607 $\pm$ 0.0027 |  0.0615 $\pm$ 0.0094 |  0.2438 $\pm$ 0.0193 |  0.0717 $\pm$ 0.0018
SWAG |  0.9588 $\pm$ 0.0018 |  0.0737 $\pm$ 0.0041 |  0.2856 $\pm$ 0.0097 |  0.0780 $\pm$ 0.0039
BBP |  0.9590 $\pm$ 0.0011 |  0.0702 $\pm$ 0.0125 |  0.2979 $\pm$ 0.0891 |  0.0766 $\pm$ 0.0052
SGLD |  0.9580 $\pm$ 0.0033 |  0.0854 $\pm$ 0.0057 |  0.3513 $\pm$ 0.0324 |  0.0826 $\pm$ 0.0055
Ensembles |  0.9604 $\pm$ 0.0018 |  0.0691 $\pm$ 0.0124 |  0.2480 $\pm$ 0.0045 |  0.0716 $\pm$ 0.0028
GROVER | Deterministic |  0.9300 $\pm$ 0.0172 |  0.0551 $\pm$ 0.0138 |  0.2746 $\pm$ 0.0256 |  0.0742 $\pm$ 0.0075
Temperature |  0.9300 $\pm$ 0.0171 |  0.0575 $\pm$ 0.0185 |  0.2689 $\pm$ 0.0252 |  0.0740 $\pm$ 0.0074
Focal Loss |  0.9294 $\pm$ 0.0198 |  0.1944 $\pm$ 0.0271 |  0.3963 $\pm$ 0.0258 |  0.1153 $\pm$ 0.0102
MC Dropout |  0.9311 $\pm$ 0.0171 | 0.0549 $\pm$ 0.0115 |  0.2701 $\pm$ 0.0242 |  0.0742 $\pm$ 0.0072
SWAG |  0.9324 $\pm$ 0.0219 |  0.0601 $\pm$ 0.0112 |  0.2736 $\pm$ 0.0300 |  0.0741 $\pm$ 0.0097
BBP |  0.9196 $\pm$ 0.0210 | 0.0549 $\pm$ 0.0103 |  0.2712 $\pm$ 0.0225 |  0.0751 $\pm$ 0.0084
SGLD |  0.9306 $\pm$ 0.0182 |  0.0588 $\pm$ 0.0140 |  0.2750 $\pm$ 0.0304 |  0.0757 $\pm$ 0.0092
Ensembles |  0.9320 $\pm$ 0.0141 |  0.0633 $\pm$ 0.0124 |  0.2739 $\pm$ 0.0305 |  0.0738 $\pm$ 0.0092
Uni-Mol | Deterministic |  0.9285 $\pm$ 0.0126 |  0.0785 $\pm$ 0.0103 |  0.3165 $\pm$ 0.0332 |  0.0810 $\pm$ 0.0108
Temperature |  0.9272 $\pm$ 0.0121 |  0.0687 $\pm$ 0.0094 |  0.2838 $\pm$ 0.0231 |  0.0790 $\pm$ 0.0099
Focal Loss |  0.9242 $\pm$ 0.0086 |  0.1001 $\pm$ 0.0261 |  0.3201 $\pm$ 0.0332 |  0.0936 $\pm$ 0.0109
MC Dropout |  0.9284 $\pm$ 0.0125 |  0.0750 $\pm$ 0.0087 |  0.3124 $\pm$ 0.0322 |  0.0807 $\pm$ 0.0107
SWAG |  0.9359 $\pm$ 0.0048 |  0.0797 $\pm$ 0.0086 |  0.3233 $\pm$ 0.0266 |  0.0792 $\pm$ 0.0091
BBP |  0.9175 $\pm$ 0.0121 |  0.0706 $\pm$ 0.0047 |  0.3256 $\pm$ 0.0262 |  0.0809 $\pm$ 0.0094
SGLD |  0.9234 $\pm$ 0.0069 |  0.0869 $\pm$ 0.0053 |  0.3671 $\pm$ 0.0262 |  0.0846 $\pm$ 0.0109
Ensembles |  0.9341 $\pm$ 0.0117 |  0.0652 $\pm$ 0.0140 |  0.2925 $\pm$ 0.0429 |  0.0766 $\pm$ 0.0107
Table 26: Test results on the randomly split ClinTox dataset. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.8659 $\pm$ 0.0483 |  0.0754 $\pm$ 0.0129 |  0.5589 $\pm$ 0.4190 |  0.0672 $\pm$ 0.0104
Temperature |  0.8628 $\pm$ 0.0300 |  0.0719 $\pm$ 0.0087 |  0.4434 $\pm$ 0.1979 |  0.0645 $\pm$ 0.0068
Focal Loss |  0.8813 $\pm$ 0.0564 |  0.0652 $\pm$ 0.0054 |  0.1980 $\pm$ 0.0696 |  0.0523 $\pm$ 0.0093
MC Dropout |  0.8757 $\pm$ 0.0382 |  0.0616 $\pm$ 0.0093 |  0.2560 $\pm$ 0.0963 |  0.0566 $\pm$ 0.0067
SWAG |  0.8819 $\pm$ 0.0424 |  0.0619 $\pm$ 0.0070 |  0.2574 $\pm$ 0.1058 |  0.0572 $\pm$ 0.0066
BBP |  0.8358 $\pm$ 0.0778 |  0.0547 $\pm$ 0.0198 |  0.2679 $\pm$ 0.1254 |  0.0534 $\pm$ 0.0083
SGLD |  0.8745 $\pm$ 0.0506 |  0.0666 $\pm$ 0.0085 |  0.2488 $\pm$ 0.0714 |  0.0572 $\pm$ 0.0068
Ensembles |  0.8241 $\pm$ 0.0587 |  0.0646 $\pm$ 0.0088 |  0.4562 $\pm$ 0.3799 |  0.0561 $\pm$ 0.0077
ChemBERTa | Deterministic |  0.9862 $\pm$ 0.0063 |  0.0193 $\pm$ 0.0026 |  0.0703 $\pm$ 0.0063 |  0.0168 $\pm$ 0.0015
Temperature |  0.9862 $\pm$ 0.0063 |  0.0181 $\pm$ 0.0045 |  0.0639 $\pm$ 0.0039 |  0.0163 $\pm$ 0.0013
Focal Loss |  0.9824 $\pm$ 0.0038 |  0.0493 $\pm$ 0.0183 |  0.0907 $\pm$ 0.0132 |  0.0193 $\pm$ 0.0025
MC Dropout |  0.9870 $\pm$ 0.0051 | 0.0162 $\pm$ 0.0029 |  0.0640 $\pm$ 0.0087 | 0.0148 $\pm$ 0.0025
SWAG |  0.9863 $\pm$ 0.0060 |  0.0216 $\pm$ 0.0036 |  0.0751 $\pm$ 0.0084 |  0.0179 $\pm$ 0.0019
BBP |  0.9855 $\pm$ 0.0077 |  0.0268 $\pm$ 0.0124 |  0.0687 $\pm$ 0.0095 |  0.0165 $\pm$ 0.0021
SGLD |  0.9851 $\pm$ 0.0053 |  0.0201 $\pm$ 0.0058 |  0.0713 $\pm$ 0.0020 |  0.0178 $\pm$ 0.0035
Ensembles | 0.9885 $\pm$ 0.0034 |  0.0242 $\pm$ 0.0108 | 0.0633 $\pm$ 0.0056 |  0.0154 $\pm$ 0.0015
GROVER | Deterministic |  0.8823 $\pm$ 0.0347 |  0.0392 $\pm$ 0.0059 |  0.1550 $\pm$ 0.0289 |  0.0409 $\pm$ 0.0069
Temperature |  0.8818 $\pm$ 0.0341 |  0.0387 $\pm$ 0.0041 |  0.1575 $\pm$ 0.0294 |  0.0412 $\pm$ 0.0069
Focal Loss |  0.8554 $\pm$ 0.0542 |  0.1687 $\pm$ 0.0024 |  0.2767 $\pm$ 0.0073 |  0.0682 $\pm$ 0.0026
MC Dropout |  0.8840 $\pm$ 0.0332 |  0.0324 $\pm$ 0.0015 |  0.1505 $\pm$ 0.0242 |  0.0392 $\pm$ 0.0058
SWAG |  0.8897 $\pm$ 0.0337 |  0.0400 $\pm$ 0.0033 |  0.1490 $\pm$ 0.0267 |  0.0397 $\pm$ 0.0060
BBP |  0.8466 $\pm$ 0.0496 |  0.0684 $\pm$ 0.0059 |  0.1748 $\pm$ 0.0226 |  0.0419 $\pm$ 0.0064
SGLD |  0.8904 $\pm$ 0.0285 |  0.0424 $\pm$ 0.0021 |  0.1545 $\pm$ 0.0297 |  0.0414 $\pm$ 0.0086
Ensembles |  0.8854 $\pm$ 0.0371 |  0.0342 $\pm$ 0.0102 |  0.1470 $\pm$ 0.0295 |  0.0385 $\pm$ 0.0071
Uni-Mol | Deterministic |  0.8391 $\pm$ 0.0620 |  0.0625 $\pm$ 0.0119 |  0.1949 $\pm$ 0.0291 |  0.0531 $\pm$ 0.0078
Temperature |  0.8433 $\pm$ 0.0611 |  0.0633 $\pm$ 0.0119 |  0.1880 $\pm$ 0.0236 |  0.0526 $\pm$ 0.0073
Focal Loss |  0.8416 $\pm$ 0.0705 |  0.0961 $\pm$ 0.0199 |  0.2153 $\pm$ 0.0136 |  0.0586 $\pm$ 0.0050
MC Dropout |  0.8394 $\pm$ 0.0616 |  0.0627 $\pm$ 0.0112 |  0.1931 $\pm$ 0.0281 |  0.0528 $\pm$ 0.0077
SWAG |  0.8211 $\pm$ 0.0871 |  0.0515 $\pm$ 0.0131 |  0.1974 $\pm$ 0.0588 |  0.0484 $\pm$ 0.0097
BBP |  0.8389 $\pm$ 0.0345 |  0.0593 $\pm$ 0.0234 |  0.1788 $\pm$ 0.0387 |  0.0472 $\pm$ 0.0132
SGLD |  0.8367 $\pm$ 0.0855 |  0.0462 $\pm$ 0.0092 |  0.1867 $\pm$ 0.0655 |  0.0475 $\pm$ 0.0110
Ensembles |  0.8387 $\pm$ 0.0463 |  0.0671 $\pm$ 0.0197 |  0.1803 $\pm$ 0.0254 |  0.0469 $\pm$ 0.0032
Table 27: Test results on the randomly split Tox21 dataset. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.8451 $\pm$ 0.0162 |  0.0335 $\pm$ 0.0012 |  0.1999 $\pm$ 0.0041 |  0.0537 $\pm$ 0.0005
Temperature |  0.8451 $\pm$ 0.0162 |  0.0338 $\pm$ 0.0023 |  0.1945 $\pm$ 0.0016 |  0.0534 $\pm$ 0.0005
Focal Loss |  0.8422 $\pm$ 0.0120 |  0.0829 $\pm$ 0.0065 |  0.2265 $\pm$ 0.0055 |  0.0587 $\pm$ 0.0010
MC Dropout |  0.8494 $\pm$ 0.0137 |  0.0259 $\pm$ 0.0009 |  0.1885 $\pm$ 0.0012 |  0.0510 $\pm$ 0.0002
SWAG |  0.8496 $\pm$ 0.0129 |  0.0256 $\pm$ 0.0009 |  0.1882 $\pm$ 0.0008 |  0.0509 $\pm$ 0.0003
BBP |  0.8216 $\pm$ 0.0033 |  0.0293 $\pm$ 0.0018 |  0.2052 $\pm$ 0.0054 |  0.0565 $\pm$ 0.0011
SGLD |  0.8114 $\pm$ 0.0067 |  0.0864 $\pm$ 0.0038 |  0.2568 $\pm$ 0.0084 |  0.0685 $\pm$ 0.0028
Ensembles |  0.8544 $\pm$ 0.0101 |  0.0271 $\pm$ 0.0025 |  0.1849 $\pm$ 0.0018 |  0.0506 $\pm$ 0.0005
ChemBERTa | Deterministic |  0.8373 $\pm$ 0.0140 |  0.0345 $\pm$ 0.0046 |  0.1993 $\pm$ 0.0110 |  0.0545 $\pm$ 0.0027
Temperature |  0.8373 $\pm$ 0.0140 |  0.0360 $\pm$ 0.0020 |  0.1973 $\pm$ 0.0066 |  0.0543 $\pm$ 0.0022
Focal Loss |  0.8339 $\pm$ 0.0128 |  0.0939 $\pm$ 0.0078 |  0.2361 $\pm$ 0.0024 |  0.0605 $\pm$ 0.0007
MC Dropout |  0.8430 $\pm$ 0.0101 | 0.0240 $\pm$ 0.0017 |  0.1859 $\pm$ 0.0069 |  0.0506 $\pm$ 0.0019
SWAG |  0.8369 $\pm$ 0.0149 |  0.0370 $\pm$ 0.0025 |  0.2027 $\pm$ 0.0110 |  0.0553 $\pm$ 0.0022
BBP |  0.8301 $\pm$ 0.0131 |  0.0403 $\pm$ 0.0038 |  0.2052 $\pm$ 0.0077 |  0.0575 $\pm$ 0.0025
SGLD |  0.8314 $\pm$ 0.0127 |  0.0376 $\pm$ 0.0018 |  0.2002 $\pm$ 0.0102 |  0.0552 $\pm$ 0.0031
Ensembles |  0.8509 $\pm$ 0.0082 |  0.0304 $\pm$ 0.0027 |  0.1890 $\pm$ 0.0100 |  0.0519 $\pm$ 0.0025
GROVER | Deterministic |  0.8754 $\pm$ 0.0155 |  0.0259 $\pm$ 0.0018 |  0.1690 $\pm$ 0.0067 |  0.0457 $\pm$ 0.0015
Temperature |  0.8754 $\pm$ 0.0154 |  0.0281 $\pm$ 0.0030 |  0.1700 $\pm$ 0.0060 |  0.0458 $\pm$ 0.0015
Focal Loss | 0.8785 $\pm$ 0.0099 |  0.1337 $\pm$ 0.0012 |  0.2623 $\pm$ 0.0046 |  0.0636 $\pm$ 0.0017
MC Dropout |  0.8760 $\pm$ 0.0149 |  0.0279 $\pm$ 0.0015 |  0.1697 $\pm$ 0.0062 |  0.0457 $\pm$ 0.0015
SWAG |  0.8776 $\pm$ 0.0135 |  0.0247 $\pm$ 0.0014 |  0.1674 $\pm$ 0.0059 |  0.0452 $\pm$ 0.0014
BBP |  0.8667 $\pm$ 0.0135 |  0.0439 $\pm$ 0.0038 |  0.1850 $\pm$ 0.0049 |  0.0483 $\pm$ 0.0016
SGLD |  0.8724 $\pm$ 0.0123 |  0.0390 $\pm$ 0.0023 |  0.1811 $\pm$ 0.0061 |  0.0473 $\pm$ 0.0019
Ensembles |  0.8775 $\pm$ 0.0137 |  0.0244 $\pm$ 0.0010 |  0.1680 $\pm$ 0.0055 |  0.0453 $\pm$ 0.0013
Uni-Mol | Deterministic |  0.8666 $\pm$ 0.0138 |  0.0302 $\pm$ 0.0016 |  0.1773 $\pm$ 0.0052 |  0.0466 $\pm$ 0.0012
Temperature |  0.8661 $\pm$ 0.0136 |  0.0295 $\pm$ 0.0008 |  0.1727 $\pm$ 0.0037 |  0.0464 $\pm$ 0.0010
Focal Loss |  0.8631 $\pm$ 0.0071 |  0.0824 $\pm$ 0.0093 |  0.2154 $\pm$ 0.0097 |  0.0543 $\pm$ 0.0028
MC Dropout |  0.8667 $\pm$ 0.0139 |  0.0292 $\pm$ 0.0021 |  0.1758 $\pm$ 0.0050 |  0.0465 $\pm$ 0.0012
SWAG |  0.8666 $\pm$ 0.0170 |  0.0308 $\pm$ 0.0018 |  0.1779 $\pm$ 0.0042 |  0.0452 $\pm$ 0.0010
BBP |  0.8731 $\pm$ 0.0107 |  0.0259 $\pm$ 0.0031 |  0.1731 $\pm$ 0.0046 |  0.0469 $\pm$ 0.0013
SGLD |  0.8663 $\pm$ 0.0149 |  0.0333 $\pm$ 0.0027 |  0.1807 $\pm$ 0.0058 |  0.0470 $\pm$ 0.0011
Ensembles |  0.8778 $\pm$ 0.0090 |  0.0262 $\pm$ 0.0024 | 0.1660 $\pm$ 0.0043 | 0.0440 $\pm$ 0.0008
Table 28: Test results on the randomly split ESOL dataset. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.6530 $\pm$ 0.0615 |  0.4788 $\pm$ 0.0279 |  0.1067 $\pm$ 0.0716 |  0.0419 $\pm$ 0.0034
MC Dropout |  0.6597 $\pm$ 0.0666 |  0.4820 $\pm$ 0.0351 |  0.1212 $\pm$ 0.0925 |  0.0436 $\pm$ 0.0037
SWAG |  0.6542 $\pm$ 0.0766 |  0.4752 $\pm$ 0.0392 |  0.0977 $\pm$ 0.0884 |  0.0435 $\pm$ 0.0034
BBP |  0.7060 $\pm$ 0.0604 |  0.5294 $\pm$ 0.0458 |  0.2153 $\pm$ 0.1182 |  0.0263 $\pm$ 0.0050
SGLD |  0.7033 $\pm$ 0.0648 |  0.5235 $\pm$ 0.0273 |  0.3825 $\pm$ 0.0019 |  0.0163 $\pm$ 0.0019
Ensembles |  0.6455 $\pm$ 0.0518 |  0.4735 $\pm$ 0.0310 |  0.0791 $\pm$ 0.0905 |  0.0405 $\pm$ 0.0031
ChemBERTa | Deterministic |  0.7760 $\pm$ 0.0921 |  0.5920 $\pm$ 0.0465 |  0.3768 $\pm$ 0.1070 |  0.0464 $\pm$ 0.0042
MC Dropout |  0.6919 $\pm$ 0.0947 |  0.5192 $\pm$ 0.0619 |  0.1908 $\pm$ 0.1299 |  0.0448 $\pm$ 0.0043
SWAG |  0.7704 $\pm$ 0.0784 |  0.5921 $\pm$ 0.0350 |  0.4094 $\pm$ 0.1176 |  0.0475 $\pm$ 0.0046
BBP |  0.7648 $\pm$ 0.0328 |  0.5923 $\pm$ 0.0176 |  0.2208 $\pm$ 0.0289 |  0.0326 $\pm$ 0.0022
SGLD |  0.7282 $\pm$ 0.0764 |  0.5539 $\pm$ 0.0290 |  0.2455 $\pm$ 0.0835 |  0.0456 $\pm$ 0.0030
Ensembles |  0.7318 $\pm$ 0.0778 |  0.5641 $\pm$ 0.0416 |  0.3073 $\pm$ 0.1107 |  0.0470 $\pm$ 0.0032
GROVER | Deterministic |  0.6137 $\pm$ 0.0392 |  0.4730 $\pm$ 0.0173 |  0.0299 $\pm$ 0.0935 |  0.0377 $\pm$ 0.0063
MC Dropout |  0.5997 $\pm$ 0.0256 |  0.4621 $\pm$ 0.0110 |  0.0124 $\pm$ 0.0890 |  0.0376 $\pm$ 0.0065
SWAG |  0.5895 $\pm$ 0.0251 |  0.4476 $\pm$ 0.0102 |  -0.0342 $\pm$ 0.0624 |  0.0419 $\pm$ 0.0035
BBP |  0.6449 $\pm$ 0.0517 |  0.4892 $\pm$ 0.0286 |  0.3780 $\pm$ 0.0474 | 0.0152 $\pm$ 0.0018
SGLD |  0.5888 $\pm$ 0.0379 |  0.4542 $\pm$ 0.0252 |  0.0748 $\pm$ 0.0683 |  0.0299 $\pm$ 0.0010
Ensembles |  0.5847 $\pm$ 0.0326 |  0.4526 $\pm$ 0.0210 |  -0.0377 $\pm$ 0.0725 |  0.0384 $\pm$ 0.0031
Uni-Mol | Deterministic |  0.6253 $\pm$ 0.0500 |  0.4691 $\pm$ 0.0291 |  0.0024 $\pm$ 0.0693 |  0.0388 $\pm$ 0.0008
MC Dropout |  0.6224 $\pm$ 0.0501 |  0.4640 $\pm$ 0.0299 |  0.0058 $\pm$ 0.0647 |  0.0375 $\pm$ 0.0007
SWAG | 0.5642 $\pm$ 0.0564 | 0.4030 $\pm$ 0.0342 | -0.0603 $\pm$ 0.1092 |  0.0361 $\pm$ 0.0028
BBP |  0.6762 $\pm$ 0.0766 |  0.4884 $\pm$ 0.0544 |  0.2844 $\pm$ 0.0276 |  0.0235 $\pm$ 0.0024
SGLD |  0.6063 $\pm$ 0.0281 |  0.4388 $\pm$ 0.0098 |  0.1947 $\pm$ 0.2311 |  0.0270 $\pm$ 0.0096
Ensembles |  0.5930 $\pm$ 0.0374 |  0.4311 $\pm$ 0.0129 |  -0.0273 $\pm$ 0.0238 |  0.0375 $\pm$ 0.0015
Table 29: Test results on the randomly split FreeSolv dataset. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
DNN-rdkit | Deterministic |  1.1867 $\pm$ 0.0369 |  0.8582 $\pm$ 0.0521 |  0.7262 $\pm$ 0.0311 |  0.0252 $\pm$ 0.0038
MC Dropout |  1.1893 $\pm$ 0.0716 |  0.8576 $\pm$ 0.0283 |  0.7085 $\pm$ 0.0277 |  0.0263 $\pm$ 0.0038
SWAG |  1.1183 $\pm$ 0.0960 |  0.7817 $\pm$ 0.0602 |  0.6026 $\pm$ 0.0144 |  0.0305 $\pm$ 0.0047
BBP |  1.8405 $\pm$ 0.2306 |  1.2685 $\pm$ 0.0533 |  1.2661 $\pm$ 0.0458 |  0.0060 $\pm$ 0.0013
SGLD |  1.5395 $\pm$ 0.1830 |  1.0855 $\pm$ 0.0925 |  1.2214 $\pm$ 0.0395 | 0.0045 $\pm$ 0.0020
Ensembles |  1.1690 $\pm$ 0.0630 |  0.8500 $\pm$ 0.0239 |  0.7444 $\pm$ 0.0502 |  0.0240 $\pm$ 0.0042
ChemBERTa | Deterministic |  1.4125 $\pm$ 0.1890 |  1.0323 $\pm$ 0.1206 |  0.8274 $\pm$ 0.1999 |  0.0387 $\pm$ 0.0059
MC Dropout |  1.3675 $\pm$ 0.2456 |  1.0278 $\pm$ 0.1732 |  0.6970 $\pm$ 0.1730 |  0.0357 $\pm$ 0.0067
SWAG |  1.3789 $\pm$ 0.2128 |  1.0024 $\pm$ 0.1570 |  0.8477 $\pm$ 0.2851 |  0.0408 $\pm$ 0.0041
BBP |  1.4355 $\pm$ 0.2128 |  1.0643 $\pm$ 0.1312 |  0.8143 $\pm$ 0.0637 |  0.0270 $\pm$ 0.0032
SGLD |  1.3864 $\pm$ 0.2747 |  0.9852 $\pm$ 0.1613 |  0.6654 $\pm$ 0.1475 |  0.0361 $\pm$ 0.0023
Ensembles |  1.2276 $\pm$ 0.2395 |  0.8745 $\pm$ 0.1727 |  0.5995 $\pm$ 0.1546 |  0.0399 $\pm$ 0.0013
GROVER | Deterministic |  1.3823 $\pm$ 0.1108 |  1.0223 $\pm$ 0.0942 |  0.9105 $\pm$ 0.0684 |  0.0188 $\pm$ 0.0018
MC Dropout |  1.3570 $\pm$ 0.1082 |  1.0093 $\pm$ 0.0940 |  0.9247 $\pm$ 0.0680 |  0.0177 $\pm$ 0.0017
SWAG |  1.3181 $\pm$ 0.0898 |  0.9858 $\pm$ 0.0727 |  0.8386 $\pm$ 0.0352 |  0.0218 $\pm$ 0.0042
BBP |  1.4832 $\pm$ 0.1322 |  1.1451 $\pm$ 0.0893 |  1.1459 $\pm$ 0.0166 |  0.0060 $\pm$ 0.0019
SGLD |  1.4965 $\pm$ 0.2643 |  1.1682 $\pm$ 0.2289 |  1.1154 $\pm$ 0.1317 |  0.0072 $\pm$ 0.0016
Ensembles |  1.2826 $\pm$ 0.1700 |  0.9424 $\pm$ 0.1316 |  0.9476 $\pm$ 0.0558 |  0.0140 $\pm$ 0.0020
Uni-Mol | Deterministic |  1.0429 $\pm$ 0.0170 |  0.7345 $\pm$ 0.0308 | 0.5741 $\pm$ 0.0637 |  0.0322 $\pm$ 0.0054
MC Dropout |  1.0381 $\pm$ 0.0249 |  0.7280 $\pm$ 0.0394 |  0.5872 $\pm$ 0.0636 |  0.0310 $\pm$ 0.0055
SWAG | 0.9959 $\pm$ 0.0902 | 0.6842 $\pm$ 0.1248 |  0.6050 $\pm$ 0.0705 |  0.0271 $\pm$ 0.0051
BBP |  1.1541 $\pm$ 0.1779 |  0.8681 $\pm$ 0.1550 |  0.9795 $\pm$ 0.0621 |  0.0154 $\pm$ 0.0027
SGLD |  1.0486 $\pm$ 0.1898 |  0.7543 $\pm$ 0.1538 |  1.0065 $\pm$ 0.1283 |  0.0126 $\pm$ 0.0016
Ensembles |  1.0009 $\pm$ 0.0416 |  0.7201 $\pm$ 0.0378 |  0.6766 $\pm$ 0.0365 |  0.0270 $\pm$ 0.0045
Table 30: Test results on the randomly split Lipophilicity dataset. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
DNN-rdkit | Deterministic |  0.6790 $\pm$ 0.0415 |  0.5007 $\pm$ 0.0253 |  0.6158 $\pm$ 0.1691 |  0.0365 $\pm$ 0.0049
MC Dropout |  0.6732 $\pm$ 0.0398 |  0.4971 $\pm$ 0.0286 |  0.5558 $\pm$ 0.1208 |  0.0366 $\pm$ 0.0048
SWAG |  0.6699 $\pm$ 0.0441 |  0.4926 $\pm$ 0.0277 |  0.6208 $\pm$ 0.1477 |  0.0370 $\pm$ 0.0052
BBP |  0.6833 $\pm$ 0.0444 |  0.5092 $\pm$ 0.0222 |  0.2681 $\pm$ 0.2026 |  0.0281 $\pm$ 0.0040
SGLD |  0.6819 $\pm$ 0.0427 |  0.5090 $\pm$ 0.0222 |  0.1218 $\pm$ 0.0554 | 0.0134 $\pm$ 0.0010
Ensembles |  0.6399 $\pm$ 0.0402 |  0.4690 $\pm$ 0.0199 |  0.3242 $\pm$ 0.1517 |  0.0349 $\pm$ 0.0004
ChemBERTa | Deterministic |  0.7033 $\pm$ 0.0521 |  0.5282 $\pm$ 0.0234 |  1.4475 $\pm$ 1.1982 |  0.0418 $\pm$ 0.0064
MC Dropout |  0.6583 $\pm$ 0.0330 |  0.4882 $\pm$ 0.0095 |  0.8234 $\pm$ 0.7527 |  0.0380 $\pm$ 0.0056
SWAG |  0.7114 $\pm$ 0.0502 |  0.5363 $\pm$ 0.0196 |  1.7413 $\pm$ 1.2705 |  0.0441 $\pm$ 0.0059
BBP |  0.7103 $\pm$ 0.0508 |  0.5377 $\pm$ 0.0212 |  0.5764 $\pm$ 0.3161 |  0.0368 $\pm$ 0.0043
SGLD |  0.6988 $\pm$ 0.0540 |  0.5271 $\pm$ 0.0249 |  0.7978 $\pm$ 0.2380 |  0.0414 $\pm$ 0.0016
Ensembles |  0.6763 $\pm$ 0.0534 |  0.5044 $\pm$ 0.0232 |  1.4618 $\pm$ 0.8428 |  0.0451 $\pm$ 0.0033
GROVER | Deterministic |  0.5717 $\pm$ 0.0320 |  0.3957 $\pm$ 0.0144 |  0.6373 $\pm$ 0.0281 |  0.0446 $\pm$ 0.0016
MC Dropout |  0.5633 $\pm$ 0.0349 |  0.3900 $\pm$ 0.0163 |  0.5493 $\pm$ 0.0500 |  0.0441 $\pm$ 0.0018
SWAG |  0.5599 $\pm$ 0.0340 |  0.3851 $\pm$ 0.0117 |  0.5192 $\pm$ 0.1369 |  0.0440 $\pm$ 0.0002
BBP |  0.5638 $\pm$ 0.0314 |  0.4021 $\pm$ 0.0088 |  -0.0783 $\pm$ 0.0532 |  0.0232 $\pm$ 0.0010
SGLD |  0.5597 $\pm$ 0.0409 |  0.3958 $\pm$ 0.0171 |  -0.0915 $\pm$ 0.0856 |  0.0271 $\pm$ 0.0006
Ensembles |  0.5539 $\pm$ 0.0314 |  0.3833 $\pm$ 0.0134 |  0.4942 $\pm$ 0.1247 |  0.0447 $\pm$ 0.0010
Uni-Mol | Deterministic |  0.5387 $\pm$ 0.0439 |  0.3824 $\pm$ 0.0203 |  -0.0690 $\pm$ 0.1128 |  0.0362 $\pm$ 0.0024
MC Dropout |  0.5367 $\pm$ 0.0437 |  0.3812 $\pm$ 0.0204 |  -0.0859 $\pm$ 0.1088 |  0.0355 $\pm$ 0.0024
SWAG | 0.5077 $\pm$ 0.0446 | 0.3598 $\pm$ 0.0178 |  -0.1168 $\pm$ 0.1292 |  0.0369 $\pm$ 0.0018
BBP |  0.5632 $\pm$ 0.0509 |  0.4139 $\pm$ 0.0207 |  -0.0682 $\pm$ 0.0724 |  0.0248 $\pm$ 0.0039
SGLD |  0.5238 $\pm$ 0.0357 |  0.3718 $\pm$ 0.0160 | -0.1525 $\pm$ 0.0618 |  0.0265 $\pm$ 0.0011
Ensembles |  0.5255 $\pm$ 0.0434 |  0.3706 $\pm$ 0.0187 |  -0.1311 $\pm$ 0.1122 |  0.0347 $\pm$ 0.0010
Table 31: Test results on the randomly split QM7 dataset. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
DNN-rdkit | Deterministic |  107.9546 $\pm$ 8.1501 |  57.5611 $\pm$ 4.8507 |  5.5231 $\pm$ 0.0677 |  0.0449 $\pm$ 0.0026
MC Dropout |  106.5277 $\pm$ 8.4035 |  56.8402 $\pm$ 4.4571 |  4.9801 $\pm$ 0.0805 |  0.0434 $\pm$ 0.0025
SWAG |  105.3930 $\pm$ 8.3626 |  56.1532 $\pm$ 3.9650 |  4.9387 $\pm$ 0.1316 |  0.0429 $\pm$ 0.0018
BBP |  111.6410 $\pm$ 8.9230 |  65.9097 $\pm$ 3.7377 |  5.1715 $\pm$ 0.2092 |  0.0388 $\pm$ 0.0020
SGLD |  111.1149 $\pm$ 7.8189 |  75.5816 $\pm$ 4.4419 |  5.2277 $\pm$ 0.0341 | 0.0127 $\pm$ 0.0004
Ensembles |  104.0890 $\pm$ 9.9660 |  55.6554 $\pm$ 4.1624 |  5.0123 $\pm$ 0.1890 |  0.0438 $\pm$ 0.0020
ChemBERTa | Deterministic |  97.4000 $\pm$ 8.8093 |  47.3906 $\pm$ 3.6007 |  4.5928 $\pm$ 0.1364 |  0.0402 $\pm$ 0.0021
MC Dropout |  97.1763 $\pm$ 9.0430 |  43.4817 $\pm$ 4.8435 |  4.3224 $\pm$ 0.3558 |  0.0458 $\pm$ 0.0017
SWAG |  97.5653 $\pm$ 8.4876 |  49.3942 $\pm$ 4.0995 |  4.7928 $\pm$ 0.1495 |  0.0397 $\pm$ 0.0019
BBP |  100.8003 $\pm$ 8.9576 |  53.0419 $\pm$ 4.4522 |  4.6938 $\pm$ 0.2195 |  0.0347 $\pm$ 0.0027
SGLD |  99.6487 $\pm$ 7.9623 |  54.8976 $\pm$ 2.4214 |  4.6314 $\pm$ 0.1117 |  0.0308 $\pm$ 0.0015
Ensembles |  97.5500 $\pm$ 8.6923 |  47.2016 $\pm$ 3.7146 |  4.5436 $\pm$ 0.2688 |  0.0392 $\pm$ 0.0020
GROVER | Deterministic |  98.7533 $\pm$ 9.8780 |  42.7787 $\pm$ 4.8753 |  4.7424 $\pm$ 0.3523 |  0.0512 $\pm$ 0.0003
MC Dropout |  96.6699 $\pm$ 9.6047 |  42.6411 $\pm$ 4.6449 |  4.7324 $\pm$ 0.3130 |  0.0511 $\pm$ 0.0003
SWAG |  97.1293 $\pm$ 10.0641 |  43.3197 $\pm$ 5.1696 |  4.6304 $\pm$ 0.2678 |  0.0495 $\pm$ 0.0008
BBP |  99.0196 $\pm$ 8.1011 |  56.8246 $\pm$ 3.9466 |  4.9841 $\pm$ 0.0630 |  0.0223 $\pm$ 0.0003
SGLD |  101.0303 $\pm$ 6.6971 |  57.6752 $\pm$ 3.7755 |  4.9718 $\pm$ 0.0556 |  0.0237 $\pm$ 0.0009
Ensembles |  98.0631 $\pm$ 9.7788 |  42.8305 $\pm$ 4.6214 |  4.6874 $\pm$ 0.3069 |  0.0498 $\pm$ 0.0006
Uni-Mol | Deterministic |  71.4475 $\pm$ 2.3163 |  36.7968 $\pm$ 2.3826 |  4.1470 $\pm$ 0.1635 |  0.0411 $\pm$ 0.0032
MC Dropout |  70.8934 $\pm$ 2.2622 |  35.5291 $\pm$ 2.4115 |  4.1425 $\pm$ 0.1709 |  0.0406 $\pm$ 0.0032
SWAG | 68.6536 $\pm$ 1.3320 | 29.1093 $\pm$ 2.1596 | 3.9460 $\pm$ 0.0134 |  0.0433 $\pm$ 0.0011
BBP |  73.0523 $\pm$ 1.1094 |  39.2194 $\pm$ 1.8792 |  4.5952 $\pm$ 0.0996 |  0.0297 $\pm$ 0.0042
SGLD |  71.2770 $\pm$ 1.6947 |  32.4820 $\pm$ 3.3687 |  4.4407 $\pm$ 0.0538 |  0.0321 $\pm$ 0.0017
Ensembles |  69.6610 $\pm$ 0.9264 |  33.8478 $\pm$ 0.0577 |  4.0436 $\pm$ 0.0870 |  0.0429 $\pm$ 0.0020
Table 32: Test results with frozen backbone model weights on BACE. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
ChemBERTa | Deterministic |  0.6289 $\pm$ 0.0090 |  0.2096 $\pm$ 0.0037 |  0.7570 $\pm$ 0.0073 |  0.2751 $\pm$ 0.0028
Temperature |  0.6289 $\pm$ 0.0090 |  0.1554 $\pm$ 0.0103 |  0.7134 $\pm$ 0.0032 |  0.2587 $\pm$ 0.0014
Focal Loss |  0.6474 $\pm$ 0.0051 |  0.1964 $\pm$ 0.0035 |  0.7450 $\pm$ 0.0023 |  0.2741 $\pm$ 0.0011
MC Dropout |  0.6515 $\pm$ 0.0164 |  0.1931 $\pm$ 0.0045 |  0.7393 $\pm$ 0.0029 |  0.2693 $\pm$ 0.0013
SWAG |  0.6305 $\pm$ 0.0081 |  0.2045 $\pm$ 0.0032 |  0.7559 $\pm$ 0.0077 |  0.2746 $\pm$ 0.0030
BBP |  0.5454 $\pm$ 0.0619 |  0.0998 $\pm$ 0.0235 |  0.7080 $\pm$ 0.0087 |  0.2573 $\pm$ 0.0042
SGLD |  0.5217 $\pm$ 0.0512 | 0.0531 $\pm$ 0.0114 |  0.6941 $\pm$ 0.0022 |  0.2505 $\pm$ 0.0011
Ensembles |  0.6308 $\pm$ - |  0.1415 $\pm$ - |  0.7081 $\pm$ - |  0.2569 $\pm$ -
GROVER | Deterministic |  0.7633 $\pm$ 0.0116 |  0.2083 $\pm$ 0.0100 |  0.6934 $\pm$ 0.0070 |  0.2514 $\pm$ 0.0030
Temperature |  0.7635 $\pm$ 0.0117 |  0.1915 $\pm$ 0.0109 |  0.6826 $\pm$ 0.0067 |  0.2461 $\pm$ 0.0029
Focal Loss |  0.7221 $\pm$ 0.0343 |  0.2125 $\pm$ 0.0066 |  0.7358 $\pm$ 0.0048 |  0.2710 $\pm$ 0.0020
MC Dropout |  0.7672 $\pm$ 0.0081 |  0.2221 $\pm$ 0.0111 |  0.7037 $\pm$ 0.0050 |  0.2561 $\pm$ 0.0020
SWAG |  0.7691 $\pm$ 0.0096 |  0.2232 $\pm$ 0.0131 |  0.7027 $\pm$ 0.0036 |  0.2559 $\pm$ 0.0013
BBP |  0.7425 $\pm$ 0.0089 |  0.1946 $\pm$ 0.0135 |  0.7046 $\pm$ 0.0038 |  0.2565 $\pm$ 0.0015
SGLD |  0.5485 $\pm$ 0.0612 |  0.0944 $\pm$ 0.0227 |  0.7007 $\pm$ 0.0135 |  0.2536 $\pm$ 0.0067
Ensembles | 0.7826 $\pm$ - |  0.2278 $\pm$ - |  0.6807 $\pm$ - |  0.2462 $\pm$ -
Uni-Mol | Deterministic |  0.3713 $\pm$ 0.1533 |  0.1999 $\pm$ 0.0357 |  0.7544 $\pm$ 0.0216 |  0.2795 $\pm$ 0.0101
Temperature |  0.3712 $\pm$ 0.1534 |  0.1811 $\pm$ 0.0340 |  0.7317 $\pm$ 0.0149 |  0.2689 $\pm$ 0.0072
Focal Loss |  0.3512 $\pm$ 0.1450 |  0.2720 $\pm$ 0.0414 |  0.7977 $\pm$ 0.0301 |  0.2985 $\pm$ 0.0131
MC Dropout |  0.3698 $\pm$ 0.1569 |  0.2052 $\pm$ 0.0513 |  0.7512 $\pm$ 0.0203 |  0.2780 $\pm$ 0.0095
SWAG |  0.4297 $\pm$ 0.1408 |  0.1513 $\pm$ 0.0237 |  0.7330 $\pm$ 0.0089 |  0.2695 $\pm$ 0.0042
BBP |  0.5958 $\pm$ 0.0671 |  0.1614 $\pm$ 0.0191 |  0.7361 $\pm$ 0.0096 |  0.2709 $\pm$ 0.0045
SGLD |  0.7189 $\pm$ 0.0291 |  0.1414 $\pm$ 0.0218 | 0.6737 $\pm$ 0.0091 | 0.2405 $\pm$ 0.0045
Ensembles |  0.6799 $\pm$ - |  0.1543 $\pm$ - |  0.7370 $\pm$ - |  0.2714 $\pm$ -
Table 33: Test results with frozen backbone model weights on BBBP. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
ChemBERTa | Deterministic | 0.6651 $\pm$ 0.0075 |  0.1025 $\pm$ 0.0103 | 0.6534 $\pm$ 0.0072 | 0.2308 $\pm$ 0.0033
Temperature | 0.6651 $\pm$ 0.0075 |  0.1681 $\pm$ 0.0189 |  0.7366 $\pm$ 0.0315 |  0.2518 $\pm$ 0.0073
Focal Loss |  0.6600 $\pm$ 0.0082 |  0.1342 $\pm$ 0.0038 |  0.6782 $\pm$ 0.0017 |  0.2430 $\pm$ 0.0009
MC Dropout |  0.6592 $\pm$ 0.0084 |  0.1358 $\pm$ 0.0159 |  0.6813 $\pm$ 0.0058 |  0.2428 $\pm$ 0.0030
SWAG |  0.6650 $\pm$ 0.0079 |  0.1068 $\pm$ 0.0112 |  0.6537 $\pm$ 0.0074 |  0.2310 $\pm$ 0.0033
BBP |  0.6280 $\pm$ 0.0074 |  0.1684 $\pm$ 0.0089 |  0.7250 $\pm$ 0.0082 |  0.2645 $\pm$ 0.0035
SGLD |  0.4822 $\pm$ 0.0247 | 0.0346 $\pm$ 0.0091 |  0.6945 $\pm$ 0.0020 |  0.2507 $\pm$ 0.0010
Ensembles |  0.6545 $\pm$ - |  0.1059 $\pm$ - |  0.6539 $\pm$ - |  0.2320 $\pm$ -
GROVER | Deterministic |  0.6088 $\pm$ 0.0124 |  0.3048 $\pm$ 0.0071 |  0.9424 $\pm$ 0.0027 |  0.3303 $\pm$ 0.0024
Temperature |  0.6083 $\pm$ 0.0122 |  0.2738 $\pm$ 0.0073 |  0.8674 $\pm$ 0.0022 |  0.3119 $\pm$ 0.0019
Focal Loss |  0.5979 $\pm$ 0.0167 |  0.0867 $\pm$ 0.0234 |  0.6872 $\pm$ 0.0071 |  0.2469 $\pm$ 0.0034
MC Dropout |  0.6089 $\pm$ 0.0135 |  0.3073 $\pm$ 0.0114 |  0.9372 $\pm$ 0.0028 |  0.3297 $\pm$ 0.0021
SWAG |  0.6179 $\pm$ 0.0112 |  0.3087 $\pm$ 0.0038 |  0.9424 $\pm$ 0.0042 |  0.3300 $\pm$ 0.0022
BBP |  0.6096 $\pm$ 0.0069 |  0.3023 $\pm$ 0.0065 |  0.9211 $\pm$ 0.0038 |  0.3271 $\pm$ 0.0012
SGLD |  0.4628 $\pm$ 0.0682 |  0.0685 $\pm$ 0.0306 |  0.6979 $\pm$ 0.0108 |  0.2523 $\pm$ 0.0053
Ensembles |  0.6107 $\pm$ - |  0.2982 $\pm$ - |  0.9338 $\pm$ - |  0.3285 $\pm$ -
Uni-Mol | Deterministic |  0.5325 $\pm$ 0.0364 |  0.3534 $\pm$ 0.0074 |  1.0741 $\pm$ 0.0298 |  0.3726 $\pm$ 0.0064
Temperature |  0.5326 $\pm$ 0.0366 |  0.2867 $\pm$ 0.0074 |  0.9013 $\pm$ 0.0181 |  0.3297 $\pm$ 0.0056
Focal Loss |  0.5405 $\pm$ 0.0458 |  0.0522 $\pm$ 0.0042 |  0.6912 $\pm$ 0.0075 |  0.2490 $\pm$ 0.0037
MC Dropout |  0.5336 $\pm$ 0.0397 |  0.3449 $\pm$ 0.0077 |  1.0457 $\pm$ 0.0290 |  0.3666 $\pm$ 0.0066
SWAG |  0.5592 $\pm$ 0.0143 |  0.3008 $\pm$ 0.0042 |  0.9277 $\pm$ 0.0109 |  0.3374 $\pm$ 0.0033
BBP |  0.4626 $\pm$ 0.0239 |  0.2631 $\pm$ 0.0447 |  0.8784 $\pm$ 0.0809 |  0.3209 $\pm$ 0.0243
SGLD |  0.4590 $\pm$ 0.0456 |  0.1206 $\pm$ 0.0298 |  0.7238 $\pm$ 0.0136 |  0.2646 $\pm$ 0.0059
Ensembles |  0.5875 $\pm$ - |  0.3370 $\pm$ - |  1.0186 $\pm$ - |  0.3614 $\pm$ -
Table 34: Test results with frozen backbone model weights on ClinTox. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
ChemBERTa | Deterministic |  0.6344 $\pm$ 0.2837 |  0.2913 $\pm$ 0.1651 |  0.4713 $\pm$ 0.2530 |  0.1618 $\pm$ 0.0952
Temperature |  0.6344 $\pm$ 0.2837 |  0.2389 $\pm$ 0.1655 |  0.4157 $\pm$ 0.2533 |  0.1398 $\pm$ 0.0964
Focal Loss |  0.6369 $\pm$ 0.2860 |  0.3267 $\pm$ 0.1183 |  0.5099 $\pm$ 0.2000 |  0.1688 $\pm$ 0.0859
MC Dropout |  0.6689 $\pm$ 0.2643 |  0.2886 $\pm$ 0.1675 |  0.4706 $\pm$ 0.2491 |  0.1612 $\pm$ 0.0934
SWAG |  0.6911 $\pm$ 0.2300 |  0.2245 $\pm$ 0.1211 |  0.3728 $\pm$ 0.1831 |  0.1142 $\pm$ 0.0607
BBP |  0.7242 $\pm$ 0.0410 |  0.2659 $\pm$ 0.1039 |  0.4515 $\pm$ 0.1399 |  0.1379 $\pm$ 0.0638
SGLD |  0.5489 $\pm$ 0.1696 |  0.4339 $\pm$ 0.0032 |  0.6898 $\pm$ 0.0048 |  0.2483 $\pm$ 0.0024
Ensembles | 0.9734 $\pm$ - |  0.2885 $\pm$ - |  0.4085 $\pm$ - |  0.1139 $\pm$ -
GROVER | Deterministic |  0.6437 $\pm$ 0.0235 |  0.0304 $\pm$ 0.0051 |  0.2357 $\pm$ 0.0041 |  0.0599 $\pm$ 0.0008
Temperature |  0.6426 $\pm$ 0.0240 | 0.0179 $\pm$ 0.0032 |  0.2337 $\pm$ 0.0050 |  0.0594 $\pm$ 0.0007
Focal Loss |  0.6328 $\pm$ 0.0220 |  0.2116 $\pm$ 0.0054 |  0.3897 $\pm$ 0.0064 |  0.1109 $\pm$ 0.0027
MC Dropout |  0.6534 $\pm$ 0.0201 |  0.0222 $\pm$ 0.0041 |  0.2329 $\pm$ 0.0029 |  0.0594 $\pm$ 0.0005
SWAG |  0.6838 $\pm$ 0.0124 |  0.0268 $\pm$ 0.0064 | 0.2306 $\pm$ 0.0031 | 0.0589 $\pm$ 0.0007
BBP |  0.6019 $\pm$ 0.0346 |  0.0339 $\pm$ 0.0028 |  0.2422 $\pm$ 0.0032 |  0.0609 $\pm$ 0.0005
SGLD |  0.4601 $\pm$ 0.0878 |  0.4365 $\pm$ 0.0355 |  0.7050 $\pm$ 0.0650 |  0.2558 $\pm$ 0.0321
Ensembles |  0.7039 $\pm$ - |  0.0348 $\pm$ - |  0.2311 $\pm$ - |  0.0591 $\pm$ -
Uni-Mol | Deterministic |  0.5750 $\pm$ 0.0185 |  0.1620 $\pm$ 0.0598 |  0.3423 $\pm$ 0.0569 |  0.0916 $\pm$ 0.0209
Temperature |  0.5752 $\pm$ 0.0182 |  0.0713 $\pm$ 0.0499 |  0.2694 $\pm$ 0.0319 |  0.0678 $\pm$ 0.0086
Focal Loss |  0.5696 $\pm$ 0.0199 |  0.2605 $\pm$ 0.0328 |  0.4397 $\pm$ 0.0359 |  0.1290 $\pm$ 0.0158
MC Dropout |  0.5797 $\pm$ 0.0259 |  0.1704 $\pm$ 0.0595 |  0.3499 $\pm$ 0.0577 |  0.0944 $\pm$ 0.0216
SWAG |  0.5887 $\pm$ 0.0228 |  0.1125 $\pm$ 0.0324 |  0.2948 $\pm$ 0.0250 |  0.0741 $\pm$ 0.0075
BBP |  0.5899 $\pm$ 0.0112 |  0.2754 $\pm$ 0.0510 |  0.4608 $\pm$ 0.0683 |  0.1391 $\pm$ 0.0313
SGLD |  0.5257 $\pm$ 0.0576 |  0.4285 $\pm$ 0.0442 |  0.7081 $\pm$ 0.0984 |  0.2556 $\pm$ 0.0463
Ensembles |  0.5551 $\pm$ - |  0.1449 $\pm$ - |  0.3199 $\pm$ - |  0.0811 $\pm$ -
Table 35: Test results with frozen backbone model weights on Tox21. |  | ROC-AUC | ECE | NLL | BS
---|---|---|---|---|---
ChemBERTa | Deterministic |  0.6615 $\pm$ 0.0039 |  0.0402 $\pm$ 0.0008 |  0.2934 $\pm$ 0.0004 |  0.0826 $\pm$ 0.0002
Temperature |  0.6615 $\pm$ 0.0039 |  0.0307 $\pm$ 0.0015 |  0.2894 $\pm$ 0.0008 |  0.0817 $\pm$ 0.0001
Focal Loss |  0.6646 $\pm$ 0.0026 |  0.1183 $\pm$ 0.0025 |  0.3428 $\pm$ 0.0021 |  0.0949 $\pm$ 0.0007
MC Dropout |  0.6729 $\pm$ 0.0048 |  0.0315 $\pm$ 0.0016 |  0.2859 $\pm$ 0.0007 |  0.0807 $\pm$ 0.0002
SWAG |  0.6619 $\pm$ 0.0038 |  0.0397 $\pm$ 0.0012 |  0.2934 $\pm$ 0.0005 |  0.0826 $\pm$ 0.0002
BBP |  0.5426 $\pm$ 0.0080 |  0.1150 $\pm$ 0.0079 |  0.3629 $\pm$ 0.0059 |  0.1009 $\pm$ 0.0019
SGLD |  0.5062 $\pm$ 0.0212 |  0.3973 $\pm$ 0.0005 |  0.6903 $\pm$ 0.0008 |  0.2486 $\pm$ 0.0004
Ensembles |  0.6663 $\pm$ - |  0.0383 $\pm$ - |  0.2916 $\pm$ - |  0.0822 $\pm$ -
GROVER | Deterministic |  0.6897 $\pm$ 0.0102 |  0.0361 $\pm$ 0.0002 |  0.2838 $\pm$ 0.0011 |  0.0807 $\pm$ 0.0003
Temperature |  0.6897 $\pm$ 0.0102 | 0.0223 $\pm$ 0.0017 | 0.2789 $\pm$ 0.0011 | 0.0799 $\pm$ 0.0003
Focal Loss |  0.6632 $\pm$ 0.0139 |  0.1326 $\pm$ 0.0023 |  0.3593 $\pm$ 0.0005 |  0.0998 $\pm$ 0.0002
MC Dropout |  0.6874 $\pm$ 0.0093 |  0.0343 $\pm$ 0.0004 |  0.2842 $\pm$ 0.0009 |  0.0810 $\pm$ 0.0002
SWAG | 0.6923 $\pm$ 0.0075 |  0.0344 $\pm$ 0.0002 |  0.2830 $\pm$ 0.0008 |  0.0807 $\pm$ 0.0002
BBP |  0.6496 $\pm$ 0.0040 |  0.0364 $\pm$ 0.0005 |  0.2951 $\pm$ 0.0003 |  0.0836 $\pm$ 0.0001
SGLD |  0.5072 $\pm$ 0.0266 |  0.3898 $\pm$ 0.0018 |  0.6806 $\pm$ 0.0024 |  0.2438 $\pm$ 0.0012
Ensembles |  0.6911 $\pm$ - |  0.0365 $\pm$ - |  0.2839 $\pm$ - |  0.0807 $\pm$ -
Uni-Mol | Deterministic |  0.5702 $\pm$ 0.0065 |  0.0556 $\pm$ 0.0008 |  0.3352 $\pm$ 0.0009 |  0.0905 $\pm$ 0.0002
Temperature |  0.5702 $\pm$ 0.0066 |  0.0153 $\pm$ 0.0012 |  0.3036 $\pm$ 0.0001 |  0.0862 $\pm$ 0.0000
Focal Loss |  0.5620 $\pm$ 0.0067 |  0.0472 $\pm$ 0.0030 |  0.3183 $\pm$ 0.0020 |  0.0888 $\pm$ 0.0004
MC Dropout |  0.5696 $\pm$ 0.0056 |  0.0519 $\pm$ 0.0004 |  0.3297 $\pm$ 0.0007 |  0.0900 $\pm$ 0.0002
SWAG |  0.5936 $\pm$ 0.0056 |  0.0293 $\pm$ 0.0011 |  0.3070 $\pm$ 0.0005 |  0.0868 $\pm$ 0.0001
BBP |  0.5505 $\pm$ 0.0184 |  0.0472 $\pm$ 0.0021 |  0.3249 $\pm$ 0.0021 |  0.0896 $\pm$ 0.0002
SGLD |  0.4927 $\pm$ 0.0089 |  0.4022 $\pm$ 0.0118 |  0.7135 $\pm$ 0.0216 |  0.2599 $\pm$ 0.0104
Ensembles |  0.5866 $\pm$ - |  0.0546 $\pm$ - |  0.3335 $\pm$ - |  0.0901 $\pm$ -
Table 36: Test results with frozen backbone model weights on ESOL. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
ChemBERTa | Deterministic |  1.9543 $\pm$ 0.0830 |  1.4348 $\pm$ 0.0895 |  1.2260 $\pm$ 0.0454 |  0.0090 $\pm$ 0.0010
MC Dropout |  1.8954 $\pm$ 0.1132 |  1.4051 $\pm$ 0.1047 |  1.1497 $\pm$ 0.0815 |  0.0080 $\pm$ 0.0006
SWAG |  1.9708 $\pm$ 0.0865 |  1.4372 $\pm$ 0.0905 |  1.2561 $\pm$ 0.0495 |  0.0105 $\pm$ 0.0011
BBP |  2.2450 $\pm$ 0.0440 |  1.7152 $\pm$ 0.0346 |  1.3560 $\pm$ 0.0285 |  0.0155 $\pm$ 0.0026
SGLD |  2.3491 $\pm$ 0.0542 |  1.7899 $\pm$ 0.0472 |  1.4684 $\pm$ 0.0373 |  0.0180 $\pm$ 0.0055
Ensembles |  1.8844 $\pm$ - |  1.3699 $\pm$ - |  1.1914 $\pm$ - |  0.0082 $\pm$ -
GROVER | Deterministic |  1.8533 $\pm$ 0.1590 |  1.4171 $\pm$ 0.1281 |  1.1335 $\pm$ 0.0833 |  0.0097 $\pm$ 0.0024
MC Dropout |  1.8629 $\pm$ 0.1562 |  1.4131 $\pm$ 0.1234 |  1.1444 $\pm$ 0.0836 |  0.0110 $\pm$ 0.0031
SWAG | 1.8059 $\pm$ 0.1521 |  1.3651 $\pm$ 0.1102 | 1.0930 $\pm$ 0.0857 |  0.0120 $\pm$ 0.0029
BBP |  1.9032 $\pm$ 0.1318 |  1.4502 $\pm$ 0.1186 |  1.3243 $\pm$ 0.0225 |  0.0088 $\pm$ 0.0014
SGLD |  2.4699 $\pm$ 0.1371 |  1.8858 $\pm$ 0.1161 |  1.3697 $\pm$ 0.0510 |  0.0285 $\pm$ 0.0050
Ensembles |  1.8185 $\pm$ - | 1.3161 $\pm$ - |  1.1063 $\pm$ - |  0.0098 $\pm$ -
Uni-Mol | Deterministic |  2.2126 $\pm$ 0.0258 |  1.7134 $\pm$ 0.0482 |  1.3168 $\pm$ 0.0112 |  0.0078 $\pm$ 0.0054
MC Dropout |  2.2155 $\pm$ 0.0223 |  1.7118 $\pm$ 0.0426 |  1.3169 $\pm$ 0.0106 |  0.0078 $\pm$ 0.0053
SWAG |  2.1809 $\pm$ 0.0258 |  1.6573 $\pm$ 0.0294 |  1.2883 $\pm$ 0.0210 |  0.0085 $\pm$ 0.0035
BBP |  2.1964 $\pm$ 0.1226 |  1.7223 $\pm$ 0.0996 |  1.3008 $\pm$ 0.0625 | 0.0053 $\pm$ 0.0042
SGLD |  2.3596 $\pm$ 0.0599 |  1.8831 $\pm$ 0.0734 |  1.4158 $\pm$ 0.0808 |  0.0139 $\pm$ 0.0037
Ensembles |  2.1476 $\pm$ - |  1.6524 $\pm$ - |  1.2669 $\pm$ - | 0.0053 $\pm$ -
Table 37: Test results with frozen backbone model weights on FreeSolv. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
ChemBERTa | Deterministic |  3.7564 $\pm$ 0.0520 |  2.7933 $\pm$ 0.0770 |  2.0303 $\pm$ 0.0828 |  0.0378 $\pm$ 0.0078
MC Dropout |  3.7861 $\pm$ 0.0542 |  2.8493 $\pm$ 0.0801 |  2.0393 $\pm$ 0.0773 |  0.0429 $\pm$ 0.0065
SWAG |  3.7629 $\pm$ 0.0297 |  2.8017 $\pm$ 0.0563 |  2.0520 $\pm$ 0.0679 |  0.0395 $\pm$ 0.0062
BBP |  4.5822 $\pm$ 0.0444 |  3.4921 $\pm$ 0.0636 |  2.3843 $\pm$ 0.0507 |  0.0669 $\pm$ 0.0037
SGLD |  4.3996 $\pm$ 0.1248 |  3.3132 $\pm$ 0.1317 |  2.3018 $\pm$ 0.1113 |  0.0546 $\pm$ 0.0088
Ensembles | 3.7094 $\pm$ - | 2.7309 $\pm$ - |  2.0012 $\pm$ - |  0.0370 $\pm$ -
GROVER | Deterministic |  4.2407 $\pm$ 0.3174 |  3.1254 $\pm$ 0.4023 |  2.0701 $\pm$ 0.1023 |  0.0233 $\pm$ 0.0141
MC Dropout |  4.2045 $\pm$ 0.3199 |  3.1098 $\pm$ 0.3737 |  2.0422 $\pm$ 0.0929 | 0.0199 $\pm$ 0.0130
SWAG |  4.1854 $\pm$ 0.2899 |  3.1039 $\pm$ 0.3888 |  2.0604 $\pm$ 0.1230 |  0.0251 $\pm$ 0.0127
BBP |  4.4898 $\pm$ 0.2067 |  3.3533 $\pm$ 0.2186 |  2.0452 $\pm$ 0.0530 |  0.0378 $\pm$ 0.0136
SGLD |  4.6731 $\pm$ 0.5142 |  3.5858 $\pm$ 0.5165 |  2.1826 $\pm$ 0.0714 |  0.0529 $\pm$ 0.0178
Ensembles |  4.0551 $\pm$ - |  3.0166 $\pm$ - |  1.9352 $\pm$ - |  0.0245 $\pm$ -
Uni-Mol | Deterministic |  3.9034 $\pm$ 0.4092 |  2.8791 $\pm$ 0.3542 | 1.8779 $\pm$ 0.0965 |  0.0207 $\pm$ 0.0150
MC Dropout |  3.9103 $\pm$ 0.4142 |  2.8836 $\pm$ 0.3612 |  1.8813 $\pm$ 0.0990 |  0.0209 $\pm$ 0.0151
SWAG |  4.1746 $\pm$ 0.2749 |  3.1272 $\pm$ 0.2718 |  1.9582 $\pm$ 0.0786 |  0.0360 $\pm$ 0.0128
BBP |  4.0001 $\pm$ 0.2701 |  3.0110 $\pm$ 0.2507 |  1.9486 $\pm$ 0.1219 |  0.0276 $\pm$ 0.0196
SGLD |  4.1575 $\pm$ 0.0222 |  3.1164 $\pm$ 0.0627 |  2.0574 $\pm$ 0.1045 |  0.0321 $\pm$ 0.0066
Ensembles |  4.0416 $\pm$ - |  2.9823 $\pm$ - |  1.9718 $\pm$ - |  0.0287 $\pm$ -
Table 38: Test results with frozen backbone model weights on Lipophilicity. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
ChemBERTa | Deterministic |  0.9958 $\pm$ 0.0033 |  0.8083 $\pm$ 0.0047 |  0.4837 $\pm$ 0.0039 | 0.0016 $\pm$ 0.0001
MC Dropout |  0.9985 $\pm$ 0.0064 |  0.8185 $\pm$ 0.0061 |  0.4885 $\pm$ 0.0063 |  0.0041 $\pm$ 0.0001
SWAG |  0.9948 $\pm$ 0.0029 |  0.8070 $\pm$ 0.0043 |  0.4823 $\pm$ 0.0031 | 0.0016 $\pm$ 0.0001
BBP |  1.0477 $\pm$ 0.0017 |  0.8578 $\pm$ 0.0018 |  0.5537 $\pm$ 0.0033 |  0.0025 $\pm$ 0.0001
SGLD |  1.1189 $\pm$ 0.0033 |  0.9266 $\pm$ 0.0036 |  0.6225 $\pm$ 0.0038 |  0.0074 $\pm$ 0.0012
Ensembles |  0.9837 $\pm$ - |  0.7981 $\pm$ - |  0.4729 $\pm$ - | 0.0016 $\pm$ -
GROVER | Deterministic |  0.9297 $\pm$ 0.0182 |  0.7568 $\pm$ 0.0165 |  0.4238 $\pm$ 0.0178 |  0.0019 $\pm$ 0.0002
MC Dropout |  0.9366 $\pm$ 0.0146 |  0.7683 $\pm$ 0.0113 |  0.4348 $\pm$ 0.0148 |  0.0033 $\pm$ 0.0004
SWAG |  0.9290 $\pm$ 0.0148 |  0.7621 $\pm$ 0.0104 |  0.4223 $\pm$ 0.0144 |  0.0034 $\pm$ 0.0003
BBP |  0.9753 $\pm$ 0.0229 |  0.7950 $\pm$ 0.0192 |  0.6023 $\pm$ 0.0346 |  0.0025 $\pm$ 0.0007
SGLD |  1.1203 $\pm$ 0.0238 |  0.9301 $\pm$ 0.0243 |  0.6134 $\pm$ 0.0206 |  0.0052 $\pm$ 0.0031
Ensembles | 0.9160 $\pm$ - | 0.7455 $\pm$ - | 0.4134 $\pm$ - |  0.0019 $\pm$ -
Uni-Mol | Deterministic |  1.0917 $\pm$ 0.0210 |  0.9069 $\pm$ 0.0203 |  0.6446 $\pm$ 0.0189 |  0.0055 $\pm$ 0.0014
MC Dropout |  1.0929 $\pm$ 0.0216 |  0.9080 $\pm$ 0.0208 |  0.6442 $\pm$ 0.0192 |  0.0054 $\pm$ 0.0014
SWAG |  1.0822 $\pm$ 0.0114 |  0.8986 $\pm$ 0.0115 |  0.6144 $\pm$ 0.0121 |  0.0045 $\pm$ 0.0004
BBP |  1.1003 $\pm$ 0.0198 |  0.9112 $\pm$ 0.0160 |  0.8454 $\pm$ 0.0580 |  0.0124 $\pm$ 0.0022
SGLD |  1.1796 $\pm$ 0.0644 |  0.9567 $\pm$ 0.0190 |  0.6812 $\pm$ 0.0469 |  0.0135 $\pm$ 0.0051
Ensembles |  1.0863 $\pm$ - |  0.9019 $\pm$ - |  0.6421 $\pm$ - |  0.0052 $\pm$ -
Table 39: Test results with frozen backbone model weights on QM7. |  | RMSE | MAE | NLL | CE
---|---|---|---|---|---
ChemBERTa | Deterministic |  137.8736 $\pm$ 2.2380 |  98.5749 $\pm$ 2.9618 |  5.4092 $\pm$ 0.0182 |  0.0049 $\pm$ 0.0001
MC Dropout |  133.8345 $\pm$ 1.8327 |  97.7798 $\pm$ 2.3986 |  5.4049 $\pm$ 0.0117 |  0.0028 $\pm$ 0.0001
SWAG |  138.2471 $\pm$ 2.2020 |  99.0168 $\pm$ 2.7806 |  5.4107 $\pm$ 0.0177 |  0.0048 $\pm$ 0.0001
BBP |  141.0033 $\pm$ 1.2990 |  104.8384 $\pm$ 1.5366 |  5.4635 $\pm$ 0.0100 | 0.0026 $\pm$ 0.0005
SGLD |  202.5163 $\pm$ 1.4235 |  165.5756 $\pm$ 1.1589 |  5.8131 $\pm$ 0.0087 |  0.0208 $\pm$ 0.0015
Ensembles |  136.6747 $\pm$ - |  97.2405 $\pm$ - |  5.4066 $\pm$ - |  0.0045 $\pm$ -
GROVER | Deterministic |  134.4406 $\pm$ 3.7901 |  95.5997 $\pm$ 4.3313 |  5.4077 $\pm$ 0.0717 |  0.0058 $\pm$ 0.0023
MC Dropout |  131.7638 $\pm$ 3.5591 |  93.2436 $\pm$ 4.3578 |  5.4005 $\pm$ 0.0761 |  0.0042 $\pm$ 0.0021
SWAG | 130.2656 $\pm$ 2.2246 | 92.1086 $\pm$ 3.1537 | 5.3670 $\pm$ 0.0446 |  0.0047 $\pm$ 0.0015
BBP |  137.9643 $\pm$ 6.4603 |  100.5829 $\pm$ 7.7898 |  5.8332 $\pm$ 0.1322 |  0.0055 $\pm$ 0.0030
SGLD |  191.3884 $\pm$ 3.4342 |  154.4454 $\pm$ 1.8662 |  5.8488 $\pm$ 0.0227 |  0.0158 $\pm$ 0.0033
Ensembles |  130.9773 $\pm$ - |  93.3082 $\pm$ - |  5.4027 $\pm$ - |  0.0051 $\pm$ -
Uni-Mol | Deterministic |  183.5096 $\pm$ 9.5881 |  148.3822 $\pm$ 9.8276 |  5.7908 $\pm$ 0.0353 |  0.0180 $\pm$ 0.0051
MC Dropout |  183.7110 $\pm$ 9.1987 |  148.5660 $\pm$ 9.3875 |  5.7908 $\pm$ 0.0336 |  0.0178 $\pm$ 0.0048
SWAG |  171.6691 $\pm$ 1.7586 |  137.1572 $\pm$ 2.1770 |  5.7147 $\pm$ 0.0078 |  0.0111 $\pm$ 0.0008
BBP |  178.0057 $\pm$ 3.9910 |  142.9159 $\pm$ 3.6271 |  6.0873 $\pm$ 0.0082 |  0.0193 $\pm$ 0.0007
SGLD |  226.7307 $\pm$ 21.2619 |  188.4545 $\pm$ 19.4483 |  6.0090 $\pm$ 0.1956 |  0.0458 $\pm$ 0.0322
Ensembles |  175.6095 $\pm$ - |  140.6629 $\pm$ - |  5.7658 $\pm$ - |  0.0129 $\pm$ -