††thanks:

# Definite Integral of $\alpha$-Fractal Functions

Md Nazimul Islam  Department of Mathematics and Statistics
Aliah University
IIA/27, New Town, Kolkata-700 160, India<EMAIL_ADDRESS>Imrul Kaish
Department of Mathematics and Statistics Aliah University IIA/27, New Town,
Kolkata-700 160, India<EMAIL_ADDRESS>
(Date: January 1, 2004)

###### Abstract.

In this article the integration of the $\alpha$-fractal interpolation function
$f^{\alpha}$ corresponding to any continuous function $f$ on a compact
interval $I$ of $\mathbb{R}$ is estimated although there is no explicit form
of $\alpha$-fractal interpolation function till now. Some results related to
the definite integral of $f^{\alpha}$ are established. Also the flipped
$\alpha$-fractal function $f_{F}^{\alpha_{F}}$ corresponding to the continuous
function $f$ is constructed and a result is proved that relates the definite
integrals of the fractal functions $f_{F}^{\alpha_{F}}$ and $f^{\alpha}$.

###### Key words and phrases:

$\alpha$-fractal function, Definite integral of $\alpha$-fractal function,
Flipped $\alpha$-fractal function.

###### 1991 Mathematics Subject Classification:

28A80, 65D05

##  1\. Introduction

There are objects in nature whose geometric structure is very irregular and
complicated. Such as surface of broken stone, the boundary when a drop of ink
falls on a paper, trees, system of blood vessels, mountain ranges, coastlines,
smoke etc. Benoit B. Mandelbrot [5] named these objects as fractals.

Barnsley [1] developed the fractal theory through introducing the concept of
fractal interpolation function (FIF) and constructed the FIF using
Hutchinson’s operator [4] on an iterated function system (IFS), whose
attractor is the graph of a continuous function interpolating a certain data
set. FIFs whose graphs are fractals have been broadly used in approximation
theory, interpolation theory, financial series, computer graphics, signal
processing etc.

Later, Navascués [7, 8], introduced the concept of $\alpha$-fractal
interpolation function $f^{\alpha}$ which is a fractal perturbation
corresponding to a continuous function $f\in\mathcal{C}(I)$ defined on a
compact interval $I$ of $\mathbb{R}$. It is discussed in details the theory
and applications of $\alpha$-fractal interpolation function in the literature
[8, 11, 12]. The function $f^{\alpha}$ is continuous but in general nowhere
differentiable.

Barnsley _et. al._ [3] introduced the calculus of FIF and showed that integral
of a FIF is also a FIF. In [14], the fractional order integral of FIF is
discussed and it is shown that FIF can be integrated on any closed interval
$[a,b]\subset[0,\infty)$. In the same paper, it is also proved that the
fractional order integral of FIF is still a FIF on the interval $[0,b](b>0)$.

In [13], a method for finding the numerical integration of affine fractal
function is mentioned and an upper bound of error in computation of the
integration of an affine fractal function with the integration of classical
one is placed.

In the present paper, in Section 3, we estimate the definite integral of the
$\alpha$-fractal function $f^{\alpha}$ corresponding to a continuous function
$f$ on a compact interval $I$. We obtain some results related to the definite
integral of the function $f^{\alpha}$. In Section 4, we constructed the
flipped $\alpha$-fractal function $f_{F}^{\alpha_{F}}$ corresponding to the
continuous function $f$ on the compact interval $I_{F}$ which is the flipped
version of the interval $I$ about the $y$-axis. We proved that the definite
integral of $f_{F}^{\alpha_{F}}$ on $I_{F}$ is equal to the definite integral
of $f^{\alpha}$ on $I$.

##  2\. Definitions and Notations

###  2.1. Fractal Interpolation Functions

Let $N\geq 2$ be an integer. Consider a set of interpolation points
$D=\\{(x_{i},y_{i})\in I\times\mathbb{R}:i=0,1,...,N\\}$, where
$\Delta:x_{0}<x_{1}<...<x_{N}$ is a partition of the closed interval
$I=[x_{0},x_{N}]$. Set $I_{i}=[x_{i-1},x_{i}]$ for $i=1,2,...,N$. Let
$L_{i}:I\rightarrow I_{i},$ $i=1,2,...,N,$ be contraction homeomorphisms with

$L_{i}(x_{0})=x_{i-1},L_{i}(x_{N})=x_{i},$ $|L_{i}(x)-L_{i}(y)|\leq a|x-y|,$

for all $x,y$ in $I$ and for some $0\leq a<1.$ Let
$F_{i}:I\times\mathbb{R}\rightarrow\mathbb{R}$, $i=1,2,...,N$, be given
continuous functions satisfying the join-up conditions

$F_{i}(x_{0},y_{0})=y_{i-1},F_{i}(x_{N},y_{N})=y_{i},$
$|F_{i}(x,y)-F_{i}(x,y^{\prime})|\leq|\alpha_{i}||y-y^{\prime}|,$

for all $x$ in $I$ and for all $y,y^{\prime}$ in $\mathbb{R}$ and for some
$0\leq\left|\alpha_{i}\right|<1,i=1,2,...,N$.

Define the mappings $W_{i}:I\times\mathbb{R}\rightarrow
I_{i}\times\mathbb{R};$ $i=1,2,...,N$ by, for all $(x,y)$ $\in
I\times\mathbb{R},$

$W_{i}(x,y)=(L_{i}(x),F_{i}(x,y)).$ (2.1)

######  Theorem 2.1.

[2] The iterated function system $\left\langle
I\times\mathbb{R};W_{i}(x,y):i=1,..,N\right\rangle$ defined in (2.1) admits a
unique attractor $G$, where $G$ is the graph of a continuous function
$g:I\rightarrow R$ which obeys $g(x_{i})=y_{i}$ for $i=1,2,...,N$.

Let
$\mathcal{C}^{\ast}(I)=\\{f\in\mathcal{C}(I):f(x_{0})=y_{0},f(x_{N})=y_{N}\\}$
and $\mathcal{C}^{\ast\ast}(I)=\\{f\in\mathcal{C}(I):f(x_{i})=y_{i};$
$i=0,1,2,...,N\\}.$

The Read-Bajraktarvic (RB) operator
$T:\mathcal{C}^{\ast}(I)\rightarrow\mathcal{C}^{\ast\ast}(I)$ defined by (see
[6])

$(Tf)(x)=F_{i}(L_{i}^{-1}(x),f(L_{i}^{-1}(x)));x\in I_{i},i=1,2,...,N,$

is a contraction with contractivity factor
$\left|\alpha\right|_{\infty}=\max\\{\left|\alpha_{i}\right|:i=1,..,N\\}<1.$
Due to Banach fixed point theorem, $T$ has a unique fixed point $g$ (say)
which also interpolates the points of $D$. This function $g$ is called a
fractal interpolation function or simply a fractal function corresponding to
the IFS (2.1) and this unique function $g$ satisfies the fixed point equation

$g(x)=F_{i}(L_{i}^{-1}(x),g(L_{i}^{-1}(x)));x\in I_{i},i=1,2,...,N.$ (2.2)

The free parameters $\alpha_{i}$, $i=1,2,...,N$ are the vertical scaling
factors of the transformation $W_{i}$ and the corresponding vector
$\alpha=(\alpha_{1},\alpha_{2},...,\alpha_{N})$ is called the scale vector of
the IFS (2.1).

###  2.2.  $\alpha$-Fractal Interpolation Functions

Let $f\in\mathcal{C}(I)$. Consider the IFS defined by the iterated mappings

$L_{i}(x)=a_{i}x+e_{i}$ (2.3)

and

$F_{i}(x,y)=\alpha_{i}y+f(L_{i}(x))-\alpha_{i}b(x),$ (2.4)

where $a_{i}=\frac{x_{i}-x_{i-1}}{x_{N}-x_{0}}$,
$e_{i}=\frac{x_{N}x_{i-1}-x_{0}x_{i}}{x_{N}-x_{0}}$,
$0\leq\left|\alpha_{i}\right|<1,i=1,2,...,N$ and $b\in\mathcal{C}(I)$, known
as base function which satisfy $b(x_{0})=f(x_{0})$ , $b(x_{N})=f(x_{N})$.

Let $f^{\alpha}(=f_{\Delta,b}^{\alpha})$ be the continuous function whose
graph is the attractor of the IFS represented by (2.3) and (2.4). This
$f^{\alpha}$ is called the $\alpha$_-_ fractal interpolation function or
simply $\alpha$-fractal function of $f$ with respect to the base function $b$
and the partition $\Delta$. From (2.2), $f^{\alpha}$ satisfies the fixed point
equation

$f^{\alpha}(x)=f(x)+\alpha_{i}(f^{\alpha}-b)(L_{i}^{-1}(x)),$ (2.5)

for all $x\in I_{i},$ $i=1,2,...,N.$ From (2.5), it is easy to deduce that

$\left\|f^{\alpha}-f\right\|_{\infty}\leq\frac{\left|\alpha\right|_{\infty}}{1-\left|\alpha\right|_{\infty}}\left\|f-b\right\|_{\infty}.$

For $\alpha=0$, the fractal function $f^{\alpha}$ agrees with $f$. More
discussion of $\alpha$-fractal function for different choices of $b$ can be
found in [7, 9, 10].

##  3\. Definite Integral of $\alpha$-Fractal Functions

In this section, the definite integral $\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx$
of the fractal function $f^{\alpha}$ corresponding to any continuous function
$f$ defined on a compact interval $I=[x_{0},x_{N}]$ is estimated and some
results related to the definite integrals of fractal functions are
established.

######  Theorem 3.1.

Let $f^{\alpha}$ be the fractal function of $f\in\mathcal{C}(I)$ with base
$b\in\mathcal{C}(I)$, then

$\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx=\frac{1}{(1-\lambda)}\int_{x_{0}}^{x_{N}}f(x)dx-\frac{\lambda}{(1-\lambda)}\int_{x_{0}}^{x_{N}}b(x)dx,$
(3.1)

where $\lambda=\sum_{i=1}^{N}a_{i}\alpha_{i}.$

###### Proof.

For $x\in I_{i},i=1,...,N,$ the self-referential equation for $f^{\alpha}$:

$f^{\alpha}(x)=f(x)+\alpha_{i}(f^{\alpha}-b)(L_{i}^{-1}(x)).$

Then

$\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx=\int_{x_{0}}^{x_{N}}f(x)dx+\sum_{i=1}^{N}\alpha_{i}\int_{x_{i-1}}^{x_{i}}(f^{\alpha}-b)(L_{i}^{-1}(x))dx.$

Letting $L_{i}^{-1}(x)=z,$

$\displaystyle\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx$ $\displaystyle=$
$\displaystyle\int_{x_{0}}^{x_{N}}f(x)dx+\left[\sum_{i=1}^{N}a_{i}\alpha_{i}\right]\int_{x_{0}}^{x_{N}}(f^{\alpha}-b)(z)dz$
$\displaystyle=$
$\displaystyle\int_{x_{0}}^{x_{N}}f(x)dx+\lambda\int_{x_{0}}^{x_{N}}(f^{\alpha}-b)(x)dx,$

where $\lambda=\sum_{i=1}^{N}a_{i}\alpha_{i}.$ Therefore

$\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx=\frac{1}{(1-\lambda)}\int_{x_{0}}^{x_{N}}f(x)dx-\frac{\lambda}{(1-\lambda)}\int_{x_{0}}^{x_{N}}b(x)dx.$

∎

In the following example, we calculate the definite integral of the fractal
function $f^{\alpha}(x)=(x^{3}+x)^{\alpha}$, where
$\alpha=(0.2,-0.3,0.5,0.3,0.4)$ with base function $b(x)=2x$, over the
interval $[0,1]$. The function $f(x)=x^{3}+x$ and its fractal function on
$[0,1]$ are shown in Figure 1.

######  Example.

Let $\Delta:0<0.2<0.4<0.6<0.8<1$ be a partion of $I=[0,1]$, then
$a_{i}=0.2,i=1,2,...,5,$ and $\lambda=\sum_{i=1}^{5}a_{i}\alpha_{i}=0.22.$
Applying Theorem 3.1,

$\displaystyle\int_{0}^{1}(x^{3}+x)^{\alpha}dx$ $\displaystyle=$
$\displaystyle\frac{1}{(1-\lambda)}\int_{0}^{1}(x^{3}+x)dx-\frac{\lambda}{(1-\lambda)}\int_{0}^{1}2xdx$
$\displaystyle=$
$\displaystyle\frac{1}{(1-0.22)}\left[\frac{x^{4}}{4}+\frac{x^{2}}{2}\right]_{0}^{1}-\frac{0.22}{(1-0.22)}\left[x^{2}\right]_{0}^{1}$
$\displaystyle=$ $\displaystyle\frac{53}{78}.$

Figure 1. The function $f(x)=x^{3}+x$ and its fractal function
$f^{\alpha}(x)=(x^{3}+x)^{\alpha}$

######  Corollary 3.2.

Let the scale vector $\alpha=(\alpha_{1},\alpha_{2},...,\alpha_{N})$ be such
that $\sum_{i=1}^{N}\alpha_{i}=0.$ Then, for any uniform partition
$\Delta:x_{0}<x_{0}+h<x_{0}+2h<...<x_{0}+Nh=x_{N}$ of $I=[x_{0},x_{N}]$ and
for any base function $b$,

$\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx=\int_{x_{0}}^{x_{N}}f(x)dx.$

###### Proof.

For uniform partition,
$a_{i}=\frac{x_{i}-x_{i-1}}{x_{N}-x_{0}}=\frac{h}{x_{N}-x_{0}}=\frac{1}{N},i=1,2,...,N.$
The desired result is obtained by setting
$\lambda=\sum_{i=1}^{N}a_{i}\alpha_{i}=\frac{1}{N}\sum_{i=1}^{N}\alpha_{i}=0$
in Theorem 3.1. ∎

######  Example.

Consider $\Delta:0<0.2<0.4<0.6<0.8<1$ as a partition of $I=[0,1]$. Let
$f(x)=\frac{1}{x+1}$ and $b(x)=1-\frac{x}{2}$ be defined on $I=[0,1].$ Let
$f^{\alpha}$ be the fractal function of $f$ with base function $b$. Suppose
the scale vector $\alpha=(-0.2,0.4,0.3,-0.6,0.1)$. Here
$a_{i}=0.2,i=1,2,...,5,$ and $\lambda=\sum_{i=1}^{5}a_{i}\alpha_{i}=0$.
Therefore, applying Corollary 3.2

$\displaystyle\int_{0}^{1}f^{\alpha}(x)dx$ $\displaystyle=$
$\displaystyle\int_{0}^{1}f(x)dx$ $\displaystyle=$
$\displaystyle\int_{0}^{1}\frac{dx}{x+1}$ $\displaystyle=$ $\displaystyle\log
2.$

######  Theorem 3.3.

Let $\alpha=(\alpha_{1},\alpha_{2},...,\alpha_{N})$ and
$\beta=(\beta_{1},\beta_{2},...,\beta_{N})$ be two scale vectors with
$\sum_{i=1}^{N}\alpha_{i}=\sum_{i=1}^{N}\beta_{i}$. Suppose that $f^{\alpha}$
and $f^{\beta}$ are respectively the $\alpha$-fractal function and
$\beta$-fractal function of $f$ with the base function $b$. Then, for any
uniform partition $\Delta:x_{0}<x_{0}+h<x_{0}+2h<...<x_{0}+Nh=x_{N}$ of
$I=[x_{0},x_{N}]$,

$\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx=\int_{x_{0}}^{x_{N}}f^{\beta}(x)dx.$

###### Proof.

For uniform partition,
$a_{i}=\frac{x_{i}-x_{i-1}}{x_{N}-x_{0}}=\frac{h}{x_{N}-x_{0}}=\frac{1}{N},i=1,2,...,N$
and for the scale vectors $\alpha$ and $\beta$,
$\lambda_{\alpha}=\sum_{i=1}^{N}a_{i}\alpha_{i}=\frac{1}{N}\sum_{i=1}^{N}\alpha_{i}=\frac{1}{N}\sum_{i=1}^{N}\beta_{i}=\sum_{i=1}^{N}a_{i}\beta_{i}=\lambda_{\beta}$.

Therefore, applying Theorem 3.1

$\displaystyle\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx$ $\displaystyle=$
$\displaystyle\frac{1}{(1-\lambda_{\alpha})}\int_{x_{0}}^{x_{N}}f(x)dx-\frac{\lambda_{\alpha}}{(1-\lambda_{\alpha})}\int_{x_{0}}^{x_{N}}b(x)dx$
$\displaystyle=$
$\displaystyle\frac{1}{(1-\lambda_{\beta})}\int_{x_{0}}^{x_{N}}f(x)dx-\frac{\lambda_{\beta}}{(1-\lambda_{\beta})}\int_{x_{0}}^{x_{N}}b(x)dx$
$\displaystyle=$ $\displaystyle\int_{x_{0}}^{x_{N}}f^{\beta}(x)dx.$

∎

######  Theorem 3.4.

If $\left|\alpha\right|_{\infty}\rightarrow 0,$ then

$\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx\rightarrow\int_{x_{0}}^{x_{N}}f(x)dx$

and if $\alpha=0,$ then

$\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx=\int_{x_{0}}^{x_{N}}f(x)dx.$

###### Proof.

For first one, use
$\lambda=\sum_{i=1}^{N}a_{i}\alpha_{i}\leq\sum_{i=1}^{N}a_{i}\left|\alpha\right|_{\infty}\rightarrow
0$ as $\left|\alpha\right|_{\infty}\rightarrow 0$ in Theorem 3.1. The second
part is straight forward. ∎

######  Theorem 3.5.

Let $f_{b}^{\alpha},g_{\widetilde{b}}^{\alpha}$ be the fractal functions of
$f,g\in\mathcal{C}(I)$ with the base functions
$b,\widetilde{b}\in\mathcal{C}(I)$ respectively, then

$\int_{x_{0}}^{x_{N}}(\gamma f+\delta g)_{\gamma
b+\delta\widetilde{b}}^{\alpha}(x)dx=\gamma\int_{x_{0}}^{x_{N}}f_{b}^{\alpha}(x)dx+\delta\int_{x_{0}}^{x_{N}}g_{\widetilde{b}}^{\alpha}(x)dx,$

for any $\gamma,\delta\in\mathbb{R}.$

###### Proof.

For $x\in I_{i},i=1,...,N,$ the self-referential equations for
$f_{b}^{\alpha}$ and $g_{\widetilde{b}}^{\alpha}$ are respectively

$f_{b}^{\alpha}(x)=f(x)+\alpha_{i}(f_{b}^{\alpha}-b)(L_{i}^{-1}(x))$

and

$g_{\widetilde{b}}^{\alpha}(x)=g(x)+\alpha_{i}(g_{\widetilde{b}}^{\alpha}-\widetilde{b})(L_{i}^{-1}(x)).$

The equality

$(\gamma f+\delta g)_{\gamma b+\delta\widetilde{b}}^{\alpha}=\gamma
f_{b}^{\alpha}+\delta g_{\widetilde{b}}^{\alpha}$ (3.2)

is obtained from the uniqueness of the self-referential equation below

$(\gamma f_{b}^{\alpha}+\delta g_{\widetilde{b}}^{\alpha})(x)=\left(\gamma
f+\delta g\right)(x)+\alpha_{i}\left[(\gamma f_{b}^{\alpha}+\delta
g_{\widetilde{b}}^{\alpha})-(\gamma
b+\delta\widetilde{b})\right](L_{i}^{-1}(x)).$

The result is follows from the equation (3.2). ∎

Setting $\gamma=-1$ and $\delta=0$ in Theorem 3.5, we get the following
result.

######  Corollary 3.6.

Let $f_{b}^{\alpha}$ be the fractal function of $f$ with base function $b$,
then

$\int_{x_{0}}^{x_{N}}(-f)_{-b}^{\alpha}(x)dx=-\int_{x_{0}}^{x_{N}}f_{b}^{\alpha}(x)dx.$

A supporting example to the Corollary 3.6 is given below.

######  Example.

Let $I=[0,1]$ and $\Delta:0<0.2<0.4<0.6<0.8<1$ be a partition of $I$. Let
$f(x)=x^{3}$ and $b(x)=x^{2}$ be defined on $I=[0,1].$ Let $f_{b}^{\alpha}$
and $(-f)_{-b}^{\alpha}$ be the fractal functions of $f$ and $-f$ with base
functions $b$ and $-b$ respectively. Suppose the scale vector
$\alpha=(-0.1,0,0.1,0.2,0.3)$. Here $a_{i}=0.2,i=1,2,...,5,$ and
$\lambda=\sum_{i=1}^{5}a_{i}\alpha_{i}=0.1.$ Therefore, using Theorem 3.1

$\displaystyle\int_{0}^{1}(x^{3})_{x^{2}}^{\alpha}(x)dx$ $\displaystyle=$
$\displaystyle\int_{0}^{1}f_{b}^{\alpha}(x)dx$ $\displaystyle=$
$\displaystyle\frac{1}{(1-\lambda)}\int_{0}^{1}f(x)dx-\frac{\lambda}{(1-\lambda)}\int_{0}^{1}b(x)dx$
$\displaystyle=$
$\displaystyle\frac{1}{(1-0.1)}\int_{0}^{1}x^{3}dx-\frac{0.1}{(1-0.1)}\int_{0}^{1}x^{2}dx$
$\displaystyle=$ $\displaystyle\frac{13}{54}.$

and

$\displaystyle\int_{0}^{1}(-x^{3})_{-x^{2}}^{\alpha}(x)dx$ $\displaystyle=$
$\displaystyle\int_{0}^{1}(-f)_{-b}^{\alpha}(x)dx$ $\displaystyle=$
$\displaystyle\frac{1}{(1-\lambda)}\int_{0}^{1}(-f(x))dx-\frac{\lambda}{(1-\lambda)}\int_{0}^{1}(-b(x))dx$
$\displaystyle=$
$\displaystyle\frac{1}{(1-0.1)}\int_{0}^{1}(-x^{3})dx-\frac{0.1}{(1-0.1)}\int_{0}^{1}(-x^{2})dx$
$\displaystyle=$ $\displaystyle-\frac{13}{54}.$

######  Lemma 3.1.

Let $f_{b}^{\alpha}$ be the $\alpha$-fractal function of $f\in\mathcal{C}(I)$
with the base function $b\in\mathcal{C}(I).$ If $g\in\mathcal{C}(I)$ is
linear, then $g\circ f_{b}^{\alpha}$ is also $\alpha$ -fractal function of
$g\circ f$ with the base function $g\circ b$. In other words,

$(g\circ f)_{g\circ b}^{\alpha}=g\circ f_{b}^{\alpha}.$

###### Proof.

For $x\in I_{i},i=1,...,N,$ we have

$f_{b}^{\alpha}(x)=f(x)+\alpha_{i}(f_{b}^{\alpha}-b)(L_{i}^{-1}(x)).$

Then, for $x\in I_{i},i=1,...,N,$

$\displaystyle(g\circ f_{b}^{\alpha})(x)$ $\displaystyle=$ $\displaystyle
g(f_{b}^{\alpha}(x))$ (3.3) $\displaystyle=$ $\displaystyle(g\circ
f)(x)+\alpha_{i}(g\circ f_{b}^{\alpha}-g\circ b)(L_{i}^{-1}(x)).$

Since $(g\circ f)(x_{0})=(g\circ b)(x_{0})$ and $(g\circ f)(x_{N})=(g\circ
b)(x_{N})$, the uniqueness of the self-referential equation (3.3) gives the
desired result. ∎

######  Theorem 3.7.

Let $f_{b}^{\alpha}$ be the $\alpha$-fractal function of $f\in\mathcal{C}(I)$
with the base function $b\in\mathcal{C}(I).$ If $g\in\mathcal{C}(I)$ is
linear, then

$\int_{x_{0}}^{x_{N}}(g\circ
f_{b}^{\alpha})(x)dx=\frac{1}{(1-\lambda)}\int_{x_{0}}^{x_{N}}(g\circ
f)(x)dx-\frac{\lambda}{(1-\lambda)}\int_{x_{0}}^{x_{N}}(g\circ b)(x)dx,$

where $\lambda=\sum_{i=1}^{N}a_{i}\alpha_{i}.$

###### Proof.

Since $g\circ f_{b}^{\alpha}$ is the $\alpha$-fractal function of $g\circ f$
with the base function $g\circ b$, using Theorem $\ref{thrm-fintegral},$

$\displaystyle\int_{x_{0}}^{x_{N}}(g\circ f_{b}^{\alpha})(x)dx$
$\displaystyle=$ $\displaystyle\int_{x_{0}}^{x_{N}}(g\circ f)_{g\circ
b}^{\alpha}(x)dx$ $\displaystyle=$
$\displaystyle\frac{1}{(1-\lambda)}\int_{x_{0}}^{x_{N}}(g\circ
f)(x)dx-\frac{\lambda}{(1-\lambda)}\int_{x_{0}}^{x_{N}}(g\circ b)(x)dx.$

∎

##  4\. Construction of Flipped $\alpha$-Fractal Function

######  Definition 4.1.

Let $I=[x_{0},x_{N}]$ and $I_{F}=[-x_{N},-x_{0}]$. We can define the flipped
function of $f\in\mathcal{C}(I)$ about $y$-axis as

$f_{F}(-x)=f(x),\text{for all }x\in I.$

or equivalently

$f_{F}(x)=f(-x),\text{for all }x\in I_{F}.$

Let $f^{\alpha}(=f_{\Delta,b}^{\alpha})$ be the $\alpha$-fractal function
associated to $f$ with the base function $b$ and the partition $\Delta$.
Recall the fixed point equation for $f^{\alpha}$:

$f^{\alpha}(x)=f(x)+\alpha_{i}(f^{\alpha}-b)(L_{i}^{-1}(x)),$

for all $x\in I_{i},$ $i=1,2,...,N$.

Let $\Delta_{F}:-x_{N}<-x_{N-1}<...<-x_{0}$ is a partition of the closed
interval $I_{F}=[-x_{N},-x_{0}]$. Set $I_{Fi}=[-x_{N+1-i},-x_{N-i}]$ for
$i=1,2,...,N$. Let $f_{F}\in\mathcal{C}(I_{F})$ such that $f_{F}(-x)=f(x),$
for all $x\in I$ and let
$\alpha_{F}=(\alpha_{N},\alpha_{N-1},...,\alpha_{1}).$ Define the RB operator
$T_{\Delta_{F},b_{F}}:\mathcal{C}^{\ast}(I_{F})\rightarrow\mathcal{C}^{\ast\ast}(I_{F})$
by

$(T_{\Delta_{F},b_{F}}g)(x)=f_{F}(x)+\alpha_{N+1-i}(g-b_{F})((L_{F})_{i}^{-1}(x)),x\in
I_{Fi},$ (4.1)

where $L_{Fi}(x)=a_{Fi}x+e_{Fi},$ $a_{Fi}=a_{N+1-i}$,
$e_{Fi}=-e_{N+1-i},i=1,2,...,N$ and $b_{F}\in\mathcal{C}(I_{F})$ satisfying
$b_{F}(-x)=b(x),$ for all $x\in I$. We can easily show that
$T_{\Delta_{F},b_{F}}$ is a contraction map.

Let $f_{F}^{\alpha_{F}}(=f_{F\Delta_{F},b_{F}}^{\alpha_{F}})$ be the
$\alpha_{F}$-fractal function of $f_{F}$ with respect to (4.1 ). From (4.1),
$f_{F}^{\alpha_{F}}$ satisfies the fixed point equation

$f_{F}^{\alpha_{F}}(x)=f_{F}(x)+\alpha_{N+1-i}(f_{F}^{\alpha_{F}}-b_{F})((L_{F})_{i}^{-1}(x)),x\in
I_{Fi},i=1,2,...,N.$ (4.2)

From (4.2), for $i=1,2,...,N$

$f_{F}^{\alpha_{F}}(-x)=f_{F}(-x)+\alpha_{N+1-i}(f_{F}^{\alpha_{F}}-b_{F})((L_{F})_{i}^{-1}(-x)),x\in
I_{N+1-i}.$ (4.3)

Since $(L_{F})_{i}^{-1}(-x)=-L_{N+1-i}^{-1}(x),i=1,2,...,N,$ then (4.3)
becomes, for $i=1,2,...,N$

$f_{F}^{\alpha_{F}}(-x)=f_{F}(-x)+\alpha_{N+1-i}(f_{F}^{\alpha_{F}}-b_{F})(-L_{N+1-i}^{-1}(x)),x\in
I_{N+1-i}.$

This above equation is the same as

$f_{F}^{\alpha_{F}}(-x)=f_{F}(-x)+\alpha_{i}(f_{F}^{\alpha_{F}}-b_{F})(-L_{i}^{-1}(x)),x\in
I_{i},i=1,2,..,N.$

Therefore, for $i=1,2,...,N$

$\displaystyle f_{F}^{\alpha_{F}}(-x)$ $\displaystyle=$ $\displaystyle
f_{F}(-x)+\alpha_{i}f_{F}^{\alpha_{F}}(-L_{i}^{-1}(x))-\alpha_{i}b_{F}(-L_{i}^{-1}(x)),x\in
I_{i}$ $\displaystyle=$ $\displaystyle
f(x)+\alpha_{i}f_{F}^{\alpha_{F}}(-L_{i}^{-1}(x))-\alpha_{i}b(L_{i}^{-1}(x)),x\in
I_{i}.$

Let $g$ be the flipped function of $f_{F}^{\alpha_{F}}$, then
$f_{F}^{\alpha_{F}}(-x)=g(x),x\in I.$ Now, for all $x\in I_{i},i=1,2,...,N$

$\displaystyle g(x)$ $\displaystyle=$ $\displaystyle f_{F}^{\alpha_{F}}(-x)$
$\displaystyle=$ $\displaystyle
f(x)+\alpha_{i}f_{F}^{\alpha_{F}}(-L_{i}^{-1}(x))-\alpha_{i}b(L_{i}^{-1}(x))$
$\displaystyle=$ $\displaystyle
f(x)+\alpha_{i}g(L_{i}^{-1}(x))-\alpha_{i}b(L_{i}^{-1}(x))$ $\displaystyle=$
$\displaystyle f(x)+\alpha_{i}(g-b)(L_{i}^{-1}(x)).$

The uniqueness of the fixed point equation of $f^{\alpha}$ implies that

$g(x)=f^{\alpha}(x)=f_{F}^{\alpha_{F}}(-x),x\in I_{i},i=1,2,..,N.$

Thus we see that $f_{F}^{\alpha_{F}}$ is the flipped function of $f^{\alpha}$
about $y$-axis. We say that $f_{F}^{\alpha_{F}}$ is the flipped
$\alpha$-fractal function of $f$.

######  Example.

Figure 2 corresponds to the function $f(x)=\sqrt{x}$ on the interval $[0,1]$
and its flipped function $(\sqrt{x})_{F}$ on $[-1,0]$. The fractal function
$f^{\alpha}$ associated with $f$ corresponding to the partition
$\Delta:0<0.2<0.4<0.6<0.8<1$, the scaling vector
$\alpha=(0.3,0.5,0.2,0.15,0.02)$, and the base function $b(x)=x$ and the
flipped $\alpha$-fractal function of $f$ are shown in Figure 3.

Figure 2. The function $\sqrt{x}$ and its flipped function $(\sqrt{x})_{F}$

Figure 3. Fractal function $(\sqrt{x})^{\alpha}$ and its flipped function
$(\sqrt{x})_{F}^{{\alpha}_{F}}$

######  Theorem 4.1.

Suppose that $I=[x_{0},x_{N}]$ be any compact interval. Let $f^{\alpha}$ be
the fractal function of $f\in\mathcal{C}(I)$ with base $b\in\mathcal{C}(I)$,
then

$\int_{-x_{N}}^{-x_{0}}f_{F}^{\alpha_{F}}(x)dx=\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx.$

###### Proof.

For $x\in I_{Fi},i=1,...,N,$ the self-referential equation for
$f_{F}^{\alpha_{F}}$ is

$f_{F}^{\alpha_{F}}(x)=f_{F}(x)+\alpha_{N+1-i}(f_{F}^{\alpha_{F}}-b_{F})((L_{F})_{i}^{-1}(x)).$

Then

$\displaystyle\int_{-x_{N}}^{-x_{0}}f_{F}^{\alpha_{F}}(x)dx$ $\displaystyle=$
$\displaystyle\int_{-x_{N}}^{-x_{0}}f_{F}(x)dx$
$\displaystyle+\sum_{i=1}^{N}\alpha_{N+1-i}\int_{-x_{N+1-i}}^{-x_{N-i}}(f_{F}^{\alpha_{F}}-b_{F})((L_{F})_{i}^{-1}(x))dx.$

Letting $(L_{F})_{i}^{-1}(x)=z,$

$\displaystyle\int_{-x_{N}}^{-x_{0}}f_{F}^{\alpha_{F}}(x)dx$ $\displaystyle=$
$\displaystyle\int_{-x_{N}}^{-x_{0}}f_{F}(x)dx$
$\displaystyle+\left[\sum_{i=1}^{N}a_{N+1-i}\alpha_{N+1-i}\right]\int_{-x_{N}}^{-x_{0}}(f_{F}^{\alpha_{F}}-b_{F})(z)dz$
$\displaystyle=$
$\displaystyle\int_{-x_{N}}^{-x_{0}}f_{F}(x)dx+\lambda_{F}\int_{-x_{N}}^{-x_{0}}(f_{F}^{\alpha_{F}}-b_{F})(x)dx,$

where
$\lambda_{F}=\sum_{i=1}^{N}a_{N+1-i}\alpha_{N+1-i}=\sum_{i=1}^{N}a_{i}\alpha_{i}=\lambda.$

Therefore

$\displaystyle\int_{-x_{N}}^{-x_{0}}f_{F}^{\alpha_{F}}(x)dx$ $\displaystyle=$
$\displaystyle\frac{1}{(1-\lambda)}\int_{-x_{N}}^{-x_{0}}f_{F}(x)dx-\frac{\lambda}{(1-\lambda)}\int_{-x_{N}}^{-x_{0}}b_{F}(x)dx$
$\displaystyle=$
$\displaystyle\frac{1}{(1-\lambda)}\int_{-x_{N}}^{-x_{0}}f(-x)dx-\frac{\lambda}{(1-\lambda)}\int_{-x_{N}}^{-x_{0}}b(-x)dx$
$\displaystyle=$
$\displaystyle\frac{1}{(1-\lambda)}\int_{x_{0}}^{x_{N}}f(x)dx-\frac{\lambda}{(1-\lambda)}\int_{x_{0}}^{x_{N}}b(x)dx$
$\displaystyle=$ $\displaystyle\int_{x_{0}}^{x_{N}}f^{\alpha}(x)dx.$

∎

The following example is an application of the Theorem 4.1.

######  Example.

Consider the integral $\int_{-1}^{0}(x^{2})_{-x}^{\alpha}(x)dx,$ where
$\alpha=(0.2,-0.1,0,0.3,0.4).$ To apply the Theorem 4.1, first we need to find
out $\int_{0}^{1}(x^{2})_{x}^{\alpha}(x)dx.$ For this, let
$\Delta:0<0.2<0.4<0.6<0.8<1$ be a partition of $I=[0,1]$, then
$a_{i}=0.2,i=1,2,...,5,$ and $\lambda=\sum_{i=1}^{5}a_{i}\alpha_{i}=0.16.$
Using Theorem 3.1 and applying the Theorem 4.1,

$\displaystyle\int_{-1}^{0}(x^{2})_{-x}^{\alpha}(x)dx$ $\displaystyle=$
$\displaystyle\int_{0}^{1}(x^{2})_{x}^{\alpha}(x)dx$ $\displaystyle=$
$\displaystyle\frac{1}{(1-\lambda)}\int_{0}^{1}x^{2}dx-\frac{\lambda}{(1-\lambda)}\int_{0}^{1}xdx$
$\displaystyle=$
$\displaystyle\frac{1}{3(1-0.16)}\left[x^{3}\right]_{0}^{1}-\frac{0.16}{2(1-0.16)}\left[x^{2}\right]_{0}^{1}$
$\displaystyle=$ $\displaystyle\frac{19}{63}.$

## References

  * [1] Barnsley, M.F. Fractal Everywhere, 2nd edn., Academic Press, USA, (1993).
  * [2] Barnsley, M.F. Fractal functions and interpolation. Constr. Approx. 2 (1986), 303–329.
  * [3] Barnsley, M.F.; Harrington A.N. The Calculus of Fractal Interpolation Functions. J. Approx. Theory 57 (1989), 14–34.
  * [4] Hutchinson, J.E. Fractals and self-similarity. Indiana Univ. Math. J. 30 (1981), 713–747.
  * [5] Mandelbrot, B.B. The Fractal Geometry of Nature, W.H. Freeman and Company, New York, (1983).
  * [6] Massopust, P.R. Fractal Functions, Fractal Surfaces, and Wavelets, Academic Press, New York, (1995).
  * [7] Navascués, M.A. Fractal Polynomial Interpolation. Z. Analysis Anwend. 25(2) (2005), 401–418.
  * [8] Navascués, M.A. Fractal trigonometric approximation. Electron Trans. Numer. Anal. 20 (2005), 64–74.
  * [9] Navascués, M.A. Non-smooth polynomials. Int. J. Math. Anal. 1 (2007), 159–174.
  * [10] Navascués, M.A.; Chand, A.K.B. Fundamental sets of fractal functions. Acta Appl. Math. 100(3) (2008), 247–261.
  * [11] Navascués, M.A. Reconstruction of sampled signals with fractal functions. Acta Appl. Math. 110(3) (2010), 1199–1210.
  * [12] Navascués, M.A. Fractal Haar system. Nonlinear Anal. 74(12) (2011), 4152–4165.
  * [13] Navascués, M.A.; Sebastián, M.V. Numerical integration of affine fractal functions. J. Comput. Appl. Math. 252 (2013), 169–176.
  * [14] Pan, X.Z. Fractional Calculus of Fractal Interpolation Function on $[0,b](b>0)$. Abstr. Appl. Anal. (2014), Article ID 640628, http://dx.doi.org/10.1155/2014/640628.


# From molecules to scaffolds to functional groups:
building context-dependent molecular representation
via multi-channel learning

Yue Wan1, Jialu Wu2, Tingjun Hou2, Chang-Yu Hsieh211footnotemark: 1, Xiaowei
Jia111footnotemark: 1
University of Pittsburgh, Computer Science Department, Pittsburgh, PA 15260,
United States1
Innovation Institute for Artificial Intelligence in Medicine of Zhejiang
University,
College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, 310058,
China2 Corresponding authors.
E-mail addresses<EMAIL_ADDRESS>(T. Hou<EMAIL_ADDRESS>(C.-Y.
Hsieh<EMAIL_ADDRESS>(X. Jia)

###### Abstract

Reliable molecular property prediction is essential for various scientific
endeavors and industrial applications, such as drug discovery. However, the
scarcity of data, combined with the highly non-linear causal relationships
between physicochemical and biological properties and conventional molecular
featurization schemes, complicates the development of robust molecular machine
learning models. Self-supervised learning (SSL) has emerged as a popular
solution, utilizing large-scale, unannotated molecular data to learn a
foundational representation of chemical space that might be advantageous for
downstream tasks. Yet, existing molecular SSL methods largely overlook domain-
specific knowledge, such as molecular similarity and scaffold importance, as
well as the context of the target application when operating over the large
chemical space. This paper introduces a novel learning framework that
leverages the knowledge of structural hierarchies within molecular structures,
embeds them through separate pre-training tasks over distinct channels, and
employs a task-specific channel selection to compose a context-dependent
representation. Our approach demonstrates competitive performance across
various molecular property benchmarks and establishes some state-of-the-art
results. It further offers unprecedented advantages in particularly
challenging yet ubiquitous scenarios like activity cliffs with enhanced
robustness and generalizability compared to other baselines.

##  1 Introduction

Empowered by the advancement of machine learning techniques, molecular machine
learning has shown its great potential in computational chemistry and drug
discovery [16, 29]. The data-driven protocol allows the model to infer
biochemical behaviors from simple representations like SMILES sequence [44]
and molecular graph, enabling fast identification of drug candidates via rapid
screening of vast chemical spaces [15], as well as prediction of binding
affinity, toxicity, and other pharmacological properties [46, 40]. These
advancements significantly accelerate the drug discovery procedures, saving
time and efforts from the traditional wet-lab experiments [17, 11]. However,
it is fundamentally challenging to learn an effective and robust molecular
representation via machine learning, limited by the expensive gathering of
precise biochemical labels and the complexity underlying the structure-
property relationships (SPR). With limited molecule data, models may become
overly adapted to specific structural patterns within the training molecules,
making it fail to generalize to the broader chemical space. In addition, the
challenge of “activity cliffs” (AC) [26] in drug discovery, where minor
changes in molecular structure significantly alter the biological activity,
further impose obstacles in developing accurate Quantitative SPR (QSPR) models
[40, 35, 32, 9].

Inspired by the success of the pretrain-finetune workflow in computer vision
[8] and natural language processing [10, 28], various methods in molecule
self-supervised learning (SSL) [18, 30, 47, 42, 50, 21, 13, 12] have emerged.
In the self-supervised setting, machine learning models are pre-trained to
learn generic molecule representations by optimizing the performance on pre-
defined pre-training tasks on large-scale unannotated molecule data. These
tasks are designed in a way such that solving them requires identification of
important structural patterns and understanding of rudimentary chemical
knowledge. Existing molecule SSL methods can be mainly classified into two
categories: predictive and contrastive. Predictive learning [18, 22, 30, 47]
aims to predict structural components given contexts at different levels,
which mainly focuses on intra-data relationship. Hu et al. [18] conduct masked
attribute prediction, where the model is pre-trained to predict the masked
atom/bond attribute (e.g., atom type, bond direction). MoleBERT [47] further
improves masked attribute prediction into masked atom prediction. GROVER [30]
attempts to predict the masked substructure. Besides, it also proposes motif
prediction, where the existence of functional groups are predicted as a multi-
label classification task. These two tasks are often framed as molecule
context prediction. Contrastive learning [55, 41, 48, 53, 47, 50, 21],
initially proposed in computer vision [6], aims to learn the inter-data
relationship by pulling semantic-similar data samples closer and pushing
semantic-dissimilar samples apart in the representation space. Note that this
idea aligns well with common heuristics in chemistry, where structurally
similar molecules are likely to exhibit similar physicochemical and biological
properties. Most works attempt to generalize the same SSL methods across
multiple graphical domains (e.g., social networks and molecular graphs),
whereas it is unsure whether the same learning schemes are compatible to all
settings. Recently, several studies have pointed out that the generalized SSL
methods may fail to learn effective molecular representation. RePRA [54] was
proposed as a metric for measuring the representations’ potential in solving
activity cliffs and scaffold hopping [5], and experiments showed that most SSL
pretrained representations perform worse than the molecule fingerprint. The
work in [36] explored various graph SSL methods on molecule and observed that
some pre-training strategies can only bring marginal improvement, while some
even induce negative transfer. Meanwhile, several studies [40, 9, 45] have
demonstrated that existing pre-trained molecular representations struggle with
challenging yet ubiquitous situations like activity cliffs in drug discovery.

This work aims to enhance molecular representation learning while bridging the
gap between domain-agnostic SSL methods and domain-specific molecular tasks.
We start by identifying the two major drawbacks in existing methods: Firstly,
in contrastive learning, the conventional formulations of the semantic-
similar/dissimilar (i.e., positive/negative) samples are not well-tailored for
molecular graphs. Most graph contrastive methods generate positive samples via
graph perturbation, such as atom masking, bond deletion/addition [50, 21, 42,
47], and subgraph masking [42] in terms of the molecular graphs. However,
chemical validity might be challenged by the perturbation (e.g., randomly
breaking a bond). Molecules may also lose essential characteristics by
perturbing important motifs like aromatic rings, shifting the semantics
distant away. The negative samples, taken from the rest of the data, are often
treated equally. It essentially neglects the existing molecule structural
relationship and the presence of specific molecular components; Second, almost
all existing works attempt to learn a context-independent molecular
representation space, aiming to generalize to various applications. However,
this contradicts the fact that molecular properties are often context-
dependent, from both the physical (e.g., surrounding environments) and
biological (e.g., interaction with proteins) perspectives. In other words, it
remains uncertain whether the same SSL tasks learned in pre-training could
align well with diverse tasks of distinct properties in fine-tuning, thereby
leading to the learning gap.

Figure 1: Framework overview. a) The prompt guided pretrain-finetune
framework. For each downstream task, the model is optimized additionally on
the prompt weight selection, locating the best pre-trained channel compatible
with the current application. b) Molecule contrastive learning, where the
positive samples $G_{i}^{\prime}$ from subgraph masking is contrasted against
negative samples $G_{j}$ by an adaptive margin. c) Scaffold contrastive
distancing, where the positive samples $G_{i}^{\prime}$ from scaffold-
invariant perturbation is contrasted by against negative samples $G_{j}$ by an
adaptive margin. d) Context prediction channel consists of masked subgraph
prediction and motif prediction tasks. e) Prompt-guided aggregation module,
which conditionally aggregates atom representations into molecule
representation by prompt token. It is realized via a multi-head attention with
prompt embedding $h_{p}$ being the query.

To approach the aforementioned challenges, we introduce a prompt-guided multi-
channel learning framework for molecular representation learning using graph
neural networks (GNNs). Each of the $k$ channels, guided by a specific prompt
token, is responsible for learning one dedicated SSL task. Essentially, the
pre-trained model is able to learn $k$ distinct representation spaces. During
fine-tuning, a prompt selection module is optimized to aggregate $k$
representations into a composite representation, which is used for molecular
property predictions. This involves determining which prompt-guided
information channel is most relevant to the current application, thereby
making the representation context-dependent. The prompt selection module is
initialized by identifying the composite representation with the smoothest
quantitative structure-property landscape [2]. We later show how this
composite formulation is more resilient to label overfitting and manifests
better robustness. We design the SSL tasks to form an interpolation from a
global view to a local view of the molecular structures. Besides leveraging
the global molecule contrastive learning [42] and the local context prediction
[30], we introduce a novel task called the scaffold contrastive distancing,
highlighting the foundational role of scaffolds in affecting molecular
characteristics and behaviors. Inspired by the facts that scaffolds are often
treated as starting points for new compound design, scaffold distancing aims
to map molecules with similar scaffolds (generated via scaffold-invariant
perturbations) closer in the representation space. Additionally, it pushes
molecules with different scaffolds apart, where the distance margin is
computed adaptively based on scaffold composition difference. Note that the
scaffold distancing tackles the partial but core view of the molecules. The
overall framework is pre-trained using ZINC15 [33], and evaluated on 7
molecular property prediction tasks in MoleculeNet [46] and 30 binding potency
prediction tasks in MoleculeACE [40]. Learning to leverage information from
different channels for different applications, our method surpasses various
baseline representation learning methods in both benchmarks. Moreover, our
method is shown to handle the challenge of activity cliffs more effectively,
whereas competing approaches are more susceptible to the negative transfer,
leading to substantial performance decline. Besides reporting the performance
metrics, we further examine the shift in representation space during fine-
tuning via epoch probing, and discover that our learned representation is
better at preserving pre-trained chemical knowledge with better
transferability and robustness compared to other baselines. Case study shows
that our method has the potential to identify crucial patterns that contribute
to activity cliffs even using merely topological cues. We also demonstrate
that the learned channel-wise representation distances are highly correlated
with the conventional fingerprint/descriptor similarity. These representations
can form an implicit representation hierarchy, visualized by iteratively
refining the chemical space clustering.

##  2 Results

###  2.1 Framework overview

The proposed framework (Figure 1) comprises three novel components compared to
the conventional pretrain-finetune paradigm on molecules: (1) The prompt-
guided multi-channel learning, (2) contrastive learning with adaptive margin,
and (3) scaffold-invariant molecule perturbation.

#### Prompt-guided multi-channel learning

We introduce a prompt-guided multi-channel learning framework for molecular
representation learning using GNNs, as shown in Figure 1a. Essentially, the
molecular graphs will first go through a unified GNN module, and then diverge
into $k$ different channels, each of which is responsible for learning
distinct SSL tasks. For each channel, a prompt token $p_{i}$ is utilized to
guide the molecule representation. This is realized via a prompt-guided
readout operation [23], which aggregates atom representations conditionally
into molecule representation given the prompt token (Figure 1e). Our
experiment involves three learning channels, which are molecule distancing,
scaffold distancing, and context prediction. Each channel focuses on a unique
aspect of the molecular structure, enabling molecular representation learning
from an hierarchical viewpoints, from a global view (i.e., entire molecule), a
partial view (i.e., core structure), and down to a local view (i.e.,
functional groups).

We now briefly introduce the three learning channels. (i) Molecule distancing
(Figure 1b) is similar to MolCLR [42], where subgraph masking is adopted for
molecule augmentation, and the augmented molecules are contrasted against
other molecules. We formulate our contrastive learning using the triplet loss
[31]. It considers triplet of data samples {anchor, positive, negative}, where
negative (i.e., dissimilar) samples are pushed apart against anchor and
positive (i.e., similar) samples by a margin $\alpha$. On top of this, we
propose the adaptive margin loss, as detailed below. It introduces another
level of distancing constraints based on the structural similarity of molecule
composition. (ii) We propose scaffold distancing (Figure 1c) as a novel
contrastive learning task that focuses on scaffold differences. Molecule
scaffolds are often viewed as the foundation for a range of biologically
active molecules. They play a crucial role in drug discovery and medicinal
chemistry by providing a starting point for the design of new compounds with
desired pharmacological properties. In other words, molecules with similar
scaffolds are more likely to possess similar physical (e.g., solubility,
lipophilicity) and biological (e.g., conformational property when interacting
with a protein) characteristics, and thereby sharing similar semantics.
Scaffold distancing contrasts the scaffold-invariant molecule perturbations,
as detailed below, against the other molecules with different scaffolds using
the adaptive margin loss. (iii) Context prediction (Figure 1d) involves masked
subgraph prediction and motif prediction, which are also adopted in GROVER
[30]. For each molecular graph, a random subgraph (i.e., a center atom and its
one-hop neighbors) is masked out, and the model aims to predict this masked
portion based on its surrounding structures. Motif prediction aims to predict
the existence of functional groups in the molecule. Both of them are
formulated as multi-label classification tasks. Note that this learning
channel mainly focuses on the local view of the molecule by identifying the
existence of substructure and functional groups, while molecule distancing and
scaffold distancing focus more on the global view and partial view,
respectively.

In the fine-tuning stage, the model consists of the same GNN and prompt-guided
aggregation modules, whose parameters are initialized from the pre-trained
model. The parameters in the aggregation modules are fixed during finetuning
as we aim to use it as a pooling layer independent of the downstream
applications. In addition, a prompt-tuning module $\tau_{\theta}(\cdot)$ is
introduced in fine-tuning to determine which channel is most relevant to the
current application. It essentially learns a task-specific prompt
distribution. The prompt weights are utilized to linearly combine the channel-
wise information into a composite molecule representation, which is then used
for the task-specific prediction. We initialize the prompt prompts by the
choosing the candidate which leads to the smoothest quantitative structure-
property landscape (i.e., lowest roughness index) [2] of the composite
representation. The framework is then evaluated on both the molecular property
prediction and binding potency prediction benchmarks. More details are
included in the Method section.

#### Contrastive learning with adaptive margin

We further introduce the adaptive margin loss, a variant of the triplet
contrastive learning loss [31], that supports the contrastive learning in the
first two channels (molecule distancing and scaffold distancing). In the
conventional triplet loss, the representation distance between the anchor
$G_{i}$ and negative (i.e., semantic-dissimilar) sample $G_{j}$ needs to be at
least by margin $\alpha$ larger than the distance between the anchor $G_{i}$
and positive (i.e., semantic similar) sample $G_{i}^{\prime}$. Conventional
contrastive learning methods use the same margin for any triplet considered.
However, when applied to molecule triplets, it neglects the known structural
relationship between molecules (e.g., co-existence of functional groups). To
learn a more fine-grained molecule representation space, we propose to
dynamically compute the molecule triplet margin based on the Tanimoto
similarity between structural fingerprints. As shown in Figure 1b and c, the
adaptive margin $\alpha_{\text{MCD}}(.)$ considers the molecule structural
similarity between $G_{i}$ and $G_{j}$, while $\alpha_{\text{SCD}}(.)$
considers the scaffold structural similarity between $s(G_{i})$ and
$s(G_{j})$. Another issue with the conventional triplet loss is that it
imposes no constraint on the representation space beyond the margin. It means
that the actual representation distances are not necessarily to be well
correlated with the computed margin, even if the margin constraints are fully
satisfied. This is further elaborated by an example in Supplementary
Information. Therefore, we include a secondary term into the adaptive margin
loss by considering the structural relationship among the anchor and different
negative samples. Detailed formulation of the adaptive margin loss is included
in Method section. With careful consideration of existing structural
similarity, the learned representation space would better capture molecule
relationships in a fine-grained representation space.

#### Scaffold-invariant molecule perturbation

To generate semantic-similar samples (i.e., positive) for scaffold contrastive
distancing, we propose to perturb only the terminal side chains of the
molecule. In other words, the molecule scaffold (i.e., core structure) is
preserved. This is done by first identifying the side chains and then
performing fragment replacement based on a candidate fragment pool. To avoid
significant alterations in molecule characteristics, we restrict the amount of
changes to be fewer than five atoms. For simplicity reason, we consider the
Bemis-Murcko framework as the scaffold. Figure 1c shows a sample perturbation
with scaffolds highlighted in blue. In this example, the benzene ring is the
identified scaffold, and either the carboxylic ester group or the carbonyl
group is perturbed by another functional groups. Note that such perturbation
is not limited to atom-level or bond-level editing, but also motif-level.

###  2.2 Molecular property prediction

To demonstrate the effectiveness of our approach, we first evaluate it on
seven challenging classification tasks from MoleculeNet [46], which is a
large-scale curated benchmark that covers multiple molecular property domains
(e.g., physiology, biophysics). The scaffold split scheme [19] is applied. The
performance is evaluated by the ROC-AUC value. Each experimental result is
averaged over three different runs following the prior works [42, 30, 12]. We
compare our method with seven competitive molecular representation learning
baselines, which cover a wide variety of pre-train SSL techniques and also
include the state-of-the-art on the same benchmark dataset. Specifically, we
first consider four contrastive SSL baselines: 1. GraphLoG [50] achieves a
hierarchical prototypical embedding space with the conventional graph
perturbation techniques; 2. D-SLA [21] proposes the discrepancy learning to
refine the embedding space with the conventional graph perturbation
techniques; 3. MolCLR [42] adapts the NT-Xent [6] contrastive loss with
molecule perturbation techniques, including atom/bond editing and subgraph
removal; 4. KANO [13] introduces the knowledge graph prompting techniques to
augment molecular graph, while the augmentations are also learned in the
contrastive manner. Besides, we include three predictive SSL baselines: 5. Hu
et. al [18] adopts masked attribute prediction and molecule subgraph
prediction; 6. GROVER [30] proposes a graph Transformer architecture and
applies context prediction learning; 7. GEM [12] utilizes the geometric
information of molecules and perform bond angle and atom distance prediction.
We also include the setting of training GNN from scratch (i.e., No-Pretrain).
Table 1 shows the performance comparison results. Our approach improves over
the No Pre-train setting by 12.4% ROC-AUC in average. When compared to the
other SSL methods, our approach reaches the new state-of-the-art performance
on BBBP, Clintox, and BACE datasets, while remaining highly competitive in the
rest of the tasks. Our overall ROC-AUC score is 1.8% higher than the second
best method (GEM).

###  2.3 Binding potency prediction

We consider MoleculeACE [40] as the second evaluation benchmark. It consists
of 30 datasets retrieved from ChEMBL [14]. Each dataset contains binding
potency measures (e.g., $K_{i}$ value) of molecules against a macromolecular
targets. These datasets mainly focus on the structure-property relationships
(SPR), where the phenomenon of activity cliffs is amplified. Activity cliffs
refer to the cases where small changes in molecular structure significantly
alters its biological activity, and understanding them is crucial for
optimizing lead compounds and designing new molecules with desired activities.
The phenomenon is also counter-intuitive in machine learning, as the machine
learning model tends to make similar predictions given similar inputs.
Stratified sampling is used in data splitting with the activity cliff label.
We take R-squared as the evaluation metrics since relative binding potency
ranking is more important than absolute prediction errors (e.g., RMSE) in the
real world.

Table 2 shows the performance comparison results. The performance variation of
No Pre-train indicate that these tasks cover different levels of difficulty,
regardless of size of the datasets. Our method outperforms the baselines by a
large margin in 24 out of 30 tasks. The largest performance gain occurs at the
dataset CHEMBL2034_Ki, where the R-squared value is improved by 35%.
Surprisingly, all baseline methods suffer a certain degree of negative
transfer. GraphLoG has negative transfer on 25 datasets, MolCLR on 17
datasets, and GROVER on 18 datasets. We argue that one of the reasons MolCLR
has less negative transfer than GraphLoG is because its sample perturbation
method is relatively more tailored for molecules. The results demonstrate that
the generic graph pertraining methods might be inefficient in modeling
challenging molecular tasks, which is consistent with the finding in [36]. The
convergence rate analysis is included in Supplementary Information.

Figure 2: Representation space probing. The dynamics of molecule
representation space of three methods at five fine-tune timestamps on dataset
CHEMBL237_Ki. For each row pair, a 2-D view of the representation space of
both the training (top) and validation set (bottom) are visualized. The dot
coloring represents the normalized potency labels. Rand index and roughness
index are reported for training, while R-squared value and cliff-noncliff
distance ratio are reported for validation. The red arrow indicates the
distance of a cliff molecule pair, while the blue arrow indicates that of a
non-cliff molecule pair. Figure 3: Representation shift decomposition.
Detailed analysis of the representation shift during fine-tuning across three
datasets. The dashed lines (top row) correspond to the representation shift
(approximated by the Rand Index) for each individual channel, while the solid
lines represent the overall method. The bottom row displays the optimized
prompt weights distributed over channels.

###  2.4 Representation robustness

To study the robustness of the learned molecule representation, we propose to
probe the fine-tuning process and evaluate the shift in the representation
space. Essentially, the shift captures how much the pre-trained chemical
knowledge is distorted during fine-tuning. We examine the representation space
of both the training and validation molecule set at five training timestamps.
To clarify, the term “learned representation” refers to the numerical
embedding generated after the atom aggregation, before it is fed to the
prediction layer. We choose CHEMBL237_Ki, one of the largest binding potency
prediction dataset in MoleculeACE, for the downstream target. We compare our
method with GraphLoG and MolCLR, as these models only differ by molecule SSL
strategies, while using the same GNN architecture and hyperparameters. In
Figure 2, each column represents a training timestamp, and each row pair
represents the mapping in training and validation molecule space by a specific
method. The coloring represents the normalized potency labels. We also report
four additional metrics that capture the representation characteristics along
the training process: 1. Roughness Index (i.e., ROGI) [2] captures the
landscape roughness of molecular property given a representation. Small ROGI
value indicates better modellability. 2. Rand Index [20] measures the
similarity between two clusterings. To study the amount of structural
information encoded in the representation, we compute the Rand index between
clusters formed using Morgan fingerprint (shown by the grey circle) and
clusters formed using the current representation. Higher Rand index means more
overlaps between clusters, and subsequently more structural information
encoded. The change in Rand index also suggests the degree of representation
shift. 3. Cliff-noncliff Distance Ratio indicates the generalizability of the
representation towards the activity cliff phenomenon. In short, cliff molecule
pairs (i.e., similar molecules with different labels) should be more distant
away compared to the non-cliff molecule pairs (i.e., similar molecules with
similar labels). This is computed on the validation set. We also pin-point
three molecules for illustrative purpose, with one cliff pair (indicated by
the red arrow) and one non-cliff pair (indicated by the blue arrow). 4. At
last, we report the validation R-squared for the out-of-sample performance.
Here are the three main takeaways from Figure 2:

  * •

With the prompt weight initialization, our composed representation yields the
lowest ROGI value to start with. In other words, our pre-trained knowledge can
be better transferedto the target application. This is also shown by our rapid
convergence rate in the validation set, reaching a validation R-squared of
0.676 at epoch 10.

  * •

Our representation preserves the structural information better than others,
leading to better robustness. While representations are optimized towards the
downstream property labels, the Rand index drops continuously for all methods.
It means that the encoded information gradually shifts from being structure-
oriented to label-oriented. However, our method has the lowest Rand index drop
of 0.128, compared to the drop of 0.141 by GraphLoG and 0.157 by MolCLR. The
visualization also shows that the representation space of MolCLR gradually
becomes label-centric (i.e., overfitting), while losing structural
relationship between molecules. This explains its low ROGI value and low
validation R-squared along the training process.

  * •

Our representation exhibits a better understanding of the concept of activity
cliffs. Our cliff-noncliff distance ratio in the validation set is always
positive and larger than that of GraphLoG and MolCLR. As illustrated by the
triplet samples, the red arrow is always larger than the blue arrow, while the
closeness of cliff and non-cliff pairs in space (i.e., structurally similar)
is being maintained. For MolCLR, even though the red arrow is much longer than
the blue one at some timestamps, the triplet points are distant away from each
other. It means that MolCLR fails to capture the structural similarity aspect
of the activity cliffs.

To further understand why our composed representation is more resilient to the
representation shift and can better preserve pre-trained knowledge, we present
more analysis on three diverse datasets, along with the individual channel-
wise representation behavior during fine-tuning. We choose CHEMBL237_Ki and
CHEMBL262_Ki as the representative regression-based datasets of different
scales, and BBBP as the typical classification-based dataset. The Rand index
of representation clustering difference between the initial and the current
timestamp is computed as a proxy for the representation shift. A smaller Rand
index indicates a larger shift. As illustrated in Figure 3, channels with the
highest prompt weights often exhibit the largest shift, and vice versa. This
is reasonable because these channels contribute more to the optimization
against the labels. Conversely, even though low-weighted channels contribute
less to the optimization, they are more likely to preserve the pre-trained
knowledge. As a result, the composed representation derived from channel
aggregation exhibits a certain level of resilience to representation shifts,
making it potentially more robust than other methods. We also include a few-
shot representation probing analysis in Supplementary Information, which
demonstrates similar patterns.

Figure 4: Activity cliffs analysis. Evaluate the bioactivity binding mode and
the atom importance captured by our model on a (a)
5-phenyl-4-phenyldiazenyl-1,2-dihydropyrazol-3-one series and a (b)
N-phenyl-4-pyrazolo[1,5-b]pyridazin-3-ylpyrimidin-2-amine series. The binding
modes of compound $a1$ and compound $b1$ in the active sites of GSK3$\beta$
(PDB ID: 3L1S) are predicted by AutoDock Vina and visualized by PyMOL [7]. The
key hydrogen bonds between compounds and the active sites are highlighted by
yellow dash lines, while the green dash line refers to the intra-molecular
halogen bond. The color intensity on each atom indicates its respective
contribution to the prediction of our method, computed from a GNNExplainer.

###  2.5 Activity cliffs analysis

We present a deeper analysis into the potential of our method in understanding
the activity cliffs. To be more specific, we evaluate the relationship between
the model explanations generated by the GNNExplainer [51] and the predicted
binding mode between the ligands and the protein pockets by AutoDock Vina
[38]. Note that this analysis merely serves as a proof of concept, such that
our representation has the potential of capturing influential and well-
established factors in binding affinities. However, the fundamental limitation
of utilizing topological information only is unavoidable, which we will
discuss in Conclusion.

As shown in Figure 4, two series of compounds sharing the same scaffolds are
potential inhibitors of glycogen synthase kinase-3 beta (GSK3$\beta$). The
molecule activity cliff pairs, determined by the formulation in [40], are
compounds <$a1,a2$>, <$a1,a3$>, <$b1,b2$>, and <$b1,b3$>. The explanations of
both our and MolCLR’s predictions are compared. In Figure 4a, the potential
intra-molecular halogen-bonding contact between the chlorine atom and
hydrazone in compound $a1$, as indicated by the green dashed line, is
disfavored for the inter-molecular hydrogen-bonding contact between the
backbone carbonyl of the active site VAL-135 and hydrazone of the compound
[3]. As shown by our model explanation, compared to the compound $a2$ and
$a3$, the chlorine atom of compound $a1$, along with its associated benzene
ring, contribute less to the overall binding affinity prediction. It aligns
well with the predicted binding mode.

The predicted binding mode in Figure 4b shows that the orientation of the
substituent (i.e., the alkoxy group in compound $b1$ at the 6 position of
pyrazolo[1,5-b]pyridazine) will cause steric clash with PHE-67 in the G-rich
loop of GSK3$\beta$, thereby leading to a loss of potency [37, 34]. In
contrast, the alkoxy groups at the 3’ and 5’ positions have no clear contact
with the protein pocket. Remarkably, using only the topological information,
our model can capture the importance of the influential alkoxy group at 6
position. In addition, the absence of the alkoxy group at 5’ position (from
compound $b2$ to $b3$) does not affect the overall atoms’ contribution to the
predicted potency. MolCLR performs equally good in terms of compound $b2$ and
$b3$, but it fails to capture the influential alkoxy group in compound $b1$.

##  3 Discussion

In this work, we propose a novel multi-channel learning framework for molecule
representation learning. Each channel is dedicated to learning one self-
supervised learning task, with varying levels of focus on the global and local
aspects of the molecule. Specifically, the molecule contrastive distancing
captures the molecule’s global similarity, the scaffold contrastive distancing
highlights the foundational role of the scaffold in affecting molecular
characteristics, and the context prediction targets the composition of
functional groups. Further analysis shows that representations learned from
these channels form an implicit feature hierarchy. When fine-tuned on
downstream tasks, the model is able to identify which channel-wise
representation is most relevant to the current application, thereby making the
composite representation context-dependent. It mitigates the learning gap
between pre-training and fine-tuning objectives. In addition, the composite
formulation endows our representation with enhanced robustness and
generalizability, shown by its resilience to representation shifts and label
overfitting.

There are several interesting directions for future research. One of the
potential improvements is to consider more the interaction between channels.
Currently, the GNN’s message-passing module is shared across channels, whereas
there is no explicit guidance on inter-channel correlations. For instance, is
it possible to explicitly formulate a representation hierarchy during pre-
training? Another promising direction is to incorporate 3D geometric
information into the framework. By merely leveraging topological molecular
structure, the model is unable to differentiate molecules with different
conformations (e.g., functional groups’ orientation or atom’s chirality),
which are fundamental in bioactivity prediction. Besides, there exist other
advanced data-driven techniques for studying the structural-activity
relationship (SAR) that might be compatible with our framework. For example,
Molecular Anatomy [24] argues that the network clustering from scaffold
fragmentation and abstraction allows high quality structure-activity
relationships (SAR) analysis. Such investigations aim to transfer knowledge
from cheminformatics to machine learning models, potentially improving both
model interpretability and robustness. More importantly, while our method has
immediate implications for drug discovery, its molecular representation
robustness further shed lights on its promising potential in other sub-fields
of chemistry, such as materials science and environmental chemistry.

##  4 Methods

#### Graph neural networks (GNNs)

A graph $G=(V,E)$ is defined by a set of nodes $V$ and edges $E$. In the
molecular graph, each node denotes an atom, and the edge denotes the chemical
bond. Let $h_{v}$ be the representation of node $v$, and $h_{G}$ be the
representation of the graph $G$. Modern GNNs follow the message-passing
framework, such that node representations are updated iteratively via
neighborhood aggregation:

$\displaystyle
h_{v}^{k}=\operatorname{UPDATE}\left(h_{v}^{k-1},\operatorname{AGGREGATE}\left(\left\\{h_{v}^{k-1},h_{u}^{k-1},e_{uv}\right\\}:\forall
u\in N(v)\right)\right)\text{ ,}$

where $N(v)$ is the neighborhood of node $v$, $k$ denotes the layer index in a
multi-layer GNN structure, and $e_{uv}$ denotes the edge connecting two nodes
$u$ and $v$. The initialization of $h_{v}^{0}$ comes from the predefined node
features $x_{v}$. The aggregate function integrates neighborhood information
into the current node representation. The update function takes the updated
node representation and the node representation from the previous $k-1$ layer
and performs operations like concatenation or summation. After the iterative
updates, a permutation-invariant pooling operation is performed to get the
representation for the entire graph $G$:
$h_{g}=\operatorname{READOUT}(h_{v}^{k}|v\in V)$. There are various number of
options for the readout operation, including simple operations of mean and
max, and advanced differentiable approaches like DiffPool [52] and GMT [4].

#### Prompt-guided aggregation

Instead of using the readout operations mentioned above, we adopt the prompt-
guided aggregation, which is achieved using the multi-head attention. As shown
in Figure 1e, the embedding of the prompt token $h_{p}$ is treated as the
attention query, while the representations $h_{x}$ of nodes/atoms $x$, are
viewed as the keys and values. The resulted prompt-aware graph representation
$h_{g}^{p}=\sum_{x}\alpha_{x}h_{x}$, where the attention weight for each node
$x$ is computed as $\alpha_{x}=\text{softmax}(\\{q\cdot
k_{x}/\sqrt{d_{k}}\\}_{x})$, and $q=W_{q}h_{p}$, $k_{x}=W_{k}h_{x}$, and
$v_{x}=W_{v}h_{x}$. Here $\\{W_{q},W_{k},W_{v}\\}$ represent parameters in the
linear transformations, and $\sqrt{d_{k}}$ is the scaling factor. Essentially,
the prompt-aware $h_{g}^{p}$ is aggregated from the weighted average of
$h_{x}$.

#### Adaptive margin contrastive loss

Contrastive learning is a technique widely used in self-supervised learning,
which aims to group semantic similar samples closer while pushing dissimilar
samples distant apart in the latent representation space. In this work, we
adopt the triplet loss [31] to formulate the contrastive learning. It
considers triplets of data samples: the anchor $G_{i}$, the positive (i.e.,
semantic-similar) sample $G_{i}^{\prime}$, and the negative (i.e., semantic-
dissimilar) sample $G_{j}$. Its formulation is shown below.

$\displaystyle\ell_{i,j(\neq
i)}=\max\big{(}0,\alpha+d(\bm{h}_{g_{i}},\bm{h}_{g_{i}}^{\prime})-d(\bm{h}_{g_{i}},\bm{h}_{g_{j}})\big{)}\text{
,}$ (1)

where $\bm{h}_{g_{i}}$ denotes the latent representation for sample $G_{i}$,
and $\bm{h}_{g_{i}}^{\prime}$ being the representation for its augmentation.
The function $d(\cdot,\cdot)$ measures the L2 distance between two vectors. In
general, this objective enforces the pair-wise distancing difference between
<$G_{i}$, $G_{i}^{\prime}$> and <$G_{i}$, $G_{j}$> to be at least $\alpha$
margin. However, as we discuss before, this formulation can only lead to a
coarse-grained representation space, while neglecting the existing structural
relationship among molecules (e.g., shared rings or functional groups). Also,
this formulation does not pose any constraints on the representation space
beyond the margin (see Supplementary Information for example illustration).
Therefore, we propose to add an additional contrastive formulation with
adaptive margin among negative samples and the anchor.

$\displaystyle\ell_{i,j(\neq i),k(\neq j\neq i)}=$
$\displaystyle\max\big{(}0,\alpha_{1}(G_{i},G_{j})+d(\bm{h}_{g_{i}},\bm{h}_{g_{i}}^{\prime})-d(\bm{h}_{g_{i}},\bm{h}_{g_{j}})\big{)}+$
$\displaystyle\max\big{(}0,\alpha_{1}({G_{i}},{G_{k}})+d(\bm{h}_{g_{i}},\bm{h}_{g_{i}}^{\prime})-d(\bm{h}_{g_{i}},\bm{h}_{g_{k}})\big{)}+$
$\displaystyle\max\big{(}0,\alpha_{2}({G_{i}},{G_{j}},{G_{k}})+d(\bm{h}_{g_{i}},\bm{h}_{g_{j}})-d(\bm{h}_{g_{i}},\bm{h}_{g_{k}})\big{)}\text{
,}$ (2)

where $\alpha_{1}(\cdot)$ and $\alpha_{2}(\cdot)$ are the adaptive margin
functions. Let $\bm{z}_{g_{i}}$ be the conventional structural features of the
sample $G_{i}$. We use the Morgan fingerprint [25], which hashes circular atom
neighborhoods into fixed-length binary strings, to represent the structural
features. In molecule contrastive distancing, the adaptive function
$\alpha_{1}(G_{i},G_{j})=\alpha_{\text{offset}}\times(1-\text{sim}(\bm{z}_{g_{i}},\bm{z}_{g_{j}}))$,
and
$\alpha_{2}(G_{i},G_{j},G_{k})=\alpha_{\text{offset}}\times(\text{sim}(\bm{z}_{g_{i}},\bm{z}_{g_{j}})-\text{sim}(\bm{z}_{g_{i}},\bm{z}_{g_{k}}))$,
where $\text{sim}(\cdot,\cdot)$ denotes the Tanimoto similarity. The scaffold
contrastive distancing has the same formulation, except that the molecule
sample $G$ is replaced by its scaffold $s(G)$. Note that the formulation now
considers quadruplet of data samples <$G_{i}$, $G_{i}^{\prime}$, $G_{j(\neq
i)}$, $G_{k(\neq j\neq i)}$>. Even though the theoretical complexity is
increased from $O(N^{2}K)$ to $O(N^{3})$, we can perform fixed-size random
sampling with respect to the computed similarity differences in
$\alpha_{2}(\cdot)$. Quadruplets are also dropped if the computed values are
negative. Eventually, the optimization goal is to minimize the loss summation
as $\min\sum_{i,j,k}\ell_{i,j(\neq i),k(\neq j\neq i)}$.

#### Prompt-guided multi-channel learning

The overall multi-channel learning framework is inspired by the work in [43].
At the pre-train stage, each channel is responsible for learning distinct SSL
tasks that focus on a unique aspect of the molecule. At the fine-tune stage,
the model parameters learned via pre-training are first used to initialize the
fine-tune model. Besides, we introduce an additional prompt selection module
to combine representations from different channels into the task-specific
(i.e., context-dependent) composite representation. Essentially, it learns the
relevance between different pre-trained molecular knowledge and the downstream
application, hence bridging the gap between pre-training and fine-tuning
objectives. Through experiments, we find that directly optimizing the pre-
trained multi-channel model with randomly initialized prompt weights could
easily reach local optimum. In other words, the downstream performance is
sensitive to the random initialization of selected prompts. Therefore, we
propose a prompt weight initialization trick, such that the initial prompt
weights should lead to a composite representation with the smoothest
quantitative structure-property landscape (i.e., low ROGI value) [2] with
respect to the current application. ROGI measures the roughness of the
molecule property landscape given a molecular numericrepresentation. A low
ROGI value indicates better modellability. We use a simple Bayesian
optimization pipeline to find the best initialization with the lowest ROGI
value on the training set. Essentially, the input parameters of the Bayesian
optimization are $k-1$ learnable scalers, where $k$ is the number of channels.
The black-box utility function first computes the distribution over $k$ values
by treating the input scalars as logits, and calculates the composite
representation as well as its corresponding roughness index. We utilize the
Quasi MC-based batch Expected Improvement as the acquisition function. After
the initialization, the entire model, except for the prompt-guided aggregation
modules, is optimized towards the molecular property labels. The parameters
within the prompt-guided aggregation modules are fixed because these layers
are set to perform application-agnostic pooling operations over the molecular
graph. Hypothetically, the aggregation module responsible for scaffold
contrastive distancing should allocate greater attention weight on atoms
constituting the scaffold.

#### Experimental setup

We pre-train our framework using the molecules from ZINC15 [33], which is an
open-sourced database that contains 2 million unlabelled drug-like compounds.
We use CReM [27], an open-sourced molecule mutation framework, for the
scaffold-invariant molecule perturbation. To be more specific, we first use
RDKit [1] to identify the Bemis-Murcko scaffold of molecules, as well as the
connection sites (i.e., atom indices) between the scaffold and the side
chains. The CReM algorithm then takes these indices and performs fragment
replacement using an external fragment pool. Perturbation results are further
filtered by chemical validity and the maximum number of changed atoms allowed.
At last, we are able to successfully compute perturbations for 1,882,537 out
of 2 million molecules. These molecules form our final pre-train database. In
the contrastive-based pre-training, we consider five positive samples for each
anchor molecule. In molecule distancing, we randomly apply subgraph masking
five times on the original molecule. In scaffold distancing, we randomly
sample five scaffold-invariant perturbations. In terms of the model
architecture, we follow the same architecture setup as the graph neural
network in [50, 18, 42, 21]. We use a 5-layer Graph Isomorphism Network (GIN)
[49] as the backbone, whose hidden dimension size is set to 300. For the
MoleculeNet benchmark, we utilize the same molecule feature sets as works in
[30, 13], while we consider the same molecule feature sets as works in [18,
21, 42, 50] for the MoleculeACE benchmark.

#### Baselines

We replicate the performance of MolCLR, GROVER, GEM, and KANO on the
MoleculeNet benchmark using their provided checkpoints and code repositories.
Note that the published reported results of GROVER and KANO are evaluated
under the balanced scaffold split, which is different than the deterministic
scaffold split used in this work and the rest of the baseline methods. For the
MoleculeACE benchmark, we again fine-tune GraphLoG, MolCLR, and GROVER using
their repositories and model checkpoints.

#### Representation space probing

We propose to analyze the dynamics of representation space during fine-tuning
to evaluate the representation robustness and generalizability. To be more
specific, we probe the mapping of the representation space at five different
training timestamps (epoch 0, epoch 10, epoch 20, epoch 50, epoch 100). The
2-D view is constructed via the T-SNE [39] dimension reduction technique.
Besides the visualization, we also report four additional metrics that capture
the representation characteristics. We report the ROGI value and Rand index in
the training set, plus the R-squared value and cliff-noncliff distance ratio
in the validation set. The Rand index is calculated from two clustering
assignments. The first clustering is formed using KMeans clustering with the
molecule Morgan fingerprint (radius=2, nBits=512). The second clustering is
formed with using KMeans clustering with the representation at the current
timestamp. We set the same number of clusters for these two clusterings. In
terms of the cliff-noncliff distance ratio, we first identify the cliff and
the noncliff molecule pairs using the same principle in MoleculeACE [40].
Then, we compute and average the representation distance of each pair. At
last, the cliff-noncliff ratio is formed by normalizing their average
distance.

Table 1: Fine-tuning results (ROC-AUC) on 7 classification tasks in
MoleculeNet. The results of MolCLR, GROVER, GEM, and KANO are taken from
rerunning their code repositories using the exact scaffold split.

Methods | BBBP | Clintox | MUV | HIV | BACE | Tox21 | SIDER | Avg.
---|---|---|---|---|---|---|---|---
No Pre-train | 65.8 $\pm$ 4.5 | 58.0 $\pm$ 4.4 | 71.8 $\pm$ 2.5 | 75.3 $\pm$ 1.9 | 70.1 $\pm$ 5.4 | 74.0 $\pm$ 0.8 | 57.3 $\pm$ 1.6 | 67.5
Hu et. al [18] | 70.8 $\pm$ 1.5 | 72.6 $\pm$ 1.5 | 81.3 $\pm$ 2.1 | 79.9 $\pm$ 0.7 | 84.5 $\pm$ 0.7 | 78.7 $\pm$ 0.4 | 62.7 $\pm$ 0.8 | 75.8
GraphLoG [50] | 72.3 $\pm$ 0.9 | 74.7 $\pm$ 2.2 | 74.2 $\pm$ 1.8 | 75.4 $\pm$ 0.6 | 82.2 $\pm$ 0.9 | 75.1 $\pm$ 0.7 | 61.2 $\pm$ 1.1 | 73.6
D-SLA [21] | 72.6 $\pm$ 0.8 | 80.2 $\pm$ 1.5 | 76.6 $\pm$ 0.9 | 78.6 $\pm$ 0.4 | 83.8 $\pm$ 1.0 | 76.8 $\pm$ 0.5 | 60.2 $\pm$ 1.1 | 75.5
MolCLR [42] | 73.5 $\pm$ 0.4 | 90.4 $\pm$ 1.7 | 75.5 $\pm$ 1.8 | 77.6 $\pm$ 3.2 | 83.5 $\pm$ 1.8 | 76.7 $\pm$ 2.1 | 60.7 $\pm$ 5.7 | 76.8
GROVER [30] | 69.5 $\pm$ 0.1 | 76.2 $\pm$ 3.7 | 67.3 $\pm$ 1.8 | 68.2 $\pm$ 1.1 | 81.0 $\pm$ 1.4 | 73.5 $\pm$ 0.1 | 65.4 $\pm$ 0.1 | 71.6
GEM [12] | 71.8 $\pm$ 0.6 | 89.7 $\pm$ 2.0 | 77.0 $\pm$ 1.5 | 78.0 $\pm$ 1.4 | 84.9 $\pm$ 1.1 | 78.2 $\pm$ 0.3 | 67.2 $\pm$ 0.6 | 78.1
KANO [13] | 69.9 $\pm$ 1.9 | 90.7 $\pm$ 2.2 | 74.7 $\pm$ 2.0 | 75.7 $\pm$ 0.3 | 82.7 $\pm$ 0.9 | 75.8 $\pm$ 0.5 | 60.2 $\pm$ 1.4 | 75.7
Ours | 74.1 $\pm$ 0.6 | 95.7 $\pm$ 1.2 | 81.2 $\pm$ 0.5 | 79.8 $\pm$ 0.3 | 85.0 $\pm$ 1.1 | 77.5 $\pm$ 0.3 | 66.7 $\pm$ 0.8 | 80.1
|  |  |  |  |  |  |  |

Table 2: Fine-tuning results (R-Squared) on 30 binding potency prediction
tasks in MoleculeACE.

Dataset | Size |  | SSL methods
---|---|---|---
| No Pre-train | GraphLoG | MolCLR | GROVER | Ours
CHEMBL4203_Ki | 731 |  | 0.09 $\pm$ 0.12 | 0.05 $\pm$ 0.02 | 0.09 $\pm$ 0.05 | 0.16 $\pm$ 0.03 | 0.24 $\pm$ 0.05
CHEMBL262_Ki | 856 |  | 0.20 $\pm$ 0.1 | 0.34 $\pm$ 0.04 | 0.45 $\pm$ 0.02 | 0.40 $\pm$ 0.07 | 0.50 $\pm$ 0.04
CHEMBL2034_Ki | 750 |  | 0.30 $\pm$ 0.06 | 0.56 $\pm$ 0.03 | 0.40 $\pm$ 0.1 | 0.52 $\pm$ 0.02 | 0.65 $\pm$ 0.01
CHEMBL218_EC50 | 1031 |  | 0.35 $\pm$ 0.06 | 0.34 $\pm$ 0.03 | 0.37 $\pm$ 0.04 | 0.48 $\pm$ 0.07 | 0.35 $\pm$ 0.03
CHEMBL3979_EC50 | 1125 |  | 0.36 $\pm$ 0.04 | 0.29 $\pm$ 0.03 | 0.35 $\pm$ 0.07 | 0.34 $\pm$ 0.05 | 0.39 $\pm$ 0.03
CHEMBL2047_EC50 | 631 |  | 0.41 $\pm$ 0.08 | 0.52 $\pm$ 0.02 | 0.44 $\pm$ 0.09 | 0.59 $\pm$ 0.02 | 0.63 $\pm$ 0.04
CHEMBL1871_Ki | 659 |  | 0.42 $\pm$ 0.05 | 0.33 $\pm$ 0.03 | 0.41 $\pm$ 0.09 | 0.42 $\pm$ 0.06 | 0.49 $\pm$ 0.04
CHEMBL4616_EC50 | 682 |  | 0.42 $\pm$ 0.06 | 0.49 $\pm$ 0.03 | 0.48 $\pm$ 0.06 | 0.44 $\pm$ 0.1 | 0.52 $\pm$ 0.02
CHEMBL219_Ki | 1859 |  | 0.44 $\pm$ 0.04 | 0.40 $\pm$ 0.04 | 0.37 $\pm$ 0.04 | 0.32 $\pm$ 0.05 | 0.53 $\pm$ 0.03
CHEMBL287_Ki | 1328 |  | 0.46 $\pm$ 0.03 | 0.21 $\pm$ 0.04 | 0.42 $\pm$ 0.05 | 0.39 $\pm$ 0.02 | 0.49 $\pm$ 0.02
CHEMBL239_EC50 | 1721 |  | 0.47 $\pm$ 0.06 | 0.41 $\pm$ 0.04 | 0.47 $\pm$ 0.07 | 0.52 $\pm$ 0.02 | 0.54 $\pm$ 0.03
CHEMBL235_EC50 | 2349 |  | 0.52 $\pm$ 0.02 | 0.51 $\pm$ 0.04 | 0.48 $\pm$ 0.03 | 0.40 $\pm$ 0.1 | 0.54 $\pm$ 0.01
CHEMBL231_Ki | 973 |  | 0.58 $\pm$ 0.02 | 0.54 $\pm$ 0.03 | 0.57 $\pm$ 0.03 | 0.64 $\pm$ 0.02 | 0.65 $\pm$ 0.02
CHEMBL237_EC50 | 955 |  | 0.60 $\pm$ 0.05 | 0.51 $\pm$ 0.03 | 0.53 $\pm$ 0.07 | 0.52 $\pm$ 0.11 | 0.63 $\pm$ 0.02
CHEMBL238_Ki | 1052 |  | 0.62 $\pm$ 0.04 | 0.56 $\pm$ 0.02 | 0.63 $\pm$ 0.06 | 0.62 $\pm$ 0.01 | 0.65 $\pm$ 0.02
CHEMBL233_Ki | 3142 |  | 0.63 $\pm$ 0.01 | 0.59 $\pm$ 0.02 | 0.64 $\pm$ 0.02 | 0.50 $\pm$ 0.09 | 0.67 $\pm$ 0.01
CHEMBL214_Ki | 3317 |  | 0.63 $\pm$ 0.01 | 0.58 $\pm$ 0.02 | 0.60 $\pm$ 0.02 | 0.45 $\pm$ 0.09 | 0.65 $\pm$ 0.01
CHEMBL228_Ki | 1704 |  | 0.63 $\pm$ 0.02 | 0.52 $\pm$ 0.03 | 0.58 $\pm$ 0.03 | 0.46 $\pm$ 0.11 | 0.62 $\pm$ 0.01
CHEMBL4005_Ki | 960 |  | 0.64 $\pm$ 0.03 | 0.57 $\pm$ 0.01 | 0.44 $\pm$ 0.06 | 0.58 $\pm$ 0.07 | 0.63 $\pm$ 0.02
CHEMBL4792_Ki | 1471 |  | 0.65 $\pm$ 0.02 | 0.55 $\pm$ 0.03 | 0.62 $\pm$ 0.04 | 0.55 $\pm$ 0.09 | 0.66 $\pm$ 0.02
CHEMBL264_Ki | 2862 |  | 0.65 $\pm$ 0.03 | 0.60 $\pm$ 0.02 | 0.62 $\pm$ 0.03 | 0.41 $\pm$ 0.04 | 0.61 $\pm$ 0.01
CHEMBL237_Ki | 2602 |  | 0.66 $\pm$ 0.01 | 0.67 $\pm$ 0.02 | 0.67 $\pm$ 0.02 | 0.60 $\pm$ 0.06 | 0.72 $\pm$ 0.02
CHEMBL234_Ki | 3657 |  | 0.66 $\pm$ 0.02 | 0.60 $\pm$ 0.01 | 0.62 $\pm$ 0.04 | 0.49 $\pm$ 0.01 | 0.70 $\pm$ 0.01
CHEMBL236_Ki | 2598 |  | 0.70 $\pm$ 0.02 | 0.65 $\pm$ 0.02 | 0.68 $\pm$ 0.02 | 0.52 $\pm$ 0.09 | 0.71 $\pm$ 0.01
CHEMBL1862_Ki | 794 |  | 0.71 $\pm$ 0.03 | 0.69 $\pm$ 0.03 | 0.76 $\pm$ 0.03 | 0.75 $\pm$ 0.01 | 0.72 $\pm$ 0.02
CHEMBL2971_Ki | 976 |  | 0.72 $\pm$ 0.01 | 0.68 $\pm$ 0.02 | 0.76 $\pm$ 0.06 | 0.78 $\pm$ 0.01 | 0.76 $\pm$ 0.03
CHEMBL204_Ki | 2754 |  | 0.76 $\pm$ 0.02 | 0.71 $\pm$ 0.02 | 0.72 $\pm$ 0.02 | 0.66 $\pm$ 0.01 | 0.76 $\pm$ 0.01
CHEMBL2835_Ki | 615 |  | 0.78 $\pm$ 0.03 | 0.69 $\pm$ 0.04 | 0.75 $\pm$ 0.04 | 0.70 $\pm$ 0.03 | 0.79 $\pm$ 0.01
CHEMBL244_Ki | 3097 |  | 0.80 $\pm$ 0.01 | 0.78 $\pm$ 0.01 | 0.80 $\pm$ 0.02 | 0.63 $\pm$ 0.06 | 0.80 $\pm$ 0.02
CHEMBL2147_Ki | 1456 |  | 0.88 $\pm$ 0.01 | 0.82 $\pm$ 0.01 | 0.87 $\pm$ 0.02 | 0.85 $\pm$ 0.01 | 0.88 $\pm$ 0.01
Average |  |  | 0.558 | 0.525 | 0.553 | 0.523 | 0.616
#Negative transfer |  |  | - | 25 | 17 | 18 | 3
|  |  |  |  |  |  |

## References

  * [1] Rdkit: Open-source cheminformatics.
  * [2] Aldeghi, M., Graff, D. E., Frey, N., Morrone, J. A., Pyzer-Knapp, E. O., Jordan, K. E., and Coley, C. W. Roughness of Molecular Property Landscapes and Its Impact on Modellability.  Journal of Chemical Information and Modeling 62, 19 (Oct. 2022), 4660–4671.
  * [3] Arnost, M., Pierce, A., ter Haar, E., Lauffer, D., Madden, J., Tanner, K., and Green, J. 3-aryl-4-(arylhydrazono)-1h-pyrazol-5-ones: Highly ligand efficient and potent inhibitors of gsk3$\beta$.  Bioorganic & medicinal chemistry letters 20, 5 (2010), 1661–1664.
  * [4] Baek, J., Kang, M., and Hwang, S. J. Accurate learning of graph representations with graph multiset pooling.  In International Conference on Learning Representations (2021).
  * [5] Böhm, H.-J., Flohr, A., and Stahl, M. Scaffold hopping.  Drug Discov Today Technol 1, 3 (Dec. 2004), 217–224.
  * [6] Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. E. A simple framework for contrastive learning of visual representations.  CoRR abs/2002.05709 (2020).
  * [7] DeLano, W. L., et al. Pymol: An open-source molecular graphics tool.  CCP4 Newsl. Protein Crystallogr 40, 1 (2002), 82–92.
  * [8] Deng, J., Socher, R., Fei-Fei, L., Dong, W., Li, K., and Li, L.-J. Imagenet: A large-scale hierarchical image database.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) (06 2009), vol. 00, pp. 248–255.
  * [9] Deng, J., Yang, Z., Wang, H., Ojima, I., Samaras, D., and Wang, F. A systematic study of key elements underlying molecular property prediction.  Nature Communications 14, 1 (Oct 2023), 6395.
  * [10] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805 (2018).
  * [11] Dowden, H., and Munro, J. Trends in clinical success rates and therapeutic focus.  Nat Rev Drug Discov 18, 7 (July 2019), 495–496.
  * [12] Fang, X., Liu, L., Lei, J., He, D., Zhang, S., Zhou, J., Wang, F., Wu, H., and Wang, H. Geometry-enhanced molecular representation learning for property prediction.  Nature Machine Intelligence 4, 2 (Feb. 2022), 127–134.
  * [13] Fang, Y., Zhang, Q., Zhang, N., Chen, Z., Zhuang, X., Shao, X., Fan, X., and Chen, H. Knowledge graph-enhanced molecular contrastive learning with functional prompt.  Nature Machine Intelligence (May 2023).
  * [14] Gaulton, A., Bellis, L. J., Bento, A. P., Chambers, J., Davies, M., Hersey, A., Light, Y., McGlinchey, S., Michalovich, D., Al-Lazikani, B., and Overington, J. P. ChEMBL: a large-scale bioactivity database for drug discovery.  Nucleic Acids Research 40, D1 (Jan. 2012), D1100–D1107.
  * [15] Gentile, F., Yaacoub, J. C., Gleave, J., Fernandez, M., Ton, A.-T., Ban, F., Stern, A., and Cherkasov, A. Artificial intelligence–enabled virtual screening of ultra-large chemical libraries with deep docking.  Nature Protocols 17, 3 (Mar 2022), 672–697.
  * [16] Goh, G. B., Hodas, N. O., and Vishnu, A. Deep learning for computational chemistry, 2017.
  * [17] Hay, M., Thomas, D. W., Craighead, J. L., Economides, C., and Rosenthal, J. Clinical development success rates for investigational drugs.  Nat Biotechnol 32, 1 (Jan. 2014), 40–51.
  * [18] Hu*, W., Liu*, B., Gomes, J., Zitnik, M., Liang, P., Pande, V., and Leskovec, J. Strategies for pre-training graph neural networks.  In International Conference on Learning Representations (2020).
  * [19] Hu, W., Liu, B., Gomes, J., Zitnik, M., Liang, P., Pande, V. S., and Leskovec, J. Pre-training graph neural networks.  CoRR abs/1905.12265 (2019).
  * [20] Hubert, L., and Arabie, P. Comparing partitions.  Journal of Classification 2, 1 (Dec 1985), 193–218.
  * [21] Kim, D., Baek, J., and Hwang, S. J. Graph self-supervised learning with accurate discrepancy learning.  CoRR abs/2202.02989 (2022).
  * [22] Kim, D., and Oh, A. How to find your friendly neighborhood: Graph attention design with self-supervision.  In International Conference on Learning Representations (2021).
  * [23] Liu, Z., Yu, X., Fang, Y., and Zhang, X. GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks, Feb. 2023.  arXiv:2302.08043 [cs].
  * [24] Manelfi, C., Gemei, M., Talarico, C., Cerchia, C., Fava, A., Lunghini, F., and Beccari, A. R. “Molecular Anatomy”: a new multi-dimensional hierarchical scaffold analysis tool.  Journal of Cheminformatics 13, 1 (Dec. 2021), 54.
  * [25] Morgan, H. L. The generation of a unique machine description for chemical structures-a technique developed at chemical abstracts service.  Journal of Chemical Documentation 5, 2 (1965), 107–113.
  * [26] Pérez-Villanueva, J., Méndez-Lucio, O., Soria-Arteche, O., and Medina-Franco, J. L. Activity cliffs and activity cliff generators based on chemotype-related activity landscapes.  Mol Divers 19, 4 (July 2015), 1021–1035.
  * [27] Polishchuk, P. CReM: chemically reasonable mutations framework for structure generation.  Journal of Cheminformatics 12, 1 (Apr. 2020), 28.
  * [28] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners.  OpenAI blog 1, 8 (2019), 9.
  * [29] Ramsundar, B., Eastman, P., Walters, P., Pande, V., Leswing, K., and Wu, Z. Deep Learning for the Life Sciences.  O’Reilly Media, 2019.  https://www.amazon.com/Deep-Learning-Life-Sciences-Microscopy/dp/1492039837.
  * [30] Rong, Y., Bian, Y., Xu, T., Xie, W., Wei, Y., Huang, W., and Huang, J. Self-Supervised Graph Transformer on Large-Scale Molecular Data.
  * [31] Schroff, F., Kalenichenko, D., and Philbin, J. FaceNet: A unified embedding for face recognition and clustering.  In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Boston, MA, USA, June 2015), IEEE, pp. 815–823.
  * [32] Sheridan, R. P., Karnachi, P., Tudor, M., Xu, Y., Liaw, A., Shah, F., Cheng, A. C., Joshi, E., Glick, M., and Alvarez, J. Experimental error, kurtosis, activity cliffs, and methodology: What limits the predictivity of quantitative Structure-Activity relationship models?  J Chem Inf Model 60, 4 (Apr. 2020), 1969–1982.
  * [33] Sterling, T., and Irwin, J. J. Zinc 15 – ligand discovery for everyone.  Journal of Chemical Information and Modeling 55, 11 (2015), 2324–2337.  PMID: 26479676.
  * [34] Stevens, K. L., Reno, M. J., Alberti, J. B., Price, D. J., Kane-Carson, L. S., Knick, V. B., Shewchuk, L. M., Hassell, A. M., Veal, J. M., Davis, S. T., et al. Synthesis and evaluation of pyrazolo [1, 5-b] pyridazines as selective cyclin dependent kinase inhibitors.  Bioorganic & medicinal chemistry letters 18, 21 (2008), 5758–5762.
  * [35] Stumpfe, D., Hu, Y., Dimova, D., and Bajorath, J. Recent progress in understanding activity cliffs and their utility in medicinal chemistry.  Journal of Medicinal Chemistry 57, 1 (2014), 18–28.  PMID: 23981118.
  * [36] Sun, R., Dai, H., and Yu, A. W. Does gnn pretraining help molecular representation?  In Advances in Neural Information Processing Systems (2022), S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35, Curran Associates, Inc., pp. 12096–12109.
  * [37] Tavares, F. X., Boucheron, J. A., Dickerson, S. H., Griffin, R. J., Preugschat, F., Thomson, S. A., Wang, T. Y., and Zhou, H.-Q. N-phenyl-4-pyrazolo [1, 5-b] pyridazin-3-ylpyrimidin-2-amines as potent and selective inhibitors of glycogen synthase kinase 3 with good cellular efficacy.  Journal of medicinal chemistry 47, 19 (2004), 4716–4730.
  * [38] Trott, O., and Olson, A. J. Autodock vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading.  Journal of computational chemistry 31, 2 (2010), 455–461.
  * [39] van der Maaten, L., and Hinton, G. Visualizing data using t-sne.  Journal of Machine Learning Research 9, 86 (2008), 2579–2605.
  * [40] van Tilborg, D., Alenicheva, A., and Grisoni, F. Exposing the limitations of molecular machine learning with activity cliffs.  Journal of Chemical Information and Modeling 62, 23 (dec 2022), 5938–5951.
  * [41] Veličković, P., Fedus, W., Hamilton, W. L., Liò, P., Bengio, Y., and Hjelm, R. D. Deep graph infomax.  In International Conference on Learning Representations (2019).
  * [42] Wang, Y., Wang, J., Cao, Z., and Barati Farimani, A. Molecular contrastive learning of representations via graph neural networks.  Nature Machine Intelligence 4, 3 (Mar. 2022), 279–287.
  * [43] Wang, Z., Zhang, Q., HU, S.-W., Yu, H., Jin, X., Gong, Z., and Chen, H. Multi-level Protein Structure Pre-training via Prompt Learning.  In The Eleventh International Conference on Learning Representations (2023).
  * [44] Weininger, D. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules.  Journal of Chemical Information and Computer Sciences 28, 1 (1988), 31–36.
  * [45] Wu, F., Qin, H., Li, S., Li, S. Z., Zhan, X., and Xu, J. Instructbio: A large-scale semi-supervised learning paradigm for biochemical problems, 2023.
  * [46] Wu, Z., Ramsundar, B., Feinberg, E., Gomes, J., Geniesse, C., Pappu, A. S., Leswing, K., and Pande, V. Moleculenet: a benchmark for molecular machine learning.  Chem. Sci. 9 (2018), 513–530.
  * [47] Xia, J., Zhao, C., Hu, B., Gao, Z., Tan, C., Liu, Y., Li, S., and Li, S. Z. Mole-BERT: Rethinking pre-training graph neural networks for molecules.  In The Eleventh International Conference on Learning Representations (2023).
  * [48] Xu, D., Cheng, W., Luo, D., Chen, H., and Zhang, X. Infogcl: Information-aware graph contrastive learning.  In Advances in Neural Information Processing Systems (2021), M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34, Curran Associates, Inc., pp. 30414–30425.
  * [49] Xu, K., Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks?  In International Conference on Learning Representations (2019).
  * [50] Xu, M., Wang, H., Ni, B., Guo, H., and Tang, J. Self-supervised graph-level representation learning with local and global structure.  In Proceedings of the 38th International Conference on Machine Learning (18–24 Jul 2021), M. Meila and T. Zhang, Eds., vol. 139 of Proceedings of Machine Learning Research, PMLR, pp. 11548–11558.
  * [51] Ying, Z., Bourgeois, D., You, J., Zitnik, M., and Leskovec, J. Gnnexplainer: Generating explanations for graph neural networks.  In Advances in Neural Information Processing Systems (2019), H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds., vol. 32, Curran Associates, Inc.
  * [52] Ying, Z., You, J., Morris, C., Ren, X., Hamilton, W., and Leskovec, J. Hierarchical Graph Representation Learning with Differentiable Pooling.
  * [53] You, Y., Chen, T., Shen, Y., and Wang, Z. Graph Contrastive Learning Automated.
  * [54] Zhang, Z., Bian, Y., Xie, A., Han, P., Huang, L.-K., and Zhou, S. Can pre-trained models really learn better molecular representations for ai-aided drug discovery?, 2022.
  * [55] Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., and Wang, L. Graph contrastive learning with adaptive augmentation.  In Proceedings of the Web Conference 2021 (New York, NY, USA, 2021), WWW ’21, Association for Computing Machinery, p. 2069–2080.