# Capturing Fine-grained Semantics in Contrastive Graph Representation
Learning

Lin Shu  Chuan Chen Corresponding Author. Email<EMAIL_ADDRESS>Zibin Zheng  Sun Yat-sen University

###### Abstract

Graph contrastive learning defines a contrastive task to pull similar
instances close and push dissimilar instances away. It learns discriminative
node embeddings without supervised labels, which has aroused increasing
attention in the past few years. Nevertheless, existing methods of graph
contrastive learning ignore the differences between diverse semantics existed
in graphs, which learn coarse-grained node embeddings and lead to sub-optimal
performances on downstream tasks. To bridge this gap, we propose a novel Fine-
grained Semantics enhanced Graph Contrastive Learning (FSGCL) in this paper.
Concretely, FSGCL first introduces a motif-based graph construction, which
employs graph motifs to extract diverse semantics existed in graphs from the
perspective of input data. Then, the semantic-level contrastive task is
explored to further enhance the utilization of fine-grained semantics from the
perspective of model training. Experiments on five real-world datasets
demonstrate the superiority of our proposed FSGCL over state-of-the-art
methods. To make the results reproducible, we will make our codes public on
GitHub after this paper is accepted.

##  1 Introduction

Graph-structured data is pervasive in a wide variety of real-world scenarios,
such as social graphs [17, 23], citation graphs [1, 6] and biological graphs
[16, 9]. Recent years have witnessed the proliferation of network
representation [34], among which Graph Neural Networks (GNNs) [29, 39] attract
a surge of interest and show their effectiveness in learning good-quality
representations from graphs. However, most GNNs are trained under the
supervision of manual labels [10, 26], which are expensive and even
unavailable in the real world, resulting in overfitting on the limited labels.
As a consequence, self-supervised graph representation learning [31, 32] comes
into being, which makes full use of unlabeled graphs to train models and thus
removes the need for expensive manual labels.

Figure 1: An illustrative example of diverse semantics existed in social
graphs and the case study of utilizing fine-grained semantics in graph
contrastive learning. The histograms depict the accuracies of node
classification on the synthetic dataset.

As a dominative category of self-supervised graph representation learning,
graph contrastive learning (GCL) [33, 41] has gained increasing attention in
the past few years. Most GCL methods achieve an instance-level contrastive
task, i.e., make each node/instance discriminative from each other.
Nevertheless, there exist various latent semantics in graphs and different
nodes might share high similarity under different inherent semantics. The
instance-level contrastive schema constraints each node far away from each
other, which ignores the differences between diverse semantics existed in
graphs and is prone to pull nodes with high semantic similarity far away,
resulting in coarse-grained representation for each node. As an illustration,
Figure 1 depicts a social graph that contains four distinct communities.
Specifically, Alice participates in four communities that imply diverse
semantics of colleagues, friends, families and classmates. Existing GCL
methods leverage the connections in graphs without distinguishing the semantic
nuances between different connections, thus leading to the information loss of
inherent semantics and sub-optimal performances on downstream tasks. To verify
our hypothesis, we conduct a case study on a synthetic dataset composed of 8
communities, where nodes might belong to 2 communities simultaneously and each
community can simulate the inherent semantics in graphs explicitly (More
detailed settings are introduced in the Supplementary Material). Concretely,
we apply the multilabel algorithm [35] on the frozen node embeddings generated
from method w/o semantics (i.e., method that ignores fine-grained semantics
[13]) and method with semantics (i.e., our proposed method), where the ground-
truth communities are regarded as labels. The histograms report the
classification accuracies, where the value of the $i$-th row and $j$-th column
denotes the proportion of nodes belonging to both community $i$ and $j$ that
are accurately classified. As a result, the diagonal parts of histograms
depict the accuracies of nodes with only one community, while the non-diagonal
parts depict the accuracies of nodes with two communities. It can be observed
that method w/o semantics has lower classification accuracies on the non-
diagonal parts compared with diagonal ones, implying that methods ignoring
fine-grained semantics have poor learning ability on nodes with multiple
semantics. In contrast, method with semantics improves the accuracies of non-
diagonal parts, indicating that considering the fine-grained semantics can
boost the embedding quality and achieve higher performances on downstream
tasks. Therefore, it is of curial significance to mine fine-grained semantics
for graph contrastive learning.

In recent years, numerous studies [2, 5] have verified the effectiveness of
motifs for capturing latent semantics in graphs. Specifically, graph motifs
are defined as essential subgraph patterns that occur frequently in graphs.
Since different graph motifs depict different semantic contexts of each node,
the semantic roles of each node can be easily distinguished according to the
specified semantic contexts. As a consequence, graph motifs have been
recognized as popular fundamental tools to mine latent semantics. For example,
Motif-CNN [22] fuses information from multiple graph motifs to mine semantic
dependencies between nodes. SHNE [38] captures structural semantics by
exploiting graph motifs to boost the expressive power of the proposed model.
Therefore, we propose to integrate graph motifs to mine fine-grained semantics
for graph contrastive learning.

Overall, we propose Fine-grained Semantics enhanced Graph Contrastive Learning
(FSGCL) in this paper. Concretely, FSGCL first introduces motif-based graph
construction, which generates multiple semantic graphs to integrate latent
distinct semantics from the perspective of input data. Then, based on the
generated semantic graphs, we combine the instance-level contrastive task with
a carefully designed semantic-level contrastive task to further enhance the
utilization of fine-grained latent semantics from the perspective of model
training. In addition, since the random negative sampling strategy used in
traditional GCL methods is prone to treat nodes with high similarity as
negative pairs and pull them away, which results in the loss of semantic
similarity, we adopt a slow-moving average approach [7] to achieve the
contrastive task without negative samples to further ensure the effective
utilization of inherent semantics.

To sum up, the major contributions of this paper are concluded as follows:

  * •

We propose a fine-grained semantics enhanced graph contrastive learning
(FSGCL) model, which boosts the expressive ability of node representations by
exploring latent semantics for contrastive learning on graphs.

  * •

The motif-based graph construction is introduced to extract diverse semantics
existed in graphs from the perspective of input data. Furthermore, the
semantic-level and instance-level contrastive task without negative samples
are explored jointly to further enhance the utilization of fine-grained
semantics from the perspective of model training.

  * •

We conduct extensive experiments to demonstrate the effectiveness of FSGCL. We
further report the results of visualization on the synthetic graph to verify
the superior ability of capturing the diverse semantics of FSGCL.

##  2 Related Work

###  2.1 Graph Contrastive Learning

Graph contrastive learning has gained increasing popularity in recent years,
which leverages unlabeled graphs to train models and thus removes the need for
expensive manual labels. Inspired by the success of contrastive learning in
computer vision [4, 25], contrastive learning for graphs achieves the
contrastive task by pulling similar instances close and pushing dissimilar
instances far away. Representative researches include GraphCL [33] which
designs various types of graph augmentations to facilitate invariant
representation learning for contrastive GNNs. Further, GCA [41] proposes
adaptive augmentation schemes to encourage the model to learn significant
topologies and attributes from graphs. However, they exploit the connections
in graphs without distinguishing the underlying semantic nuances, thus leading
to sub-optimal node representations. Recently, PGCL [15] proposes a
clustering-based approach that encourages semantically similar graphs closer
and investigates the bias issue of negative sampling. DGCL [14] disentangles
the latent semantic factors of the graph to learn factorized graph
representations for contrastive learning. Nevertheless, these methods are
tailored for graph-level contrastive learning, which don’t take the underlying
node-level semantics into account and are not suitable for node-level graph
contrastive learning.

###  2.2 Motifs in Graph Learning

Graph motif, a fundamental subgraph pattern that occurs frequently in graphs,
plays a significant role in mining latent semantics. For instance, Motif-CNN
[22] fuses information from multiple graph motifs to mine semantic
dependencies between nodes. SHNE [38] captures structural semantics by
exploiting graph motifs to boost the expressive power of the proposed model.
However, these models are trained under the supervision of manual labels,
which are expensive and even unavailable. Therefore, it is of crucial
significance to utilize graph motifs to capture fine-grained semantics via
graph contrastive learning, a dominative category of self-supervised graph
representation learning. It should be noted that MICRO-Graph [36] and MGSSL
[37] also incorporate graph motifs with self-supervised learning, but the goal
of them is to generate informative motifs via self-supervised learning, where
the latter model further leverages learned motifs to sample subgraphs for
graph-level contrastive learning, whose aims are entirely different from that
of our node-level graph contrastive learning model, which considers the
underlying semantics to generate high-quality node embeddings.

Figure 2: An illustrative example of graph motifs and corresponding co-
occurrence matrices.

##  3 Preliminaries

In this section, we first describe the main definitions and notations that
appeared in this paper. Specifically, calligraphic math font (e.g.,
$\mathcal{V}$) represents set, boldface upperclass letters (e.g.,
$\mathbf{A}$) represent matrices and boldface lowerclass letters (e.g.,
$\mathbf{w}$) represent vectors.

Graph Contrastive Representation Learning. A graph can be represented as
$\mathcal{G}=(\mathcal{V},\mathcal{E})$, where
$\mathcal{V}=\\{v_{1},v_{2},...,v_{n}\\}$ denotes the set of $n$ nodes and
$\mathcal{E}$ denotes the set of links. Given a graph $\mathcal{G}$, graph
contrastive representation learning aims at learning discriminative
$d$-dimensional embeddings $\mathbf{Z}\in\mathbb{R}^{n\times d}$ for $n$ nodes
in $\mathcal{G}$ via a contrastive task, which can be served for downstream
tasks such as node classification.

Graph Motif. A graph motif is denoted as
$\mathcal{M}=(\mathcal{V}_{\mathcal{M}},\mathcal{E}_{\mathcal{M}})$, where
$\mathcal{V}_{\mathcal{M}}$ and $\mathcal{E}_{\mathcal{M}}$ are the set of
nodes and links of motif $\mathcal{M}$ respectively. Given a motif pattern
$\mathcal{M}$, a motif instance is defined as a subgraph of $\mathcal{G}$ that
matches the pattern of motif $\mathcal{M}$, which is denoted as
$(\mathcal{V}_{S},\mathcal{E}_{S})$, where
$\mathcal{V}_{S}\subseteq\mathcal{V}$ and
$\mathcal{E}_{S}\subseteq\mathcal{E}$, satisfying (1) $\forall
u\in\mathcal{V}_{S}$, $\Phi(u)\in\mathcal{V}_{\mathcal{M}}$, where
$\Phi:\mathcal{V}_{S}\rightarrow\mathcal{V}_{\mathcal{M}}$ is a bijection, (2)
$\forall u,v\in\mathcal{V}_{S}$, if
$(\Phi(u),\Phi(v))\in\mathcal{E}_{\mathcal{M}}$, then
$(u,v)\in\mathcal{E}_{S}$. We denote the instance set of motif $\mathcal{M}$
as $S_{\mathcal{M}}=\\{(\mathcal{V}_{S},\mathcal{E}_{S})\\}$.

Motif-based Co-occurrence Matrix. Given a specific motif $\mathcal{M}$ and
corresponding instance set
$S_{\mathcal{M}}=\\{(\mathcal{V}_{S},\mathcal{E}_{S})\\}$, the co-occurrence
matrix $\mathbf{O}^{\mathcal{M}}$ of $\mathcal{M}$ is defined as:

$\mathbf{O}^{\mathcal{M}}_{v,u}=\sum_{(\mathcal{V}_{S},\mathcal{E}_{S})}I((v,u)\in\mathcal{E}_{S}),$
(1)

where $I(s)$ is the indicative function, i.e., the value of $I(s)$ equals to 1
if the condition $s$ is true and 0 otherwise. $\mathbf{O}^{\mathcal{M}}_{v,u}$
depicts the number of co-occurrences of node $v$ and $u$ in all motif
instances in $S_{\mathcal{M}}$. Figure 2 illustrates an example of graph
motifs and corresponding co-occurrence matrices.

Figure 3: Framework of FSGCL. FSGCL utilizes the online and target network to
learn, where the target network is stop-gradient and optimized by the slow-
moving average of the online network.

##  4 Framework

###  4.1 Components of FSGCL

Overall, FSGCL first generates two graph views $\mathcal{G}^{1st}$ and
$\mathcal{G}^{2nd}$ based on the semantic graphs constructed by graph motifs
(Section 4.2). Then, FSGCL utilizes two main neural networks to learn
embeddings, i.e., the online network $f_{\theta}$ and target network
$f_{\xi}\;(\theta\neq\xi)$. Concretely, the online network is composed of
graph encoder, projector and predictor, while the target network has the same
architecture as online network except for the absence of predictor. The
asymmetric architecture prevents the training model from falling into trivial
solutions. The semantic-wise graph encoder are introduced to extract latent
semantics existed in graphs (Section 4.3), followed by the semantic-level and
instance-level contrastive objective formalized in Section 4.4. The whole
architecture of FSGCL is illustrated in Figure 3.

###  4.2 Motif-based Graph Construction

To explicitly explore latent semantics in graphs, we leverage graph motifs to
capture the semantic similarity. Specifically, given $T$ graph motifs, for
each motif $\mathcal{M}_{i},i\in\\{1,...,T\\}$, we first obtain the motif
instance set $\mathcal{S_{M}}_{i}$ via an efficient motif matching algorithm
[12], and calculate the corresponding motif-based co-occurrence matrix
$\mathbf{O}^{\mathcal{M}_{i}}$, where node $u$ and $v$ share the same
semantics within motif $\mathcal{M}_{i}$ if
$\mathbf{O}^{\mathcal{M}_{i}}_{u,v}$ is non-zero. It is straightforward to
employ the co-occurrence matrix as the semantic adjacency matrix to achieve
the subsequent graph encoding procedure to generate node embeddings. However,
on the one hand, graph motifs can only capture underlying semantics from the
perspective of structural information, which ignore the role of node features
played in semantic similarity. On the other hand, there exists a large number
of similar node pairs in the co-occurrence matrix, thus it is unscalable and
inflexible to regard all similar node pairs as neighbor pairs. As a
consequence, we propose to facilitate both the feature similarity and motif
co-occurrence matrix to select top-K neighbors for each node. Concretely,
given the node feature $\mathbf{X}\in\mathbb{R}^{n\times d_{f}}$, where
$d_{f}$ denotes the dimension of initial features, we first calculate the
feature similarity matrix $\mathbf{C}$ through the widely used cosine
similarity, which is formulated as:

$\mathbf{C}_{ij}=\frac{\mathbf{X}_{i}\cdot\mathbf{X}_{j}}{\left\|\mathbf{X}_{i}\cdot\mathbf{X}_{j}\right\|}.$
(2)

Then, the non-zero position matrix of the co-occurrence matrix
$\mathbf{O}^{\mathcal{M}_{i}}$ is extracted, which is denoted as
$\mathbf{R}^{\mathcal{M}_{i}}$:

$\mathbf{R}^{\mathcal{M}_{i}}_{u,v}=\left\\{\begin{array}[]{lr}1,&\mathbf{O}^{\mathcal{M}_{i}}_{u,v}\neq
0,\\\ 0,&\mathbf{O}^{\mathcal{M}_{i}}_{u,v}=0.\end{array}\right.$ (3)

Ultimately, we select the top-K elements from
$\mathbf{M}=\mathbf{C}\odot\mathbf{R}^{\mathcal{M}_{i}}$ for each node to
construct the semantic graph $G^{SG_{i}}$, whose adjacency matrix
$\mathbf{A}^{SG_{i}}$ is calculated as

$\mathbf{A}^{SG_{i}}_{u,v}=\left\\{\begin{array}[]{lr}\mathbf{M}_{u,v},&\mathbf{M}_{u,v}\in
TopK(\mathbf{M}_{u}),\\\ 0,&\mathbf{M}_{u,v}\notin
TopK(\mathbf{M}_{u}),\end{array}\right.$ (4)

where $\odot$ denotes the operation of Hadamard product. Therefore, we
construct $T$ semantic graphs via $T$ graph motifs respectively, where the
non-zero elements of each adjacency matrix $\mathbf{A}^{SG_{i}}$ denotes that
two nodes share high semantic similarity within both motif $\mathcal{M}_{i}$
and node features.

Based on the semantic graphs, we generate two different graph views $G^{1st}$
and $G^{2nd}$ via graph augmentations, which help learn discriminative node
embeddings for subsequent graph encoding and contrastive tasks. To further
strengthen the information of the input graph, we collectively utilize the
original graph and semantics graphs to build different graph views.
Specifically, inspired by MVGRL [8], we first adopt the Personalized PageRank
(PPR) [18] diffusion on the input graph to compute the diffusion matrix
$\mathbf{U}$:

$\mathbf{U}=\alpha{(\mathbf{I}_{n}-(1-\alpha)\mathbf{D}^{-\frac{1}{2}}\mathbf{A}\mathbf{D}^{-\frac{1}{2}})}^{-1},$
(5)

where $\mathbf{A}$ is the adjacency matrix of the input graph,
$\mathbf{D}\in\mathbb{R}^{n\times n}$ is the diagonal degree matrix of
$\mathbf{A}$ and $\alpha$ denotes the teleport probability in a random walk
[11]. Furthermore, we perturb the initial node feature matrix $\mathbf{X}$ to
generate augmented features $\mathbf{\widetilde{X}}^{1st}$ and
$\mathbf{\widetilde{X}}^{2nd}$ of different views, where each elements of
$\mathbf{\widetilde{X}}^{1st}$ and $\mathbf{\widetilde{X}}^{2nd}$ is set to
zero with a probability $r$. Overall, as depicted in Figure 3, we build two
graph views $G^{1st}$ and $G^{2nd}$ by

$\small\left\\{\begin{array}[]{lr}G^{1st}=\\{(\mathbf{\widetilde{X}}^{1st},\mathbf{\widetilde{A}}^{1st}_{i})|\mathbf{\widetilde{A}}^{1st}_{0}=\mathbf{A},\mathbf{\widetilde{A}}^{1st}_{j}=\mathbf{A}^{SG_{j}}\\},\\\
\\\
G^{2nd}=\\{(\mathbf{\widetilde{X}}^{2nd},\mathbf{\widetilde{A}}^{2nd}_{i})|\mathbf{\widetilde{A}}^{2nd}_{0}=\mathbf{U},\mathbf{\widetilde{A}}^{2nd}_{j}=\mathbf{A}^{SG_{j}}\\},\end{array}\right.$
(6)

where $i\in\\{0,...,T\\}$ and $j\in\\{1,...,T\\}$. By applying graph diffusion
and feature perturbation, we can not only simultaneously encode rich local and
global information existed in the original graph [8], but also extract the
intrinsic feature information. As a result, each graph view is composed of a
holistic graph (i.e., the original whole graph) and $T$ semantic graphs that
have been perturbed.

###  4.3 Semantic-wise Graph Encoder

After generating two graph views based on the holistic graph and semantic
graphs, we employ graph neural networks (GNNs) to learn node embeddings on
each view. Since the $T+1$ graphs of each view depict distinct aspects of the
input graph, encoding all graphs with the same GNN will result in the loss of
unique information of each graph. Therefore, we propose to learn separate GNNs
for each graph of each view, which are collectively referred to as semantic-
wise graph encoder. More specifically, for each graph
$(\mathbf{\widetilde{X}}^{m},\mathbf{\widetilde{A}}^{m}_{i}),i\in\\{0,...,T\\}$,
where $m$ denotes the $m$-th augmented graph view, we utilize graph
convolutional layers [10] to learn node embeddings $\mathbf{Z}^{m}_{i}$, which
update node embeddings by aggregating neighbors’ embeddings. The propagation
of each layer can be represented as:

$\mathbf{Z}^{m}_{i}=\sigma(\mathbf{\widetilde{D}}_{i,m}^{-\frac{1}{2}}\mathbf{\overline{A}}^{m}_{i}\mathbf{\widetilde{D}}_{i,m}^{-\frac{1}{2}}\mathbf{Z}^{m}_{i}\mathbf{W}^{m}_{i}),$
(7)

where $\sigma$ is the activation function (i.e., PReLU),
$\mathbf{\overline{A}}^{m}_{i}=\mathbf{\widetilde{A}}^{m}_{i}+\mathbf{I}_{n}$
is the adjacency matrix added with self-connections, $\mathbf{I}_{n}$ is the
identity matrix, $\mathbf{\widetilde{D}}_{i,m}$ is the diagonal degree matrix
of $\mathbf{\overline{A}}^{m}_{i}$,
$\mathbf{W}_{i}^{m}\in\mathbb{R}^{d_{in}\times d_{out}}$ is the layer-specific
transformation matrix for the $i$-th graph in the $m$-th view, $d_{in}$ and
$d_{out}$ denote the input and output embedding dimensions of the propagation
layer and $d_{out}$ is set to be $d$ in each propagation layer. Ultimately,
the GNNs employed in semantic-wise graph encoder are built by stacking
multiple propagation layers defined in Equation 7. For the convenience of
presentation, we denote $\mathbf{Z}^{m}_{0}$ generated from the holistic graph
as holistic embeddings and $\mathbf{Z}^{m}_{i},i\in\\{1,...,T\\}$ generated
from semantic graphs as semantic embeddings in the following sections.

Generally, for each augmented graph view, since different semantics emphasize
distinct aspects of the whole graph, we apply the weighted sum function on $T$
semantic embeddings before combining them with the holistic embeddings to
obtain the final node embeddings $\mathbf{Z}^{m}_{combine}$:

$\mathbf{Z}^{m}_{combine}=\beta\sum_{i=1}^{T}{w^{m}_{i}\mathbf{Z}^{m}_{i}}+\mathbf{Z}^{m}_{0},$
(8)

where $w^{m}_{i}$ is the coefficient that balances different importance of $T$
semantics and $\beta$ is the weight coefficient that controls the balance
between the holistic and semantic embeddings. As a consequence, the proposed
semantic-wise graph encoder generates $T+2$ embeddings for each node $v$ on
each graph view, which contains the holistic embeddings $\mathbf{Z}^{m}_{0}$,
$T$ semantic embeddings $\mathbf{Z}^{m}_{i},i\in\\{1,...,T\\}$, and the final
node embeddings $\mathbf{Z}^{m}_{T+1}=\mathbf{Z}^{m}_{combine}$.

###  4.4 Contrastive Objective

The downstream task requires the output embeddings of graph encoders to learn,
yet the optimization objectives of the downstream task and contrastive task
are not consistent. As a consequence, before defining the contrastive
objectives, we employ graph projectors to project the encoded embeddings
$\mathbf{Z}_{i}^{m}$ to new subspaces, which have been confirmed to be an
effective way to improve the representation quality [3]. Specifically, we
leverage a non-linear perceptron layer to project node embeddings:

$\mathbf{Q}^{m}_{i}=\sigma(\mathbf{U}_{i}^{m}\mathbf{Z}_{i}^{m}+b_{i}^{m}),i\in\\{0,...,T+1\\},$
(9)

where $\mathbf{U}_{i}^{m}\in\mathbb{R}^{d\times d}$ and
$b_{i}^{m}\in\mathbb{R}^{d\times 1}$ are the transformation matrix and bias
for the $i$-th embeddings of $m$-th graph view. Furthermore, for the online
network, we stack $L$ perceptron layers as predictors to build an asymmetric
architecture to prevent the training model from falling into trivial
solutions:

$\mathbf{P}_{i}^{1st}=\sigma(\mathbf{E}_{i}\mathbf{Q}_{i}^{1st}+a_{i}),i\in\\{0,...,T+1\\},$
(10)

where $\mathbf{E}_{i}\in\mathbb{R}^{d\times d}$ and
$a_{i}\in\mathbb{R}^{d\times 1}$ are the transformation matrix and bias for
the $i$-th embeddings in graph predictor.

In the following, we introduce the instance-level and semantic-level
contrastive task in detail.

####  4.4.1 Instance-level Contrastive Learning

The instance-level contrastive task regards the graph as a perceptual whole
and achieves the contrastive task on the whole graph, i.e., it doesn’t
distinguish different semantics. In this section, we formalize the instance-
level contrastive task on the holistic and final combined node embeddings
since they don’t distinguish diverse semantics.

Traditional contrastive learning methods randomly select negative instances
for each node to achieve the contrastive task. Nevertheless, random negative
sampling introduces false-negative instance pairs, i.e., nodes that share
similar semantics might be pulled far away, thus weakening the utilization of
node similarity. Therefore, inspired by BYOL, we propose to design the
instance-level contrastive task without negative samples. Specifically, we
utilize the normalized predicted embeddings of online network
$\mathbf{P}_{i}^{1st}$ to predict the normalized projected embeddings of
target network $\mathbf{Q}_{i}^{2nd}$ and leverage the cosine similarity to
calculate the loss:

$\small
L_{Holistic}=-\frac{1}{N}\sum_{v=1}^{N}{\frac{\mathbf{P}_{(0,v)}^{1st}\cdot\mathbf{Q}_{(0,v)}^{2nd}}{\left\|\mathbf{P}_{(0,v)}^{1st}\right\|\cdot\left\|\mathbf{Q}_{(0,v)}^{2nd}\right\|}},$
(11) $\small
L_{Combine}=-\frac{1}{N}\sum_{v=1}^{N}{\frac{\mathbf{P}_{(T+1,v)}^{1st}\cdot\mathbf{Q}_{(T+1,v)}^{2nd}}{\left\|\mathbf{P}_{(T+1,v)}^{1st}\right\|\cdot\left\|\mathbf{Q}_{(T+1,v)}^{2nd}\right\|}},$
(12)

where $\mathbf{P}_{(i,v)}^{1st}$ and $\mathbf{Q}_{(i,v)}^{2nd}$ denote node
$v$’s embeddings of $\mathbf{P}_{i}^{1st}$ and $\mathbf{Q}_{i}^{2nd}$. As a
result, the predicting loss only requires pushing pairs with high similarity
closer.

####  4.4.2 Semantic-level Contrastive Learning

Since the instance-level contrastive learning ignores the fine-grained
semantics existed in graphs, we propose the semantic-level contrastive task to
strengthen the utilization of semantic information. Specifically, we leverage
$T$ semantic embeddings to achieve the semantic-level contrastive learning.
Since different semantic graphs convey distinct information, separate
contrastive objectives should be designed on $T$ semantic graphs to
distinguish semantic nuances. Therefore, for the $i$-th semantic graph, we
formalize the semantic predicting loss without negative samples as

$\small
L_{Semantic}^{i}=-\frac{1}{N}\sum_{v=1}^{N}{\frac{\mathbf{P}_{(i,v)}^{1st}\cdot\mathbf{Q}_{(i,v)}^{2nd}}{\left\|\mathbf{P}_{(i,v)}^{1st}\right\|\cdot\left\|\mathbf{Q}_{(i,v)}^{2nd}\right\|}},i\in\\{1,...,T\\},$
(13)

where $\mathbf{P}_{(i,v)}^{1st}$ and $\mathbf{Q}_{(i,v)}^{2nd}$ denote node
$v$’s embeddings of $\mathbf{P}_{i}^{1st}$ and $\mathbf{Q}_{i}^{2nd}$. The
whole semantic-level contrastive loss is composed by the combination of $T$
semantic contrastive subtasks:

$L_{Semantic}=\sum_{i=1}^{T}{L_{Semantic}^{i}},i\in\\{1,...,T\\}.$ (14)

Table 1: Node classification accuracies ($\pm$ std) results (in $\%$) on five real-world datasets. OOM: out of memory on a 24GB GPU. We emphasize the highest values in bold and the second ones with underlines.  | Amazon-Photos | Amazon-Computers | Coauthor-CS | Coauthor-Physics | WikiCS
---|---|---|---|---|---
GCN |  93.18 $\pm$ 0.18 |  89.37 $\pm$ 0.29 |  92.34 $\pm$ 0.04 |  95.06 $\pm$ 0.06 |  79.19 $\pm$ 0.60
DGI |  91.61 $\pm$ 0.22 |  83.95 $\pm$ 0.47 |  92.15 $\pm$ 0.63 |  94.51 $\pm$ 0.52 |  75.35 $\pm$ 0.14
GMI |  90.68 $\pm$ 0.17 |  82.21 $\pm$ 0.31 | OOM | OOM |  74.85 $\pm$ 0.08
MVGRL |  91.74 $\pm$ 0.07 |  87.52 $\pm$ 0.11 |  92.11 $\pm$ 0.12 |  95.33 $\pm$ 0.03 |  77.52 $\pm$ 0.08
GRACE |  92.59 $\pm$ 0.16 |  88.55 $\pm$ 0.30 |  89.81 $\pm$ 0.19 | OOM |  79.27 $\pm$ 0.67
GCA |  92.39 $\pm$ 0.32 |  87.57 $\pm$ 0.45 |  92.49 $\pm$ 0.14 | OOM |  79.14 $\pm$ 0.25
BGRL | 93.23 $\pm$ 0.28 | 90.42 $\pm$ 0.15 |  92.68 $\pm$ 0.14 |  95.37 $\pm$ 0.10 | 80.07 $\pm$ 0.49
ProGCL |  92.01 $\pm$ 0.24 |  87.44 $\pm$ 0.38 | OOM | OOM |  79.61 $\pm$ 0.63
AFGRL |  93.04 $\pm$ 0.26 |  89.41 $\pm$ 0.34 | 93.27 $\pm$ 0.17 | 95.71 $\pm$ 0.10 |  77.78 $\pm$ 0.42
FSGCL | 94.65 $\pm$ 0.16 | 90.54 $\pm$ 0.21 | 94.22 $\pm$ 0.07 | 96.10 $\pm$ 0.08 | 80.25 $\pm$ 0.02

###  4.5 Model Training

Combining the semantic-level and instance-level contrastive learning, we train
the FSGCL model by the joint loss:

$L_{Joint}=\gamma L_{Semantic}+L_{Holistic}+L_{Combine},$ (15)

where $\gamma$ is the weight coefficient that controls the significance of the
semantic-level contrastive task. In practice, we symmetrize the loss by
feeding $G^{2nd}$ and $G^{1st}$ to the online and target network respectively
to compute $\widetilde{L}_{Joint}$. Furthermore, we only update the parameters
$\theta$ of online network by minimizing the total loss
$L=L_{Joint}+\widetilde{L}_{Joint}$, while the parameter $\xi$ of target
network is stop-gradient and optimized by a slow-moving average of the online
network:

$\theta\leftarrow optimize(\theta,\lambda,\nabla_{\theta}L),$ (16)
$\xi\leftarrow\tau\xi+(1-\tau)\theta,$ (17)

where $\lambda$ is the learning rate and $\tau$ is the decay rate. At the end
of the training, we only keep the encoder of online network and treat the
final node embeddings $\mathbf{Z}^{1st}_{final}$ as the input of downstream
inference tasks.

Since the online and target network are initialized randomly, the randomness
makes all node embeddings distinct from each other at the initial stage of
training, thus achieving the effect of distinguishing different instances
(i.e., the effect of negative instances). Moreover, $\tau$ is usually set to
be a large number (e.g., 0.99), thus the slow-moving average algorithm can not
only make augmented instances pairs (i.e., positive instance pairs) closer,
but also implicitly preserve the distinction between different instances to a
large extent. Compared to methods with random negative sampling, FSGCL only
pulls positive instances closer, which not only reduces the computation cost,
but also avoids pushing instances of similar semantics far away while
optimization, thus strengthening the utilization of semantic similarity.

##  5 Experiments

###  5.1 Datasets

We conduct node classification on five public real-world datasets: Amazon-
Photos, Amazon-Computers111https://github.com/shchur/gnn-
benchmark/tree/master/data/npz, Coauthor-CS, Coauthor-
Physics222https://github.com/shchur/gnn-benchmark/tree/master/data/npz and
WikiCS333https://github.com/pmernyei/wiki-cs-dataset/raw/master/dataset, where
we use a random train/validation/test split (10/10/80$\%$) for the datasets of
Amazon and Coauthor since these four datasets have no standard dataset splits,
and follow the 20 canonical train/validation/test splits for WikiCS dataset.
The details of datasets as listed in Table 2.

Table 2: Description of datasets and corresponding matching numbers of motif instances. Datasets |  | Amazon-
---
Photos
| Amazon-
---
Computers
| Coauthor-
---
CS
| Coauthor-
---
Physics
WikiCS
Nodes | 7,650 | 13,572 | 18,333 | 34,493 | 11,701
Edges | 119,081 | 245,861 | 81,894 | 247,962 | 216,123
Attribute | 754 | 767 | 6,805 | 8,415 | 300
Classes | 8 | 10 | 15 | 5 | 10
Motif A | 12,136,593 | 42,532,480 | 1,409,965 | 7,499,633 | 36,875,788
Motif B | 717,400 | 1,527,469 | 85,799 | 468,550 | 3,224,375
Motif C | 14,516,676 | 38,630,055 | 202,620 | 1,886,051 | 79,649,534

###  5.2 Baselines and Experimental Settings

We compare our FSGCL with semi-supervised GCN [10] and 8 state-of-the-art
unsupervised contrastive learning methods including DGI [27], GMI [21], MVGRL
[8], GRACE [40], GCA [41], BGRL [24], ProGCL [30], AFGRL [13]. We use the
authors’ released codes and follow the papers’ guidance to train all
comparison methods. Since the three motif in Figure 2 are the most basic and
simples motifs consisting of three nodes and four nodes, we utilize three
motifs shown in Figure 2 in this paper, which are denoted as Motif-A,B,C in
the following. More detailed experimental settings and parameter sensitivity
analysis are presented in the Supplementary Material due to the limitation of
pages.

###  5.3 Experimental Results

We repeat all experiments 5 times and report the average accuracies ($\pm$
std) on five real-world datasets. The experimental results are reported in
Table 1, where we emphasize the highest values in bold and the second ones
with underlines. Overall, we summarize major observations as follows:

  * •

Our proposed FSGCL consistently outperforms baselines of graph contrastive
learning, which can be attributed to the ability of graph motifs that mine
fine-grained semantics existed in graphs. Therefore, distinguishing nuances
between diverse semantics in contrastive learning helps learn more informative
node embeddings, thus boosting the performances of downstream tasks.

  * •

It can be observed that FSGCL has large improvements on Amazon-Photos and
Coauthor datasets, while the improvements are relatively small on Amazon-
Computers and WikiCS. We conjecture that this phenomenon is because the
matching numbers of motif instances are higher on Amazon-Computers and WikiCS
than other datasets (shown in Table 2). To be more specific, we extract the
non-zero node pairs in the motif co-occurrence matrices, and select top-K
elements for each node based on the feature similarity of the aforementioned
non-zero node pairs to construct semantic graphs. As a result, when there
exist excess motif instances, a large amount of non-zero node pairs in the co-
occurrence matrices are extracted, resulting in the adjacency matrices of
semantic graphs being extremely close to the top-K matrices of feature
similarity, thus having less informative semantics and achieving smaller
improvements.

  * •

Contrastive learning methods that don’t require negative instances (i.e., our
FSGCL, BGRL and AFGRL) perform better than those with negative sampling (i.e.,
DGI, GMI, MVGRL, GRACE, GCA and ProGCL). What’s more, methods without negative
sampling are more scalable and can easily be applied to larger datasets such
as Coauthor-CS and Coauthor-Physics. It verifies that removing the procedure
of negative sampling can not only reduce the computation cost, but also
improve the embedding quality since it avoids pushing instances of similar
semantics far away while optimizing.

(a) ProGCL

(b) AFGRL

(c) BGRL

(d) FSGCL

Figure 4: Visualization analysis on the synthetic datasets.

###  5.4 Visualization Analysis on Synthetic Graphs

To provide a more intuitive understanding of the functionality of mining fine-
grained semantics of FSGCL, we employ the well-known LFR toolkit [12] to
generate a synthetic graph with overlapping communities, i.e., each node might
exist in multiple communities, which can simulate the underlying diverse
semantics more explicitly. Concretely, we generate a graph with 10,000 nodes
and 8 communities, where 8,000 nodes simultaneously belong to two different
communities (More detailed parameters are introduced in the Supplementary
Material). Then, the multilabel K nearest neighbors algorithm [35] is applied
on the frozen node embeddings generated from ProGCL, AFGRL, BGRL and our
FSGCL, where the ground-truth communities are used as labels.

Figure 4 depicts the heatmaps of classification accuracies on 8 communities,
where the value of the $i$-th row and $j$-th column denotes the proportion of
nodes belonging to both community $i$ and $j$ that are accurately classified.
Therefore, the diagonal elements of heatmaps imply the predicting accuracies
of nodes which have only one community, while the non-diagonal elements imply
the accuracies for nodes which have two distinct communities. It is apparent
that ProGCL, AFGRL and BGRL have higher predicting accuracies on the diagonals
compared with non-diagonal parts of heatmaps, which indicates that existing
methods learn poor-quality embeddings for nodes that simultaneously
participate in multiple communities since they ignore the distinct semantics
existed in graphs. Instead, our FSGCL generates higher accuracies overall,
especially on the non-diagonal parts of heatmaps, which demonstrates that
FSGCL can indeed distinguish fine-grained semantics and boost the expressive
ability.

Table 3: Ablation study of key components of FSGCL. Datasets |  | Amazon-
---
Photos
| Amazon-
---
Computers
| Coauthor-
---
CS
| Coauthor-
---
Physics
WikiCS
FSGCL | 94.65 | 90.54 | 94.22 | 96.10 | 80.25
w/o $w_{m}^{i}$ | 94.07 | 90.45 | 94.20 | 96.08 | 80.26
w/o slow | 92.20 | 89.80 | 92.09 | 95.75 | 78.45
w/o $\mathbf{A}^{SG_{i}}$ | 93.13 | 90.05 | 93.32 | 95.75 | 79.55
w/o top-k $\mathbf{A}^{SG_{i}}$ | 93.97 | 89.96 | 94.10 | 96.03 | 78.97
w/o $L_{Semantic}$ | 94.19 | 90.49 | 93.92 | 96.02 | 80.14
w/o $L_{Holistic}$ | 94.15 | 90.11 | 93.87 | 96.02 | 79.60

###  5.5 Ablation Study

The ablation study is performed on six variants to investigate the
effectiveness of key components:

  * •

w/o $w_{i}^{m}$: it removes the weighted coefficients of different motifs in
Eq.(8), i.e., it trains with an average weighted sum in Eq.(8).

  * •

w/o slow: it replaces the slow-moving average algorithm with the random
negative sampling strategy to optimize the model.

  * •

w/o $\mathbf{A}^{SG_{i}}$: it sets
$\mathbf{A}^{SG_{i}}=\mathbf{A},i\in\\{1,...,T\\}$, i.e., it trains with
original graphs instead of semantic graphs.

  * •

w/o top-k $\mathbf{A}^{SG_{i}}$: it sets
$\mathbf{A}^{SG_{i}}=topK(\mathbf{C}),i\in\\{1,...,T\\}$, i.e., it substitutes
the top-K matrices of feature similarity for the adjacency matrices of
semantic graphs to train.

  * •

w/o $L_{Semantic}$ and w/o $L_{Holistic}$: they set $L_{Semantic}=0$ and
$L_{Holistic}=0$, respectively.

The results of FSGCL and its variants are reported in Table 3. From this
table, we can conclude that:

  * •

$w/o\;w_{i}^{m}$ performs slightly worse than FSGCL in most situations,
demonstrating that considering different contributions made by distinct motifs
helps improving the performances in downstream tasks. It should be noted that
$w/o\;w_{i}^{m}$ is still superior than all comparison methods in Table 1,
demonstrating that our FSGCL can achieve improvements with only few
parameters.

  * •

${w/o\;slow}$ performs much worse than FSGCL, proving that the non-negative
schema (i.e., the slow-moving average algorithm) can indeed improve the
embedding quality by reducing the information loss of semantics.

  * •

We observe a performance drop in w/o $\mathbf{A}^{SG_{i}}$ and w/o top-k
$\mathbf{A}^{SG_{i}}$, verifying the effectiveness of graph motifs of
capturing informative semantic nuances.

  * •

Compared with w/o $L_{Semantic}$ and w/o $L_{Holistic}$, we can see that both
semantic-level and instance-level contrastive loss can boost the performances
of FSGCL, confirming the necessity of combining these two contrastive tasks.

Table 4: Ablation study of different motif combinations of FSGCL. Datasets |  | Amazon-
---
Photos
| Amazon-
---
Computers
| Coauthor-
---
CS
| Coauthor-
---
Physics
WikiCS
FSGCL | 94.65 | 90.54 | 94.22 | 96.10 | 80.25
motif-A | 94.60 | 90.51 | 93.81 | 96.02 | 79.47
motif-B | 93.74 | 89.97 | 93.49 | 95.85 | 79.89
motif-C | 94.11 | 90.21 | 93.54 | 95.85 | 79.86
motif-AB | 94.49 | 90.50 | 94.15 | 96.07 | 79.98
motif-AC | 94.62 | 90.52 | 94.16 | 96.05 | 79.99
motif-BC | 94.29 | 94.30 | 93.78 | 95.83 | 80.14

Besides, to investigate how graph motifs affect the performances of FSGCL, we
conduct experiments on all combinations of motif-A,B,C used in this paper.
From Table 4, we can find that all combinations are worse than FSGCL which
contains all three motifs, demonstrating that more graph motifs can help mine
more semantic information and boost the expressive ability of node embeddings.
It should be noted that our method can be seamlessly extended to utilize more
complex graph motifs, we don’t conduct extra experiments with more motifs in
this paper.

###  5.6 Runtime Analysis

In this section, we compare the runtime of our FSGCL and three methods
(ProGCL, AFGRL and BGRL) that perform the best among all comparison methods.
Specially, we not only report the model training time of FSGCL, but also
report the time of semantic construction, i.e., the time of generating motif
instances and constructing semantic graphs in the preprocessing process. The
runtime results on Amazon-Photos and Amazon-Computers are depicted in Figure
5:

  * •

Compared with ProGCL requiring negative samples in model training, FSGCL
reduces training time by removing the need of massive negative instances.

  * •

FSGCL requices less training time than AFGRL. Though AFGRL also adopts the
schema of non-negative training, this phenomenon is reasonable since AFGRL
performs K-means clustering in each iteration, which is more time-consuming
than our FSGCL that constructs semantic graphs during data preprocessing.

  * •

Although FSGCL consumes more total time than BGRL, FSGCL owns less model
training time than BGRL, which might because FSGCL converges faster by
capturing fine-grained semantics and thus requiring fewer iterations and less
training time.

(a) Amazon-Photos

(b) Amazon-Computers

Figure 5: Runtime of FSGCL and three comparison methods on Amazon-Photos and
Amazon-Computers.

###  5.7 Conclusions

In this paper, we propose a fine-grained semantics enhanced graph contrastive
learning model - FSGCL. Concretely, FSGCL first employs graph motifs to
construct multiple semantic graphs to mine the distinct semantics from the
perspective of input data. Then, the semantic-level contrastive task without
negative samples is introduced to further enhance the utilization of fine-
grained latent semantics from the perspective of model training. The conducted
experiments on five real-world datasets indicate that FSGCL is consistently
superior to state-of-the-art methods, verifying the effectiveness of our
proposed model.

(a) Amazon Photos

(b) Amazon Computers

(c) Coauthor CS

(d) Coauthor Physics

(e) WikiCS

Figure 6: Parameter sensitivity analysis of $\tau,k,r,\gamma,\beta$ on five
real-world datasets.

##  Appendix A Experimental Settings

The proposed model FSGCL is implemented by PyTorch [19] and DGL [28] with the
AdamW optimizer with the base learning rate $\eta_{b}=0.001$ and weight decay
of $10^{-5}$. Following the settings of BGRL [24], the learning rate is
annealed via a cosine schedule during the $n_{total}$ steps with an initial
warmup period of $n_{w}$ steps. As a result, the learning rate at step $i$ is
computed as:

$\small\eta_{i}=\left\\{\begin{array}[]{lr}\frac{i\times\eta_{base}}{n_{w}},&if\;i\leq
n_{w}\\\ \\\
\eta_{b}\times(1+cos{\frac{(i-n_{w})\times\pi}{n_{total}-n_{w}}})\times
0.5,&if\;n_{w}\leq i\leq n_{total}\end{array}\right.$ (18)

For node classification, the final evaluation is achieved by fitting an
$l_{2}$-regularized LogisticRegression classifier from Scikit-Learn [20] using
the liblinear solver on the frozen node embeddings, where the regularization
strength is chosen by grid-search from
$\\{2^{-10},2^{-9},...,2^{9},2^{10}\\}$. The dimension of hidden and output
representations are set to be 512. The number of $k,\tau,\beta,\gamma$ and
layers of GCN are set to be 7, 0.99, 0.5, 0.5, 2 for Amazon-Computers and
WikiCS, while 5, 0.996, 1, 1, 1 for other datasets. The MLP layers $L$ of
online predictor is set to be 2 for Amazon and Wiki datasets, and 1 for
Coauthor datasets. The augmentation ratio $r$ is set to be 0.2 for Amazon-
Computers, 0.1 for WikiCS and 0.3 for other datasets. As for the motif
coefficient $w_{i}^{m},i\in{motif-A,B,C}$, we set the value of $w^{m}$ to be
$[0.7,0.1,0.2]$ for Amazon-Photos, $[0.3,0.1,0.6]$ for Amazon-Computers,
$[0.4,0.2,0.4]$ for Coauthor-CS, $[0.5,0.45,0.05]$ for Coauthor-Physics and
$[0.1,0.5,0.4]$ for WikiCS. The value of $\alpha$ is set to be 0.2 on all
datasets.

##  Appendix B Settings of Synthetic Dataset

We employ the well-known LFR toolkit [12] to generate a synthetic graph with
10,000 nodes and 8 overlapping communities. More specifically, we set the
average degree $k$ and maximum degree to be 20 and 50 respectively, the mixing
parameter $mu$ (each node shares a fraction of its edges with nodes in other
communities) to be 0.2, the minimum number $minc$ and maximum number $maxc$ of
community size to be 1500 and 3000, the number of overlapping nodes $on$ to be
8,000, and the number of memberships of the overlapping nodes $om$ to be 2.

##  Appendix C Parameter Sensitivity Analysis

In this section, we investigate the sensitivity of five major hyperparameters:
the decay rate $\tau$ of the slow-moving average strategy, $k$ that is used to
select the top-K elements to build semantic graphs, the augmentation ratio
$r$, the weight coefficient $\gamma$ that controls the significance of the
semantic-level contrastive objective, and the weight coefficient $\beta$ that
balances the holistic and semantic embeddings. The results of the above
parameters are illustrated in Figure 6, where $\tau$ takes the value from the
list {0.9,0.93,0.96,0.99,0.993,0.996,0.999}, $k$ ranges in {3,5,7,9,11,13,15},
$r$ ranges in {0.1,0.2,0.3,0.4,0.5,0.6,0.7}, $\gamma$ and $\beta$ take the
value from {0.1,0.3,0.5,0.7,0.9,1,2} and {0.1,0.3,0.5,0.7,0.9,1,2},
respectively.

As shown in Figure 6, FSGCL achieves higher performances when $\tau$ ranges
from 0.99 to 0.999, demonstrating that a larger value of $\tau$ is beneficial
for the slow-moving average strategy for learning discriminative embeddings.
Besides, FSGCL always performs better when $k$ is small, which is reasonable
since the semantic graphs own more unique semantic information when $k$ is
smaller. For the augmentation ratio $r$, the performances drop sharply when
$r$ becomes too large, indicating that over-perturbation leads to the loss of
useful information in the original graphs. For the weight coefficient $\gamma$
and $\beta$, it can be observed that our model is relatively stable with
regard to $\gamma$, while the performance decreases when $\beta$ is too small
or too large.

## References

  * [1] Kimitaka Asatani, Junichiro Mori, Masanao Ochi, and Ichiro Sakata, ‘Detecting trends in academic research from a citation network using network representation learning’, PloS one, 13(5), e0197260, (2018).
  * [2] Chris Biemann, Lachezar Krumov, Stefanie Roos, and Karsten Weihe, ‘Network motifs are a powerful tool for semantic distinction’, in Towards a Theoretical Framework for Analyzing Complex Linguistic Networks, 83–105, Springer, (2016).
  * [3] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton, ‘A simple framework for contrastive learning of visual representations’, in International conference on machine learning, pp. 1597–1607. PMLR, (2020).
  * [4] Bo Dai and Dahua Lin, ‘Contrastive learning for image captioning’, Advances in Neural Information Processing Systems, 30, (2017).
  * [5] Manoj Reddy Dareddy, Mahashweta Das, and Hao Yang, ‘motif2vec: Motif aware node representation learning for heterogeneous networks’, in 2019 IEEE International Conference on Big Data (Big Data), pp. 1052–1059. IEEE, (2019).
  * [6] Travis Ebesu and Yi Fang, ‘Neural citation network for context-aware citation recommendation’, in Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp. 1093–1096, (2017).
  * [7] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al., ‘Bootstrap your own latent-a new approach to self-supervised learning’, Advances in neural information processing systems, 33, 21271–21284, (2020).
  * [8] Kaveh Hassani and Amir Hosein Khasahmadi, ‘Contrastive multi-view representation learning on graphs’, in International Conference on Machine Learning, pp. 4116–4126. PMLR, (2020).
  * [9] Shuting Jin, Xiangxiang Zeng, Feng Xia, Wei Huang, and Xiangrong Liu, ‘Application of deep learning methods in biological networks’, Briefings in bioinformatics, 22(2), 1902–1917, (2021).
  * [10] Thomas N Kipf and Max Welling, ‘Semi-supervised classification with graph convolutional networks’, arXiv preprint arXiv:1609.02907, (2016).
  * [11] Johannes Klicpera, Stefan Weißenberger, and Stephan Günnemann, ‘Diffusion improves graph learning’, arXiv preprint arXiv:1911.05485, (2019).
  * [12] Andrea Lancichinetti and Santo Fortunato, ‘Benchmarks for testing community detection algorithms on directed and weighted graphs with overlapping communities’, Physical Review E, 80(1), 016118, (2009).
  * [13] Namkyeong Lee, Junseok Lee, and Chanyoung Park, ‘Augmentation-free self-supervised learning on graphs’, in Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 7372–7380, (2022).
  * [14] Haoyang Li, Xin Wang, Ziwei Zhang, Zehuan Yuan, Hang Li, and Wenwu Zhu, ‘Disentangled contrastive learning on graphs’, Advances in Neural Information Processing Systems, 34, 21872–21884, (2021).
  * [15] Shuai Lin, Chen Liu, Pan Zhou, Zi-Yuan Hu, Shuojia Wang, Ruihui Zhao, Yefeng Zheng, Liang Lin, Eric Xing, and Xiaodan Liang, ‘Prototypical graph contrastive learning’, IEEE Transactions on Neural Networks and Learning Systems, (2022).
  * [16] Giulia Muzio, Leslie O’Bray, and Karsten Borgwardt, ‘Biological network analysis with deep learning’, Briefings in bioinformatics, 22(2), 1515–1530, (2021).
  * [17] David F Nettleton, ‘Data mining of social networks represented as graphs’, Computer Science Review, 7, 1–34, (2013).
  * [18] Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd, ‘The pagerank citation ranking: Bringing order to the web.’, Technical report, Stanford InfoLab, (1999).
  * [19] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al., ‘Pytorch: An imperative style, high-performance deep learning library’, Advances in neural information processing systems, 32, (2019).
  * [20] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al., ‘Scikit-learn: Machine learning in python’, the Journal of machine Learning research, 12, 2825–2830, (2011).
  * [21] Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Junzhou Huang, ‘Graph representation learning via graphical mutual information maximization’, in Proceedings of The Web Conference 2020, pp. 259–270, (2020).
  * [22] Aravind Sankar, Xinyang Zhang, and Kevin Chen-Chuan Chang, ‘Motif-based convolutional neural network on graphs’, arXiv preprint arXiv:1711.05697, (2017).
  * [23] Qiaoyu Tan, Ninghao Liu, and Xia Hu, ‘Deep representation learning for social network analysis’, Frontiers in big Data, 2, 2, (2019).
  * [24] Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Mehdi Azabou, Eva L Dyer, Remi Munos, Petar Veličković, and Michal Valko, ‘Large-scale representation learning on graphs via bootstrapping’, arXiv preprint arXiv:2102.06514, (2021).
  * [25] Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola, ‘What makes for good views for contrastive learning?’, Advances in Neural Information Processing Systems, 33, 6827–6839, (2020).
  * [26] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio, ‘Graph attention networks’, arXiv preprint arXiv:1710.10903, (2017).
  * [27] Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm, ‘Deep graph infomax.’, ICLR (Poster), 2(3), 4, (2019).
  * [28] Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, et al., ‘Deep graph library: A graph-centric, highly-performant package for graph neural networks’, arXiv preprint arXiv:1909.01315, (2019).
  * [29] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip, ‘A comprehensive survey on graph neural networks’, IEEE transactions on neural networks and learning systems, 32(1), 4–24, (2020).
  * [30] Jun Xia, Lirong Wu, Ge Wang, Jintao Chen, and Stan Z Li, ‘Progcl: Rethinking hard negative mining in graph contrastive learning’, in International Conference on Machine Learning, pp. 24332–24346. PMLR, (2022).
  * [31] Yaochen Xie, Zhao Xu, Jingtun Zhang, Zhengyang Wang, and Shuiwang Ji, ‘Self-supervised learning of graph neural networks: A unified review’, IEEE Transactions on Pattern Analysis and Machine Intelligence, (2022).
  * [32] Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, and Jian Tang, ‘Self-supervised graph-level representation learning with local and global structure’, in International Conference on Machine Learning, pp. 11548–11558. PMLR, (2021).
  * [33] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen, ‘Graph contrastive learning with augmentations’, Advances in Neural Information Processing Systems, 33, 5812–5823, (2020).
  * [34] Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang, ‘Network representation learning: A survey’, IEEE transactions on Big Data, 6(1), 3–28, (2018).
  * [35] Min-Ling Zhang and Zhi-Hua Zhou, ‘Ml-knn: A lazy learning approach to multi-label learning’, Pattern recognition, 40(7), 2038–2048, (2007).
  * [36] Shichang Zhang, Ziniu Hu, Arjun Subramonian, and Yizhou Sun, ‘Motif-driven contrastive learning of graph representations’, arXiv preprint arXiv:2012.12533, (2020).
  * [37] Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Chee-Kong Lee, ‘Motif-based graph self-supervised learning for molecular property prediction’, Advances in Neural Information Processing Systems, 34, 15870–15882, (2021).
  * [38] Ziyang Zhang, Chuan Chen, Yaomin Chang, Weibo Hu, Xingxing Xing, Yuren Zhou, and Zibin Zheng, ‘Shne: Semantics and homophily preserving network embedding’, IEEE Transactions on Neural Networks and Learning Systems, (2021).
  * [39] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun, ‘Graph neural networks: A review of methods and applications’, AI open, 1, 57–81, (2020).
  * [40] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang, ‘Deep graph contrastive representation learning’, arXiv preprint arXiv:2006.04131, (2020).
  * [41] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang, ‘Graph contrastive learning with adaptive augmentation’, in Proceedings of the Web Conference 2021, pp. 2069–2080, (2021).


# Controlling nonautonomous matter waves in “smart” transient trap variations

S. Sree Ranjani<EMAIL_ADDRESS>Department of Physics, Faculty of
Science and Technology (IcfaiTech), ICFAI Foundation for Higher Education,
Hyderabad, India.  Tangirala Shreecharan<EMAIL_ADDRESS>Department
of Physics, Faculty of Science and Technology (IcfaiTech), ICFAI Foundation
for Higher Education, Hyderabad, India.  Thokala Soloman Raju
<EMAIL_ADDRESS>Department of Physics, National Institute of
Science and Technology, Palur Hills, Brahmapur, Odisha 761008, India.

###### Abstract

In this paper, we study the controllable behavior of nonautonomous matter
waves in different “smart” transient trap variations in the context of the
cigar-shaped Bose-Einstein condensates. By utilizing a self-similarity
transformation we reduce the nonautonomous Gross-Pitaevskii (GP) equation to
the elliptic equation that admits soliton solutions. This procedure leads to a
consistency equation which is in the form of Riccati equation. The connection
between the Riccati and the linear Schrödinger equation, through the Cole-Hopf
transformation, is exploited profitably here to introduce temporal trap
variations. For our study, we explore the possibility of using one dimensional
exactly solvable (ES) potentials and their newly constructed rational
extensions, as functions of time to introduce interesting temporal trap
modulations. The fact that the regular potentials and their rational
extensions being structurally different, leads to different temporal
modulations. It is exhibited that the soliton behavior with respect to
compression in both these cases is quite different.

##  I Introduction

Observation of Bose–Einstein condensates (BECs) in dilute vapors of alkali
atoms is a fascinating phenomenon that has opened up a plethora of
technological applications references needed. Indeed, BECs bose ; MOM ; scin
can be confined to one and two dimensions by applying suitable traps bloch ;
kad ; lecoq . Last few years have seen a flurry of investigations
demonstrating variety of atom traps. This has lead to interesting
configurations of BECs where a two-dimensional configuration refers to BEC on
a chip pas . Similarly in a quasi-one-dimensional scenario billy ; ying ; fat
one obtains a cigar cigar-shaped BECs. In the latter case, bright kha ; car2 ;
lia ; cor ; kon ; boris  and dark stef ; ADJ ; cal  solitons have been
observed in repulsive and attractive coupling regimes, respectively. Here the
bright and dark solitons correspond to rarefactions and density lumps compared
to the background respectively. It is intriguing to note that the
corresponding mean field Gross–Pitaevskii (GP) equation is nothing but the
familiar nonlinear Schrödinger equation (NLSE) in quasi-one dimensions carr1 ,
that admits solitons as solutions due to the delicate balance between
dispersion and nonlinearity.

The coherent control of solitons in cigar-shaped BECs can be achieved by
controlling the variation in the parameters like: scattering length and trap
frequencies. The temporal variation of scattering length due to Feshbach
resonance has been extensively studied in the literature Xi ; zhang ; jing .
The variations in the transverse trap frequency leads to a modulation of the
nonlinearity with time. This has been systematically studied in the context of
Faraday modes Nich ; atre . Recently, Serkin et al have revealed the main
features of nonautonomous matter-wave solitons near the Feshbach resonance in
a one-dimensional Bose-Einstein condensate confined by a harmonic potential
with a varying-in-time longitudinal trapping frequency serkin1 ; serkin2 ;
serkin3 . In the context of optical fiber, nonautonomous solitons and their
dynamical behavior has been studied extensively kruglov ; soloman1 ; soloman2
.

So far in the literature, few studies have shown that the dynamics of the
solitons in BECs can be controlled by modulating the trap frequencies in time
(refe: sree solomon etc). In the present study, we have elucidated the effect
of temporal modulations of the trap, on the ensuing soliton solutions. The
modulations are introduced through a consistency conditions obtained in the
process of mapping nonautonomous GPE to the elliptic equation atre . The
consistency condition is in the form of the well-known Riccati equation and
can be transformed into a second order differential equation of the Strum
Liouville-type using the Cole-Hopf transformation.

One of the well studied Strum Liouville-type differential equations is the
Schrödinger equation. The eigenvalues and eigenfunctions of the latter for one
dimensional potentials, can be exploited in the context of NLSE. In our case,
we consider these potentials as functions of time which can be used to provide
a wide range of temporal trap modulations. In this paper we explore these
connections and more pertinently, exploit the Cole-Hopf transformation and its
intimate connection with the Ricatti and the Schrödinger equations. The
potential functions we are going to consider are ES potentials as functions of
time.

Recently the number of ES potentials in quantum mechanics has seen a large
growth with the construction of rational extensions of the regular ES
potentials. By rational extensions we mean that the regular ES potentials has
additional rational terms. Interestingly the regular and the corresponding
rational potentials are isospectral to each other. It is worth mentionaing
that the regular potentials have the classical orthogonal polynomials as their
eigenfunctions while the rational potentials have exceptional orthogonal
polynomials. To the best of the knowledge of the authors, these rational
potentials as “smart” transient trap variations have been exploited for the
first time, in the context of BECs.

##  II Solutions of the NLSE

A cigar shaped BEC is obtained by confining a dilute Bose gas in a cylindrical
harmonic trap given by

$V=\frac{1}{2}m^{\prime}\omega^{2}_{\bot}(x^{2}+y^{2})+\frac{1}{2}m^{\prime}\omega^{2}_{0}(t)z^{2},$
(1)

with the time dependent confinement in the $z$ direction. The mean field GP
equation reduces to the quasi-one dimensional nonlinear Schrödinger equation
(NLSE) using a Guassian trial wave function atre ; jack ; sal  in a tight
confinement regime, $(\omega_{\perp}>>\omega_{z})$, applied in the $z$
direction

$\imath\partial_{t}\psi=-\frac{1}{2}\partial_{zz}\psi+\gamma(t)|\psi|^{2}\psi+\frac{1}{2}M(t)z^{2}\psi.$
(2)

Where $\gamma(t)=2a_{s}(t)N\hbar\omega_{r}$,
$M(t)=\omega_{0}^{2}(t)/\omega^{2}_{\perp}$,
$a_{\perp}=\hbar/m^{\prime}\omega_{\perp}$, $\omega_{r}$ is the transverse
harmonic oscillator frequency, $a_{s}$ is the scattering length and $N$ is the
total number of particles. The trap potential becomes confining if $M(t)>0$
and repulsive if $M(t)<0$.

The following ansatz is used to obtain the exact solutions of the NLSE

$\psi(z,t)=\sqrt{A(t)}F\\{A(t)[z-\ell(t)]\\}\exp[\imath\Phi(z,t)],$ (3)

where

$\Phi(z,t)=a(t)-\frac{1}{2}c(t)z^{2}$ (4)

with

$a(t)=a_{0}+\frac{\lambda-1}{2}\int_{0}^{t}A^{2}(t^{\prime})dt^{\prime}$ (5)

and $a_{0}$ being a constant. In the above equations $a(t)$ and $c(t)$
represent the phase offset and the phase front curvature of the phase profile
respectively. $A(t)$ and $\ell(t)$ give the amplitude and the location of the
center of mass of the soliton respectively. Substituting the ansatz Eq. (3) of
in Eq. (2) leads to the elliptic differential equation

$F^{\prime\prime}(T)-\lambda F(T)+2\kappa F^{3}(T)=0$ (6)

with $\kappa=-\gamma_{0}/A_{0}$, $T=A(t)(z-\ell(t))$ and the prime denotes
differentiation with respect to $T$. Jacobi elliptic functions han , like
${\rm sn\,}(T,m),{\rm cn\,}(T,m)$ etc with elliptic modulus $m$ appear as
solutions to the above differential equation. Furthermore one obtains the
following consistency conditions for the control parameters

$A(t)=A_{0}\exp\left(\int_{0}^{t}c(t^{\prime})dt^{\prime}\right),\quad\gamma(t)=\gamma_{0}\frac{A(t)}{A_{0}},$
(7) $\frac{d\ell(t)}{dt}+c(t)\ell(t)=0,$ (8) $\frac{dc(t)}{dt}-c^{2}(t)=M(t),$
(9)

with $A_{0}$ and $\gamma_{0}$ being constants.

In this study we look at the dynamics of the bright solition which arises in
the attractive nonlinear regime ($\gamma_{0}<0$) and the limit $m\rightarrow
1$. Under these conditions

$F(T)={\rm cn\,}(T/\tau_{0},m)\rightarrow{\rm sech\,}(T/\tau_{0}).$ (10)

Thus the bright soliton solution is

$\psi(z,t)=\sqrt{A(t)}{\rm sech\,}(T/\tau_{0})\exp[\imath
a(t)-\frac{\imath}{2}c(t)z^{2}],$ (11)

with $\tau_{0}^{2}=-A_{0}/\gamma_{0}$ and $\lambda=-1/\tau_{0}^{2}$.

##  III Temporal modulations of the trap

In the present section we present a method that has been developed and used in
Ref. to generate temporal trap modulations leading to interesting soliton
dynamics. Our starting point is the equation governing the phase front
curvature $c(t)$ given by equation (9). It can be noticed that this is the
Riccati equation. Using the Cole-Hopf transformation:

$c(t)=-\frac{d}{dt}\ln[\phi(t)],$ (12)

in Eq.(9) we obtain

$-\phi^{\prime\prime}(t)-M(t)\phi(t)=0.$ (13)

By choosing the ratio of the trap frequencies as

$M(t)=M_{0}+V(t)$ (14)

Eq. (13) becomes the Schrödinger equation in $t$. This enables us to introduce
a large variety of temporal modulations of the trap through the function
$V(t)$. This is accomplished by choosing $V(t)$ to be of the same functional
form as the one dimensional potential models that one is familiar in quantum
mechanics. We therefore have the advantage of knowing the exact eigenfunctions
which in turn allow us to determone $c(t)$ via equation Eq. (12). In addition
to the aforementioned advantage, Eq. (12) allows us to determine the control
parameters via equations (7) and (8) in terms of $\phi(t)$ as

$A(t)=A_{0}\frac{\phi(0)}{\phi(t)}\quad\gamma(t)=\gamma_{0}\frac{\phi(0)}{\phi(t)}\quad\ell(t)=\ell_{0}\frac{\phi(t)}{\phi(0)},$
(15)

where $\ell_{0}$ is a constant. These control parameters are dependent on time
and can be exploited to control the soliton dynamics.

##  IV Soliton dynamics

In this section, we discuss the spatio-temporal dynamics of BECs for different
trap modulations introduced using the regular potentials and their rational
extensions. This study concentrates on how this rationalization affects the
soliton dynamics. The modulations considered are of the form of the regular
oscillator, Scarf-I potentials and their rational extensions.

We first present the trap and the spatio-temporal dynamics of the bright
soliton, where the ratio of the trap frequencies $M(t)$ is taken as a
constant. We refer to this as an unmodulated trap. $M(t)$ is a constant
implies $V(t)=0$ in Eq. (14), which now takes the form of the free particle
Schroödinger equation in time, with one solution being
$\phi(t)=\exp(\sqrt{M_{0}}t)$. The trap potential in Eq.(2) now becomes
$\frac{1}{2}M_{0}z^{2}$, which is a confining parabolic trap as shown in Figs.
(1) and (3) for $M_{0}=0.001$ and $M_{0}=0.1$ respectively. As can be seen
from these figures, the parameter $M_{0}$ controls the height of the trap.

For constant value of $M_{0}$ Eq. (13) takes the form a free particle
Schrödinger equation whose eigenfunction is $\phi(t)=\exp(\sqrt{M_{0}}\ t)$.
The control parameters for different values of $M_{0}$ are obtained by
substituting $\phi(t)=\exp(\sqrt{M_{0}}\ t)$ in Eq. (15). These control
parameters affect the spatio-temporal dynamics of the bright solitons which
are revealed in Figs. (2) and (4).

It can be seen in Fig. (2), the soliton, corresponding to $M_{0}=0.001$ is
non-singular. For $M_{0}=0.01$ the soliton becomes singular as it evolves as
shown in Fig. (4). As $t$ increases, the soliton localizes further gaining in
amplitude. A possible explanation for the above singular soliton behaviour is
as follows. Traps with bigger height have more energetic condensate atoms as
compared to traps with lower height. Hence the dynamics of the soliton in
traps with more height pick up in amplitude resulting in a singular soliton.
Additionally, we emphasize that the self-focusing effect is responsible for
this singularity weinstein ; gaeta .

Figure 1: Unmodulated trap $M_{0}=0.001$. Figure 2: Soliton solution for the
unmodulated trap with $M_{0}=0.001$ Figure 3: Unmodulated trap $M_{0}=0.01$.
Figure 4: Soliton solution for the unmodulated trap with $M_{0}=0.01$

###  IV.1 Temporal modulations using regular and rational oscillator potential
functions

Here we introduce temporal trap modulations by considering $V(t)$ in the form
of the regular and rational oscillator. The regular oscillator type temporal
modulations are of the form

$V_{\mathrm{reg}}(t)=t^{2}-1$ (16)

while the rational oscillator type modulation is as follows

$V_{\mathrm{rat}}(t)=t^{2}+\frac{16(4t^{2}-2)}{(4t^{2}+2)^{2}}-2.$ (17)

The subscripts reg and rat will be used to indicate functions related to the
regular and the rational type modulations. It can be clearly seen from the
above equations that
$V_{\mathrm{rat}}(t)=V_{\mathrm{reg}}(t)+\mathrm{rational\ terms}$.

The corresponding $\phi(t)$ obtained from (13) for $V_{\mathrm{reg}}(t)$ and
$V_{\mathrm{rat}}(t)$ are

$\phi_{\mathrm{reg}}(t)=\sqrt{\frac{1}{\sqrt{\pi}}}\exp{(-t^{2}/2)}$ (18)

and

$\phi_{\mathrm{rat}}(t)=\sqrt{\frac{8}{\sqrt{\pi}}}\frac{\exp{(-t^{2}/2)}}{4t^{2}+2}$
(19)

respectively. The phase-front curvatures $c(t)$ for the above modulations turn
out to be

$c_{\mathrm{reg}}(t)=t\,\,\,\mathrm{and}\,\,\,c_{\mathrm{rat}}(t)=t+\frac{8t}{4t^{2}+2}.$
(20)

The corresponding modulated traps for $V_{\mathrm{reg}}$ and
$V_{\mathrm{rat}}$ are depicted in the figures (5) and (6) respectively.

The confining traps show initial flattening with the introduction of these
temporal modulations. Though the flattening is pronounced initially for both
the cases, in the rational case, we see that the presence of the rational
terms in fact make the trap repulsive initially, which gradually flattens and
becomes confining again as $t$ increases. We can also see that as $t$
increases both the traps become indistinguishable.

Figure 5: Trap with regular oscillator modulation. Figure 6: Trap with
rational oscillator modulation.

Substitution of equations (18) and (19) in Eq. (15) for these two modulations
will introduce the affect of the modulations into the solutions of the NLSE
given in equation (11). The surface plots of the condensate, for the two types
of modulations are shown in figures (7), (8), (9), (10), (11), (12).

We observe that both these temporal modulations lead to the condensate
undergoing rapid compression as it propagates through the atomic wave guide.
This is clearly visible in the short term behaviour of the condensate in the
Fig. (7) and Fig. (8). Figures (9) and (10) depict the long term behaviour for
the same parameter values. It can be seen that the compression rate in the
case of $V_{rat}(t)$ is faster. In figures (11) and (12), the soliton dynamics
is captured when the center of mass parameter is nonzero. It can be seen that
this leads to the soliton evolving off the symmetry axis of the trap. In
addition $\ell_{0}$ being nonzero impacts the soliton amplitude.

Figure 7: Regular oscillator modulation. Parameter values used are
$A_{0}=0.5$, $\gamma_{0}=-0.5$, $\ell_{0}=0$ Figure 8: Rational oscillator
modulation. Parameter values used are $A_{0}=0.5$, $\gamma_{0}=-0.5$,
$\ell_{0}=0$ Figure 9: Regular oscillator modulation.Parameter values used are
$A_{0}=0.5$, $\gamma_{0}=-0.5$, $\ell_{0}=0$ Figure 10: Rational oscillator
modulation.Parameter values used are $A_{0}=0.5$, $\gamma_{0}=-0.5$,
$\ell_{0}=0$ Figure 11: Regular oscillator modulation. Parameter values used
are $A_{0}=0.5$, $\gamma_{0}=-0.5$, $\ell_{0}=-4$ Figure 12: Rational
oscillator modulation. Parameter values used are $A_{0}=0.5$,
$\gamma_{0}=-0.5$, $\ell_{0}=-4$

###  IV.2 Modulation using Regular and Rational Scarf I potentials

Another set of temporal modulations can be produced using the regular and the
rational Scarf-I potentials which are of the form

$V_{reg}(t)=[\alpha(\alpha-1)+\beta^{2}]\sec^{2}t-\beta(2\alpha-1)\sec t\tan
t$ (21) $\displaystyle V_{rat}(t)$ $\displaystyle=$
$\displaystyle[\alpha(\alpha-1)+\beta^{2}]\sec^{2}t-\beta(2\alpha-1)\sec t\tan
t$ (22) $\displaystyle+$
$\displaystyle\frac{2(2\alpha-1)}{2\alpha-1-2\beta{\rm
sin\,}t}-\frac{2[(2\alpha-1)^{2}-4\beta^{2}]}{(2\alpha-1-2\beta{\rm
sin\,}t)^{2}}$

Here, $\beta<(\alpha-1)$. The traps modulated using equations (21) and (22)
are given in figures respectively. These two modulations are doubly periodic
in time, which deliver kicks of varying intensity at different times to the
condensate.

From the figures (13) and (14) is can be seen that the rational modulation is
distinctly different from the regular one for small $t$.

As in the preceding case $\phi(t)$ for the above modulations are

$\phi_{reg}(t)=(1-{\rm sin\,}t)^{(\alpha-\beta)/2}(1+{\rm
sin\,}t)^{(\alpha+\beta)/2}$ (23)

and

$\displaystyle\phi_{rat}(t)$ $\displaystyle=$ $\displaystyle\frac{(1-{\rm
sin\,}t)^{(\alpha-\beta)/2}(1+{\rm
sin\,}t)^{(\alpha+\beta)/2}}{2\alpha-1-2\beta{\rm sin\,}t}$
$\displaystyle\frac{1}{2(\beta_{1}-\alpha_{1})}\big{[}\alpha_{1}+\beta_{1}+2-(\beta_{1}-\alpha_{1}){\rm
sin\,}t\big{]}.$

In the above $\alpha_{1}=\alpha-\beta-1/2$ and $\beta_{1}=\alpha+\beta-1/2$.
For the sake of completeness we also give the the corresponding phase front
curvatures for the above two modulations

$c_{reg}(t)=\alpha\tan t-\beta\sec t$ (25)

and

$\displaystyle c_{rat}(t)$ $\displaystyle=$ $\displaystyle\alpha\tan
t-\beta\sec t$ $\displaystyle-$ $\displaystyle\frac{2\beta{\rm
cos\,}t}{2\alpha-1-2\beta{\rm sin\,}t}+\frac{2\beta{\rm
cos\,}t}{2\alpha+1-2\beta{\rm sin\,}t}$

Figure 13: Trap with regular Scarf-I modulation $\alpha=6$, $\beta=4.9$.
Figure 14: Trap with rational Scarf-I modulation $\alpha=6$, $\beta=4.9$.
Figure 15: Regular Scarf-I modulation.Parameter values used are $A_{0}=0.5$,
$\gamma_{0}=-0.5$, $\ell_{0}=0$, $\alpha=6$, $\beta=4.9$ Figure 16: Rational
Scarf-I modulation.Parameter values used are $A_{0}=0.5$, $\gamma_{0}=-0.5$,
$\ell_{0}=0$, $\alpha=6$, $\beta=4.9$ Figure 17: Regular Scarf-I modulation.
Parameter values used are $A_{0}=0.5$, $\gamma_{0}=-0.5$, $\ell_{0}=0$,
$\alpha=6$, $\beta=4.9$ Figure 18: Rational Scarf-I modulation. Parameter
values used are $A_{0}=0.5$, $\gamma_{0}=-0.5$, $\ell_{0}=0$, $\alpha=6$,
$\beta=4.9$

From figures (15) and (16) we can see that the long term behaviour of the
soliton is same for both the modulations. The affect of the rational terms can
be seen in figure (18) around $t=1.5$ where the soliton compression is
slightly faster that the regular modulation as shown in Fig. (17). This
behaviour coincides with the behaviour of the trap modulation around $t=1.5$
in figure (14).

##  V Conclusions

In conclusion, we have studied the controllable behavior of nonautonomous
matter waves in different ”smart” transient trap variations in the context of
the cigar-shaped BECs. To accomplish this task, use was made of a novel self-
similarity transformation by which we have reduced the nonautonomous GP
equation to the elliptic equation that admits soliton solutions. This
procedure led to a consistency equation which is in the form of Riccati
equation. The connection between the Riccati and the linear Schrödinger
equation through the Cole-Hopf transformation has been successfully used to
introduce temporal trap variations. For our study, we ave explored the one
dimensional ES potentials and their newly constructed rational extensions, as
functions of time, to introduce interesting temporal trap modulations. As an
application, we delineated the rapid pulse compression experienced by the
solition due to the rational modulations in the form of rational oscillator
and rational Scarf-I. It is observed that the modulations induced by the
rational terms have a definite effect only for small values of $t$,
nevertheless the amplitude of the soliton remains the same for rational as
well as regular modulations for large $t$. We envision that these newly
introduced trap variations may be experimentally realized in the case of
cigar-shaped BEC.

Acknowledgments
TS thanks SERB, India for financial support via Grant: ECR/2015/000081. SSR
acknowledges financial support from SERB, India via Grant: EMR/2016/005002.

## References

  * (1) S. N. Bose, Z. Phys. 26, 178 (1924); A. Einstein, Sitzungsber. Preuss. Akad. Wiss., Phys. Math. Kl. Bericht 3, 18 (1925).
  * (2) M.-O. Mewes, M. R. Andrews, D. M. Kurn, D. S. Durfee, C. G. Townsend, and W. Ketterle, Phys. Rev. Lett. 78, 582 (1997).
  * (3) E. W. Hagley, L. Deng, M. Kozuma, J. Wen, K. Helmerson, S. L. Rolston, and W. D. Phillips Science, 283, 1706 (1999).
  * (4) I. Bloch, T. W. Hänsch, and T. Esslinger, Phys. Rev. Lett. 82, 3008 (1999).
  * (5) D. Kadio and Y. B. Band, Phys. Rev. A 74, 053609 (2006).
  * (6) Y. Le Coq, J.A. Retter, S. Richard, A. Aspect and P. Bouyer, App. Phys. B: Lasers and Optics, 84, 0946 (2006).
  * (7) T. A. Pasquini at. al., Jour. Phys: Conf series, 19, 139 (2005).
  * (8) J. Billy et. al., Annales de Physique 32,2-3, 17 (2007).
  * (9) Y. Wang et. al., Phys. Rev. Lett. 94, 090405 (2005).
  * (10) M. Fattori et. al., Phys. Rev. Lett. 100, 080405 (2008).
  * (11) L. Khaykovich et al., Science 296, 1290 (2002).
  * (12) L. D. Carr and Y. Castin, Phys. Rev. A 66, 063602 (2002); U. Al Khawaja et al., Phys. Rev. Lett. 89, 200404 (2002); K. E. Strecker et al., New J. Phys. 5, 73 (2003).
  * (13) Z. X. Liang, Z. D. Zhang, and W. M. Liu, Phys. Rev. Lett. 94, 050402 (2005); L. Salasnich, Phys. Rev. A 70, 053617 (2004).
  * (14) F. Kh. Abdullaev et. al., Int. Jour. Mod. Phys. B 19, 3415 (2005); S. L. Cornish, S. T. Thompson and C. E. Wieman, Phys. Rev. Lett. 96, 170401 (2006).
  * (15) J. Belmonte-Beitia, V. M. Pérez-García, V. Vekslechik and V. V. Konotop, Phy. Rev. Lett. 100, 164102 (2008).
  * (16) B. A. Malomed et al., J. Opt. B: Quantum Semiclass. Opt. 7, R53 (2005).
  * (17) S. Schmid et. al., New Jour. Phys. 8, 159 (2006).
  * (18) L. D. Carr, C. W. Clark, and W. P. Reinhardt, Phys. Rev. A 62, 063611 (2000).
  * (19) F. S. Cataliotti et al., Science 293, 843 (2001); K. E. Strecker et al., Nature (London) 17, 10 (2002).
  * (20) L. D. Carr and J Brand, Phys. Rev.A 70, 033607 (2004).
  * (21) X. Liu, H. Jing, M. Ge, Chinese Phys. Lett. 23 1184\.
  * (22) W. Zhang, D. F. Walls, and B. C. Sanders, Phys. Rev. Lett. 72, 60 (1994).
  * (23) H. Jing, Int. J. Theo. Phys. 46, 1763 (2007).
  * (24) N. Robins, C. Savage, and E. A. Ostrovskaya, Phys. Rev. A 64, 043605 (2001).
  * (25) R. Atre, P. K. Panigrahi and G. S. Agarwal, Phys. Rev. E 73, 056611 (2006); R. Atre and P. K. Panigrahi, Phys. Rev. A 76, 043838 (2007).
  * (26) V.N. Serkin, A. Hasegawa and T.L. Belyaeva, Phys. Rev. Lett. 98, 074102 (2007).
  * (27) V.N. Serkin, A. Hasegawa and T.L. Belyaeva, J. Mod. Optics 57, 1456 (2010).
  * (28) V.N. Serkin, A. Hasegawa and T.L. Belyaeva, Phys. Rev. A 81, 023610 (2010).
  * (29) V.I. Kruglov, A.C. Peacock, and J.D. Harvey, Phys. Rev. Lett. 90, 113902 (2003).
  * (30) T.S. Raju, P.K. Panigrahi and K. Porsezian, Phys. Rev. E 71, 026608 (2005).
  * (31) T.S. Raju and P.K. Panigrahi, Phys. Rev. A 84, 033807 (2011).
  * (32) M. Weinstein, Commun. Math. Phys. 87, 567 (1983).
  * (33) G. Fibich and A.L. Gaeta, Opt. Lett. 25, 335 (2000).
  * (34) S. Sree Ranjani, Utpal Roy, P. K. Panigrahi and A. K. Kapoor, J. Phys. B 41, 235301 (2008).
  * (35) A. D. Jackson, G. M. Kavoulakis and C. J. Pethick, Phys. Rev. A 58, 2417 (1998).
  * (36) L. Salasnich, A. Parola and L. Reatto, Phys. Rev. A 65, 043614 (2002).
  * (37) H. Hancock, Theory of Elliptic Functions ( Dover, New York, 1958); Handbook of Mathematical Functions, Natl. Bur. Stand. Appl. Math. Ser. No. 55, edited by M. Abromowitz and I. Stegun (U. S. GPO, Washigton, DC, 1964).
  * (38) J. Pappademos , U. Sukhatme and A. Pagnamenta, Phys. Rev. A 48 3525 (1993).
  * (39) E. Tiesinga, B. J. Verhaar and H. T. C. Stoof, Phys. Rev. A 47, 4114 (1993); A. J. Moerdijk, B. J. Verhaar, and A. Axelsson, Phys. Rev. A 51, 4852 (1995).
  * (40) S. Inouye et al., Nature 392, 151 (1998); J. Stenger et al., Phys. Rev. Lett. 82, 2422 (1999).
  * (41) D. B. M. Dickerscheid et al., Phys. Rev. A 71, 043604 (2005).