# Physics-Informed Graph Learning

Ciyuan Peng
Vidya Saikrishna  Institute of Innovation, Science and Sustainability
Federation University Australia
Ballarat, Australia
<EMAIL_ADDRESS>Global Professional School
Federation University Australia
Ballarat, Australia
<EMAIL_ADDRESS>Feng Xia∗
Huan Liu *Corresponding Author Institute of Innovation, Science and
Sustainability
Federation University Australia
Ballarat, Australia
<EMAIL_ADDRESS>School of Computing and Augmented Intelligence
Arizona State University
Tempe, AZ, USA
<EMAIL_ADDRESS>
###### Abstract

An expeditious development of graph learning in recent years has found
innumerable applications in several diversified fields. Of the main associated
challenges are the volume and complexity of graph data. The graph learning
models suffer from the inability to efficiently learn graph information. In
order to indemnify this inefficacy, physics-informed graph learning (PIGL) is
emerging. PIGL incorporates physics rules while performing graph learning,
which has enormous benefits. This paper presents a systematic review of PIGL
methods. We begin with introducing a unified framework of graph learning
models followed by examining existing PIGL methods in relation to the unified
framework. We also discuss several future challenges for PIGL. This survey
paper is expected to stimulate innovative research and development activities
pertaining to PIGL.

###### Index Terms:

graph learning, network representation learning, graph neural networks,
physics, network embedding

##  I Introduction

Graph learning (GL) is a rapidly growing artificial intelligence (AI)
technique that refers to applying machine learning on graph data [1]. It has
attracted great attention in recent years. Many GL algorithms, such as graph
neural networks (GNNs) and random walks [2], have shown great capability to
learn complex relations and dependencies between vertices of graphs, and
capture rich knowledge embedded in graphs [3]. This makes, GL, a primary
approach for graph analysis and also effective for accomplishing various
downstream tasks. In the subsections below, we first discuss the challenges
facing graph learning, followed by some solutions for addressing the
challenges and at the end, we outline the contributions of this survey paper.

###  I-A The Challenges for Graph Learning

Despite the popularity of GL algorithms, a lot of challenges still remain for
GL. We discuss some major challenges below.

The first challenge is concerned with the success of data-driven models. The
performance of data-driven models is highly dependent on the quality and
availability of data. In view of this issue, data processing has become a
crucial step in GL methods. The biggest challenge is to efficiently extract
valid knowledge and information from data deluge [4]. Owing to the complex
nature of graph data, it is difficult to collect data that has no quality
issues [5]. Few examples of quality problems include: false information (e.g.,
the wrong triplet in knowledge graphs), missing data (e.g., missing important
nodes or edges) and data uncertainty (e.g., corrupted or distorted data).
Therefore, imperfection in graph data often leads to the degradation of
performance in GL models.

The second challenge concerns the embedding of graph data into feature space
in which the model tries to learn hidden characteristics of data. The learning
method requires to keep intact the structural information of data [6].
However, many existing GL algorithms cannot fully preserve the structure of
graph data [7]. An example is random negative sampling of nodes [8], because
of which the features of graph structure can easily get destroyed. There is a
consequence in the destruction of data structure during graph representation.

Thirdly, most existing GL models initialize parameters randomly and heavily
rely on manual parameters optimization to achieve better performance. This
could result in costly human efforts and low efficiency of models.

Finally, many GL systems are black boxes or unintelligible to human beings
[9]. Deep learning models generally conduct training without any support of
scientific principles. To achieve trust and transparency, there is a strong
need for explainability or interpretability of the models.

###  I-B Overcoming Challenges with Physics-Informed Graph Learning

Inspired by physical-informed machine learning algorithms, some researchers
have worked on integrating physics principles with GL calling it as physics-
informed graph learning (PIGL). PIGL refers to the graph learning models are
trained from the additional information gained by enforcing physical laws,
such as the distribution rule of continuous space. PIGL solves various
challenges faced by GL[10, 11, 12].

As traditional GL models are incapable of dealing with a variety of problems
caused by complex data, physics-based graph data processing approaches have
emerged. When dealing with imperfect data in data processing step, PIGL models
can combine physical information with data to transform useless data into
valid data [13, 14]. Specially, PIGL intergrates flawed data and mathematical
models reflecting physical laws, and then implements them through graph neural
networks. For example, Yang et al. [15] integrated the Bayesian model with
physics-informed neural networks for quantifying the uncertainty of noisy
data. In addition, by embedding and enforcing the physics principles, physical
models can generate simulation data to enrich sparse data and even pre-train
the GL models to reduce the required amount of data.

In the graph representation process, the structure of graph data is easily
destroyed in the feature space because the relational structure is ignored
[16, 17]. When the layout of network structure is influenced by physical
principles (e.g., repulsive and attractive force), the graph structure can be
preserved in the feature space by utilizing physics constraints to model the
interaction of nodes [18, 19].

It is worth noting that some PIGL models can reasonably initialize model
parameters to improve the efficiency of the model and reduce the model
overhead. The core idea is to physically inform the initial states of the
learning models that satisfy some physics principles [20]. The main method of
physics-based initialization revolves around using synthetic data generated by
physical approaches that are used to pre-train the GL models [21]. In this
way, not only the model parameters can be reasonably initialized, the data
paucity issues can also be alleviated. Moreover, physics principles can
provide scientific theories to elucidate the inner mechanisms of deep learning
[22].

###  I-C Contributions

In this survey, we provide a systematic overview of PIGL methods. To the best
of our knowledge, this is the first survey paper on this emerging field. We
first discuss the unified framework of GL and explain the three main steps
required to implement GL. Then, this paper elucidates the significance of PIGL
models from three aspects: physics-based graph data processing, graph
representation with physical properties, and physics-driven learning models
from the perspective of the unified framework of GL. Following this, the
future directions of PIGL are outlined. The last section finally concludes the
paper.

##  II A Unified Framework of Graph Learning

This section discusses the unified framework of GL (see Fig. 1). The three
main steps to be considered for graph learning are: data processing,
representation, and model training.

Figure 1: The unified framework of graph learning.

###  II-A Data Processing

The data processing step first requires collection of corresponding graph data
based on the needs of tasks and applications. This is normally followed by
data analysis for future processing.

####  II-A1 Graph Data Collection

Unlike unstructured data, graph data represents not only the vertices, but
also the relationships between vertices are included. It is, therefore, more
complex than unstructured data. A vast number of graph databases, such as
TigerGraph111https://www.tigergraph.com/ and Neo4j222https://neo4j.com/, have
been made public for various projects. Also, many benchmark graph datasets are
built for a variety of research. For example, Cora dataset [23], a citation
network, can be used for academic network analysis.

####  II-A2 Graph Theory and Analysis

A graph is represented as $G=(V,E)$, where $V$ indicates the set of nodes
(vertices), and $E$ is the set of edges. $e_{ij}$ is the edge between node
$v_{i}$ and $v_{j}$, denoted as $e_{ij}=(v_{i},v_{j})\in E$. Graphs are
categorosed as: undirected and directed graphs [24]. Directed graphs also
known as digraphs are defined as graphs in which edges have direction. The
directions are usually indicated by arrow on the edges. Undirected graphs are
special case of directed graphs where direction is not mentioned or on other
words the edges are bi-directional.

###  II-B Representation

The second step of GL is graph representation. This step requires extraction
of graph features first and then it is followed by network embedding.

####  II-B1 Graph Feature Extraction

To facilitate graph embedding algorithms, the graph features are first
obtained. Among the features of graph, extraction of node features (e.g.,
research topics of scholars in citation networks) is especially important [25,
26, 27]. For a set of $n$ nodes ${\\{v_{1},v_{2},...,v_{n}}\\}$ with $d$
features, the feature vector of a node $v$ can be indicated as
$x_{v}\in\mathbb{R}^{d}$, and the feature matrix of the graph is
$X\in\mathbb{R}^{n\times d}$. In general, the features of graphs are high
dimensional. Therefore, dimensionality reduction, projecting the high-
dimensional features into a new low-dimensional space, is needed [28].

####  II-B2 Network Embedding

Network embedding (a.k.a graph embedding) aims at finding a mapping function
to translate nodes in the original network into low-dimensional and dense
vector forms [29, 30]. This helps in the node vectors to be easily utilised by
the machine learning models. In the vector space, the relations among nodes
are represented by the distance between node vectors. Some of well-known
widely used network embedding models are random walk, matrix factorization,
and deep neural networks [31].

TABLE I: Some studies of PIGL. Type | Objective | Reference |  Approach
---|---|---|---
Physics-based Graph Data Processing | Sparse data enhancement | [12] |  Utilizing physical sparsity property of data to locate unlabelled data.
Filling missing data | [32] |  Incorporating differentiable physics equations with graph learning.
Noisy value removal | [33] |  Incorporating the physical characteristics with GNNs to capture the noisy values.
Graph Representation with Physical Properties | Precise graph visualisation | [18] |  Combining force-directed graph layout algorithm and deep learning algorithms.
Graph data structure preserving | [7] |  Embedding the principles of force interaction into graph learning models.
Physics-driven Learning Models | Data dependency reduction | [34] |  Generating simulated data to pre-train the driving algorithm.
Reasonable parameter initialization | [11] |  Generating simulated physical variables to pre-train recurrent graph network models.
Accurate and explicable learning | [35] |  Physics-driven loss function of LSTM model.

###  II-C Model Training

The final step of GL is model training, which aims to learn the features of
training data and get augmented with dealing of downstream tasks. To learn the
features of training data, the learning algorithm is first designed. The
training and validation data sets are then used to build and evaluate the
model, respectively. In order to obtain a model with the high performance, the
model should be tuned according to the validation results. Finally, the model
is tested by the test data set.

##  III Physics-Guided Design of Graph Learning Models

In this section, we discuss some representative existing PIGL methods, and
classify these methods by the framework of GL. Firstly, we introduce some
works that are related to physics-based graph data processing. Then, the
physics-inspired graph representation methods are discussed. Finally, we
summarize how to embed physics into the learning algorithms. Table I gives
some representative research on PIGL.

###  III-A Physics-based Graph Data Processing

As graph data is complex and data quality cannot always be guaranteed, the
step of graph data processing is quite challenging [5]. Specifically, the
performance of GL models could seriously be impacted by imperfect data that
includes sparse, noisy and incomplete data. To alleviate this challenge, the
use of physical information to enhance graph data has attracted much attention
recently [36, 37].

Sparse data (e.g., low ratios of labelled data) is one of the most challenging
issues for data collection [38]. Therefore, some studies suggest to combine
physical models to deal with such problems. Representatively, Li and Deka [12]
took use of physical information contented in observation data to achieve
sparse data enhancement and proposed a physics-informed graph neural network
model to locate faults in power grids. The power grid resembles a graph, a
fault at one point in the power grid results in abnormal data (voltage,
current), and the faulty position is regarded as the label of the fault. To
detect the faults in power grids using data-driven models, correctly labelled
data sets are required. However, the collected power grid data sets are
usually insufficiently labelled data because of the complexity and variability
of the power grid. To better address this problem, the physical sparsity
property of data is leveraged to reduce the requirements on labelled data.

Consider a power grid as a graph with $n$ nodes.
$\mathbf{Y}\in\mathbb{C}^{3\times 3}$ is the admittance matrix. When there is
no fault, the voltages and currents of all the points in the power grid are in
normal conditions, denoted as $\mathbf{U}_{0}\in\mathbb{C}^{3\times 1}$ and
$\mathbf{C}_{0}\in\mathbb{C}^{3\times 1}$, respectively. The formula
$\mathbf{Y}\mathbf{U}_{0}=\mathbf{C}_{0}$ can be satisfied. While if there is
a fault at node $f$, which is between the nodes $i$ and $j$, the voltages and
currents of the power grid will be changed. The real voltages and currents are
$\mathbf{U}$ and $\mathbf{C}$, and according to the Kirchhoff’s law and the
substitution theory [39], the following formula holds:

$\mathbf{YU}-\Delta_{ij}=\mathbf{C}.$ (1)

Here, $\Delta_{ij}\in\mathbb{C}^{3n}$ is a sparse vector with nonzero values
corresponding to the two nodes $i$ and $j$. Then, according to physical laws
of conservation, let $\Delta\mathbf{U}=\mathbf{U}-\mathbf{U}_{0}$ and
$\Delta\mathbf{C}=\mathbf{C}-\mathbf{C}_{0}$. Thereby, the Equation 1 can be
interpreted as:

$\mathbf{Y}\Delta\mathbf{U}=\Delta\mathbf{C}+\Delta_{ij}.$ (2)

The summation of $\Delta\mathbf{C}$ and $\Delta_{ij}$ is the physical
interpretation of the linear combinations. Due to the non-zero values in
$\Delta_{ij}$, at nodes $i$ and $j$, the dominant entries of
$\Delta\mathbf{C}+\Delta_{ij}$ are closely related to the fault location $f$.
Therefore, when the labelled faults in the observed data are lacking, this
physical information of voltages and currents can help to estimate the
unlabelled faults and enhance the sparse data. Especially, utilizing the
physical similarity of unlabelled and labelled data to transform useless
sparse data into valid data is a trending topic for graph data processing
[40].

Li and Deka [12] implemented their model in the 123-node test feeder [41].
Three performance metrics including location accuracy rate ($LAR$),
$LAR^{1-hop}$, and F1-score were adapted. Compared with three well-known
baselines: neural network, convolutional neural network, and graph
convolutional network, their model showed better locates various faults at low
ratios of labeled data. Thereby, it shows the physics-based graph data
processing can enhance the sparse data and improve the model performance.

In addition to overcoming the sparse data problems, other common problems
associated with data processing (e.g., noisy values and missing data) are also
expected to be solved by physical methods. For example, Seo and Liu [32]
presented DPGN (differentiable physics-informed graph networks) model
incorporating differentiable physics equations with the graph learning. They
transformed physical information in the physical model into usable data to
fill the missing values. Therefore, the known physical rules are used to
compensate for incomplete graph data caused by inadequate observations.

Salehi and Giannacopoulos [33] proposed PhysGNN model, which combines the
physical characteristics of human soft tissue and graph neural networks, to
analyse the preoperative data and guide the neurosurgical procedures. The
preoperative data has the property to constantly change and be imprecise. This
results in a lot of noise being present in the data set. They took advantage
of the physical characteristics of human soft tissue to capture the noisy
values, so that the data set can be more accurate.

###  III-B Graph Representation with Physical Properties

In graph representation step, to preserve the structure of graph data in the
feature vector space, some studies import physical constraints when embedding
data into low-dimensional space. In particular, adding physical properties to
the input feature vector can ensure that the graph structure is not damaged to
a certain extent and the data features are better preserved in the vector
space.

In the field of graph visualisation [42], to keep the original topology of
graphs, researchers begin to utilize the physical interaction between nodes in
the graph to force the graph structures to be undamaged in the two or three
dimensions [43]. For instance, Haleem et al. [18] combined a force-directed
graph layout algorithm with deep learning algorithms to visualize the
networks. The approach suggested has the advantage of preserving the original
graph structure by observing the physical attraction and repulsion between the
nodes induced by force-directed graph layout algorithm.

Figure 2: The example of graph representation considering physical properties.
The force interaction of nodes (repulsive and attractive force) is presented
to preserve the data structure in the feature space.

By the application of physical principles, such as repulsive and attractive
force, on the layout of network structures, Sun et al [7] proposed a GForce
model. GForce is a graph learning model embedding the principles of force
interaction related to physics. This helps in fully preserving the data
structure in the vector space. Sun et al suggested that if the connected nodes
have similarities, they are close to each other, otherwise, they are far from
each other. However, in the feature space, the embedding vector of nodes may
not represent the same structure as in the original graph. Thereby, inspired
by the spring-electrical model [44], they considered that there is a repulsive
force between each node, and each node is attractive to its neighbours which
are connected to it. In the feature space, due to the attraction, two
connected nodes are relatively closer to each other, while unconnected nodes
are far away from each other because of repulsion. Fig. 2 is an example of
physics-based graph representation considering the force interaction of nodes.
The graph structure can be well preserved in the feature space.

In the method suggested, the main step is importing attractive and repulsive
relations, respectively. Given an undirected graph $G={(V,E)}$, after
embedding graph data into feature space, the feature vectors of the graph can
be denoted as $\mathbf{H}\in R^{|V|\times n}$. Here, the dimension of vector
space is $n$ and satisfies $n\gg 3$. The feature vector $\mathbf{H}$ is
updated by importing attractive and repulsive relations to the feature
vectors:

$\mathbf{H}=\mathbf{H}+u\ast(\nabla F_{a}+\nabla F_{r}),$ (3)

here, $\nabla F_{a}$ and $\nabla F_{r}$ are the update of attractive and
repulsive relations, respectively. $u$ is a parameter which is used to control
the learning speed. The distance between nodes $i$ and $j$ in the vector space
is $d_{ij}=\mathbf{h}_{i}-\mathbf{h}_{j}$, here
$\mathbf{h}_{i},\mathbf{h}_{j}\in\mathbf{H}$. The attractive relation of node
$i$ to node $j$ is updated as the following:

$\nabla f_{a}(i)=-p\ast\widehat{w}_{ij}\ast d_{ij},$ (4)

where $p$ indicates the learning rate, and $\widehat{w}_{ij}\in\widehat{W}$ is
the positive weight between nodes $i$ and $j$. On the other hand, the update
of repulsive relation between node $i$ and node $j$ is as the following:

$\nabla f_{r}(i)=q\ast\widetilde{w}_{ij}\ast\frac{d_{ij}}{||d_{ij}|+b|^{2}},$
(5)

where $q$ indicates the learning rate, $b$ denotes the distance bias, and
$\widetilde{w}_{ij}\in\widetilde{W}$ is the negative weight between nodes $i$
and $j$.

After importing the attractive relations, the connected nodes (or similar
nodes) can be closer to each other in the feature vector space, while
unconnected nodes (or different nodes) are far away from each other after
importing repulsive relations. Sun et al [7] evaluated their model on five
datasets including Wikidata333https://www.wikidata.org/
Cora444https://relational.fit.cvut.cz/dataset/CORA,
Citeseer555https://relational.fit.cvut.cz/dataset/CiteSeer, 20
Newsgroups666http://qwone.com/ jason/20Newsgroups/, and
WebKB777http://www.cs.cmu.edu/ webkb/. Compared with various state-of-the-art
baselines, such as SDNE [45], GForce showed better capacity in preserving the
original structural information in low-dimensional feature spaces. It
indicates that physics-inspired graph representation methods can effectively
preserve the graph data structure in vector spaces.

###  III-C Physics-Driven Learning Models

The step after representing training data in feature space, is model training.
Specific learning algorithm needs to be identified to gain knowledge from
training data and the trained model is then used in tasks such as
classification and detection. There are still some challenges associated with
the learning aspect of the models [46, 47]. The challenges are for instance:
loosing efficiency of the models by initialising using random parameters,
model degradation by the imprecise loss function, lack of interpretation in
deep graph learning models. The challenges are mitigated by physics-driven
learning models proposed by researchers [48].

In this section, we introduce some advanced works focusing on physics-driven
learning models. In particular, we discuss how to reduce the overhead of
models by initializing the parameters physically. Then, we give some examples
of studies that improve the performance of learning models by adding physics
laws to the loss functions.

####  III-C1 Parameter Initialization

To improve the efficiency of training models and reduce the model overhead,
recent works are carried out on informing the initial state of training models
by satisfying some physical principles [49, 50]. The main method is to use
physical approaches to generate simulated data, which is subsequently used to
pre-train the GL models [51]. Therefore, the parameters of GL models can be
reasonably initialized rather than getting randomly selected. The obvious
advantage is that the problem associated with data paucity can be controlled
in this way.

Jia et al [11] presented a physical technique to initialize recurrent graph
network model, and then the pre-trained model is applied to predict the water
temperature and flow in river networks. They first used a physical model to
generate simulated target variables and intermediate physical variables. Then,
the simulated variables were used to initialize and tune the GL model. The
physical model they used is an energy balance model, which is based on the
physics of thermodynamics and stream-flow. For example, the simulated
temperature variables are generated according to the energy fluxes in the
river segments. They took advantage of the relations between incoming and
outgoing energy in the river segments to capture the temperature change. The
temperature change conforms to the following equation:

$\Delta T\propto E_{in}-E_{out}+E_{up},$ (6)

where $\Delta T$ indicates the simulated temperature change, $E_{in}$ is the
incoming energy fluxes, such as rainfall and solar radiation, while $E_{out}$
indicates the outgoing energy fluxes, such as evaporation. $E_{up}$ is the net
heat, which is advocated into the current river segment from the upstream
segments.

After using simulated variables to pre-train the recurrent graph network
model, they collected the real river segments data from the river network to
train the model. Compared with other baselines, the model is more
generalizable because the physical relations between river segments are
enforced. Also, the pre-trained model will be more efficient when training is
done with real data.

In addition, the simulated data can also help to alleviate the data paucity
issues and reduce the data dependency of models. For instance, in the computer
vision area, the simulated images have shown great ability to deal with real-
world tasks including object localization. Shah et al. [34] proposed a method
using a simulator, which is built on a video physical engine, that is used to
pre-train the driving algorithm. Significantly, their method allows for less
volume of training data.

####  III-C2 Loss Function

(a) Traditional architecture of neural networks.

(b) Architecture of physics-driven neural networks.

Figure 3: Comparison of traditional neural networks and physics-driven neural
networks. As shown in (a), due to the black-box property, neural networks are
unexplainable. In contrast, (b) is the neural networks infusing physics
through physical intermediate variables $z$ or physics-guided connections
among neurons.

Data-driven models ignore the influence of physical variables and therefore,
the standard GL model fails to gain advantage from the physical relationships
of data. Numerous scientific problems emphasize the necessity to include
physical constraints to explain the physical relationships of entities in data
[52]. Therefore, to help GL models capture physical relationships and
generalizable dynamic patterns which is consistent with known physical laws,
researchers incorporate physical knowledge (e.g., physical intermediate
variables) into the loss functions of GL models [53]. The physical laws-
informed loss function is expressed as follows:

$\displaystyle\mathcal{L}=\mathcal{L}_{t}(Y_{t},Y_{p})+\lambda
R(W)+\gamma\mathcal{L}_{p}(Y_{p}),$ (7)

here, the training loss $\mathcal{L}_{t}$ measures errors between true values
$Y_{t}$ and predicted values $Y_{p}$. The trade-off parameter $\lambda$ is to
control the weight of regularization loss $R(W)$. $\mathcal{L}_{t}$ and
$\lambda R(W)$ are two terms of standard loss of learning models, while
$\mathcal{L}_{p}$ denotes the physics-based loss. Particularly,
$\mathcal{L}_{p}$ is to ensure the predicted values $Y_{p}$ consistent with
physical laws. $\gamma$ is the hyper-parameter weighted $\mathcal{L}_{p}$.

Daw et al [35] presented a physics-guided loss function of the Long Short-Term
Memory (LSTM) model to achieve accurate lake temperature prediction. Standard
data-driven models typically only take the depth of water in lake data, that
is, the position of nodes in graph data, as the main impact factor in
temperature prediction. However, some other variables in lakes also have
physical correlations with temperature. Among them, the temperature is closely
related to density. That means temperature varies with density. Hence, in
their method, not only the target temperature values $Y$ are considered, but
also the physical intermediate values, density $Z$, are produced to build the
loss function. They minimized the empirical loss over $Y$ and $Z$:

$\mathop{\arg\min}\mathcal{L}(Y_{t},Y_{p})+\lambda
R(W)+\gamma\mathcal{L}_{p}(Z_{t},Z_{p}).$ (8)

Here, $Y_{t}$ and $Z_{t}$ denote the observed values of temperature and
density, respectively. $Y_{p}$ and $Z_{p}$ are output temperature and
ancillary output density of the model.

Daw et al [35] tested their model on two real lake datasets: Lake Mendota and
Falling Creek Reservoir. Their proposed model outperformed other LSTM-based
methods when taking the root mean square error (RMSE) of models on the test
datasets and physical inconsistency of lake data as the two evaluation
metrics.

As time or space changes, the observed data is affected by some physical
factors. Therefore, compared with the general models, the models with the loss
functions adding physical principles can have better performance. Moreover,
physical laws provide reliable scientific theories to interpret neural
networks [54]. Thus, the explicable deep learning models can be significantly
applied to scientific areas. Fig. 3 shows the comparison of traditional neural
networks and physics-driven neural networks.

##  IV Open Challenges for PIGL

Physics-informed graph learning expedited as more and more research works
started incorporating physics models into graph learning models. Despite its
obvious advantages, it still suffers from several challenges. This section
briefly reviews some major challenges faced by PIGL.

###  IV-A Cross-disciplinary Collaboration

The integration of GL (AI and machine learning in general) and physics
principles creates unprecedented potentials for addressing challenges that
face GL, including data efficiency, trust and transparency. The realization of
these potentials demands effective interaction and collaboration among
multiple disciplines. Most of current techniques for integrating GL and
physics were developed by researchers from distinct disciplines and for
isolated applications [55, 56]. To diminish this disciplinary gap, the
pollination of research ideas and paradigms across diverse disciplines needs
to be fostered.

###  IV-B Physics-driven Co-design

To alleviate specific problems such as data issues and model robustness, the
existing PIGL models focus on integrating physics into a certain step of GL.
It might be worthwhile to embed interrelated physics at multiple steps. An
example is to use cross-layer physical principles to deal with multiple
problems [12], for example, sparse data and data structure destruction. As
such, the problems can be alleviated by taking advantage of relevant physics
laws in multiple steps. In view of the advantages, an innovative PIGL design
pattern/model that feature co-design of multiple (physics-based) modules may
be devised.

###  IV-C Hybrid Systems

Machine learning concepts are based on discrete time and space constraints. On
the other hand, the physical concepts are realised in continuous time and
space constraints [57]. As the traditional graph learning approach is based on
discreteness of the feature space, it is a tremendous challenge for the GL
model to deal with tasks in real time constraints effectively. Therefore, by
importing the concept of continuous space-time into PIGL, the performance can
be drastically improved. In other words, the PIGL can be regarded as a hybrid
system, which combines both continuous and discrete systems [12]. Due to the
complex nature of hybrid systems, it is important to understand that,
effective blending of continuous and discrete plays a major role in the
performance of the PIGL models. An example could be demonstrating the changes
in physical variables over time and space without breaking the properties of
the data.

###  IV-D Data Discrepancy

The process of many PIGL methods is to first use physical models to generate
synthetic data, and then train GL models by using the synthetic data. One
obvious advantage associated with using synthetic data is that it can easily
tackle data deficiency and missing problems [58]. In addition, synthetic data
can be used to pre-train the GL models to initialize model parameters.
However, due to the influence of various objective factors, the actual
observation data could be inconsistent with the ideal state and so the ideal
synthetic data might deviate from the actual observation data [59]. Because of
the discrepancy, the model trained with synthetic data will have errors when
processing real tasks with real data. Therefore, one research pointer can be
directed towards improving the robustness of PIGL models in the presence of
discrepancy between the synthetic and actual data.

###  IV-E Multi-domain Applications

PIGL models have been applied to some specific tasks, such as power grid
maintenance and water temperature detection. However, most of the current
studies have a limited range of applications in scientific areas. Expanding
the horizon of PIGL into diversified fields such as finance and social
networks [60] by taking full advantage of its potential is another promising
research direction. An example could be, importing physical constraints into
the social network data, and then using the processed data to train the GL
model for completing assigned tasks such as relationship recognition in social
networks [26].

##  V Conclusion

Graph learning is one of the powerful AI techniques, attracting a great deal
of attention from both academia and industry. Researchers have been working
constantly on addressing challenges such as processing of complex graph data
and solving the black-box property of deep learning. It could be a promising
way to solve these problems by incorporating physics principles with GL. In
principle, various physical laws can be effectively embedded into one or
multiple steps of GL, yielding improved performance. To the best of our
knowledge, this survey paper is the first review of PIGL. We expect that this
survey paper will spark new interest in physics-informed design of graph
learning systems.

## References

  * [1] F. Xia, K. Sun, S. Yu, A. Aziz, L. Wan, S. Pan, and H. Liu, “Graph learning: A survey,” _IEEE Transactions on Artificial Intelligence_ , vol. 2, no. 2, pp. 109–127, 2021.
  * [2] F. Xia, J. Liu, H. Nie, Y. Fu, L. Wan, and X. Kong, “Random walks: A review of algorithms and applications,” _IEEE Transactions on Emerging Topics in Computational Intelligence_ , vol. 4, no. 2, pp. 95–107, 2019.
  * [3] S. Wang, L. Hu, Y. Wang, X. He, Q. Z. Sheng, M. A. Orgun, L. Cao, F. Ricci, and P. S. Yu, “Graph learning based recommender systems: A review,” _arXiv preprint arXiv:2105.06339_ , 2021.
  * [4] M. A. Chao, C. Kulkarni, K. Goebel, and O. Fink, “Fusing physics-based and deep learning models for prognostics,” _Reliability Engineering & System Safety_, vol. 217, p. 107961, 2022.
  * [5] S. Yu, H. Huang, M. N. Dao, and F. Xia, “Graph augmentation learning,” in _Companion Proceedings of the Web Conference 2022_ , ser. WWW ’22, 2022, p. 1063–1072.
  * [6] P. Wang, Y. Fu, Y. Zhou, K. Liu, X. Li, and K. A. Hua, “Exploiting mutual information for substructure-aware graph representation learning.” in _IJCAI_ , 2020, pp. 3415–3421.
  * [7] K. Sun, J. Liu, S. Yu, B. Xu, and F. Xia, “Graph force learning,” in _2020 IEEE International Conference on Big Data (Big Data)_. IEEE, 2020, pp. 2987–2994.
  * [8] A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in _Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining_ , 2016, pp. 855–864.
  * [9] R. Henderson, D.-A. Clevert, and F. Montanari, “Improving molecular graph neural network explainability with orthonormalization and induced sparsity,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 4203–4213.
  * [10] K. Yan, X. Chen, X. Zhou, Z. Yan, and J. Ma, “Physical model informed fault detection and diagnosis of air handling units based on transformer generative adversarial network,” _IEEE Transactions on Industrial Informatics_ , 2022\.
  * [11] X. Jia, J. Zwart, J. Sadler, A. Appling, S. Oliver, S. Markstrom, J. Willard, S. Xu, M. Steinbach, J. Read _et al._ , “Physics-guided recurrent graph model for predicting flow and temperature in river networks,” in _Proceedings of the 2021 SIAM International Conference on Data Mining (SDM)_. SIAM, 2021, pp. 612–620.
  * [12] W. Li and D. Deka, “Physics-informed graph learning for robust fault location in distribution systems,” _arXiv e-prints_ , pp. arXiv–2107, 2021.
  * [13] A. L. da Silva, F. Kocayusufoglu, S. Jafarpour, F. Bullo, A. Swami, and A. Singh, “Combining physics and machine learning for network flow estimation,” in _International Conference on Learning Representations_ , 2020\.
  * [14] Z. Xiang, M. Fan, G. V. Tovar, W. Trehern, B.-J. Yoon, X. Qian, R. Arroyave, and X. Qian, “Physics-constrained automatic feature engineering for predictive modeling in materials science,” in _Proc. AAAI Conf. Artif. Intell_ , vol. 35, no. 12, 2021, pp. 10 414–10 421.
  * [15] L. Yang, X. Meng, and G. E. Karniadakis, “B-pinns: Bayesian physics-informed neural networks for forward and inverse pde problems with noisy data,” _Journal of Computational Physics_ , vol. 425, p. 109913, 2021.
  * [16] J. Xu, S. Yu, K. Sun, J. Ren, I. Lee, S. Pan, and F. Xia, “Multivariate relations aggregation learning in social networks,” in _Proceedings of the ACM/IEEE Joint Conference on Digital Libraries_ , ser. JCDL’20, 2020, p. 77–86.
  * [17] Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph wavenet for deep spatial-temporal graph modeling,” in _Proceedings of the 28th International Joint Conference on Artificial Intelligence_ , ser. IJCAI’19, 2019, p. 1907–1913.
  * [18] H. Haleem, Y. Wang, A. Puri, S. Wadhwa, and H. Qu, “Evaluating the readability of force directed graph layouts: A deep learning approach,” _IEEE computer graphics and applications_ , vol. 39, no. 4, pp. 40–53, 2019.
  * [19] S. Yu, F. Xia, and H. Liu, “Academic team formulation based on liebig’s barrel: Discovery of anticask effect,” _IEEE Transactions on Computational Social Systems_ , vol. 6, no. 5, pp. 1083–1094, 2019.
  * [20] J. Willard, X. Jia, S. Xu, M. Steinbach, and V. Kumar, “Integrating physics-based modeling with machine learning: A survey,” _arXiv preprint arXiv:2003.04919_ , vol. 1, no. 1, pp. 1–34, 2020.
  * [21] M. M. Almajid and M. O. Abu-Al-Saud, “Prediction of porous media fluid flow using physics informed neural networks,” _Journal of Petroleum Science and Engineering_ , vol. 208, p. 109205, 2022.
  * [22] M. Geiger, A. Jacot, S. Spigler, F. Gabriel, L. Sagun, S. d’Ascoli, G. Biroli, C. Hongler, and M. Wyart, “Scaling description of generalization with number of parameters in deep learning,” _Journal of Statistical Mechanics: Theory and Experiment_ , vol. 2020, no. 2, p. 023401, 2020.
  * [23] C. Cabanes, A. Grouazel, K. von Schuckmann, M. Hamon, V. Turpin, C. Coatanoan, F. Paris, S. Guinehut, C. Boone, N. Ferry _et al._ , “The cora dataset: validation and diagnostics of in-situ ocean temperature and salinity measurements,” _Ocean Science_ , vol. 9, no. 1, pp. 1–18, 2013.
  * [24] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” _IEEE transactions on neural networks and learning systems_ , vol. 32, no. 1, pp. 4–24, 2020.
  * [25] K. Sun, L. Wang, B. Xu, W. Zhao, S. W. Teng, and F. Xia, “Network representation learning: From traditional feature learning to deep learning,” _IEEE Access_ , vol. 8, pp. 205 600–205 617, 2020.
  * [26] J. Liu, F. Xia, L. Wang, B. Xu, X. Kong, H. Tong, and I. King, “Shifu2: A network representation learning based model for advisor-advisee relationship mining,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 33, no. 4, pp. 1763–1777, 2021.
  * [27] F. Xia, J. Wang, X. Kong, D. Zhang, and Z. Wang, “Ranking station importance with human mobility patterns using subway network datasets,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 21, no. 7, pp. 2840–2852, 2019.
  * [28] J. Lu, H. Wang, J. Zhou, Y. Chen, Z. Lai, and Q. Hu, “Low-rank adaptive graph embedding for unsupervised feature extraction,” _Pattern Recognition_ , vol. 113, p. 107758, 2021.
  * [29] G. Xue, M. Zhong, J. Li, J. Chen, C. Zhai, and R. Kong, “Dynamic network embedding survey,” _Neurocomputing_ , vol. 472, pp. 212–223, 2022.
  * [30] M. Hou, J. Ren, D. Zhang, X. Kong, D. Zhang, and F. Xia, “Network embedding: Taxonomies, frameworks and applications,” _Computer Science Review_ , vol. 38, p. 100296, 2020.
  * [31] P. Cui, X. Wang, J. Pei, and W. Zhu, “A survey on network embedding,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 31, no. 5, pp. 833–852, 2018.
  * [32] S. Seo and Y. Liu, “Differentiable physics-informed graph networks,” _arXiv preprint arXiv:1902.02950_ , 2019.
  * [33] Y. Salehi and D. Giannacopoulos, “Physgnn: A physics-driven graph neural network based model for predicting soft tissue deformation in image-guided neurosurgery,” _arXiv preprint arXiv:2109.04352_ , 2021.
  * [34] S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Airsim: High-fidelity visual and physical simulation for autonomous vehicles,” in _Field and service robotics_. Springer, 2018, pp. 621–635.
  * [35] A. Daw, R. Q. Thomas, C. C. Carey, J. S. Read, A. P. Appling, and A. Karpatne, “Physics-guided architecture (pga) of neural networks for quantifying uncertainty in lake temperature modeling,” in _Proceedings of the 2020 siam international conference on data mining_. SIAM, 2020, pp. 532–540.
  * [36] V. Iakovlev, M. Heinonen, and H. Lähdesmäki, “Learning continuous-time pdes from sparse data with graph neural networks,” _arXiv preprint arXiv:2006.08956_ , 2020.
  * [37] F. Sun, Y. Liu, and H. Sun, “Physics-informed spline learning for nonlinear dynamics discovery,” _arXiv preprint arXiv:2105.02368_ , 2021.
  * [38] J. Liu, F. Xia, X. Feng, J. Ren, and H. Liu, “Deep graph learning for anomalous citation detection,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2022.
  * [39] M. Majidi, A. Arabali, and M. Etezadi-Amoli, “Fault location in distribution networks by compressive sensing,” _IEEE Transactions on Power Delivery_ , vol. 30, no. 4, pp. 1761–1769, 2014.
  * [40] Z. Zhang, “A physics-informed deep convolutional neural network for simulating and predicting transient darcy flows in heterogeneous reservoirs without labeled data,” _Journal of Petroleum Science and Engineering_ , p. 110179, 2022.
  * [41] W. H. Kersting, “Radial distribution test feeders,” _IEEE Transactions on Power Systems_ , vol. 6, no. 3, pp. 975–985, 1991.
  * [42] J. Liu, T. Tang, W. Wang, B. Xu, X. Kong, and F. Xia, “A survey of scholarly data visualization,” _Ieee Access_ , vol. 6, pp. 19 205–19 221, 2018.
  * [43] A. Arleo, W. Didimo, G. Liotta, and F. Montecchiani, “A distributed multilevel force-directed algorithm,” in _International Symposium on Graph Drawing and Network Visualization_. Springer, 2016, pp. 3–17.
  * [44] M. E. Ko, C. M. Williams, K. I. Fread, S. M. Goggin, R. S. Rustagi, G. K. Fragiadakis, G. P. Nolan, and E. R. Zunder, “Flow-map: a graph-based, force-directed layout algorithm for trajectory mapping in single-cell time course datasets,” _Nature protocols_ , vol. 15, no. 2, pp. 398–420, 2020\.
  * [45] D. Wang, P. Cui, and W. Zhu, “Structural deep network embedding,” in _Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining_ , 2016, pp. 1225–1234.
  * [46] F. Xia, L. Wang, T. Tang, X. Chen, X. Kong, G. Oatley, and I. King, “Cengcn: Centralized convolutional networks with vertex imbalance for scale-free graphs,” _IEEE Transactions on Knowledge and Data Engineering_ , 2022.
  * [47] F. Xia, S. Yu, C. Liu, J. Li, and I. Lee, “Chief: Clustering with higher-order motifs in big networks,” _IEEE Transactions on Network Science and Engineering_ , vol. 9, no. 3, pp. 990–1005, 2022.
  * [48] Y. Alanazi, N. Sato, P. Ambrozewicz, A. N. H. Blin, W. Melnitchouk, M. Battaglieri, T. Liu, and Y. Li, “A survey of machine learning-based physics event generation,” _arXiv preprint arXiv:2106.00643_ , 2021.
  * [49] R. Shi, Z. Mo, and X. Di, “Physics-informed deep learning for traffic state estimation: A hybrid paradigm informed by second-order traffic models,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 35, no. 1, 2021, pp. 540–547.
  * [50] X. Jia, J. Willard, A. Karpatne, J. S. Read, J. A. Zwart, M. Steinbach, and V. Kumar, “Physics-guided machine learning for scientific discovery: An application in simulating lake temperature profiles,” _ACM/IMS Transactions on Data Science_ , vol. 2, no. 3, pp. 1–26, 2021.
  * [51] S. Schoenholz and E. D. Cubuk, “Jax md: a framework for differentiable physics,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 11 428–11 441, 2020.
  * [52] Y. Li, J. Wang, Z. Huang, and R. X. Gao, “Physics-informed meta learning for machining tool wear prediction,” _Journal of Manufacturing Systems_ , vol. 62, pp. 17–27, 2022.
  * [53] M. J. Schuetz, J. K. Brubaker, and H. G. Katzgraber, “Combinatorial optimization with physics-inspired graph neural networks,” _arXiv preprint arXiv:2107.01188_ , 2021.
  * [54] M. Russell and P. Wang, “Physics-informed deep learning for signal compression and reconstruction of big data in industrial condition monitoring,” _Mechanical Systems and Signal Processing_ , vol. 168, p. 108709, 2022.
  * [55] J. Park and J. Park, “Physics-induced graph neural network: An application to wind-farm power estimation,” _Energy_ , vol. 187, p. 115883, 2019.
  * [56] X. Zhou, S. Li, Z. Li, and W. Li, “Information diffusion across cyber-physical-social systems in smart city: A survey,” _Neurocomputing_ , vol. 444, pp. 203–213, 2021.
  * [57] T. Matsubara, A. Ishikawa, and T. Yaguchi, “Deep energy-based modeling of discrete-time physics,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 13 100–13 111, 2020.
  * [58] X. Chen, M. Hou, T. Tang, A. Kaur, and F. Xia, “Digital twin mobility profiling: A spatio-temporal graph learning approach,” in _7th Int Conf on Data Science and Systems_ , 2021, pp. 1178–1187.
  * [59] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, “Physics-informed machine learning,” _Nature Reviews Physics_ , vol. 3, no. 6, pp. 422–440, 2021.
  * [60] J. Zhang, W. Wang, F. Xia, Y.-R. Lin, and H. Tong, “Data-driven computational social science: A survey,” _Big Data Research_ , p. 100145, 2020.