# ARC: A Generalist Graph Anomaly Detector
with In-Context Learning

Yixin Liu1,∗, Shiyuan Li2,, Yu Zheng3, Qingfeng Chen2, Chengqi Zhang4, Shirui
Pan5
1Monash University, 2Guangxi University, 3La Trobe University,
4University of Technology Sydney, 5Griffith University
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Equal
Contribution.

###### Abstract

Graph anomaly detection (GAD), which aims to identify abnormal nodes that
differ from the majority within a graph, has garnered significant attention.
However, current GAD methods necessitate training specific to each dataset,
resulting in high training costs, substantial data requirements, and limited
generalizability when being applied to new datasets and domains. To address
these limitations, this paper proposes ARC, a generalist GAD approach that
enables a “one-for-all” GAD model to detect anomalies across various graph
datasets on-the-fly. Equipped with in-context learning, ARC can directly
extract dataset-specific patterns from the target dataset using few-shot
normal samples at the inference stage, without the need for retraining or
fine-tuning on the target dataset. ARC comprises three components that are
well-crafted for capturing universal graph anomaly patterns: 1) smoothness-
based feature Alignment module that unifies the features of different datasets
into a common and anomaly-sensitive space; 2) ego-neighbor Residual graph
encoder that learns abnormality-related node embeddings; and 3) cross-
attentive in-Context anomaly scoring module that predicts node abnormality by
leveraging few-shot normal samples. Extensive experiments on multiple
benchmark datasets from various domains demonstrate the superior anomaly
detection performance, efficiency, and generalizability of ARC.

##  1 Introduction

Graph anomaly detection (GAD) aims to distinguish abnormal nodes that show
significant dissimilarity from the majority of nodes in a graph. GAD has broad
applications across various real-world scenarios, such as fraud detection in
financial transaction networks [1] and rumor detection in social networks [2].
As a result, GAD has attracted increasing research attention in recent years
[3, 4, 5, 6, 7]. Conventional GAD methods employ shallow mechanisms to model
node-level abnormality [8, 9, 10]; however, they face limitations in handling
high-dimensional features and complex interdependent relations on graphs.
Recently, graph neural network (GNN)-based approaches have emerged as the go-
to solution for the GAD problem due to their superior performance [4, 6]. Some
GNN-based GAD approaches regard GAD as a supervised binary classification
problem and use specifically designed GNN architectures to capture anomaly
patterns [6, 11, 12, 13]. Another line of approaches targets the more
challenging unsupervised paradigm, employing various unsupervised learning
objectives and frameworks to identify anomalies without relying on labels [4,
14, 15, 16].

Despite their remarkable detection performance, the existing GAD approaches
follow a “one model for one dataset” learning paradigm (as shown in Fig. 1 (a)
and (b)), necessitating dataset-specific training and ample training data to
construct a detection model for each dataset. This learning paradigm
inherently comes with the following limitations: ❶ Expensive training cost.
For each dataset, we need to train a specialized GAD model from scratch, which
incurs significant costs for model training, especially when dealing with
large-scale graphs. ❷ Data requirements. Training a reliable GAD model
typically needs sufficient in-domain data, sometimes requiring labels as well.
The data requirements pose a challenge when applying GAD to scenarios with
sparse data, data privacy concerns, or high label annotation costs. ❸ Poor
generalizability. On a new-coming dataset, existing GAD methods require
hyperparameter tuning or even model architecture modifications to achieve
optimal performance, which increases the cost of applying them to new data and
domains.

Given the above limitations, a natural question arises: Can we train a “one-
for-all” GAD model that can generalize to detect anomalies across various
graph datasets from different application domains, without any training on the
target data? Following the trend of artificial general intelligence and
foundation models, a new paradigm termed “generalist anomaly detection”,
originating from image anomaly detection, is a potential answer to this
question [17]. As shown in Fig. 1 (c), in the generalist paradigm, we only
need to train the GAD model once; afterward, the well-trained generalist GAD
model can directly identify anomalies on diverse datasets, without any re-
training or fine-tuning. Considering the diversity of graph data across
different domains and datasets, the labels of few-shot normal nodes are
required during the inference stage to enable the model to grasp the
fundamental characteristics of the target dataset. Compared to conventional
paradigms, the generalist paradigm eliminates the need for dataset-specific
training, resulting in fewer computations, lower data costs, and stronger
generalizability when applying GAD models to new datasets.

Figure 1: Sketch maps of (a) supervised, (b) unsupervised, and (c) generalist
GAD paradigms.

Nevertheless, due to the unique characteristics of graph data and GAD problem,
it is non-trivial to design a generalist GAD approach. The challenge is three-
fold: C1 \- Feature alignment. Unlike image data, which are typically
represented in a consistent RGB feature space, the feature dimensionality and
semantic space can vary significantly across different graph datasets.
Substituting features with unified representations generated by large language
models may be a potential solution [18]; however, this approach is limited to
specific feature semantics and cannot address more general cases [19]. C2 \-
Representation encoding. As the core of a generalist GAD model, a GNN-based
encoder is expected to learn dataset-agnostic and abnormality-aware node
embeddings for anomaly detection. However, in the absence of universal pre-
trained foundation models [17] for graph data, crafting a potent encoder for a
generalist GAD model presents a challenge. C3 \- Few-shot sample-guided
prediction. Existing GAD methods typically focus on single dataset settings,
where dataset-specific knowledge is embedded in the model through training,
enabling it to predict abnormality for each node independently. In contrast, a
generalist GAD model should derive such knowledge from a small number of
normal nodes. In this case, how to effectively utilize the few-shot normal
samples during inference remains an open question.

To tackle these challenges, we introduce ARC, a generalist GAD approach based
on in-context learning. ARC comprises three meticulously designed modules,
each targeting a specific challenge. To address C1, we introduce a smoothness-
based feature Alignment module, which not only standardizes features across
diverse datasets to a common dimensionality but also arranges them in an
anomaly-sensitive order. To deal with C2, we design an ego-neighbor Residual
graph encoder. Equipped with a multi-hop residual-based aggregation scheme,
the graph encoder learns attributes that indicate high-order affinity and
heterophily, capturing informative and abnormality-aware embeddings across
different datasets. Last but not least, to solve C3, we propose a cross-
attentive in-Context anomaly scoring module. Following the in-context learning
schema, we treat the few-shot normal nodes as context samples and utilize a
cross-attention block to reconstruct the embeddings of unlabeled samples based
on the context samples. Then, the reconstruction distance can serve as the
anomaly score for each unlabeled node. In summary, this paper makes the
following contributions:

  * •

Problem. We, for the first time, propose to investigate the generalist GAD
problem, aiming to detect anomalies from various datasets with a single GAD
model, without dataset-specific fine-tuning.

  * •

Methodology. We propose a novel generalist GAD method ARC, which can detect
anomalies in new graph datasets on-the-fly via in-context learning based on
few-shot normal samples.

  * •

Experiments. We conduct extensive experiments to validate the anomaly
detection capability, generalizability, and efficiency of ARC across multiple
benchmark datasets from various domains.

##  2 Related Work

In this section, we offer a brief review of pertinent related works, with a
more extensive literature review available in Appendix A.

Anomaly Detection. Anomaly detection (AD) aims to identify anomalous samples
that deviate from the majority of samples [20]. Mainstream AD methods focus on
unsupervised settings and employ various unsupervised techniques to build the
models [21, 22, 23, 24, 25, 26]. To enhance the generalizability of AD methods
across diverse datasets, RegAD [27] considers few-shot setting and trains a
single generalizable model capable of being applied to new in-domain data
without re-training or fine-tuning. WinCLIP [28] utilizes visual-language
models (VLMs, e.g., CLIP [29]) with well-crafted text prompts to perform
zero/few-shot AD for image data. InCTRL [17], as the first generalist AD
approach, integrates in-context learning and VLMs to achieve domain-agnostic
image AD with a single model. However, due to their heavy reliance on pre-
trained vision encoders/VLMs and image-specific designs, these approaches
excel in AD for image data but face challenges when applied to graph data.

Graph Anomaly Detection (GAD). In this paper, we focus on the node-level AD on
graphs and refer to it as “graph anomaly detection (GAD)” following [6, 30,
31]. While shallow methods [8, 9, 10] show limitations in handling complex
real-world graphs [4], the advanced approaches are mainly based on GNNs [32].
The GNN-based approaches can be divided into supervised and unsupervised
approaches [3, 5, 7]. Supervised GAD approaches assume that the labels of both
normal and anomalous nodes are available for model training [7]. Hence,
related studies mainly introduce GAD methods in a binary classification
paradigm [6, 11, 12, 13, 33, 34]. In contrast, unsupervised GAD approaches do
not require any labels for model training. They employ several unsupervised
learning techniques to learn anomaly patterns on graph data, including data
reconstruction [4, 31, 35], contrastive learning [14, 36, 37], and other
auxiliary objectives [15, 16, 38]. Nevertheless, all the above methods adhere
to the conventional paradigm of “one model for one dataset”. Although some GAD
approaches [39, 40] can handle cross-domain scenarios, their requirement for
high correlation (e.g., aligned node features) between source and target
datasets limits their generalizability. Differing from existing methods, our
proposed ARC is a “one-for-all” GAD model capable of identifying anomalies
across target datasets from diverse domains, without the need for re-training
or fine-tuning.

In-Context Learning (ICL). ICL enables a well-trained model to be effectively
(fine-tuning-free) adapted to new domains, datasets, and tasks based on
minimal in-context examples (a.k.a. context samples), providing powerful
generalization capability of large language models (LLMs) [41, 42, 43] and
computer vision (CV) models [17, 44, 45, 46]. Two recent approaches, PRODIGY
[47] and UniLP [48] attempt to use ICL for GNNs to solve the node
classification and link prediction tasks, respectively. However, how to use
ICL to deal with the generalist GAD problem where only normal context samples
are available still remains open.

##  3 Problem Statement

Notations. Let $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X})$ be an
attributed graph with $n$ nodes and $m$ edges, where
$\mathcal{V}=\\{v_{1},\cdots,v_{n}\\}$ and $\mathcal{E}$ are the set of nodes
and edges, respectively. The node-level attributes are included by feature
matrix $\mathbf{X}\in\mathbb{R}^{n\times d}$, where each row $\mathbf{X}_{i}$
indicates the feature vector for node $v_{i}$. The inter-node connectivity is
represented by an adjacency matrix $\mathbf{A}\in\\{0,1\\}^{n\times n}$, where
the $i,j$-th entry $\mathbf{A}_{ij}=1$ means $v_{i}$ and $v_{j}$ are connected
and vice versa.

Conventional GAD Problem. GAD aims to differentiate abnormal nodes
$\mathcal{V}_{a}$ from normal nodes $\mathcal{V}_{n}$ within a given graph
$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X})$, where $\mathcal{V}_{a}$
and $\mathcal{V}_{n}$ satisfy
$\mathcal{V}_{a}\cup\mathcal{V}_{n}=\mathcal{V}$,
$\mathcal{V}_{a}\cap\mathcal{V}_{n}=\emptyset$, and
$|\mathcal{V}_{a}|\ll|\mathcal{V}_{n}|$. An anomaly label vector
$\mathbf{y}\in\\{0,1\\}^{n}$ can be used to denote the abnormality of each
node, where the $i$-th entry $\mathbf{y}_{i}=1$ iff $v\in\mathcal{V}_{a}$ and
$\mathbf{y}_{i}=0$ iff $v\in\mathcal{V}_{n}$. Formally, the goal of GAD is to
learn an anomaly scoring function (i.e., GAD model)
$f:\mathcal{V}\rightarrow\mathbb{R}$ such that $f(v^{\prime})>f(v)$ for
$\forall v^{\prime}\in\mathcal{V}_{a}$ and $\forall v\in\mathcal{V}_{n}$. In
the conventional GAD setting of “one model for one dataset”, the GAD model $f$
is optimized on the target graph dataset
$\mathcal{D}=(\mathcal{G},\mathbf{y})$ with a subset of anomaly labels (in
supervised setting) or without labels (in unsupervised setting). After
sufficient training, the model $f$ can identify anomalies within the target
graph $\mathcal{G}$ during the inference phase.

Generalist GAD Problem. In this paper, we investigate the generalist GAD
problem, wherein we aim to develop a generalist GAD model capable of detecting
abnormal nodes across diverse graph datasets from various application domains
without any training on the specific target data. Formally, we define the
generalist GAD setting, aligning it with its counterpart in image AD as
introduced by Zhu et al. [17]. Specifically, let
$\mathcal{T}_{train}=\\{\mathcal{D}^{(1)}_{train},\cdots,\mathcal{D}^{(N)}_{train}\\}$
be a collection of training datasets, where each
$\mathcal{D}^{(i)}_{train}=(\mathcal{G}^{(i)}_{train},\mathbf{y}^{(i)}_{train})$
is a labeled dataset from an arbitrary domain. We aim to train a generalist
GAD model $f$ on $\mathcal{T}_{train}$, and $f$ is able to identify anomalies
within any test graph dataset $\mathcal{D}^{(i)}_{test}\in\mathcal{T}_{test}$,
where
$\mathcal{T}_{test}=\\{\mathcal{D}^{(1)}_{test},\cdots,\mathcal{D}^{(N^{\prime})}_{test}\\}$
is a collection of testing datasets. Note that
$\mathcal{T}_{train}\cap\mathcal{T}_{test}=\emptyset$ and the datasets in
$\mathcal{T}_{train}$ and $\mathcal{T}_{test}$ can be drawn from different
distributions and domains. Following [17], we adopt a “normal few-shot”
setting during inference: for each $\mathcal{D}^{(i)}_{test}$, only a handful
of $n_{k}$ normal nodes ($n_{k}\ll n$) are available, and the model $f$ is
expected to predict the abnormality of the rest nodes without re-training and
fine-tuning.

##  4 ARC: A generalist GAD approach

Figure 2: The overall pipeline of ARC, the proposed generalist GAD approach.

In this section, we introduce ARC, a generalist GAD approach capable of
identifying anomalies across diverse graph datasets without the need for
specific fine-tuning. The overall pipeline of ARC is demonstrated in Fig. 2.
Firstly, to align the features of different datasets, we introduce a
smoothness-based feature alignment module (Sec. 4.1), which not only projects
features onto a common plane but also sorts the dimensions in an anomaly-
sensitive order. Next, to capture abnormality-aware node embeddings, we
propose a simple yet effective GNN model termed ego-neighbor residual graph
encoder (Sec. 4.2), which constructs node embeddings by combining residual
information between an ego node and its neighbors. Finally, to leverage
knowledge from few-shot context samples for predicting node-level abnormality,
we introduce a cross-attentive in-context anomaly scoring module (Sec. 4.3).
Using the cross-attention block, the model learns to reconstruct query node
embeddings based on context node embeddings. Ultimately, the drift distance
between the original and reconstructed query embeddings can quantify the
abnormality of each node.

###  4.1 Smoothness-Based Feature Alignment

Graph data from diverse domains typically have different features,
characterized by differences in dimensionality and unique meanings for each
dimension. For example, features in a citation network usually consist of
textual and meta-information associated with each paper, whereas in a social
network, the features may be the profile of each user. Therefore, in the first
step, we need to align the features into a shared feature space. To achieve
this, we introduce the feature alignment module in ARC, consisting of two
phases: feature projection, which aligns dimensionality, and smoothness-based
feature sorting, which reorders features according to their smoothness
characteristics.

Feature Projection. At the first step of ARC, we employ a feature projection
block to unify the feature dimensionality of multiple graph datasets [19].
Specifically, given a feature matrix
$\mathbf{X}^{(i)}\in\mathbb{R}^{n^{(i)}\times d^{(i)}}$ of
$\mathcal{D}^{(i)}\in\mathcal{T}_{train}\cup\mathcal{T}_{test}$, the feature
projection is defined by a linear mapping:

$\tilde{\mathbf{X}}^{(i)}\in\mathbb{R}^{n^{(i)}\times
d_{u}}=\operatorname{Proj}\left(\mathbf{X}^{(i)}\right)=\mathbf{X}^{(i)}\mathbf{W}^{(i)},$
(1)

where $\tilde{\mathbf{X}}^{(i)}$ is the projected feature matrix for
$\mathcal{D}^{(i)}$, $d_{u}$ is a predefined projected dimension shared across
all datasets, and $\mathbf{W}^{(i)}\in\mathbb{R}^{d^{(i)}\times d_{u}}$ is a
dataset-specific linear projection weight matrix. To maintain generality,
$\mathbf{W}^{(i)}$ can be defined using commonly used dimensionality reduction
approaches such as singular value decomposition [49] (SVD) and principal
component analysis [50] (PCA).

Smoothness-Based Feature Sorting. Although feature projection can align
dimensionality, the semantic meaning of each projected feature across
different datasets remains distinct. Considering the difficulty of semantic-
level matching without prior knowledge and specific fine-tuning [18, 19], in
this paper, we explore an alternative pathway: aligning features based on
their contribution to anomaly detection tasks. Through analytical and
empirical studies, we pinpoint that the smoothness of each feature is strongly
correlated with its contribution to GAD. Building on this insight, in ARC, we
propose to sort the features according to their contribution as our alignment
strategy.

From the perspective of graph signal processing, Tang et al. [6] have
demonstrated that the inverse of the low-frequency energy ratio monotonically
increases with the anomaly degree. In other words, high-frequency graph
signals tend to play a more significant role in detecting anomalies. Similar
findings have also been observed from the standpoint of spatial GNNs [34, 51],
where heterophily information has been shown to be crucial in discriminating
anomalies. Motivated by these findings, can we develop a metric to gauge the
contribution of each feature to GAD based on its frequency/heterophily?

(a) Cora

(b) Facebook

Figure 3: AUROC on data with 5 groups of features.

Considering its correlation to frequency [52] and heterophily [53], in this
paper, we select feature-level smoothness as the measure for contribution.
Formally, given a graph $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X})$
with a normalized feature matrix $\mathbf{X}$, the smoothness of the $k$-th
feature dimension is defined as:

$s_{k}(\mathbf{X})=-\frac{1}{|\mathcal{E}|}\sum_{(v_{i},v_{j})\in\mathcal{E}}\left(\mathbf{X}_{ik}-\mathbf{X}_{jk}\right)^{2},$
(2)

where a lower $s_{k}$ indicates a significant change in the $k$-th feature
between connected nodes, implying that this feature corresponds to a high-
frequency graph signal and exhibits strong heterophily.

To verify whether smoothness can indicate the contribution of features in GAD,
we further conduct empirical analysis (experimental setup and more results can
be found in Appendix B). Concretely, we sort the raw features of each dataset
based on the smoothness $s_{k}$ and divide them into 5 groups according to the
percentile of $s_{k}$. Then, we train different GAD models using each group of
features separately, and the performance is shown in Fig. 3 and 8. On both
datasets, a model-agnostic observation is that the features with lower $s_{k}$
are more helpful in discriminating anomalies. The consistent trend
demonstrates the effectiveness of $s_{k}$ as an indicator of the role of
features in GAD.

In light of this, given the projected features of different datasets, we can
align their feature spaces by rearranging the permutation of features based on
the descending order of $s_{k}$ w.r.t. each projected feature. For all
datasets, the feature in the first column is the one with the lowest $s_{k}$,
which deserves more attention by ARC; conversely, features with less
contribution (i.e. higher $s_{k}$) are placed at the end. In this way, the
GNN-based model can learn to filter graph signals with different smoothness
levels automatically and predict anomalies accordingly. During inference, the
smoothness-related information remains transferable because we adhere to the
same alignment strategy.

###  4.2 Ego-Neighbor Residual Graph Encoder

Once the features are aligned, we employ a GNN-based graph encoder to learn
node embeddings that capture both semantic and structural information for each
node. The learned embedding can be utilized to predict the abnormality of the
corresponding node with the downstream anomaly scoring module. A naive
solution is directly employing commonly used GNNs, such as GCN [54] or GAT
[55], as the graph encoder. However, due to their low-pass filtering
characteristic, these GNNs face difficulty in capturing abnormality-related
patterns that are high-frequency and heterophilic [6, 34]. Moreover, most
GNNs, including those tailored for GAD, tend to prioritize capturing node-
level semantic information while disregarding the affinity patterns of local
subgraphs [16]. Consequently, employing existing GNN models as the encoder may
overemphasize dataset-specific semantic knowledge, but overlook the shared
anomaly patterns (i.e. local node affinity) across different datasets.

To address the above issues, we design an ego-neighbor residual graph encoder
for ARC. Equipped with a residual operation, the encoder can capture multi-hop
affinity patterns of each node, providing valuable and comprehensive
information for anomaly identification. Similar to the “propagation then
transformation” GNN architecture in SGC [56], our graph encoder consists of
three steps: multi-hop propagation, shared MLP-based transformation, and ego-
neighbor residual operation. In the first two steps, we perform propagation on
the aligned feature matrix $\mathbf{X}^{\prime}=\mathbf{X}^{[0]}$ for $L$
iterations, and then conduct transformation on the raw and propagated features
with a shared MLP network:

$\mathbf{X}^{[l]}=\tilde{\mathbf{A}}\mathbf{X}^{[l-1]},\quad\mathbf{Z}^{[l]}=\operatorname{MLP}\left(\mathbf{X}^{[l]}\right),$
(3)

where $l\in\\{0,\cdots,L\\}$, $\mathbf{X}^{[l]}$ is the propagated feature
matrix at the $l$-th iteration, $\mathbf{Z}^{[l]}$ is the transformed
representation matrix at the $l$-th iteration, and $\tilde{\mathbf{A}}$ is the
normalized adjacency matrix [54, 56]. Note that, unlike most GNNs that only
consider the features/representations after $L$-iter propagation, here we
incorporate both the raw features and intermediate propagated features and
transform them into the same representation space. After obtaining
${\mathbf{Z}^{[0]},\cdots,\mathbf{Z}^{[L]}}$, we calculate the residual
representations by taking the difference between $\mathbf{Z}^{[l]}$ ($1\leq
l\leq L$) and $\mathbf{Z}^{[0]}$, and then concatenate the multi-hop residual
representations to form the final embeddings:

$\mathbf{R}^{[l]}=\mathbf{Z}^{[l]}-\mathbf{Z}^{[0]},\quad\mathbf{H}=[\mathbf{R}^{[1]}||\cdots||\mathbf{R}^{[L]}],$
(4)

where $\mathbf{R}^{[l]}$ is the residual matrix at the $l$-th iteration,
$\mathbf{H}\in\mathbb{R}^{n\times d_{e}}$ is the output embedding matrix, and
$||$ denotes the concatenation operator.

Discussion. Compared to existing GNNs, our graph encoder offers the following
advantages. Firstly, with the residual operation, the proposed encoder
emphasizes the difference between the ego node and its neighbors rather than
ego semantic information. This approach allows for the explicit modeling of
local affinity through the learned embeddings. Since local affinity is a
crucial indicator of abnormality and this characteristic can be shared across
diverse datasets [16], the learned embeddings can offer valuable
discriminative insights for downstream prediction. Second, the residual
operation performs as a high-pass filter on the graph data, aiding ARC in
capturing more abnormality-related attributes, i.e., high-frequency signals
and local heterophily. Moreover, unlike existing approaches [14, 16] that only
consider 1-hop affinity, our encoder also incorporates higher-order affinity
through the multi-hop residual design, which enables ARC to capture more
complex graph anomaly patterns. More discussion and comparison to the existing
GNNs/GAD methods are conducted in Appendix C.

###  4.3 Cross-Attentive In-Context Anomaly Scoring

To utilize the few-shot normal samples (denoted by context nodes) to predict
the abnormality of the remaining nodes (denoted by query nodes), in ARC, we
devise an in-context learning module with a cross-attention mechanism for
anomaly scoring. The core idea of our in-context learning module is to
reconstruct the node embedding of each query node using a cross-attention
block to blend the embeddings of context nodes. Then, the drift distance
between the original and reconstructed embeddings of a query node can serve as
the indicator of its abnormality.

Specifically, we partition the embedding matrix $\mathbf{H}$ into two parts by
indexing the corresponding row vectors: the embeddings of context nodes
$\mathbf{H}_{k}\in\mathbb{R}^{n_{k}\times d_{e}}$ and the embeddings of query
nodes $\mathbf{H}_{q}\in\mathbb{R}^{n_{q}\times d_{e}}$. Then, a cross-
attention block is utilized to reconstruct each row of $\mathbf{H}_{q}$
through a linear combination of $\mathbf{H}_{k}$:

$\mathbf{Q}=\mathbf{H}_{q}\mathbf{W}_{q},\quad\mathbf{K}=\mathbf{H}_{k}\mathbf{W}_{k},\quad\tilde{\mathbf{H}}_{q}=\operatorname{Softmax}\left(\frac{\mathbf{Q}\mathbf{K}^{\top}}{\sqrt{d_{e}}}\right)\mathbf{H}_{k},$
(5)

where $\mathbf{Q}\in\mathbb{R}^{n_{q}\times d_{e}}$ and
$\mathbf{K}\in\mathbb{R}^{n_{k}\times d_{e}}$ are the query and key matrices
respectively, $\mathbf{W}_{q}$ and $\mathbf{W}_{k}$ are learnable parameters,
and $\tilde{\mathbf{H}}_{q}$ is the reconstructed query embedding matrix. Note
that, unlike the conventional cross-attention blocks [57, 58, 59] that further
introduce a value matrix $\mathbf{V}$, our block directly multiplies the
attention matrix with $\mathbf{H}_{k}$. This design ensures that
$\tilde{\mathbf{H}}_{q}$ is in the same embedding space as ${\mathbf{H}}_{q}$
and ${\mathbf{H}}_{k}$. Thanks to this property, given a query node $v_{i}$,
we can calculate its anomaly score $f(v_{i})$ by computing the L2 distance
between its query embedding vector ${\mathbf{H}}{q_{i}}$ and the corresponding
reconstructed query embedding vector ${\tilde{\mathbf{H}}q}_{i}$, i.e.,
$f(v_{i})=d({\mathbf{H}}{q_{i}},\tilde{\mathbf{H}}{q_{i}})=\sqrt{\sum_{j=1}^{d_{e}}\left({\mathbf{H}}{q_{ij}}-\tilde{\mathbf{H}}{q_{ij}}\right)^{2}}$.

Figure 4: Toy examples of query embeddings ($\bullet$), reconstructed query
embeddings ($\bullet$), and context embeddings ($\blacksquare$).

Discussion.  The design of cross-attentive in-context anomaly scoring follows
a basic assumption: normal query nodes have similar patterns to several
context nodes, and hence their embeddings can be easily represented by the
linear combination of context node embeddings. Consequently, given a normal
node, its original and reconstructed embeddings can be close to each other in
the embedding space. In contrast, abnormal nodes may display distinct patterns
compared to normal ones, making it difficult to depict their corresponding
abnormal query embeddings using context embeddings. As a result, their drift
distance $s_{i}$ can be significantly larger. Fig. 4 provides examples for the
scenarios of (a) single-class normal and (b) multi-class normal. In both
cases, the drift distance (➞) can be a significant indicator for
distinguishing anomaly (5) from normal nodes (1~4). Interestingly, if the
attention matrix assigns uniform weights to all context nodes, then our
scoring module becomes a one-class classification model [21]. This property
ensures the anomaly detection capability of ARC even without extensive
training. A detailed discussion is conducted in Appendix D.

Model Training.  To optimize ARC on training datasets $\mathcal{T}_{train}$,
we employ a marginal cosine similarity loss to minimize the drift distance of
normal nodes while maximizing the drift distance of abnormal nodes.
Specifically, given graph data with anomaly labels, we randomly select $n_{k}$
normal nodes as context nodes and sample an equal number of normal and
abnormal nodes as query nodes. Then, given a query node $v_{i}$ with embedding
${\mathbf{H}}{q_{i}}$, reconstructed embedding $\tilde{\mathbf{H}}{q_{i}}$,
and anomaly label $\mathbf{y}_{i}$, the sample-level loss function can be
written by:

$\mathcal{L}=\begin{cases}1-\cos\left({\mathbf{H}}{q_{i}},\tilde{\mathbf{H}}{q_{i}}\right),&\text{
if }\mathbf{y}_{i}=0\\\
\max\left(0,\cos\left({\mathbf{H}}{q_{i}},\tilde{\mathbf{H}}{q_{i}}\right)-\epsilon\right),&\text{
if }\mathbf{y}_{i}=1\end{cases}$ (6)

where $\cos(\cdot,\cdot)$ and $\max(\cdot,\cdot)$ denote the cosine similarity
and maximum operation, respectively, and $\epsilon$ is a margin
hyperparameter. Detailed algorithmic description and complexity analysis of
ARC can be found in Appendix E.

##  5 Experiments

###  5.1 Experimental Setup

Datasets. To learn generalist GAD models, we train the baseline methods and
ARC on a group of graph datasets and test on another group of datasets. For
comprehensive evaluations, we consider graph datasets spanning a variety of
domains, including social networks, citation networks, and e-commerce co-
review networks, each of them with either injected anomalies or real anomalies
[7, 14, 16]. Inspired by [48], we train the models on the largest dataset of
each type and conduct testing on the remaining datasets. Specifically, the
training datasets $\mathcal{T}_{train}$ comprise PubMed, Flickr, Questions,
and YelpChi, while the testing datasets $\mathcal{T}_{test}$ consist of Cora,
CiteSeer, ACM, BlogCatalog, Facebook, Weibo, Reddit, and Amazon. For detailed
information, please refer to Appendix F.1.

Baselines. We compare ARC with both supervised and unsupervised methods.
Supervised methods include two conventional GNNs, i.e., GCN [54] and GAT [55],
and three state-of-the-art GNNs specifically designed for GAD, i.e., BGNN
[60], BWGNN [6], and GHRN [34]. Unsupervised methods include four
representative approaches with distinct designs, including the generative
method DOMINANT [4], the contrastive method CoLA [14], the hop predictive
method HCM-A [15], and the affinity-based method TAM [16]. For detailed
information, refer to Appendix F.2.

Evaluation and Implementation. Following [7, 16, 61], we employ AUROC and
AUPRC as our evaluation metrics for GAD. We report the average AUROC/AUPRC
with standard deviations across 5 trials. We train ARC on all the datasets in
$\mathcal{T}_{train}$ jointly, and evaluate the model on each dataset in
$\mathcal{T}_{test}$ in an in-context learning manner ($n_{k}=10$ as default).
For the supervised baselines, we follow the same training and testing
procedure (denoted as “pre-train only”), since no labeled anomaly is available
for fine-tuning. For the unsupervised baselines, we consider two settings:
“pre-train only” and “pre-train & fine-tune”. In the latter, we additionally
conduct dataset-specific fine-tuning with a few epochs. To standardize the
feature space in baseline methods, we utilize either learnable projection or
random projection as an adapter between the raw feature and the model input
layer. We utilize random search to determine the optimal hyperparameters for
both the baselines and ARC. Since our goal is to train generalist GAD models,
we do not perform dataset-specific hyperparameter search, but instead use the
same set of hyperparameters for all testing datasets. More implementation
details can be found in Appendix F.3.

###  5.2 Experimental Results

Table 1: Anomaly detection performance in terms of AUROC (in percent,
mean$\pm$std). Highlighted are the results ranked first, second, and third.
“Rank” indicates the average ranking over 8 datasets.

Method | Cora | CiteSeer | ACM | BlogCatalog | Facebook | Weibo | Reddit | Amazon | Rank
---|---|---|---|---|---|---|---|---|---
Supervised - Pre-Train Only
GCN | $59.64{\scriptstyle\pm 8.30}$ | $60.27{\scriptstyle\pm 8.11}$ | $60.49{\scriptstyle\pm 9.65}$ | $56.19{\scriptstyle\pm 6.39}$ | $29.51{\scriptstyle\pm 4.86}$ | $76.64{\scriptstyle\pm 17.69}$ | $50.43{\scriptstyle\pm 4.41}$ | $46.63{\scriptstyle\pm 3.47}$ | $8.9$
GAT | $50.06{\scriptstyle\pm 2.65}$ | $51.59{\scriptstyle\pm 3.49}$ | $48.79{\scriptstyle\pm 2.73}$ | $50.40{\scriptstyle\pm 2.80}$ | $51.88{\scriptstyle\pm 2.16}$ | $53.06{\scriptstyle\pm 7.48}$ | $51.78{\scriptstyle\pm 4.04}$ | $50.52{\scriptstyle\pm 17.22}$ | $10.0$
BGNN | $42.45{\scriptstyle\pm 11.57}$ | $42.32{\scriptstyle\pm 11.82}$ | $44.00{\scriptstyle\pm 13.69}$ | $47.67{\scriptstyle\pm 8.52}$ | $54.74{\scriptstyle\pm 25.29}$ | $32.75{\scriptstyle\pm 35.35}$ | $50.27{\scriptstyle\pm 3.84}$ | $52.26{\scriptstyle\pm 3.31}$ | $11.1$
BWGNN | $54.06{\scriptstyle\pm 3.27}$ | $52.61{\scriptstyle\pm 2.88}$ | $67.59{\scriptstyle\pm 0.70}$ | $56.34{\scriptstyle\pm 1.21}$ | $45.84{\scriptstyle\pm 4.97}$ | $53.38{\scriptstyle\pm 1.61}$ | $48.97{\scriptstyle\pm 5.74}$ | $55.26{\scriptstyle\pm 16.95}$ | $9.0$
GHRN | $59.89{\scriptstyle\pm 6.57}$ | $56.04{\scriptstyle\pm 9.19}$ | $55.65{\scriptstyle\pm 6.37}$ | $57.64{\scriptstyle\pm 3.48}$ | $44.81{\scriptstyle\pm 8.06}$ | $51.87{\scriptstyle\pm 14.18}$ | $46.22{\scriptstyle\pm 2.33}$ | $49.48{\scriptstyle\pm 17.13}$ | $9.8$
Unsupervised - Pre-Train Only
DOMINANT | $66.53{\scriptstyle\pm 1.15}$ | $69.47{\scriptstyle\pm 2.02}$ | $70.08{\scriptstyle\pm 2.34}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{74.25{\scriptstyle\pm 0.65}}}$ | $51.01{\scriptstyle\pm 0.78}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{92.88{\scriptstyle\pm 0.32}}}}$ | $50.05{\scriptstyle\pm 4.92}$ | $48.94{\scriptstyle\pm 2.69}$ | $5.8$
CoLA | $63.29{\scriptstyle\pm 8.88}$ | $62.84{\scriptstyle\pm 9.52}$ | $66.85{\scriptstyle\pm 4.43}$ | $50.04{\scriptstyle\pm 3.25}$ | $12.99{\scriptstyle\pm 11.68}$ | $16.27{\scriptstyle\pm 5.64}$ | $52.81{\scriptstyle\pm 6.69}$ | $47.40{\scriptstyle\pm 7.97}$ | $9.5$
HCM-A | $54.28{\scriptstyle\pm 4.73}$ | $48.12{\scriptstyle\pm 6.80}$ | $53.70{\scriptstyle\pm 4.64}$ | $55.31{\scriptstyle\pm 0.57}$ | $35.44{\scriptstyle\pm 13.97}$ | $65.52{\scriptstyle\pm 12.58}$ | $48.79{\scriptstyle\pm 2.75}$ | $43.99{\scriptstyle\pm 0.72}$ | $11.4$
TAM | $62.02{\scriptstyle\pm 2.39}$ | $72.27{\scriptstyle\pm 0.83}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{74.43{\scriptstyle\pm 1.59}}}$ | $49.86{\scriptstyle\pm 0.73}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{65.88{\scriptstyle\pm 6.66}}}$ | $71.54{\scriptstyle\pm 0.18}$ | $55.43{\scriptstyle\pm 0.33}$ | $56.06{\scriptstyle\pm 2.19}$ | $5.6$
Unsupervised - Pre-Train & Fine-Tune
DOMINANT | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{72.23{\scriptstyle\pm 0.34}}}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{74.69{\scriptstyle\pm 0.32}}}$ | $74.34{\scriptstyle\pm 0.12}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{74.61{\scriptstyle\pm 0.04}}}$ | $49.92{\scriptstyle\pm 0.55}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{92.21{\scriptstyle\pm 0.10}}}$ | $52.14{\scriptstyle\pm 5.06}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{59.06{\scriptstyle\pm 2.80}}}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{3.6}}$
CoLA | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{67.62{\scriptstyle\pm 4.26}}}$ | $70.75{\scriptstyle\pm 3.42}$ | $69.11{\scriptstyle\pm 0.67}$ | $62.49{\scriptstyle\pm 3.38}$ | $64.70{\scriptstyle\pm 18.86}$ | $31.55{\scriptstyle\pm 6.02}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{58.12{\scriptstyle\pm 0.67}}}$ | $52.51{\scriptstyle\pm 6.66}$ | $5.4$
HCM-A | $56.45{\scriptstyle\pm 4.93}$ | $55.54{\scriptstyle\pm 4.07}$ | $57.69{\scriptstyle\pm 3.59}$ | $55.10{\scriptstyle\pm 0.29}$ | $36.57{\scriptstyle\pm 10.72}$ | $71.89{\scriptstyle\pm 2.79}$ | $49.15{\scriptstyle\pm 2.72}$ | $42.20{\scriptstyle\pm 0.55}$ | $10.1$
TAM | $62.56{\scriptstyle\pm 2.10}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{76.54{\scriptstyle\pm 1.33}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{86.29{\scriptstyle\pm 1.57}}}}$ | $57.69{\scriptstyle\pm 0.88}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{76.26{\scriptstyle\pm 3.70}}}}$ | $71.73{\scriptstyle\pm 0.16}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{56.62{\scriptstyle\pm 0.49}}}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{57.13{\scriptstyle\pm 1.59}}}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{3.4}}$
Ours
ARC | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{85.33{\scriptstyle\pm 0.44}}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{90.64{\scriptstyle\pm 0.32}}}}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{79.27{\scriptstyle\pm 0.17}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{74.81{\scriptstyle\pm 0.28}}}}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{69.57{\scriptstyle\pm 1.47}}}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{89.24{\scriptstyle\pm 0.35}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{58.95{\scriptstyle\pm 1.25}}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{69.77{\scriptstyle\pm 3.88}}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{1.5}}}$

Performance Comparison. Table 1 shows the comparison results of ARC with
baseline methods in terms of AUROC. Results in AUPRC are provided in Appendix
G.1. We have the following observations. ❶ ARC demonstrates strong anomaly
detection capability in the generalist GAD scenario, without any fine-tuning.
Specifically, ARC achieves state-of-the-art performance on 5 out of 8 datasets
and demonstrates competitive performance on the remainder.

(a) Cora

(b) Facebook

Figure 5: Performance with varying #context nodes.

On several datasets, ARC demonstrates significant improvement compared to the
best baseline (e.g., $\uparrow$18.1% on Cora, $\uparrow$18.4% on CiteSeer, and
$\uparrow$18.1% on Amazon). ❷ Simply pre-training the dataset-specific GAD
methods typically results in poor generalization capability to new datasets.
Specifically, the AUROC of the majority of “pre-train only” approaches is
close to random guessing (50%) or even lower. ❸ With dataset-specific fine-
tuning, the baseline methods achieve better performance in the majority of
cases. However, the improvement can be minor or even negative in some cases,
demonstrating the limitations of fine-tuning. Additionally, we observe that
their performance is sometimes lower than the reported results from training
models from scratch [4, 14, 16], indicating potential risk of negative
transfer within the "pre-train & fine-tune" paradigm. ❹ Unsupervised baselines
(except HCM-A) generally outperform the supervised ones, highlighting the
difficulty of training a generalist GAD model using the binary classification
paradigm.

Effectiveness of #Context Nodes. To investigate how the number of context
nodes $n_{k}$ affects the performance of ARC during inference, we vary $n_{k}$
within the range of 2 to 100. The results are shown in Fig. 5 (more results
are in Appendix G.2). From the figure, we observe that the performance of ARC
increases as more context nodes are involved, demonstrating its capability to
leverage these labeled normal nodes with in-context learning. Furthermore, we
can conclude that ARC is also label-efficient: when $n_{k}\geq 10$, the
performance gain from using more context nodes becomes minor; moreover, even
when $n_{k}$ is extremely small, ARC can still perform well on the majority of
datasets.

Table 2: AUROC of ARC and its variants.

Variant | Cora | Cite. | ACM | Blog. | Face. | Wei. | Red. | Ama.
---|---|---|---|---|---|---|---|---
ARC | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{85.33}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{90.64}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{79.27}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{74.81}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{69.57}}}$ | $89.24$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{58.95}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{69.77}}}$
w/o A | $78.46$ | $83.76$ | $78.53$ | $74.44$ | $65.29$ | $90.66$ | $53.81$ | $65.95$
w/o R | $30.35$ | $29.30$ | $60.74$ | $60.56$ | $13.82$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{97.32}}}$ | $46.52$ | $54.83$
w/o C | $49.98$ | $50.69$ | $52.88$ | $72.01$ | $47.95$ | $49.34$ | $47.82$ | $44.64$

Ablation Study. To verify the effectiveness of each component of ARC, we make
corresponding modifications to ARC and designed three variants: 1) w/o A:
using random projection to replace smoothness-based feature alignment; 2) w/o
R: using GCN to replace ego-neighbor residual graph encoder; and 3) w/o C:
using binary classification-based predictor and loss to replace cross-
attentive in-context anomaly scoring. The results are demonstrated in Table 2
(full results are in Appendix G.3). From the results, we can conclude that all
three components significantly contribute to the performance. Among them, the
in-context anomaly scoring module has a significant impact, as the performance
of w/o C is close to random guessing on most datasets. The residual graph
encoder also has a significant impact on the final performance. Notably, Weibo
dataset is an exception where the GCN encoder performs better. A possible
reason is that the Weibo dataset exhibits different anomaly patterns compared
to the others.

Figure 6: Time comparison.

Efficiency Analysis. To assess the runtime efficiency of ARC, we compare the
inference and fine-tuning time on the ACM dataset. As depicted in Fig. 6, ARC
demonstrates comparable runtime performance with the fastest GNNs (e.g., GCN
and BWGNN), and significantly outperforms the unsupervised methods in terms of
efficiency. Additionally, we observe that dataset-specific fine-tuning
consumes more time compared to inference.

(a) Cora

(b) Facebook

Figure 7: Visualization results.

Visualization. To investigate the weight allocation mechanism of the cross-
attention module in ARC, we visualize the attention weights between context
nodes and query nodes in Fig.7 (additional results are in Appendix G.4). From
Fig. 7(a), it is evident that ARC tends to assign uniform attention weights to
normal nodes, leading to reconstructed embeddings that closely resemble the
average embedding of the context nodes. Conversely, anomalies are
reconstructed using a combination of 1 or 2 context nodes, suggesting that
their embeddings are farther from the center. This allocation aligns with the
case of “single-class normal” in Fig. 4(a). Differently, in Fig. 7(b), we
observe that each normal query node is assigned to several context nodes
following two fixed patterns, corresponding to the case of “multi-class
normal” in Fig. 4(b). In summary, the cross-attention module enables ARC to
adapt to various normal/anomaly distribution patterns, enhancing its
generalizability.

##  6 Conclusion

In this paper, we take the first step towards addressing the generalist GAD
problem, aiming to detect anomalies across diverse graph datasets with a “one-
for-all” GAD model, without requiring dataset-specific fine-tuning. We
introduce ARC, a novel and well-crafted in-context learning-based generalist
GAD approach, capable of identifying anomalies on-the-fly using only few-shot
normal nodes. Extensive experiments on real-world datasets from various
domains demonstrate the detection prowess, generalizability, and efficiency of
ARC compared to existing approaches. One limitation is that ARC can only use
normal context samples during inference but cannot directly utilize abnormal
context samples, even when they are available. A potential future direction
could involve developing generalist GAD methods that utilize context samples
containing both anomalies and normal instances.

## References

  * [1] Ranran Li, Zhaowei Liu, Yuanqing Ma, Dong Yang, and Shuaijie Sun.  Internet financial fraud detection based on graph learning.  Ieee Transactions on Computational Social Systems, 2022.
  * [2] Tian Bian, Xi Xiao, Tingyang Xu, Peilin Zhao, Wenbing Huang, Yu Rong, and Junzhou Huang.  Rumor detection on social media with bi-directional graph convolutional networks.  In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 549–556, 2020.
  * [3] Xiaoxiao Ma, Jia Wu, Shan Xue, Jian Yang, Chuan Zhou, Quan Z Sheng, Hui Xiong, and Leman Akoglu.  A comprehensive survey on graph anomaly detection with deep learning.  IEEE Transactions on Knowledge and Data Engineering, 35(12):12012–12038, 2021.
  * [4] Kaize Ding, Jundong Li, Rohit Bhanushali, and Huan Liu.  Deep anomaly detection on attributed networks.  In Proceedings of the 2019 SIAM International Conference on Data Mining, pages 594–602. SIAM, 2019.
  * [5] Kay Liu, Yingtong Dou, Yue Zhao, Xueying Ding, Xiyang Hu, Ruitong Zhang, Kaize Ding, Canyu Chen, Hao Peng, Kai Shu, et al.  Bond: Benchmarking unsupervised outlier node detection on static attributed graphs.  Advances in Neural Information Processing Systems, 35:27021–27035, 2022.
  * [6] Jianheng Tang, Jiajin Li, Ziqi Gao, and Jia Li.  Rethinking graph neural networks for anomaly detection.  In International Conference on Machine Learning, pages 21076–21089. PMLR, 2022.
  * [7] Jianheng Tang, Fengrui Hua, Ziqi Gao, Peilin Zhao, and Jia Li.  Gadbench: Revisiting and benchmarking supervised graph anomaly detection.  Advances in Neural Information Processing Systems, 36, 2024.
  * [8] Bryan Perozzi and Leman Akoglu.  Scalable anomaly ranking of attributed neighborhoods.  In Proceedings of the 2016 SIAM International Conference on Data Mining, pages 207–215. SIAM, 2016.
  * [9] Jundong Li, Harsh Dani, Xia Hu, and Huan Liu.  Radar: Residual analysis for anomaly detection in attributed networks.  In IJCAI, volume 17, pages 2152–2158, 2017.
  * [10] Zhen Peng, Minnan Luo, Jundong Li, Huan Liu, Qinghua Zheng, et al.  Anomalous: A joint modeling approach for anomaly detection on attributed networks.  In IJCAI, volume 18, pages 3513–3519, 2018.
  * [11] Yingtong Dou, Zhiwei Liu, Li Sun, Yutong Deng, Hao Peng, and Philip S Yu.  Enhancing graph neural network-based fraud detectors against camouflaged fraudsters.  In Proceedings of the 29th ACM international conference on information & knowledge management, pages 315–324, 2020.
  * [12] Ao Li, Zhou Qin, Runshi Liu, Yiqun Yang, and Dong Li.  Spam review detection with graph convolutional networks.  In Proceedings of the 28th ACM International conference on information and knowledge management, pages 2703–2711, 2019.
  * [13] Yang Liu, Xiang Ao, Zidi Qin, Jianfeng Chi, Jinghua Feng, Hao Yang, and Qing He.  Pick and choose: a gnn-based imbalanced learning approach for fraud detection.  In Proceedings of the web conference 2021, pages 3168–3177, 2021.
  * [14] Yixin Liu, Zhao Li, Shirui Pan, Chen Gong, Chuan Zhou, and George Karypis.  Anomaly detection on attributed networks via contrastive self-supervised learning.  IEEE transactions on neural networks and learning systems, 33(6):2378–2392, 2021.
  * [15] Tianjin Huang, Yulong Pei, Vlado Menkovski, and Mykola Pechenizkiy.  Hop-count based self-supervised anomaly detection on attributed networks.  In Joint European conference on machine learning and knowledge discovery in databases, pages 225–241. Springer, 2022.
  * [16] Hezhe Qiao and Guansong Pang.  Truncated affinity maximization: One-class homophily modeling for graph anomaly detection.  In Advances in Neural Information Processing Systems, volume 36, 2023.
  * [17] Jiawen Zhu and Guansong Pang.  Toward generalist anomaly detection via in-context residual learning with few-shot sample prompts.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
  * [18] Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang.  One for all: Towards training one graph model for all classification tasks.  In International Conference on Learning Representations, 2024.
  * [19] Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, and Jia Li.  All in one and one for all: A simple yet effective method towards cross-domain graph pretraining.  arXiv preprint arXiv:2402.09834, 2024.
  * [20] Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel.  Deep learning for anomaly detection: A review.  ACM computing surveys (CSUR), 54(2):1–38, 2021.
  * [21] Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft.  Deep one-class classification.  In International conference on machine learning, pages 4393–4402. PMLR, 2018.
  * [22] Sachin Goyal, Aditi Raghunathan, Moksh Jain, Harsha Vardhan Simhadri, and Prateek Jain.  Drocc: Deep robust one-class classification.  In International conference on machine learning, pages 3711–3721. PMLR, 2020.
  * [23] Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, and Peter Gehler.  Towards total recall in industrial anomaly detection.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14318–14328, 2022.
  * [24] Chong Zhou and Randy C Paffenroth.  Anomaly detection with robust deep autoencoders.  In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 665–674, 2017.
  * [25] Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Ursula Schmidt-Erfurth, and Georg Langs.  Unsupervised anomaly detection with generative adversarial networks to guide marker discovery.  In International conference on information processing in medical imaging, pages 146–157. Springer, 2017.
  * [26] Vikash Sehwag, Mung Chiang, and Prateek Mittal.  SSD: A unified framework for self-supervised outlier detection.  In International Conference on Learning Representations, 2021.
  * [27] Chaoqin Huang, Haoyan Guan, Aofan Jiang, Ya Zhang, Michael Spratling, and Yan-Feng Wang.  Registration based few-shot anomaly detection.  In European Conference on Computer Vision, pages 303–319. Springer, 2022.
  * [28] Jongheon Jeong, Yang Zou, Taewan Kim, Dongqing Zhang, Avinash Ravichandran, and Onkar Dabeer.  Winclip: Zero-/few-shot anomaly classification and segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19606–19616, 2023.
  * [29] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  * [30] Yu Zheng, Ming Jin, Yixin Liu, Lianhua Chi, Khoa T Phan, and Yi-Ping Phoebe Chen.  Generative and contrastive self-supervised learning for graph anomaly detection.  IEEE Transactions on Knowledge and Data Engineering, 35(12):12220–12233, 2021.
  * [31] Xuexiong Luo, Jia Wu, Amin Beheshti, Jian Yang, Xiankun Zhang, Yuan Wang, and Shan Xue.  Comga: Community-aware attributed graph anomaly detection.  In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pages 657–665, 2022.
  * [32] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip.  A comprehensive survey on graph neural networks.  IEEE transactions on neural networks and learning systems, 32(1):4–24, 2020.
  * [33] Mingguo He, Zhewei Wei, Hongteng Xu, et al.  Bernnet: Learning arbitrary graph spectral filters via bernstein approximation.  Advances in Neural Information Processing Systems, 34:14239–14251, 2021.
  * [34] Yuan Gao, Xiang Wang, Xiangnan He, Zhenguang Liu, Huamin Feng, and Yongdong Zhang.  Addressing heterophily in graph anomaly detection: A perspective of graph spectrum.  In Proceedings of the ACM Web Conference 2023, pages 1528–1538, 2023.
  * [35] Haoyi Fan, Fengbin Zhang, and Zuoyong Li.  Anomalydae: Dual autoencoder for anomaly detection on attributed networks.  In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5685–5689. IEEE, 2020.
  * [36] Jingcan Duan, Siwei Wang, Pei Zhang, En Zhu, Jingtao Hu, Hu Jin, Yue Liu, and Zhibin Dong.  Graph anomaly detection via multi-scale contrastive learning networks with augmented view.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7459–7467, 2023.
  * [37] Bo Chen, Jing Zhang, Xiaokang Zhang, Yuxiao Dong, Jian Song, Peng Zhang, Kaibo Xu, Evgeny Kharlamov, and Jie Tang.  Gccad: Graph contrastive learning for anomaly detection.  IEEE Transactions on Knowledge and Data Engineering, 2022.
  * [38] Tong Zhao, Chuchen Deng, Kaifeng Yu, Tianwen Jiang, Daheng Wang, and Meng Jiang.  Error-bounded graph anomaly loss for gnns.  In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 1873–1882, 2020.
  * [39] Kaize Ding, Kai Shu, Xuan Shan, Jundong Li, and Huan Liu.  Cross-domain graph anomaly detection.  IEEE Transactions on Neural Networks and Learning Systems, 33(6):2406–2415, 2021.
  * [40] Qizhou Wang, Guansong Pang, Mahsa Salehi, Wray Buntine, and Christopher Leckie.  Cross-domain graph anomaly detection via anomaly-aware contrastive alignment.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 4676–4684, 2023.
  * [41] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020.
  * [42] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al.  Flamingo: a visual language model for few-shot learning.  Advances in neural information processing systems, 35:23716–23736, 2022.
  * [43] Yaru Hao, Haoyu Song, Li Dong, Shaohan Huang, Zewen Chi, Wenhui Wang, Shuming Ma, and Furu Wei.  Language models are general-purpose interfaces.  arXiv preprint arXiv:2206.06336, 2022.
  * [44] Ting Chen, Saurabh Saxena, Lala Li, Tsung-Yi Lin, David J Fleet, and Geoffrey E Hinton.  A unified sequence interface for vision tasks.  Advances in Neural Information Processing Systems, 35:31333–31346, 2022.
  * [45] Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang.  Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework.  In International Conference on Machine Learning, pages 23318–23340. PMLR, 2022.
  * [46] Alexander Kolesnikov, André Susano Pinto, Lucas Beyer, Xiaohua Zhai, Jeremiah Harmsen, and Neil Houlsby.  Uvim: A unified modeling approach for vision with learned guiding codes.  Advances in Neural Information Processing Systems, 35:26295–26308, 2022.
  * [47] Qian Huang, Hongyu Ren, Peng Chen, Gregor Kržmanc, Daniel Zeng, Percy S Liang, and Jure Leskovec.  Prodigy: Enabling in-context learning over graphs.  Advances in Neural Information Processing Systems, 36, 2024.
  * [48] Kaiwen Dong, Haitao Mao, Zhichun Guo, and Nitesh V Chawla.  Universal link predictor by in-context learning.  arXiv preprint arXiv:2402.07738, 2024.
  * [49] Gilbert W Stewart.  On the early history of the singular value decomposition.  SIAM review, 35(4):551–566, 1993.
  * [50] Hervé Abdi and Lynne J Williams.  Principal component analysis.  Wiley interdisciplinary reviews: computational statistics, 2(4):433–459, 2010.
  * [51] Yuan Gao, Xiang Wang, Xiangnan He, Zhenguang Liu, Huamin Feng, and Yongdong Zhang.  Alleviating structural distribution shift in graph anomaly detection.  In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pages 357–365, 2023.
  * [52] Yushun Dong, Kaize Ding, Brian Jalaian, Shuiwang Ji, and Jundong Li.  Adagnn: Graph neural networks with adaptive frequency response filter.  In Proceedings of the 30th ACM international conference on information & knowledge management, pages 392–401, 2021.
  * [53] Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao-Wen Chang, and Doina Precup.  Revisiting heterophily for graph neural networks.  Advances in neural information processing systems, 35:1362–1375, 2022.
  * [54] Thomas N Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  In International Conference on Learning Representations, 2017.
  * [55] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio.  Graph attention networks.  In International Conference on Learning Representations, 2018.
  * [56] Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger.  Simplifying graph convolutional networks.  In International conference on machine learning, pages 6861–6871. PMLR, 2019.
  * [57] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [58] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.
  * [59] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi.  Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.  In International conference on machine learning, pages 19730–19742. PMLR, 2023.
  * [60] Sergei Ivanov and Liudmila Prokhorenkova.  Boost then convolve: Gradient boosting meets graph neural networks.  In International Conference on Learning Representations, 2021.
  * [61] Guansong Pang, Anton van den Hengel, Chunhua Shen, and Longbing Cao.  Toward deep supervised anomaly detection: Reinforcement learning from partially labeled anomaly data.  In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 1298–1308, 2021.
  * [62] Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier.  Padim: a patch distribution modeling framework for anomaly detection and localization.  In International Conference on Pattern Recognition, pages 475–489. Springer, 2021.
  * [63] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj.  Reconstruction by inpainting for visual anomaly detection.  Pattern Recognition, 112:107706, 2021.
  * [64] Yiru Zhao, Bing Deng, Chen Shen, Yao Liu, Hongtao Lu, and Xian-Sheng Hua.  Spatio-temporal autoencoder for video anomaly detection.  In Proceedings of the 25th ACM international conference on Multimedia, pages 1933–1941, 2017.
  * [65] Julian Wyatt, Adam Leach, Sebastian M Schmon, and Chris G Willcocks.  Anoddpm: Anomaly detection with denoising diffusion probabilistic models using simplex noise.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 650–656, 2022.
  * [66] Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Georg Langs, and Ursula Schmidt-Erfurth.  f-anogan: Fast unsupervised anomaly detection with generative adversarial networks.  Medical image analysis, 54:30–44, 2019.
  * [67] Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Pfister.  Cutpaste: Self-supervised learning for anomaly detection and localization.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9664–9674, 2021.
  * [68] Choubo Ding, Guansong Pang, and Chunhua Shen.  Catching both gray and black swans: Open-set supervised anomaly detection.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7388–7398, 2022.
  * [69] Yiwei Lu, Frank Yu, Mahesh Kumar Krishna Reddy, and Yang Wang.  Few-shot scene-adaptive anomaly detection.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16, pages 125–141. Springer, 2020.
  * [70] Tri Cao, Jiawen Zhu, and Guansong Pang.  Anomaly detection under distribution shift.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6511–6523, 2023.
  * [71] Abhishek Aich, Kuan-Chuan Peng, and Amit K Roy-Chowdhury.  Cross-domain video anomaly detection without target domain adaptation.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2579–2591, 2023.
  * [72] Wenchao Yu, Wei Cheng, Charu C Aggarwal, Kai Zhang, Haifeng Chen, and Wei Wang.  Netwalk: A flexible deep embedding approach for anomaly detection in dynamic networks.  In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2672–2681, 2018.
  * [73] Yixin Liu, Shirui Pan, Yu Guang Wang, Fei Xiong, Liang Wang, Qingfeng Chen, and Vincent CS Lee.  Anomaly detection in dynamic graphs via transformer.  IEEE Transactions on Knowledge and Data Engineering, 35(12):12081–12094, 2021.
  * [74] Rongrong Ma, Guansong Pang, Ling Chen, and Anton van den Hengel.  Deep graph-level anomaly detection by glocal knowledge distillation.  In Proceedings of the fifteenth ACM international conference on web search and data mining, pages 704–714, 2022.
  * [75] Yixin Liu, Kaize Ding, Qinghua Lu, Fuyi Li, Leo Yu Zhang, and Shirui Pan.  Towards self-interpretable graph-level anomaly detection.  Advances in Neural Information Processing Systems, 36, 2023.
  * [76] Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov, and Zornitsa Kozareva.  Improving in-context few-shot learning via self-supervised training.  arXiv preprint arXiv:2205.01703, 2022.
  * [77] Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi.  Metaicl: Learning to learn in context.  arXiv preprint arXiv:2110.15943, 2021.
  * [78] Amir Bar, Yossi Gandelsman, Trevor Darrell, Amir Globerson, and Alexei Efros.  Visual prompting via image inpainting.  Advances in Neural Information Processing Systems, 35:25005–25017, 2022.
  * [79] Yulong Pei, Tianjin Huang, Werner van Ipenburg, and Mykola Pechenizkiy.  Resgcn: Attention-based deep residual modeling for anomaly detection on attributed networks.  Machine Learning, 111(2):519–541, 2022.
  * [80] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad.  Collective classification in network data.  AI magazine, 29(3):93–93, 2008.
  * [81] Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su.  Arnetminer: extraction and mining of academic social networks.  In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 990–998, 2008.
  * [82] Lei Tang and Huan Liu.  Relational learning via latent social dimensions.  In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 817–826, 2009.
  * [83] Julian John McAuley and Jure Leskovec.  From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews.  In Proceedings of the 22nd international conference on World Wide Web, pages 897–908, 2013.
  * [84] Shebuti Rayana and Leman Akoglu.  Collective opinion spam detection: Bridging review networks and metadata.  In Proceedings of the 21th acm sigkdd international conference on knowledge discovery and data mining, pages 985–994, 2015.
  * [85] Shijie Zhang, Hongzhi Yin, Tong Chen, Quoc Viet Nguyen Hung, Zi Huang, and Lizhen Cui.  Gcn-based user representation learning for unifying robust recommendation and fraudster detection.  In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pages 689–698, 2020.
  * [86] Arjun Mukherjee, Vivek Venkataraman, Bing Liu, and Natalie Glance.  What yelp fake review filter might be doing?  In Proceedings of the international AAAI conference on web and social media, volume 7, pages 409–418, 2013.
  * [87] Zhiming Xu, Xiao Huang, Yue Zhao, Yushun Dong, and Jundong Li.  Contrastive attributed network anomaly detection with data augmentation.  In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 444–457. Springer, 2022.
  * [88] Srijan Kumar, Xikun Zhang, and Jure Leskovec.  Predicting dynamic embedding trajectory in temporal interaction networks.  In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1269–1278, 2019.
  * [89] Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova.  A critical look at the evaluation of gnns under heterophily: Are we really making progress?  arXiv preprint arXiv:2302.11640, 2023.
  * [90] Kaize Ding, Jundong Li, and Huan Liu.  Interactive anomaly detection on attributed networks.  In Proceedings of the twelfth ACM international conference on web search and data mining, pages 357–365, 2019.
  * [91] David B Skillicorn.  Detecting anomalies in graphs.  In 2007 IEEE Intelligence and Security Informatics, pages 209–216. IEEE, 2007.
  * [92] Xiuyao Song, Mingxi Wu, Christopher Jermaine, and Sanjay Ranka.  Conditional anomaly detection.  IEEE Transactions on knowledge and Data Engineering, 19(5):631–645, 2007.

##  Appendix A Detailing Related Work

Anomaly Detection. The objective of anomaly detection (AD) is to identify
anomalous samples that deviate from the majority of samples [20]. Due to the
difficulty of collecting labeled anomaly data, mainstream AD methods mainly
focus on unsupervised settings. To capture anomaly patterns without guidance
by annotated labels, existing studies employ several advanced techniques to
learn powerful AD models, such as one-class classification [21, 22], distance
measurement [23, 62], data reconstruction [24, 63, 64], generative models [25,
65, 66], and self-supervised learning [26, 67]. For example, DeepSVDD [21]
introduces a fully deep one-class classification objective for unsupervised
anomaly detection, optimizing a data-enclosing hypersphere in output space to
extract common factors of variation and demonstrating theoretical properties
such as the $v$-property. AnoDDPM [65], a simplex noise-based approach for
anomaly detection, enhances anomaly capture with a partial diffusion strategy
and multiscale simplex noise processing, facilitating faster inference and
training on high-resolution images. While effective, these approaches are
tailored to identify abnormal samples within a predetermined target dataset
(i.e., the dataset for training), restricting their generalizability to new
domains.

Cross-Dataset Anomaly Detection. Recently, some advanced AD methods aim to
transcend dataset limitations, enhancing their generalizability across diverse
datasets. A research line aims to address the AD problem under domain or
distribution shifts [68, 69, 70, 71]; however, these approaches require domain
relevance between the source and target datasets, thus limiting their
generalizability. To enable the model to better understand the patterns in
target datasets, a viable approach is to incorporate a few-shot setting,
allowing access to a limited number of normal samples from the target
datasets. Under the few-shot setting, RegAD [27] is a pioneering approach that
trains a single generalizable model capable of being applied to new in-domain
data without re-training or fine-tuning. WinCLIP [28] utilizes visual-language
models (VLMs, e.g., CLIP [29]) with well-crafted text prompts to perform
zero/few-shot AD for image data. InCTRL [17], as the first generalist AD
approach, integrates in-context learning and VLMs to achieve domain-agnostic
image AD with a single model. However, due to their heavy reliance on pre-
trained vision encoders/VLMs and image-specific designs, these approaches
excel in anomaly detection for image data but face challenges when applied to
graph data.

Anomaly Detection on Graph Data. Based on the granularity of anomaly samples
within graph data, existing AD approaches can be primarily categorized into
three classes: node-level [4, 6], edge-level [72, 73], and graph-level [74,
75] AD. Due to its broad real-world applications, node-level AD receives the
most research attention [3]. In this paper, we focus on the node-level AD and
refer to it as “graph anomaly detection (GAD)”, following the convention of
most previous papers [6, 30, 31].

Early GAD methods aimed to detect anomalies through shallow mechanisms. For
example, AMEN [8] detects anomalies by utilizing the attribute correlations of
nodes in each ego-network on the attribute network. In addition, residual
analysis is another common method to measure the anomalies of nodes on an
attribute network. In particular, Radar [9] characterizes the residuals of
attribute information and their coherence with network information for anomaly
detection. Further, ANOMALOUS [10] proposes joint learning of attribute
selection and anomaly detection as a whole based on CUR decomposition and
residual analysis. Despite the success of these methods on low-dimensional
attribute network data, they do not work well when the network has a complex
structure and high-dimensional attributes due to the limitations of their
shallow mechanisms.

To overcome the limitations of shallow approaches, recently, GNN-based methods
have become the de facto solution for GAD tasks. Existing GNN-based GAD
approaches can be divided into two research lines: supervised GAD and
unsupervised GAD [3, 5, 7]. Supervised GAD approaches assume that the labels
of both normal and anomalous nodes are available for model training [7].
Hence, related studies mainly focus on improving graph convolutional
operators, model architectures, and supervised objective functions, to
leverage labels to learn node-level anomaly patterns [6, 11, 12, 13, 33, 34].
For example, the spatial GNN has been redesigned mainly in terms of message
passing and aggregation mechanisms. In particular, considering that GNN-based
fraud detectors fail to effectively identify fraudsters in disguise, CARE-GNN
[11] uses a label-aware similarity metric and introduces reinforcement
learning to aggregate selected neighbors with different relationships to
counteract disguises. Concurrently, spectral GNNs, relate graphical anomalies
to high-frequency spectral distributions. Tang et al. [6] analyzed anomalies
for the first time from the perspective of the spectral spectrum of the graph
and proposed the Beta wavelet GNN (BWGNN), which has spectrally and spatially
localized band-pass filters to better capture anomalies. In addition,
anomalies are usually associated with high-frequency components in the
spectral representation of a graph. Therefore, GHRN [34] prunes inter-class
edges by emphasizing the high-frequency components of the graph, which can
effectively isolate the anomalous nodes and thus obtain better anomaly
detection performance.

In contrast, unsupervised GAD approaches do not require any labels for model
training. Similar to unsupervised AD for image data, unsupervised GAD
approaches employ several unsupervised learning techniques to learn anomaly
patterns on graph data, including data reconstruction [4, 31, 35], contrastive
learning [14, 36, 37], and other auxiliary objectives [15, 16, 38]. For
example, DOMINANT [4] is a reconstruction-based approach that employs a graph
convolution autoencoder to reconstruct both the adjacency and attribute
matrices simultaneously, assessing node abnormality through a weighted sum of
the reconstruction error terms. Similarly, ComGA [31] is a community-aware
attribute GAD framework based on tailored GCNs to capture local, global, and
structural anomalies. CoLA [14], the first contrastive self-supervised
learning framework for GAD, samples novel contrast instance pairs in an
unsupervised manner, utilizing contrastive learning to capture local
information. HCM-A [15] integrates both local and global contextual
information, employs hop count prediction as a self-supervised task, and
utilizes Bayesian learning to enhance anomaly identification. TAM [16]
optimizes the proposed anomaly metric (affinity) on the truncated graph end-
to-end, considering one-class homophily and local affinity. Nevertheless, all
the above methods adhere to the conventional paradigm of “one model for one
dataset”.

Although some GAD approaches [39, 40] can handle cross-domain scenarios, their
requirement for high correlation (e.g., aligned node features) between source
and target datasets limits their generalizability. Differing from existing
methods, our proposed ARC is a “one-for-all” GAD model capable of identifying
anomalies across target datasets from diverse domains, without the need for
re-training or fine-tuning.

In-Context Learning. In-context learning (ICL) can be effectively adapted to
new tasks based on minimal in-context examples, providing a powerful
generalization capability of large language models (LLMs) [41, 42, 43]. For
example, Brown et al. [41] demonstrate the remarkable ability of language
models to perform diverse tasks with minimal training examples. Through few-
shot learning, these models exhibit robustness and adaptability across various
domains, exhibiting their potential as general-purpose learners in natural
language processing (NLP). Further, given that pre-training objectives are not
specifically optimized for ICL [76], Min et al. introduced MetaICL [77] as a
solution to bridge the divide between pre-training and downstream ICL
utilization. This method involves continuously training the pre-trained LLMs
across diverse tasks using demonstration examples, thereby enhancing its few-
shot capabilities. In contrast, supervised in-context fine-tuning [76]
suggests constructing self-supervised training data aligned with the ICL
format to utilize the original corpora for warm-up in downstream tasks. They
converted raw text into input-output pairs and investigated four self-
supervised objectives, such as masked token prediction and classification
tasks.

ICL has also generated research attention in the field of computer vision
(CV), where it has been widely used in vision tasks by designing specialized
discretization tokens as prompts [17, 44, 45, 46]. For instance, Chen et al.
[44] proposed to use a unified interface to represent the output of each task
as a sequence of discrete tokens, the neural network can be trained for a
variety of tasks using a single model architecture and loss function, thus
eliminating the need for customization for specific tasks. Furthermore, given
a shot prompt as a task description, the sequence output adapts to the prompt
to generate task-specific results. Differently, Amir et al. [78] proposes an
innovative method for in-context visual prompting by framing a wide range of
vision tasks as grid in-painting problems. This approach leverages image in-
painting to generate visual prompts, guiding models to complete various vision
tasks by filling in missing parts of an image based on contextual information,
thereby enhancing adaptability and performance across different visual
challenges.

Very recently, few studies applied ICL to graph learning and GNNs. As an
initial attempt to use ICL for GNNs, PRODIGY [47] leverages a prompt graph-
based framework to conduct few-shot ICL for node-level classification and
edge-level prediction tasks. Further, UniLP [48] introduces ICL to resolve
conflicting connectivity patterns caused by distributional differences between
different graphs. Nevertheless, these methods require context samples in
multiple classes for ICL during inference. Then, how to leverage one-class
context samples with ICL in the scenario of generalist GAD remains an open
question.

##  Appendix B Motivated Experiments for Smoothness-Based Feature Sorting

(a) CiteSeer

(b) PubMed

(c) BlogCatalog

(d) Flickr

(e) ACM

Figure 8: AUROC on data with 5 groups of features split by $s_{k}$.

Experimental Setup. To verify whether smoothness can indicate the contribution
of features to GAD, we conducted motivated experiments. We consider three
mainstream GAD methods: DOMINANT [4], CoLA [14], and TAM [16]. For each
method, we conduct the experiments with hyper-parameters reported in the
original article across all datasets. At the data pre-processing phase, we
first calculate $s_{k}$ based on the raw features and sort them in descending
order. According to the new order, we divide the features into 5 groups of
feature subsets, denoted by the percentile of 80%-100%, 60%-80%, $\cdots$,
0%-20%. Subsequently, the five feature subsets are sequentially used as inputs
for different methods, and the methods are trained to obtain their AUROC
values. We statistic the results of 5 random experiments for each method and
reported the average AUROC.

Results. Additional experimental results are shown in Fig. 8. As we can
witness in the figure, in most cases, features with lower $s_{k}$ are observed
to be more helpful in distinguishing anomalies on most of the datasets. This
consistent model-independent trend indicates the effectiveness of $s_{k}$ as
an indicator of the contribution of each feature in GAD.

##  Appendix C Discussion of Ego-Neighbor Residual Graph Encoder

In this section, we discuss the connections and differences between the ego-
neighbor residual graph encoder in ARC (R-ENC for short) and the existing GNNs
and GAD methods.

R-ENC v.s. Radar [9]. Radar, one of the representative shallow GAD methods,
also leverages a residual-based mechanism for anomaly detection. Our proposed
R-ENC differs from Radar in the following key aspects:

  * •

Radar calculates the residual on the original feature space, which inevitably
suffers from high complexity and less representative capability on high-
dimensional graph data. In contrast, R-ENC computes residuals based on the
output embeddings of a GNN, which enables more efficient processing and
enhances the representative power of the model. This approach allows R-ENC to
better capture the underlying structure and anomalies within the graph data.

  * •

In Radar, the summation of residual $\mathbf{R}$ directly serves as the
anomaly score for each node, where the irrelevant information within several
residual entries may hinder the model’s accuracy and effectiveness.
Differently, in ARC, we specifically use R-ENC to generate the embedding of
each node, and employ another learnable scoring module to calculate the
anomaly score based on these embeddings. In this case, our method can
effectively filter out noise and irrelevant features, allowing the scoring
module to focus on the most informative aspects of the node embeddings.

  * •

Radar employs the Laplacian matrix to calculate the residual, meaning that
only first-order residual information is considered. In contrast, our R-ENC
incorporates multi-hop residual aggregation, enhancing its ability to detect
subtle anomalies by considering both local and global graph structures through
high-order residuals.

R-ENC v.s. ResGCN [79]. ResGCN is a GNN-based GAD approach with a residual-
based mechanism. Similar to Radar, ResGCN also uses the summation of residual
$\mathbf{R}$ as the anomaly score. However, the inclusion of irrelevant
information within several residual entries can impair the model’s accuracy
and effectiveness. Moreover, ResGCN employs a two-branch design, with the node
representation by GCN and residual information by MLP calculated by two
different network modules. Compared to R-ENC with simpler designs, ResGCN has
lower operational efficiency.

R-ENC v.s. CoLA [14]. CoLA learns the anomaly patterns by maximizing the
agreement between the embedding of each node and its neighboring nodes.
Specifically, CoLA employs a bilinear module to compute the agreement score.
Such an agreement can also be modeled by the first-order residual in R-ENC,
which is computed by the difference between the ego embedding and the 1-hop
aggregated embeddings. Compared to CoLA, a significant advantage of R-ENC is
its ability to capture not only first-order residuals but also high-order
residuals. This makes R-ENC a more robust and comprehensive encoder for graph
anomaly detection, addressing a wider range of anomaly patterns across
different datasets.

R-ENC v.s. TAM [16]. TAM utilizes local affinity, i.e., the feature-level or
embedding-level similarity between a node and its neighbors, as the indicator
of each node’s abnormality. R-ENC, with its residual operation, can also
capture local affinity. Specifically, by computing the first-order residual
between an ego node and its 1-hop neighbors, the local affinity can be
indicated by the negative summation of the first-order residuals since they
are highly correlated. Again, R-ENC can not only capture the first-order
affinity but also the high-order affinity with the multi-hop residual
operator.

R-ENC v.s. Heterophily-aware GNNs [34]. Existing studies indicate that a key
solution to handle the GAD problem is to enable heterophily-aware graph
convolutional operation with high-pass filtering [34]. A feasible filter is
graph Laplacian, whose normalized and self-loop added version can be written
by $\mathbf{L}=\mathbf{I}-\tilde{\mathbf{A}}$. For R-ENC, its first-order
residual can also be viewed as a Laplacian-based graph convolution.
Specifically, if we simplify the MLP-based feature transformation as a weight
matrix $\mathbf{W}$, the ego information can be written by
$\mathbf{Z}^{[0]}=\mathbf{X}\mathbf{W}$, while the representation
$\mathbf{Z}^{[1]}=\tilde{\mathbf{A}}\mathbf{X}\mathbf{W}$. Then, the first-
order residual can be written by:

$\mathbf{R}^{[1]}=\mathbf{Z}^{[1]}-\mathbf{Z}^{[0]}=\tilde{\mathbf{A}}\mathbf{X}\mathbf{W}-\mathbf{X}\mathbf{W}=-\mathbf{L}\mathbf{X}\mathbf{W}.$
(7)

That is to say, the first-order residual in R-ENC can be regarded as
Laplacian-based high-pass filtering (note that the negative sign can be fused
into the learnable weight $\mathbf{W}$). Such a nice property enables ARC to
capture high-frequency graph signals and heterophily information through
residual-based embeddings, thereby enhancing its capability to detect
anomalies in diverse and complex graph structures.

##  Appendix D Discussion of Cross-Attentive In-Context Anomaly Scoring

In this section, we first introduce the basic definition of one-class
classification (OC) model, and then discuss the connection between one-class
classification and cross-attentive in-context anomaly scoring module (C-AS for
short).

One-class classification. The core idea of OC model is to measure the
abnormality of each sample according to the distance between its
representation $\mathbf{h}$ and a center representation $\mathbf{c}$ [21].
Here $\mathbf{c}$ can be a fixed random representation vector or dynamically
adjusted as the mean of all samples’ representation vectors. Formally, the
anomaly score by OC model can be written by:

$f(\mathbf{x}_{i})=\left\|\phi\left(\mathbf{x}_{i}\right)-\mathbf{c}\right\|^{2}=\left\|\mathbf{h}_{i}-\mathbf{c}\right\|^{2},$
(8)

where $\phi(\cdot)$ is a neural network model, as defined in Deep SVDD [21].
Intuitively, a normal sample tends to have a similar representation to the
majority of samples, and hence the distance between its representation and
$\mathbf{c}$ should be closer.

C-AS as an OC model. In C-AS, we use a cross-attention block to calculate the
weighted sum of context embeddings $\mathbf{H}_{k}$ into
$\tilde{\mathbf{H}}_{q_{i}}$ for a query node $q_{i}$. For an initialized
model, we assume that the parameters $\mathbf{W}_{q}$ and $\mathbf{W}_{k}$ are
random enough, making $\mathbf{Q}$ and $\mathbf{K}$ become uniform noise
matrices. In this case, each entry in the attention matrix
$\mathbf{T}=\operatorname{Softmax}\left(\frac{\mathbf{Q}\mathbf{K}^{\top}}{\sqrt{d_{e}}}\right)$
can be $\frac{1}{n_{k}}$, indicating that the attention matrix assigns uniform
weights to all context nodes for all query nodes. Then, all the reconstructed
query embeddings are equal to the average embedding of context nodes:

$\tilde{\mathbf{H}}_{q_{1}}=\cdots=\tilde{\mathbf{H}}_{q_{n_{q}}}=\frac{1}{n_{k}}\mathbf{1}^{T}\mathbf{H}_{k}.$
(9)

Since the average context embedding is the center embedding of a group of few-
shot normal samples, we can naturally define the center embedding
$\mathbf{c}=\frac{1}{n_{k}}\mathbf{1}^{T}\mathbf{H}_{k}$. Recalling that we
define the anomaly score as the L2 distance between
$\tilde{\mathbf{H}}_{q_{i}}$ and ${\mathbf{H}}_{q_{i}}$, then for all query
nodes, the anomaly scoring can be rewritten by:

$f(v_{i})=d({\mathbf{H}}{q_{i}},\tilde{\mathbf{H}}{q_{i}})=d({\mathbf{H}}{q_{i}},\mathbf{c})=\left\|\mathbf{H}_{q_{i}}-\mathbf{c}\right\|^{2}.$
(10)

That is to say, the C-AS module serves as an OC model under random
initialization. Note that in practice, the attention matrix cannot be so
ideal, but it can still assign relevantly average weights for the context
embeddings. Such merit ensures that ARC can perform like an OC model,
effectively detecting anomalies in the case of single-class normal (Fig. 4
(a)) even without costly training.

C-AS goes beyond OC model. Thanks to its inherent mechanism, the OC model can
effectively handle single-class normal scenarios. However, in the case of
multi-class normals (Fig. 4 (b)), a single center is not sufficient to model
multiple normal class centers. Unlike the OC model, C-AS can address this
issue through cross-attention. Specifically, the cross-attention block learns
to reconstruct a query embedding by assigning higher weights to several (but
not all) context embeddings that are close to the query node. This way, for a
query node, the cross-attention block can automatically learn the center of
its corresponding normal class, rather than simply using the average context
embedding. The awareness of multiple normal classes ensures that ARC can
handle both single-class and multi-class normal cases.

##  Appendix E Algorithm and Complexity

###  E.1 Algorithmic description

The algorithmic description of the feature alignment in ARC, the training
process of ARC, and inference process of ARC are summarized in Algo. 1, Algo.
2, and Algo. 3, respectively.

Input: Graph $\mathcal{G}$.

Parameters : Projected dimension $d_{u}$.

1 Extract $\mathbf{X}$, $\mathcal{E}$, and $\mathcal{V}$ from $\mathcal{G}$

2 $\tilde{\mathbf{X}}\in\mathbb{R}^{n\times d_{u}}\leftarrow$ Calculate
projected features by linear projection via Eq. (1)

3 for  _$k=1:d_{u}$_ do

4 $s_{k}\leftarrow$ Calculate feature-level smoothness of the $k$-th column of
$\tilde{\mathbf{X}}$ via Eq. (2)

5 end for

6${\mathbf{X}^{\prime}}\leftarrow$ Rearrange the permutation of features of
$\tilde{\mathbf{X}}$ based on the descending order of $\mathbf{s}$

Return
$\mathcal{G}=\left(\mathcal{V},\mathcal{E},{\mathbf{X}}^{\prime}\right)$

Algorithm 1 Smoothness-based Feature Alignment

Input: Training datasets $\mathcal{T}_{train}$.

Parameters : Number of epoch $E$; Propagation iteration: $L$.

1 Initialize model parameters

2 for  _$\mathcal{D}^{(i)}\in\mathcal{T}_{train}$_ do

3 Align features in $\mathcal{G}^{(i)}$ via Algo. 1

4 end for

5for  _$e=1:E$_ do

6 for  _$\mathcal{D}^{(i)}\in\mathcal{T}_{train}$_ do

7 Obtain
${\mathbf{X}}^{(i)^{\prime}},\mathcal{E}^{(i)},\mathcal{V}^{(i)},\mathbf{y}^{(i)}$
from $\mathcal{D}^{(i)}$

8 for  _$l=1:L$_ do

9 $\mathbf{Z}^{(i),[l]}\leftarrow$ Propagate and transform
${\mathbf{X}}^{(i)^{\prime}}=\mathbf{X}^{(i),[0]}$ via Eq. (3)

10 $\mathbf{R}^{(i),[l]}\leftarrow$ Calculate residual of
$\mathbf{Z}^{(i),[l]}$ via Eq. (4)

11 end for

12 $\mathbf{H}^{(i)}\leftarrow$ Concatenate
$[\mathbf{R}^{(i),[1]}||\cdots||\mathbf{R}^{(i),[L]}]$ via Eq. (4)

13 $\mathbf{H}^{(i)}_{q}$, $\mathbf{H}^{(i)}_{k}\leftarrow$ Randomly split
query and context node sets and indexing from $\mathbf{H}^{(i)}$

14 $\tilde{\mathbf{H}}^{(i)}\leftarrow$ Calculate cross attention from
$\mathbf{H}^{(i)}_{q}$, $\mathbf{H}^{(i)}_{k}$ via Eq. (5)

15 Calculate loss $\mathcal{L}$ from $\tilde{\mathbf{H}}^{(i)}_{q}$,
$y^{(i)}_{q}$ via Eq. (6)

16 Update model parameters via gradient descent.

17

18 end for

19

20 end for

Algorithm 2 The Training algorithm of ARC

Input: Test dataset $\mathcal{D}$ with few-shot normal nodes
$\\{v_{k_{1}},\cdots,v_{k_{n_{k}}}\\}$.

Parameters : Well-trained model weight parameters.

1

2Align features in $\mathcal{G}$ via Algo. 1

3 Obtain ${\mathbf{X}}^{\prime},\mathcal{E},\mathcal{V}$ from $\mathcal{G}$

4 for  _$l=1:L$_ do

5 $\mathbf{Z}^{[l]}\leftarrow$ Propagate and transform
${\mathbf{X}}^{(i)^{\prime}}=\mathbf{X}^{[0]}$ via Eq. (3)

6 $\mathbf{R}^{[l]}\leftarrow$ Calculate residual of $\mathbf{Z}^{[l]}$ via
Eq. (4)

7 end for

8$\mathbf{H}\leftarrow$ Concatenate
$[\mathbf{R}^{[1]}||\cdots||\mathbf{R}^{[L]}]$ via Eq. (4)

9 $\mathbf{H}_{q}$, $\mathbf{H}_{k}\leftarrow$ Separate query and context node
sets and indexing from $\mathbf{H}$

10 $\tilde{\mathbf{H}}_{q}\leftarrow$ Calculate cross attention from
$\mathbf{H}_{q}$, $\mathbf{H}_{k}$ via Eq. (5)

11 $\mathbf{d}\leftarrow$ Computing the L2 distance between
$\tilde{\mathbf{H}}_{q}$ and $\mathbf{H}_{q}$

Return $\mathbf{d}$ as the anomaly scores $f(\cdot)$ for query nodes

Algorithm 3 The Inference algorithm of ARC

###  E.2 Complexity Analysis

In the testing phase, the time complexity consists of two main components:
feature alignment and model inference. For feature alignment, the overall
complexity is $\mathcal{O}(ndd_{u}+d_{u}m+d_{u}log(d_{u}))$, where $m=|E|$ is
the number of edges. Here, the first term is used for feature projection,
while the second and third terms are used for smoothness computation and
feature reordering, respectively. The model inference is divided into two main
parts: embedding generation and anomaly scoring. The complexity of node
embedding generation is $\mathcal{O}(L(md_{u}+nd_{u}h+nh^{2}))$, where the
first term is used for feature propagation and the rest of the terms are used
for residual encoding by MLP. The anomaly scoring, on the other hand, mainly
involves cross-attention computation with time complexity of
$\mathcal{O}(n_{q}n_{k}h+n_{q}h)$, where $n_{q}$ is the number of query nodes
and $n_{k}$ is the number of context nodes.

##  Appendix F Details of Experimental Setup

###  F.1 Description of Datasets

Table 3: The statistics of datasets.

Dataset | Train | Test | #Nodes | #Edges | #Features | Avg. Degree | #Anomaly | %Anomaly
---|---|---|---|---|---|---|---|---
Citation network with injected anomalies
Cora | - | $\checkmark$ | 2,708 | 5,429 | 1,433 | 3.90 | 150 | 5.53
CiteSeer | - | $\checkmark$ | 3,327 | 4,732 | 3,703 | 2.77 | 150 | 4.50
ACM | - | $\checkmark$ | 16,484 | 71,980 | 8,337 | 8.73 | 597 | 3.62
PubMed | $\checkmark$ | - | 19,717 | 44,338 | 500 | 4.50 | 600 | 3.04
Social network with injected anomalies
BlogCatalog | - | $\checkmark$ | 5,196 | 171,743 | 8,189 | 66.11 | 300 | 5.77
Flickr | $\checkmark$ | - | 7,575 | 239,738 | 12,047 | 63.30 | 450 | 5.94
Social network with real anomalies
Facebook | - | $\checkmark$ | 1,081 | 55,104 | 576 | 50.97 | 25 | 2.31
Weibo | - | $\checkmark$ | 8,405 | 407,963 | 400 | 48.53 | 868 | 10.30
Reddit | - | $\checkmark$ | 10,984 | 168,016 | 64 | 15.30 | 366 | 3.33
Questions | $\checkmark$ | - | 48,921 | 153,540 | 301 | 3.13 | 1,460 | 2.98
Co-review network with real anomalies
Amazon | - | $\checkmark$ | 10,244 | 175,608 | 25 | 17.18 | 693 | 6.76
YelpChi | $\checkmark$ | - | 23,831 | 49,315 | 32 | 2.07 | 1,217 | 5.10

In total, we considered 12 benchmark datasets. We divide the datasets into 4
groups: ❶ citation network with injected anomalies, ❷ social network with
injected anomalies, ❸ social network with real anomalies, and ❹ co-review
network with real anomalies. Within each type, we consider the largest dataset
as one of the training datasets, and the rest datasets as the testing
datasets. The detailed statistics of the datasets are shown in Table 3. These
datasets are selected from different domains and with injected or real
anomalies to ensure that our proposed ARC model learns extensive anomaly
patterns. The diversity of the above data can maximally ensure that ARC can
effectively adapt to new and unseen graphs. Specifically, the detailed
descriptions for the datasets are given as follows:

  * •

Cora, CiteSeer, PubMed [80], and ACM [81] are four citation networks. In these
datasets, nodes represent scientific publications, while edges denote the
citation links between them. Each publication is characterized by a bag-of-
words representation for its node attribute vector, with the dimensionality
determined by the size of the respective dictionary.

  * •

BlogCatalog and Flickr [82, 4] stand as typical social blog directories,
facilitating user connections through following relationships. Each user is
depicted as a node, with inter-node links symbolizing mutual following. Node
attributes encompass the personalized textual content generated by users
within social network, such as blog posts or shared photos with tag
descriptions.

  * •

Amazon and YelpChi [83, 84] are datasets about the relationship between users
and reviews. Amazon is designed to identify users paid to write fake reviews
for products, and three different graph datasets are derived from Amazon using
different types of relations to construct adjacency matrix [85, 16]. YelpChi
aims to identify anomalous reviews on Yelp.com that unfairly promote or demote
products or businesses. Based on [84, 86], three different graph datasets
derived from Yelp using different connections in user, product review text,
and time. In this work, we focus on Amazon-UPU (users who have reviewed at
least one of the same product) and YelpChi-RUR (reviews posted by the same
user).

  * •

Facebook [87] is a social network in which users can build relationships with
others and share their friends.

  * •

Reddit [88] serves as a forum posts network sourced from the social media
platform Reddit, where users labeled as banned are identified as anomalies.
Textual content from posts is transformed into vectors to serve as node
attributes.

  * •

Weibo [88] dataset encompasses a graph of users and their associated hashtags
from the Tencent Weibo platform. Within a defined temporal window (e.g., 60
seconds), consecutive posts by a user are labeled as potentially suspicious
behavior. Users engaging in a minimum of five such instances are classified as
“suspicious”. The raw feature vector includes the location of a micro-blog
post and bag-of-words features.

  * •

Questions [89] dataset originates from Yandex Q, a platform dedicated to
question-answering. Users represent the nodes, while the connections between
them signify the presence or absence of a question-and-answer interaction
within a one-year timeframe. The node features are derived from the average of
the FastText embeddings of the words in the user description, with an
additional binary feature indicating users without description.

Anomaly Injection. For the datasets with injected anomalies, we use the
strategy introduced in [4, 14] to inject anomalous nodes. Specifically, we
inject a set of anomaly combinations for each dataset by perturbing the
topology and node attributes, respectively [90]. In terms of structural
perturbation, this is done by generating small cliques of otherwise unrelated
nodes as anomalies. The intuition for this strategy is that small cliques in
the real world are a typical substructure of anomalies, with much more closely
linked within the cliques than the mean [91]. Thus for a dataset, we can
specify the size of the cliques (i.e., the number of nodes) $p$ and its amount
$q$ for anomaly generation. Specifically, randomly sample $p$ nodes from the
graph making them fully connected and labeled as anomaly nodes. We iteratively
repeat the above process $q$ times to inject a total of $p\times q$ anomalies.
Finally, we control the number of injected anomalies according to the size of
the dataset. In particular, we fix $p=15$ and $q=10,15,20,5,5,20$ on
BlogCatalog, Flickr, ACM, Cora, Citeseer, and Pubmed, respectively. On the
other hand, for attribute perturbations, we base the schema introduced by
[92]. Specifically, for each perturbation target node $v_{i}$, $k$ nodes are
randomly sampled in the graph and their distance from the target node is
computed. Then, the node $v_{j}$ with the largest deviation from the target
node $v_{i}$ is selected, and the attribute $\mathbf{X}_{i}$ of the node
$v_{i}$ to $\mathbf{X}_{j}$. We set the number of anomalies of the attribute
perturbation to $p\times q$ to maintain the balance of different anomalies. In
addition, we set $k=50$ to ensure that the perturbation magnitude is large
enough.

###  F.2 Description of Baselines

In our evaluation, we provide a comprehensive comparsion of ARC with various
supervised and unsupervised GAD methods. On the supervised side, two classic
GNNs are included as well as 3 state-of-the-art (SOTA) models specifically
tailored for the GAD task. For supervised models, it is assumed that the
labels of both normal and abnormal nodes can be used for model training.
Therefore, the main binary classification task is used to identify the
anomalies:

  * •

GCN [54], as a seminal model in the field of GNN, is known for its ability to
process graph-structured data using neighborhood aggregation, facilitating
efficient node feature extraction and representation learning.

  * •

GAT [55] incorporates the attention mechanism into the GNN framework to
achieve dynamic weighting of node contributions. It optimizes its attention
according to different downstream tasks to achieve high-quality node
representations.

  * •

BGNN [60] is a GNN that combines gradient boost decision trees (GBDT) with GNN
for graphs with tabular node features. It utilizes the GBDT to handle
heterogeneous features while the GNN considers the graph structure and
significantly improves performance on a variety of graphs with tabular
features.

  * •

BWGNN [6] has spectral and spatial localized band-pass filters to better
handle the “right-shift” phenomenon in anomalies, i.e., the distribution of
spectral energy is concentrated at high frequencies rather than at low
frequencies.

  * •

GHRN [34] is a heterophily-aware supervised GAD method based on graph spectra.
By emphasizing the high-frequency components of the graph, the method can
effectively cut down inter-class edges, thus improving the overall performance
of anomaly detection.

For the unsupervised alternative, we consider 4 representative SOTA GAD
methods, each of them belonging to a sub-type: data reconstruction,
contrastive learning, hop-based auxiliary goal, or affinity-based auxiliary
goal:

  * •

DOMINANT [4] combines GCN and deep auto-encoder, and its learning objective is
to reconstruct the adjacency matrix and node features jointly. It aims to
identify structural and attribute anomalies based on reconstruction errors.

  * •

CoLA [14] is a contrastive self-supervised learning for anomaly detection on
graphs with node attributes. The framework captures the relationship between
each node and its neighborhood substructure in an unsupervised manner by
sampling novel pairs of contrasting instances and leveraging the local
information of the graph.

  * •

HCM-A [15] uses hop-count prediction as a self-supervised task to better
identify anomalies by modeling both local and global context information. In
addition, HCM-A designs two new anomaly scores and introduces Bayesian
learning to train the model to capture anomalies.

  * •

TAM [16] is designed based on one-class homophily and local affinity. The
learning target of TAM is to optimize the proposed anomaly metric (i.e.
affinity) end-to-end on the truncated adjacency matrix.

Table 4: Anomaly detection performance in terms of AUPRC (in percent,
mean$\pm$std). Highlighted are the results ranked first, second, and third.
“Rank” indicates the average ranking over 8 datasets.

Method | Cora | CiteSeer | ACM | BlogCatalog | Facebook | Weibo | Reddit | Amazon | Rank
---|---|---|---|---|---|---|---|---|---
Supervised - Pre-Train Only
GCN | $7.41{\scriptstyle\pm 1.55}$ | $6.40{\scriptstyle\pm 1.40}$ | $5.27{\scriptstyle\pm 1.12}$ | $7.44{\scriptstyle\pm 1.07}$ | $1.59{\scriptstyle\pm 0.11}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{67.21{\scriptstyle\pm 15.20}}}$ | $3.39{\scriptstyle\pm 0.39}$ | $6.96{\scriptstyle\pm 2.04}$ | $9.6$
GAT | $6.49{\scriptstyle\pm 0.84}$ | $5.58{\scriptstyle\pm 0.62}$ | $4.70{\scriptstyle\pm 0.75}$ | $12.81{\scriptstyle\pm 2.08}$ | $3.14{\scriptstyle\pm 0.37}$ | $33.34{\scriptstyle\pm 9.80}$ | $3.73{\scriptstyle\pm 0.54}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{15.74{\scriptstyle\pm 17.85}}}}$ | $7.1$
BGNN | $4.90{\scriptstyle\pm 1.27}$ | $3.91{\scriptstyle\pm 1.01}$ | $3.48{\scriptstyle\pm 1.33}$ | $5.73{\scriptstyle\pm 1.47}$ | $3.81{\scriptstyle\pm 2.12}$ | $30.26{\scriptstyle\pm 29.98}$ | $3.52{\scriptstyle\pm 0.50}$ | $7.51{\scriptstyle\pm 0.58}$ | $10.5$
BWGNN | $7.25{\scriptstyle\pm 0.80}$ | $6.35{\scriptstyle\pm 0.73}$ | $7.14{\scriptstyle\pm 0.20}$ | $8.99{\scriptstyle\pm 1.12}$ | $2.54{\scriptstyle\pm 0.63}$ | $12.13{\scriptstyle\pm 0.71}$ | $3.69{\scriptstyle\pm 0.81}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{13.12{\scriptstyle\pm 11.82}}}$ | $8.6$
GHRN | $9.56{\scriptstyle\pm 2.40}$ | $7.79{\scriptstyle\pm 2.01}$ | $5.61{\scriptstyle\pm 0.71}$ | $10.94{\scriptstyle\pm 2.56}$ | $2.41{\scriptstyle\pm 0.62}$ | $28.53{\scriptstyle\pm 7.38}$ | $3.24{\scriptstyle\pm 0.33}$ | $7.54{\scriptstyle\pm 2.01}$ | $8.4$
Unsupervised - Pre-Train Only
DOMINANT | $12.75{\scriptstyle\pm 0.71}$ | $13.85{\scriptstyle\pm 2.34}$ | $15.59{\scriptstyle\pm 2.69}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{35.22{\scriptstyle\pm 0.87}}}$ | $2.95{\scriptstyle\pm 0.06}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{81.47{\scriptstyle\pm 0.22}}}}$ | $3.49{\scriptstyle\pm 0.44}$ | $6.11{\scriptstyle\pm 0.29}$ | $6.0$
CoLA | $11.41{\scriptstyle\pm 3.51}$ | $8.33{\scriptstyle\pm 3.73}$ | $7.31{\scriptstyle\pm 1.45}$ | $6.04{\scriptstyle\pm 0.56}$ | ${1.90{\scriptstyle\pm 0.68}}$ | $7.59{\scriptstyle\pm 3.26}$ | $3.71{\scriptstyle\pm 0.67}$ | $11.06{\scriptstyle\pm 4.45}$ | $9.0$
HCM-A | $5.78{\scriptstyle\pm 0.76}$ | $4.18{\scriptstyle\pm 0.75}$ | $4.01{\scriptstyle\pm 0.61}$ | $6.89{\scriptstyle\pm 0.34}$ | $2.08{\scriptstyle\pm 0.60}$ | $21.91{\scriptstyle\pm 11.78}$ | $3.18{\scriptstyle\pm 0.23}$ | $5.87{\scriptstyle\pm 0.07}$ | $12.1$
TAM | $11.18{\scriptstyle\pm 0.75}$ | $11.55{\scriptstyle\pm 0.44}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{23.20{\scriptstyle\pm 2.36}}}$ | $10.57{\scriptstyle\pm 1.17}$ | $8.40{\scriptstyle\pm 0.97}$ | $16.46{\scriptstyle\pm 0.09}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{3.94{\scriptstyle\pm 0.13}}}$ | $10.75{\scriptstyle\pm 3.10}$ | $5.9$
Unsupervised - Pre-Train & Fine-Tune
DOMINANT | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{21.35{\scriptstyle\pm 0.74}}}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{23.02{\scriptstyle\pm 1.55}}}$ | $22.74{\scriptstyle\pm 0.95}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{35.79{\scriptstyle\pm 0.63}}}}$ | $3.56{\scriptstyle\pm 0.15}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{77.69{\scriptstyle\pm 1.43}}}$ | $3.84{\scriptstyle\pm 0.74}$ | $7.48{\scriptstyle\pm 0.46}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{3.9}}$
CoLA | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{13.91{\scriptstyle\pm 5.56}}}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{19.51{\scriptstyle\pm 3.73}}}$ | $8.48{\scriptstyle\pm 0.51}$ | $10.43{\scriptstyle\pm 1.22}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{15.19{\scriptstyle\pm 11.04}}}}$ | $8.03{\scriptstyle\pm 1.19}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{4.07{\scriptstyle\pm 0.13}}}$ | $7.27{\scriptstyle\pm 1.13}$ | $5.8$
HCM-A | $6.41{\scriptstyle\pm 1.33}$ | $4.76{\scriptstyle\pm 0.51}$ | $4.41{\scriptstyle\pm 0.63}$ | $6.62{\scriptstyle\pm 0.14}$ | $2.23{\scriptstyle\pm 0.76}$ | $27.20{\scriptstyle\pm 5.53}$ | $3.10{\scriptstyle\pm 0.19}$ | $5.64{\scriptstyle\pm 0.09}$ | $11.9$
TAM | $13.62{\scriptstyle\pm 0.53}$ | $18.66{\scriptstyle\pm 1.41}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{58.04{\scriptstyle\pm 8.17}}}}$ | $13.90{\scriptstyle\pm 0.53}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{11.11{\scriptstyle\pm 3.20}}}$ | $16.47{\scriptstyle\pm 0.08}$ | $3.93{\scriptstyle\pm 0.09}$ | $11.56{\scriptstyle\pm 1.80}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{4.1}}$
Ours
ARC | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{47.26{\scriptstyle\pm 0.91}}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{48.38{\scriptstyle\pm 0.70}}}}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{40.14{\scriptstyle\pm 0.09}}}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{34.80{\scriptstyle\pm 0.02}}}$ | ${\color[rgb]{0.92578125,0.328125,0.078125}\underline{8.96{\scriptstyle\pm 1.74}}}$ | ${65.46{\scriptstyle\pm 0.95}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{4.14{\scriptstyle\pm 0.25}}}}$ | ${\color[rgb]{0.078125,0.40625,0.66015625}\underline{15.45{\scriptstyle\pm 4.30}}}$ | ${\color[rgb]{0.078125,0.5,0.33203125}\underline{\mathbf{2.1}}}$

###  F.3 Details of Implementation

Hyper-parameters. We select some key hyper-parameters of ARC through random
search within specified grids. Specifically, the random search was performed
within the following search space:

  * •

Hidden layer dimension: {64, 128, 256, 512, 1024}

  * •

Number of MLP layers: {1, 2, 3, 4}

  * •

Propagation iteration: {1, 2, 3, 4, 5}

  * •

Dropout rate: {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8}

  * •

Learning rate: floats between $10^{-5}$ and $10^{-2}$

  * •

Weight decay: floats between $10^{-6}$ and $10^{-3}$

Implementation Pipeline. We employ a fixed set of hyper-parameters to build a
generalist GAD model for all datasets. First, we train all the methods
(including baselines and ARC) on the training set $\mathcal{T}_{train}$ with
full labels. Then, the methods are evaluated on each dataset from
$\mathcal{T}_{test}$ respectively. For baselines that require fine-tuning, we
further conduct dataset-specific tuning at this stage.

Metrics. Following [7, 16, 61], we employ two popular and complementary
evaluation metrics for evaluation, including area under the receiver operating
characteristic Curve (AUROC) and area under the precision-recall curve
(AUPRC). A higher AUROC/AUPRC value indicates better performance. We report
the average AUROC/AUPRC with standard deviations across 5 trials.

Computing Infrastructures. We implemented the proposed ARC using PyTorch
2.1.2, PyTorch Geometric (PyG) 2.3.1, and DGL 0.9.0. All experiments were
performed on a Linux server with an Inter Xeon microprocessor E-2288G CPU and
a Quadro RTX 6000 GPU.

##  Appendix G Supplemental Experiments

###  G.1 Performance Comparison in Terms of AUPRC

In terms of AUPRC, Table 4 gives comprehensive comparative results with
consistent observations with the AUROC results. Specifically, we have the
following observations. ❶ ARC still demonstrates strong anomaly detection in
generalist GAD scenarios without any fine-tuning. Specifically, ARC achieves
state-of-the-art performance on three of the eight datasets and demonstrates
competitive performance on the remaining datasets. On several datasets, ARC
showed significant improvement over the best baseline (e.g., $\uparrow$ 121.4%
on Cora, $\uparrow$ 110.2% on Citeseer). ❷ GAD methods that only pre-train
specific to a dataset usually result in poor generalization to new datasets.
Specifically, existing methods perform very erratically on different datasets,
which can be attributed to capturing only specific anomaly patterns. ❸ Using
dataset-specific fine-tuning, baseline methods can achieve better performance
in most case. However, in some cases the improvement can be small or even
negative, demonstrating the limitations of fine-tuning.

###  G.2 Effectiveness of Context Sample Number

For all test sets, we varied $n_{k}$ in the range of 2 to 100 and the results
are shown in Fig. 5 and Fig. 9. From the figure, we observe that in most cases
the performance of ARC increases with the involvement of more context nodes,
which indicates its ability to utilize these labeled normal nodes for context
learning. Moreover, even if $n_{k}$ is very small, ARC can still perform well
on most datasets.

(a) CiteSeer

(b) ACM

(c) BlogCatalog

(d) Weibo

(e) Reddit

(f) Amazon

Figure 9: Performance with varying $n_{k}$ on the rest of six datasets.

###  G.3 Detailed Results of Ablation Study

Table 5: Performance of ARC and its variants in terms of AUROC.

Variant | Cora | CiteSeer | ACM | BlogCatalog | Facebook | Weibo | Reddit | Amazon
---|---|---|---|---|---|---|---|---
ARC w/o A | $78.46{\scriptstyle\pm 1.21}$ | $83.76{\scriptstyle\pm 1.06}$ | $78.53{\scriptstyle\pm 0.19}$ | $74.44{\scriptstyle\pm 0.11}$ | $65.29{\scriptstyle\pm 1.72}$ | $90.66{\scriptstyle\pm 0.13}$ | $53.81{\scriptstyle\pm 1.71}$ | $65.95{\scriptstyle\pm 1.64}$
ARC w/o R | $30.35{\scriptstyle\pm 0.51}$ | $29.30{\scriptstyle\pm 0.24}$ | $60.74{\scriptstyle\pm 2.90}$ | $60.56{\scriptstyle\pm 4.54}$ | $13.82{\scriptstyle\pm 0.42}$ | $97.32{\scriptstyle\pm 0.45}$ | $46.52{\scriptstyle\pm 1.83}$ | $54.83{\scriptstyle\pm 1.48}$
ARC w/o C | $49.98{\scriptstyle\pm 0.33}$ | $50.69{\scriptstyle\pm 0.25}$ | $52.88{\scriptstyle\pm 0.88}$ | $72.01{\scriptstyle\pm 0.20}$ | $47.95{\scriptstyle\pm 0.63}$ | $49.34{\scriptstyle\pm 0.52}$ | $47.82{\scriptstyle\pm 0.42}$ | $44.64{\scriptstyle\pm 1.03}$
ARC | $85.33{\scriptstyle\pm 0.44}$ | $90.64{\scriptstyle\pm 0.32}$ | $79.27{\scriptstyle\pm 0.17}$ | $74.81{\scriptstyle\pm 0.28}$ | $69.57{\scriptstyle\pm 1.47}$ | $89.24{\scriptstyle\pm 0.35}$ | $58.95{\scriptstyle\pm 1.25}$ | $69.77{\scriptstyle\pm 3.88}$

Table 6: Performance of ARC and its variants in terms of AUPRC.

Variant | Cora | CiteSeer | ACM | BlogCatalog | Facebook | Weibo | Reddit | Amazon
---|---|---|---|---|---|---|---|---
ARC w/o A | $26.60{\scriptstyle\pm 1.18}$ | $25.16{\scriptstyle\pm 1.81}$ | $27.48{\scriptstyle\pm 1.39}$ | $34.28{\scriptstyle\pm 0.42}$ | $4.49{\scriptstyle\pm 0.53}$ | $68.87{\scriptstyle\pm 0.50}$ | $3.57{\scriptstyle\pm 0.10}$ | $9.76{\scriptstyle\pm 1.02}$
ARC w/o R | $6.77{\scriptstyle\pm 0.08}$ | $8.06{\scriptstyle\pm 0.46}$ | $7.74{\scriptstyle\pm 0.14}$ | $16.65{\scriptstyle\pm 1.22}$ | $1.36{\scriptstyle\pm 0.01}$ | $90.37{\scriptstyle\pm 0.66}$ | $2.95{\scriptstyle\pm 0.12}$ | $8.34{\scriptstyle\pm 0.47}$
ARC w/o C | $8.91{\scriptstyle\pm 0.31}$ | $10.98{\scriptstyle\pm 0.21}$ | $18.33{\scriptstyle\pm 0.31}$ | $33.73{\scriptstyle\pm 0.26}$ | $3.35{\scriptstyle\pm 0.14}$ | $13.67{\scriptstyle\pm 0.09}$ | $3.13{\scriptstyle\pm 0.05}$ | $5.85{\scriptstyle\pm 0.10}$
ARC | $47.26{\scriptstyle\pm 0.91}$ | $48.38{\scriptstyle\pm 0.70}$ | $40.14{\scriptstyle\pm 0.09}$ | $34.80{\scriptstyle\pm 0.02}$ | $8.96{\scriptstyle\pm 1.74}$ | $65.46{\scriptstyle\pm 0.95}$ | $4.14{\scriptstyle\pm 0.25}$ | $15.45{\scriptstyle\pm 4.30}$

To assess the effectiveness of the key design in the ARC, we conducted an
ablation study with three variants of the ARC, 1) w/o A: using random
projection to replace smoothness-based feature alignment; 2) w/o R: using GCN
to replace ego-neighbor residual graph encoder; and 3) w/o C: using binary
classification-based predictor and loss to replace cross-attentive in-context
anomaly scoring. As can be seen from Table 5 and Table 6, the two metrics
AUROC and AUPRC of ARC achieved the best in all datasets except Weibo dataset.
A possible explanation is that the Weibo dataset exhibits a particular pattern
of anomalies. In addition, all three key designs provide significant
improvements in performance.

###  G.4 Visualization

(a) CiteSeer

(b) ACM

(c) BlogCatalog

(d) Facebook

(e) Weibo

(f) Reddit

Figure 10: Attention visualization results for more datasets.

The attention weights between context nodes and query nodes in other datasets
are shown in Fig. 10. As can be seen in Fig. 10, for most of the datasets, the
“single-class normal” case in Fig. 4 (a) is met: ARC tends to assign a uniform
attention weight to normal nodes. This results in reconstructed embedding that
are very similar to the average embedding of context nodes; in contrast,
reconstructing anomalies using a combination of a few context nodes results in
their embedding being farther from the center. Moreover, corresponding to the
“multi-class normal” case in Fig. 4 (b): in Fig. 10 (f), it is observed that
each normal query nodes that follow two fixed patterns. In summary, the cross-
attention module enables ARC to adapt to various normal/abnormal distribution
patterns, conferring it generality.