11institutetext: The University of British Columbia, Vancouver, BC, Canada
11email<EMAIL_ADDRESS>
11institutetext:  Department of Electrical and Computer Engineering,
The University of British Columbia, Vancouver, BC V6Z 1Z4, Canada
22institutetext: Department of Medicine,
The University of British Columbia Vancouver, BC V6Z 1Z4, Canada

# Community-Aware Transformer for Autism Prediction in fMRI Connectome

Anushree Bannadabhavi 11 Soojin Lee 22 Wenlong Deng 11 Xiaoxiao Li 11
<EMAIL_ADDRESS>
###### Abstract

Autism spectrum disorder(ASD) is a lifelong neurodevelopmental condition that
affects social communication and behavior. Investigating functional magnetic
resonance imaging (fMRI)-based brain functional connectome can aid in the
understanding and diagnosis of ASD, leading to more effective treatments. The
brain is modeled as a network of brain Regions of Interest (ROIs), and ROIs
form communities and knowledge of these communities is crucial for ASD
diagnosis. On the one hand, Transformer-based models have proven to be highly
effective across several tasks, including fMRI connectome analysis to learn
useful representations of ROIs. On the other hand, existing transformer-based
models treat all ROIs equally and overlook the impact of community-specific
associations when learning node embeddings. To fill this gap, we propose a
novel method, Com-BrainTF, a hierarchical local-global transformer
architecture that learns intra and inter-community aware node embeddings for
ASD prediction task. Furthermore, we avoid over-parameterization by sharing
the local transformer parameters for different communities but optimize unique
learnable prompt tokens for each community. Our model outperforms state-of-
the-art (SOTA) architecture on ABIDE dataset and has high interpretability,
evident from the attention module. Our code is available at
https://github.com/ubc-tea/Com-BrainTF. ††This work is supported in part by
the Natural Sciences and Engineering Research Council of Canada (NSERC) and
Compute Canada.
Corresponding author: Xiaoxiao Li ()

##  1 Introduction

Autism spectrum disorder (ASD) is a developmental disorder that affects
communication, social interaction, and behaviour [18]. ASD diagnosis is
challenging as there are no definitive medical tests such as a blood tests,
and clinicians rely on behavioural and developmental assessments to accurately
diagnose ASD. As a more objective alternative measurement, neuroimaging tools,
such as fMRI has been widely used to derive and validate biomarkers associated
with ASD [9]. Conventionally, fMRI data is modelled as a functional
connectivity (FC) matrix, e.g., by calculating Pearson correlation
coefficients of pairwise brain ROIs to depict neural connections in the brain.
FC-based brain connectome analysis is a powerful tool to study connectivity
between ROIs and their impact on cognitive processes, facilitating diagnosis
and treatment of neurological disorders including ASD [14, 28].

Deep learning (DL) models have led to significant advances in various fields
including fMRI-based brain connectome analysis. For instance, BrainNetCNN[17]
proposes a unique convolutional neural network (CNN) architecture with special
edge-to-edge, edge-to-node, and node-to-graph convolutional filters.
Considering brain network’s non-euclidean nature, graph neural networks (GNNs)
[30] have emerged as a promising method for analyzing and modelling brain
connectome by constructing graph representation from FC matrices [30, 19, 16,
4]. More recently, transformers [29] have been utilized for brain connectome
analysis as they can learn long-range interaction between ROIs without a
predefined graph structure. Brain Network Transformer (BNT) [15] has
transformer encoders that learn node embeddings using a pearson correlation
matrix and a readout layer that learns brain clusters. BNT [15] shows that
transformer-based methods outperform CNN and GNN models for fMRI-based
classification.

It is worth noting that our brain is comprised of functional communities [12]
(often referred to as functional networks), which are groups of ROIs that
perform similar functions as an integrated network [25, 26]. These communities
are highly reproducible across different studies [3, 5] and are essential in
understanding the functional organization of the brain [11]. Their importance
extends beyond basic neuroscience, as functional communities have been shown
to be relevant in understanding cognitive behaviours [12], mental states [8],
and neurological and psychiatric disorders [24, 2]. For example, numerous
studies have found that individuals with ASD exhibit abnormalities in default
mode network (DMN) [22], which is characterized by a combination of hypo- and
hyper-connectivity with other functional networks. ASD is also associated with
lifelong impairments in attention across multiple domains, and significant
alterations in the dorsal attention network (DAN) and DMN have been linked to
these deficits [7]. Such prior knowledge of functional relationships between
functional communities, can be highly beneficial in predicting ASD. However,
existing DL models often overlook this information, resulting in sub-optimal
brain network analysis.

To address this limitation, we propose a novel community-aware transformer for
brain network analysis, dubbed Com-BrainTF, which integrates functional
community information into the transformer encoder. Com-BrainTF consists of a
hierarchical transformer with a local transformer that learns community-
specific embeddings, and a global transformer that fuses the whole brain
information. The local transformer takes FC matrices as input with its
parameters shared across all communities, while, personalized prompt tokens
are learnt to differentiate the local transformer embedding functions. The
local transformer’s output class tokens and node embeddings are passed to the
global transformer, and a pooling layer summarizes the final prediction. Our
approach enhances the accuracy of fMRI brain connectome analysis and improves
understanding of the brain’s functional organization. Our key contributions
are:
1\. We propose a novel local-global hierarchical transformer architecture to
efficiently learn and integrate community-aware ROI embeddings for brain
connectome analysis by utilizing both ROI-level and community-level
information.
2\. We avoid over-parameterization by sharing the local transformer parameters
and design personalized learnable prompt tokens for each community.
3\. We prove the efficacy of Com-BrainTF with quantitative and qualitative
experiment results. Our visualization demonstrates the ability of our model to
capture functional community patterns that are crucial for ASD vs. Healthy
Control (HC) classification.

##  2 Method

###  2.1 Overview

Problem Definition In brain connectome analysis, we first parcellate the brain
into $N$ ROIs based on a given atlas. FC matrix is constructed by calculating
the Pearson correlation coefficients between pairs of brain ROIs based on the
strength of their fMRI activations. Formally, given a brain graph with $N$
number of nodes, we have a symmetric FC matrix $X\in\mathbb{R}^{N\times N}$.
Node feature vector of ROI $j$ is defined as the $j^{th}$ row or column of
this matrix. Given $K$ functional communities and the membership of ROIs, we
rearrange the rows and columns of the FC matrix, resulting in $K$ input
matrices $\\{X_{1},X_{2},\dots,X_{K}\\}$. $X_{k}\in\mathbb{R}^{N_{k}\times N}$
is viewed as a $N_{k}$ length sequence with $N$ dimensional tokens and
$\sum_{k}N_{k}=N$ (Fig. 1(1)). This process helps in grouping together regions
with similar functional connectivity patterns, facilitating the analysis of
inter-community and intra-community connections. Com-BrainTF inputs $X_{k}$ to
the community $k$-specific local transformer and outputs $N$ dimensional
tokens $H_{k}\in\mathbb{R}^{N_{k}\times N}$. A global transformer learns
embedding $H=[H_{1},\dots,H_{k}]\mapsto Z_{L}\in\mathbb{R}^{N\times N}$,
followed by a pooling layer and multi-layer perceptrons (MLPs) to predict the
output.

Overview of our Pipeline Human brain connectome is a hierarchical structure
with ROIs in the same community having greater similarities compared to inter-
community similarities. Therefore, we designed a local-global transformer
architecture(Fig. 1(2)) that mimics this hierarchy and efficiently leverages
community labels to learn community-specific node embeddings. This approach
allows the model to effectively capture both local and global information. Our
model has three main components: a local transformer, a global transformer and
a pooling layer details of which are discussed in the following subsections.

Figure 1: 1) fMRI images parcellated by an atlas to obtain the Functional
connectivity matrix for ’N’ ROIs. Rows and columns are rearranged based on
community labels of each ROI to obtain matrices $X_{1},X_{2}...X_{K}$, inputs
to local transformer 2) Overview of our local-global transformer architecture
3) Transformer encoder module

###  2.2 Local-global transformer encoder

The transformer encoder[29] (Fig. 1(3)) is a crucial component of both local
and global transformers. The transformer encoder takes as input an FC matrix
and has a multi-head-attention module to capture interdependencies between
nodes using attention mechanism. The learnt node feature matrix $H_{i}$ is
given by

$H_{i}=(\parallel_{m=1}^{M}h^{m})W_{O},$ (1)
$h^{m}=\mbox{softmax}\Bigg{(}\frac{Q^{m}(K^{m})^{T}}{\sqrt{d_{k}^{m}}}\Bigg{)}V^{m},\mbox{where,}$
(2)

$Q^{m}=W_{Q}X_{i}^{\prime},K^{m}=W_{K}X_{i}^{\prime}$,
$V^{m}=W_{V}X_{i}^{\prime}$, $X_{i}^{\prime}=[p_{i},X_{i}]$, $M=$ number of
attention heads, $\parallel$ is concatenation operator, $W_{Q},W_{K},W_{V}$,
$W_{O}$ are model parameters.

Local Transformer: For optimal analysis of fMRI brain connectomes, it is
important to incorporate both functional community information and node
features. Therefore, we introduce the knowledge of community labels to the
network by grouping node features based on community labels producing inputs
$\\{X_{1},X_{2},\dots,X_{K}\\}$. However, separate local transformers for each
input would result in a significant increase in model parameters. Hence, we
use the same local transformer for all inputs, but introduce unique learnable
personalized prompt tokens $\\{p_{1},p_{2},\dots,p_{K}\\}$, where
$p_{i}\in\mathbb{R}^{1\times N}$, that learn to distinguish between node
feature matrices of each community and therefore avoid over-parameterization
of the model. Previous transformer-based models like BNT[15] do not have any
prompt tokens and only use the FC matrix as input. Using attention mechanism,
pairwise connection strength between ROIs is learnt, producing community-
specific node embeddings $H_{i}$ and prompt tokens $p^{{}^{\prime}}_{i}$,

$p^{{}^{\prime}}_{i},H_{i}=\mbox{LocalTransformer}([p_{i},X_{i}])\mbox{ where,
}i\in[1,2...K].$ (3)

Global Transformer: On obtaining community-specific node embeddings and prompt
tokens from the local transformer, it is essential to combine this information
and design a module to learn the brain network at a global level. Therefore,
we introduce a global transformer encoder to learn inter-community
dependencies. Input to the global transformer is the concatenated, learnt node
feature matrices from the local transformer and a prompt token (Fig. 1(2)).
Prompt tokens learnt by the local transformer contain valuable information to
distinguish different communities and thus are concatenated and passed through
an MLP to obtain a prompt token input for the global transformer as follows:

$p_{global}=\mbox{MLP }(\mbox{Concat
}(p^{{}^{\prime}}_{1},p^{{}^{\prime}}_{2}\dots p^{{}^{\prime}}_{K})),$ (4)
$H_{global}=\mbox{Concat}(H_{1},H_{2},\dots,H_{K}),$ (5)
$p^{{}^{\prime}},Z_{L}=\mbox{GlobalTransformer}([p_{global},H_{global}]).$ (6)

The resulting attention-enhanced, node embedding matrix $Z_{L}$ is then passed
to a pooling layer for further coarsening of the graph. Extensive ablation
studies are presented in section 3.3 to justify the choice of inputs and
output.

###  2.3 Graph Readout Layer

The final step involves aggregating global-level node embeddings to obtain a
high-level representation of the brain graph. We use OCRead layer [15] for
aggregating the learnt node embeddings. OCRead initializes orthonormal cluster
centers $E\in\mathbb{R}^{K\times N}$ and softly assigns nodes to these
centers. Graph level embedding $Z_{G}$ is then obtained by
$Z_{G}=A^{\top}Z^{L}$, where $A\in\mathbb{R}^{K\times N}$is a learnable
assignment matrix computed by OCRead. $Z_{G}$ is then flattened and passed to
an MLP for a graph-level prediction. The whole process is supervised with
Cross-Entropy (CE) loss.

##  3 Experiments

###  3.1 Datasets and Experimental Settings

ABIDE is a collection of resting-state functional MRI (rs-fMRI) data from 17
international sites [1] with Configurable Pipeline for the Analysis of
Connectomes (CPAC), band-pass filtering (0.01 - 0.1 Hz), no global signal
regression, parcellated by Craddock 200 atlas[23]. ROIs of ABIDE dataset
belong to either of the eight functional communities namely, cerebellum and
subcortical structures (CS & SB), visual network (V), somatomotor network
(SMN), DAN, ventral attention network (VAN), limbic network (L),
frontoparietal network (FPN) and DMN, following the assignments in [31]. ABIDE
has fMRI data of 1009 subjects, 51.14% of whom were diagnosed with ASD.
Pearson correlation matrix is computed using mean time series of ROIs.
Stratified sampling strategy[15] is used for train-validation-test data split.

Experimental Settings: All models are implemented in PyTorch and trained on
NVIDIA V100 with 8GB memory. For both local and global transformers, the
number of attention heads is equal to the number of communities. We used Adam
optimizer with an initial learning rate of $10^{-4}$ and a weight decay of
$10^{-4}$. The train/validation/test data spilt ratio is 70:10:20, and the
batch size is 64. Prediction of ASD vs. HC is the binary classification task.
We use AUROC, accuracy, sensitivity, and specificity on the test set to
evaluate performance. Models are trained for 50 epochs and we use an early
stopping strategy. The epoch with the highest AUROC performance on the
validation set is used for performance comparison on the test set.

###  3.2 Quantitative and Qualitative Results

Comparison with Baselines (Quantitative results) Following the comparison
method mentioned in BNT[15], we compare the performance of our model with
three types of baselines (i) Comparison with transformer-based models - BNT
(ii) Comparison with neural network model on fixed brain network -
BrainNetCNN[17] (iii) Comparison with neural network models on learnable brain
network - FBNETGEN[16]. Note that the selected baselines are reported to be
the best among other alternatives in the same category on ABIDE dataset by
BNT[15]. As seen in Table.1, our model outperforms all the other architectures
on three evaluation metrics. In contrast to other models, Com-BrainTF is
hierarchical, similar to the brain’s functional organization and therefore is
capable of learning relationships within and between communities.

Table 1: Performance comparison with baselines (Mean $\pm$ standard deviation) Model |  AUROC |  Accuracy |  Sensitivity |  Specificity
---|---|---|---|---
BrainNetTF[15] |  78.3 $\pm$ 4.4 |  68.1 $\pm$ 3.1 |  78.1 $\pm$ 10.0 |  58.9 $\pm$ 12.0
BrainNetCNN[17] |  74.1 $\pm$ 5.1 |  67.5 $\pm$ 3.1 |  65.3 $\pm$ 4.3 |  69.6 $\pm$ 4.1
FBNETGNN[16] |  72.5 $\pm$ 8.3 |  64.9 $\pm$ 8.9 |  60.9 $\pm$ 11.3 |  67.5 $\pm$ 13.1
Com-BrainTF |  79.6 $\pm$ 3.8 |  72.5 $\pm$ 4.4 |  80.1 $\pm$ 5.8 |  65.7 $\pm$ 6.4
Figure 2: (1) Attention matrix of the global transformer of Com-BrainTF
highlighting prominent communities that influence ASD prediction (2) Attention
matrix of the first transformer encoder module of BNT (3) Chord plot of the
top 1% of attention values (4) Neurosynth results generated from averaged
prompt vectors (5) Correlation between functional communities and functional
keywords decoded by Neurosynth [27].

Interpretibility of Com-BrainTF (Qualitative results) We perform extensive
experiments to analyze the interpretability of Com-BrainTF. The following
results are discussed in detail:
1\. Interpretation of the learned attention matrix: Fig. 2(1) and Fig. 2(2)
show the averaged attention matrices of Com-BrainTF and BNT[15] respectively.
These attention matrices are obtained by averaging attention scores over
correctly classified test data. In comparison to BNT, our learned attention
scores clearly highlight the communities important for ASD prediction.
Specifically, attention scores in the SMN region (green arrow) are high
indicating that within SMN, connectivity plays an essential role in ASD
prediction. This aligns with prior studies on autism which have shown reduced
FC within SMN [13], reflecting altered sensory and motor processing [20, 10,
21]. Additionally, attention scores in DMN, DAN, and FPN regions were found to
be high (magenta arrows) suggesting that the functional connectivity between
DMN and DAN and between DMN and FPN are also crucial for ASD prediction which
is in line with previous studies that report abnormalities in DMN[22] and its
functional connectivity with other functional networks including DAN and FPN.
These connections can be better visualized from the chord plot in Fig. 2(3).
To summarize, Com-BrainTF is able to correctly identify functional communities
associated with ASD and the results are consistent with neuroscience findings.
2\. Ability of the prompt’s attention vector to differentiate between ASD and
HC subjects: We investigate the first row of the learned attention matrix,
namely the attention of ROIs corresponding to the prompt. The ROI-wise
normalized attention scores shown in Fig. 2(4) are generated using averaged
prompt vectors over correctly classified test data, where red to blue indicate
attention scores from high to low (dark red:1 to dark blue: 0). As evident
from the figure, clear differences can be seen in the important brain regions
among ASD and HC subjects. Additionally, most differences in the prompt vector
embeddings are seen in DMN and SMN (supporting figures in the appendix),
consistent with previous neuroscience literature.
3\. Meta-analysis of the important functional networks that influence ASD
prediction: DMN and SMN networks have been found to be crucial for ASD
prediction based on Fig. 2(4). Using the difference of the prompt vector
values in those regions between ASD and HC subjects, Fig. 2(5) was generated
based on Meta-analysis using Neurosynth222The meta-analytic framework
(www.neurosynth.org) provides the posterior probability
$P$(Feature$\mid$Coordinate) for psychological features (i.e., word or phrase)
at a given spatial location based on neuroscience literature. database[27].
Regions in DMN were found to be associated with trait keywords such as
empathy, painful, stimulation, orienting and spatial, whereas regions in the
SMN showed higher correlations with eye movement, ventral premotor,
somatosensory and speech production.

###  3.3 Ablation studies

We conduct ablation studies to justify the designs of input and output pairing
of the global transformer that facilitates effective learning of node
embeddings, consequently resulting in superior performance.

Input: node features vs. class tokens of local transformers. We evaluated two
input possibilities for the global transformer: (i) use only prompt tokens
from the local transformer (ii) incorporate both prompt tokens and updated
node features. Table.2 reveals that prompt tokens alone are unable to capture
the complete intra-functional connectivity and negatively affect performance.

Output: Cross Entropy loss on the learned node features vs. prompt token. In
comparison to vision models like ViT[6], which only use the class token
(similar to the prompt token in our structure) for classification, we use
output node embeddings from the global transformer for further processing
steps. We justify this design by conducting a performance comparison
experiment: (i) use learnt node features as the global transformer output (ii)
use only the prompt token as output. The results (Table.2) demonstrated a
significant decrease in performance when prompt token alone was used. This is
expected since brain networks are non-euclidean structures and the learnt node
features capture more information about the underlying graph, making them
essential for accurately capturing the inter-community dependencies.

Table 2: Ablation study: Examining different input and output possibilities
Different inputs to the global transformer
---
Inputs |  AUROC |  Accuracy |  Sensitivity |  Specificity
Prompt tokens only |  73.2 $\pm$ 4.5 |  65.7 $\pm$ 5.8 |  88.8 $\pm$ 4.1 |  43.7 $\pm$ 9.1
Node features and prompt token |  79.6 $\pm$ 3.8 |  72.5 $\pm$ 4.4 |  80.1 $\pm$ 5.8 |  65.7 $\pm$ 6.4
CE loss on node features vs prompt token
Outputs |  AUROC |  Accuracy |  Sensitivity |  Specificity
Prompt token |  71.9 $\pm$ 4.6 |  66.1 $\pm$ 3.6 |  80.4 $\pm$ 8.2 |  51.5 $\pm$ 7.6
Node features |  79.6 $\pm$ 3.8 |  72.5 $\pm$ 4.4 |  80.1 $\pm$ 5.8 |  65.7 $\pm$ 6.4

##  4 Conclusion

In this work, we introduce Com-BrainTF, a hierarchical, community-aware local-
global transformer architecture for brain network analysis. Our model learns
intra- and inter-community aware node embeddings for ASD prediction tasks.
With built-in interpretability, Com-BrainTF not only outperforms SOTA on the
ABIDE dataset but also detects salient functional networks associated with
ASD. We believe that this is the first work leveraging functional community
information for brain network analysis using transformer architecture. Our
framework is generalizable for the analysis of other neuroimaging modalities,
ultimately benefiting neuroimaging research. Our future work includes
investigating alternate variants for choosing different atlases and community
network parcellations.

## References

  * [1] C, C., et al.: The neuro bureau preprocessing initiative: open sharing of preprocessed neuroimaging data and derivatives (2013)
  * [2] Canario, E., Chen, D., Biswal, B.: A review of resting-state fMRI and its use to examine psychiatric disorders. Psychoradiology (2021)
  * [3] Chen, S., et al.: Group independent component analysis reveals consistent resting-state networks across multiple sessions. Brain Research (2008)
  * [4] Cui, H., et al.: Interpretable graph neural networks for connectome-based brain disorder analysis. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VIII (2022)
  * [5] Damoiseaux, J.S., et al.: Consistent resting-state networks across healthy subjects. Proceedings of the National Academy of Sciences (2006)
  * [6] Dosovitskiy, A., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. CoRR (2020)
  * [7] Farrant, K., Uddin, L.Q.: Atypical developmental of dorsal and ventral attention networks in autism. Developmental Science (2015)
  * [8] Geerligs, L., et al.: State and trait components of functional connectivity: Individual differences vary with mental state. The Journal of Neuroscience (2015)
  * [9] Goldani, A.A.S., et al.: Biomarkers in autism. Frontiers in Psychiatry (2014)
  * [10] Gowen, E., Hamilton, A.: Motor abilities in autism: A review using a computational context. Journal of Autism and Developmental Disorders (2012)
  * [11] van den Heuvel, M.P., et al.: Functionally linked resting-state networks reflect the underlying structural connectivity architecture of the human brain. Human Brain Mapping (2009)
  * [12] den Heuvel, M.P.V., Pol, H.E.H.: Exploring the brain network: A review on resting-state fMRI functional connectivity. European Neuropsychopharmacology (2010)
  * [13] Ilioska, I., et al.: Connectome-wide mega-analysis reveals robust patterns of atypical functional connectivity in autism. Biological Psychiatry (2022)
  * [14] Kaiser, M.D., et al.: Severe impairments of social interaction and associated abnormalities in children: epidemiology and classification. Proceedings of the National Academy of Sciences, 107(49), 21223–21228 (2010)
  * [15] Kan, X., et al.: Brain network transformer. In: Advances in Neural Information Processing Systems (2022)
  * [16] Kan, X., et al.: FBNETGEN: Task-aware GNN-based fMRI analysis via functional brain network generation. In: Medical Imaging with Deep Learning (2022)
  * [17] Kawahara, J., et al.: Brainnetcnn: Convolutional neural networks for brain networks; towards predicting neurodevelopment. NeuroImage (2017)
  * [18] L, W., J, G.: Severe impairments of social interaction and associated abnormalities in children: epidemiology and classification. Journal of autism and developmental disorders (1979)
  * [19] Li, X., et al.: Braingnn: Interpretable brain graph neural network for fmri analysis. Medical Image Analysis (2021)
  * [20] Marco, E.J., et al.: Sensory processing in autism: A review of neurophysiologic findings. Pediatric Research (2011)
  * [21] Mostofsky, S.H., Ewen, J.B.: Altered connectivity and action model formation in autism is autism. The Neuroscientist (2011)
  * [22] Padmanabhan, A., et al.: The default mode network in autism. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging (2017)
  * [23] RC, C., et al.: A whole brain fmri atlas generated via spatially constrained spectral clustering. Hum Brain Mapp. 2012;33(8):1914-1928 (2012)
  * [24] Rosazza, C., Minati, L.: Resting-state brain networks: literature review and clinical applications. Neurological Sciences (2011)
  * [25] Smith, S.M., et al.: Correspondence of the brain's functional architecture during activation and rest. Proceedings of the National Academy of Sciences (2009)
  * [26] Smith, S.M., et al.: Functional connectomics from resting-state fMRI. Trends in Cognitive Sciences (2013)
  * [27] T, Y., et al.: Large-scale automated synthesis of human functional neuroimaging data. Nature methods vol. 8,8 665-70 (2011)
  * [28] True, P., et al.: Multiple-network classification of childhood autism using functional connectivity dynamics. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014 (2014)
  * [29] Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (2017)
  * [30] Veličković, P., et al.: Graph attention networks. In: International Conference on Learning Representations (2018)
  * [31] Yeo, B.: Kriene nf, sepulcre j, sabuncu m, lashkari d, hollinshead m, et al. the organization of the human cerebral cortex estimated by intrinsic functional connectivity. J Neurophysiol 106(3), 1125–1165 (2011)

##  Appendix 0.A Supplementary Materials

###  0.A.1 Variations on the Number of Prompts

Table 3: Experiments with different numbers of input prompt tokens to the
local and global transformers (Please see the detailed definition in Sec 2.2)
to justify our final choice. Using more prompt tokens means learning more
parameters. We noticed that one prompt token for both local and global
transformers is sufficient for achieving satisfactory performance.

Num of input prompt tokens | AUROC | Accuracy | Sensitivity | Specificity
---|---|---|---|---
local: 2, global: 2 | 78.4 $\pm$ 4.2 | 69.5 $\pm$ 4.3 | 82.3 $\pm$ 9.9 | 55.2 $\pm$ 14.0
local:1, global: 1 | 79.6 $\pm$ 3.8 | 72.5 $\pm$ 4.4 | 80.1 $\pm$ 5.8 | 65.7 $\pm$ 6.4

###  0.A.2 Attention Scores of ASD vs. HC in Comparison between Com-BrainTF
(ours) and BNT (baseline)

Figure 3: Comparison of Com-BrainTF’s prompt vector with BNT’s attention
vector to demonstrate improved interpretability of Com-BrainTF (1)
Visualization of prompt’s attention vector that shows most differences in DMN
and SMN communities (2) Visualization of the sum of rows of the attention
matrix in the baseline, BNT. In comparison to Com-BrainTF where clear
differences between ASD and HC are seen in DMN and SMN, we cannot tell the
difference between ASD and HC groups using BNT.

###  0.A.3 Decoded Functional Group Differences of ASD vs. HC

Figure 4: We follow the analysis similar to Sec 3.2 but focus on the positive
components of the normalized attention scores differences between ASD vs HC.
Namely, we analyze the average $score_{\rm ASD}=(attn_{\rm ASD}-attn_{\rm
HC})_{+}$ for ASD and $score_{\rm HC}=(attn_{\rm HC}-attn_{\rm ASD})_{+}$ for
HC. (1) Visualization of $score_{\rm ASD}$ and $score_{\rm HC}$ showing
differences in HC and ASD subjects in the DMN functional community. (2)
Correlation between highlighted ROIs of ASD and HC in DMN community, and
functional keywords decoded by Neurosynth. The interpreted results indicate
functional differences between ASD and HC, potentially providing new insights
for discovering ASD biomarkers. Figure 5: We follow the analysis similar to
Sec 3.2 but focus on the positive components of the normalized attention
scores differences between ASD vs HC. Namely, we analyze the average
$score_{\rm ASD}=(attn_{\rm ASD}-attn_{\rm HC})_{+}$ for ASD subjects and
$score_{\rm HC}=(attn_{\rm HC}-attn_{\rm ASD})_{+}$ for HC subjects. (1)
Visualization of $score_{\rm ASD}$ and $score_{\rm HC}$ showing differences in
HC and ASD subjects in the SMN functional community. (2) Correlation between
highlighted ROIs of ASD and HC in SMN community, and functional keywords
decoded by Neurosynth. The interpreted results indicate functional differences
between ASD and HC, potentially providing new insights for discovering ASD
biomarkers.