0.8pt

# Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series
Classification

Yihe Wang∗
University of North Carolina - Charlotte
<EMAIL_ADDRESS>
&Nan Huang∗
University of North Carolina - Charlotte
<EMAIL_ADDRESS>
&Taida Li∗
University of North Carolina - Charlotte
<EMAIL_ADDRESS>
& Yujun Yan
Dartmouth College
<EMAIL_ADDRESS>
& Xiang Zhang
University of North Carolina - Charlotte
<EMAIL_ADDRESS>

###### Abstract

Medical time series data, such as Electroencephalography (EEG) and
Electrocardiography (ECG), play a crucial role in healthcare, such as
diagnosing brain and heart diseases. Existing methods for medical time series
classification primarily rely on handcrafted biomarkers extraction and CNN-
based models, with limited exploration of transformers tailored for medical
time series. In this paper, we introduce Medformer, a multi-granularity
patching transformer tailored specifically for medical time series
classification. Our method incorporates three novel mechanisms to leverage the
unique characteristics of medical time series: cross-channel patching to
leverage inter-channel correlations, multi-granularity embedding for capturing
features at different scales, and two-stage (intra- and inter-granularity)
multi-granularity self-attention for learning features and correlations within
and among granularities. We conduct extensive experiments on five public
datasets under both subject-dependent and challenging subject-independent
setups. Results demonstrate Medformer’s superiority over 10 baselines,
achieving top averaged ranking across five datasets on all six evaluation
metrics. These findings underscore the significant impact of our method on
healthcare applications, such as diagnosing Myocardial Infarction,
Alzheimer’s, and Parkinson’s disease. We release the source code at
https://github.com/DL4mHealth/Medformer.

**footnotetext: These authors contributed equally to this work.

##  1 Introduction

Medical time series refers to sequences of health-related data points recorded
at successive times, tracking various physiological signals over time [1, 2].
Classifying medical time series enables continuous monitoring and real-time
analysis of a patient’s physiological state, allowing for early detection of
abnormalities, accurate diagnosis, timely intervention, and personalized
treatment, enhancing patient outcomes and healthcare efficiency [3, 4]. For
instance, Electroencephalography (EEG) provides insights into a subject’s
neurological status [5, 6], while Electrocardiography (ECG) aids in diagnosing
heart conditions [7, 8, 9]. Existing works on medical time series
classification mostly utilize handcrafted biomarker extraction [10, 11, 12]
and CNN-based models [13, 14, 15, 16], such as EEGNet [17] and TCN [18]. There
is a notable absence of effective transformer-based methods for medical time
series classification.

Transformers have proven to be effective in time series representation
learning across various tasks, including forecasting [19, 20, 21],
classification [22, 23], and anomaly detection [24, 25], with a predominant
focus on forecasting. While these methods can be applied to medical time
series classification, their design motivations and mechanisms may not be
ideally suited for this purpose. For instance, as shown in Figure 1,
Autoformer [26] and Informer [27] adopt the token embedding method from the
vanilla transformer [28], embedding a single cross-channel timestamp as an
input token. This approach struggles to capture coarse-grained features along
the temporal dimension. Conversely, iTransformer [29] treats the entire series
of one channel as an attention token, often overlooking fine-grained temporal
features and focusing more on multi-channel correlations. Additionally,
PatchTST [30] embeds a sequence of timestamps from one channel as a patch for
self-attention, which limits the model’s ability to learn multi-channel
correlations.

These existing methods fail to fully exploit the distinctive characteristics
of medical time series data, including local temporal dynamics, inter-channel
correlations, and multi-scale analysis. Firstly, capturing temporal dynamics
requires multi-timestamp input to recognize local temporal patterns with
multi-timestamp input, which have been extensively discussed in literature
like PatchTST [30]. Secondly, it is essential to harness cross-channel
information. For instance, brain activities are generally captured by multi-
channel EEG (i.e., 32 channels/electrodes), where each channel monitors a
specific brain region. Given that the brain functions as a cohesive unit, the
inter-channel correlations (i.e., brain connectome) are crucial for EEG
analysis [31, 32, 33]. Thirdly, representation learning across multiple
temporal scales and periods is vital to uncover a broad spectrum of health
patterns. Existing research has shown that some signs of disease only exist in
specific frequency bands [10, 12].

Figure 1: Token embedding methods. Vanilla transformer, Autoformer, and
Informer [28, 26, 27] employ a single cross-channel timestamp as a token;
iTransformer [29] utilizes an entire channel as a token; and PatchTST and
Crossformer [30, 34] adopt a patch of timestamps from one channel as a token.
For medical time series classification, we propose Medformer considering
inter-channel dependencies (multi-channel), temporal properties (multi-
timestamp), and multifaceted scale of temporal patterns (multi-granularity).

To bridge this gap, we propose Medformer, a multi-granularity patching
transformer specifically designed for medical time series classification. Our
method introduces three mechanisms to enhance learning ability. Firstly, we
propose a novel token embedding approach by implementing cross-channel
patching, effectively capturing both multi-timestamp and multi-channel
features. Secondly, instead of fixed-length patching, we adopt multi-
granularity patching using a list of patch lengths to capture features at
different granularities. The multi-granularity approach also simulates
different frequency bands and captures band features without relying on
handcrafted up/downsampling and band filters. Lastly, we introduce a two-stage
(intra- and inter-granularity) multi-granularity self-attention to learn
features within specific granularity and the correlations among different
granularities. This allows the model to learn features at various
granularities and compel them to integrate and complement information.

We conduct extensive experiments using ten baselines and five public datasets,
including three EEG datasets and two ECG datasets on detecting Alzheimer’s
Disease and cardiovascular diseases, under both subject-dependent and
challenging subject-independent setups (Figure 2). The experimental results
show that Medformer achieves the best average ranking on all six evaluation
metrics across five datasets (Figure 4), demonstrating its superior
effectiveness and stability and highlighting its potential for real-world
applications.

##  2 Related Work

Medical Time Series. Medical time series are a special type of time series
data collected from the human body, used for disease diagnosis [3, 7], health
monitoring [6, 1], and brain-computer interfaces [2]. Various types of medical
time series include EEG [35, 36, 37], ECG [7, 8, 9], EMG [38, 39], and EOG
[40, 41]. Each type offers unique capabilities for different healthcare
applications. For instance, EEG and ECG can diagnose the health of the brain
and heart [35, 7]. Recent research on brain-computer interfaces explores using
EEG to control objects, benefiting disabled individuals [2, 42]. Research on
medical time series mainly focuses on decoding signals, which involves
classifying the hidden information in a sequence of medical time series.
Existing works rely on identifying biomarkers and utilizing CNN-based deep-
learning models. For example, band features such as relative band power and
band correlations [11, 43] have proven useful for EEG-based Alzheimer’s
disease diagnosis. Deep learning models like EEGNet [17] and TCN [18, 13] have
also demonstrated effectiveness in many medical time series classification
tasks.

Table 1: Existing methods do not fully utilize all potential aspects of
features in medical time series.

Models |  Multi- Timestamp |  Multi- Channel |  Multi- Granularity |  Granularity- Interaction
---|---|---|---|---
Autoformer [26] |  | ✓ |  |
Crossformer [34] | ✓ | ✓ |  |
FEDformer [44] |  | ✓ |  |
Informer [27] |  | ✓ |  |
iTransformer [29] | ✓ | ✓ |  |
MTST [45] | ✓ |  | ✓ |
Nonformer [46] |  | ✓ |  |
PatchTST [30] | ✓ |  |  |
Pathformer [47] | ✓ |  | ✓ |
Reformer [48] |  | ✓ |  |
Transformer [28] |  | ✓ |  |
Medformer(Ours) | ✓ | ✓ | ✓ | ✓

Transformers for Time Series. Existing transformer-based methods for time
series can be categorized into two main directions: modifying token embedding
methods and self-attention mechanisms, or both. For example, PatchTST [30]
uses a sequence of single-channel timestamps as a patch for token embedding.
Methods like Autoformer [26], Informer [27], Nonformer [46], and FEDformer
[44] develop new self-attention mechanisms or replace the self-attention
module to improve learning ability and reduce complexity. Crossformer [34] and
iTransformer [29] modify both token embedding methods and self-attention
mechanisms. Patching. Patch embedding has been widely used in time series
transformers since the proposal of PatchTST [30]. Existing methods of
patching, such as Crossformer [34], CARD [21], and MTST [45], inherit from
PatchTST [30] and utilize a sequence of single-channel timestamps for
patching. This channel-independent patching might benefit learning ability in
time series forecasting but may not be as effective in medical time series
classification. Multi-Granularity. Existing methods such as Pyraformer [19],
MTST [45], Pathformer [47], and Scaleformer [49], utilize multi-granularity
embedding to capture features at different scales, allowing models to learn
both fine-grained and coarse-grained patterns. We discuss the differences
between our method and existing multi-granularity approaches in Appendix F.1.

Medformer includes both novel token embedding and self-attention mechanisms.
Figure 1 and Table 1 present a comparison of token embedding methods and
feature utilization between our method and existing methods. The components of
our method can be easily incorporated into existing methods to improve
classification learning ability. For example, cross-channel multi-granularity
patching can be integrated with methods that modify self-attention mechanisms,
such as Autoformer [26] and Informer [27], for token embedding. Similarly, the
two-stage multi-granularity self-attention can be combined with existing
multi-granularity methods, like MTST [45], to enhance the learning of inter-
granularity features.

##  3 Preliminaries and Problem Formulation

Medical time series typically exhibit multiple data levels, including subject,
session, trial, and sample levels [13]. In medical time series collected for
disease diagnosis tasks, each subject is usually assigned a data label, such
as indicating the presence or absence of Alzheimer’s disease. Multiple labels
assigned to one subject are also possible if the subject has different
diseases. Long sequences of time series(Trial/Session) data from each subject
are often segmented into multiple shorter samples for deep learning tasks.
Thus, each sample of medical time series generally includes a class label
indicating the disease type and a subject ID specifying its origin subject.
Given the ultimate goal of diagnosing whether a subject has a particular
disease, experimental setups must be meticulously designed to align with real-
world medical applications. Diverse experimental setups can yield
significantly different results, potentially leading to erroneous conclusions.
Here, we introduce two widely employed setups in medical time series
classification and highlight their distinctions. Figure 2 provides a simple
illustration of these two setups.

Subject-Dependent. In this setup, the division into training, validation, and
test sets is based on time series samples. All samples from various subjects
are randomly shuffled and then allocated into the respective sets.
Consequently, samples with identical subject IDs may be present in the
training, validation, and test sets. This scenario potentially introduces
“information leakage," wherein the model could inadvertently learn the
distribution specific to certain subjects during the training phase. This
setup is typically employed for assessing whether a dataset exhibits cross-
subject features and has limited applications under real-world medical time-
series-based disease diagnosis scenarios. The reason is simple: we cannot know
the label of unseen subjects and their corresponding samples during training.
Generally, the results of the subject-dependent setup tend to be notably
higher than those from the subject-independent setup, often showing the upper
limit of a dataset’s learning capability.

Figure 2: Subject-dependent/independent setups (adopted from [13]). In the
subject-dependent setup, samples from the same subject can appear in both the
training and test sets, causing information leakage. In a subject-independent
setup, samples from the same subject are exclusively in either the training or
test set, which is more challenging and practically meaningful but less
studied.

Subject-Independent. In this setup, the division into training, validation,
and test sets is based on subjects. Each subject and their corresponding
samples are exclusively distributed into one of the training, validation, or
test sets. Consequently, samples with identical subject IDs can only be
present in one of these sets. This setup holds significant importance in
disease diagnosis tasks as it closely simulates real-world scenarios. It
enables us to train a model on subjects with known labels and subsequently
evaluate its performance on unseen subjects; in other words, evaluate if a
subject has a specific disease. However, this setup poses significant
challenges in medical time series classification tasks. Due to the variability
in data distribution and the potential presence of unknown noise within each
subject’s data, capturing general features across subjects becomes challenging
[50, 13, 51, 52]. Even if subjects share the same label, the personal noise
inherent in each subject’s data may obscure these common features. Developing
a method that effectively captures common features among subjects while
disregarding individual noise remains an unsolved problem.

In this work, we evaluate our model mainly in subject-independent setup to
better align with real-world applications, and draw the attention of time
series researchers to focus on the significant challenges posed by subject-
independence. Although this work is not specifically designed for subject-
independent problems, our model comprehensively integrates multi-timestamp,
multi-channel, and multi-granularity information from each subject. We aim to
minimize information loss and enable the model to freely exploit subject-
invariant representations. Consequently, our model is well-equipped to tackle
the subject-independent challenge to a certain extent, and our results
(Section 5) confirm such capability of Medformer.

Next, we present the problem formulation in the context of multivariate
medical time series classification for disease diagnosis.

######  Problem (Medical Time Series Classification).

Consider an input medical time series sample
$\mathrm{\bm{x}}_{\text{in}}\in\mathbb{R}^{T\times C}$, where $T$ represents
the number of timestamps and $C$ represents the number of channels. We aim to
learn a representation $\bm{h}$, which can be used to predict the
corresponding label $\bm{y}\in\mathbb{R}^{K}$. Here, $K$ denotes the number of
classes that have medical meanings, such as different disease types.

##  4 Method

Figure 3: Overview of Medformer. a) Workflow. b) For the input sample
$\bm{x}_{\textrm{in}}$, we apply $n$ different patch lengths in parallel to
create patched features $\bm{x}_{p}^{(i)}$, where $i$ ranges from 1 to $n$.
Each patch length represents a different granularity. These patched features
are then linearly transformed into $\bm{x}_{e}^{(i)}$, which are subsequently
augmented into $\widetilde{\bm{x}}_{e}^{(i)}$. c) We obtain the final patch
embedding $\bm{x}_{(i)}$ by fusing augmented $\widetilde{\bm{x}}_{e}^{(i)}$
with the positional embedding $\bm{W}_{\text{pos}}$ and the granularity
embedding $\bm{W}_{\text{gr}}^{(i)}$. Additionally, we design a granularity-
specific router $\bm{u}^{(i)}$ to capture integrated information for its
respective granularity. We compute both intra-granularity attention, which
concentrates within individual granularities, and inter-granularity attention,
which leverages the routers to focus across different granularities, for
extensive representation learning.

In this section, we first describe the cross-channel multi-granularity
patching mechanism for learning spatial-temporal features at different
granularities. Next, we analyze the two-stage multi-granularity self-attention
mechanism, which leverages features within the same granularity and
correlations among different granularities. The architecture of the proposed
Medformer is illustrated in Figure 3.

Cross-Channel Multi-Granularity Patch Embedding. From a medical perspective,
the brain or heart functions as a cohesive unit, suggesting a naive assumption
that there are inherent correlations among different channels in medical time
series [31, 32, 33].. Motivated by the above assumption, we reasonably propose
multi-channel patching for token embedding, which is different from existing
patch embedding methods that embed patches in a channel-independent manner and
fail to capture inter-channel correlations [30, 34, 21]. Figure 1 provides an
overview comparison of existing token embedding methods and ours.
Additionally, existing research on EEG biomarker extraction has shown that
certain features are linked to different frequency bands, such as
$\alpha,\beta,$ and $\gamma$ bands [10, 12]. This motivates us to embed patch
tokens in a multi-granularity way. Instead of using traditional methods like
up/downsampling or handcrafted band filtering, multi-granularity patching
automatically corresponds to different sampling frequencies, which can
simulate different frequency bands and capture band-related features.

Given the above rationales, we propose a novel token embedding approach:
cross-channel multi-granularity patching. Given an input multivariate time
series sample $\bm{x}_{\text{in}}\in\mathbb{R}^{T\times C}$, and a list of
different patch lengths $\left\\{{L}_{1},{L}_{2},\ldots,{L}_{n}\right\\}$. For
the $i$-th patch length $L_{i}$ denoting granularity $i$, we segment the input
sample into $N_{i}$ cross-channel non-overlapping patches
$\bm{x}_{\text{p}}^{(i)}\in\mathbb{R}^{N{i}\times\left(L_{i}\cdot C\right)}$.
Zero padding is applied to ensure that the number of timestamps $T$ is
divisible by $L_{i}$, making $N_{i}=\left\lceil{T/L_{i}}\right\rceil$.

The patches are mapped into latent embeddings space using a linear projection:
$\bm{x}_{\text{e}}^{(i)}=\bm{x}_{\text{p}}^{(i)}\bm{W}^{(i)}$, where
$\bm{x}_{\text{e}}^{(i)}\in\mathbb{R}^{N_{i}\times D}$ and
$\bm{W}^{(i)}\in\mathbb{R}^{\left(L_{i}\cdot C\right)\times D}$. Inspired by
the augmented views contrasting in the contrastive learning framework [53, 13,
54], we further apply data augmentations such as masking and jittering on
$\bm{x}_{\text{e}}^{(i)}$ to obtain augmented embeddings
$\widetilde{\bm{x}}_{\text{e}}^{(i)}\in\mathbb{R}^{N_{i}\times D}$. We assume
the augmentation can improve the learning ability in the following inter-
granularity self-attention stage by forcing different granularities to learn
and complement information from each other.

A fixed positional embedding $\bm{W}_{\text{pos}}\in\mathbb{R}^{G\times D}$ is
generated for positional encoding [28], where $G$ is a very large number. We
add $\bm{W}_{\text{pos}}[1:N_{i}]\in\mathbb{R}^{N_{i}\times D}$, the first
$N_{i}$ rows of the positional embedding $\bm{W}_{\text{pos}}$, and a
learnable granularity embedding
$\bm{W}_{\text{gr}}^{(i)}\in\mathbb{R}^{1\times D}$ for the $i$-th patch
length $L_{i}$, to obtain the final patch embedding:

$\bm{x}^{(i)}=\widetilde{\bm{x}}_{\text{e}}^{(i)}+\bm{W}_{\text{pos}}[1:N_{i}]+\bm{W}_{\text{gr}}^{(i)},$
(1)

where $\bm{x}^{(i)}\in{\mathbb{R}}^{N_{i}\times D}$. Note that the granularity
embedding $\bm{W}_{\text{gr}}^{(i)}$ is broadcasted to all $N_{i}$ embeddings
during addition.

To reduce time and space complexity, we initialize a router for multi-
granularity self-attention (as described later):

$\bm{u}^{(i)}=\bm{W}_{\text{pos}}[N_{i}+1]+\bm{W}_{\text{gr}}^{(i)},$ (2)

where
$\bm{u}^{(i)},\bm{W}_{\text{pos}}[N_{i}+1],\bm{W}_{\text{gr}}^{(i)}\in\mathbb{R}^{1\times
D}$. Here, $\bm{W}_{\text{pos}}[N_{i}+1]$ is not used for positional embedding
but to inform the router about the number of patches with the current $L_{i}$
granularity, and $\bm{W}_{\text{gr}}^{(i)}$ contains the granularity
information. Both components help distinguish the routers from one another.

Finally, we obtain a list of patch embeddings
$\left\\{\bm{x}^{(1)},\bm{x}^{(2)},\ldots,\bm{x}^{(n)}\right\\}$ and router
embeddings $\left\\{\bm{u}^{(1)},\bm{u}^{(2)},\ldots,\bm{u}^{(n)}\right\\}$ of
different patch lengths $\left\\{L_{1},L_{2},\ldots,L_{n}\right\\}$. We feed
the embeddings to the two-stage multi-granularity self-attention.

Multi-Granularity Self-Attention. Our goal is to learn multi-granularity
features and granularity interactions during self-attention. A naive approach
to achieve this goal is to concatenate all the patch embeddings
$\left\\{\bm{x}^{(1)},\bm{x}^{(2)},\ldots,\bm{x}^{(n)}\right\\}$ into a large
patch embedding
$\bm{X}\in{\mathbb{R}}^{\left(N_{1}+N_{2}+\ldots+N_{n}\right)\times D}$ and
perform self-attention on this new embedding, where $n$ denotes the number of
different granularities. However, this results in a time complexity of
$O\left(\left(\sum_{i=1}^{n}N_{i}\right)^{2}\right)$, which is impractical for
a large $n$.

To reduce the time complexity, we propose a router mechanism and split the
self-attention module into two stages: a) intra-granularity and b) inter-
granularity self-attention. The intra-granularity stage performs self-
attention within the same granularity to capture the distinctive features of
each granularity. The inter-granularity stage performs self-attention across
different granularities to capture their correlations.

a) Intra-Granularity Self-Attentionwhitea) Intra-Granularity Self-Attention.
For the $i$-th patch length $L_{i}$ denoting granularity $i$, we vertically
concatenate the patch embedding $\bm{x}^{(i)}\in{\mathbb{R}}^{N_{i}\times D}$
and router embedding $\bm{u}^{(i)}\in{\mathbb{R}}^{1\times D}$ to form an
intermediate sequence of embeddings
$\bm{z}^{(i)}\in{\mathbb{R}}^{\left(N_{i}+1\right)\times D}$:

$\displaystyle\bm{z}^{(i)}=$
$\displaystyle\left[\bm{x}^{(i)}\|\bm{u}^{(i)}\right]$ (3)

where $\left[\cdot\|\cdot\right]$ denotes concatenation. We perform self-
attention on the new $\bm{z}^{(i)}$ for both the patch embedding
$\bm{x}^{(i)}$ and the router embedding $\bm{u}^{(i)}$:

$\displaystyle\bm{x}^{(i)}\leftarrow$
$\displaystyle\operatorname{Attn^{Intra}}\left(\bm{x}^{(i)},\bm{z}^{(i)},\bm{z}^{(i)}\right)$
(4) $\displaystyle\bm{u}^{(i)}\leftarrow$
$\displaystyle\operatorname{Attn^{Intra}}\left(\bm{u}^{(i)},\bm{z}^{(i)},\bm{z}^{(i)}\right)$

where $\operatorname{Attn}\left(\bm{Q},\bm{K},\bm{V}\right)$ denotes the
scaled dot-product attention mechanism in [28]. Note that the router embedding
$\bm{u}^{(i)}$ is updated in the same way as the patch embedding
$\bm{x}^{(i)}$ to maintain consistency, ensuring that the router can summarize
temporal features within the same granularity while the patch embeddings
receive global information from the router. The intra-granularity self-
attention mechanism enables the model to capture temporal features within the
same granularity, facilitating the extraction of local features and
correlations among timestamps of the same scale.

Moreover, unlike existing multi-granularity methods such as MTST [45] and
Pathformer [47] that use different attention blocks for different
granularities, we use a shared attention block for all granularities. This
greatly reduces the model’s space complexity and memory consumption, making
our method more efficient and scalable.

b) Inter-Granularity Self-Attentionwhiteb) Inter-Granularity Self-Attention.
We concatenate all router embeddings
$\left\\{\bm{u}^{(1)},\bm{u}^{(2)},\ldots,\bm{u}^{(n)}\right\\}$ to form a
sequence of routers $\bm{U}\in{\mathbb{R}}^{n\times D}$:

$\bm{U}=\left[\bm{u}^{(1)}\|\bm{u}^{(2)}\|\ldots\|\bm{u}^{(n)}\right]\\\ $ (5)

where $n$ is the number of different granularities. For granularity $i$ with
patch length $L_{i}$, we apply self-attention to the router embedding
$\bm{u}^{(i)}\in{\mathbb{R}}^{1\times D}$ with all the routers $\bm{U}$:

$\displaystyle\bm{u}^{(i)}\leftarrow$
$\displaystyle\operatorname{Attn^{Inter}}\left(\bm{u}^{(i)},\bm{U},\bm{U}\right)$
(6)

Each router contains global information specific to one granularity by doing
intra-granularity self-attention. By performing self-attention among routers,
information can be exchanged and learned across different granularities,
effectively capturing features across various scales. Additionally, the use of
the router mechanism successfully reduces the time complexity of the naive
approach from $O\left(\left(\sum_{i=1}^{n}N_{i}\right)^{2}\right)$ to
$O\left(\sum_{i=1}^{n}N_{i}^{2}+n^{2}\right)$. Given that $N_{i}\leq T$, the
worst-case time complexity for our self-attention mechanism is
$O\left(nT^{2}+n^{2}\right)$. However, a reasonable choice of patch lengths as
a power series, i.e., $L_{i}=2^{i}$, leads to a time complexity of $O(T^{2})$.
See appendix E for more details about complexity analysis.

Summary. Our method utilizes the standard transformer architecture shown in
Figure 3. For given sample $\bm{x}_{\textrm{in}}$, after $M$ layers of self-
attention learning, we obtain a list of updated patch embeddings
$\left\\{\bm{x}^{(1)},\bm{x}^{(2)},\ldots,\bm{x}^{(n)}\right\\}$, which we
concatenate them to form a final representation $\bm{h}$ that can be used to
predict label $y\in\mathbb{R}^{K}$ in a downstream classification task. Note
that although we discuss multi-granularity here, our method is flexible and
can be easily adapted to variants such as single-granularity or even
repetitive same granularities. See Appendix D.2 for more details.

##  5 Experiments

We compare our Medformer with 10 baselines across 5 datasets, including 3 EEG
datasets and 2 ECG datasets. Our method is evaluated under two setups (Section
3): subject-dependent and subject-independent. In the subject-dependent setup,
training, validation, and test sets are split based on samples, while in the
subject-independent setup, they are split based on subjects.

Datasets. (1) APAVA [55] has 23 subjects and 5,967 16-channel multivariate EEG
samples. Each sample consists of a one-second time sequence with 256
timestamps, sampled at a rate of 256Hz. A binary label indicating whether the
subject has Alzheimer’s disease is assigned to each sample. (2) TDBrain [56]
has 72 subjects and 6,240 33-channel multivariate EEG samples. Each sample is
a one-second time sequence with 256 timestamps, sampled at 256Hz. A binary
label indicating whether the subject has Parkinson’s disease is assigned to
each sample. (3) ADFD [57, 58] has 88 subjects and 69,762 19-channel
multivariate EEG samples. Each sample is a one-second time sequence with 256
timestamps, sampled at 256Hz. A three-class label indicating whether the
subject is Healthy, Dementia, or Alzheimer’s disease is assigned to each
sample. (4) PTB [59] has 198 subjects and 64,356 15-channel multivariate ECG
samples. Each sample is a heartbeat with 300 timestamps, sampled at a rate of
250Hz. A binary label indicating whether the subject has Myocardial Infarction
is assigned to each sample. (5) PTB-XL [60] has 17,596 subjects and 191,400
12-channel multivariate ECG samples. Each sample is a one-second time sequence
with 250 timestamps, sampled at a rate of 250Hz. A five-class label indicating
different heart situations is assigned to each sample. For more details
regarding data information, train-validation-test split under different
setups, and data preprocessing, please refer to Appendix B.

Baselines. We compare with 10 state-of-the-art time series transformer
methods: Autoformer [26], Crossformer [34], FEDformer [44], Informer [27],
iTransformer [29], MTST [45], Nonformer [46], PatchTST [30], Reformer [48],
and vanilla Transformer [28].

Implementation. We employ six evaluation metrics: accuracy, precision (macro-
averaged), recall (macro-averaged), F1 score (macro-averaged), AUROC (macro-
averaged), and AUPRC (macro-averaged). The training process is conducted with
five random seeds (41-45) on fixed training, validation, and test sets to
compute the mean and standard deviation of the models. All experiments are run
on an NVIDIA RTX 4090 GPU and a server with 4 RTX A5000 GPUs.

For data augmentations in our method, we choose to use masking, jittering, and
scaling—three widely used methods in time series augmentation [61, 54]. For
more details about these three methods, see Appendix A. For the parameter
tuning in our method and all baselines, we employ 6 layers for the encoder,
set the dimension $D$ to 128, and the hidden dimension of feed-forward
networks to 256. We utilize the Adam optimizer with a learning rate of 1e-4.
The batch size is set to {32,32,128,128,128} for datasets APAVA, TDBrain,
ADFD, PTB, and PTB-XL, respectively. The training epoch is set to 100, with
early stopping triggered after 10 epochs without improvement in the F1 score
on the validation set. We save the model with the best F1 score on the
validation set and evaluate it on the test set. See Appendix C for any
additional implementation details of our method and all baselines.

Table 2: Results of Subject-Dependent Setup. The training, validation, and
test sets are split based on samples according to a predetermined ratio.
Results of the ADFD dataset under this setup are presented here.

Datasets | Models | Accuracy | Precision | Recall | F1 score | AUROC | AUPRC
---|---|---|---|---|---|---|---
ADFD (3-Classes) | Autoformer | 87.83$\pm$1.62 | 87.63$\pm$1.66 | 87.22$\pm$1.97 | 87.38$\pm$1.79 | 96.59$\pm$0.88 | 93.82$\pm$1.64
Crossformer | 89.35$\pm$1.32 | 89.00$\pm$1.44 | 88.79$\pm$1.37 | 88.88$\pm$1.40 | 97.52$\pm$0.58 | 95.45$\pm$1.03
FEDformer | 77.63$\pm$2.37 | 76.76$\pm$2.17 | 76.68$\pm$2.48 | 76.60$\pm$2.46 | 91.67$\pm$1.34 | 84.94$\pm$2.11
Informer | 90.93$\pm$0.90 | 90.74$\pm$0.71 | 90.50$\pm$1.14 | 90.60$\pm$0.94 | 98.19$\pm$0.27 | 96.51$\pm$0.49
iTransformer | 64.90$\pm$0.25 | 62.53$\pm$0.27 | 62.21$\pm$0.26 | 62.25$\pm$0.33 | 81.52$\pm$0.29 | 68.87$\pm$0.49
MTST | 65.08$\pm$0.69 | 63.85$\pm$0.80 | 62.71$\pm$0.64 | 63.03$\pm$0.58 | 81.36$\pm$0.56 | 69.34$\pm$0.89
Nonformer | 96.12$\pm$0.47 | 95.94$\pm$0.56 | 95.99$\pm$0.38 | 95.96$\pm$0.47 | 99.59$\pm$0.09 | 99.08$\pm$0.16
PatchTST | 66.26$\pm$0.40 | 65.08$\pm$0.41 | 64.97$\pm$0.51 | 64.95$\pm$0.42 | 83.07$\pm$0.45 | 71.70$\pm$0.61
Reformer | 91.51$\pm$1.75 | 91.15$\pm$1.79 | 91.65$\pm$1.56 | 91.14$\pm$1.83 | 98.85$\pm$0.35 | 97.88$\pm$0.60
Transformer | 97.00$\pm$0.43 | 96.87$\pm$0.53 | 96.86$\pm$0.36 | 96.86$\pm$0.44 | 99.75$\pm$0.04 | 99.42$\pm$0.07
Medformer (Ours) | 97.62$\pm$0.34 | 97.53$\pm$0.33 | 97.48$\pm$0.40 | 97.50$\pm$0.36 | 99.83$\pm$0.05 | 99.62$\pm$0.12

Table 3: Results of Subject-Independent Setup. The training, validation, and
test sets are distributed based on subjects according to a predetermined
ratio/IDs. Results of the APAVA, TDBrain, ADFD, PTB, and PTB-XL datasets under
this setup are presented here.

Datasets | Models | Accuracy | Precision | Recall | F1 score | AUROC | AUPRC
---|---|---|---|---|---|---|---
APAVA (2-Classes) | Autoformer | 68.64$\pm$1.82 | 68.48$\pm$2.10 | 68.77$\pm$2.27 | 68.06$\pm$1.94 | 75.94$\pm$3.61 | 74.38$\pm$4.05
Crossformer | 73.77$\pm$1.95 | 79.29$\pm$4.36 | 68.86$\pm$1.70 | 68.93$\pm$1.85 | 72.39$\pm$3.33 | 72.05$\pm$3.65
FEDformer | 74.94$\pm$2.15 | 74.59$\pm$1.50 | 73.56$\pm$3.55 | 73.51$\pm$3.39 | 83.72$\pm$1.97 | 82.94$\pm$2.37
Informer | 73.11$\pm$4.40 | 75.17$\pm$6.06 | 69.17$\pm$4.56 | 69.47$\pm$5.06 | 70.46$\pm$4.91 | 70.75$\pm$5.27
iTransformer | 74.55$\pm$1.66 | 74.77$\pm$2.10 | 71.76$\pm$1.72 | 72.30$\pm$1.79 | 85.59$\pm$1.55 | 84.39$\pm$1.57
MTST | 71.14$\pm$1.59 | 79.30$\pm$0.97 | 65.27$\pm$2.28 | 64.01$\pm$3.16 | 68.87$\pm$2.34 | 71.06$\pm$1.60
Nonformer | 71.89$\pm$3.81 | 71.80$\pm$4.58 | 69.44$\pm$3.56 | 69.74$\pm$3.84 | 70.55$\pm$2.96 | 70.78$\pm$4.08
PatchTST | 67.03$\pm$1.65 | 78.76$\pm$1.28 | 59.91$\pm$2.02 | 55.97$\pm$3.10 | 65.65$\pm$0.28 | 67.99$\pm$0.76
Reformer | 78.70$\pm$2.00 | 82.50$\pm$3.95 | 75.00$\pm$1.61 | 75.93$\pm$1.82 | 73.94$\pm$1.40 | 76.04$\pm$1.14
Transformer | 76.30$\pm$4.72 | 77.64$\pm$5.95 | 73.09$\pm$5.01 | 73.75$\pm$5.38 | 72.50$\pm$6.60 | 73.23$\pm$7.60
Medformer (Ours) | 78.74$\pm$0.64 | 81.11$\pm$0.84 | 75.40$\pm$0.66 | 76.31$\pm$0.71 | 83.20$\pm$0.91 | 83.66$\pm$0.92
TDBrain (2-Classes) | Autoformer | 87.33$\pm$3.79 | 88.06$\pm$3.56 | 87.33$\pm$3.79 | 87.26$\pm$3.84 | 93.81$\pm$2.26 | 93.32$\pm$2.42
Crossformer | 81.56$\pm$2.19 | 81.97$\pm$2.25 | 81.56$\pm$2.19 | 81.50$\pm$2.20 | 91.20$\pm$1.78 | 91.51$\pm$1.71
FEDformer | 78.13$\pm$1.98 | 78.52$\pm$1.91 | 78.13$\pm$1.98 | 78.04$\pm$2.01 | 86.56$\pm$1.86 | 86.48$\pm$1.99
Informer | 89.02$\pm$2.50 | 89.43$\pm$2.14 | 89.02$\pm$2.50 | 88.98$\pm$2.54 | 96.64$\pm$0.68 | 96.75$\pm$0.63
iTransformer | 74.67$\pm$1.06 | 74.71$\pm$1.06 | 74.67$\pm$1.06 | 74.65$\pm$1.06 | 83.37$\pm$1.14 | 83.73$\pm$1.27
MTST | 76.96$\pm$3.76 | 77.24$\pm$3.59 | 76.96$\pm$3.76 | 76.88$\pm$3.83 | 85.27$\pm$4.46 | 82.81$\pm$5.64
Nonformer | 87.88$\pm$2.48 | 88.86$\pm$1.84 | 87.88$\pm$2.48 | 87.78$\pm$2.56 | 97.05$\pm$0.68 | 96.99$\pm$0.68
PatchTST | 79.25$\pm$3.79 | 79.60$\pm$4.09 | 79.25$\pm$3.79 | 79.20$\pm$3.77 | 87.95$\pm$4.96 | 86.36$\pm$6.67
Reformer | 87.92$\pm$2.01 | 88.64$\pm$1.40 | 87.92$\pm$2.01 | 87.85$\pm$2.08 | 96.30$\pm$0.54 | 96.40$\pm$0.45
Transformer | 87.17$\pm$1.67 | 87.99$\pm$1.68 | 87.17$\pm$1.67 | 87.10$\pm$1.68 | 96.28$\pm$0.92 | 96.34$\pm$0.81
Medformer (Ours) | 89.62$\pm$0.81 | 89.68$\pm$0.78 | 89.62$\pm$0.81 | 89.62$\pm$0.81 | 96.41$\pm$0.35 | 96.51$\pm$0.33
ADFD (3-Classes) | Autoformer | 45.25$\pm$1.48 | 43.67$\pm$1.94 | 42.96$\pm$2.03 | 42.59$\pm$1.85 | 61.02$\pm$1.82 | 43.10$\pm$2.30
Crossformer | 50.45$\pm$2.31 | 45.57$\pm$1.63 | 45.88$\pm$1.82 | 45.50$\pm$1.70 | 66.45$\pm$2.03 | 48.33$\pm$2.05
FEDformer | 46.30$\pm$0.59 | 46.05$\pm$0.76 | 44.22$\pm$1.38 | 43.91$\pm$1.37 | 62.62$\pm$1.75 | 46.11$\pm$1.44
Informer | 48.45$\pm$1.96 | 46.54$\pm$1.68 | 46.06$\pm$1.84 | 45.74$\pm$1.38 | 65.87$\pm$1.27 | 47.60$\pm$1.30
iTransformer | 52.60$\pm$1.59 | 46.79$\pm$1.27 | 47.28$\pm$1.29 | 46.79$\pm$1.13 | 67.26$\pm$1.16 | 49.53$\pm$1.21
MTST | 45.60$\pm$2.03 | 44.70$\pm$1.33 | 45.05$\pm$1.30 | 44.31$\pm$1.74 | 62.50$\pm$0.81 | 45.16$\pm$0.85
Nonformer | 49.95$\pm$1.05 | 47.71$\pm$0.97 | 47.46$\pm$1.50 | 46.96$\pm$1.35 | 66.23$\pm$1.37 | 47.33$\pm$1.78
PatchTST | 44.37$\pm$0.95 | 42.40$\pm$1.13 | 42.06$\pm$1.48 | 41.97$\pm$1.37 | 60.08$\pm$1.50 | 42.49$\pm$1.79
Reformer | 50.78$\pm$1.17 | 49.64$\pm$1.49 | 49.89$\pm$1.67 | 47.94$\pm$0.69 | 69.17$\pm$1.58 | 51.73$\pm$1.94
Transformer | 50.47$\pm$2.14 | 49.13$\pm$1.83 | 48.01$\pm$1.53 | 48.09$\pm$1.59 | 67.93$\pm$1.59 | 48.93$\pm$2.02
Medformer (Ours) | 53.27$\pm$1.54 | 51.02$\pm$1.57 | 50.71$\pm$1.55 | 50.65$\pm$1.51 | 70.93$\pm$1.19 | 51.21$\pm$1.32
PTB (2-Classes) | Autoformer | 73.35$\pm$2.10 | 72.11$\pm$2.89 | 63.24$\pm$3.17 | 63.69$\pm$3.84 | 78.54$\pm$3.48 | 74.25$\pm$3.53
Crossformer | 80.17$\pm$3.79 | 85.04$\pm$1.83 | 71.25$\pm$6.29 | 72.75$\pm$7.19 | 88.55$\pm$3.45 | 87.31$\pm$3.25
FEDformer | 76.05$\pm$2.54 | 77.58$\pm$3.61 | 66.10$\pm$3.55 | 67.14$\pm$4.37 | 85.93$\pm$4.31 | 82.59$\pm$5.42
Informer | 78.69$\pm$1.68 | 82.87$\pm$1.02 | 69.19$\pm$2.90 | 70.84$\pm$3.47 | 92.09$\pm$0.53 | 90.02$\pm$0.60
iTransformer | 83.89$\pm$0.71 | 88.25$\pm$1.18 | 76.39$\pm$1.01 | 79.06$\pm$1.06 | 91.18$\pm$1.16 | 90.93$\pm$0.98
MTST | 76.59$\pm$1.90 | 79.88$\pm$1.90 | 66.31$\pm$2.95 | 67.38$\pm$3.71 | 86.86$\pm$2.75 | 83.75$\pm$2.84
Nonformer | 78.66$\pm$0.49 | 82.77$\pm$0.86 | 69.12$\pm$0.87 | 70.90$\pm$1.00 | 89.37$\pm$2.51 | 86.67$\pm$2.38
PatchTST | 74.74$\pm$1.62 | 76.94$\pm$1.51 | 63.89$\pm$2.71 | 64.36$\pm$3.38 | 88.79$\pm$0.91 | 83.39$\pm$0.96
Reformer | 77.96$\pm$2.13 | 81.72$\pm$1.61 | 68.20$\pm$3.35 | 69.65$\pm$3.88 | 91.13$\pm$0.74 | 88.42$\pm$1.30
Transformer | 77.37$\pm$1.02 | 81.84$\pm$0.66 | 67.14$\pm$1.80 | 68.47$\pm$2.19 | 90.08$\pm$1.76 | 87.22$\pm$1.68
Medformer (Ours) | 83.50$\pm$2.01 | 85.19$\pm$0.94 | 77.11$\pm$3.39 | 79.18$\pm$3.31 | 92.81$\pm$1.48 | 90.32$\pm$1.54
PTB-XL (5-Classes) | Autoformer | 61.68$\pm$2.72 | 51.60$\pm$1.64 | 49.10$\pm$1.52 | 48.85$\pm$2.27 | 82.04$\pm$1.44 | 51.93$\pm$1.71
Crossformer | 73.30$\pm$0.14 | 65.06$\pm$0.35 | 61.23$\pm$0.33 | 62.59$\pm$0.14 | 90.02$\pm$0.06 | 67.43$\pm$0.22
FEDformer | 57.20$\pm$9.47 | 52.38$\pm$6.09 | 49.04$\pm$7.26 | 47.89$\pm$8.44 | 82.13$\pm$4.17 | 52.31$\pm$7.03
Informer | 71.43$\pm$0.32 | 62.64$\pm$0.60 | 59.12$\pm$0.47 | 60.44$\pm$0.43 | 88.65$\pm$0.09 | 64.76$\pm$0.17
iTransformer | 69.28$\pm$0.22 | 59.59$\pm$0.45 | 54.62$\pm$0.18 | 56.20$\pm$0.19 | 86.71$\pm$0.10 | 60.27$\pm$0.21
MTST | 72.14$\pm$0.27 | 63.84$\pm$0.72 | 60.01$\pm$0.81 | 61.43$\pm$0.38 | 88.97$\pm$0.33 | 65.83$\pm$0.51
Nonformer | 70.56$\pm$0.55 | 61.57$\pm$0.66 | 57.75$\pm$0.72 | 59.10$\pm$0.66 | 88.32$\pm$0.36 | 63.40$\pm$0.79
PatchTST | 73.23$\pm$0.25 | 65.70$\pm$0.64 | 60.82$\pm$0.76 | 62.61$\pm$0.34 | 89.74$\pm$0.19 | 67.32$\pm$0.22
Reformer | 71.72$\pm$0.43 | 63.12$\pm$1.02 | 59.20$\pm$0.75 | 60.69$\pm$0.18 | 88.80$\pm$0.24 | 64.72$\pm$0.47
Transformer | 70.59$\pm$0.44 | 61.57$\pm$0.65 | 57.62$\pm$0.35 | 59.05$\pm$0.25 | 88.21$\pm$0.16 | 63.36$\pm$0.29
Medformer (Ours) | 72.87$\pm$0.23 | 64.14$\pm$0.42 | 60.60$\pm$0.46 | 62.02$\pm$0.37 | 89.66$\pm$0.13 | 66.39$\pm$0.22

###  5.1 Results of Subject-Dependent

Setup. In this setup, the training, validation, and test sets are split based
on samples. All samples from all subjects are randomly shuffled and
distributed into the training, validation, and test sets according to a
predetermined ratio, allowing samples from the same subject to appear in three
sets simultaneously. As discussed in the Preliminaries section 3, this setup
has limited applicability for medical time-series-based disease diagnosis in
real-world scenarios. However, it can be utilized to evaluate whether the
dataset exhibits cross-subject features quickly. The results obtained from
this setup are typically much higher than those from the subject-independent
setup, showing a dataset’s upper limit of learnability.

Results. We evaluate the EEG dataset ADFD using this setup to provide a direct
comparison of results with the subject-independent setup. The results are
presented in Table 2. Our method outperforms all the baselines, achieving the
top-1 results in all six evaluations, with an impressive F1 score of 97.50%.
Notably, baseline methods like Informer, Nonformer, Reformer, and Transformer
also demonstrate strong performance, achieving F1 scores exceeding 90%. The
overall results indicate the presence of discernible and learnable features
related to Alzheimer’s Disease within this dataset.

###  5.2 Results of Subject-Independent

Setup. In this setup, the training, validation, and test sets are split based
on subjects. All subjects and their corresponding samples are distributed into
the training, validation, and test sets according to a predetermined ratio or
subject IDs. Samples from the same subjects should exclusively appear in one
of these three sets. This setup simulates real-world medical time-series-based
disease diagnosis, wherein the aim is to train a model on subjects with known
labels and then test it on unseen subjects to determine if they have a
specific disease. The challenges associated with this setup are discussed in
section 2. All five datasets are evaluated using this setup.

Figure 4: Average Rank of Subject-Independent Setup. The heatmap table shows
the average rank of Medformer and 10 baselines across 5 datasets using the
subject-independent setup. A lower number indicates better results. The
average rank is calculated across the 5 datasets to obtain the overall average
rank.

Results. Table 3 presents the results of the subject-independent setup. Our
method achieves the top-1 F1 scores on 4 out of 5 datasets. Overall, our
method achieves 15 top-1 and 30 top-3 rankings out of 30 evaluations conducted
across 5 datasets and 10 baselines, considering 6 different metrics. Figure 4
provides an overview heatmap table of average rank across 5 datasets on 6
metrics for all methods. Lower rank numbers indicate better results, with rank
1 representing the best performance among all methods. Our method demonstrates
the best average rank among all methods across the 6 metrics. Additionally, it
is notable that the result for ADFD is a 50.65% F1 score under the subject-
independent setup, which is significantly lower than the 97.50% F1 score
achieved under the subject-dependent setup. This comparison highlights the
challenge of the subject-independent setup, which better simulates real-world
scenarios.

Ablation Study. 1) Module Study: We conduct a module study to evaluate the
effectiveness of each proposed mechanism in our method (Appendix D.1). 2)
Patch Length Study: We perform parameter tuning on the list of patch lengths
to evaluate the effectiveness of multi-granularities. (Appendix D.2).

##  6 Conclusion

This paper introduces Medformer, a multi-granularity patching transformer
designed for medical time series classification. We develop three novel
mechanisms to leverage the unique characteristics of medical time series, such
as channel correlations and frequency band biomarkers. These mechanisms
include cross-channel patching to learn multi-timestamp and multi-channel
features, multi-granularity embedding to capture features at different scales,
and two-stage multi-granularity self-attention to capture features within and
among granularities. Results on five datasets, compared with ten baselines
under a subject-independent setup, demonstrate the effectiveness and stability
of our method, highlighting its potential for real-world applications. The
limitations and future works of our method are discussed in Appendix F.2.

## References

  * [1] Yara Badr, Usman Tariq, Fares Al-Shargie, Fabio Babiloni, Fadwa Al Mughairbi, and Hasan Al-Nashash.  A review on evaluating mental stress by deep learning using eeg signals.  Neural Computing and Applications, pages 1–26, 2024.
  * [2] Hamdi Altaheri, Ghulam Muhammad, Mansour Alsulaiman, Syed Umar Amin, Ghadir Ali Altuwaijri, Wadood Abdul, Mohamed A Bencherif, and Mohammed Faisal.  Deep learning techniques for classification of electroencephalogram (eeg) motor imagery (mi) signals: A review.  Neural Computing and Applications, 35(20):14681–14722, 2023.
  * [3] Xinwen Liu, Huan Wang, Zongjin Li, and Lang Qin.  Deep learning in ecg diagnosis: A review.  Knowledge-Based Systems, 227:107187, 2021.
  * [4] Fatma Murat, Ozal Yildirim, Muhammed Talo, Ulas Baran Baloglu, Yakup Demir, and U Rajendra Acharya.  Application of deep learning techniques for heartbeats detection using ecg signals-analysis and review.  Computers in biology and medicine, 120:103726, 2020.
  * [5] Aniqa Arif, Yihe Wang, Rui Yin, Xiang Zhang, and Ahmed Helmy.  Ef-net: Mental state recognition by analyzing multimodal eeg-fnirs via cnn.  Sensors, 24(6):1889, 2024.
  * [6] Mahboobeh Jafari, Afshin Shoeibi, Marjane Khodatars, Sara Bagherzadeh, Ahmad Shalbaf, David López García, Juan M Gorriz, and U Rajendra Acharya.  Emotion recognition in eeg signals using deep learning methods: A review.  Computers in Biology and Medicine, page 107450, 2023.
  * [7] Qiao Xiao, Khuan Lee, Siti Aisah Mokhtar, Iskasymar Ismail, Ahmad Luqman bin Md Pauzi, Qiuxia Zhang, and Poh Ying Lim.  Deep learning-based ecg arrhythmia classification: A systematic review.  Applied Sciences, 13(8):4964, 2023.
  * [8] Zekai Wang, Stavros Stavrakis, and Bing Yao.  Hierarchical deep learning with generative adversarial network for automatic cardiac diagnosis from ecg signals.  Computers in Biology and Medicine, 155:106641, 2023.
  * [9] Dani Kiyasseh, Tingting Zhu, and David A Clifton.  Clocs: Contrastive learning of cardiac signals across space, time, and patients.  In International Conference on Machine Learning, pages 5606–5615. PMLR, 2021.
  * [10] Katerina D Tzimourta, Vasileios Christou, Alexandros T Tzallas, Nikolaos Giannakeas, Loukas G Astrakas, Pantelis Angelidis, Dimitrios Tsalikakis, and Markos G Tsipouras.  Machine learning algorithms and statistical approaches for alzheimer’s disease analysis based on resting-state eeg recordings: A systematic review.  International journal of neural systems, 31(05):2130002, 2021.
  * [11] Golshan Fahimi, Seyed Mahmoud Tabatabaei, Elnaz Fahimi, and Hamid Rajebi.  Index of theta/alpha ratio of the quantitative electroencephalogram in alzheimer’s disease: a case-control study.  Acta Medica Iranica, pages 502–506, 2017.
  * [12] Salah S Al-Zaiti, Christian Martin-Gill, Jessica K Zègre-Hemsey, Zeineb Bouzid, Ziad Faramand, Mohammad O Alrawashdeh, Richard E Gregg, Stephanie Helman, Nathan T Riek, Karina Kraevsky-Phillips, et al.  Machine learning for ecg diagnosis and risk stratification of occlusion myocardial infarction.  Nature Medicine, 29(7):1804–1813, 2023.
  * [13] Yihe Wang, Yu Han, Haishuai Wang, and Xiang Zhang.  Contrast everything: A hierarchical contrastive framework for medical time-series.  Advances in Neural Information Processing Systems, 36, 2024.
  * [14] Xiaojun Bi and Haibo Wang.  Early alzheimer’s disease diagnosis based on eeg spectral images using deep learning.  Neural Networks, 114:119–135, 2019.
  * [15] Roberta Avanzato and Francesco Beritelli.  Automatic ecg diagnosis using convolutional neural network.  Electronics, 9(6):951, 2020.
  * [16] Hisaki Makimoto, Moritz Höckmann, Tina Lin, David Glöckner, Shqipe Gerguri, Lukas Clasen, Jan Schmidt, Athena Assadi-Schmidt, Alexandru Bejinariu, Patrick Müller, et al.  Performance of a convolutional neural network derived from an ecg database in recognizing myocardial infarction.  Scientific reports, 10(1):8445, 2020.
  * [17] Vernon J Lawhern, Amelia J Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung, and Brent J Lance.  Eegnet: a compact convolutional neural network for eeg-based brain–computer interfaces.  Journal of neural engineering, 15(5):056013, 2018.
  * [18] Shaojie Bai, J Zico Kolter, and Vladlen Koltun.  An empirical evaluation of generic convolutional and recurrent networks for sequence modeling.  arXiv preprint arXiv:1803.01271, 2018.
  * [19] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar.  Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting.  In International conference on learning representations, 2021.
  * [20] Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi.  Etsformer: Exponential smoothing transformers for time-series forecasting.  arXiv preprint arXiv:2202.01381, 2022.
  * [21] Xue Wang, Tian Zhou, Qingsong Wen, Jinyang Gao, Bolin Ding, and Rong Jin.  Card: Channel aligned robust blend transformer for time series forecasting.  In International Conference on Learning Representations, 2023.
  * [22] Minghao Liu, Shengqi Ren, Siyuan Ma, Jiahui Jiao, Yizhou Chen, Zhiguang Wang, and Wei Song.  Gated transformer networks for multivariate time series classification.  arXiv preprint arXiv:2103.14438, 2021.
  * [23] Zekun Li, Shiyang Li, and Xifeng Yan.  Time series as images: Vision transformer for irregularly sampled time series.  Advances in Neural Information Processing Systems, 36, 2024.
  * [24] Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long.  Anomaly transformer: Time series anomaly detection with association discrepancy.  International Conference on Learning Representations, 2021.
  * [25] Junho Song, Keonwoo Kim, Jeonglyul Oh, and Sungzoon Cho.  Memto: Memory-guided transformer for multivariate time series anomaly detection.  Advances in Neural Information Processing Systems, 36, 2024.
  * [26] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long.  Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting.  Advances in Neural Information Processing Systems, 34:22419–22430, 2021.
  * [27] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.  Informer: Beyond efficient transformer for long sequence time-series forecasting.  In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021.
  * [28] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [29] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long.  itransformer: Inverted transformers are effective for time series forecasting.  International Conference on Learning Representations, 2024.
  * [30] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam.  A time series is worth 64 words: Long-term forecasting with transformers.  ICLR, 2023.
  * [31] Vincent Bazinet, Justine Y Hansen, and Bratislav Misic.  Towards a biologically annotated brain connectome.  Nature reviews neuroscience, 24(12):747–760, 2023.
  * [32] Saeid Sanei and Jonathon A Chambers.  EEG signal processing.  John Wiley & Sons, 2013.
  * [33] Selcan Kaplan Berkaya, Alper Kursat Uysal, Efnan Sora Gunal, Semih Ergin, Serkan Gunal, and M Bilginer Gulmezoglu.  A survey on ecg analysis.  Biomedical Signal Processing and Control, 43:216–235, 2018.
  * [34] Yunhao Zhang and Junchi Yan.  Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting.  In The Eleventh International Conference on Learning Representations, 2022.
  * [35] Siyi Tang, Jared Dunnmon, Khaled Kamal Saab, Xuan Zhang, Qianying Huang, Florian Dubost, Daniel Rubin, and Christopher Lee-Messer.  Self-supervised graph neural networks for improved electroencephalographic seizure analysis.  In International Conference on Learning Representations, 2021.
  * [36] Chaoqi Yang, M Brandon Westover, and Jimeng Sun.  Manydg: Many-domain generalization for healthcare applications.  In The Eleventh International Conference on Learning Representations, 2023.
  * [37] Xiaodong Qu, Zepeng Hu, Zhaonan Li, and Timothy J Hickey.  Ensemble methods and lstm outperformed other eight machine learning classifiers in an eeg-based bci experiment.  In International Conference on Learning Representations, 2020.
  * [38] Dezhen Xiong, Daohui Zhang, Xingang Zhao, and Yiwen Zhao.  Deep learning for emg-based human-machine interaction: A review.  IEEE/CAA Journal of Automatica Sinica, 8(3):512–533, 2021.
  * [39] Yuanchao Dai, Jing Wu, Yuanzhao Fan, Jin Wang, Jianwei Niu, Fei Gu, and Shigen Shen.  Mseva: A musculoskeletal rehabilitation evaluation system based on emg signals.  ACM Transactions on Sensor Networks, 19(1):1–23, 2022.
  * [40] Yingying Jiao, Yini Deng, Yun Luo, and Bao-Liang Lu.  Driver sleepiness detection from eeg and eog signals using gan and lstm networks.  Neurocomputing, 408:100–111, 2020.
  * [41] Jiahao Fan, Chenglu Sun, Meng Long, Chen Chen, and Wei Chen.  Eognet: a novel deep learning model for sleep stage classification based on single-channel eog signal.  Frontiers in Neuroscience, 15:573194, 2021.
  * [42] Saifuddin Mahmud, Xiangxu Lin, and Jong-Hoon Kim.  Interface for human machine interaction for assistant devices: A review.  In 2020 10th Annual computing and communication workshop and conference (CCWC), pages 0768–0773. IEEE, 2020.
  * [43] Magali T Schmidt, Paulo AM Kanda, Luis FH Basile, Helder Frederico da Silva Lopes, Regina Baratho, Jose LC Demario, Mario S Jorge, Antonio E Nardi, Sergio Machado, Jéssica N Ianof, et al.  Index of alpha/theta ratio of the electroencephalogram: a new marker for alzheimer’s disease.  Frontiers in aging neuroscience, 5:60, 2013.
  * [44] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin.  Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting.  In International Conference on Machine Learning, pages 27268–27286. PMLR, 2022.
  * [45] Yitian Zhang, Liheng Ma, Soumyasundar Pal, Yingxue Zhang, and Mark Coates.  Multi-resolution time-series transformer for long-term forecasting.  In International Conference on Artificial Intelligence and Statistics, pages 4222–4230. PMLR, 2024.
  * [46] Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long.  Non-stationary transformers: Exploring the stationarity in time series forecasting.  Advances in Neural Information Processing Systems, 35:9881–9893, 2022.
  * [47] Peng Chen, Yingying ZHANG, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo.  Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting.  In The Twelfth International Conference on Learning Representations, 2024.
  * [48] Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya.  Reformer: The efficient transformer.  In International Conference on Learning Representations, 2019.
  * [49] Amin Shabani, Amir Abdi, Lili Meng, and Tristan Sylvain.  Scaleformer: Iterative multi-scale refining transformers for time series forecasting.  arXiv preprint arXiv:2206.04038, 2022.
  * [50] Joseph Y Cheng, Hanlin Goh, Kaan Dogrusoz, Oncel Tuzel, and Erdrin Azemi.  Subject-aware contrastive learning for biosignals.  arXiv preprint arXiv:2007.04871, 2020.
  * [51] Isabela Albuquerque, João Monteiro, Olivier Rosanne, Abhishek Tiwari, Jean-François Gagnon, and Tiago H Falk.  Cross-subject statistical shift estimation for generalized electroencephalography-based mental workload assessment.  In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pages 3647–3653. IEEE, 2019.
  * [52] Chaoqi Yang, M Brandon Westover, and Jimeng Sun.  Manydg: Many-domain generalization for healthcare applications.  In The Eleventh International Conference on Learning Representations, 2022.
  * [53] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In International conference on machine learning, pages 1597–1607. PMLR, 2020.
  * [54] Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu.  Ts2vec: Towards universal representation of time series.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8980–8987, 2022.
  * [55] J Escudero, Daniel Abásolo, Roberto Hornero, Pedro Espino, and Miguel López.  Analysis of electroencephalograms in alzheimer’s disease patients with multiscale entropy.  Physiological measurement, 27(11):1091, 2006.
  * [56] Hanneke van Dijk, Guido van Wingen, Damiaan Denys, Sebastian Olbrich, Rosalinde van Ruth, and Martijn Arns.  The two decades brainclinics research archive for insights in neurophysiology (tdbrain) database.  Scientific data, 9(1):333, 2022.
  * [57] Andreas Miltiadous, Katerina D Tzimourta, Theodora Afrantou, Panagiotis Ioannidis, Nikolaos Grigoriadis, Dimitrios G Tsalikakis, Pantelis Angelidis, Markos G Tsipouras, Euripidis Glavas, Nikolaos Giannakeas, et al.  A dataset of scalp eeg recordings of alzheimer’s disease, frontotemporal dementia and healthy subjects from routine eeg.  Data, 8(6):95, 2023.
  * [58] Andreas Miltiadous, Emmanouil Gionanidis, Katerina D Tzimourta, Nikolaos Giannakeas, and Alexandros T Tzallas.  Dice-net: a novel convolution-transformer architecture for alzheimer detection in eeg signals.  IEEE Access, 2023.
  * [59] PhysioToolkit PhysioBank.  Physionet: components of a new research resource for complex physiologic signals.  Circulation, 101(23):e215–e220, 2000.
  * [60] Patrick Wagner, Nils Strodthoff, Ralf-Dieter Bousseljot, Dieter Kreiseler, Fatima I Lunze, Wojciech Samek, and Tobias Schaeffter.  Ptb-xl, a large publicly available electrocardiography dataset.  Scientific data, 7(1):1–15, 2020.
  * [61] Johannes Pöppelbaum, Gavneet Singh Chadha, and Andreas Schwung.  Contrastive learning based self-supervised time-series analysis.  Applied Soft Computing, 117:108397, 2022.
  * [62] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long.  Timesnet: Temporal 2d-variation modeling for general time series analysis.  In International Conference on Learning Representations, 2023.

##  Appendix A Data Augmentation Banks

In the embedding stage, we apply data augmentation to the patch embeddings. We
utilize a bank of data augmentation techniques to enhance the model’s
robustness and generalization. During the forward pass in training, each patch
will pick one augmentation from available augmentation options with equal
probability. The data augmentation techniques include masking, jittering,
scaling, and can be further expanded to more choices. We provide a detailed
description of each technique below.

###  A.1 Masking

Masking is a common data augmentation technique in sequence modeling tasks. It
randomly masks a portion of the input sequence to encourage the model to learn
from the remaining information. In our implementation, we randomly mask each
element of $\mathbf{x}_{\text{emb}}^{(i)}$ with a probability of
$p_{\text{mask}}$. The masked elements are replaced with zeros. The masking
probability $p_{\text{mask}}$ is a hyperparameter that controls the masking
rate.

###  A.2 Jittering

Jittering is also a common data augmentation technique by adding random noise
to the input sequence. It helps the model to learn robust features by
introducing variations in the input data. In our implementation, we add
Gaussian noise to each element of $\mathbf{x}_{\text{emb}}^{(i)}$ with a mean
of $0$ and a standard deviation of $\sigma_{\text{jitter}}$. The jittering
standard deviation $\sigma_{\text{jitter}}$ is a hyperparameter that controls
the noise level.

###  A.3 Scaling

Scaling is yet another data augmentation technique that rescales the input
sequence by a random factor. It helps the model to learn invariant features by
adjusting the input data’s scale. In our implementation, we multiply each
element of $\mathbf{x}_{\text{emb}}^{(i)}$ by a random scaling factor sampled
from a Gaussian distribution with a mean of $1$ and a standard deviation of
$\sigma_{\text{scale}}$. The scaling standard deviation
$\sigma_{\text{scale}}$ is a hyperparameter that controls the scaling level.

##  Appendix B Data Preprocessing

###  B.1 APAVA Preprocessing

The Alzheimer’s Patients’ Relatives Association of Valladolid (APAVA)
dataset***https://osf.io/jbysn/, referenced in the paper [55], is a public EEG
time series dataset with 2 classes and 23 subjects, including 12 Alzheimer’s
disease patients and 11 healthy control subjects. On average, each subject has
30.0 $\pm$ 12.5 trials, with each trial being a 5-second time sequence
consisting of 1280 timestamps across 16 channels. Before further
preprocessing, each trial is scaled using the standard scaler. Subsequently,
we segment each trial into 9 half-overlapping samples, where each sample is a
1-second time sequence comprising 256 timestamps. This process results in
5,967 samples. Each sample has a subject ID to indicate its originating
subject. For the training, validation, and test set splits, we employ the
subject-independent setup. Samples with subject IDs {15,16,19,20} and
{1,2,17,18} are assigned to the validation and test sets, respectively. The
remaining samples are allocated to the training set.

###  B.2 TDBrain Preprocessing

The TDBrain dataset***https://brainclinics.com/resources/, referenced in the
paper [56], is a large permission-accessible EEG time series dataset recording
brain activities of 1274 subjects with 33 channels. Each subject has two
trials: one under eye open and one under eye closed setup. The dataset
includes a total of 60 labels, with each subject potentially having multiple
labels indicating multiple diseases simultaneously. In this paper, we utilize
a subset of this dataset containing 25 subjects with Parkinson’s disease and
25 healthy controls, all under the eye-closed task condition. Each eye-closed
trial is segmented into non-overlapping 1-second samples with 256 timestamps,
and any samples shorter than 1-second are discarded. This process results in
6,240 samples. Each sample is assigned a subject ID to indicate its
originating subject. For the training, validation, and test set splits, we
employ the subject-independent setup. Samples with subject IDs
{18,19,20,21,46,47,48,49} are assigned to the validation set, while samples
with subject IDs {22,23,24,25,50,51,52,53} are assigned to the test set. The
remaining samples are allocated to the training set.

###  B.3 ADFD Preprocessing

The Alzheimer’s Disease and Frontotemporal Dementia (ADFD)
dataset***https://openneuro.org/datasets/ds004504/versions/1.0.6, referenced
in the papers [57, 58], is a public EEG time series dataset with 3 classes,
including 36 Alzheimer’s disease (AD) patients, 23 Frontotemporal Dementia
(FD) patients, and 29 healthy control (HC) subjects. The dataset has 19
channels, and the raw sampling rate is 500Hz. Each subject has a trial, with
trial durations of approximately 13.5 minutes for AD subjects (min=5.1,
max=21.3), 12 minutes for FD subjects (min=7.9, max=16.9), and 13.8 minutes
for HC subjects (min=12.5, max=16.5). A bandpass filter between 0.5-45Hz is
applied to each trial. We downsample each trial to 256Hz and segment them into
non-overlapping 1-second samples with 256 timestamps, discarding any samples
shorter than 1 second. This process results in 69,752 samples. For the
training, validation, and test set splits, we employ both the subject-
dependent and subject-independent setups. For the subject-dependent setup, we
allocate 60%, 20%, and 20% of total samples into the training, validation, and
test sets, respectively. For the subject-independent setup, we allocate 60%,
20%, and 20% of total subjects with their corresponding samples into the
training, validation, and test sets, respectively.

###  B.4 PTB Preprocessing

The PTB dataset***https://physionet.org/content/ptbdb/1.0.0/, referenced in
the paper [59], is a public ECG time series recording from 290 subjects, with
15 channels and a total of 8 labels representing 7 heart diseases and 1 health
control. The raw sampling rate is 1000Hz. For this paper, we utilize a subset
of 198 subjects, including patients with Myocardial infarction and healthy
control subjects. We first downsample the sampling frequency to 250Hz and
normalize the ECG signals using standard scalers. Subsequently, we process the
data into single heartbeats through several steps. We identify the R-Peak
intervals across all channels and remove any outliers. Each heartbeat is then
sampled from its R-Peak position, and we ensure all samples have the same
length by applying zero padding to shorter samples, with the maximum duration
across all channels serving as the reference. This process results in 64,356
samples. For the training, validation, and test set splits, we employ the
subject-independent setup. Specifically, we allocate 60%, 20%, and 20% of the
total subjects, along with their corresponding samples, into the training,
validation, and test sets, respectively.

###  B.5 PTB-XL Preprocessing

The PTB-XL dataset***https://physionet.org/content/ptb-xl/1.0.3/, referenced
in the paper [60], is a large public ECG time series dataset recorded from
18,869 subjects, with 12 channels and 5 labels representing 4 heart diseases
and 1 healthy control category. Each subject may have one or more trials. To
ensure consistency, we discard subjects with varying diagnosis results across
different trials, resulting in 17,596 subjects remaining. The raw trials
consist of 10-second time intervals, with sampling frequencies of 100Hz and
500Hz versions. For our paper, we utilize the 500Hz version, then we
downsample to 250Hz and normalize using standard scalers. Subsequently, each
trial is segmented into non-overlapping 1-second samples with 250 timestamps,
discarding any samples shorter than 1 second. This process results in 191,400
samples. For the training, validation, and test set splits, we employ the
subject-independent setup. Specifically, we allocate 60%, 20%, and 20% of the
total subjects, along with their corresponding samples, into the training,
validation, and test sets, respectively.

##  Appendix C Implementation Details

We implement our method and all the baselines based on the Time-Series-Library
project***https://github.com/thuml/Time-Series-Library from Tsinghua
University [62], which integrates all methods under the same framework and
training techniques to ensure a relatively fair comparison. The 10 baseline
time series transformer methods are Autoformer [26], Crossformer [34],
FEDformer [44], Informer [27], iTransformer [29], MTST [45], Nonformer [46],
PatchTST [30], Reformer [48], and vanilla Transformer [28].

For all methods, we employ 6 layers for the encoder, with the self-attention
dimension $D$ set to 128 and the hidden dimension of the feed-forward networks
set to 256. The optimizer used is Adam, with a learning rate of 1e-4. The
batch size is set to {32,32,128,128,128} for the datasets APAVA, TDBrain,
ADFD, PTB, and PTB-XL, respectively. Training is conducted for 100 epochs,
with early stopping triggered after 10 epochs without improvement in the F1
score on the validation set. We save the model with the best F1 score on the
validation set and evaluate it on the test set. We employ six evaluation
metrics: accuracy, precision (macro-averaged), recall (macro-averaged), F1
score (macro-averaged), AUROC (macro-averaged), and AUPRC (macro-averaged).
Both subject-dependent and subject-independent setups are implemented for
different datasets. Each experiment is run with 5 random seeds (41-45) and
fixed training, validation, and test sets to compute the average results and
standard deviations.

Medformer (Our Method) We use a list of patch lengths in patch embedding to
generate patches with different granularities. Instead of flattening the
patches and mapping them to dimension $D$ during patch embedding, we use a
conv2d network to directly map patches into a 1-D representation with
dimension $D$. These patch lengths can vary, including different numbers of
patch lengths such as $\\{2,4,8,16\\}$, repetitive numbers such as
$\\{8,8,8,8\\}$, or a mix of different and repetitive lengths such as
$\\{8,8,8,16,16,16\\}$. It is also possible to use only one patch length, such
as $\\{8\\}$, which indicates a single granularity. The patch lists used for
the datasets APAVA, TDBrain, ADFD, PTB, and PTB-XL are
$\\{2,2,2,4,4,4,16,16,16,16,32,32,32,32,32\\}$, $\\{8,8,8,16,16,16\\}$,
$\\{2,4,8,8,16,16,16,16,32,32,32,32,32,32,32,32\\}$,
$\\{2,4,8,8,16,16,16,32,32,32,32,32\\}$, and
$\\{2,4,8,8,16,16,16,16,32,32,32,32,32,32,32,32\\}$, respectively. The data
augmentations are randomly chosen from a list of four possible options: none,
jitter, scale, and mask. The number following each augmentation method
indicates the degree of augmentation. Detailed descriptions of these methods
can be found in Appendix A. The augmentation methods used for the datasets
APAVA, TDBrain, ADFD, PTB, and PTB-XL are {none, mask0.35}, {none, mask0.25},
{mask0.5}, {mask0.5}, and {jitter0.2, scale0.2, mask0.5}, respectively.

Autoformer Autoformer [26] employs an auto-correlation mechanism to replace
self-attention for time series forecasting. Additionally, they use a time
series decomposition block to separate the time series into trend-cyclical and
seasonal components for improved learning. The raw source code is available at
https://github.com/thuml/Autoformer.

Crossformer Crossformer [34] designs a single-channel patching approach for
token embedding. They utilize two-stage self-attention to leverage both
temporal features and channel correlations. A router mechanism is proposed to
reduce time and space complexity during the cross-dimension stage. The raw
code is available at https://github.com/Thinklab-SJTU/Crossformer.

FEDformer FEDformer [44] leverages frequency domain information using the
Fourier transform. They introduce frequency-enhanced blocks and frequency-
enhanced attention, which are computed in the frequency domain. A novel time
series decomposition method replaces the layer norm module in the transformer
architecture to improve learning. The raw code is available at
https://github.com/MAZiqing/FEDformer.

Informer Informer [27] is the first paper to employ a one-forward procedure
instead of an autoregressive method in time series forecasting tasks. They
introduce ProbSparse self-attention to reduce complexity and memory usage. The
raw code is available at https://github.com/zhouhaoyi/Informer2020.

iTransformer iTransformer [29] questions the conventional approach of
embedding attention tokens in time series forecasting tasks and proposes an
inverted approach by embedding the whole series of channels into a token. They
also invert the dimension of other transformer modules, such as the layer norm
and feed-forward networks. The raw code is available at
https://github.com/thuml/iTransformer.

MTST MTST [45] uses the same token embedding method as Crossformer and
PatchTST. It highlights the importance of different patching lengths in
forecasting tasks and designs a method that can take different sizes of patch
tokens as input simultaneously. The raw code is available at
https://github.com/networkslab/MTST.

Nonformer Nonformer [46] analyzes the impact of non-stationarity in time
series forecasting tasks and its significant effect on results. They design a
de-stationary attention module and incorporate normalization and
denormalization steps before and after training to alleviate the over-
stationarization problem. The raw code is available at
https://github.com/thuml/Nonstationary_Transformers.

PatchTST PatchTST [30] embeds a sequence of single-channel timestamps as a
patch token to replace the attention token used in the vanilla transformer.
This approach enlarges the receptive field and enhances forecasting ability.
The raw code is available at https://github.com/yuqinie98/PatchTST.

Reformer Reformer [48] replaces dot-product attention with locality-sensitive
hashing. They also use a reversible residual layer instead of standard
residuals. The raw code is available at
https://github.com/lucidrains/reformer-pytorch.

Transformer Transformer [28], commonly known as the vanilla transformer, is
introduced in the well-known paper "Attention is All You Need." It can also be
applied to time series by embedding each timestamp of all channels as an
attention token. The PyTorch version of the code is available at
https://github.com/jadore801120/attention-is-all-you-need-pytorch.

##  Appendix D Ablation Study

###  D.1 Module Study

To assess the efficacy of our proposed mechanisms—inter-granularity self-
attention, embedding augmentation, and multi-channel patching—we conduct
ablation studies on five datasets across three distinct settings: without
inter-granularity attention, without embedding augmentation, and with single-
channel patching. We maintain the other two modules intact in each setting and
fix all hyperparameters as described in the implementation details C. Table 4
presents a comparison between our full Medformer model and these three
variants. The complete Medformer model secures 28 top-1 and 30 top-2 rankings
across 30 evaluations, demonstrating robust performance. We observe that each
module significantly enhances performance: on average, across the datasets,
inter-granularity attention contributes to a 3.64% improvement in F1 score,
embedding augmentation leads to a 4.46% increase and multi-channel patching
results in a 6.10% enhancement in F1 score. We find multi-channel patching
particularly beneficial for results, especially in EEG data. Overall, these
results underscore the critical role of each component in our design.

Table 4: Module Study.

Datasets | Models | Accuracy | Precision | Recall | F1 score | AUROC | AUPRC
---|---|---|---|---|---|---|---
APAVA | No Inter-Attention | 76.90$\pm$1.50 | 78.08$\pm$2.12 | 73.87$\pm$1.48 | 74.59$\pm$1.58 | 80.29$\pm$3.75 | 81.32$\pm$3.37
No Augmentation | 75.21$\pm$2.94 | 76.69$\pm$3.41 | 71.72$\pm$3.22 | 72.30$\pm$3.46 | 77.05$\pm$5.22 | 78.15$\pm$5.42
Single-Channel Patching | 73.08$\pm$1.34 | 76.43$\pm$1.46 | 68.6$\pm$1.93 | 68.68$\pm$2.37 | 69.54$\pm$0.64 | 69.43$\pm$1.36
Medformer | 78.74$\pm$0.64 | 81.11$\pm$0.84 | 75.40$\pm$0.66 | 76.31$\pm$0.71 | 83.20$\pm$0.91 | 83.66$\pm$0.92
TDBrain | No Inter-Attention | 88.17$\pm$0.72 | 88.27$\pm$0.72 | 88.17$\pm$0.72 | 88.16$\pm$0.72 | 96.06$\pm$0.40 | 96.18$\pm$0.39
No Augmentation | 88.56$\pm$0.66 | 88.67$\pm$0.61 | 88.56$\pm$0.66 | 88.55$\pm$0.66 | 96.11$\pm$0.39 | 96.20$\pm$0.39
Single-Channel Patching | 80.94$\pm$0.95 | 81.84$\pm$1.55 | 80.94$\pm$0.95 | 80.81$\pm$0.92 | 89.65$\pm$0.85 | 89.48$\pm$0.91
Medformer | 89.62$\pm$0.81 | 89.68$\pm$0.78 | 89.62$\pm$0.81 | 89.62$\pm$0.81 | 96.41$\pm$0.35 | 96.51$\pm$0.33
ADFD | No Inter-Attention | 52.14$\pm$1.11 | 51.13$\pm$2.57 | 46.15$\pm$0.86 | 45.59$\pm$1.18 | 67.99$\pm$1.77 | 49.68$\pm$2.05
No Augmentation | 49.99$\pm$6.86 | 48.21$\pm$6.16 | 44.88$\pm$4.59 | 44.07$\pm$4.75 | 65.03$\pm$6.12 | 47.11$\pm$5.64
Single-Channel Patching | 47.09$\pm$1.22 | 45.42$\pm$1.30 | 43.94$\pm$0.80 | 44.11$\pm$0.84 | 62.07$\pm$0.86 | 44.57$\pm$0.95
Medformer | 53.27$\pm$1.54 | 51.02$\pm$1.57 | 50.71$\pm$1.55 | 50.65$\pm$1.51 | 70.93$\pm$1.19 | 51.21$\pm$1.32
PTB | No Inter-Attention | 78.02$\pm$2.70 | 80.96$\pm$1.39 | 68.65$\pm$4.58 | 69.97$\pm$5.27 | 92.94$\pm$0.86 | 90.19$\pm$1.12
No Augmentation | 77.64$\pm$1.65 | 81.03$\pm$1.60 | 67.88$\pm$2.61 | 69.31$\pm$3.22 | 92.19$\pm$0.71 | 89.37$\pm$0.96
Single-Channel Patching | 79.02$\pm$1.62 | 81.14$\pm$1.59 | 70.43$\pm$2.47 | 72.24$\pm$2.76 | 85.74$\pm$1.59 | 82.23$\pm$1.48
Medformer | 83.50$\pm$2.01 | 85.19$\pm$0.94 | 77.11$\pm$3.39 | 79.18$\pm$3.31 | 92.81$\pm$1.48 | 90.32$\pm$1.54
PTB-XL | No Inter-Attention | 72.51$\pm$0.16 | 63.61$\pm$0.28 | 59.75$\pm$0.30 | 61.25$\pm$0.22 | 89.48$\pm$0.08 | 65.74$\pm$0.26
No Augmentation | 72.68$\pm$0.19 | 63.99$\pm$0.62 | 59.73$\pm$0.41 | 61.26$\pm$0.34 | 89.49$\pm$0.05 | 66.00$\pm$0.22
Single-Channel Patching | 72.79$\pm$0.35 | 64.80$\pm$0.51 | 59.57$\pm$0.44 | 61.43$\pm$0.38 | 88.97$\pm$0.19 | 65.91$\pm$0.34
Medformer | 72.87$\pm$0.23 | 64.14$\pm$0.42 | 60.60$\pm$0.46 | 62.02$\pm$0.37 | 89.66$\pm$0.13 | 66.39$\pm$0.22

###  D.2 Patch Length Study

To investigate the effects of multi-granularity and computational complexity,
we conduct an empirical analysis using various patch lengths on the APAVA
dataset. Table 5 presents the evaluation results for different combinations of
patch lengths. Initially, we compare the performance of models using a single
patch length against models using five identical patch lengths (e.g.,
$\\{8\\}$ vs $\\{8,8,8,8,8\\}$). Our findings indicate that using repetitive
patch lengths generally enhances performance, except when $L=2$, suggesting
that additional identical patch lengths can capture more information,
analogous to multi-head attention mechanisms.

Furthermore, we assess the performance of a manually selected combination of
varying patch lengths, specifically $\\{2,2,4,16,32\\}$. This configuration
achieves the highest performance across all evaluated metrics, underscoring
the effectiveness of our designed attention module in accommodating multi-
granularity patches. However, it is worth noting that mixing different patch
lengths does not guarantee improved performance. See F for more detailed
discussion.

Table 5: Patch Length Study

Datasets | Models | Accuracy | Precision | Recall | F1 score | AUROC | AUPRC
---|---|---|---|---|---|---|---
APAVA | {2} | 71.82 $\pm$8.13 | 73.23 $\pm$8.91 | 69.69 $\pm$6.31 | 69.95 $\pm$7.08 | 69.34 $\pm$4.72 | 69.18 $\pm$5.17
{4} | 75.72 $\pm$3.73 | 78.15 $\pm$5.84 | 72.14 $\pm$3.39 | 72.83 $\pm$3.64 | 72.75 $\pm$4.76 | 73.84 $\pm$5.08
{8} | 71.29 $\pm$3.02 | 72.83 $\pm$2.73 | 67.38 $\pm$4.25 | 67.17 $\pm$4.98 | 76.12 $\pm$4.25 | 76.74 $\pm$4.29
{12} | 69.77 $\pm$4.01 | 69.72 $\pm$5.65 | 67.09 $\pm$3.34 | 67.36 $\pm$3.53 | 75.19 $\pm$3.00 | 75.36 $\pm$3.48
{16} | 70.92 $\pm$1.99 | 71.46 $\pm$3.09 | 67.67 $\pm$2.26 | 67.81 $\pm$2.45 | 76.97 $\pm$2.53 | 77.21 $\pm$2.87
{24} | 71.68 $\pm$2.44 | 74.26 $\pm$3.49 | 67.14 $\pm$2.55 | 67.13 $\pm$2.85 | 79.07 $\pm$3.34 | 78.73 $\pm$3.32
{32} | 72.55 $\pm$1.51 | 75.74 $\pm$1.49 | 68.38 $\pm$2.99 | 68.19 $\pm$3.23 | 79.17 $\pm$2.17 | 78.44 $\pm$2.40
{2,2,2,2,2} | 65.52 $\pm$8.24 | 65.97 $\pm$7.41 | 64.14 $\pm$6.06 | 63.71 $\pm$7.23 | 63.15 $\pm$3.43 | 61.84 $\pm$4.81
{4,4,4,4,4} | 76.91 $\pm$1.72 | 78.66 $\pm$3.20 | 73.71 $\pm$1.26 | 74.46 $\pm$1.42 | 74.90 $\pm$3.21 | 76.36 $\pm$3.12
{8,8,8,8,8} | 71.81 $\pm$3.81 | 74.25 $\pm$6.34 | 67.72 $\pm$3.45 | 67.89 $\pm$3.68 | 74.95 $\pm$5.34 | 75.59 $\pm$5.63
{12,12,12,12,12} | 71.17 $\pm$3.85 | 72.18 $\pm$5.97 | 67.65 $\pm$3.27 | 67.96 $\pm$3.48 | 76.71 $\pm$4.82 | 77.27 $\pm$4.91
{16,16,16,16,16} | 71.13 $\pm$3.33 | 72.14 $\pm$5.69 | 67.82 $\pm$2.45 | 68.13 $\pm$2.58 | 76.34 $\pm$4.52 | 76.39 $\pm$4.94
{24,24,24,24,24} | 73.18 $\pm$2.15 | 75.72 $\pm$3.46 | 68.98 $\pm$2.11 | 69.27 $\pm$2.33 | 81.10 $\pm$2.61 | 81.20 $\pm$2.68
{32,32,32,32,32} | 74.34 $\pm$2.20 | 78.92 $\pm$1.57 | 69.66 $\pm$2.80 | 69.80 $\pm$3.42 | 81.11 $\pm$1.10 | 80.69 $\pm$1.02
{2,2,4,16,32} | 78.21 $\pm$2.60 | 80.82 $\pm$4.30 | 74.92 $\pm$2.07 | 75.78 $\pm$2.31 | 80.73 $\pm$2.34 | 81.38 $\pm$2.38

##  Appendix E Complexity Analysis

Let the number of timestamps $T$, and patch list
$\left\\{L_{1},L_{2},\ldots,L_{n}\right\\}$ be given, where the $i$-th patch
length $L_{i}$ produce $N_{i}=\left\lceil{T/L_{i}}\right\rceil$ number of
patches. During intra-granularity attention, we perform self-attention among
the patch embeddings within the same granularity. The total complexity is
$O\left(\sum_{i=1}^{n}N_{i}^{2}\right)$. During intra-granularity attention,
we perform self-attention among $n$ routers, with a time complexity of
$O(n^{2})$. Therefore, the total time complexity is
$O\left(n^{2}+\sum_{i=1}^{n}N_{i}^{2}\right)$.

One potentially useful patch list is the power series $\\{2^{1},2^{2},\ldots
2^{n}\\}$, where $2^{n}<T$. In this case, the complexity of intra-granularity
attention reduces as follows:

$\displaystyle
O\left(\sum_{i=1}^{n}N_{i}^{2}\right)=O\left(\sum_{i=1}^{n}\left\lceil{\frac{T}{2^{i}}}\right\rceil^{2}\right)\leq
O\left(\sum_{i=1}^{n}\left(\frac{T}{2^{i}}+1\right)^{2}\right)$
$\displaystyle=O\left(\sum_{i=1}^{n}\left(\frac{T^{2}}{2^{2i}}+2\frac{T}{2^{i}}+1\right)\right)=O\left(T^{2}\sum_{i=1}^{n}\frac{1}{2^{2i}}+2T\sum_{i=1}^{n}\frac{1}{2^{i}}+n\right)$
$\displaystyle\leq O\left(\frac{1}{3}T^{2}+2T+\log T\right)=O(T^{2})$

The complexity of inter-granularity attention is $O(n^{2})\leq O(\log^{2}T)$.
Therefore, the total time complexity of the two-stage multi-granularity self-
attention module is $O(T^{2})$, which is the same complexity as the vanilla
transformer. This analysis demonstrates our model’s ability to incorporate
different granularities without significantly increasing computational
overhead.

##  Appendix F Discussion

###  F.1 Comparision with Other Multi-Granularity Methods

MTST [45] and Pathformer [47] differ from our Medformer in three significant
aspects: (1) Patching & Embedding MTST and Pathformer utilize single-channel
patching, presupposing channel independence. In contrast, Medformer employs
multi-channel patching to capture potential channel correlations. (2)
Architecture Structure MTST and Pathformer adopt a parallel multi-branch
architecture, where each branch comprises independent attention modules
tailored to specific granularities. Conversely, Medformer utilizes a shared
attention module across different granularities, thereby significantly
reducing model complexity. (3) Granularity Interactions MTST assimilates
multi-granularity information by concatenating outputs from different
branches, while Pathformer uses adaptive pathways for weighted aggregation of
these outputs without any inter-granularity interactions within the attention
modules. In contrast, Medformer introduces a novel inter-granularity attention
mechanism specifically designed for granularity interaction, thereby
effectively integrating multi-granularity information.

Scaleformer [49] operates as a model-agnostic structural framework that
employs variable downsampling and upsampling rates on embeddings outside of
attention modules. Although it integrates seamlessly with non-patching methods
like Autoformer and FEDformer, its incorporation into patching methods is not
straightforward and may result in sub-optimal patch representations [45].
Consequently, the design objectives of Scaleformer are largely orthogonal to
ours, which concentrate on multi-granularity patching and attention
mechanisms.

###  F.2 Limitations and Future Works

The design of Medformer enables the input of various patch lengths, offering
both benefits and challenges. The ability to choose variable patch lengths
suggests the potential for discovering configurations that outperform those
with uniform lengths, as evidenced by our experiments (See Appendix D.2 and
Appendix C). Nonetheless, not all combinations of patch lengths yield optimal
results; some configurations may perform worse than uniform patch lengths.
Consequently, this flexibility necessitates meticulous tuning of patch lengths
as hyperparameters. Future research could explore the development of
mechanisms that automatically select the most effective patch lengths,
optimizing for the most relevant granularities. Additionally, investigating
the performance of large medical time series models across different datasets
is also an interesting direction.

##  Appendix G Broader Impacts

Our proposed model demonstrates performance comparable to or surpassing state-
of-the-art baselines on medical time series classification tasks. The model’s
design, which includes specialized patching and self-attention mechanisms,
specifically targets channel correlations and multi-granularity information.
We anticipate our findings will encourage further research into effective
strategies for capturing multi-scale information in medical time series data.
Additionally, this work could broaden interest in medical time series
classification, an area that remains less explored compared to time series
forecasting.

Besides, different experiment setups based on medical perspectives, such as
subject-dependent and subject-independent, are evaluated to simulate real-
world applications. On a societal level, our model has potential applications
in healthcare, such as facilitating the diagnosis of diseases using medical
time series data. For instance, it could be employed to detect neurological
disorders through EEG data. However, practitioners should be cognizant of the
model’s limitations.