# Braced Fourier Continuation and Regression for Anomaly Detection

Josef Sabuda

###### Abstract

In this work, the concept of Braced Fourier Continuation and Regression (BFCR)
is introduced. BFCR is a novel and computationally efficient means of finding
nonlinear regressions or trend lines in arbitrary one-dimensional data sets.
The Braced Fourier Continuation (BFC) and BFCR algorithms are first outlined,
followed by a discussion of the properties of BFCR as well as demonstrations
of how BFCR trend lines may be used effectively for anomaly detection both
within and at the edges of arbitrary one-dimensional data sets. Finally,
potential issues which may arise while using BFCR for anomaly detection as
well as possible mitigation techniques are outlined and discussed. All source
code and example data sets are either referenced or available via GitHub, and
all associated code is written entirely in Python.

Keywords: Regression, Trend Finding Algorithm, Fourier Continuation, Fourier
Analysis, FFT, Anomaly Detection, Outlier Detection

##  1 Introduction

Braced Fourier Continuation for Regression (BFCR) is a novel and
computationally efficient means of finding nonlinear regressions or trend
lines in arbitrary one-dimensional data sets. The main idea behind BFCR is as
follows: If one can take a given data set and obtain an accurate Fourier
representation of it via the FFT, then one can use a suitable low-pass filter
to remove the high-frequency components of the data set while preserving the
overall trend with an IFFT, thereby creating a trend line or nonlinear
regression. However, since general data sets will almost never be periodic and
hence will almost always suffer from Gibbs phenomena in their Fourier
representations, any given data set will first need to be made periodic before
carrying out this process. A modified version of Fourier Continuation -Braced
Fourier Continuation (BFC)- is what accomplishes this task in BFCR and allows
the overall idea to work.

Implementations of the algorithms discussed in this paper, written entirely in
Python, are available via GitHub [5]

Figure 1: Illustration of the BFCR algorithm on an example data set taken from
[6], with select portions magnified.

###  1.1 Background: Fourier Continuation

As a brief summary, Fourier Continuation (FC) [1, 2, 3] is a means of taking
arbitrary, non-periodic data sets and extending or “continuing” them to be
periodic via appending special, dataset-dependant synthetic data. Once a given
non-periodic data set has been continued with FC, one can take an FFT of the
now periodic continued data set and obtain an accurate Fourier representation
of the data without any Gibbs phenomena at the endpoints of the original data
set. Then the added synthetic data can be removed, and one can use the Fourier
representation of the original data set as they wish.

###  1.2 Braced Fourier Continuation

FC, while powerful, has a problem which needs to be addressed prior to its use
for regression that stems from the process by which it creates the synthetic
data it uses to continue the input data. That process is susceptible to
creating continuations whose values grow exceptionally large for data sets
which are sufficiently “noisy” or non-smooth around the endpoints, an example
of which is shown in Figure 2 below.

Figure 2: Illustration of FC on a non-smooth data set taken from [4]. Figure
2a depicts the input data, while Figure 2b illustrates the output of the FC
process on this data, when FC hyper-parameter $d=12$.

This “explosion” of the FC synthetic data is a significant issue, as if it
dominates in the combined data set, it will cause the contributions of the
original dataset in the FFT output to be pushed to comparatively higher
frequencies, i.e., to become the noise which the algorithm seeks to eliminate.

To solve this problem in BFCR, FC is applied not to the original data set but
to precomputed and appropriately scaled “bracing” data which is known to have
a smooth and bounded continuation, hence the term “Braced Fourier
Continuation”. It is this continued bracing data which is then appended to the
original data set in step 1 of the BFCR algorithm. An important consequence
and benefit of this approach is that the synthetic data used in BFCR can be
precalculated once and then scaled based on two, dataset-specific scalars;
much like how the matrices used in FC process need only be calculated once for
a given choice of hyper-parameters.

The BFC synthetic data creation algorithm is described in more technical
detail below.

###  1.3 The Braced Fourier Continuation Algorithm

Given some input data $X$, the BFC algorithm is as follows:

Figure 3: Illustration of select steps of the BFC algorithm visualized on
example data taken from [4]. Figure 3a depicts the input data, while Figure 3b
depicts the output of steps 2-6. Figure 3c depicts steps 7-9, and Figure 3d
depicts the end result of the algorithm.

  1. 1.

Select the hyper-parameters to be used in the FC process, including $d$, which
will determine the number of bracing data points needed.

  2. 2.

Take the last four points of the input data $X$,
$\\{x_{N-3},x_{N-2},x_{N-1},x_{N}\\}$, and enumerate them as pairs:
$\\{(0,x_{N-3}),(1,x_{N-2}),(2,x_{N-1}),(3,x_{N})\\}$

  3. 3.

Find the lines of best fit, in the least squares sense, through
$\\{(1,x_{N-2}),$ $(2,x_{N-1}),(3,x_{N})\\}$: $L_{1}=a_{1}*x+b_{1}$ (Right
line of best fit 1 in Figure 3b), and
$\\{(0,x_{N-3}),(1,x_{N-2}),(2,x_{N-1})\\}$: $L_{2}=a_{2}*x+b_{2}$ (Right line
of best fit 2 in Figure 3b)

  4. 4.

Project $L_{1}$ forward by 1, $r_{1}=a_{1}*4+b_{1}$, and $L_{2}$ forward by 2,
$r_{2}=a_{2}*4+b_{2}$

  5. 5.

Take the average of $r_{1}$ and $r_{2}$ to arrive at the Right Scaling Point
$RSP$; $RSP=(r_{1}+r_{2})/2$

  6. 6.

Repeat steps 1-4 using the first four points of the input data
$\\{x_{1},x_{2},x_{3},x_{4}\\}$ enumerated as pairs as follows:
$\\{(0,x_{4}),(1,x_{3}),(2,x_{2}),(3,x_{1})\\}$, labelling the resulting Left
Scaling Point $LSP$

  7. 7.

Take the first $d$ points of your selected bracing data ($\\{S_{1}\\}$) and
scale them such that the last point of ($\\{S_{1}\\}$) equals $LSP$

  8. 8.

Take the last $d$ points of your selected bracing data ($\\{S_{2}\\}$) and
scale them such that the first point of ($\\{S_{2}\\}$) equals $RSP$

  9. 9.

Append $\\{S_{1}\\}$ and $\\{S_{2}\\}$ to $X$ such that
$X_{ext}=\\{S_{1},X,S_{2}\\}$, as shown Figure 3c

  10. 10.

Apply the FC process to $X_{ext}$ to reach $X_{cont}$, as shown in Figure 3d

In practice, this algorithm can be simplified in that the FC process can
actually be run on the selected bracing first, then the results can scaled by
the multipliers found in steps 4 and 5. This means that once bracing data has
been selected and the FC process has been run on it, the FC process does not
need to be run again unless and until one decides to use different bracing
data.

Note that in all of the examples shown in this paper, the choice of hyper-
parameters used in the FC process are as follows: $d=Z=12$, $C=27$, $E=0$, and
$N_{over}=20$.

##  2 The BFCR Algorithm

Given an input data set $X$ with $N$ points, $X=\\{x_{i},i=1,2,3,...,N\\}$,
the BFCR algorithm is as follows:

Figure 4: Illustration of select steps of the BFCR Algorithm on example data
taken from [6]. Figure 4a depicts step 1 of the algorithm on the example data.
Figure 4b depicts steps 3 and part of step 4, namely the FFT modes of the
continued data set with zero mean $Y$ both with and without a Sigma
Approximation filter. Figure 4c depicts the reconstructed trend from second
part of step 4. Figure 4d depicts the end result of the BFCR algorithm on the
input data set.

  1. 1.

Extend $X$ to be periodic via BFC and obtain the continued data set
$X_{cont}=\\{x_{cont,i},i=1,2,3,...,N+C\\}$ with $N+C$ points. An example of
this is shown in the top left panel of Figure 5

  2. 2.

Calculate the mean of the $X_{cont}$, $\mu$, and subtract it from each point
in $X_{cont}$, so that new data set $Y$ has zero mean; $Y=\\{y_{i}\\}$, where
$y_{i}=x_{cont,i}-\mu,\forall i\in[1,2,3,...,N+C]$

  3. 3.

Take the FFT of $Y$

  4. 4.

Use the FFT modes of $Y$ with a low-pass filter based on Sigma Approximation
in an IFFT to reconstruct the data as
$Y^{{}^{\prime}}=\\{{y^{{}^{\prime}}}_{i},i=1,2,3,...,N+C\\}$. A comparison of
the FFT modes from the example data, with and without the Sigma Approximation
filter, is shown in the top right panel of Figure 5, and the resulting
reconstruction is shown in the bottom left panel.

  5. 5.

Remove the points associated with the synthetic data from the reconstruction;
$Y^{{}^{\prime}}=\\{{y^{{}^{\prime}}}_{i},i=1,2,3,...,N\\}$

  6. 6.

Add back the mean subtracted in step 2 to each point in the reconstructed
dataset $Y^{{}^{\prime}}$ to arrive at $X^{{}^{\prime}}$;
$X^{{}^{\prime}}=\\{{x^{{}^{\prime}}}_{i}\\}$, where
${x^{{}^{\prime}}}_{i}=({y^{{}^{\prime}}}_{i}+\mu),\forall i\in[1,2,3,...,N]$

  7. 7.

Calculate the average difference between the reconstruction and the original
dataset, $\mu^{{}^{\prime}}=(\sum_{i=1}^{N}({x^{{}^{\prime}}}_{i}-x_{i}))/N$,
and subtract it from the reconstruction to arrive at the final trend line
$X_{trend}=\\{x_{trend,i}\\}$, where
$x_{trend,i}={x^{{}^{\prime}}}_{i}-\mu^{{}^{\prime}},\forall
i\in[1,2,3,...,N]$. The final result of this step, and the algorithm overall
on the example data, is shown in the bottom right panel of Figure 5.

The choice of low-pass filter used in step 3 can actually be varied and is up
to the user, but each of the examples presented in this paper as well as in
the associated source code utilize a filter based on Sigma Approximation with
each Lanczos $\sigma$ factor raised to the $4^{th}$ power. Sigma approximation
is an advantageous choice of low-pass filter in BFCR as, by design, it will
greatly reduce any Gibbs phenomena arising from any internal jump
discontinuities which may exist in a given data set.

##  3 Properties of BFCR

BFCR has a number of important properties that make it stand out from other
regression models/methodologies, including:

  1. 1.

BFCR can be used on general, one-dimensional data sets regardless of data
volatility or behavior so long as they contain at least 4 data points. Note:
It is possible to generalize BFCR to general, N-dimensional data sets, but
this is beyond the scope of this paper. Additionally, only examples of BFCR
used on real datasets will be shown in this paper.

  2. 2.

BFCR requires no assumptions about the structure of the underlying trend
within the data (e.g., linear, quadratic, etc.) in order to create a
regression.

  3. 3.

Assuming all bracing and FC data has been precomputed, BFCR has a
computational complexity of $O((N+C)*log(N+C))$, where $N$ is the number of
points in the input data set, and $C$ is the number of points added through
the BFC process.

##  4 Anomaly Detection with BFCR

BFCR can be used effectively for anomaly/outlier detection in general, one-
dimensional data sets. Generally speaking, the larger the input data set, the
more effective the below algorithms using BFCR will be, with an important
caveat as well as other notable potential issues and associated mitigation
techniques discussed in the next section. Additionally, while the BFCR
algorithm can be used on data sets with as few as 4 points, the author
recommends when using BFCR for anomaly detection that data sets have at least
6 points of data.

Due to the nature of the BFCR algorithm, there are two cases to consider when
using BFCR for anomaly detection which are discussed below. Both cases,
however, share the same fundamental idea of taking a sample (from a single
data point) and comparing it to a population (derived from the rest of the
data points) via population statistics with an assumed statistical
distribution.

###  4.1 Anomaly Detection Away from Edges

Non-edge anomaly detection with BFCR (i.e., for data points which are neither
the first nor last points in a given data set) is straightforward, and the
algorithm is as follows:

Figure 5: Illustration of the BFCR internal anomaly detection algorithm on
some example data taken from [4]. Figure 5a depicts the sample data set.
Figure 5b shows the results of step 1 of the algorithm, Figure 5c shows the
process and result of step 2 of the algorithm, and Figure 5d shows the end
result of the algorithm when assuming a normal distribution in step 3.

  1. 1.

Given an input data set $X$ with $N$ points, calculate a trend line $Y$
through the input data set using BFCR.

  2. 2.

Find the mean and standard deviation of absolute difference between each point
in the data set and its corresponding point in the trend;

$\mu=\dfrac{\sum_{i=1}^{N}|x_{i}-y_{i}|}{N}$,
$\sigma=\sqrt{\dfrac{\sum_{i=1}^{N}(|x_{i}-y_{i}|-\mu)^{2}}{N}}$

These values, $\mu$ and $\sigma$, are the population statistics.

  3. 3.

Assume a statistical distribution for the population statistics, and see if
any internal points (i.e, the samples) are outliers according to the
properties of that distribution. For example, if one assumes the absolute
differences will be normally distributed, then one would loop through every
internal data point of interest and see if any lie more than two standard
deviations away from the mean absolute difference. If any do, those points
would be flagged as anomalies or outliers. All of the BFCR anomaly detection
examples shown in this paper assume that the population statistics derived
from BFCR trend lines follow normal distributions.

###  4.2 Edge Anomaly Detection

While similar, anomaly detection with BFCR for the edges of a data set differs
from internal anomaly detection in important ways. The algorithm for edge
anomaly detection presented below assumes one is trying to determine whether
the very last point in a given data set is anomalous/an outlier.

Figure 6: Illustration of the BFCR edge anomaly detection algorithm on some
example data taken from [4]. Figure 6a depicts the sample data set. Figure 6b
shows the results of step 1 of the algorithm, Figure 6c shows the process and
result of steps 2 and 3 of the algorithm, and Figure 6d shows the end result
of the algorithm when assuming a normal distribution in step 4.

  1. 1.

Given an input data set $X$ with $N$ points, calculate two BFCR trend lines,
$Y_{1}$ and $Y_{2}$; $Y_{1}$ is calculated using all but the very last data
point in the input dataset, and $Y_{2}$ by using the entire input dataset.

  2. 2.

Using $Y_{1}$, find the mean and standard deviation of the absolute
differences between $Y_{1}$ and all but the very last point in the input data;

$\mu=\dfrac{\sum_{i=1}^{N-1}|x_{i}-y_{1,i}|}{N-1}$,
$\sigma=\sqrt{\dfrac{\sum_{i=1}^{N-1}(|x_{i}-y_{1,i}|-\mu)^{2}}{N-1}}$

These values, $\mu$ and $\sigma$, are the population statistics.

  3. 3.

Using $Y_{2}$, measure the absolute difference between the last point in
$Y_{2}$ with the last point of the input data;

$s=|y_{2,N}-x_{N}|$

This measurement, $s$, is the sample.

  4. 4.

Assume a statistical distribution for the population statistics, and see if
the sample is an outlier according to the properties of that distribution. For
example, if one assumes the absolute differences will be normally distributed,
then one would check to see if the sample lies more than two standard
deviations away from the mean absolute difference. If it does, then that point
would be flagged as an anomaly or outlier. Again, all of the BFCR anomaly
detection examples shown in this paper assume that the population statistics
from BFCR trend lines follow normal distributions.

To check the first point in a given data set, simply reverse sort the data set
and then use the same process presented above.

##  5 Anomaly Detection with BFCR; Potential Issues and Mitigation Techniques

###  5.1 Data Sets with Varying Volatility

Figure 7: Illustration of volatility regime change and the effect it has on
internal anomaly detection on example data taken from [6]. Figure 7a depicts
BFCR internal anomaly detection algorithm results on a sample data set which
contains significant amounts of data from two distinct volatility regimes
(before and after the large spike). Figure 7b shows the results the algorithm
on a portion of the same data set but with most of the data from the first
volatility regime excluded.

BFCR for anomaly detection, like any trend-based anomaly detection method, is
vulnerable to false positives/negatives when the underlying volatility of the
data, what will hereto be referred to as the volatility regime, changes within
the data set. Figure 7 above depicts this situation; when significant amounts
of data from the previous volatility regime (before the spike) are included
when analyzing points in the data for internal anomalies (Figure 7a), more
points are flagged as anomalies than if most of the data from the old regime
is excluded (Figure 7b). Note: If the volatility regime change was reversed,
i.e., if the data went from being more volatile to significantly less
volatile, fewer points would have been identified as anomalous if significant
data from the more volatile regime was included.

This issue of volatility regime change within a data set can be handled in
various ways, with one potential method being the following which is simple
and efficient using BFCR:

  1. 1.

Given an input data set $X$ with $N$ points, use BFCR to draw a trend $Y$
through the data.

  2. 2.

Measure the absolute differences $D$ between each point in the input data and
the trend;

$D=\\{|y_{i}-x_{i}|,i=1,2,3,...,N\\}$

  3. 3.

Cut $D$ into equal (or nearly equal) halves and calculate the standard
deviation of each half ($\sigma_{1},\sigma_{2})$.

  4. 4.

Divide $\sigma_{1}$ by $\sigma_{2}$ (or vice versa) and see if the result
falls outside some specified percentage range (e.g., 25%).

$0.75\leq\dfrac{\sigma_{1}}{\sigma_{2}}\leq 1.25$

If it does, remove the first some percentage (e.g., 20%) of the input data and
repeat the process over again until either the resulting data halves have
similar volatility, or only some specified cut-off percentage (e.g., 50%) of
the input data remains.

Once this process is run and terminates, assuming the input data set isn’t
very long and does not contain more than two volatility regimes, the (likely
truncated) data set will now likely be of mostly a single volatility regime.

###  5.2 Data Sets with Pre-Existing Outliers

Figure 8: Illustration of the potential effect of pre-existing internal
outliers while using BFCR for edge anomaly detection on example data taken
from [4]. Figures 8a and 8b depict the results of the unmodified BFCR for edge
anomaly detection algorithm on an example data set which contains a
significant internal outlier. Figures 8c and 8d show the results of the
modified algorithm which first finds and filters out internal anomalies on the
same data set.

When using BFCR for edge anomaly detection, it is possible for data sets to
contain internal outliers which, if ignored, would hamper the ability of the
algorithm to detect anomalous edge data points whose distances from the trend
line are still extreme compared to most points in the data set, but not as
extreme as the internal outlier points. This situation is depicted in Figure 8
above, and can be mitigated by screening for and removing the influence of
these internal outliers in the population statistics. This can be done by
inserting an additional step in the BFCR edge anomaly detection algorithm
after step 2; namely step 3 of the BFCR internal anomaly detection algorithm.
In other words, after steps 1 and 2 of the edge anomaly detection algorithm,
use the already calculated population statistics from step 2 to see if any
internal points are outliers according to the assumed statistical distribution
the population statistics describe (e.g., a normal distribution). If any do,
recalculate the population mean and standard deviation while neglecting the
contributions of those internal outliers before moving on to step 3 and
proceeding as normal.

###  5.3 Data Sets with Little to No Noise

While the BFCR and BFCR edge anomaly detection algorithms can be used on any
one-dimensional data set, the algorithms work best on data sets in which there
exists a non-trivial amount of noise. If there is little to no noise present
within a given input data set, then the resulting BFCR regression(s) will
likely deviate from the data mostly around the end-points, hence the edge
anomaly detection algorithm will be prone to false positives. This phenomena
is demonstrated in the plots in Figure 9 below. The issue stems from the
likely mismatch between one’s selected bracing data and the underlying trend
in the input data around the end points (especially if one is using
precomputed bracing data). It is this mismatch, combined with the lack of
noise, that causes whatever low-pass filter being used in the algorithms to
filter out FFT modes which correspond to the underlying trend at the end
points of the input data set, thereby causing the calculated trend to deviate
from the underlying data around those end points.

For edge anomaly detection this potential issue can, in practice, largely be
mitigated via using either one or both of the following pre-processing methods
to the input data before applying the BFCR edge anomaly detection algorithm:

  1. 1.

Specify a minimum percent change threshold for the change between the last two
points in any given data set (e.g., $x_{N}/x_{N-1}\geq 10\%$) that must be met
in order for the algorithm to be run. This will generally work because, in
practice, many if not most real world data sets do not vary exponentially over
time (where this mitigation technique would not necessarily work).

  2. 2.

Calculate the coefficient of variation of the differences between the last $m$
(e.g., 4) points in the input data (e.g.,
$\\{(x_{N}-x_{N-1}),(x_{N-1}-x_{N-2}),(x_{N-2}-x_{N-3}),...,(x_{N-m+2}-x_{N-m+1})\\}$).
If this coefficient is less than some chosen threshold (e.g., 0.2), then the
edge of the data set has low volatility and likely does not contain an
outlier, hence the edge anomaly detection algorithm is unnecessary.

Figure 9: Illustration of false positives when the BFCR edge anomaly detection
algorithm is run on data sets with no noise. Figures 9a and 9b depict the
algorithm being run on linear data of the form $y=x$. Figures 9c and 9d depict
the algorithm being run on quadratic data of the form $y=x^{2}$. Figures 9e
and 9f depict the algorithm being run on exponential data of the form
$y=e^{x}$.

##  6 Conclusion

This paper introduced the Braced Fourier Continuation and Regression (BFCR)
algorithm, a novel and computationally efficient means of finding nonlinear
regressions or trend lines in arbitrary, one-dimensional data sets.
Additionally, the use of BFCR for efficient and flexible outlier detection,
both within and at the edges of general data sets, was also introduced.
Possible future work includes extension of the BFCR algorithm to higher
dimensional data sets, modification of the algorithm to handle data sets with
non-uniform spacing between points, as well as improving the computational
complexity of the algorithm to $O(N+C)$ from $O((N+C)*log(N+C))$, where $N$ is
the number of points in the input data set and $C$ is the number of points
added through the BFC process.

## References

  * [1] Nathan Albin and Oscar Bruno  “A spectral FC solver for the compressible Navier–Stokes equations in general domains I: Explicit time-stepping”  In _Journal of Computational Physics_ 230.16 Elsevier BV, 2011, pp. 6248–6270  DOI: 10.1016/j.jcp.2011.04.023
  * [2] Faisal Amlani and Oscar P. Bruno  “An FC-based spectral solver for elastodynamic problems in general three-dimensional domains”  In _Journal of Computational Physics_ 307, 2016, pp. 333–354  DOI: https://doi.org/10.1016/j.jcp.2015.11.060
  * [3] Oscar P. Bruno and Jagabandhu Paul  “Two-Dimensional Fourier Continuation and Applications”  In _SIAM Journal on Scientific Computing_ 44.2, 2022, pp. A964–A992  DOI: 10.1137/20M1373189
  * [4] NIC  “Financial Data Download”  URL: https://www.ffiec.gov/npw/FinancialReport/FinancialDataDownload
  * [5] Josef Sabuda  “Braced-Fourier-Continuation-and-Regression”  In _GitHub repository_ GitHub, 2024  URL: https://github.com/j4sabuda/Braced-Fourier-Continuation-and-Regression
  * [6] Matt Wyrick  “SOMA-Asset-Forecasting”  In _GitHub repository_ GitHub, 2022  URL: https://github.com/mattrwyrick/SOMA-Asset-Forecasting


# Meta-FL: A Novel Meta-Learning Framework for Optimizing Heterogeneous Model
Aggregation in Federated Learning

Zahir Alsulaimawi

###### Abstract

Federated Learning (FL) enables collaborative model training across diverse
entities while safeguarding data privacy. However, FL faces challenges such as
data heterogeneity and model diversity. The Meta-Federated Learning (Meta-FL)
framework has been introduced to tackle these challenges. Meta-FL employs an
optimization-based Meta-Aggregator to navigate the complexities of
heterogeneous model updates. The Meta-Aggregator enhances the global model’s
performance by leveraging meta-features, ensuring a tailored aggregation that
accounts for each local model’s accuracy. Empirical evaluation across four
healthcare-related datasets demonstrates the Meta-FL framework’s adaptability,
efficiency, scalability, and robustness, outperforming conventional FL
approaches. Furthermore, Meta-FL’s remarkable efficiency and scalability are
evident in its achievement of superior accuracy with fewer communication
rounds and its capacity to manage expanding federated networks without
compromising performance.

###### Index Terms:

Federated Learning, Meta-Learning, Model Aggregation, Data Heterogeneity,
Privacy Preservation, Distributed Machine Learning, Healthcare Informatics.

##  I Introduction

Federated Learning (FL) represents a paradigm shift in distributed machine
learning, allowing for collaborative model training across disparate clients
while maintaining data privacy. This evolution addresses critical concerns
around data sovereignty and privacy regulations [1]. Despite its potential,
traditional FL faces data heterogeneity and model diversity challenges among
clients [2]. The assumption of homogeneity in client models falls short in
real-world applications, where data can be non-independently and identically
distributed (non-IID) across different nodes, leading to significant
performance degradation of the aggregated global model.

The intricacy of federated environments is further magnified by the
architectural diversity among client models tailored to specific data
characteristics. This variance necessitates more sophisticated aggregation
mechanisms beyond simple parameter averaging, which traditional approaches
struggle to provide [3]. Additionally, the logistical challenge of efficiently
transmitting model updates, especially in bandwidth-constrained scenarios,
presents a significant bottleneck [4].

To address these multifaceted challenges, we introduce the Meta-Federated
Learning (Meta-FL) framework, an innovative approach that employs an
optimization-based Meta-Aggregator. This Meta-Aggregator harnesses the power
of meta-learning, dynamically adjusting the weighting of client updates
according to their performance and extracted meta-features. Such a methodology
enables the seamless integration of heterogeneous models into a unified, high-
performing global model [5]. Our framework is designed to improve the
convergence and accuracy of the global model amid data and model diversity and
optimize communication efficiency by prioritizing the most impactful updates
for global learning [6].

This paper outlines the Meta-FL framework’s architecture, details the
optimization challenges tackled by the Meta-Aggregator, and provides a robust
theoretical foundation underpinning our approach. Through rigorous
experiments, we demonstrate Meta-FL’s superiority over existing FL
methodologies in handling heterogeneous environments [7, 1], marking a pivotal
advancement in the quest to realize FL’s full potential across diverse and
distributed settings [8].

Meta-FL embodies a new frontier for FL, where diversity among participants
enriches the collective learning journey and overcomes previously
insurmountable barriers to effective, decentralized machine learning. This
work contributes to the ongoing discourse within the FL community and sets the
stage for future innovations in this rapidly evolving field.

##  II Related Work

Substantial research has marked the evolution of FL, aiming to address its
inherent challenges, such as data heterogeneity, privacy preservation, model
aggregation, and the efficient training of models across distributed
environments. This section delves into these challenges, highlighting
significant contributions in each area and discussing how the proposed Meta-FL
framework advances state-of-the-art.

###  II-A Addressing Data Heterogeneity

Data heterogeneity across clients in FL poses significant challenges to model
accuracy and convergence. Zhao et al. [9] first quantified the impact of non-
IID data on model performance, proposing strategies to mitigate its effects.
Li et al. [10] then developed an algorithmic solution to adapt local model
updates, enhancing the robustness of FL against data diversity. More recently,
Hsieh et al. [11] have further explored the implications of data heterogeneity
on the efficiency of FL algorithms, emphasizing the need for adaptable
aggregation strategies. Our Meta-FL framework utilizes an optimization-based
Meta-Aggregator that dynamically adjusts to the characteristics of data
heterogeneity, optimizing aggregation without significant computational
overhead.

###  II-B Enhancing Privacy Preservation

The intersection of FL with privacy-preserving techniques has garnered
significant attention. McMahan et al. [12] laid the groundwork by introducing
FL as a privacy-enhancing technology. Truex et al. [13] explored hybrid
approaches combining FL with secure multi-party computation and differential
privacy. Recent advancements by Geyer et al. [14] have introduced more
efficient mechanisms to integrate differential privacy within the FL paradigm,
reducing the trade-off between privacy and model performance. The Meta-FL
framework incorporates privacy considerations directly into the meta-
aggregation process, offering a balanced and efficient approach to privacy
preservation.

###  II-C Optimization and Aggregation Strategies

The efficiency of FL heavily relies on its ability to aggregate local updates
into a coherent global model. Wang et al. [15] optimized the weighting of
local updates based on relevance. Following this, Karimireddy et al. [16]
introduced SCAFFOLD, an approach that corrects for the client drift in FL,
significantly improving convergence. Building upon these insights, the Meta-FL
framework introduces a novel meta-learning-based aggregation strategy that
surpasses existing convergence speed and accuracy methods.

###  II-D Transitioning Towards Meta-Federated Learning

The Meta-FL framework synthesizes these advancements into a cohesive approach
that addresses key challenges in FL. By leveraging meta-learning for dynamic
aggregation, Meta-FL improves upon traditional FL’s limitations in handling
data heterogeneity, ensures privacy, and optimizes the aggregation process to
enhance the global model’s performance. For instance, Fallah et al. [17] have
explored the potential of meta-learning to personalize models in a federated
setting, which aligns with the adaptive aggregation strategies Meta-FL
employs. Similarly, the work by Chen et al. [18] on FedMeta introduces meta-
learning techniques to address the challenge of non-IID data, showcasing the
effectiveness of such approaches. These developments underscore Meta-FL’s
innovative use of meta-learning to personalize learning and dynamically adjust
aggregation strategies, thereby addressing some of the most pressing issues in
FL. This framework represents a significant step forward, promising to reshape
the future of distributed machine learning by offering a scalable, efficient,
and privacy-preserving solution. In line with the discussions on advancements
in privacy and efficiency in FL, Park et al. [19] and Smith et al. [20]
provide insights into the latest methodologies for enhancing privacy and
optimizing computational resources in federated settings, further validating
the approaches integrated within Meta-FL.

In conclusion, while existing research has laid a strong foundation for FL,
the Meta-FL framework advances the field by providing a comprehensive solution
to its most pressing challenges. Through innovative use of meta-learning and
optimization, Meta-FL sets a new benchmark for efficiency, privacy, and
performance in FL systems.

##  III Preliminaries

###  III-A Federated Learning

FL is a machine learning setting where many clients (e.g., mobile devices or
whole organizations) collaboratively train a model under the coordination of a
central server (or service) while keeping the training data decentralized.
Formally, let $\mathcal{K}$ be the set of client indices, where
$K=|\mathcal{K}|$ is the number of clients. Each client $k\in\mathcal{K}$
possesses a local dataset $D_{k}$ of size $n_{k}$. The goal of FL is to train
a global model characterized by parameters $\theta$ on the union of the
clients’ data, i.e., $D=\bigcup_{k=1}^{K}D_{k}$, without exchanging data
samples. The objective function for FL can be written as:

$\min_{\theta}f(\theta)=\sum_{k=1}^{K}\frac{n_{k}}{n}F_{k}(\theta)$ (1)

where $n=\sum_{k=1}^{K}n_{k}$ is the total number of samples across all
clients, and $F_{k}(\theta)$ is the local objective function for client $k$,
often taken to be the empirical risk on $D_{k}$.

###  III-B Optimization in FL

The optimization challenge in FL is to find the global model parameters
$\theta$ that minimize the global loss function $f(\theta)$. Due to the
potentially non-IID nature of data across clients, direct optimization of
$f(\theta)$ is non-trivial. The FedAvg algorithm, introduced by [12], is a
widely adopted approach where each client computes an update to the model
based on its local data, which are averaged to update the global model.

###  III-C Meta-Learning

Meta-learning, or learning to learn, refers to the paradigm where a model is
trained to adapt to new tasks quickly. In the context of FL, meta-learning can
be used to learn the optimal way to aggregate model updates from various
clients. Let $\mathcal{M}$ be the meta-model with meta-parameters $\phi$,
which parameterizes the aggregation of client updates. The meta-learning
objective is:

$\min_{\phi}\mathcal{L}(\phi)=\sum_{k=1}^{K}\ell\big{(}\mathcal{M}(\phi;\theta_{k}),\theta^{*}\big{)}$
(2)

where $\ell$ is a loss function that measures the discrepancy between the
meta-aggregated parameters $\mathcal{M}(\phi;\theta_{k})$ and the optimal
global model parameters $\theta^{*}$.

###  III-D Optimization-Based Meta-Aggregation

In the Meta-FL framework, the Meta-Aggregator aims to learn an aggregation
function that effectively combines the updates $\theta_{k}$ from each client.
This aggregation is formulated as an optimization problem where the objective
is to maximize the performance of the aggregated model, possibly subject to
regularization or constraints that encode prior knowledge about the problem
domain.

Given the meta-features $F_{k}$ and performance metrics $P_{k}$ of the updates
from each client, the optimization problem for the Meta-Aggregator can be
expressed as:

$\min_{\phi}\mathcal{L}(\phi)+\lambda R(\phi)$ (3)

where $\mathcal{L}(\phi)$ is the meta-learning objective defined above,
$R(\phi)$ is a regularization term, and $\lambda$ is a regularization
coefficient.

The solution to this optimization problem provides the weights for aggregating
the local updates into the global model. It can be solved using techniques
such as gradient descent, evolutionary algorithms, or other heuristic methods.

##  IV Proposed Approach

The Meta-FL framework introduces a novel paradigm in FL by leveraging an
optimization-based Meta-Aggregator to dynamically integrate heterogeneous
client updates, using meta-features to guide the aggregation process. This
approach aims to enhance the global model’s performance by accounting for the
diversity in data distributions and model architectures across clients.

####  IV-1 System Model and Notations

Consider a FL system with $K$ clients, each possessing a distinct dataset
$D_{k}$ and a local model $M_{k}$ parameterized by $\theta_{k}$. The
collective goal is to construct a global model $M_{g}$ with parameters
$\theta_{g}$, which effectively aggregates the insights from all clients. Let
$F_{k}(\theta_{k})$ represent the loss function for client $k$, reflecting the
performance of $M_{k}$ on $D_{k}$.

####  IV-2 Meta-Aggregator Formulation

The Meta-Aggregator synthesizes $\theta_{g}$ by solving an optimization
problem that strategically weights the contributions of each client based on
performance metrics $P_{k}$ and meta-features $X_{k}$. Meta-features may
include data characteristics, model complexity, or learning dynamics. The
objective is formulated as follows:

$\min_{\theta_{g}}\mathcal{L}(\theta_{g})=\sum_{k=1}^{K}w_{k}\cdot
F_{k}(\theta_{k}),$ (4)

where $w_{k}$ denotes the weight assigned to the $k$-th client’s update, and
$\mathcal{L}(\theta_{g})$ represents the global loss function. The weights
$w_{k}$ are derived by solving an optimization problem that aims to minimize
the global loss, subject to constraints that ensure fairness and efficiency in
aggregation. This problem is given by:

$\min_{w_{k}}\Phi(w_{k},X_{k},P_{k})\quad\text{s.t.}\quad\sum_{k=1}^{K}w_{k}=1,\quad
w_{k}\geq 0,\forall k,$ (5)

where $\Phi$ encapsulates the Meta-Aggregator’s strategy for weighting
updates, incorporating each client’s meta-features $X_{k}$ and performance
metrics $P_{k}$.

####  IV-3 Optimization Strategy

The Meta-Aggregator’s optimization problem can be solved using gradient
descent or other suitable optimization techniques, depending on the complexity
of $\Phi$. The iterative update rule for $w_{k}$ is as follows:

$w_{k}^{(t+1)}=w_{k}^{(t)}-\eta\frac{\partial\Phi}{\partial w_{k}},$ (6)

where $\eta$ represents the learning rate. This process iteratively refines
the weights $w_{k}$, thereby optimizing the global model $\theta_{g}$ over
successive FL rounds.

####  IV-4 Meta-Feature Extraction and Utilization

Meta-features $X_{k}$ play a critical role in informing the Meta-Aggregator’s
decisions. Extracting meaningful meta-features requires analyzing the local
datasets $D_{k}$ and the models $M_{k}$. Potential meta-features include
dataset size, model accuracy, data distribution metrics, or learning rates.
The choice of meta-features is pivotal to the Meta-FL framework’s ability to
integrate diverse client updates adaptively.

##  V Derivation of the Loss Function

A tailored loss function is essential for optimizing the Meta-FL framework. It
guides the learning process and ensures effective aggregation of diverse
client updates. The proposed loss function encapsulates client models’
individual performances and integrates meta-features to adjust their
contributions to the global model dynamically. Here, we derive the loss
function utilized in the Meta-FL approach.

Given a federated network of $K$ clients, the goal is to minimize the global
loss, a function of each client’s local losses and meta-features. Formally,
the global loss function, $\mathcal{L}_{global}$, is defined as:

$\mathcal{L}_{global}(\theta_{g})=\sum_{k=1}^{K}w_{k}\cdot\mathcal{L}_{k}(\theta_{k})+\lambda\cdot
R(\theta_{g}),$ (7)

where $\mathcal{L}_{k}(\theta_{k})$ is the loss function for the $k$-th client
model parameterized by $\theta_{k}$, and $R(\theta_{g})$ is a regularization
term for the global model parameters $\theta_{g}$. The weights $w_{k}$ are
derived from the Meta-Aggregator, based on performance metrics $P_{k}$ and
meta-features $X_{k}$ associated with each client, and $\lambda$ is a
regularization coefficient.

The weights $w_{k}$ for each client are computed by solving an optimization
problem that aims to balance the contributions of each client based on their
relevance and performance. Specifically, the weights are obtained as:

$w_{k}=\frac{e^{-\alpha\cdot\mathcal{E}_{k}}}{\sum_{j=1}^{K}e^{-\alpha\cdot\mathcal{E}_{j}}},$
(8)

where $\mathcal{E}_{k}$ is a composite error metric for the $k$-th client,
incorporating both the loss $\mathcal{L}_{k}(\theta_{k})$ and relevant meta-
features $X_{k}$, and $\alpha$ is a hyperparameter that controls the
sensitivity of weight distribution to the error metrics.

The regularization term $R(\theta_{g})$ is introduced to prevent overfitting
and promote the generalization of the global model. It can be formulated as:

$R(\theta_{g})=\|\theta_{g}\|_{2}^{2},$ (9)

which represents the $L_{2}$ norm of the global model parameters, encouraging
the model to maintain small weights.

The final optimization objective for the Meta-FL framework is to minimize the
global loss function $\mathcal{L}_{global}(\theta_{g})$ concerning the global
model parameters $\theta_{g}$. This objective encapsulates the dual goals of
accurately aggregating local models and ensuring the generalizability of the
global model.

The derived loss function provides a comprehensive mechanism for integrating
diverse client updates in the Meta-FL framework, enabling the construction of
a robust and performant global model. Including meta-features in the weighting
scheme further enhances the adaptability of the aggregation process, ensuring
that the global model benefits from the unique strengths of each client’s data
and model.

##  VI Meta-Federated Learning Algorithm

We present the Meta-FL aggregation algorithm, which employs an optimization-
based mechanism to synthesize diverse client updates into an enhanced global
model through meta-learning.

Algorithm 1 Meta-FL Aggregation

1:$K$: Total number of client

2:$\\{M_{k}\\}_{k=1}^{K}$: Set of local models from client

3:$\\{D_{k}^{train},D_{k}^{val}\\}_{k=1}^{K}$: Local dataset

4:$T$: Number of FL round

5:$\theta$: Initial Meta-Aggregator parameter

6:$\theta^{*}$: Optimized global model parameters

7:for $t=1$ to $T$ do

8: for $k=1$ to $K$ do $\triangleright$ In parallel

9: Train $M_{k}$ on $D_{k}^{train}$ to get $\theta_{k}$

10: Evaluate $M_{k}$ on $D_{k}^{val}$ for $P_{k}$

11: Extract meta-features $F_{k}$ from $M_{k}$

12: Send $\theta_{k},P_{k},F_{k}$ to Meta-Aggregator

13: end for

14:
$\theta\leftarrow\text{MetaAgg}(\\{\theta_{k},P_{k},F_{k}\\}_{k=1}^{K};\theta)$

15: Update Meta-Aggregator using the global validation set

16: Distribute $\theta$ to clients for next round

17:end for

18:$\theta^{*}\leftarrow\theta$

###  VI-A Meta-Aggregator Aggregation Function

The MetaAgg function is an optimization algorithm that solves for the optimal
weights to be used in aggregating the client model updates. It performs the
following steps:

  1. 1.

Formulate an optimization problem where the objective is to maximize the
performance of the global model on the validation set.

  2. 2.

Define the decision variables as the weights $w_{k}$ for each client’s model
update.

  3. 3.

Incorporate constraints to ensure the weights are non-negative and sum up to
one.

  4. 4.

Use performance metrics $P_{k}$ and meta-features $F_{k}$ to define the
objective function.

  5. 5.

Solve the optimization problem using an appropriate solver to obtain the
optimal weights $w_{k}^{*}$.

  6. 6.

Aggregate the client updates by computing a weighted sum:
$\theta=\sum_{k=1}^{K}w_{k}^{*}\cdot\theta_{k}$

##  VII Theoretical Analysis

###  VII-A Convergence of Meta-Aggregator

The convergence of the Meta-Aggregator is crucial to ensure that the iterative
process leads to a stable global model that accurately reflects the aggregated
knowledge of all clients.

######  Lemma 1 (Weight Convergence).

Given a sequence of meta-features $\\{F_{k}^{t}\\}_{t=1}^{\infty}$ and
corresponding performance metrics $\\{P_{k}^{t}\\}_{t=1}^{\infty}$ for each
client $k$, the weights $w_{k}(\phi^{t})$ generated by the Meta-Aggregator
converge to a fixed point as $t\to\infty$, assuming $F_{k}^{t}$ and
$P_{k}^{t}$ satisfy certain regularity conditions.

###### Proof.

Consider the weight update mechanism defined by the Meta-Aggregator as a
mapping $\Phi:\mathbb{R}^{K}\to\mathbb{R}^{K}$, where each component
$\Phi_{k}$ of $\Phi$ is defined by the weight update rule for client $k$. Let
$w^{t}=(w_{1}^{t},w_{2}^{t},\ldots,w_{K}^{t})$ be the vector of weights at
iteration $t$.

Assuming that the sequences $\\{F_{k}^{t}\\}$ and $\\{P_{k}^{t}\\}$ satisfy
boundedness and Lipschitz continuity, we assert that the mapping $\Phi$
induced by the weight update rule is a contraction concerning a suitable
metric on $\mathbb{R}^{K}$, say the Euclidean metric. Specifically, there
exists a constant $L<1$ such that for any two vectors of weights
$w,w^{\prime}\in\mathbb{R}^{K}$,

$\|\Phi(w)-\Phi(w^{\prime})\|\leq L\|w-w^{\prime}\|.$ (10)

To show this, we leverage the properties of the exponential function used in
the weight update rule, which, combined with the Lipschitz continuity of
$F_{k}^{t}$ and $P_{k}^{t}$, ensures that the updates are bounded and changes
in $w_{k}^{t}$ over successive iterations are contractive.

Given that $\Phi$ is a contraction, by Banach’s Fixed Point Theorem, there
exists a unique fixed point $w^{*}\in\mathbb{R}^{K}$ such that
$\Phi(w^{*})=w^{*}$, and for any initial vector of weights $w^{0}$, the
sequence $\\{w^{t}\\}$ generated by iteratively applying $\Phi$ converges to
$w^{*}$, i.e.,

$\lim_{t\to\infty}w^{t}=w^{*}.$ (11)

Thus, the weights $w_{k}(\phi^{t})$ generated by the Meta-Aggregator converge
to a fixed point as $t\to\infty$, ensuring the stability and convergence of
the Meta-Aggregator’s iterative process. ∎

######  Theorem 2 (Convergence to Optimal Aggregation).

If the weights $w_{k}(\phi^{t})$ converge as stated in Lemma 1, and the global
loss function $\mathcal{L}(\theta)$ is convex concerning $\theta$, then the
Meta-Aggregator converges to an optimal aggregation of client models,
minimizing the global loss function.

###### Proof.

Given that the weights $w_{k}(\phi^{t})$ converge to a fixed point as
$t\to\infty$, let us denote this fixed point as
$w^{*}=(w_{1}^{*},w_{2}^{*},\ldots,w_{K}^{*})$. This convergence implies that
the contribution of each client’s model towards the global model stabilizes
over time.

Consider the global loss function $\mathcal{L}(\theta)$ which is convex in
$\theta$. By definition, a function $f(x)$ is convex if, for any
$x_{1},x_{2}\in\text{dom}f$ and any $\lambda\in[0,1]$,

$f(\lambda x_{1}+(1-\lambda)x_{2})\leq\lambda f(x_{1})+(1-\lambda)f(x_{2}).$
(12)

Applying this property to the global loss function $\mathcal{L}(\theta)$, and
considering the weighted sum of local models as the argument, we have:

$\mathcal{L}\left(\sum_{k=1}^{K}w_{k}^{*}\theta_{k}\right)\leq\sum_{k=1}^{K}w_{k}^{*}\mathcal{L}(\theta_{k}).$
(13)

Given the convergence of weights $w_{k}(\phi^{t})$ to $w^{*}$ and the convex
nature of $\mathcal{L}(\theta)$, we utilize the properties of convex functions
to establish that the weighted aggregate of client models under the fixed
weight distribution $w^{*}$ yields a global model $\theta_{g}$ that minimizes
$\mathcal{L}(\theta)$.

Furthermore, since the global loss function $\mathcal{L}(\theta)$ is convex,
the point of convergence corresponds to the global minimum of $\mathcal{L}$,
thereby ensuring that the Meta-Aggregator’s convergence leads to an optimal
aggregation of client models that minimizes the global loss function.

Therefore, under the stated conditions of weight convergence and convexity of
the global loss function, the Meta-Aggregator is guaranteed to converge to an
optimal aggregation of client models, effectively minimizing the global loss
function $\mathcal{L}(\theta)$. ∎

###  VII-B Generalization of the Global Model

Another important aspect is the global model’s ability to generalize well to
unseen data. A bound on the generalization error would be instrumental in
establishing Meta-FL’s efficacy.

######  Theorem 3 (Generalization Bound).

The expected generalization error of the global model $\theta^{*}$ obtained
from the Meta-Aggregator is bounded above by a function of the divergence
between client data distributions and the number of clients, given sufficient
conditions on the learning rate and the complexity of the model class.

###### Proof.

Let $\mathcal{H}$ be the hypothesis space from which the client models are
drawn, and $\mathcal{D}$ represent the distribution of the data across
clients. Assume the global model $\theta^{*}$ is obtained by aggregating
client models with weights $w_{k}$ that minimize the global loss
$\mathcal{L}_{global}(\theta)$.

The expected generalization error, $\epsilon_{gen}$, can be expressed as the
difference between the expected loss over the distribution $\mathcal{D}$ and
the empirical loss observed on the training data. Given the aggregation scheme
and the convexity of $\mathcal{L}$, we leverage a result from the theory of
learning with convex losses to bound $\epsilon_{gen}$:

$\epsilon_{gen}\leq\sqrt{\frac{2\log|\mathcal{H}|}{m}}+\sqrt{\frac{2KL(\mathcal{D}\|\mathcal{D}_{avg})}{m}}+\frac{1}{\sqrt{m}},$
(14)

where $m$ is the total number of samples across all clients,
$KL(\mathcal{D}\|\mathcal{D}_{avg})$ is the Kullback-Leibler divergence
measuring the average divergence between the clients’ data distributions and
the average data distribution $\mathcal{D}_{avg}$, and $|\mathcal{H}|$ denotes
the cardinality of the hypothesis space, reflecting the complexity of the
model class.

This bound indicates that the generalization error is influenced by the model
complexity, the amount of data, and the heterogeneity in data distributions
across clients. The learning rate affects how quickly weights converge, and
under sufficient conditions (e.g., appropriate choice of learning rate), the
bound ensures that the generalization error remains controlled, underscoring
the efficacy of Meta-FL in achieving good performance on unseen data. ∎

##  VIII Experiments

This section details the experimental setup and results obtained by applying
the Meta-FL framework to four distinct datasets, each within the healthcare
domain but presenting unique challenges in medical imaging and diagnostics.
These datasets allow for a comprehensive evaluation of the Meta-FL framework’s
capabilities in addressing data heterogeneity, model diversity, and the
specific demands of healthcare applications.

We utilize four publicly accessible healthcare datasets to demonstrate the
versatility and efficacy of the Meta-FL framework. Each dataset presents its
unique set of challenges:

####  VIII-1 The Cancer Imaging Archive (TCIA)

TCIA provides a substantial collection of cancer-related imaging data, with
172 distinct collections categorized by disease type, imaging modalities, and
research focus. Given the complexity of cancer identification and
classification across diverse imaging techniques, this dataset offers a
rigorous testbed for the meta-FL framework. It is also crucial for evaluating
the framework’s advanced image processing and analysis capabilities, making it
the most complex dataset in our study.

####  VIII-2 COVID-19 X-ray Dataset

This dataset includes over 6,500 chest X-ray images with detailed annotations
for various pneumonia types, including COVID-19. Its specificity for COVID-19
detection tasks allows us to assess the Meta-FL framework’s effectiveness in a
high-stakes, real-world scenario. The dataset’s complexity stems from the high
variability in lung imaging associated with COVID-19 and other types of
pneumonia, positioning it as the second most complex dataset in our analysis.

####  VIII-3 MedPix Database

The MedPix Database is a vast repository of medical images and teaching cases,
comprising over 59,000 images from 12,000 cases. This dataset’s diversity in
disease locations and patient profiles provides a comprehensive challenge for
the Meta-FL framework, especially in computer vision tasks relevant to the
medical field. While complex, it ranks third in our complexity analysis due to
the broader range of conditions it covers.

####  VIII-4 MIMIC-IV Dataset

The MIMIC-IV (Medical Information Mart for Intensive Care) dataset is a
publicly available healthcare dataset that provides a rich repository of de-
identified health-related data associated with over forty thousand patients
who were admitted to critical care units of the Beth Israel Deaconess Medical
Center. It includes detailed information such as laboratory test results,
vital signs, medications, and more, spanning various medical conditions and
patient demographics. The dataset’s breadth and depth present a unique
opportunity to explore the Meta-FL framework’s performance in processing and
analyzing complex, real-world medical data. Given its comprehensive coverage
of critical care patient data, MIMIC-IV is the most extensive dataset in our
study, providing a critical benchmark for evaluating the Meta-FL framework’s
adaptability and scalability to real-world healthcare challenges.

##  IX Results

Our extensive evaluation showcases the Meta-FL framework’s exemplary
performance across diverse metrics, underscoring its adaptability, efficiency,
scalability, and robustness. The experiments were conducted on four
healthcare-related datasets, each presenting unique challenges to FL systems.

###  IX-A Adaptability to New Tasks

The framework’s adaptability is underscored by its ability to enhance model
accuracy significantly within a short span, as evidenced by the improvement
from 70% to 85% accuracy within just five epochs, as shown in Figure 1. This
rapid learning capability is crucial for FL environments continually
encountering new tasks and data distributions.

Figure 1: Adaptability to New Tasks

###  IX-B Meta-feature Relevance Analysis

Analysis of meta-feature relevance reveals that ‘Data Complexity‘ and
‘Learning Rate Sensitivity‘ are paramount, as depicted in Figure 2. This
finding supports the notion that a deep understanding of the data and how
models respond to learning rates is essential for customizing the aggregation
strategy, thereby optimizing the global model’s performance.

Figure 2: Meta-feature Relevance Analysis

###  IX-C Model Generalization

Meta-FL’s generalization capability is evidenced by its consistent accuracy
across four heterogeneous datasets, demonstrating minimal variance in
performance and underscoring the framework’s ability to develop robust global
models applicable across various domains (Figure 3).

Figure 3: Model Generalization

###  IX-D Efficiency in Learning

When compared to traditional FL approaches, Meta-FL demonstrates heightened
efficiency by achieving superior accuracy with fewer communication rounds
needed, as illustrated in Figure 4. This efficiency is attributed to the
framework’s strategic aggregation approach, which prioritizes impactful
updates and integrates meta-learning principles to accelerate convergence.

Figure 4: Efficiency in Learning

###  IX-E Scalability

The framework’s scalability is further validated through experiments showing
performance improvement or stability with increasing clients without
significantly increasing training time, as depicted in Figures 5 and
LABEL:fig:scalability_training_time. This scalability indicates Meta-FL’s
capability to effectively manage larger federated networks without
compromising performance.

Figure 5: Scalability: Average Accuracy vs. Number of Clients

###  IX-F Robustness to Distribution Shifts

Meta-FL’s robustness against distribution shifts is confirmed by its sustained
high performance across datasets with varying complexities, as shown in Figure
6. This resilience is critical for FL systems that operate across diverse and
evolving data landscapes.

Figure 6: Robustness to Distribution Shifts: Accuracy Across Datasets

The results affirm the Meta-FL framework’s advanced capabilities in addressing
FL’s intrinsic challenges, such as data heterogeneity and model diversity.
Meta-FL significantly surpasses traditional FL methodologies by leveraging
meta-learning for dynamic aggregation and emphasizing crucial meta-features.
These advancements suggest that Meta-FL elevates the efficacy and efficiency
of FL systems and broadens their applicability across many real-world
scenarios, marking a significant progression in the field.

##  X Discussion

The Meta-FL framework introduced in this study represents a significant leap
forward in addressing the inherent challenges of FL, particularly regarding
data heterogeneity and model diversity. Our approach, centered around an
optimization-based Meta-Aggregator, dynamically integrates heterogeneous
client models by leveraging meta-features, thus ensuring adaptive, efficient,
and theoretically sound aggregation. This novel strategy not only enhances the
synergy among diverse client models but also significantly improves the
overall performance of the global model. Empirical evaluations across multiple
healthcare-related datasets have demonstrated the Meta-FL framework’s superior
adaptability, meta-feature relevance, generalization capabilities, efficiency,
scalability, and robustness compared to traditional FL approaches. The
framework’s ability to rapidly adapt to new tasks, prioritize impactful meta-
features for optimized aggregation, and maintain high model generalization
across varied datasets is particularly noteworthy. Additionally, the Meta-FL
framework achieves superior learning efficiency and scalability, validating
its potential for broad applicability and improved model performance in real-
world distributed learning scenarios. These findings underscore the Meta-FL
framework’s capacity to facilitate collaborative learning in a privacy-
preserving, decentralized manner while effectively managing the complexity and
diversity inherent in federated networks. The Meta-FL framework sets a new
benchmark for efficiency, privacy, and performance in FL systems by addressing
critical limitations of existing FL methodologies.

## References

  * [1] K. A. Bonawitz, P. Kairouz, H. B. McMahan, and D. Ramage, “Federated learning and privacy,” _Queue_ , vol. 19, pp. 87 – 114, 2021.
  * [2] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” _IEEE Signal Processing Magazine_ , vol. 37, pp. 50–60, 2019.
  * [3] Z. Li, V. Sharma, and S. Mohanty, “Preserving data privacy via federated learning: Challenges and solutions,” _IEEE Consumer Electronics Magazine_ , vol. 9, pp. 8–16, 2020.
  * [4] M. Smietanka, H. Pithadia, and P. Treleaven, “Federated learning for privacy-preserving data access,” _Available at SSRN 3696609_ , 2020.
  * [5] A. Tan, H. Yu, L. zhen Cui, and Q. Yang, “Towards personalized federated learning,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 34, pp. 9587–9603, 2021.
  * [6] P. K. et al., “Advances and open problems in federated learning,” _Found. Trends Mach. Learn._ , vol. 14, pp. 1–210, 2019.
  * [7] K. B. et al, “Towards federated learning at scale: System design,” _ArXiv_ , vol. abs/1902.01046, 2019.
  * [8] X. Yin, Y. Zhu, and J. Hu, “A comprehensive survey of privacy-preserving federated learning,” _ACM Computing Surveys (CSUR)_ , vol. 54, pp. 1 – 36, 2021.
  * [9] Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” _arXiv preprint arXiv:1806.00582_ , 2018.
  * [10] Q. Li, Z. Wen, and B. He, “Federated learning systems: Vision, hype and reality for data privacy and protection,” _IEEE Transactions on Knowledge and Data Engineering_ , 2020.
  * [11] K. Hsieh _et al._ , “Non-iid data in federated learning: Challenges and solutions,” _arXiv preprint arXiv:2006.11477_ , 2020.
  * [12] H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in _AISTATS_ , 2017.
  * [13] S. Truex, N. Baracaldo, A. Anwar, T. Steinke, H. Ludwig, R. Zhang, and Y. Zhou, “A hybrid approach to privacy-preserving federated learning,” in _ACM CCS_ , 2019.
  * [14] R. C. Geyer _et al._ , “Differentially private federated learning: A client level perspective,” _arXiv preprint arXiv:2102.04917_ , 2021.
  * [15] J. Wang, Q. Liu, S. Liang, W. Zhang, and X. Zhang, “Federated learning with non-iid data,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2020.
  * [16] S. P. Karimireddy _et al._ , “Scaffold: Stochastic controlled averaging for federated learning,” in _International Conference on Machine Learning_ , 2020.
  * [17] A. Fallah _et al._ , “Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach,” in _NeurIPS_ , 2020.
  * [18] M. Chen _et al._ , “Fedmeta: Federated learning via meta-learning on non-iid data,” _arXiv preprint arXiv:2101.06340_ , 2021.
  * [19] J. Park _et al._ , “Fed++: Advanced federated learning framework with comprehensive privacy protection,” _arXiv preprint arXiv:2105.06787_ , 2021.
  * [20] V. Smith _et al._ , “Federated learning: Strategies for improving communication efficiency,” _arXiv preprint arXiv:2102.03548_ , 2021.


# A System for Differentiation of Schizophrenia and Bipolar Disorder based on
rsfMRI

Daniela Janeva, Stefan Krsteski, Matea Tashkovska, Nikola Jovanovski, Tomislav
Kartalov, Dimitar Taskovski,
Zoran Ivanovski, and Branislav Gerazov

Faculty of Electrical Engineering and Information Technologies,
Ss. Cyril and Methodius University, Skopje, Macedonia
<EMAIL_ADDRESS>
###### Abstract

Schizophrenia and bipolar disorder are debilitating psychiatric illnesses that
can be challenging to diagnose accurately. The similarities between the
diseases make it difficult to differentiate between them using traditional
diagnostic tools. Recently, resting-state functional magnetic resonance
imaging (rsfMRI) has emerged as a promising tool for the diagnosis of
psychiatric disorders. This paper presents several methods for differentiating
schizophrenia and bipolar disorder based on features extracted from rsfMRI
data. The system that achieved the best results, uses 1D Convolutional Neural
Networks to analyze patterns of Intrinsic Connectivity time courses obtained
from rsfMRI and potentially identify biomarkers that distinguish between the
two disorders. We evaluate the system’s performance on a large dataset of
patients with schizophrenia and bipolar disorder and demonstrate that the
system achieves a 0.7078 Area Under Curve (AUC) score in differentiating
patients with these disorders. Our results suggest that rsfMRI-based
classification systems have great potential for improving the accuracy of
psychiatric diagnoses and may ultimately lead to more effective treatments for
patients with this disorder.

###### Index Terms:

Schizophrenia, Bipolar disorder, resting-state Functional Magnetic Resonance
Imaging (rsfMRI), 1D Convolutional Neural Networks, biomedical engineering,
AUC;

##  I Introduction

Schizophrenia and bipolar disorder are two of the most challenging psychiatric
illnesses, affecting millions of people worldwide. Schizophrenia is a severe
mental disorder characterized by a wide range of symptoms, including
delusions, hallucinations, disorganized thinking, and abnormal behaviors [1].
On the other hand, bipolar disorder is a mood disorder characterized by
recurrent episodes of mania and depression [2]. Schizophrenia and bipolar
disorder are chronic illnesses that can severely impact an individual’s daily
life and functioning [3]. The symptoms of these disorders can be distressing
and debilitating, making it difficult for patients to maintain relationships,
work, or engage in everyday activities. Unfortunately, accurate diagnosis of
these disorders is often delayed or missed, resulting in inappropriate or
ineffective treatment.

While the two disorders have distinct clinical features, they also share some
similarities in terms of symptoms and genetic risk factors. Both disorders
register problems in cognitive achievements reporting deficits in visuospatial
performance as a precursor of both disorders. This overlap has led some
researchers to suggest that the two disorders may be part of a broader
spectrum of mental illnesses that share underlying genetic and environmental
risk factors [4].

It is essential to distinguish between schizophrenia and bipolar disorder
because although they share some common symptoms, they require different
treatments. Misdiagnosis or delayed diagnosis can lead to inappropriate or
ineffective treatments, resulting in poor outcomes for patients. For example,
antipsychotic medications, which are typically used to treat schizophrenia,
may exacerbate symptoms of mania in bipolar disorder [5]. Conversely, mood
stabilizers and antidepressants, typically used to treat bipolar disorder, may
not be effective for treating symptoms of schizophrenia [6].

In recent years, the development of new techniques for brain imaging has led
to significant advances in the diagnosis and treatment of schizophrenia and
bipolar disorder. Resting-state functional magnetic resonance imaging (rsfMRI)
has emerged as a promising tool for understanding the underlying neural
mechanisms of these disorders. RsfMRI measures brain activity by detecting
changes in blood flow to different regions of the brain during periods of
rest. Studies have shown that there are distinct patterns of brain activity
associated with schizophrenia and bipolar disorder, and these patterns can be
used to differentiate between the two disorders [7], [8]. In this paper, we
present methods for differentiating schizophrenia and bipolar disorder based
on rsfMRI data. We apply machine learning algorithms to analyze patterns of
resting-state brain activity and evaluate the models’ performances on a large
dataset of patients with schizophrenia and bipolar disorder. The proposed
system was submitted to the IEEE Signal Processing Cup (SPC) 2023. [9].

##  II Dataset

The dataset used in this work was provided by the Brain Space Initiative for
the IEEE SPC [9]. It consists of features extracted from the rsfMRI data of
individuals with Schizophrenia and Bipolar disorder. The dataset was obtained
by using 105 intrinsic connectivity network (ICN) time courses derived from a
multi-spatial-scale spatially constrained ICA approach and their functional
network connectivity (FNC).

The provided features were extracted using the following steps [9]:

  1. 1.

Quality control was applied to identify high-quality data.

  2. 2.

Each subject’s rsfMRI data were preprocessed using a common procedure,
including rigid body motion correction, slice timing correction, and
distortion correction.

  3. 3.

Preprocessed subject data were registered into a common space, resampled to 3
mm3 isotropic voxels and spatially smoothed using a Gaussian kernel with a 6
mm full width at half-maximum (FWHM).

  4. 4.

A multi-spatial-scale template of 105 ICNs obtained from 100k+ subjects was
used and a constrained ICA approach to obtain subject-specific ICN time
courses.

  5. 5.

To calculate FNC, ICN time courses were cleaned using a common standard and
FNC is estimated by calculating the Pearson correlation between each pair of
ICN time courses resulting in one FNC matrix for each individual.

The training dataset consists of ICN and FCN features for 471 individuals and
the test set contains the features of 315 individuals. An additional test set
was withheld to evaluate the submitted models in the IEEE SPC.

##  III Methods

We applied different methods for the differentiation of the two diagnostic
groups for the FCN and ICN features according to their nature. For the
differentiation of the FCN features between diagnostic groups, we applied
statistical methods for feature selection and machine learning algorithms for
binary classifications. For the classification of ICNs, we applied digital
signal processing techniques as well as machine learning techniques for
feature extraction and binary classification. With each of the trained models,
we predicted the labels of the test set using soft probability scores, which
we used to evaluate the models’ AUC scores for the IEEE SPC.

###  III-A Intrinsic Connectivity Network

Intrinsic connectivity networks are a set of brain networks, defined based on
the intrinsic functional connectivity of different brain regions that are
identified using fMRI [10]. They are thought to reflect the underlying
organization of the brain which is helpful for understanding brain function
[11]. ICNs are mainly identified using techniques such as independent
component analysis (ICA), which separates the fMRI data into independent
components that correspond to different functional networks [12], [13].
Research shows alterations in ICNs in various neuropsychiatric disorders,
emphasizing their importance for understanding the neural mechanisms
underlying those conditions [14]. ICNs provide a powerful tool for
investigating the functional organization of the brain and its implications
for cognition and behavior [15], which are altered processes in patients with
schizophrenia and bipolar disorder.

####  III-A1 Preprocessing

To compensate for the varying ICN time course lengths provided by the IEEE SPC
organizers, we padded shorter components with zeros to match the maximum ICN
length of the dataset.

Figure 1: Sample ICN component of an individual with bipolar disorder (BP)
with the original signal and the low, mid, and high band signals obtained with
the filter bank

Figure 2: Sample ICN component of an individual with schizophrenia (SZ), with
the original signal and the low, mid, and high band signals obtained with the
filterbank

We used a filter bank to obtain signals in three frequency sub-bands: low,
mid, and high. The low band contains frequencies between $0.01-0.3$ Hz, the
mid band contains frequencies between $0.3-0.7$ Hz, and the high band
frequencies are between $0.7-0.99$ Hz. The filter bank comprises three
bandpass Butterworth IIR filters of order 6. In Fig. 1 and Fig. 2 we show a
sample ICN component of an individual with bipolar disorder (BP) and an
individual with schizophrenia (SZ) in the filtered frequency sub-bands.

####  III-A2 Spectrogram

For obtaining the time-frequency representation of the ICN time courses, we
applied the Short Time Fourier Transformation algorithm (STFT) using a sliding
Tukey window of length 22.

We then calculate the power spectrogram.

By stacking the spectrograms of each ICN, we create a volumetric
representation for the individual with dimensions $12\times 11\times 105$
where the $x$-axis represents the frequency components, the $y$-axis the time
components, and the $z$-axis the number of ICNs as shown in Fig. 3.

####  III-A3 Scalogram

For obtaining a 2D representation of the ICNs with a better time-frequency
resolution, we applied the Continuous Wavelet transformation (CWT). CWT is a
formal tool that provides an overcomplete representation of a signal by
“daughter” wavelets which are scaled and translated copies of the finite-
length oscillating waveforms known as the “mother wavelet”.

The wavelet analysis provides not only accurate frequency information but at
the same time it provides information for accurate time localization of the
frequency components. This property makes the wavelet transformation highly
applicable for the analysis of signals which are characterized by the
occurrence of transient events.

We generated scalograms of the ICN components using the Morlet wavelet and 50
scales. We stacked the scalograms to create 3D information with dimensions
$49\times 234\times 105$ where the $x$-axis represents the scales, the
$y$-axis represents the time courses and the $z$-axis represents the number of
ICNs as shown in Fig 3.

Figure 3: 3D representation of spectrograms (left) and scalograms (right).

Figure 4: 1D (top) and 3D (bottom) CNN architectures.

####  III-A4 Classification

For the classification of raw ICNs, and filtered ICNs in different bandwidths,
we used 1D Convolutional Neural Networks (CNNs). The architecture of the model
is shown in Fig 4. The outputs of each convolution layer are passed through
the ReLU activation function [16].

For the classification of the 3D stack of scalograms and spectrograms, we used
the 3D CNN model shown in Fig. 4. We used a simpler architecture, because of
memory and computational constraints.

We trained both models using Adam [17] in 100 epochs with a batch size of 32.
To prevent overfitting, we used Early Stopping with patience of 20 Epochs
based on the binary cross entropy loss function [18].

To evaluate the performance of our models we used the AUC score [19] on 20% of
the training dataset.

###  III-B Functional Connectivity Network

Functional connectivity networks refer to patterns of synchronous activity
between different brain regions that are thought to underlie specific
cognitive functions [20]. These networks are typically identified using
functional magnetic resonance imaging (fMRI) and other neuroimaging techniques
that can measure the correlation between the activity of different brain
regions. Resting-state networks are commonly studied in the absence of any
explicit task or stimulus. The study of functional connectivity networks has
important implications for understanding the neural basis of complex cognitive
processes and for identifying biomarkers of neurological and psychiatric
disorders [21].

The IEEE SPC Dataset FCNs are generated by calculating Pearson correlation
coefficients of the ICN time courses which results in a symmetrical matrix as
shown in Fig. 5. The lower triangular matrix is flattened and provided in the
IEEE SPC dataset in the form of a vector with dimensions $1\times 5460$.

Figure 5: Functional connectivity network

####  III-B1 Feature selection

We normalized the vector values in the range of 0 to 1 and applied the chi-
square test [22], for feature selection. We selected the 20 best features that
explain most of the variability in the dataset.

####  III-B2 Classification

We applied different machine-learning algorithms for binary classification of
the raw ICN features in the train set, including: Logistic Regression (LR),
Support Vector Machines (SVM), Linear Discriminant Analysis (LDA), Gaussian
Naive Bayes (GNB), K-Nearest Neighbours (KNN). For hyperparameters tuning, we
did a Grid Search with 5-fold cross-validation [23].

We applied the same methodology for training the machine learning algorithms
using only the selected features.

###  III-C Evaluation

For the evaluation of the models, the AUC score was used in the IEEE SPC. We
used the best models to predict the soft probability score of the labels of
the test set provided in the competition dataset. The soft probability score
indicates the confidence the model has in its prediction and it is used to
calculate the AUC score. For the competition, the AUC was averaged for the
public test set, i.e. the Public AUC score, and the withheld test set, i.e.
the private AUC Score.

##  IV Results

The AUC scores from our CNN models for the filtered raw ICNs, and stacked ICN
spectrograms and scalograms are shown in Table I. We can see that the best
score is obtained by 1D CNN model trained on raw ICN timecourses. The filtered
signal in the range of 0.3 - 0.7 Hz achieved a better AUC score compared to
the other bandwidths. Classification of scalograms produced better AUC score,
compared to the classification of spectrograms. However, the extracted
features from the ICN components achieved slightly worse AUC scores, overall.
The AUC scores from the classical machine learning models for all of the FCNs
and the selected FCNs are shown in Tables II & III. We can see that the best
score is obtained by the Linear Discriminant Analysis algorithm.
Classification of all features resulted in a slightly better overall
performance compared to using only the top 20 features. However, while
including all features provided enhancement in the classification results,
using a smaller subset of the most relevant features can yield comparable
outcomes.

TABLE I: Classification of ICN features Features | Method | AUC Public | AUC Private
---|---|---|---
raw ICN | 1D CNN | 0.705 | 0.637
ICN $0.01-0.3$ Hz | 1D CNN | 0.509 | 0.551
ICN $0.3-0.7$ Hz | 1D CNN | 0.5881 | 0.597
ICN $0.7-0.99$ Hz | 1D CNN | 0.588 | 0.563
spectrograms | 3D CNN | 0.509 | 0.551
scalograms | 3D CNN | 0.626 | 0.528
TABLE II: Classification of all FCN features Algorithm | AUC Public | AUC Private
---|---|---
LR | 0.678 | 0.589
SVM | 0.675 | 0.510
LDA | 0.686 | 0.589
GNB | 0.624 | 0.547
KNN | 0.549 | 0.502
DT | 0.500 | 0.500
RF | 0.658 | 0.540
TABLE III: Classification of selected 20 FCN features Algorithm | AUC Public | AUC Private
---|---|---
LR | 0.653 | 0.530
SVM | 0.634 | 0.520
LDA | 0.658 | 0.525
GNB | 0.656 | 0.512
KNN | 0.465 | 0.516
DT | 0.574 | 0.475
RF | 0.636 | 0.589

##  V Conclusion

The differentiation between schizophrenia and bipolar disorder is an important
task for accurate diagnosis and proper treatment of patients. We proposed a
number of systems based on rs-fMRI features for achieving this objective. The
best-trained 1D CNN model trained on raw ICN timecourses achieved the highest
AUC score. The FCNs showed potential but their classification achieved
slightly worse AUC scores than raw ICN timecourses. Overall, the results are
promising but future work needs to be done for improving the differentiation
between Schizophrenia and Bipolar disorder. Future work will include combining
the ICN and FCN features and applying more advanced classification algorithms
such as NN, LSTM, GRU, RNN, and transformers.

## References

  * [1] R. A. McCutcheon, T. R. Marques, and O. D. Howes, “Schizophrenia—an overview,” _JAMA psychiatry_ , vol. 77, no. 2, pp. 201–210, 2020.
  * [2] I. Grande, M. Berk, B. Birmaher, and E. Vieta, “Bipolar disorder,” _The Lancet_ , vol. 387, no. 10027, pp. 1561–1572, 2016.
  * [3] M. F. Green, “Cognitive impairment and functional outcome in schizophrenia and bipolar disorder,” _Journal of Clinical Psychiatry_ , vol. 67, p. 3, 2006\.
  * [4] W. Maier, A. Zobel, and M. Wagner, “Schizophrenia and bipolar disorder: differences and overlaps,” _Current Opinion in Psychiatry_ , vol. 19, no. 2, pp. 165–170, 2006.
  * [5] M. Narasimhan, T. O. Bruce, and P. Masand, “Review of olanzapine in the management of bipolar disorders,” _Neuropsychiatric disease and treatment_ , vol. 3, no. 5, pp. 579–587, 2007.
  * [6] R. Nayak, I. Rosh, I. Kustanovich, and S. Stern, “Mood stabilizers in psychiatric disorders and mechanisms learnt from in vitro model systems,” _International Journal of Molecular Sciences_ , vol. 22, no. 17, p. 9315, 2021\.
  * [7] E. A. Reavis, J. Lee, J. K. Wynn, S. A. Engel, M. S. Cohen, K. H. Nuechterlein, D. C. Glahn, L. L. Altshuler, and M. F. Green, “Assessing neural tuning for object perception in schizophrenia and bipolar disorder with multivariate pattern analysis of fmri data,” _Neuroimage: Clinical_ , vol. 16, pp. 491–497, 2017.
  * [8] B. Rashid, E. Damaraju, G. D. Pearlson, and V. D. Calhoun, “Dynamic connectivity states estimated from resting fmri identify differences among schizophrenia, bipolar disorder, and healthy control subjects,” _Frontiers in human neuroscience_ , vol. 8, p. 897, 2014.
  * [9] Brain Space Initiative, “Psychosis classification with rsfMRI,” 2023. [Online]. Available: https://kaggle.com/competitions/psychosis-classification-with-rsfmri
  * [10] A. R. Laird, P. M. Fox, S. B. Eickhoff, J. A. Turner, K. L. Ray, D. R. McKay, D. C. Glahn, C. F. Beckmann, S. M. Smith, and P. T. Fox, “Behavioral interpretations of intrinsic connectivity networks,” _Journal of cognitive neuroscience_ , vol. 23, no. 12, pp. 4022–4037, 2011.
  * [11] A. Dovern, G. R. Fink, A. C. B. Fromme, A. M. Wohlschläger, P. H. Weiss, and V. Riedl, “Intrinsic network connectivity reflects consistency of synesthetic experiences,” _Journal of Neuroscience_ , vol. 32, no. 22, pp. 7614–7621, 2012.
  * [12] X.-N. Zuo, C. Kelly, J. S. Adelstein, D. F. Klein, F. X. Castellanos, and M. P. Milham, “Reliable intrinsic connectivity networks: test–retest evaluation using ica and dual regression approach,” _Neuroimage_ , vol. 49, no. 3, pp. 2163–2177, 2010.
  * [13] W. W. Seeley, V. Menon, A. F. Schatzberg, J. Keller, G. H. Glover, H. Kenna, A. L. Reiss, and M. D. Greicius, “Dissociable intrinsic connectivity networks for salience processing and executive control,” _Journal of Neuroscience_ , vol. 27, no. 9, pp. 2349–2356, 2007.
  * [14] A. Mohan, A. J. Roberto, A. Mohan, A. Lorenzo, K. Jones, M. J. Carney, L. Liogier-Weyback, S. Hwang, and K. A. Lapidus, “Focus: the aging brain: the significance of the default mode network (dmn) in neurological and neuropsychiatric disorders: a review,” _The Yale journal of biology and medicine_ , vol. 89, no. 1, p. 49, 2016.
  * [15] L. Wang, H. Shen, Y. Lei, L.-L. Zeng, F. Cao, L. Su, Z. Yang, S. Yao, and D. Hu, “Altered default mode, fronto-parietal and salience networks in adolescents with internet addiction,” _Addictive Behaviors_ , vol. 70, pp. 1–6, 2017.
  * [16] A. F. Agarap, “Deep learning using rectified linear units (relu),” _arXiv preprint arXiv:1803.08375_ , 2018.
  * [17] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [18] L. Prechelt, “Early stopping-but when?” in _Neural Networks: Tricks of the trade_. Springer, 2002, pp. 55–69.
  * [19] C. X. Ling, J. Huang, H. Zhang _et al._ , “Auc: a statistically consistent and more discriminating measure than accuracy,” in _Ijcai_ , vol. 3, 2003, pp. 519–524.
  * [20] R. L. Buckner, F. M. Krienen, and B. T. Yeo, “Opportunities and limitations of intrinsic functional connectivity mri,” _Nature neuroscience_ , vol. 16, no. 7, pp. 832–837, 2013.
  * [21] M. J. Jafri, G. D. Pearlson, M. Stevens, and V. D. Calhoun, “A method for functional network connectivity among spatially independent resting-state components in schizophrenia,” _Neuroimage_ , vol. 39, no. 4, pp. 1666–1681, 2008.
  * [22] M. L. McHugh, “The chi-square test of independence,” _Biochemia medica_ , vol. 23, no. 2, pp. 143–149, 2013.
  * [23] S. M. LaValle, M. S. Branicky, and S. R. Lindemann, “On the relationship between classical grid search and probabilistic roadmaps,” _The International Journal of Robotics Research_ , vol. 23, no. 7-8, pp. 673–692, 2004\.