# Modern Views of Machine Learning for Precision Psychiatry

Zhe Sage Chen*, Prathamesh (Param) Kulkarni, Isaac R. Galatzer-Levy,
Benedetta Bigio, Carla Nasca and Yu Zhang  Z.S. Chen is with the Department of
Psychiatry, Department of Neuroscience and Physiology, the Neuroscience
Institute, New York University Grossman School of Medicine, New York, NY
10016, USA. He is also with the Department of Biomedical Engineering, New York
University Tandon School of Engineering, Brooklyn, NY 11201, USA. P. Kulkarni
is with the Headspace Health, San Francisco, CA, USA. I. R. Galatzer-Levy is
with the Meta Realty Lab, New York, NY, USA, and also with the Department of
Psychiatry, New York University Grossman School of Medicine, New York, NY
10016, USA B. Bigio is with with Department of Psychiatry, New York University
Grossman School of Medicine, New York, NY 10016, USA. C. Nasca is with the
Department of Psychiatry and also with the Neuroscience Institute, New York
University Grossman School of Medicine, New York, NY 10016, USA. Y. Zhang is
with the Department of Bioengineering, Lehigh University, Bethlehem, PA 18015,
USA.*Corresponding authors<EMAIL_ADDRESS>(Z.S. Chen, ORCID:
0000-0002-6483-6056) and<EMAIL_ADDRESS>(Y. Zhang, ORCID:
0000-0003-4087-6544)

###### Abstract

In light of the NIMH’s Research Domain Criteria (RDoC), the advent of
functional neuroimaging, novel technologies and methods provide new
opportunities to develop precise and personalized prognosis and diagnosis of
mental disorders. Machine learning (ML) and artificial intelligence (AI)
technologies are playing an increasingly critical role in the new era of
precision psychiatry. Combining ML/AI with neuromodulation technologies can
potentially provide explainable solutions in clinical practice and effective
therapeutic treatment. Advanced wearable and mobile technologies also call for
the new role of ML/AI for digital phenotyping in mobile mental health. In this
review, we provide a comprehensive review of the ML methodologies and
applications by combining neuroimaging, neuromodulation and advanced mobile
technologies in psychiatry practice. Additionally, we review the role of ML in
molecular phenotyping, cross-species biomarker identification in precision
psychiatry. We further discuss explainable AI (XAI) and causality testing in a
closed-human-in-the-loop manner, and highlight the ML potential in multimedia
information extraction and multimodal data fusion. Finally, we discuss
conceptual and practical challenges in precision psychiatry and highlight ML
opportunities in future research.

###### Index Terms:

Machine learning (ML), artificial intelligence (AI), deep learning, precision
psychiatry, digital psychiatry, computational psychiatry, neuroimaging,
neurobiomarker, molecular biomarker, digital phenotyping, multimodal data
fusion, neuromodulation, causality, explainable AI (XAI), teletherapy

##  1 Introduction

Mental health is epidemic in the United States and the world. According to the
National Institute of Mental Health (NIMH), nearly one in five American adults
suffer from a form of mental illness or psychiatric disorder
(www.nimh.nih.gov/health/statistics/). According to the Centers for Disease
Control and Prevention (CDC), The COVID-19 pandemic has witnessed a
significant impact on our lifestyle, and considerably elevated adverse mental
health conditions caused by fear, worry and uncertainty [1]. Increased suicide
rates, opioid abuse, antidepressant usage have been observed in both adults
and teenagers. The diagnosis and treatment of mental health has imposed a
burden to the healthcare system and the society. For instance, the economic
burden of depression alone is estimated to be at least $210 billion annually
(www.workplacementalhealth.org/mental-health-topics/depression/). Precision
medicine (or personalized medicine) is an innovative approach to tailoring
disease prevention, diagnosis, and treatment that account for the differences
in subjects’ genes, environments, and lifestyles. The goal of precision
medicine is to target timely and accurate diagnosis/prognosis/therapeutics for
individualized patient’s health problem, and further provide feedback
information to patients and surrogate decision-makers. Recent decades have
witnessed various degrees of successes in precision medicine, especially in
oncology (Lancet, vol. 397, 2021, p. 1781). Traditional diagnoses of mental
illnesses rely on physical exam, lab tests, psychological and behavioral
evaluation. Meanwhile, precision psychiatry has increasingly received its
deserved attention in the field [2, 3]. Although psychiatry has not yet
benefited fully from the advanced diagnostic and therapeutic technologies that
have an impact on other clinical specialties, these technologies have the
potential to transform the future psychiatric landscape.

The NIMH’s RDoC (Research Domain Criteria) initiative aims to address the
heterogeneity of mental illness and provide a biologically-based (as opposed
to symptom-based) framework for understanding these mental illnesses in terms
of varying degrees of dysfunction in general psychological or neurobiological
systems; it attempts to bridge the power of modern multidisciplinary (such as
in genetics, neuroscience, and behavioral science) research approaches [4]. In
contrast, the DSM (Diagnostic and Statistical Manual of Mental Disorders)
maintained by the American Psychiatric Association (APA), the assessment of
diagnosis and treatment outcome in mental disorders are often based on the
clinician’s observations, behavioral symptoms and patient reporting, which is
susceptible to a high degree of variability. Therefore, it is imperative to
develop quantitative neurobiological markers for mental disorders while
accounting for their heterogeneity and comorbidity.

One important goal in neuropsychiatry research is to identify the relationship
between neurobiological/neurophysiological findings and clinical
behavioral/self-report observations. Machine learning (ML) and artificial
intelligence (AI) have generated growing interest in psychiatry because of
their strong predictive power and generalization ability, which show promising
medicine applications in prognosis and diagnosis [5, 6, 7]. The interest of
applying ML/AI in psychiatry has grown steadily in the past two decades, as
reflected in the number of PubMed publications (Fig. 1A). To improve mental
health outcome with digital technologies, the so-called “digital psychiatry”
is focused on developing ML/AI methods intelligence for assessing, diagnosing,
and treating mental health issues [8]. A recent global survey indicated that
psychiatrists were somewhat skeptical that AI could replace human empathy, but
many predicted that ’man and machine’ would increasingly collaborate in
undertaking clinical decisions, and they were optimistic that AI might improve
efficiencies and access to care, and reduce costs [9].

The past two decades have witnessed substantial growth of ML applications for
psychiatry in the literature, reflected in applications and reviews [10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20]. Although multiple reviews of ML for
psychiatry are available, the majority of reviews are restricted to relatively
narrow scopes. In this paper, we try to provide a comprehensive review of ML
and ML-powered technologies in mental health applications. Our view is
“modern” in a sense that the development of new technologies, consumer market
demand and public health crises (such as COVID-19) have constantly redefined
the role of ML and reshaped our thinking in the field of precision psychiatry.
Specifically, we will cover state-of-the-art technological and methodological
developments in ML, multimodal neuroimaging, neuromodulation, large-scale
circuit modeling and human-machine interface. It is noteworthy that our
reviewed literature is by no means exhaustive due to space limitation. To
distinguish our review from others, we will focus on several issues central to
the ML applications for psychiatry: generalizability, interpretability,
causality, clinical and behavioral integration.

Our view towards this research direction is optimistic for several reasons:
first, with the growing amount of data and computational power, there is an
emerging demand for psychiatrists to use ML to reevaluate clinical, behavioral
and neuroimaging data. Additionally, the past few years have witnessed a
substantially growing interest in mental health funding within the industry
(Fig. 1B). Second, it is becoming increasingly important to leverage the power
of ML and develop explainable artificial intelligence (XAI) tools for unbiased
risk diagnosis, personalized medicine recommendation, and neurostimulation.
Advances in neuroimaging have provided improved spatial and temporal
resolution to examine brain activity; the integration of ML with neuroimaging
can potentially lead to further advances that help us identify and validate
biomarkers in diagnosis and treatment of mental illnesses. Third, there is an
increasing demand of psychiatrists in the US, and the shortage is particularly
acute in poorer countries [21]. ML/AI technologies may change the practice of
psychiatry for both clinicians and patients. Finally, new advanced
technologies such as social media, multimedia (speech and vision), mobile and
wearable devices also call for the development of ML/AI tools to assist the
assessment, diagnosis or treatment of patients who are mentally ill or at
risk. From now on, we will use ML and AI interchangeably throughout the paper.

The rest of the paper is organized as follows. Section 2 will present a brief
background of modern neuroimaging technologies. Section 3 will explain why and
how ML methods can help psychiatry. We will also provide a few case studies to
illustrate the power of these methods. Section 4 will review various ML-
powered technologies for psychiatry. In Section 5, we will discuss multimodal
data fusion methods in diagnostic analytics. Section 6 will discuss ML
applications for discovering molecular biomarkers for precision psychiatry.
Section 7 will discuss XAI, causality validation and computational modeling
approaches in psychiatry. Finally, we will discuss the research challenges and
outstanding questions, and conclude the paper in Section 8.

Figure 1: (A) The number of PubMed publications with keywords ”machine
Learning or AI” and ”psychiatry or mental health” in the title or abstract
(Year 2000-2021). (B) Growth of mental health tech funding in the US market
(Year 2017-2021, data source: cbinsights.com). (C) Human neuroimaging at
various spatial and temporal resolution ($\copyright$ IEEE, figure reproduced
from [22] with permission).

##  2 Background of Neuroimaging

###  2.1 Mind vs. Brain

The brain is a physical organ, which provides the center that supports all
cognitive functions. It can be considered as the hardware of the human body.
In contrast, the mind is abstract; it creates emotions and enables
consciousness, perception, thinking, judgment, and memory. The World Health
Organization (WHO) defines mental health as ”a state of well-being in which
the individual realizes his or her own abilities, can cope with the normal
stresses of life, can work productively and fruitfully, and is able to make a
contribution to his or her community.” Psychiatry is seeking to measure the
mind, which is not quite the same thing as the brain; therefore, it relies on
the quantification of how people feel under specific tasks or behavioral
conditions. Psychiatry disorders are often described as disorders of the mind,
which disrupt the brain’s ability to function normally to complete certain
processes. For instance, anxiety, stress, depression, autism, obsessive-
compulsive disorder (OCD), and post-traumatic stress disorder (PTSD) are
categorized by varying degrees of psychomotor, cognitive, affective, and
volitional impairment. Since mind and brain are internally interleaved, the
syndromes of mental disorders are associated with the abnormality of the mind
and brain [23], or to be more precise, as syndromes of disrupted neural,
cognitive, and behavioral systems [2].

###  2.2 Advances in Neuroimaging

Human neuroimaging provides a window to examine a healthy and diseased brain,
in terms of both structural and functional forms. Advances in neuroimaging
techniques offer increasing scale and resolutions to examine brain activity at
macroscopic, mesoscopic, and microscopic scales with various temporal and
spatial resolutions (Fig. 1C).

Our understanding of brain and behavior relationships has expanded
exponentially over the last few decades. While this improvement may be
attributed to a multitude of factors, advancement in neuroimaging has played a
prominent role [24]. Modern-day advancements in neuroimaging have provided an
increased ability to connect neuroanatomical structure and function with
clinical presentations, refining our understanding of associations of specific
neural regions and circuits with cognition and behavior, which in turn has led
to improvements in clinical practice [25]. The rapidly advancing field of
neuroimaging has allowed scientists to examine brain processes in new and
exciting ways. Ranging from increased utilization of structural neuroimaging
techniques to the significant scientific advancements brought about by the
increased availability of functional neuroimaging, these technologies have
provided significant benefits to advance the understanding of neural
correlates and to the discovery of neurobiological markers in psychiatric
disorders [26, 27]. Some of the most common neuroimaging methods include the
utilization of magnetic resonance imaging (MRI), functional MRI (fMRI),
diffusion tensor imaging (DTI), electroencephalography (EEG),
Magnetoencephalography (MEG), electrocorticography (ECoG), Functional Near-
Infrared Spectroscopy (fNIRS), and positron emission tomography (PET) to probe
brain function and understand the neurobiological mechanism underlying various
psychiatric conditions.

  * •

MRI. MRI is a non-invasive imaging technology that produces three-dimensional
detailed anatomical images and has been widely used in clinics for diagnosis,
staging, and follow-up of brain disease [28].

  * •

DTI. DTI is a technological advancement that allows researchers and clinicians
to assess human white matter pathways in vivo [29]. DTI has shown its promise
in characterizing neuroanatomical connectivity underlying white matter tracts,
which is hard to be captured by MRI.

  * •

fMRI. fMRI is a hemodynamic technique that uses MR imaging to measure brain
functional activity; it works by detecting the small regional changes in blood
oxygenation and flow, known as blood oxygenation level-dependent (BOLD)
signals, occurring in response to neural activity [30]. Due to the excellent
spatial resolution, fMRI has been extensively used for studying
neurobiological basis of various psychiatric disorders.

  * •

PET. PET is a functional imaging technique that uses radioactive substances
known as radiotracers to visualize and measure changes in metabolic processes,
and other physiological activities including blood flow, regional chemical
composition, and absorption. The successful application of PET has been
reported in biomarker studies for psychotic disorders [31].

  * •

EEG. EEG is a non-invasive neuroimaging technique to measure electrical
activity from neuronal populations via electrodes on the scalp. Due to its low
cost and ease of data collection, EEG provides a practical tool to be used in
a variety of clinical environments [32].

  * •

MEG. MEG is an imaging technique that uses sensitive devices called
superconducting quantum interference devices to measure the magnetic fields
produced by electrical activity in the brain. MEG is less affected by
secondary currents and demonstrates superior spatial resolution compared with
EEG [33].

  * •

ECoG. ECoG can be viewed as invasive intracranial EEG that measures brain
signals from the cortical surface. Because the subdural electrodes are placed
directly on the cortical surface, the ECoG signals have better signal-to-noise
ratio (SNR) and excellent spatial and spectral resolution compared with EEG
[34]. ECoG has a long history of use in clinical neurosurgery since its
initial application in epilepsy surgery, and has been recently used in
research studies of mental disorders, such as depression and mood disorders
[35, 36]. Micro-ECoG ($\mu$ECoG) utilizes micro-scale electrodes with contact
site diameters many orders of magnitude smaller than traditional clinical ECoG
electrode sites and minimized inter-electrode spacing, allowing for even
greater spatial resolution of the measured brain signals.

  * •

fNIRS. fNIRS is an optical brain monitoring technique that uses near-infrared
spectroscopy to measure cortical hemodynamic activities for functional
neuroimaging [37]. Alongside EEG, fNIRS can also be used in portable settings
for studying psychiatric disorders [38].

###  2.3 Neuroimaging Analysis

The rich neuroimaging modalities allow us to comprehensively probe brain
functions. Numerous research efforts have been devoted to revealing the
neurobiological basis of various psychiatric disorders using sophisticated
neuroimaging analyses. These research efforts have focused on the following
aspects.

  * •

Task-related brain activation. Under specifically designed cognitive
paradigms, the collected neuroimaging data enable us to examine brain
activities associated with certain experimental tasks and study their
relationship with cognitive dysfunctions. Typical measures of task-related
brain activities include event-related potential and event-related spectral
perturbation, and reward or emotional processing-related functional activation
[39, 40].

  * •

Brain structural and functional connectivity. A promising direction for
probing brain function using neuroimaging is to investigate brain connectivity
(also brain networks or connectome) [41]. Studying the resting-state brain
connectome provides a promising way to characterize the complex brain
architecture and uncover brain dysfunctions in intrinsic brain networks [42].

  * •

Brain dynamics. Increasing neuroimaging studies suggest that functional
connectivity may fluctuate, rather than being stationary during an entire
session of data collection [43]. Studies examining spatiotemporal dynamics of
brain networks have recently received increasing attention and may reveal
meaningful brain states associated with different psychiatric conditions [44].

  * •

Multimodal neuroimaging. Another promising approach to establish robust
biomarkers for psychiatry is to combine multiple neuroimaging modalities.
Multimodal neuroimaging analysis offers opportunities to exploit cross-
modality complementary information that a single modality approach may not
capture [45].

In the remaining sections, we argue that neuroimaging, when combined with
modern ML and other ML-powered technologies, can provide powerful tools in
advancing diagnosis, prognosis, and intervention of psychiatric disorders.

##  3 What and How Machine Learning Can Help Psychiatry?

###  3.1 What Makes Psychiatry Different from Other Medicine Disciplines?

The nature and etiology of mental illnesses remain unclear and challenging to
study. Psychiatric disorders are typically diagnosed according to a
combination of clinical symptoms based on the Diagnostic and Statistical
Manual of Mental Disorders (DSM) or the International Classiﬁcation of
Diseases (ICD). Traditional studies for the neurobiology of psychiatric
disorders have followed a categorical classification framework based on a
case-control design whereby all patients with a given diagnosis are compared
with healthy individuals. This framework largely relies on clinically derived
diagnostic labels, assuming that a biomarker may differ enough between healthy
people and patients. The symptom-based diagnosis covered hundreds of thousands
of different symptom combinations, which has caused extensive clinical
heterogeneity [46, 47]. It is increasingly recognized that existing clinical
diagnostic categories could misrepresent the causes underlying mental
disturbance and the case-control study design has limited strength in
delineating the significant clinical and neurobiological heterogeneity of
psychiatric disorders, thereby hindering the understanding of psychopathology
and the search for biomarkers. Previous studies have broadly explored the
group effects of neurobiology to explain its connection to behavior and
disease on the populational level. The widely used group-level analyses do not
fully capture individual variability, but this variability is crucial for
developing personalized medicine and better understanding the neural
correlates of individual brain abnormalities. In addition, many psychiatric
disorders may be considered as falling along multiple dimensions. Co-
occurrence of multiple psychiatric disorders might reflect different patterns
of symptoms that result from shared risk factors and perhaps the same
underlying disease processes. The high comorbidity in these disorders
significantly affects the characterization of psychopathology according to the
traditional diagnostic categories. Conventional studies focusing on a single
diagnostic domain will therefore prevent us from uncovering the neural
correlates of comorbidity among multiple disorders and essentially identify
the dimensions of neural circuits and behavioral phenotypes.

TABLE I: Categories of ML, concepts, typical methods, and their representative applications. Learning category | Concepts | Representative methods | Applications
---|---|---|---
Supervised | Learning from labeled data | SVM, random forest, | Disease diagnosis, prognosis,
| to predict class/clinical measures | sparse learning, ensemble learning | treatment outcome prediction
Unsupervised | Learning from unlabeled data to uncover | Hierarchical clustering, | Disease subtyping, normative
| structure and identify subgroups | K-means, PCA, CCA | modeling, identify behavioral
|  |  | and neurobiological dimension
Semi-supervised | Learning from both labeled and | Multi-view learning, | Multimodal analysis, Joint
| unlabeled data to perform supervised | Laplacian regularization, | disease subtyping and diagnosis,
| or unsupervised tasks | Semi-supervised clustering | Prediction with incomplete data
Deep | Learning hierarchies and nonlinear mappings | CNN, Deep autoencoder, GCN, | A large class of generic
| of features for higher-level representations | RNN, LSTM, GAN | learning problems

Distinct from the traditional case-control design, the NIMH’s RDoC aims to
address the heterogeneity and comorbidity in psychiatry by linking symptom
dimensions with biological systems, cutting across the diagnostic spectrum
[48]. The ultimate goal of RDoC is to ﬁnd “new ways of classifying psychiatric
diseases based on multiple dimensions of biology and behavior” [49]. These
newly defined disease dimensions could further be utilized to discover
neurobiological phenotypes and clarify the causal mechanism underlying the
associated brain dysfunctions. To achieve this important vision, new
analytical approaches are urgently needed. Thanks to the advancement in
cutting-edge techniques in ML/AI, psychiatrists and investigators now have an
unprecedented opportunity to beneﬁt from complex patterns in brain, behavior,
and genes using machine learning tools [5]. Combining these analysis
techniques with rich multimodal data from increasing large-scale multi-center
cohorts holds significant promise in advancing a biologically grounded
redeﬁnition of psychiatric disorders.

Despite rapid progresses in the past decade, several areas appear highly
underexplored but may carry the substantial potential for achieving
breakthroughs in psychiatry studies. The capacity to identify individual
neurobiological characteristics and dissect inter- and intra-individual
variability is a crucial step toward personalized medicine and better
understanding the neural basis of variation in human cognition and behavior
[50]. Studies and analytic approaches focusing on the level of the individual
may find greater success over conventional group-level studies. Translational
study-orientated approaches for psychiatric neuroimaging may further enhance
the ability to ﬁnd statistically significant effect sizes that can be used in
individuals [51]. On the other hand, identifying subgroups (i.e., subtypes) in
psychiatric disorders provides us with a promising way to delineate
neurobiological and clinical heterogeneity. Increasing evidence suggests that
data-driven subtyping could drive novel neurobiological phenotypes associated
with distinctive behavior and cognitive functioning [52]. These stratified
phenotypes may further show improved predictability for clinical outcomes than
DSM/ICD diagnoses can and serve as potential markers for treatment selection
[53]. Another promising area focuses on transdiagnostic approaches to uncover
neural correlates of specific domains, such as cognition, arousal, and emotion
regulation, which are implicated in psychopathology across the diagnostic
spectrum [54]. By leveraging advanced ML tools, recent efforts have been
dedicated to identifying transdiagnostic brain dysfunctions and dimensions of
psychopathology [55, 56, 57, 58]. Additionally, leveraging “big data” from a
longitudinal perspective offers a promising way to track the trajectories of
neural phenotypes that have been rarely examined in previous cross-sectional
studies of psychiatric disorders [59, 60, 61]. Investigating the association
between the trajectories of neural phenotypes and cognitive or behavioral
dysfunctions across different stages of the disease may further reveal the
neural mechanism underlying the disease progression and psychopathology,
thereby providing new insights for the development of targeted interventions.
These new frontiers in studying psychiatric disorders can be substantially
empowered by modern ML methodologies summarized in Table I, with demonstrated
potential to stratify patients into clinically meaningful subtypes, discover
novel transdiagnostic disease dimensions crossing diagnostic boundaries, and
tailor treatment decisions to individual patients. The research outcome can
deliver a significant promise in promoting the development of objective
neurobiological markers-based precision medicine in psychiatry.

In the application domain, ML can be used in four types of applications in
mental illnesses: diagnosis, prognosis, treatment, and readmission. In
contrast to most medical disciplines, traditional diagnoses in psychiatry
remain restricted to subjective symptoms and observable signs. The limitations
of this clinical practice have been criticized and the field has called for a
paradigm shift in precision psychiatry. In the following subsections, we will
review several key ML paradigms in mental health applications based on
neuroimaging, behavioral and clinical measurements. A tabular review of
representative applications is shown in Table II. In this section, we will
focus on the review of ML in psychiatry based on neuroimaging data, but the ML
applications can also cover other types of data, such as genetic, clinical,
behavioral, and social media data. Detailed reviews of the other data domains
will be reviewed separately in later sections.

TABLE II: Representative ML applications in psychiatry based on neuroimaging and clinical data. Application | Method | Disease | Data type | Reference
---|---|---|---|---
Diagnosis | Classification (dynamic GCN) | ADHD | rs-fMRI + Phenotypic data | [62]
Classification (Ensemble learning) | ADHD | Multimodal | [63]
Classification (GCN) | ASD | Task fMRI | [64]
Classification (Ensemble learning + GCN) | ASD | rs-fMRI | [65]
Classification (PCA + LASSO) | Bipolar | DWI + Cognitive data | [66]
Classification (RVM) | PTSD | rs-fMRI | [67]
Classification (ICA + LSTM) | Schizophrenia | fMRI | [68]
Classification (SVM) | Schizophrenia | sMRI | [69]
Classification (CNN) | Depression | rs-EEG | [70]
Classification (Autoencoder + MLP) | ASD | rs-fMRI | [71]
Classification (GNN) | ASD | rs-fMRI + Phenotypic data | [72]
Subtyping (Normative modeling + clustering) | PTSD | rs-fMRI | [73]
Subtyping (CCA + Hierarchical clustering) | Depression | rs-fMRI | [52]
Subtyping (Sparse K-means) | PTSD and Depression | rs-EEG | [53]
Subtyping (Latent class analysis) | ADHD | Task fMRI | [74]
Transdiagnostic (Normative modeling + GP regression) | Multiple disorders | rs-fMRI | [75]
| Transdiagnostic (Sparse CCA) | Multiple disorders | rs-fMRI | [56]
| Transdiagnostic (PLS) | Multiple disorders | rs-fMRI | [57]
Prognosis | SVM | Psychosis, Depression | multimodal | [76]
LASSO | Psychosis | rs-EEG | [77]
SVM | Depression | rs-EEG | [78]
GP classifier | Depression | Task fMRI | [79]
LASSO | Substance use | MRI/task fMRI | [80]
LSTM | PTSD | MEG | [81]
DNN | PTSD | rs-fMRI / task fMRI | [82]
SVM | Schizophrenia | sMRI | [83]
| MLP | Schizophrenia | Task fMRI | [84]
Treatment | Latent space learning | Depression | rs-EEG | [85]
prediction | RVM | Depression | Task fMRI | [86]
SVM | Psychosis | sMRI | [87]
SVM + GP classifier | Depression | sMRI | [88]
SVM | ADHD | sMRI | [89]
GP classifier | PTSD | MRI/rs-fMRI | [90]
MVPA regression | ASD | Task fMRI | [91]
LASSO | Anxiety | rs-fMRI | [92]
SVM | Schizophrenia | rs-fMRI | [93]
Readmission | SVM | Depression | multimodal | [94]
Classification tree | Bipolar | EHR | [95]
Ensemble learning | Substance use | Phenotypic data | [96]
Growth mixture modeling | Depression | Clinical data | [97]

###  3.2 Supervised and Unsupervised Learning

ML holds substantial promise in promoting the research from small case-control
studies to large transdiagnostic samples, from prior specified brain regions
to whole-brain circuit dysfunction for individual-level precision medicine
[98, 24, 99]. In a new era of evidence-based psychiatry tailored to individual
patients, objectively measurable endophenotypes identified using advanced
data-driven approaches could allow for early disease detection, personalized
treatment selection, and dosage adjustment to reduce the burden of disease
[10, 100, 101]. These significantly promising applications in psychiatric
disorders have become possible by leveraging the powerful strength of modern
machine learning techniques [13, 102, 103, 104].

Supervised Learning. Supervised learning, as the most popularly used category,
has been widely applied to neuroimaging-based predictive modeling tasks in
psychiatric disorders [105]. These classic machine learning algorithms include
logistic regression, support vector machine, and random forest. Given the
high-dimensional nature of neuroimaging data, these approaches are commonly
accompanied by a feature selection step to obtain low-dimensionality
representations. Connectome-based predictive modeling [106, 107] is one of
such approaches that combine simple linear regression and feature selection to
predict individual differences in traits and behavior from connectivity data.
LASSO provides an alternative approach that embeds feature selection into the
prediction model training to learn a compact feature pattern for the accurate
prediction of a specialized brain disease or clinical outcome [92]. Relevance
vector machine (RVM) is an approach built upon a probabilistic framework by
leveraging automatic relevance determination (ARD) to learn a sparse solution
and penalize unnecessary complexity in the model [108, 109]. RVM has recently
demonstrated its strength in quantifying neuroimaging biomarkers for the
diagnosis of PTSD [67] and for the prediction of depression [86]. As an
extension of the conventional single-task methods, multi-task learning
approaches have been increasingly designed to exploit the complementary
features jointly from multiple views of neuroimaging data [110, 111, 112].

Due to the complex nature of brain function, informative features may not be
observable in the raw high-dimensional feature space. To address this
challenge, latent space-based supervised learning has been increasingly
applied to uncover latent dimensions of neural circuits in psychiatric
disorders. For example, a sparse latent space regression model was developed
and tailored for EEG data to discover antidepressant responsive brain
signatures in major depression [85]. By jointly learning the spatial mapping
and regression weights, the machine learning model was able to successfully
amplify the signal-to-noise ratio and quantify treatment-predictive signatures
in a low-dimensional latent space. To address the comorbidity among
psychiatric disorders, dimensional approaches have been developed using
powerful multivariate statistical models capable of discovering the complex
linear relationship between high-dimensional datasets. For instance, low-
dimensional representations of depression-related connectivity features have
been successfully identified using canonical correlation analysis (CCA)
between resting-state fMRI (rs-fMRI) connectivity and clinical symptoms [52].
These newly discovered representations defined two disease dimensions
corresponding to an anhedonia-related component and an anxiety-related
component, respectively. Similar dimensional analysis was also utilized to
examine the neural correlates of neuropsychiatric symptoms in dementia and
captured latent modes with the distinct neuroanatomical basis of common and
mood-specific factors [113]. A sparsity-constrained CCA approach has been
applied to reveal linked dimensions of psychopathology and functional
connectivity in brain networks [56]. This approach sucessfully identified
interpretable dimensions involving mood, psychosis, fear, and externalizing
behavior guided by neural circuit patterns across the clinical diagnostic
spectrum. The partial least squares (PLS) approach was also applied to
identify latent components linking a broader set of behavioral measures to
whole-brain resting-state functional connectivity [57]. The latent components
defined distinct dimensions with dissociable brain functional signatures,
providing potential intermediate phenotypes spanning diagnostic categories.
These new dimensional analytics hold substantial promise in uncovering novel
transdiagnostic phenotypes for the development of targeted interventions.

Ensemble Learning. Though extensive ML approaches have been designed for
supervised learning, using a single model may not produce the optimal
generalization performance for a specific prediction task. By combining
multiple ML models to decrease variance and bias, ensemble learning improves
prediction performance over a single model and has demonstrated successful
applications in the robust discovery of biomarkers for various psychiatric
disorders. For instance, multiple brain parcellation ensemble-learning
algorithms have been proposed for improved schizophrenia detection [114] and
ASD diagnosis [65]. By utilizing multimodal neuroimaging including sMRI, fMRI,
and DTI, a bagging-based SVM was devised to yield a significant improvement in
the prediction of adult outcomes in childhood-onset ADHD [63]. Based on the
selective ensemble algorithm, a sparse multi-view prediction model has been
designed with rs-fMRI connectivity for ASD diagnosis [115]. The model combines
multiple classifiers under a bootstrap framework and significantly
outperformed other single-model approaches for ASD detection.

Unsupervised Learning. Unsupervised learning aims to uncover the intrinsic
data structure by either identifying potential clusters (e.g., using latent
class analysis or K-means clustering) or learning a feature mapping that
captures the most variance of the data (e.g., using PCA). Identifying patient
subtypes offers a promising strategy to delineate neurobiological
heterogeneity in psychiatric disorders [46]. With rs-fMRI, hierarchical
clustering was applied to successfully identify four subtypes of functional
connectivity in depression [52]. These subtypes were found to correlate with
differing clinical-symptom profiles and predict responsiveness to brain
stimulation therapy. From rs-EEG, two psychiatric subtypes were identified
using sparse K-means clustering with distinct power envelope connectivity
patterns and found to respond differently to antidepressant medication and
psychotherapy [53]. As a non-distance probability-based clustering approach,
latent class analysis has also been applied to discover homogeneous subgroups
in psychiatric disorders. A proof-of-concept study was conducted using latent
class analysis to identify ADHD subtypes from fMRI activation profiles [74]
and reveal that the subtype with attenuated brain activity showed fewer
behavior problems in daily life. By leveraging data resources from multiple
time points, psychiatry studies have been shifting from cross-sectional
analysis to longitudinal modeling [24]. Finite mixture modeling became
increasingly popular for the analysis of longitudinally repeated measures
data, which can identify latent classes following similar paths of temporal
development [116, 117]. Typical finite mixture models include growth mixture
modeling, group-based trajectory modeling, and latent transition analysis. The
use of latent growth mixture modeling (LGMM) and group-based trajectory
modeling has been growing in various psychiatric disorders, such as
depression, anxiety, and ASD, as flexible ways of identifying latent
subpopulations that manifest heterogeneous symptom trajectories [118, 119,
120]. LGMM approaches have also been successfully used to predict PTSD course
among the population at risk [121]. As an extension of latent class analysis
to longitudinal data, latent transition analysis has been applied to predict
longitudinal service use for individuals with substance use disorder [122].
Together, these approaches provide powerful tools to delineate longitudinal
heterogeneity and the corresponding distinctive phenotypes during the course
of psychiatric disorders.

Another way to characterize the heterogeneity can be achieved by quantifying
individual deviations using unsupervised normative modeling [123]. Through
building a normative model on a large-scale healthy population, brain
abnormalities of individual patients can be quantified by examining their
statistical differences from the distribution of the norm. Gaussian process
(GP) regression-based normative modeling has been increasingly applied to
quantify individual deviations and dissect neurobiological heterogeneity in
various psychiatric disorders [124]. With this tool, an association was
successfully discovered between transdiagnostic dimensions of psychopathology
and individual’s unique deviations from normative neurodevelopment in brain
structure [75]. By combining tolerance interval-based normative modeling and
clustering analysis, individual abnormities in rs-fMRI were quantified to
define two stable subtypes in patients with PTSD [73]. These subtypes showed
distinct patterns of functional connectivity with respect to healthy people
and differed clinically on levels of reexperiencing symptoms. These studies
provide novel data-driven approaches to delineate heterogeneity within
patients, thereby advancing clinical and mechanistic investigations in
psychiatric disorders.

Semi-supervised Learning. Semi-supervised learning is an ML approach that
combines a small amount of labeled data with a large amount of unlabeled data
during training. Popular semi-supervised learning techniques include self-
training, mixture models, co-training and multi-view learning, graph-based
methods, and semi-supervised SVM [125]. These paradigms have been increasingly
applied to psychiatry studies. For instance, semi-supervised learning has been
incorporated into an SVM framework to form a method called heterogeneity
through discriminative analysis, which can achieve joint disease subtyping and
diagnosis [126]. This approach has shown strength in delineating
neurostructural heterogeneity in bipolar and major depressive disorders [127],
schizophrenia [128], as well as in youth with internalizing symptoms [129]. By
unifying autoencoder and supervised classification, an unsupervised
autoencoder model was also developed for improved diagnosis of ASD [130].
Additionally, semi-supervised learning has gained increasing mental health
applications in digital data from electronic health records (EHRs), social
media and mobile phones [131, 132, 133]. See Section 4 for a detailed
discussion.

Although complex supervised learning models often produce better
classification or prediction outcomes, their interpretability also decreases
with the increasing complexity. We will discuss the interpretable ML methods
in more detail later (Section 6). Additionally, the labeled data require the
knowledge of the ground truth, which is not always accurate or reliable in the
case of mental illnesses. For instance, the skin cancer diagnosis may rely on
training samples that have been biopsied and cataloged, leaving no doubt as to
whether they are malignant or not; however, there is no equivalent of the
biopsy in mental disorder. In contrast, unsupervised learning relaxes the
assumption of labeled samples and can be useful for exploratory data analysis.
Unsupervised learning models can be either generative or discriminative.

Figure 2: Various ML models for mental health applications. (A) Left:
Multimodal supervised classification scheme. Three modality-specific factors
are optimized on the training data: classifier types, parameters and weights.
The final diagnostic classification is based on a weighted sum of decision
values, where weights correspond to those estimated during training. Right:
Feature importance maps of functional neuroimaging modalities ([134]). (B)
Unsupervised learning. Left: whole-brain functional-connectivity matrix
averaged across all subjects. z = Fischer transformed correlation coefficient.
Right: Hierarchical clustering analysis. ($\copyright$ Springer Nature, [52].
Figure reproduced with permission). (C) Semi-supervised learning pipeline for
phenotype stratification based on EHRs ([135], Figure reproduced with
permission) (D) Deep neural networks (DNNs) for group-level and individualized
treatment predictions. Future data points could then be used to forecast
symptom onset, treatment response, or other mental health-related variables
([18], Creative Commons licenses 4.0).

###  3.3 Deep Learning

Deep learning describes a collection of methods that use artiﬁcial neural
networks for various tasks in machine learning. Through a specifically
designed deep neural network structure, high-level feature representation can
be learned from raw features, which may offer an end-to-end analytic framework
for disease analysis and prediction. With the advancement in imaging
technologies, an increasing number of large-scale multi-center datasets have
been established for building robust ML models to fully explore the
informative feature representations from the complex brain and genomic data.
By training on large-scale cohorts, deep learning encodes robust informative
neuroimaging representations and has been found to outperform standard machine
learning in a variety of application scenarios [136, 137, 18, 138].

Deep autoencoder. The deep autoencoder, also known as stacked autoencoder
(SAE), aims to learn latent representations of input data through an encoder
part and uses these representations to reconstruct output data through a
decoder part. By stacking multiple layers of autoencoders, a deep autoencoder
model is formed to discover more complicated and potentially nonlinear feature
patterns. The deep autoencoder model has been applied to extract low-
dimensional features from the amplitude of low-frequency fluctuations in fMRI
[139]. Clustering analysis with the latent features uncovered by the deep
autoencoder successfully identified two subtypes within major psychiatric
disorders including schizophrenia, bipolar disorder, and major depressive
disorder. A deep learning model was also designed based on a sparse stacked
autoencoder and applied to lower the dimensionality of fMRI connectivity. The
sparsity constraint utilized in this model drove interpretable neural patterns
for improved ASD diagnosis [140]. Deep autoencoder has also been applied to
implement normative modeling with structural MRI for the quantification of
individual abnormalities in neuropsychiatric disorders, including
schizophrenia and ASD. The abnormal features extracted using the normative
model showed improved diagnosis performance compared with the traditional
case-control analysis [141].

Convolutional neural networks (CNNs). Different from conventional multi-layer
perception or autoencoder working on inputs of feature vectors, CNNs were
designed to better capture the spatial and local structure information from
pixels or voxels [142, 104]. The CNN has been one of the most successful deep
learning models applied in various medical applications, due to its strength
in exploiting image features and spatial context utilizing neighborhood
information to learn hierarchies of features [143]. An EEG-based diagnosis
model was established through advanced computational neuroscience methodology
coupled with the CNN for accurate detection of major depression disorder (MDD)
[70]. This model provided an end-to-end framework to successfully identify
translational biomarkers from resting-state EEG in distinguishing depressive
patients from healthy people. With whole-brain structure MRI, a 3D CNN model
has also been designed to automatically extract multilayer high-dimensional
features for the diagnosis of conduct disorder [144].

Graph neural networks (GNNs). Though deep learning models have shown strength
in capturing complex neuroimaging patterns, they are hard to generalize well
to non-Euclidean data types, e.g., brain network or genome data. Relying on
its non-Euclidean data type, GNNs provide a clever way of learning the deep
graph structure and are rapidly becoming state-of-the-art and are leading to
enhanced performance in various network neuroscience tasks [145]. For
instance, a framework based on graph convolutional networks has been designed
for the diagnosis of ASD [72]. By representing the population as a sparse
graph to integrate rs-fMRI data as nodal features and phenotypic measures as
edges, the designed model showed improved classification performance over
state-of-the-art results. An inductive graph neural network model was also
devised to embed the graphs containing different properties of task fMRI and
drive interpretable connectome biomarkers for improved ASD detection [64].
More recently, a novel graph network model was developed to incorporate
dynamic graph computation and 2-hop neighbor nodes feature aggregation into
graph convolution for brain network modeling [62]. This dynamic GNN
demonstrated significantly improved performance in ADHD diagnosis and reveal
the circuit-level association between the identified connectomic abnormalities
and symptom severity.

Recurrent neural networks (RNNs). As a specific extension of the feed-forward
neural network, RNN has the ability in learning features and long-term
dependencies from sequential and time-series data. Long-short-term memory
(LSTM) model is the most popular RNN and has shown its advantage in capturing
temporal dynamic information of neuroimaging data for various psychiatric
disorder studies [146]. An LSTM-based RNN architecture was built with the time
course of fMRI independent components to exploit the temporal dynamic
information and validated to yield an improved diagnosis of schizophrenia
[68]. By combining RNN with other deep neural networks, novel machine learning
models have also been proposed to model the spatio-temporal dynamics in
neuroimaging data. A spatio-temporal CNN model was proposed for 4D modeling of
fMRI data with confirmed robustness in identifying the key features in the
default mode network [147]. LSTM has also been successfully applied to
incorporate multi-stage neuroimaging data into longitudinal analytic
frameworks for modeling the trajectories of psychopathology development in
various psychiatric disorders. A recent LSTM-based model was built with MEG
data to achieve accurate longitudinal tracking of pathological brain states
and prediction of clinical outcomes in PTSD [81].

Generative Adversarial Networks (GANs). GAN belongs to a type of generative
model and has gained a lot of attention in computer vision, natural language
processing; the method has become increasingly popular for conducting
neuroimaging analysis [104]. GAN consists of two competing neural networks
(one as generator and the other as discriminator) and can learn deep feature
representations without extensively labeled training data. Due to this unique
advantage, GAN has been increasingly applied to perform data augmentation for
more robust prediction model training with enhanced sample size [148].
Moreover, GAN has been used to impute missing values in multimodal data, a
common problem in psychiatry studies, rather than discarding an entire
multivariate data point [149]. The adversarial model has also been
incorporated into other ML models for specific applications in psychiatry
studies. For instance, the discriminative and generative components were
incorporated in LSTM to form a multitask learning approach for fMRI-based
classification and achieved an improved diagnostic accuracy of ASD compared
with the standard LSTM model [150]. By integrating GAN with group ICA, a
functional network connectivity-based deep learning model was developed for
the diagnosis of major depressive disorder and schizophrenia [151]. The
generator with fake connectivity was trained to match the discriminator with
real connectivity in the intermediate layers while simultaneously a new
objective loss was determined for the generator to improve the diagnosis
accuracy.

The strength of deep learning algorithms is that they can implement complex
predictor-response mappings, but the power also comes at the cost of requiring
a very large sample size to feed a large number of model parameters. This
poses some overfitting and result interpretation challenges in psychiatric
applications [18].

Figure 3: Concepts and major findings in case studies 1 and 2. (A)
Illustration of the sparse EEG latent space regression (SELSER) framework in
Case Study 1 for treatment outcome prediction. (B) Interpretable cortical
pattern derived from the scalp pattern ($\copyright$ Springer Nature, figures
are modified from [85] with permission). (C) Distinctive EEG connectivity
profiles were identified by sparse K-means for defining psychiatric subtypes
in Case Study 2 on PTSD and MDD. The two identified subtypes were further
found to predict treatment responsiveness to psychotherapy and antidepressant
medication. (D) The EEG connectivity-defined subtypes are distinguishable by
fMRI connectivity patterns derived from an RVM-based classifier ($\copyright$
Springer Nature, figures are modified from [53] with permission).

###  3.4 Key ML Concepts for Precision Psychiatry

Regardless of the ML paradigms in psychiatric applications, there are some
common themes that distinguish between automated or human-in-the-loop machine
intelligence and human intelligence. In a recently published white paper
“Machine intelligence for healthcare”, four important features are emphasized
for ML systems [152]. These concepts are equally applicable to precision
psychiatry [11].

  * •

Trustworthiness – the ability to access the validity and reliability of an ML-
derived output across varying inputs and environments. In other words,
psychiatrists need to be able to evaluate the limitations of an ML system and
confidently apply system-derived information for psychiatric evaluation.

  * •

Explainability – the ability to understand and evaluate the internal mechanism
of a machine. The development of ML systems will need to account for data
quality, quality metrics for the system’s functioning and impact, standards
for applications in the environment, and future updates to the system.

  * •

Usability – the extent to which an ML system can be used to achieve specified
goals with effectiveness, efficiency, and patient satisfaction in multiple
environments. These applications need to be scalable across multiple settings
while preventing additional burdens on providers and patients.

  * •

Transparency and Fairness – the right to know and understand the aspects of an
input that could influence outputs (clinical decision support) from the
system. Such factors should be available to the people who use, regulate, and
are affected by any type of care decision that employs the ML algorithms. The
potential bias in the data needs to be identified and informed prior to
decision making.

The first two features are related to interpretability, which we will discuss
in more detail in Section 7. The other two features will be discussed in
Section 8.

Figure 4: Illustrations of concepts and major findings in case study 3. (A)
Experimental paradigm. (B) Schematic of the CNN. (C) AUC curves produced by
CNN vs. chance level. (D) The prediction score positively correlated with the
anxiety sensitivity index (ASI) for the control group ($r=0.41$, $p<0.001$),
but at the chance level for anxious brains ($r=-0.05$, $p=0.65$). (E)
Distribution of AUCs based on brain activations within the 10-node fear
randomly selected brain regions (Figures were adapted from [153] with
permission). Figure 5: Illustrations of concepts and major findings in case
study 4. (A) Schematic of cross-validation. An IMS point (e.g., $S_{N}$) is
left out as the test IMS to be predicted. The other IMS points (i.e., training
IMS, using $S_{1}$ to $S_{N–1}$) and the associated neural activity are used
within the modeling framework to train a neural encoding model. (B) Main
components of the modeling framework based on both unsupervised and supervised
learning. (C) Cross-validated prediction of the mood state is shown against
the true measured mood state ($\copyright$ Springer Nature; Figures were
modified from [35] with permission).

###  3.5 Case Studies

In this subsection, we provide several ML examples to illustrate the strengths
in prediction/classification diagnosis analytics for different mental
disorders. These representative case studies cover different data modalities
(rs-EEG, task fMRI, and ECoG) using different ML strategies.

Case Study 1: Sparse latent space learning for treatment-predictive EEG
signature quantification in depression. Antidepressants have shown only modest
superiority over placebo medications, which is in part because the clinical
diagnosis of MDD encompasses biologically heterogeneous conditions that relate
differentially to treatment outcomes. There is a robust neurobiological
signature for an antidepressant-responsive phenotype that could identify which
patients will derive a large benefit from medication. To address the
challenge, a sparse EEG latent space regression (“SELSER”) model, Wu et al.
successfully identified the antidepressant responsive EEG brain signature in
major depression (Fig. 3A) [85]. By jointly learning the spatial mapping and
regression weights, this new machine learning tool was able to amplify the
signal-to-noise ratio (SNR) and quantify treatment-predictive signatures in a
low-dimensional latent space. The identified brain signature yielded
significant prediction for the antidepressant outcome ($n=228$), which holds
promise in benefiting treatment stratification toward the improved clinical
outcome. An interpretable cortical pattern was further observed through a
source mapping from the scalp spatial pattern, mainly contributed by the right
parietal-occipital regions and the lateral prefrontal regions (Fig. 3B). The
validation analysis with an independent cohort showed that treatment outcomes
predicted by the brain signature are significantly higher in the partial
responder group versus the treatment-resistant group demonstrating its
replicability.

Case Study 2: Unsupervised learning-based identification of neurophysiological
subtypes in psychiatric disorders. The neurobiological heterogeneity has
substantial effects on treatment outcome even while often being independent of
pre-treatment clinical symptoms. For example, although psychotherapy is at
present the most effective treatment for PTSD, many patients are nonetheless
non-responsive and display differences in brain function relative to
responsive patients. With sparse K-means clustering, Zhang et al. [53]
developed a data-driven subtyping framework to accomplish simultaneous feature
selection and sample clustering on the high-dimensional power envelope
connectivity of rs-EEG source-reconstructed signals. This approach
successfully identified two psychiatric subtypes with distinct functional
connectivity patterns in PTSD and MDD ($n=648$), prominently within the
frontoparietal control network and default mode network (Fig. 3C).
Importantly, linear mixed models in an intent-to-treat analysis on symptom
severity revealed that the two subtypes differentially responded to
psychotherapy and antidepressant versus placebo. A classification analysis
based on RVM further confirmed that the EEG connectivity-driven subtypes can
be distinguished by rs-fMRI connectivity with a discriminative pattern
consistent with the EEG connectivity pattern (Fig. 3D). The data-driven
subtyping framework allows the detection of subpopulations with distinct
dysfunctional signatures so that further clinical and mechanistic
investigations can be focused on more neurobiologically homogeneous subtypes.

Case Study 3: Classification of anxious vs. non-anxious brains from fear
extinction learning task-based fMRI. Using a neuroimaging cohort study
($n=304$ adults, 92 anxiety patients, 74 trauma-exposed individuals, 138
matched controls), Wen et al. [153] examined how the fMRI activations of 10
brain regions that were commonly activated during fear conditioning and
extinction (Fig. 4A) might distinguish anxious or trauma-exposed brains from
controls. They proposed a CNN classifier (Fig. 4B) to map the fear-induced
fMRI activities in space and time to a prediction probability score indicating
that the subject belongs to the anxious (or trauma) group. The CNN achieved an
AUC of $0.84\pm 0.01$, $0.75\pm 0.03$ sensitivity, and $0.77\pm 0.02$
specificity in 5-fold cross-validation (Fig. 4C), outperforming other ML
methods (e.g., SVM, random forest). The prediction score was predictive of the
anxiety sensitivity index (ASI) in the control group (Fig. 4D). Three
different specificity analyses were conducted to examine the specificity of
the fear network in discrimination. First, the activations of ten brain
regions were randomly selected from the whole brain for classification.
Second, 10 regions from the somatomotor network were randomly selected for
classification (Fig. 4E). Third, $N$ ($1\leqslant N\leqslant 9$) brain regions
from the fear network were replaced with $N$ randomly selected brain regions
outside of the fear network.

Case Study 4: Decoding mood state from multi-site intracranial brain activity.
From intracranial ECoG signals and simultaneously collected self-reported mood
state measurements over multiple days in 7 epilepsy patients, Sani et al. [35]
developed a dynamic state-space model (SSM) framework to track the patients’
mood state variations over time (Fig. 5A). The modeling framework consists of
unsupervised and supervised learning components (Fig. 5B). The spectro-spatial
features were extracted from the mood-predictive networks within the limbic
brain region. The neural decoders were also highly predictive of the immediate
mood scaler (IMS) points at the population level. Furthermore, the same
trained decoder could be used for mood state prediction across hours and days,
and generalized across a wide range of IMS. In cross-validation, the decoders
could predict IMS variations that covered 73% and $33\pm 7.2$% of the total
possible IMS range across 7 subjects and within individuals, respectively
(Fig. 5C). These results suggest that ML-based decoders can predict mood state
variations from neural activity across multiple days of recordings in
patients.

##  4 ML-powered Technologies for Psychiatry

The ML technologies can be applied to a wide range of digital platforms,
including software (e.g., mobile apps), hardware (e.g., wearable devices,
robots), social services (e.g., online chatbots) and clinical practice (e.g.,
EHRs). In this section, we will review various ML-powered technologies in the
non-neuroimaging domains and discuss their applications for precision
psychiatry. We will highlight the emerging digital platforms and their
underlying ML technologies.

A recent McKinsey study showed that use of telehealth has increased by 38-fold
as compared to the pre-COVID baseline [154]. With a steep increase in
teletherapy demand and consumption, many companies (such as Talkspace and
Headspace Health) provide chat conversations with licensed mental health
professionals, the definition of teletherapy has expanded to include these
newer modalities of care delivery. These advances in care delivery have
enabled collecting massive amounts of text, audio and video data on a routine
basis, which was previously only done in controlled research settings.
Furthermore, the recent advancements of natural language, speech, and video
analysis technologies, combined with the ML tools, have made the field ripe
for innovation. The global psychiatrist community is increasingly aware of
these developments. For example, a recent survey among more than 700
psychiatrists showed that 49% believed that in the next 5-10 years, ML
technology will be able to help analyze patient information to establish
prognosis and 54% believed that is likely that this technology can help
synthesize patient information to reach a diagnosis [9].

ML can be applied across all stages of a patient’s journey [155, 156]: risk
assessment, diagnosis, prognosis, treatment, and remission in a variety of
disorders, including: depression, anxiety, schizophrenia, PTSD, and so on
[157] to various sources of data [14, 158] including: natural language,
speech, facial expressions, body language, social media, as well as
traditional clinical surveys and neuroimaging results. Table III summarizes
key recent studies that use ML to support various stages of patient journey.
Applying ML makes it possible to build models that are optimized for each
patient [5], as opposed to using traditional statistical models that optimize
for group effects. While building ML models, data for representative patients
can be used as held-out validation or test sets. Furthermore, given the inter
and intra-disease variability between clinical diagnosis and symptoms, ML can
be used to model the differential diagnosis between disease categories using
methods like multi-task learning. All of these ways of ML applications can be
considered to be the first level of precision added to ML-powered psychiatry.

Figure 6: (A) Two levels of precision in applying ML for mental health. (B)
Example application of ML at various stages of a patient’s journey in case of
major depressive disorder (MDD).

The amount of precision that can be modeled using ML goes beyond this [159,
160]. During psychiatric evaluation, psychiatrists are trying to build a
mental model of what is going on in the patient’s life in about 30 minutes.
They are trying to understand as much as possible about the patient’s history
in a very short time and define what “normal” looks like for the patient and
identify deviations from that normal. This is done by asking questions to the
patient and examining their speech, body language, and their behavioral
responses. It is very challenging and almost unrealistic to expect
psychiatrists to build an accurate baseline model of the patient’s entire life
in such a short time span whilst interacting with the patient who is in a
compromised psychological state. ML can have a great impact here by building
baseline models specific to each patient before their visit and present the
bounds for various observations as a reference to psychiatrists during the
exam [161]. This is the second level of precision in psychiatry that can be
made possible by ML (Fig. 6A). Take MDD as an example, Figure 6B shows how ML
can be applied at different stages of a patient’s journey. Similar
applications have also been developed in studies of other disorders.

In the following subsections, we describe how ML technologies can be applied
to clinically relevant data (e.g., speech, body language, natural language,
activity, online behaviors, etc) and to support one or more stages of the
patient’s journey.

TABLE III:  Representative ML applications of multimedia data in mental disorders. Study | Data source(s) | Patient journey stage | ML approach | Test sample size
---|---|---|---|---
Depression spectrum |
[162] | Audio - clinical interviews | Diagnosis | CNN ensemble |  $47$ speakers
[163] | Audio - answers to personal questions | Diagnosis | Transfer learning |  $3078$ speakers
[164] | Audio - clinical interviews | Diagnosis | SVM w/ speech landmark features |  $47$ speakers
[165] | Face video -reading & personal questions | Diagnosis | CNN |  $50$ videos
[166] | Gait-only video - casual walking in a corridor | Diagnosis | LSTM+CNN weighted fusion |  $40$ videos
[167] | Language - answers to personal questions | Diagnosis | LSTM fine-tuned w/ health forum data |  $2425$ subjects
[168] | Language - Facebook posts | Risk assessment | Logistic regression |  $68$ patients
[169] | Audio, video - clinical interviews | Diagnosis | Transformer + multimodal fusion |  $56$ subjects
Bipolar spectrum |
[170] | Audio - verbal fluency tasks | Remission | SVM | 56 subjects
[171] | Sensor - GPS | Diagnosis | Linear regression | 36 subjects
PTSD |
[172] | Audio - clinical interviews | Diagnosis | Random forest | 43 veterans
[173] | Audio, video, skin conductance | Remission | SVM | 110 subjects
Schizophrenia spectrum |
[174] | Audio - clinical interviews | Diagnosis | SVM | 70 subjects
[175] | Video - neutral open-ended questions | Diagnosis | Logistic regression | 16 subjects
[176] | Language - internet search queries | Remission | Random forest | 23 subjects
[177] | Audio, video - clinical interviews | Diagnosis | Gradient boosting | 17 subjects

###  4.1 Mobile and Sensing Technologies

With the development of smart phones, smart watches and other wearable sensing
devices, technologies have enabled us to access more information of our
physical and mental health than ever [178]. Specifically, several types of
signals are relevant for mental health monitoring or assessment (Fig. 7A):

  * •

Behavioral and physical signals: location (e.g., GPS coordinates), mobility
(e.g., accelerometer)

  * •

Multimedia signals: face expression, speech patterns

  * •

Social signals: social interactions (e.g., call and text message logs),
communication patterns, engagement, online gaming

  * •

Physiological signals: skin conductance, hear rate variability (HRV), eye
movement, electrodermal activity (EDA)

  * •

Sleep activity: phone on/off status, sleep duration, sleep staging

These signals have different implications and relevance to mental illnesses.
Although none of single signals is indicative of mental disorders, combination
of these physical/physiological/social cues may reveal important clues of
individual mental health (as compared to the control of his or her own). In
the subsequent subsections, we will focus on the analysis of multimedia,
language, and social media data and on their mental health applications.

Figure 7: Illustrations of ML-powered technologies for mental health. (A) ML
applications in mobile health. (B) Different types of data collection
strategies for digital measurement tools. (C) A technological infrastructure
for the integration of digital measurement tools. Independent platforms for
measurement of health will have their own data repositories, depicted as
clouds. This data could be safely transferred across platforms using transfer
tools such as secure APIs (application program interfaces), depicted using
dashed arrows. Such tools could allow for both unidirectional and
bidirectional movement of data. ML can be applied to integrate all measures
for clinical decision-making (Figure panels B and C are reproduced from [158]
with permission).

###  4.2 Speech and Video Analyses

Voice and visual (video of facial expressions and body language behaviors)
data have recently gained increasing attention in the studies of mental
disorders. ML technologies using speech samples obtained in the clinic or
remotely may help identify biomarkers to improve diagnosis and treatment. In
the early stage of practice, psychologists have already used auditory and
visual cues to assist the mental illness diagnosis [179]. Among all the types
of data that may carry biomarkers for mental illness, speech and video are not
only the readily available in traditional teletherapy settings but also the
most interpretable as the most natural form of human communication.

Audio and speech features. Features derived from audio data (acoustic
features) have been found to be relevant in many mental health disorders [172,
180], with depression, bipolar, and schizophrenia being the most commonly
studied disorders with regards to speech analysis. Table IV lists some
commonly used acoustic feature categories in the analysis of mental illnesses
[181]. These categories have enabled standardization and ease of
interpretation of ML methods applied on speech data, which is the key for
making ML methods useful for clinical applications.

Models built from speech-based features, have been particularly effective in
predicting the diagnosis of depression and suicidality [182]. Modeling
approaches for depression include predicting the presence, severity, or score
with varying levels of success [162, 170]. These models use prosodic, spectral
or other features computed from raw speech data to quantify flattered speech,
slow speech and other relevant markers. The target outcome variable is derived
from a clinically valid scale such as a patient health questionnaire (PHQ-9).
Furthermore, models for suicidality have explored similar features and have
been used in multi-class settings to differentiate between healthy, depressed,
and suicidal speech.

One key challenge in applying speech-based models in clinical practice is the
lack of longitudinal validation of data acquired in real-world settings.
However, this issue is beginning to get addressed in studies like [183], which
detect manic and depressive speech from recordings of outgoing speech from
phone conversations of consenting participants. Another key challenge remains
is the lack of large labeled datasets for evaluating performance across
various methods. To this end, it is encouraging to note that recent efforts
backed by companies like Ellipsis Health [184, 163], have used deep learning
and transfer learning to predict depression and anxiety scores with a high
performance based on a large labelled dataset of over 10,000 unique speakers.
Some commercial applications claim to have human-level accuracy in detecting
depression using only 20-30 seconds of audio clip [185, 186].

Visual features. Although body language and facial expressions form a key part
of a psychiatric exam, ML has only recently been applied to analyze such data
objectively. To date, most work has been targeted to support the care for
suicidal ideation [187] and depression [165, 188, 166], with few studies also
dedicated to schizophrenia [175] and autism spectrum disorders [189]. Features
derived from video data have been found to be relevant across all of these
disorders. These features can be roughly categorized into features derived
from overall facial expression, eyes, gait, and posture (Table IV).

Studies in suicidal ideation have mainly focused on using interpretable ML for
characterizing the disorder. This makes the ML models more applicable in
augmenting human caregivers by bringing up specific insight that they’d like
to measure. The approaches to depression have been mixed between work focused
on using deep learning for predicting the depression score and those using
predefined features in an interpretable model. Some approaches [190] have also
involved fusion of video features derived from each frame that are used to
train a sequential deep network and most have used pre-training to account for
the relatively small size of depression datasets [191]. While these models
perform very well on the same held-out test set, their clinical application
remains limited due to a lack of interpretability. To improve the
interpretability, few researchers have acknowledged this and built depression
activation maps to highlight the facial areas corresponding to depression
severity as learned by the model [192]; however, using predefined features has
been most successful in providing interpretable results [193, 191].

TABLE IV:  Multimodal data features and their uses in mental health. Features | Example(s) | Example relevant in mental disorder(s)
---|---|---
Acoustic
Source of sound features | Jitter | Increase with depression severity
Filtering features by vocal and nasal tracks | First resonant peak in the spectrum | increase with bipolar severity
Spectral features of speech | Mel frequency cepstral coefficients | A variety of disorders
Prosodic features of speech | Pause duration | Higher in SCZ
Video
Facial | Smile duration, eyebrow movement, disgust expression | Increased disgust expression in SI
Eyes | Gaze angle | More non-mutual gazes in MDD
Gait | Arm swing and stride | Reduced arm swing in MDD
Posture | Head pitch variance, upper body movements | Reduced head movement in SCZ
|  | Higher head movement in ASD
Language
Grandiosity | Unrealistic sense of superiority | Increased in bipolar
Semantic coherence | Flow of meaning | Decreased in psychosis
Rumination | Repetitive thought patterns | Increased in MDD
Self-focus | Self-referent information | Increased in stress

###  4.3 Natural Language Processing (NLP)

NLP allows computers to analyze, understand, and derive meaning from text and
speech similarly to how humans do. NLP is opening up new possibilities for how
mental health professionals can evaluate patterns in language to help identify
and predict psychiatric illness in patients (Table IV). Language being one of
the primary expressions of human behavior carries a variety of implicit and
explicit markers relevant to mental health [194, 195]. Language is also more
abundantly available compared to speech. For example, social media platforms
contain a large quantity of real-world language data, whereas the matched
scale of speech data is limited. There are two sorts of NLP applications for
detection of specific mental health symptoms. The first type of applications
is directly applied to patients: speech and language are the important
information sources used to diagnose mental disorders, varying from predicting
the risk of suicide and early psychiatric readmission to identifying
phenotypes and comorbidities. The second type of applications is indirectly
applied to EHR and clinical records (tests, transcripts), which can be used
for automating chart reviews, clustering patients into phenotype subtypes,
predicting patient-specific outcomes. EHRs (including pathology reports, lab
results, clinical tests and clinical session transcripts) are systematic
collections of longitudinal, patient-centered clinical records. Patients’ EHRs
consist of both structured and unstructured data: the structured data include
information about a patient’s diagnosis, medications, and laboratory test
results, and the unstructured data include information in clinical notes.

Massive EHR datasets have provided opportunities to adapt computational
approaches to track and identify target areas for quality improvement in
mental health care. According to a 2015 national survey, 61.3% of US
psychiatrists use EHRs [196]. The EHR language is at least one level
abstracted from the symptoms expressed directly by a patient since it consists
of notes taken by a clinician. However, the unique advantage of EHR data is
the ease with which demographic and socioeconomic features can be combined
with language data. Symptoms derived from the free text in EHR data can be
used to predict a diagnosis for disorders like bipolar [197] and symptoms like
situational aggression [198] as well as suicidal ideation [199] with
performance comparable to clinicians. Furthermore, discharge summaries from
EHRs have also been used to predict remission [200]. Aside from symptoms, a
variety of relevant mental health data such as intervention status and
physical health comorbidities can be routinely extracted from EHRs using NLP
methods to enhance outcomes research [201]. Privacy concerns around sharing
EHR data remain one of the key challenges in validating generalization of NLP
methods. Encouragingly, there has been growing interests in using transformers
for generating artificial mental health clinical notes to mitigate this issue
[201, 202].

The advances in text-based mental health interventions (e.g., Talkspace and
CrisisTextLine) have made transcripts of clinical sessions easily amenable via
NLP. Aside from developing models for detecting suicide ideation [203], NLP
can also be applied to these datasets to understand population-level trends,
such as the increase in anxiety and decrease in quality of personal
relationships during the COVID-19 pandemic [204]. Since language data are
ubiquitous, one of the challenges in applying NLP for advancing mental health
is standardizing data sources. Depending on the task, different types of data
may yield different levels of “signal”. For example, for predicting first
episode psychosis, language data from clinical tests has higher performance
compared to transcripts of free speech [205]. On the other hand, data
collected “free-speech” samples for diagnostic purposes has been found to be
highly effective in developing a language-based depression screening that
generalizes well across various age groups [167, 206, 184].

###  4.4 Social Media

Social media companies collect a wide variety and a large amount of language
data which contains clinically-relevant information (Table V). While this
information can also be extracted on a population level, such as the notable
rise in cognitive distortions over time [207], the same information can be
attributed on an individual level [208], making social media a powerful tool
to support mental health risk assessment and diagnosis. Language from Facebook
posts, for example, has been shown to contain markers for depression such as
rumination and sadness can be detected in the data up to 6 months prior to a
clinical diagnosis at hospital [168]. Models applied to Facebook and other
social platforms like Twitter, Reddit and so on have been successful in
predicting diagnosis of other disorders like psychosis, anorexia, anxiety, and
stress levels [209, 210, 211]. Aside from language present in user posts and
comments, these models also include processing media data such as Instagram
images [212], or combining images and text [213, 212]. It is important to note
that online search forms an equally compelling dataset that can be used
alongside social media activity to enhance overall performance.

Recent developments of transformer models, including those learning
multilingual language representations, have enabled researchers to build NLP
models that can generalize across languages and apply them to social media
data, for example, to detect depression or self-harm [214, 215]. Furthermore,
specialized language representations that were trained on mental health
specific conversations and are made publicly available [216], have been shown
to improve performance compared to non-specific representations. Finally, the
use of transformer embedding can be applied to pre-identified sections in
language which correspond to responses to standard clinical assessments such
as the subjective well-being scales [217], supporting the high-accuracy
prediction of the standard survey scale responses without directly running the
survey.

While social media solves the scale issue with millions of samples available,
one of the key challenges for social media data is the lack of clinically-
valid labels [218]. Most work has relied on using self-disclosure of mental
illness as the labels, which tend to be noisy and bring the additional problem
of defining a healthy control. Despite the challenges, the validity of social
media data to support mental health diagnosis and risk assessment has been
repeatedly proven.

TABLE V: Social media data access (OAuth 2.0 is the industry-standard protocol
for authorization).

Platform | Examples of accessible data | OAuth 2.0 required
---|---|---
Instagram | Media, captions, total posts | Yes
TikTok | Videos, descriptions, likes | Yes
SnapChat | Bitmoji, public stories/media | Yes
Twitter | Tweets, timestamps, likes, retweets | No
Pinterest | Pins, boards | Yes
Kik | Messages through a bot | Yes
Tumblr | Blogs, profile, likes | Yes
Reddit | Posts, post timing | No
Discord | Conversations on a server | Yes
Facebook | Posts, media | Yes
YouTube | Watch history, comments | Yes
Twitch | Chat history | Yes
Google | Query Strings | Yes

###  4.5 Sensing Technologies and Mobile Mental Health

Smartphones, wearables and other connected devices equipped with ambient
sensors (Fig. 7B) are increasingly capable of recording physiological
measurements like motion and location sensing, heart rate, and galvanic skin
response that are known to affect mental health [219]. In addition, some of
the less obvious measurements (such as keystroke usage patterns) have also
been shown to be implicated by mental illness [220, 221]. Additionally, online
gaming behaviors, such as interaction patterns with non-player characters
(NPCs) and other game behavior patterns, can be used to measure cognitive
performance and their relationship with mental illness [222, 223].

TABLE VI: Mobile sensor measurements and potential applications for mental
health monitoring.

Measurement | Feature | Effect in mental health
---|---|---
Movement | Psychomotor agitation | Increased in Anxiety
Location | Social avoidance | Increased in MDD
Social activity | Call/text volume | Reduced in MDD
Keystroke | Keystroke latency | Impaired in ADHD
Heart rate | Heart rate variability | Impaired in Stress
Gaming | NPC interactions | Impaired in social anxiety

These measurements from mobile sensors (Table VI) form some of the most widely
available sources of mental health data, and can be useful at various levels
of granularity–from raw sensor data (e.g., the accelerometer) to derived high-
level features (such as psychomotor activity). This has inspired big
corporations like Apple Inc. to invest in detecting depression and cognitive
decline using data collected from their wearable devices [224]. Sensor-based
measurements are found to be correlated with high-stress levels and a variety
of ailments including depression, anxiety, psychosis, and bipolar disorder
[225, 226]. Since sensor-based data are widespread and readily available,
these data offer an opportunity to build baseline models for individual users;
these baseline models can be used to identify significant physiological
changes in users and possibly inform clinical interventions.

Using data from mobile devices, digital phenotyping is collected in real-time
and targets specific individuals. Devices that collect data streams from
patients, such as surveys, cognitive tests, social medial interactions, GPS
coordinates, and behavioral patterns (e.g., keyboard typing), have great
potentials for monitoring, managing and predicting the individual’s mental
health [227, 228]. Overall, continuous quantification of these data streams
may result in clinically useful markers that can be used to refine diagnostic
processes, tailor treatment choices, improve condition monitoring for
actionable outcomes (such as early signs of relapse), and develop new
intervention models [229].

TABLE VII: Commercial and research platforms and services for mental health
applications.

Platform | Primary data source | Mental health appl.
---|---|---
WoeBot [230] | Language | Depression, Anxiety
Mindstrong [220] | Keystrokes | Serious mental illness
Sonde Health [164] | Voice | Mental fitness
Ellipsis Health [163] | Voice | Stress
Amazon Halo [231] | Voice | Emotion detection
Apple Watch [186] | Mobility, Sleep | Depression
Alphabet Fitbit [232] | Skin conductance | Stress
Kintsugi [233] | Voice | Depression, Anxiety
Bewie [227] | Raw data from smartphone | Multiple
MindDoc | Language (ask daily question) | Depression
Clarigent Health | Voice | Suicide risk

###  4.6 Commercial and Research Platforms and Services

While research studies have demonstrated promising results in using ML to
support the patient’s journey in mental health, the applicability in clinical
practice has been limited. Table VII lists examples of platforms and services
that use ML for mental health. While most platforms focus on developing a risk
assessment using single modality such as voice, the initial commercial
viability of these platforms is promising for the success of using ML in
mental health since they enable collection of large amounts of data which can
be used in further development of biomarkers with applications in clinical
practice.

Many areas of mental health technology development aim to scale approaches
that are applied in clinical research or practice today. For example, there
are between 10,000 to 20,000 smartphone apps that digitize mindfulness or
cognitive behavioral therapy techniques [234], allowing the user to engage in
psychotherapy on their own at a greatly reduced price compared to in-person
therapy. Like the apps in this example, the quality is highly variable and the
mechanisms used to validate them is often dubious. Finally, since this area is
relatively new, the industry and governmental standards to validate such a
technology are still developing. We will briefly outline two interrelated
areas of development: digital measurements and digital interventions.

Digital measurement applications. We are entering a new era of digital
psychiatry [8, 235]. In 2016, the Harvard professor Jukka-Pekka Onnela coined
the term digital phenotype [236], which refers to the use of mobile devices
and other digital data sources to measure behavior and physiology that is
relevant to understand underlying brain activity that is relevant to
pathological states. These techniques utilize measurement paradigms from
translational neuroscience that were developed in laboratory settings such as
direct quantification of motor (i.e. movement, muscle activation) and
physiological activity (i.e. heart rate, electro-dermal response) more than
traditional clinical scales or self-report scales. The advantage of this
approach is that it better aligns with emerging knowledge of rapid-acting
biological processes and provided high measurement accuracy through direct
rapid sampling rather than broad clinical measures that are taken sporadically
and measure a broad set of mental and behavioral states over a long
measurement period [237, 238, 239].These measurement approaches have relevance
in every area where measurement is of relevance including treatment
development, treatment selection, and ongoing monitoring.

Medications that target mental health conditions have a significant history of
failure. Most psychiatric medications were discovered capriciously rather than
being developed based on knowledge of the underlying biological mechanisms. As
new medications emerge from basic and translational neuroscience research,
both drug developers and clinicians struggle with how to measure the effects
of new treatments and how to properly target old treatments. For example,
traditional antidepressant medications are designed to slowly titrate
serotonin levels, resulting in slow global effects over a 2-4 week period.
Correspondingly, measures of depression based on the DSM, query about the
presence of depressive states over a 2-week period. New classes of anti-
depressant such as ketamine and psilocybinpsylocibin affect specific
depressive symptoms in minutes. Further, the mechanistic effects, and thus the
need for measurement, is much more specific and granular. In fact, most
classes of anti-depressants including serotonin reuptake inhibitors (SSRI) and
psilocybinpsylocibin, and ketamine, act on serotonin receptors which
ultimately impact peripheral motor and physiological activity [240, 241].
Serotonin regulation will likely have a direct effect on depression symptoms
such as psychomotor retardation, but minimal direct effects on feelings of
guilt. As such, methods used to directly measure motor output have a higher
likelihood of capturing both pathology and treatment effects.

As an example, research effort has been dedicated to using computer vision and
voice to directly quantify motor activity. Some recent work has demonstrated
that digital phenotyping parameters that reflect gross motor activity
including speech characteristics (rate of speech, tone) and facial/head
movements are associated with suicidal risk [242], SSRI response in major
depressive disorder [243], negative symptomatology in schizophrenia [244], and
Parkinsonian tremor [245]. Such approaches are now being commercialized for
all phases of drug development from proof-of-concept to direct measurement to
make decisions about ongoing treatment needs. Such measures solve many of the
current problems in clinical measurement since they can be captured remotely
in an automated way. These measures can also be captured at a much higher
frequency and can provide a sensitive numeric value.

These new approaches to measurement are not without significant challenges.
First, methods that are adapted from the laboratory often lack the tight
experimental control necessary to interpret the data correctly. For example, a
rapid increase in physiology can indicate stress, but also exercise or other
forms of exertion. Second, while the scientific basis of these measurement
paradigms may be sound, commercial approaches are rarely validated to the
extent required to be of clinical utility and rarely sufficiently transparent
in their approach to be used for regulatory approval.

Digital interventions. The other rapidly emerging area of mental health
technology are digital approaches to clinical care. We will briefly outline
some of the leading approaches. Importantly, digital approaches to clinical
care are often aligned with a digital measurement approach as these approaches
are ”blind” without some sort of remote data. A number of companies such as
Mindstrong Health [220], IesoTrigger Health [246], and Headspace Health [247,
248], have attempted to integrate digital phenotyping of some kind to identify
when patients are in acute clinical need. However, it is unclear how accurate
these methods are as they are typically unpublished. This work has led to
development of models that identify patterns in patient language and clinician
language that are markers of improved outcomes [249], which can be used to
measure success of various therapy modalities, treatment design, as well as to
improve care quality [250].

Digital therapeutics is generally referred to the use of mobile devices to
offer automated cognitive behavioral therapy (CBT), mindfulness, or other
validated psychotherapy approaches in an automated fashion. These app-based
approaches undergo the same clinical validation process and traditional
medications and are often developed in collaboration with large drug
developers. Examples that are in development or have received FDA (Food and
Drug Administration) approval include treatments for substance use disorder,
ADHD, schizophrenia, autism, MDD, PTSD, and generalized anxiety disorder [251]

While digital therapeutics attempt to scale treatment, telehealth aims to
scale the treatment provider network. Mental health is an area of treatment
where there are many effective treatments but little access to treatment
providers [252]. The issue of access became most acute during the emergence of
COVID-19 when significant wide-spread mental health needs emerged along with
greatly decreased access to care. To address this need, a large array of
options have emerged, many reinforced by the emergency COVID-19 Telehealth Act
of 2021 that allowed for remote patient care [253]. These services, which are
accessible directly to consumers, or more often, provided by a third-party
payer, provide access to coaches and clinicians via different mobile platforms
including text, voice, and video communication. Services vary from mental
health coaching provided by lay-professionals to psychiatric and psychological
services. While still in their infancy, early evidence has shown that
telehealth services can perform at parity with traditional in person therapy
[254]. Therefore, teletherapy is likely to be a dominant form of mental health
treatment in the future.

##  5 Multimodal Data fusion in Diagnostic Analytics

A central goal of precision psychiatry is to integrate all clinical,
physiological, neuroimaging and behavioral data to derive reliable
individualized diagnosis. Importantly, the health-related data are produced
daily, especially those from personal devices. The most essential effort in
multimodal data analysis tasks is exploratory: to explore the relationship
between modalities, complementarity, shared versus modality-specific
information and other mutual properties. Multimodal data fusion techniques
present a framework to infer information how different data modalities
interact and inform each other, are used in medical diagnosis [255, 256, 45].
In this section, we will review several data fusion methods in diagnostic
analytics (Section 5.1). We will focus on multimodal neuroimaging data
(Section 5.2), and then extend the discussion to other modalities that include
vocal and visual expression data (Section 5.3).

###  5.1 Popular ML Methods for Multimodal Fusion

In the past decades, numerous research efforts have been dedicated to
developing powerful ML methods for multimodal data fusion [257, 258, 45, 259,
260]. Some commonly used approaches are summarized below.

Multivariate Correlation Analysis. Canonical correlation analysis (CCA), as a
standard statistical method based on second-order statistics for data fusion,
aims at finding a pair of linear transformations to drive latent variables
(aka. canonical variates) that have maximized correlation between two
different data modalities [261]. For a more general setting, multiset/multiway
CCA (mCCA) has been developed as an extension of the standard CCA to
multimodal fusion by maximizing the overall correlation among latent variables
from more than two sets of modalities [262, 45]. Similar to CCA, partial least
squares (PLS) and its extensions, i.e., multiway PLS (N-PLS), provide
alternative approaches to integrate multimodal data by maximizing the
covariance between latent variables from different modalities [263, 264].

Matrix and Tensor Factorization. Based on matrix and tensor factorization
techniques, joint blind source separation (BSS) approaches have been developed
and successfully applied to multimodal fusion of biomedical data [265, 258].
As a typical example, joint independent component analysis (jICA) aims to
maximize the independence among jointly estimated components from multiple
modalities that are assumed to share the same mixing matrix [266]. The jICA
approach involves concatenating modality features alongside each other and
then performing ICA on the composite feature matrix [267]. Independent vector
analysis (IVA) is another extension of ICA to multiple datasets, which makes
use of dependence across datasets by defining source component vectors
concatenating a specific source estimated from multiple modalities [258, 257].
Coupled matrix and tensor factorization (CMTF) was developed to simultaneously
factorize multiple datasets in the form of matrices and high-orders tensors
using tensor decomposition [268], showing strength in capturing the potential
multilinear structure for multimodal fusion. Besides extracting shared common
components, some multimodal fusion tasks are also interested in deriving
individual components that are modality-specific. Common and individual
feature analysis (CIFA) [269] and joint and individual variation explained
(JIVE) [270] models have been proposed to achieve this goal. By jointly
decomposing multiple feature matrices, CIFA and JIVE are able to
simultaneously estimate common and individual feature subspaces. A further
extension of CIFA has been achieved by leveraging high-order tensor
factorization [265], which provides an efficient way to perform a
multidimensional fusion of multiple data modalities.

Multi-Kernel Learning. Multi-kernel learning (MKL) has won many successful
applications in multimodal data fusion due to the full utilization of multiple
kernels that enable simultaneous learning from various modalities with
heterogeneous data [271, 272]. Different kernels naturally correspond to
different modalities, such as neuroimaging, clinical, behavior, speech
features, etc., which may provide complementary information to drive improved
modal learning performance. The MKL problem can be set as a linear combination
of kernel matrices or a nonlinear function with specified forms of
regularization. MKL may be designed under different ML models including SVM,
Gaussian process, and clustering. Among them, MKL-SVM has been most popularly
applied to integrate heterogeneous data modalities in studies of mental health
[273, 274, 275].

Deep Learning-based Fusion. Empowered by cutting-edge deep learning
techniques, emerging methods have been increasingly developed for deep
multimodal fusion [276]. Deep learning-based fusion allows integrating
multiple modalities based on learned high-level feature representations that
are theoretically more comparable to each other and more informative for
predicting the targets [277]. By exploiting cross-modal manifolds as a feature
graph, a deep manifold-regularized learning model was recently designed to
achieve improved phenotype prediction by integrating transcriptomics and
electrophysiology data from neuronal cells [278]. Graph neural networks show
capability in information fusion for multimodal causability by defining casual
links between features with graph structures, thereby enhancing the
explainability of the derived multimodal feature representation [279]. By
extending graph neural networks to multimodal structures, deep representation
approaches have also been designed for integrating brain networks constructed
from diverse modalities [280, 281, 282].

Figure 8: Summary of typical approaches for multimodal data fusion in
psychiatry studies.

###  5.2 Multimodal Neuroimaging Studies

Neuroimaging data types are intrinsically dissimilar in nature, having
different spatial and temporal resolution [259]. Instead of entering the
entire data sets into a combined analysis, an alternate approach is to reduce
each modality to low-dimensional (latent) features of selected brain activity
or structure and then explore associations across these feature sets through
variations across individuals. Exploiting such latent feature representations
from multiple neuroimaging modalities for diagnosis has generally shown to
improve performance compared to using a single modality alone [283].
Multimodal fusion allows for the integration of neuroimaging data modalities
from different scales of spatial and temporal resolutions and offers an
elegant way to exploit complementary information for more accurate and robust
characterization of brain dysfunctions. Therefore, multimodal learning
mechanisms that take advantage of multimodal data sources are instrumental in
optimal decisions for diagnosis, treatment, prognosis, and planning in many
applications in medicine.

A combination of mCCA and jICA was successfully applied to fMRI and DTI fusion
in the diagnosis of schizophrenia and bipolar disorder [284]. CMTF has been
applied to identify diagnostic biomarkers of schizophrenia by integrating
sMRI, fMRI, and EEG [285]. MKL-SVM has been successfully applied to integrate
multimodal structural neuroimaging for predicting differential diagnosis
between bipolar and unipolar depression [286] and to combine sMRI and fMRI for
improved classification of trauma survivors with and without PTSD [287]. More
recently, it also showed efficacy in the diagnosis of early adolescent ADHD by
integrating sMRI, fMRI, and DTI [288]. In learning low dimensional
representations of functional and structural MRI [289], the functional MRI can
be split into several independent component networks, each treated as a
separate modality along with the structural scan for learning using
autoencoders. Furthermore, MKL methods have been used for diagnosing
schizophrenia by combining markers from MRI and DTI [290]. A multimodal graph
convolutional network was designed to integrate functional and structural
connectomics data for an improved prediction of phenotypic characterizations
in ASD [280]. By combining multiple typical neural network structures,
multimodal deep learning models have also been developed to effectively
integrate fMRI connectivity and sMRI features [291], and also genomic data
[292] for discovering schizophrenia-associated brain dysfunction. Methods for
learning joint representations from neuroimaging and non-neuroimaging data are
still in early development [293] and there is an opportunity for ML methods to
evolve for this task. For example, transformer networks with late fusion can
be used to learn joint representations from various modalities such as EEG and
facial video data [294].

###  5.3 Multimodal Fusion of Non-imaging Data

Multimodal approaches consist of combining data from various sources to
jointly arrive at an answer. Given how little is conclusively known about
which type of data, neuroimaging, social media, speech, video, sensor data
carries the most phenotypes for mental illness, it only makes sense to combine
the information from these data sources. In addition, this also enables
modeling the inter-dependencies between these data which may not be observable
by a human expert at the same time. For example, many features listed in Table
IV are known to be relevant in depression; however, observing them
simultaneously can be very challenging for a clinician. MKL and multi-task
learning methods have been used to jointly learn from sensor and smartphone
usage data to predict subjective well-being [295]. The success of transformer
networks in jointly modeling video, speech, and language data has catalyzed
multimodal modeling in mental health [296]. Multimodal modeling techniques can
also be used in modeling symptoms such as emotion dysregulation [297],
loneliness [298] and sentiment analysis [299]. In a prognostic study, an SVM-
based multimodal ML approach was developed to integrate clinical,
neurocognitive, neuroimaging, and genetic information to predict psychosis in
patients with clinical high-risk states [76]. Deep autoencoder-based fusion
approaches have been designed to integrate dynamics of facial and head
movement and vocalization, and successfully applied to the prediction of
depression severity [300].

##  6 Machine Learning for Molecular Phenotyping in Psychiatry

Figure 9: (A) Different levels of interacting variables (genes, cells,
circuits) to behaviors in mental illnesses. (B) Combining ML with novel
molecular biology technologies (for deep molecular phenotyping of brain
plasticity) opens up opportunities to develop new mechanistic models for
prevention and treatment of clinical endophenotypes of mood and cognitive
disorders.

Molecular phenotyping is referred to as the technique of quantifying pathway
reporter genes (i.e. pre-selected genes that are modulated specifically by
metabolic and signaling pathways) in order to infer activity of these
pathways. Mapping genes and genomics to behaviors can identify risk factors
and biomarkers in mental disorders. The brain is the central organ exposed to
stressors and external behavioral interventions, and therefore is a vulnerable
organ subject to changes in multiple interacting biological networks at the
systems level. ML methods have been contributing enormously to capturing the
complexities of interacting variables within and across multiple levels (Fig.
9A)– particularly at the molecular level – for identifying mechanistic-based
phenotyping models as new targets for the prevention and treatment of mood and
cognitive disorders. The advent of unbiased next-generation sequencing (NGS)
prompted the development of advanced bioinformatics and ML tools to profile
and decode large molecular datasets (e.g.: transcriptomics, epigenomics,
metabolomics) at the genome-wide level in health and disease states (Fig. 9B).
To date, there is increasing recognition of the utility of ML methods to
integrate these multi-level molecular datasets with clinical characteristics
to map specific neurobiological substrates into the complexity of symptom
clusters, which may aid the classification of diseases, prediction of
treatment outcomes, and selection of personalized treatment. Emerging avenues
in molecular neuroscience and computational psychiatry for new mechanistic
models for the prevention and treatment of CNS disorders.

Animal models has been playing a vital role towards precision psychiatry in
understanding mechanisms of diseases and predicting treatment responses [301,
302]. To bridge the scientific knowledge gap from animals to humans, gene
expression studies offer an example of how integrating basic neuroscience, ML,
and bioinformatics approaches can contribute to advancing understanding of the
molecular basis of MDD. Using RNA sequencing (RNA-seq) assays and gene
coexpression network analyses (based on hierarchical clustering to identify
gene modules), differential gene expression profiles have been shown across
six key brain regions, such as the ventromedial prefrontal cortex (PFC) and
dorsolateral PFC among others, in post-mortem tissues of subjects suffering
from MDD as compared to age and sex-matched controls with remarkable sex
differences in these molecular pathways [303]. Recent work using RNA-seq
assays at single nucleus resolution (snRNA-seq) and t-distributed stochastic
neighbor (t-SNE) embedding analyses showed cell-type specific transcriptomic
profiles in the post-mortem dorsolateral PFC that are differentially regulated
in MDD cases as compared to respective controls, with the greatest gene
expression changes in deep layer excitatory neurons and immature
oligodendrocyte precursor cells [304]. Importantly, these gene expression
studies in humans were supported by findings in rodents showing a brain that
continually changes with experience [305]. Several groups using RNA-seq assays
and bioinformatic analyses showed striking transcriptomic differences in the
ventral and dorsal hippocampus in the responses to stress–a primary risk
factor for multiple psychiatric diseases–with the ventral hippocampus being
particularly sensitive not only to the effects of stress [306] but also a
target for the responses to next-generation antidepressants [307, 308].

The expansion of NGS to single-cell resolution assays requires more advanced
bioinformatics analyses, which utilize ML to analyze these large datasets,
including denoising and dimensionality reduction, cell-type classification,
gene regulatory network inference, and multimodal data integration [309, 310].
Bioinformatic approaches, such as Seurat, combine unsupervised nonlinear
dimensionality reduction, K-nearest neighbor graph analysis for cell-type
clustering, and weighted nearest neighbor analysis for multimodal data
integration [311]. Deep learning approaches, such as the autoencoder, provide
another example of tools used for denoising and dimensionality reduction with
computational scalability [312, 313, 314]. Autoencoders can also be used in a
supervised manner for transfer learning across datasets, e.g., to learn the
embedding from a larger, already annotated dataset and transfer this knowledge
to cluster new datasets [315]. The combination of multimodal data generated
from the simultaneous assessment of transcriptomic profiles with regulatory
landscape or spatial location in the same single cell [316, 317, 318, 319]
will allow a deeper molecular characterization of discrete cellular states
[311].

Integrating multidimensional factors for new mechanistic treatment models. It
has been increasingly recognized that mood and cognitive disorders are
unlikely to be only brain-based diseases. Additionally, growing evidence
suggest that they are system-level disorders affecting multiple interacting
biological pathways [320], involving the dynamic cross-talk between the brain
and the body. Using hierarchical clustering to integrate in-vivo molecular
measures of brain metabolism with clinical symptoms in MDD patients, recent
work showed that, at least in these cases, the specific neurobiological
substrates map into discrete clinical symptoms, including anhedonia [321].
Furthermore, the integration of multidimensional factors spanning
mitochondrial metabolism, cellular aging, metabolic function, and childhood
trauma has been shown to provide more detailed signatures to predict
longitudinal changes in depression severity in response to the metabolic
agents used as antidepressant treatment than individual factors [322]. Use of
multi-omics approaches and random forest classifier has been shown to achieve
85% sensitivity and 77% specificity in prediction of the PTSD status. This
system-level diagnostic panel of multiple molecular and physiological measures
outperformed separate panels composed of each individual data type, with
certain mitochondrial metabolites among the most important predictors [323,
324].

An additional ML application example include the integration of
multidimensional phenotypic measures to identify those mechanisms that
predispose apparently healthy individuals to develop maladaptive coping
strategies from those that confer resilience. Using a high-throughput unbiased
automated phenotyping platform that collects more than 2000 behavioral
features and supervised ML that minimizes Bayesian misclassification
probability, recent work showed that a rich set of behavioral alterations
distinguish susceptible versus resilient phenotypes after exposure to social
defeat stress (SDS) in rodents [325, 326]. At the individual level, a ML
classifier integrated a priori constructs, including measures of anxiety and
immune system function, predicted if a given animal developed SDS-induced
social withdrawal, or remained resilient, with 80% sensitivity, which is
better than the categorization power based on either individual measure alone
[327].

To develop personalized psychiatry strategies for better diagnosing and
treating mental disorders, it is essential to meet the demand for ML enforced
by the recent advent of molecular biology protocols that opened up the
opportunity to capture central nervous system nanovesicles (known as exosomes)
for examining specific neurobiological substrates, including transcriptomic
profiles, dynamically and temporally in vivo. The development of advanced ML
methods for dynamic network analyses will permit to link brain molecular
targets and signaling pathways with other levels of analyses (e.g., functional
connectivity) in vivo and to incorporate complex relationships between the
brain and the rest of the body to redefine thinking about the modifiable
mechanisms throughout the complex clinical disease course.

##  7 Explainable AI and Causality Testing in Psychiatry

Explainable Artificial Intelligence (XAI) aims to provide strong predictive
values along with a mechanistic understanding of AI by combining ML techniques
with effective explanatory techniques and make it easily understood by the end
users. XAI has found emergent applications in medicine, finance, economy,
security, and defense [328, 329]. In psychiatry, the mission of XAI is to help
clarify the link between neural circuits to behavior, and to improve our
understanding of therapeutic strategies to enhance cognitive, affective, and
social functions [330, 331]. Notably, XAI distinguishes the standard AI in two
important ways: (i) promote transparency, interpretability, and
generalizability; (ii) transform classical “black box” ML models into “glass
box” models, while achieving similar or improved performance. From the
diagnosis or prognosis perspective, it is crucial to know whether the ML
solutions can be explainable to the point of providing hidden mechanistic
insights, such as how does the brain accomplish a particular function or a set
of complex behaviors. For instance, a classification function learned by the
machine to predict a disease outcome would not only need to report a
probability outcome but also need to address additional questions for the end-
user: why is this outcome instead of the alternative? How reliable is the
outcome? When does it fail if something is missing or misrepresented? When and
why the prediction is wrong? Accordingly, a model with improved
interpretability comes with parameter/structure/connectivity constraints and
some prior domain knowledge. The models can be continuously adapted such that
an iterative process for interpretability may be required to force ML methods
to fit models with specific interpretations. From the treatment perspective,
an improved understanding of brain dynamics responsible for dysfunctional
cognitive functions and/or maladaptive behaviors in mental illnesses is also
critical. To find the hidden cause, it is useful to discuss the concept of
“causality”.

Neuroimaging also provides a passive sensing approach to observe the
(correlational) brain-behavior relationship. However, correlation is not the
same as causation. Correlational dependencies describe associations of
measurements that experiments do not control, whereas causal dependencies link
a dependent variable to an experimentally controlled variable. The key concept
in causal inference is to introduce randomization to perturb the mapping. The
relationship between every dependable variable and the randomized variable is
causal, whereas the relationship between non-randomized variables and
behavior, remains correlational [332]. Closed-loop experimental design would
help to test the potential causality [333]. In human experiments, we classify
closed-loop testing into two categories: one being fully automated, and the
other being closed-human-in-the-loop.

In this section, we will systematically review several important topics along
these lines. We will review some “glass box” ML models in the literature
(Section 7.1), provides a brief introduction on neuroimaging-based circuit-
level modeling (Section 7.2), and adaptive neuromodulation (Section 7.3). We
argue that combining these efforts in the loop (“Neuroimaging $\rightarrow$
Circuit modeling $\rightarrow$ Neurostimulation $\rightarrow$ Observing
behaviors $\rightarrow$ Revising models”, Fig. 10A) will provide an effective
way towards interpretable neuropsychiatry in understanding brain-behavior
causation.

Figure 10: (A) Schematic of the closed loop of
neuroimaging/modeling/neurostimulation. (B) A wide spectrum of
interpretability in representative ML models.

###  7.1 Explanation and Constraints in Interpretable ML Models

In terms of taxonomy, intrinsic interpretability refers to ML models that are
considered interpretable due to their simple structure, such as short decision
trees or sparse linear models (Fig. 10B). Post-hoc interpretability refers to
the application of interpretation methods after model training [334].
Interpretation may come with different forms: (i) finite feature summary
statistics, (ii) meaningful model parameters, (iii) ease of visualization of
model outcome (e.g., feature summary or decision boundary). An ML model that
is highly interpretable because it has a few or more of the following
properties for explanation [334]: high expressive power, low translucency,
high portability, low algorithmic complexity and informative constraints.
Generally, there is a trade-off between explainablity and performance. For
instance, a constrained linear or bilinear model will fit many of these
criteria, but the linear model does not warrant a good performance.
Additionally, a model that is potentially explainable does not guarantee
explainability. For example, co-dependence of input variables may make
explanations ambiguous; latent variables of probabilistic generative model may
face the problem of “explaining away” [335]. There are several categories of
interpretable ML models, which will be discussed briefly below.

Hybrid rule-based ML models: This type of ML models can be used for generating
rules, such as a decision rule set: IF (condition) THEN (outcome 1) ELSE
(outcome 2) statement, where the conditional clause will be learned from the
data [336]. This type of model has more expressive power but less portability.

Constrained ML models: This type of ML models imposes parameter constraints to
either avoid overfitting or enhance interpretability. Examples of such include
the constrained convolutional filters in the CNN [337], or constrained mixture
models [338]. As a result, these constrained models have low translucency.

Feedback ML models: ML models can be provided with user feedback in the human-
in-the-loop system, where the user feedback is treated as constraints in the
optimization problem [339, 340]. The feedback can appear as a form of rule
sets, which can be either known or unknown in advance. Iterating the feedback-
rule optimization can generate more accurate rule sets. This type of model has
good expressive power and high portability.

###  7.2 Circuit-Level Modeling for Computational Psychiatry

Sharing the same goal of XAI, computational psychiatry tries to combine
multiple levels and types of computation with multiple types of data in an
effort to improve understanding, prediction and treatment of mental illness
[341]. Two complementary approaches are used in computational psychiatry:
first, the data-driven approaches apply ML methods to high-dimensional data,
multimodal to tackle classification and prediction problems (Section 3);
second, the theory-driven approaches develop empirical or mechanistic models
to test hypothesis.

For mechanistic modeling approaches, circuit-level mechanistic modeling of
macroscopic or mesoscopic brain dynamics represents an important research
topic in computational modeling for psychiatric disorders [342, 343, 344]. A
common strategy is to first use a biologically-inspired model to simulate
neural activity based on a network of interacting neural masses, and then
within each brain area, to model the neuronal population activity as the
Wilson-Cowan neural mass or the mean-field model, with each consisting of
excitatory and inhibitory populations [345]. Furthermore, individual brain
nodes are coupled according to the empirically-derived anatomical network
[346]. The computational model can be driven by data (such as EEG and fMRI),
or driven by hypothesis.

One data-driven macroscopic level modeling approach is Dynamic causal modeling
(DCM). DCM has been widely used in characterizing the effective connectivity
of a functional network based on task or resting-state fMRI [347, 348], where
the model parameter estimation and inference is done by unsupervised learning.
By incorporating prior knowledge or hypotheses of network connections, DCM may
reveal important brain mechanisms based on fMRI data and offer experimental
predictions. One of potential applications of DCM is to characterize the
neural plasticity in the human brain, especially the change in functional
connectivity informed by neuroimaging studies. The functional connectivity can
either change gradually during the course of tasks or behavioral protocols, or
induced by perturbation or neurostimulation. These changes are often, but not
always, associated with changes in functional activations of the studied brain
regions. Specifically, combination of neuromodulation and DCM may provide a
way to test the impact of neurostimulation on neural plasticity that underlies
the change in adaptive or maladaptive behaviors.

###  7.3 Closing the Loop for Testing Causality

One big challenge in human psychiatric neuroscience is the causality gap
[349]. Statistical causality or Granger causality between two variables is not
equivalent to brain-behavioral causality. To identify an effective treatment
strategy for mental illnesses, it is critical to causally modulate neural
circuitry that is responsible for maladaptive behaviors. Human neuroimaging
alone only demonstrates correlations but not causation. To understand the
causal mechanisms, it is important to close the loop in experiments by
manipulating or perturbing the brain circuits and measuring its outcome, as
commonly done in animal experiments [332, 333]. Unfortunately, a rigorous and
causal grounding of clinical symptoms and behavior in specific neural circuit
alternations is still missing. Specifically, since the clinical symptoms are
diverse, how to define the dimension of brain function that defines one or few
clinical symptoms and effectively manipulate them remains unknown.

Causal and temporally precise neurostimulation tools provide a plausible tool
to perturb or stimulate the brain by neuromodulation. To date, human
neuromodulation methods include invasive deep brain stimulation (DBS),
noninvasive transcranial magnetic stimulation (TMS), noninvasive transcranial
direct/alternating current stimulation (tDCS/tACS), and transcranial focused
ultrasound stimulation (tFUS). A review of advances in neuromodulation
technologies for treating mental disorders can be found in the literature
[350, 351, 352]. To date, repetitive TMS (rTMS) has been cleared by the FDA
for the treatment of depression and recently used in the studies of neural
functioning and behavior [353, 354].

The brain connectivity and dynamics can be studied from a network
communication and control perspective [355, 356]. The distinction between a
healthy and a pathological brain can be characterized by their different
ability or efficiency to route the information between distributed brain
nodes, control/modulate the target node under specific constraints, or
influence its behavior in order to perform specific tasks (”cognitive
control”) [357]. Brain connectivity and neurostimulation can be studied using
the network and graph theory. Specifically, the control-theoretic models have
also been applied to quantify the response of brain networks to exogenous and
endogenous perturbations. Within the XAI-neuromodulation framework, it is
convenient to formulae a mathematical framework for important research
questions: (i) can a target node stimulation rewire the brain connectivity in
evoked and steady-state conditions? (ii) can the neurostimulation-induced
change of evoked or resting-state brain connectivity distinguish a
pathological from a healthy brain? (iii) Given an input constraint (such as
the energy of neurostimulation), what is the optimal neurostimulation policy?
Can alternate or simultaneous neurostimulations at multiple sites more
effectively influence the network connectivity or bring additional benefit in
treatment [358]?

XAI for neurostimulation in mental health can be seen as an extension in the
design of brain-machine interface (BMI) [359]. Specifically, XAI can search
and identify behaviorally activated targets through active and scheduled
stimulation strategies. Traditional neurostimulation strategies are designed
in an on/off stimulation fashion triggered by predetermined stimulation
parameters. However, these stimulation parameters will not be optimal. To
accommodate an adaptive subject-specific stimulation strategy, adaptive
stimulation uses neurofeedback to adjust the stimulation parameters or control
policy for achieving various optimality criteria; reinforcement learning and
active learning (two other important ML topics not reviewed here) can play a
crucial role in online adaptive experiments [360, 361]. Additionally,
simultaneous or post-stimulation neuroimaging provides a window of examining
the change in brain network connectivity patterns. Can the induced brain
patterns or changes in network connectivity be used to predict the treatment
outcome? ML provides a potential tool to address such a question by providing
individualized treatment-response likelihood in precision psychiatry [362].

##  8 Discussion and Conclusion

###  8.1 Challenge and Opportunities

The past few decades have witnessed growing interests and rapid developments
in ML methods toward precision psychiatry. However, caution has been raised
regarding the unrealistic hope for ML applications in clinical practice [160,
363], and the field is facing both conceptual and practical challenges.

At the conceptual level, first, the term “disorder” was used to specifically
avoid the term “disease,” which implies a level of neurobiological and
physiological understanding that is lacking in psychiatry. This makes it very
challenging to build clinical inference models for mental disorders since the
neurological and physiological mechanisms are not always observable with
sufficient precision. For example, this precision is limited due to the
temporal and spatial limitations of neuroimaging methods. As a result, it is
not yet feasible to develop treatments that target underlying physiological
risk factors similar to how it is possible in other areas of medicine, for
example, treating hypertension since it plays a causal role in heart disease.
Furthermore, each mental disorder has various types of overlapping symptoms
with varying degrees, bringing an additional challenge to uniquely define the
disorder in psychiatry (unlike a clear cut in cardiology or oncology).

Second, many disorders are presented as a spectrum, for example, autism
spectrum disorder, generalized anxiety spectrum, and schizophrenia spectrum,
and presented differently in different patients, creating a wide range of
subtypes and subject variability diagnosed with the same mental disorder.
Third, due to various genetic, biochemical, neuropathological factors, the
same mental disorder may have different causes and symptoms in different
patients. Fourth, many mental disorders have overlapping symptoms with other
physical or mental disorders [364]. For example, changes in sleep and energy
level, often found in depression and generally measured using the PHQ-9
questionnaire, are very common across many other disorders. This makes it
challenging to accurately diagnose mental disorders. One of the challenges of
precision psychiatry is to fully dissect the mechanisms and reveal the one-to-
one relationship. This can be catalyzed by rigorous and continuous