11institutetext: School of Computer Sciences, University of East Anglia,
Norwich, NR9 7TJ, UK 11email: {andy.davies<EMAIL_ADDRESS>
22institutetext: Faculty of Social Sciences, Institute of Psychology, Leiden
University, Rapenburg 70, 2311, Netherlands 22email:
<EMAIL_ADDRESS>33institutetext: Computer-Aided Programming
Research Group, MIT Computer Science and Artificial Intelligence Laboratory
(CSAIL), MA, 02139, USA 33email<EMAIL_ADDRESS>
# Individual Behavioral Insights in Schizophrenia: A Network Analysis and
Mobile Sensing Approach

Andy Davies 11 0009-0009-5807-1972 Eiko Fried 22 0000-0001-7469-594X Hane Aung
11 0000-0002-4376-0849 Omar Costilla-Reyes 33 0000-0001-8331-7262

###### Abstract

Digital phenotyping in mental health often consists of collecting behavioral
and experience-based information through sensory and self-reported data from
devices such as smartphones. Such rich and comprehensive data could be used to
develop insights into the relationships between daily behavior and a range of
mental health conditions. However, current analytical approaches have shown
limited application due to these datasets being both high dimensional and
multimodal in nature. This study demonstrates the first use of a principled
method which consolidates the complexities of subjective self-reported data
(Ecological Momentary Assessments - EMAs) with concurrent sensor-based data.
In this study the CrossCheck dataset is used to analyse data from 50
participants diagnosed with schizophrenia. Network Analysis is applied to EMAs
at an individual (n-of-1) level while sensor data is used to identify periods
of various behavioral context. Networks generated during periods of certain
behavioral contexts, such as variations in the daily number of locations
visited, were found to significantly differ from baseline networks and
networks generated from randomly sampled periods of time. The framework
presented here lays a foundation to reveal behavioural contexts and the
concurrent impact of self-reporting at an n-of-1 level. These insights are
valuable in the management of serious mental illnesses such as schizophrenia.

###### Keywords:

Schizophrenia CrossCheck n-of-1 Digital Phenotyping Network Analysis Mobile
Sensing.

##  1 Introduction

Schizophrenia is a complex, Serious Mental health Illness (SMI) that develops
in approximately 1% of the global population [41] and represents a significant
personal and economic burden at an individual, familial and societal level
[15, 44]. Symptoms can include hallucinations (both visual and auditory),
disordered and delusional thinking, impaired cognitive ability, disorganized
speech and behavior [37], as well as increased social isolation, withdrawal
and amotivation [29]. Although characterised as a chronic condition, the
disease course is not static, with diagnosed individuals typically fluctuating
between periods of partial remission and periods of symptomatic relapse [46,
35, 17]. Studies have identified symptomatic and behavioral changes that can
manifest prior to relapse [3, 17, 9, 20], however, these changes often remain
undetected until the occurrence of significant negative consequences [45].
Evidence further suggests that timely clinical intervention poses an effective
strategy in the prevention of further deterioration, and the transition into a
state of full relapse [33, 45].

In this paper, we seek to demonstrate a method which consolidates the
complexities of subjective self-reported Ecological Momentary Assessments
(EMAs) when accounting for variations in behavioral context. Using the
CrossCheck dataset, a first of its kind dataset combining real-world,
longitudinal behavioral data, and self-reported EMAs specific to schizophrenia
[45]; we aim to demonstrate the effectiveness of using sensor-based data to
identify periods of various behavioral context, from which network analysis
can be applied to observe and compare differences in network connectivity and
the relationships between corresponding EMAs. Specifically, we focus on
behaviors that can be categorised according to periods of sociability and
social isolation, both noted symptoms associated with schizophrenia symptom
severity [20]. Figure 1 provides a high-level overview of this process, from
individual-level (n-of-1) data through to behavioral filtering and network
analysis. Ultimately, the goal of this framework is to reveal behavioral
contexts and their resulting impact on self-reported EMAs at an n-of-1 level,
in particular providing insights into symptomatic improvement or disease
exacerbation.

Figure 1: Overview of the processes involved in this study; networks are
generated using EMA responses concurrent with specific sensor-based behavioral
contexts.

##  2 Related Work

Conventional research into human behavior often relies on time and resource
intensive data collection through face-to-face engagement in a controlled or
clinical environment. However, the pervasiveness of mobile technology [42] in
everyday life is affording researchers and clinicians access to vast
quantities of moment-by-moment, in-situ, individual-level data captured by
personal digital devices; the granular level quantification of which is
referred to as digital phenotyping [43, 32]. These personal phenotypes [5, 36]
provide a digital fingerprint from which psychological, cognitive and
behavioural characteristics can be measured and assessed [16, 23, 22, 31];
providing valuable insights into symptomatic markers and effective psychiatric
treatments [38]. Within mental health research, digital phenotyping has been
employed in a number of studies, including student mental health [30],
depression [26], prediction of suicidal urges [12], anxiety disorders [24],
social anxiety [25], and psychosis spectrum illnesses [8].

With an increasing emphasis on patient-centred healthcare and individualised
medicine [11], digital phenotyping lends itself to move away from the
population level [18] and instead conduct n-of-1 trials (or single subject
trials); these focus on an individual patient as the sole unit of observation
throughout a study [27]. Typically, n-of-1 studies have been used within both
clinical and research settings to assess pharmaceutical efficacy and treatment
viability within individual participants [39]. The focus of these trials
enables the identification of observations or characteristics that may not be
evident in a collective population-level analysis. However for larger
population samples, the insights gained from n-of-1 trials can contribute to
larger-scale Randomized Control Trials (RCTs).

The CrossCheck collection emulated a Randomized Control Trial (RCT) design
[14] that explored the viability of continuous remote patient monitoring
through a multimodal sensing system; the core aim of which sought to
accurately predict indicators of symptomatic and psychotic relapse in
Schizophrenia Spectrum Disorders (SSD) [45]. CrossCheck’s digital phenotyping
dataset identified unique digital indicators of psychotic relapse; for some
participants changes in self-reported EMAs provided actionable descriptors of
symptom exacerbation, whilst in others, passively recorded behavioural and
sensory data proved useful in identifying changes in established behaviors or
daily functioning [7]. A recent study demonstrated the detection of decreases
in symptoms using change-point algorithms and counterfactual explanations
[13]. Additional research using the CrossCheck dataset mapped features on a
two-dimensional space using t-Distributed Stochastic Neighbor Embedding
(t-SNE), a technique used for dimensionality reduction that projects each
high-dimensional data point to a two-dimensional data point [28]. Using t-SNE,
CrossCheck visualized data points that represented a participant’s behavioural
features used to predict EMA responses; when plotted, these data points
clustered according to each specific study participant. This clearly
demonstrated that there are observable differences between study participants
and that CrossCheck’s sensor data is highly person dependent [45]. At a
population level, the initial study found significant associations between
recorded behavioral features and changes in mental health indicators; in
particular decreased levels of physical activity and sociability was
associated with negative mental health, whilst improvements in established
sleep patterns and getting up earlier collated with positive mental health
[45]. These findings were further supported through research into behavioral
stability using the same dataset, this stability index drew on participant’s
passively recorded features and behaviours to assess the extent to which a
diagnosed participant adheres to a stable routine. This study identified
correlations between the stability index of recorded features and symptomatic
severity, the results of which demonstrated that greater periods of stability
in social activities - such as calls and SMS messages - was associated with
reduced symptoms. In contrast, increased stability in periods of inactivity -
time spent still - exhibited an association with increased symptom severity
[20]. The findings of these studies highlight not only the highly person-
centric nature of CrossCheck’s multimodal data, but also the close association
between daily behaviors and symptom severity; both of which are of particular
importance to this study as we seek to analyse the impact recorded behavioral
contexts have on self-reported EMAs at an individual level.

In recent years the use of network analysis within psychological research has
become an important tool in the estimation and visualization of psychological
data, and can be used to identify multivariate patterns and relationships [10,
21]. Within these networks, nodes represent variables such as mood states
collected via EMAs, with edges between nodes denoting statistical
relationships between said nodes [21]. The process by which these associations
and relationships are calculated can vary depending on the initial dataset and
selected statistical model [10]. In recent years, network analysis techniques
have been applied to a large number of datasets in order to gain deeper
insights into a range of mental health problems, including drug and alcohol
dependency [40], suicidal behavior in adolescents [19], depression and anxiety
[6], and the treatment of psychosis [4]. Network analysis consists of three
stages; network structure estimation, network description, and network
stability analysis [10]. Network structure estimation refers to the process by
which the underlying structure of a network is inferred, involving the
selection of relevant nodes and edges as well as selecting an optimal
statistical model. Network description is the characterisation of a network
which involves understanding network topology and node centrality. Finally,
network stability analysis refers to the examination of a network’s
robustness, consistency and the accuracy of edge weights [10]. Recently,
estimating network models on time-series data with numerous repeated
observations using EMA data has gained traction, however this presents three
distinct challenges. First, most work is estimated at the group level,
ignoring potential variation across participants in network structures.
Second, networks are stationary, i.e., one network is obtained throughout the
time period, assuming network structure does not vary by context. Third, it is
unclear how network analysis ought to deal with multimodal (e.g., sensor and
EMA) data. Here we tackle all three challenges, by estimating n-of-1 networks
according to a selected behavioral (sensor-based) context prior to EMA network
generation, as a result enabling more nuanced, context-dependent qualitative
networks.

##  3 Dataset

The CrossCheck [45] dataset originally consisted of $150$ participants, each
of whom met the criteria for schizophrenia as defined in the DSM-IV [1] and
DSM-V [2], whilst also meeting CrossCheck’s inclusion criteria [7, 45].
Organised into two groups of $75$, participants within the CrossCheck study
arm were each issued with a smartphone that continuously recorded a range of
behavioral, sensory and self-reported EMA data over a 12 month period [45, 7,
47]. Embedded smartphone sensors passively record daily behaviours and
activities continuously, whilst EMAs were self-reported every 2 to 3 days
[45]. Of particular relevance to this paper are the following features:

#### Ecological Momentary Assessment (EMA):

EMAs afford a viable way of capturing real-time psychological data within a
natural environment. Every 2 to 3 days, CrossCheck administered a 10-item
self-reporting assessment designed to measure schizophrenia-related thoughts,
feelings, and behaviors [7, 45]. Each question (see Table 1) was answered on a
scale from 0 (”Not at all”) to 3 (”Extremely”). For ease of analysis and
understanding, each EMA is grouped according to either its positive or
negative association.

Table 1: CrossCheck EMA Questions

Positive EMAs |  Negative EMAs
---|---
\+ Have you been feeling CALM? |  \- Have you been DEPRESSED?
\+ Have you been SOCIAL? |  \- Have you been feeling STRESSED?
\+ Have you been SLEEPING well? |  \- Have you been bothered by VOICES?
\+ Have you been able to THINK clearly? |  \- Have you been SEEING THINGS other people can’t see?
\+ Have you been HOPEFUL about the future? |  \- Have you been worried about people trying to HARM you?

#### Behavioural Sensing:

Study smartphones continuously collected a wide range of behavioral features
for each participant, however only the following behavioral features are
relevant to this paper due to their close association with sociability and
social isolation. - Geo-spatial Activity: refers to timestamped locational
data derived using a combination of device GPS, Wifi and cellular network
towers [45, 7]. - Speech Frequency & Duration: periods of human speech was
inferred from ambient sound using the inbuilt device microphone [7]. - Calls &
SMS: The frequency and duration of incoming and outgoing calls, as well as the
number of incoming and outgoing SMS messages is passively logged and recorded
by the CrossCheck application [45, 7].

##  4 Method

The following section presents the proposed methodology employed in this
study, from prerequisite data pre-processing, through to initial network
generation and statistical analyses. Data pre-processing requires basic
resampling and thresholding, with network generation simply based on the
correlations within the 10 set EMA questions (see Table 1). As such giving low
computational requirements and ease of re-implementation.

###  4.1 Data Pre-processing

From CrossCheck’s original study arm, $50$ participants have been identified
from which further analysis can be conducted, these individuals were selected
for the quantity and quality of their recorded data. Participants whose
engagement was limited, or who recorded inconsistent and unusable data were
omitted from further analysis. Whilst both sensory and device usage data is
temporally continuous, self-reported EMA responses are only given every 2 to 3
days [45]. However, each EMA question also pertains to days prior to a given
response, as such we can retrospectively replicate each EMA score to also be
concurrent with sensor data recorded between EMA responses. Any days that fall
outside of the 2 day back fill window are omitted from a participant’s
dataset.

Selection of sensor features was based on their ability to effectively capture
defined behavioral contexts, without the need for further processing to map
sensor data to a particular context (e.g. it is a reasonable assumption that
no location data outside of the primary residence indicates not leaving the
home, and that no calls or detected conversations indicates not verbally
socializing). Table 2 lists these selected features and their corresponding
categories. These categories are created based on 2 factors; first relating to
the veracity of the behavioral context that the data represents as stated
above, with the second factor being based on well understood behavioral
contexts for this SMI, such as social isolation and sociability. Along side
these features random sampling of unfiltered data is also conducted from which
an empirical distribution is generated, this serves as a baseline from which
comparisons can then be made.

Table 2: Selected behavioral contexts and their corresponding categories.

Behavioral Feature in a 24hr period |  Periods of Social Isolation |  Periods of Sociability
---|---|---
Baseline |  Random Unfiltered Sample |  Random Unfiltered Sample
Locations Visited |  No locations visited |  $\geq 1$ locations visited
Calls Made |  No calls made |  $\geq 1$ calls made
Calls Received |  No calls received |  $\geq 1$ calls received
SMS Messages Sent |  No SMS messages sent |  $\geq 1$ messages sent
SMS Messages Received |  No SMS messages received |  $\geq 1$ messages received
Conversations Detected |  No detected conversations |  $\geq 1$ detected conversations

###  4.2 Network Structure Estimation & Description

Although participant data is initially chronologically ordered, filtering
according to a selected behavioral category, either social isolation or
sociability, results in time-series segmentation. This segmentation
necessitates the identification of a statistical model that can account for
this lack of temporal consistency. Correlation network models applied to
cross-sectional data deal well with this lack of consistency, as they are
effective at visualizing relationships between variables at a specific point
in time [10]. As a result we can use this model to generate networks for each
person that represent an aggregated average for a sample taken from each
selected behavioral category. This allows us to compare network structures of
EMAs when participants are, for instance, spending time socially isolated
(e.g. remaining at home) or engaging socially (e.g. visiting locations outside
of their home). Structurally, EMAs are represented as network nodes with edges
between nodes visualizing the linear relationship between each. The strength
and sign of any given relationship is defined by the correlation coefficient.
Numerically, correlations between nodes range between $-1$ and $1$; with $-1$
indicating a perfect negative relationship, $1$ a perfect positive
relationship, and $0$ no relationship at all. Pearson’s R Correlation
Coefficient is used to calculate the associations between all 10 EMA nodes.
For each behavioral context two networks are generated, one for each category
within a given behavior (Table 2).

###  4.3 Permutation Testing

To discern whether variations in each behavioral context and their observed
network structures differ in statistically meaningful ways, permutation
testing is used. The goal of which is to evaluate the null hypothesis that
these variations have no discernible concurrence with a participant’s self-
reported EMAs.

Figure 2: Permutation testing process for Daily Number of Locations Visited,
networks are generated using concurrent EMAs from each behavioral category, in
turn allowing for the calculation of differences in network connectivity.

Procedurally, permutation testing requires a selected behavior to be filtered
according to predefined categories (see Table 2). A network is generated for
each category using a $25$ day sample, at the end of each permutation the
observed difference in network connectivity is calculated by subtracting the
sum of the network matrices from each other. In probability theory, Central
Limit Theorem (CLT) suggests a sample size of approximately $30$, however, to
maximise the number of viable participants a reduced sample size of $25$ days
is used throughout the permutation testing process. Repeated $2000$ times, a
new observed difference in network connectivity is calculated for each
permutation, from which a distribution of these differences is then produced.
Figure 2 demonstrates this process using the daily number of locations visited
as an example.

The resulting distribution of differences in network connectivity for a given
behavior can then be compared with a baseline distribution. This baseline
undergoes the same testing process but is generated using randomly sampled
data, and serves as an empirical distribution from which comparisons can then
be made. To measure statistical significance, paired-sample t-testing is used
to compare a given behavioral distribution with the empirical baseline
distribution. The resulting t-score and p-value can then be used to confirm or
reject the null hypothesis that a selected sensor based behavioural context
has no discernible influence over an individual’s network connectivity.

##  5 Results

In this section we present the findings of our analysis; first detailing
results for a single participant, and then at a wider level for multiple
participants.

###  5.1 Example of an Individual CrossCheck Participant

Having recorded 330 days of usable data, this individual returned results
across 5 of the 6 selected behavioral contexts. Figure 3 visualizes the
results produced during permutation testing for each behavior according to
networks generated using only positive, and only negative EMAs (see Table 1).

(a) Positive EMAs Only

(b) Negative EMAs Only

Figure 3: Side-by-side comparison of permutation test results for a single
individual across all selected behavioral contexts as well a baseline test -
networks generated during testing process used positive EMAs only versus
negative EMAs only

In each instance, we observe differences when comparing each behavior to a
baseline distribution, for example in Figure 3(a), we observe a baseline mean
of 0 compared to a mean of 4.7 in detected conversations. Focusing in on daily
number of locations visited, there is not only an observable difference
between behavior and baseline, but there is a notable difference when
comparing positive and negative EMAs.

(a) Positive EMAs Only

(b) Negative EMAs Only

Figure 4: More detailed analysis of participants results for both their
baseline distribution (Blue) and Number of Locations Visited distribution
(Red). Fig 4(a) clearly shows an observable difference between the baseline
distribution and the behavioral context distribution for positive EMAs.

Figure 4 provides a more detailed visualization of this participant’s baseline
distribution (in blue) and variations in their daily number of locations
visited (in red). Upon visual inspection, there is a clear observable
difference in distributions produced using positive EMAs (fig 4(a)) when
compared to those produced using negative EMAs (fig 4(b)). This suggests that,
for this participant, there is a noticeable impact on their self-reported
positive EMAs when factoring in variations in the daily number of locations
visited. Table 3 provides a breakdown of these results across all EMA groups,
with a consistent p-value $<0.001$ indicating these results are statistical
significant. Moreover, we see a more sizeable t-score for this participant’s
positive EMAs ($-72.78$), further suggesting that variations in this
behavioral context impacts this individual’s self-reporting habits -
particularly for their positive EMAs.

Table 3: Participant’s baseline and Daily Number of Locations Visited test
results, including mean ($\bar{x}$), standard deviation ($\sigma$), t-score
($t$) and p-value ($p$)

| Baseline | Daily Number of Locations Visited
---|---|---
|  $\bar{x}$ |  $\sigma$ |  $\bar{x}$ |  $\sigma$ |  $t$
All EMAs |  -0.44 |  9.06 |  17.30 |  6.22 |  -55.62 *
Positive EMAs |  0.05 |  2.28 |  4.61 |  1.55 |  -72.72 *
Negative EMAs |  -0.05 |  3.80 |  1.83 |  3.07 |  -13.21 *
00footnotetext: * $p<0.001$

Whilst numerically these results indicate a statistical significance for this
particular behavioral context; network generation provides a visualization of
the relationship between EMAs. Figure 5 provides a side-by-side comparison
between two networks that each visualize one of the two categories for
variations in the daily number of locations this individual participant has
visited.

(a) Days with 0 locations visited

(b) Days with 1 or more locations visited

Figure 5: Cross-Sectional Networks for Daily Number of Locations Visited.
Nodes: DEPressed, HARm, SEEing Things, STRessed, Hearing VOIces, CALm, HOPe,
SLEep, SOCial, THInking Clearly. Thicker edges denote stronger relationships
with blue edges indicating positive correlations and red negative. As
expected, one can observe positive relations between positive EMAs (i.e., when
this person reports one positive EMA, they are more likely to report others,
too); positive relations between negative EMAs; and negative relations between
positive and negative EMAs.

Both networks in Figure 5 present strong positive relationships between
auditory and visual hallucinations, however, on days where this participant
remained at home (see Figure 5(a)) we see a much more complex network with the
presence of stronger edges in greater numbers. A visual analysis of these two
networks suggests that, for this particular participant, there is a beneficial
link between social engagement (interacting with locations away from home) and
improvements in self-reported EMAs. In particular, Figure 5(a) illustrates
strong negative relationships between each hallucinatory node and feelings of
calm and hopeful, this suggests that increased instances of one has a
detrimental impact on the other. For example, hallucinations experienced at
home could have a more noticeable detrimental impact on this participant’s
ability to feel calm and hopeful; likewise it could also suggest that feeling
calm within this participant’s own home reduces the likelihood of them
experiencing increased hallucinations.

###  5.2 By Behavioral Context

In the interest of brevity the following results focus on the analysis of all
10 EMAs for three specific behavioral contexts. The daily number of calls
made, daily number of conversations detected and daily number of locations
visited.

#### Daily Number of Calls Made & Detected Conversations:

Figure 6 presents the distributions of 6 participants who each returned
results for both the daily number of calls made and number of conversations
detected.

(a) Baseline

(b) Calls Made

(c) Conversations

Figure 6: Results across for all 10 EMAs for participants who returned results
for both selected behavioral contexts - Daily Number of Calls Made and Daily
Number of Detected Conversations. Plot color varies depending on statistical
significance when using pair-sampled t-testing to compare with the baseline
distribution - blue if $p<0.05$ otherwise plot remains grey.

As expected, the baseline distributions in fig 6(a) indicate a mean close to
$0$, demonstrating the consistency of randomized sampling. However following
the same process as with the previous individual’s results, a comparison
between each participant’s baseline distribution and behavioral context
distribution demonstrates varying degrees of difference - further highlighting
the individuality of participant data.

Table 4: Participant results for paired-sample t-testing between their
baseline distribution and both Number of Calls Received and Calls Made. For
each behavior results include; mean ($\bar{x}$), standard deviation
($\sigma$), t-score ($t$), and p-value ($p$).

| Baseline | Daily Number of Calls Made | Daily Number of Conversations
---|---|---|---
ID |  $\bar{x}$ |  $\sigma$ |  $\bar{x}$ |  $\sigma$ |  $t$ |  $\bar{x}$ |  $\sigma$ |  $t$
07 |  -0.18 |  8.75 |  15.45 |  4.97 |  -49.65 * |  14.67 |  6.57 |  -46.80 *
11 |  -0.02 |  6.81 |  -8.47 |  5.43 |  44.06 * |  0.24 |  5.41 |  -1.24
13 |  -0.01 |  5.33 |  -4.73 |  4.90 |  19.93 * |  1.00 |  4.09 |  -4.91 *
14 |  -0.27 |  10.62 |  -12.40 |  9.58 |  39.43 * |  3.88 |  9.51 |  -13.01 *
43 |  -0.42 |  13.63 |  -5.45 |  12.12 |  12.91 * |  14.43 |  11.44 |  -37.31 *
64 |  -0.11 |  4.88 |  1.55 |  4.61 |  -9.79 * |  -4.16 |  5.00 |  23.47 *

00footnotetext: * $p<0.001$

Table 4 provides a breakdown of results for each participant visualized in
Figure 6. Whilst in most cases analysis of these two behaviors produces a
statistically significant p-value ($p<0.001$), there is one instance where
this is not the case - participant 11 for daily number of detected
conversations. A population level analysis, or an unfiltered analysis would
have obscured this outlier individual given the variability in distributions
across each participant. Moreover participant 07, the same participant
analysed previously (see section 5.1), presents significant differences for
each of these behavioral contexts; further strengthening the hypothesis that
for this participant activities associated with sociability and social
engagement have a marked influence on their self-reported EMAs.

#### Daily Number of Locations Visited:

The following results are from 8 participant’s who returned distributions for
this behavioral context across all 10 EMAs.

(a) Baseline

(b) Locations Visited

Figure 7: Results across for all 10 EMAs for participants who only returned
results for Daily Number of Locations Visited. Plot color varies depending on
statistical significance when using pair-sampled t-testing to compare with the
baseline distribution - blue if $p<0.05$ otherwise plot remains grey.

The side-by-side plots in Figure 7(a) visualize both the baseline and
behavioral context distributions for each valid participant, again, we observe
statistical significance ($p<0.05$) across each individual. Table 5 provides a
numerical breakdown of these distributions.

Table 5: Results across for all 10 EMAs for participants who returned results
for Daily Number of Locations Visited. Results include; mean ($\bar{x}$),
standard deviation ($\sigma$), t-score ($t$), and p-value ($p$).

| Baseline | Daily Number of Locations Visited
---|---|---
ID |  $\bar{x}$ |  $\sigma$ |  $\bar{x}$ |  $\sigma$ |  $t$
07 |  -0.07 |  8.76 |  17.42 |  6.40 |  -55.33 *
14 |  0.44 |  10.28 |  -32.71 |  8.16 |  112.87 *
30 |  0.09 |  5.60 |  8.96 |  4.15 |  -56.95 *
31 |  0.10 |  4.43 |  3.80 |  4.39 |  -21.34 *
41 |  -0.03 |  7.84 |  3.34 |  7.14 |  -6.05 *
44 |  -0.09 |  8.21 |  -6.26 |  8.56 |  22.57 *
54 |  0.09 |  10.32 |  10.81 |  6.80 |  -35.19 *
59 |  0.13 |  16.00 |  1.54 |  13.96 |  -2.25 **

00footnotetext: * $p<0.001$, ** $p<0.05$

As with previous behavioral contexts, the results presented in Table 5 further
demonstrate the unique behavioral patterns of each participant. Whilst in
certain participants we observe a reduced difference between distributions,
participant 14 returns a significantly larger result ($t=112.87$), suggesting
that visiting locations outside of the home has an influence over how this
individual reports their EMAs. Continued analysis of each behavioral context
at a positive and negative EMA level would yield insights into whether or not
this influence is specific to one set of EMAs more so then the other.

##  6 Discussion

This paper presents an application of an n-of-1 network analysis, leveraging
qualitative EMA data whilst factoring in changes and differences in behavioral
context measured via sensor data. Specifically, our method allows researchers
and clinicians to study, in both exploratory and confirmatory ways, to which
degree mental health variables collected via EMA as well as the relation among
variables in networks differ across situations. As such, the proposed method
provides an inroad to combining multimodal data sources in clinical research
and practice, with the goal to enable the potential development of bespoke
treatment/management pathways. As an example, the results produced in Figure 4
reveal that, for this participant at least, not leaving the home significantly
changes the way this person reports on his/her positive EMA questions. This
new insight coupled with the structure of generated networks could give
qualitative actionable information, enabling timely and adaptive interventions
for this person’s care going forward [34]. The next step from this approach
will be to expand this methodology and to represent this information in a
format this is comprehensible and explainable to the larger psychiatry
community.

In summary, the task of how to meaningfully analyse multimodal behavioral
sensor data with a complex array of concurrent qualitative self reported data
is not well understood, particularly for SMI applications. In this analysis we
demonstrate an n-of-1 network analysis approach applied solely to self-
reported contextual behavioral data. Networks generated from these distinct
periods of behavioral context reveal differences in self-reporting habits,
differences beyond chance. This is a first stage indicative approach for
datasets similar to CrossCheck which is computationally inexpensive, easily
deployed and may lead to actionable clinical insights. However further studies
are required to better understand how such insights can be utilized in
practice, which are clinically effective but that are also compliant of
ethical, regulator and legal requirements.

## References

  * [1] American Psychiatric Association, A., Association, A.P., et al.: Diagnostic and statistical manual of mental disorders: DSM-IV, vol. 4. American psychiatric association Washington, DC (1994)
  * [2] American Psychiatric Association, D., Association, A.P., et al.: Diagnostic and statistical manual of mental disorders: DSM-5, vol. 5. American psychiatric association Washington, DC (2013)
  * [3] Ascher-Svanum, H., Zhu, B., Faries, D.E., Salkever, D., Slade, E.P., Peng, X., Conley, R.R.: The cost of relapse and the predictors of relapse in the treatment of schizophrenia. BMC psychiatry 10, 1–7 (2010)
  * [4] Bak, M., Drukker, M., Hasmi, L., van Os, J.: An n= 1 clinical network analysis of symptoms and treatment in psychosis. PloS one 11(9), e0162811 (2016)
  * [5] Barnett, I., Torous, J., Staples, P., Sandoval, L., Keshavan, M., Onnela, J.P.: Relapse prediction in schizophrenia through digital phenotyping: a pilot study. Neuropsychopharmacology 43(8), 1660–1666 (2018)
  * [6] Beard, C., Millner, A.J., Forgeard, M.J., Fried, E.I., Hsu, K.J., Treadway, M.T., Leonard, C.V., Kertz, S., Björgvinsson, T.: Network analysis of depression and anxiety symptom relationships in a psychiatric sample. Psychological medicine 46(16), 3359–3369 (2016)
  * [7] Ben-Zeev, D., Brian, R., Wang, R., Wang, W., Campbell, A.T., Aung, M.S., Merrill, M., Tseng, V.W., Choudhury, T., Hauser, M., et al.: Crosscheck: Integrating self-report, behavioral sensing, and smartphone use to identify digital indicators of psychotic relapse. Psychiatric rehabilitation journal 40(3), 266 (2017)
  * [8] Benoit, J., Onyeaka, H., Keshavan, M., Torous, J.: Systematic review of digital phenotyping and machine learning in psychosis spectrum illnesses. Harvard Review of Psychiatry 28(5), 296–304 (2020)
  * [9] Birchwood, M., Spencer, E., McGovern, D.: Schizophrenia: early warning signs. Advances in Psychiatric Treatment 6(2), 93–101 (2000)
  * [10] Borsboom, D., Deserno, M.K., Rhemtulla, M., Epskamp, S., Fried, E.I., McNally, R.J., Robinaugh, D.J., Perugini, M., Dalege, J., Costantini, G., et al.: Network analysis of multivariate data in psychological science. Nature Reviews Methods Primers 1(1), 58 (2021)
  * [11] Bradbury, J., Avila, C., Grace, S.: Practice-based research in complementary medicine: could n-of-1 trials become the new gold standard? In: Healthcare. vol. 8, p. 15. MDPI (2020)
  * [12] Brown, L.A., Taylor, D.J., Bryan, C., Wiley, J.F., Pruiksma, K., Khazem, L., Baker, J.C., Young, J., O’Leary, K.: Digital phenotyping to improve prediction of suicidal urges in treatment: study protocol. Aggression and violent behavior 66, 101733 (2022)
  * [13] Canas, J.S., Gomez, F., Costilla-Reyes, O.: Counterfactual explanations and predictive models to enhance clinical decision-making in schizophrenia using digital phenotyping. arXiv preprint arXiv:2306.03980 (2023)
  * [14] Chalmers, T.C., Smith Jr, H., Blackburn, B., Silverman, B., Schroeder, B., Reitman, D., Ambroz, A.: A method for assessing the quality of a randomized control trial. Controlled clinical trials 2(1), 31–49 (1981)
  * [15] Chong, H.Y., Teoh, S.L., Wu, D.B.C., Kotirum, S., Chiou, C.F., Chaiyakunapruk, N.: Global economic burden of schizophrenia: a systematic review. Neuropsychiatric disease and treatment pp. 357–373 (2016)
  * [16] Davidson, B.I.: The crossroads of digital phenotyping. General Hospital Psychiatry 74, 126–132 (2022)
  * [17] Emsley, R., Chiliza, B., Asmal, L., Harvey, B.H.: The nature of relapse in schizophrenia. BMC psychiatry 13, 1–8 (2013)
  * [18] Fisher, A.J., Medaglia, J.D., Jeronimus, B.F.: Lack of group-to-individual generalizability is a threat to human subjects research. Proceedings of the National Academy of Sciences 115(27), E6106–E6115 (2018)
  * [19] Fonseca-Pedrero, E., Al-Halabí, S., Pérez-Albéniz, A., Debbané, M.: Risk and protective factors in adolescent suicidal behaviour: a network analysis. International journal of environmental research and public health 19(3), 1784 (2022)
  * [20] He-Yueya, J., Buck, B., Campbell, A., Choudhury, T., Kane, J.M., Ben-Zeev, D., Althoff, T.: Assessing the relationship between routine and schizophrenia symptoms with passively sensed measures of behavioral stability. NPJ schizophrenia 6(1), 35 (2020)
  * [21] Hevey, D.: Network analysis: a brief overview and tutorial. Health Psychology and Behavioral Medicine 6(1), 301–328 (2018)
  * [22] Insel, T.R.: Digital phenotyping: technology for a new science of behavior. Jama 318(13), 1215–1216 (2017)
  * [23] Insel, T.R.: Digital phenotyping: a global tool for psychiatry. World Psychiatry 17(3), 276 (2018)
  * [24] Jacobson, N.C., Feng, B.: Digital phenotyping of generalized anxiety disorder: using artificial intelligence to accurately predict symptom severity using wearable sensors in daily life. Translational Psychiatry 12(1), 336 (2022)
  * [25] Jacobson, N.C., Summers, B., Wilhelm, S.: Digital biomarkers of social anxiety severity: digital phenotyping using passive smartphone sensors. Journal of medical Internet research 22(5), e16875 (2020)
  * [26] Kamath, J., Barriera, R.L., Jain, N., Keisari, E., Wang, B.: Digital phenotyping in depression diagnostics: Integrating psychiatric and engineering perspectives. World Journal of Psychiatry 12(3), 393 (2022)
  * [27] Lillie, E.O., Patay, B., Diamant, J., Issell, B., Topol, E.J., Schork, N.J.: The n-of-1 clinical trial: the ultimate strategy for individualizing medicine? Personalized medicine 8(2), 161–173 (2011)
  * [28] Van der Maaten, L., Hinton, G.: Visualizing data using t-sne. Journal of machine learning research 9(11) (2008)
  * [29] McCutcheon, R.A., Marques, T.R., Howes, O.D.: Schizophrenia—an overview. JAMA psychiatry 77(2), 201–210 (2020)
  * [30] Melcher, J., Hays, R., Torous, J.: Digital phenotyping for mental health of college students: a clinical review. BMJ Ment Health 23(4), 161–166 (2020)
  * [31] Mohr, D.C., Shilton, K., Hotopf, M.: Digital phenotyping, behavioral sensing, or personal sensing: names and transparency in the digital age. NPJ digital medicine 3(1), 45 (2020)
  * [32] Mohr, D.C., Zhang, M., Schueller, S.M.: Personal sensing: understanding mental health using ubiquitous sensors and machine learning. Annual review of clinical psychology 13, 23–47 (2017)
  * [33] Morriss, R., Vinjamuri, I., Faizal, M.A., Bolton, C.A., McCarthy, J.P.: Training to recognise the early signs of recurrence in schizophrenia. Cochrane Database of Systematic Reviews (2013)
  * [34] Nahum-Shani, I., Smith, S.N., Spring, B.J., Collins, L.M., Witkiewitz, K., Tewari, A., Murphy, S.A.: Just-in-time adaptive interventions (jitais) in mobile health: key components and design principles for ongoing health behavior support. Annals of Behavioral Medicine 52(6), 446–462 (2018)
  * [35] National Collaborating Centre for Mental Health (UK and others): Psychosis and schizophrenia in adults: treatment and management. London: National Collaborating Centre for Mental Health. doi (2014)
  * [36] Onnela, J.P.: Opportunities and challenges in the collection and analysis of digital phenotyping data. Neuropsychopharmacology 46(1), 45–54 (2021)
  * [37] Patel, K.R., Cherian, J., Gohil, K., Atkinson, D.: Schizophrenia: overview and treatment options. Pharmacy and Therapeutics 39(9), 638 (2014)
  * [38] Perez-Pozuelo, I., Spathis, D., Clifton, E.A., Mascolo, C.: Wearables, smartphones, and artificial intelligence for digital phenotyping and health. In: Digital Health, pp. 33–54. Elsevier (2021)
  * [39] Punja, S., Bukutu, C., Shamseer, L., Sampson, M., Hartling, L., Urichuk, L., Vohra, S.: N-of-1 trials are a tapestry of heterogeneity. Journal of Clinical Epidemiology 76, 47–56 (2016)
  * [40] Rhemtulla, M., Fried, E.I., Aggen, S.H., Tuerlinckx, F., Kendler, K.S., Borsboom, D.: Network analysis of substance abuse and dependence symptoms. Drug and alcohol dependence 161, 230–237 (2016)
  * [41] Saha, S., Chant, D., Welham, J., McGrath, J.: A systematic review of the prevalence of schizophrenia. PLoS medicine 2(5), e141 (2005)
  * [42] Silver, L.: Smartphone ownership is growing rapidly around the world, but not always equally (2019)
  * [43] Torous, J., Kiang, M.V., Lorme, J., Onnela, J.P., et al.: New tools for new research in psychiatry: a scalable and customizable platform to empower data driven smartphone research. JMIR mental health 3(2), e5165 (2016)
  * [44] Wander, C.: Schizophrenia: opportunities to improve outcomes and reduce economic burden through managed care. Am J Manag Care 26, S62–S68 (2020)
  * [45] Wang, R., Aung, M.S., Abdullah, S., Brian, R., Campbell, A.T., Choudhury, T., Hauser, M., Kane, J., Merrill, M., Scherer, E.A., et al.: Crosscheck: Toward passive sensing and detection of mental health changes in people with schizophrenia. In: 2016 ACM Int. Joint Conf. Pervasive & Ubiquitous Comput. pp. 886–897 (2016)
  * [46] Wang, R., Wang, W., Aung, M.S., Ben-Zeev, D., Brian, R., Campbell, A.T., Choudhury, T., Hauser, M., Kane, J., Scherer, E.A., et al.: Predicting symptom trajectories of schizophrenia using mobile sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1(3), 1–24 (2017)
  * [47] Wang, W., Mirjafari, S., Harari, G., Ben-Zeev, D., Brian, R., Choudhury, T., Hauser, M., Kane, J., Masaba, K., Nepal, S., et al.: Social sensing: assessing social functioning of patients living with schizophrenia using mobile phone sensing. In: Proceedings of the 2020 CHI conference on human factors in computing systems. pp. 1–15 (2020)


# Uncertainty Modeling of Emerging Device based Computing-in-Memory Neural
Accelerators with Application to Neural Architecture Search

Zheyu Yan<EMAIL_ADDRESS>University of Notre Dame ,  Da-Cheng Juan
<EMAIL_ADDRESS>National Tsing Hua University  ,  Xiaobo Sharon Hu<EMAIL_ADDRESS>University of Notre Dame and  Yiyu Shi<EMAIL_ADDRESS>University of Notre Dame

(2021)

###### Abstract.

Emerging device-based Computing-in-memory (CiM) has been proved to be a
promising candidate for high-energy efficiency deep neural network (DNN)
computations. However, most emerging devices suffer uncertainty issues,
resulting in a difference between actual data stored and the weight value it
is designed to be. This leads to an accuracy drop from trained models to
actually deployed platforms. In this work, we offer a thorough analysis of the
effect of such uncertainties-induced changes in DNN models. To reduce the
impact of device uncertainties, we propose UAE, an uncertainty-aware Neural
Architecture Search scheme to identify a DNN model that is both accurate and
robust against device uncertainties.

††journalyear: 2021††copyright: acmcopyright††conference: ACM Asia South
Pacific Design Automation Conference; Tokyo; Japan††booktitle: ASP-DAC 2021:
ACM Asia South Pacific Design Automation Conference, June 18–21, 2021, Tokyo,
Japan††price: 15.00††doi: 10.1145/3394885.3431635††isbn:
978-1-4503-7999-1/21/01

##  1\. Introduction

Deep Neural Networks (DNNs) have achieved superhuman performance in various
perception tasks and have become one of the most popular solutions for these
applications. Thus, there is an obvious trend in deploying DNNs on edge
devices such as automobiles, smartphones, and smart sensors. However,
implementing computational intense DNNs directly on edge devices is a
significant challenge due to the limited computation resource and constrained
power budget of these devices. Moreover, most of the DNN accelerator designs
are confined in a design space where the researchers only consider
conventional von-Neumann architectures (e.g., GPUs, mobile CPUs, or FPGAs) as
candidate platforms. In von-Neumann architectures, data movement inevitably
becomes the bottleneck for system efficiency, due to the well-known memory
wall where the computational unit must fetch and store data from the memory
hierarchy.

Figure 1. Accuracy differences between models trained in data center and
deployed on CiM simulations for different neural architectures. FC Net and
LeNet target MNIST, ResNet 56 targets CIFAR-10 and VGG-19 targets ImageNet. An
accuracy drop close to 10% can be observed.

Emerging device-based Compute-in-Memory (CiM) neural accelerators (Ielmini and
Wong, 2018) offer a great opportunity to break the memory wall with special
architectural advantages. CiM architectures offer reduced data movement by in-
situ weight data access (Sze et al., 2017). Highly efficient emerging devices
(_e.g._ RRAMs, STT-RAMS, and FeFETs) can be devised to offer higher energy
efficiency and higher memory density compared with traditional MOSFET (Shafiee
et al., 2016) based designs. However, such accelerators suffer greatly from
design limitations. Non-ideal conditions of emerging devices due to their non-
ideal manufacturing process induce uncertainties on emerging devices. These
uncertainties, such as device-to-device (D2D) variations, thermal noise, and
retrieval limitations, cause value changes that, the weights in the actually
deployed accelerators may be different from the desired weight value trained
offline in data centers. This weight value change leads to performance
degradation in actual accelerator implementations. As an illustration, we
train the four models, multilayer perceptron (MLP) and LeNet for MNIST, ResNet
56 for CIFAR-10, and VGG-19 for ImageNet, to state-of-the-art accuracy and
deploy them on CiM simulation tools (Feinberg et al., 2018). As shown in Fig.
1, an accuracy degradation of close to 10% is observed in each model
implementation.

The device uncertainty-induced performance degradation has been studied from
different perspectives, including device-level observations (Zhao et al.,
2017), architecture level analysis (Jiang et al., 2020a), and behavioral level
explorations (Yan et al., 2020b). Finding suitable pairs of DNN models and
hardware designs that can together offer both desirable hardware reliability
and high inference accuracy requires great effort.

Neural Architecture Search (NAS) (Zoph and Le, 2017; Zoph et al., 2018; Zeng
et al., 2020) is one of the most successful efforts to address this issue. NAS
liberates human labor from endlessly exploring optimal handcrafted DNN models
by automatically identifying neural architectures that can offer desired
performances from a pre-determined search space. Co-exploration of neural
architecture and hardware design (Jiang et al., 2019; Jiang et al., 2020b;
Jiang et al., 2020c) pushes this concept further by incorporating hardware
design specifications into NAS search spaces, so as to offer neural
architecture-hardware design pairs that are accurate, efficient, and robust
against hardware uncertainties.

In this work, we adopt a statistical analysis perspective to study the effect
of device uncertainties on the performance of DNNs. We model the emerging
device uncertainty as a whole into Gaussian noise on weights and thoroughly
investigate the behavior of different DNN models under such uncertainties. We
conduct a Monte-Carlo simulation-based analysis on the statistical behavior of
the models under the influence of device uncertainties. We then abstract our
analysis results to offer supports for NAS applications. The detailed
contributions of this work are:

  * •

We propose a Monte-Carlo simulation-based experimental flow to measure the
device uncertainty-induced perturbations to DNN models.

  * •

We then thoroughly investigate the behaviors of different DNN models under
such perturbations and show that the value changes of their output vectors
follow Gaussian distribution.

  * •

To alleviate this effect, we then propose UAE, a device uncertainty-aware NAS
framework, to search for architectures that are more robust to device
uncertainties.

Experimental results show that UAE offers a 2.49% higher accuracy than NACIM
(Jiang et al., 2020a) with 1.2x of time consumption. By further increasing
search complexity, UAE reaches 6.39% higher accuracy than NACIM with 2.5x of
search time.

##  2\. Background

###  2.1. CiM DNN Accelerators

Researchers have proposed different crossbar-based CiM DNN accelerator
architectures (Shafiee et al., 2016; Chi et al., 2016) for efficient DNN
inference. We assume an ISAAC-like architecture (Shafiee et al., 2016), and
the architecture of the system is organized into multiple tiles, with crossbar
arrays as the heart of each tile. The crossbar not only stores the synaptic
weights but also performs dot-product computations. One certain crossbar is
dedicated to processing a set of neurons in a given DNN layer. The outputs of
that layer are fed to other crossbars that are dedicated to processing the
next layer. The computation in the crossbar is performed inanalog domain.
However, ADC and DAC are used to convert the signal from and to the analog
domain dot-product computation and other digital domain operations needed in
DNN computation.

Crossbar is the key component of CiM DNN accelerators. As shown in Fig. 2, a
crossbar can be considered as a processing element for matrix-vector
multiplication where matrix value (_i.e._ weights for NNs) are stored at the
cross point of each vertical and horizontal line with resistive emerging
devices such as RRAMs and FeFETs, and each vector value is propagated through
horizontal data lines. In this work, we assume an RRAM-based design. The
calculation in crossbar is performed in analog domain but additional
peripheral digital circuits are needed for other key NN operations (_e.g._ ,
non-linear activation), so DAC and ADCs are adopted between different
components.

Device-level limitations confine the application of crossbars. The precision
of ADC and DACs limits the precision of DNN activations for each layer and the
non-ideal characteristics of emerging devices impose noises on the weights of
deployed DNNs.

Figure 2. Illustration on crossbar architecture.

###  2.2. Device Variation

In this work, we assume an RRAM-based crossbar design. RRAM devices suffer
various types of faults due to manufacturing and runtime non-idealities. Noise
sources that are directly relevant to crossbar-based designs include thermal
noise, shot noise, random telegraph noise (RTN), and programming errors
(Feinberg et al., 2018). When the circuitry is used for inference, programming
errors due to device-to-device variations could be the dominant error source.

Write-and-verify (Alibart et al., 2012; Niu et al., 2012; Xu et al., 2013) is
a simple, accurate, and widely used programming scheme for RRAMs. The key
operation is to iteratively apply a series of short pulses and check the
difference between current and target resistance, converging progressively on
the target resistance. In deploying accelerators for Neural Network inference,
this time-consuming progress is tolerable because once programmed, no more
modifications to the resistance are needed during the entire life span of the
accelerator. Although this scheme pulls down the D2D variation-induced error
to less than 1%, a significant error drop can still be observed in conditions
shown in Fig. 1.

###  2.3. Neural Architecture Search

Neural Architecture Search (NAS) has achieved state-of-the-art performance in
various perceptual tasks, such as image classifications (Yang et al., 2020a;
Yang et al., 2020b), inference security (Bian et al., 2020) and image
segmentation (Yan et al., 2020a). NAS is becoming increasingly successful
mainly because it liberates human designing labors by automatically
identifying high-performance neural architectures. Co-exploration of neural
architecture and hardware design (Jiang et al., 2019; Jiang et al., 2020b;
Jiang et al., 2020c) push this concept further by incorporating hardware
design specifications into NAS search spaces, so as to offer neural
architecture-hardware design pairs that are accurate, efficient, and robust
against hardware uncertainties.

Formally speaking, NAS deals with a problem that, given a perceptual task $T$,
a human-defined design space $\mathbf{S}$, and a set of figures of merit (FOM)
$\mathbf{P}$, what is the best neural architecture in $\mathbf{S}$ that can
offer optimal performance (in terms of FOM in $\mathbf{P}$) on task $T$.

A typical reinforcement learning (RL)-based NAS that solves this issue, such
as the framework proposed in (Zoph and Le, 2017), is composed of three key
components, a _controller_ , a _trainer_ and an _evaluator_. In one iteration
(named _episode_) of RL-based NAS, (1) the _controller_ generates a neural
architecture from the design space; (2) the _trainer_ builds the generated
neural architecture into a DNN model, named child network, and trains the
child network on a held-out training dataset; (3) the _evaluator_ collects the
figures of merit (FOM), _e.g._ , test accuracy of the trained child network on
test dataset, its latency and/or energy consumption; and (4) the _controller_
use a user-defined reward function to calculate a $reward$ data from FOM
collected by the _evaluator_ and use the $reward$ to update itself so that it
can predict neural architectures with higher FOM.

This iterative method terminates under two circumstances: (1) the _controller_
repeatedly predicts the same child network; and (2) the number of predicted
architectures exceeds a predefined threshold (episode limit). The child
network that offers the highest _reward_ among all the generated neural
architectures is presented as the search result. The chosen neural
architecture is then re-trained on the training dataset for a longer training
time to offer optimal performance.

More recently, differentiable NAS (Liu et al., 2018; Wu et al., 2019; Li et
al., 2020) has achieved state-of-the-art performance with a much-reduced
search time by transforming the search process into training an over-
parameterized neural network. However, those approaches suffer from
flexibility. More specifically, in the field of research considered in this
paper, differentiable NAS struggles in handling large search spaces with
multiple different hardware design parameters and complex designs where the
number of channels varies for each layer. Thus, in this work, we adopt RL
based NAS as our search algorithm.

##  3\. Uncertainty Modeling

###  3.1. Uncertainty Model

In this work, we model device uncertainties as a whole and use a Gaussian
distribution to represent them (Jiang et al., 2020a; Feinberg et al., 2018).
We set the mean of the uncertainty distribution to be zero, its variation to
be 0.04, and for each device, its uncertainty is independently distributed.
which is referred from (Zhao et al., 2017), where the uncertainties are
measured from actual physical devices. For an easier representation of the
latter part of this paper, the uncertainty model is depicted as:

(1) $W_{Dep}=W_{Exp}+\mathbf{N}(\mu,\sigma)$

where N is a Gaussian variable which, on each individual element of the
weights, is independent and identically distributed. $W_{Exp}$ and $W_{Dep}$
are the expected weights trained in the data center and the actual weights
deployed on the accelerators, respectively.

###  3.2. Effects on DNN Outputs

In this work, we focus on the impact of device uncertainty on classification
tasks, and the reasons go as follows: (1) most emerging device-based DNN
accelerators target classification tasks, analyzing the effect of device
uncertainties on these tasks helps the majority of the researchers to improve
their work; (2) DNNs for classification tasks are typically composed of
convolution layers and fully connected layers, which are also the basic
components of DNNs targeting other application. The effect of device
uncertainties on these two components is essential for all types of DNNs.

We start by understanding the effect of device uncertainties on the output of
a DNN model. Formally speaking, a DNN model $M$ can be defined as a
combination of its neural architecture and its trained weights. Thus, the
inference process of a DNN model with input $\mathbf{I}$ that generate an
output $\mathbf{O}$ can be defined as:

(2) $\mathbf{O}=F\ (W,\mathbf{I})$

where $F$ is the neural architecture of $M$, $W$ is its weights, $\mathbf{I}$
is this input vector and $\mathbf{O}$ is the output vector.

During training, $\mathbf{O}$ is then passed through a loss function, where a
version of $\mathbf{O}$ after SoftMax is compared with the ground truth
classification label $GT$ to generate a _loss_ for the backpropagation
process. During inference, the final predicted class of $\mathbf{I}$ can be
calculated by $argmax(\mathbf{O})$, which is the index of the item in
$\mathbf{O}$ that has the maximum value.

Although in inference, classification result is the final outcome of a DNN
model, the output vector $\mathbf{O}$ serves as a better representative of the
behavior of this model. The classification result is only an index of the
maximum value of $\mathbf{O}$ and is thus only a simplified discrete proxy of
$\mathbf{O}$. The continuous, multi-dimension vector $\mathbf{O}$ contains
more information than the classification result. In order to understand how
uncertainties in weights may affect the network, it is of crucial importance
to understand how it affects $\mathbf{O}$.

As defined in 1 and 2, a deployed neural network under the effect of device
uncertainties can be depicted as:

(3) $\mathbf{O}_{Dep}=F\ (W_{Dep},\ I)=F\ (W_{Exp}+N_{j},\ I)$

where $W_{Exp}$ is the trained value of the neural network to be deployed,
$N_{j}$ is one sample from the noise distribution, and $\mathbf{O}_{Dep}$ is
the affected output.

We analyze the distributional behavior of the effect of device uncertainties
on the output vector of a DNN model. To conduct this analysis, we first (1)
train a DNN model $F$ to converge and collects its trained weight $W_{Exp}$.
We then (2) fix one input image $I$ and collect its output on the trained
weight $W_{Exp}$. We denote this output as the original output
$\mathbf{O}_{Ori}$. After that, we (3) sample $K$ different instances of
noises $N_{1},N_{2},...N_{K}$ and then feed them to Eq. 3, collecting $K$
different output vectors. Finally, we calculate _output change_ using Eq. 4

(4) $\mathbf{O}_{CG}=\mathbf{O}_{Dep}-\mathbf{O}_{Ori}$

where the output change is the element-wise subtraction of the perturbed and
original output.

###  3.3. Experimental Results

In order to get a glance at the statistical behavior of _output change_ ,
according to the workflow introduced in Sect. 3.2, we train a LeNet model for
the MNIST dataset (LeCun et al., 1998) to state-of-the-art accuracy. We then
randomly choose one input image in the test dataset and sampled 10k different
instances of noise. with this setup, we gathered 10k different _output change_
vectors.

The _output change_ is a vector of 10, with each element representing the
confidence of classifying the input image into one certain category. Because a
high-dimensional vector is not a good choice for analytical study and
visualization, we analyze each element of these vectors. We analyze the
statistical behavior of each element across different vectors and gathered 10
instances of distribution data.

Surprisingly, each element of the _output change_ follows Gaussian
distribution. To visualize this finding, we plot the histogram of the
distribution of each element of _output change_ vector and the corresponding
Gaussian distribution that fits it. The visualization result for the first
element of _output change_ is shown in Fig. 3 that the first element of
_output change_ vector is a Gaussian variable.

Figure 3. _Output change_ distribution of LeNet for MNIST. 10k _output change_
vectors are gathered from one trained LeNet model affected by 10k different
instances of noise sampled from $\mathcal{N}(0,0.04)$. This figure shows the
distribution of the fist item of the gathered _output change_ vectors.

To verify this observation, We tested various networks in various datasets.
With the MNIST dataset, we analyze both LeNet and multilayer perceptrons (MLP)
using ReLU and Sigmoid activation with 2 layers. With the CIFAR-10 dataset
(Krizhevsky et al., 2009), we test a conventional floating-point CNN, a
quantized CNN, and two ResNets, ResNet-56 and ResNet-110. We also train these
models with 3 different initializations to get different trained weights.

We evaluate how these _output change_ vectors fit into Gaussian variables by
two widely used standards: mean square error (MSE) and Chi-square ($\chi^{2}$)
test. MSE can be described as:

(5) $MSE=\frac{1}{N}\sum^{N}_{i=1}(O_{i}-E_{i})^{2}$

and $\chi^{2}$ test can be depicted as:

(6) $\chi^{2}=\sum^{N}_{i=1}\frac{(O_{i}-E_{i})^{2}}{E_{i}}$

where $O_{i}$ and $E_{i}$ are the observed (_output change_) and estimated
(Gaussian) value of probability and $N$ is a user-defined granularity. We
define $N=100$ because it is precise enough when we have a total of 10k
instances of data.

Model | Dataset |  $\chi^{2}$ ($10^{-2}$) | MSE ($10^{-4}$)
---|---|---|---
MLP-ReLU | MNIST | 5.22 | 3.20
MLP-Sigmoid | MNIST | 5.81 | 2.20
LeNet | MNIST | 4.59 | 2.67
Float-Conv | CIFAR-10 | 7.01 | 3.03
Fixed-Conv | CIFAR-10 | 6.79 | 2.74
ResNet-56 | CIFAR-10 | 4.56 | 1.79
ResNet-110 | CIFAR-10 | 4.81 | 2.01
Table 1. Gaussian fit for different models. The $\chi^{2}$ test result and MSE
between the _output change_ and Gaussian distribution is presented. Both tests
show that the _output change_ follows Gaussian distribution w.r.t different
instances of noise.

The evaluation result is shown in Table 1. Note that we have test 3 different
initializations for each model and for each model, $outputchange$ is a vector
of 10. The result shown in Table 1 is an average of them. For each model
tested, $\chi^{2}$ test results are all below 0.1 and MSE are all below
$1\times 10^{-3}$, which indicates that they are well fit into Gaussian
distributions. Moreover, both errors do not increase when the model is
extremely shallow (_e.g._ 2-layer MLP) and very deep (RestNet-110), so this
observation generalizes across different DNN models.

The study on each of the models supports the previous observation that their
output vectors values follow Gaussian distribution. Based on these studies, we
can claim that,

with any independent and identically distributed Gaussian noise on weight, the
output vector of the same input image follows a multi-dimensional Gaussian
distribution111Note that each element of the output are deeply co-related, not
independent over different samples of noise.

This is a very strong claim but is not counter-intuitive. The output of the
first convolution layer is the summation of the multiplication result of
deterministic inputs and Gaussianly distributed weights and is thus a
summation of Gaussian distributions. The summation of Gaussian variables is
also a Gaussian variable, so the output of the first layer is a Gaussian
variable. After activation, the input of the second layer is a transformed
Gaussian variable and after propagating through this layer, with enough number
of operands, the accumulated variable can be approximated by Gaussian
variables. Thus, although the final output may not strictly be a Gaussian
variable, a Gaussian approximation can be observed.

##  4\. Uncertainty Aware Search

###  4.1. Methodology

In addition to understanding the effect of device uncertainties, we propose a
remedy method to reduce the effect of this issue by adopting NAS.

In this work, we propose Uncertainty Aware sEarch (UAE), a more comprehensive
uncertainty aware NAS for better exploration of neural architectures in non-
ideal emerging devices based CiM NN accelerators.

Similar to the state-of-the-art Reinforcement Learning based NAS framework
NACIM (Jiang et al., 2020a), UAE works iteratively and in each iteration: (1)
an LSTM-based _controller_ is used to learn and generate neural architectures;
(2) an uncertainty aware _trainer_ is used to train each generated neural
architecture to get a model to deploy; (3) an uncertainty aware _evaluator_ is
adopted to evaluate the actual performance of the deployed model; (4) the
evaluated performance is used as a reward to update the _controller_ so that
it can generate neural architectures with higher rewards.

The detailed implementation of the uncertainty aware _trainer_ and _evaluator_
are described below.

###  4.2. Uncertainty-Aware Training & Evaluation

We adopt an uncertainty-aware training scheme similar to the one used in NACIM
(Jiang et al., 2020a). The training process is organized the same as
traditional DNN training that, in each iteration, a subset (batch) of the
training data is used to train the model, and after the whole training dataset
has been used to train the model, and an _epoch_ of training is finished and
another _epoch_ is started. The _trainer_ trains the model for multiple
_epochs_ to get a trained model.

The uncertainty-aware training augments the training process for each batch to
learn a DNN model that is more robust against device variations. In each
training batch, before feeding the input into the model, the _trainer_ (1)
save the original weight $W_{Ori}$ of the model; (2) sample a noise from the
uncertainty distribution and add the noise to the weight of the model to form
a $W_{Dep}$; (3) perform forward inference and back propagation in the
perturbed model and collect gradient data for each weight; (4) load the saved
$W_{Ori}$ back to the model and update $W_{Ori}$ with the collected gradient
data via stochastic gradient descent.

Uncertainty-aware training simulates the process of training DNNs directly on
CiM-based accelerators. Experiments in NACIM show that uncertainty-aware
training learns DNN models that are robust against device uncertainties.

The uncertainty-aware evaluation is performed similarly to the training
process. Before evaluation, the _evaluator_ samples an instance of noise from
the uncertainty distribution and add the noise to the trained weight $W_{Ori}$
to get a $W_{Dep}$. The _evaluator_ then evaluates the classification accuracy
of the perturbed model on a test dataset. This process is performed for $K$
times and $K$ different accuracy data are gathered. The _evaluator_ then
report one distributional property (_e.g._ , mean, maximum value, 95% minimum
value) of the $K$ accuracy data to form a _reward_. The distributional
property to be used is specified by the user.

###  4.3. Experimental Results

We demonstrate the effectiveness of UAE by searching for an optimal quantized
CNN for CIFAR-10. The fixed design parameters and hyper-parameters included in
the search space are shown in Table 2. For device uncertainty specifications,
we assume an ISAAC-like (Shafiee et al., 2016) neural accelerator architecture
and a four-bit RRAM device, whose behavioral model is extracted from (Zhao et
al., 2017). The search process is conducted in a GPU server machine with an
Nvidia GTX 1080ti accelerator.

Hyper-Parameters |  | Value choices
---|---|---
Dataset |  | CIFAR-10
Type |  | Quantized CNN
# of Conv Layers |  | 6
# of FC Layers |  | 2
FC Hidden size |  | 1024
# channels |  | (24, 36, 48, 64)
Filter Height/Width |  | (1, 3, 5, 7)
# of integer bits |  | (0, 1, 2, 3)
# of fraction bits |  | (0, 1, 2, 3, 4, 5, 6)
Table 2. Quantized CNN for CIFAR-10 search setups. Upper-half: configurations
fixed to be the same among all searched architectures; lower half: hyper-
parameters to be searched.

As described in Sect. 4.2, there are two major search parameters: the
instances ($K$) of noise sampled for each architecture and the distributional
properties used to form the accuracy data collected by the _evaluator_ into a
_reward_. We test two different values of $K$, 5, and 100 samples, for the
reason that will be explained afterward. We also use two different
distributional properties, one is the mean value of all accuracy data (mean)
and the other is 95% minimum of the accuracy data (95). The mean value
indicates how a model performs under the effect of device uncertainty in
average circumstances and the 95% minimum shows the models’ behavior in worst-
case scenarios.

We offer a comparison for different specifications of UAE and two baseline
methods, quantNAS (Lu et al., 2019), a state-of-the-art NAS framework to
search for the optimal quantized CNN and NACIM (Jiang et al., 2020a), another
uncertainty aware searching framework for CiM-based accelerators. In each
experiment, the NAS _controller_ searches for 2000 different architectures
(episodes) and the _trainer_ trains each generated architecture for 15 epochs.

The DNN models finally presented by each search framework are also evaluated
by mean and 95% minimum value with their accuracy data collected by 10k Monte-
Carol simulation. The experimental result is shown in Table. 3.222Because the
data for quantNAS and NACIM are collected from published work, we do not have
the 95% minimum accuracy result for them.

Method | $K$ | w/o noise | mean | 95 | Time (h)
---|---|---|---|---|---
QuantNAS (Lu et al., 2019) | 0 | 84.92% | 08.48% | N/A | 53
NACIM (Jiang et al., 2020a) | 1 | 73.88% | 73.45% | N/A | 98
UAE-M | 5 | 77.48% | 75.94% | 75.55% | 118
UAE-M | 100 | 82.99% | 79.84% | 77.82% | 255
UAE-95 | 100 | 80.64% | 78.39% | 77.98% | 255
Table 3. Comparison for different specifications of UAE and two other
baselines. Different methods sample different instances of noise ($K$) in
uncertainty-aware evaluation. UAE-M uses mean value of the accuracy data
collected by the _evaluator_ to form a reward and UAE-95 uses the 95% minimum
data. Accuracy without noise shows the model accuracy with an ideal device and
the mean and 95% minimum (95) value shows the behavior of the searched DNN
models under the effect of device uncertainty, evaluated by Monte-Carol
simulation.

Experimental results show that, without uncertainty-aware training, QuantNAS
can identify an optimal DNN model that can offer close to 85% of test
accuracy, but struggles in finding proper neural architectures that are robust
to device uncertainties, as the test accuracy of the DNN model identified by
quantNAS is down to 8.5%, even worse than random guessing (10%). With the help
of uncertainty-aware training, NACIM can identify DNN models that are robust
against the impact of device uncertainties. However, as NACIM only evaluates
the performance of the searched architecture once, the randomness of the
device uncertainty hinders the performance of NACIM, resulting in a model with
only 73.45% of average accuracy. UAE, on the contrary, is able to find DNN
models that are both accurate and robust against the impact of device
uncertainties. When collecting only 5 samples in uncertainty-aware evaluation,
UAE achieves 2.49% higher accuracy than NACIM with a time overhead of only
20%. When collecting 100 samples in uncertainty-aware evaluation, UAE achieves
6.39% higher accuracy than NACIM a search time overhead of 2.5x. Though
further increasing the number of samples is possible, the search time overhead
will be too large to endure. The adoption of a 95% minimum value standard is
also effective. UAE-95 offers 0.16% higher worst-case accuracy than its UAE-M
counterpart with a 1.45% lower average accuracy.

##  5\. Conclusions and Future Works

In this work, we propose a Monte-Carlo simulation-based experimental flow to
measure the device uncertainty-induced perturbations to DNN models. Our
thorough investigation of the behaviors of different DNN models under such
perturbations and shows that the value changes of their output vectors follow
Gaussian distribution. We also propose UAE, a device uncertainty-aware NAS
framework that identifies DNN models that are both accurate and robust against
device uncertainty-induced perturbations. Based on the observations made on
the impact of device uncertainties on the DNN models, the possible future
directions include the formal mathematical proof of the analyzed statistical
behaviors and a time-efficient estimation method for the impact of device
uncertainties.

## Acknowledgment

This work is supported in part by National Science Foundation under grant
CNS-1919167.

## References

  * (1)
  * Alibart et al. (2012) Fabien Alibart, Ligang Gao, Brian D Hoskins, and Dmitri B Strukov. 2012.  High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm.  _Nanotechnology_ 23, 7 (2012), 075201.
  * Bian et al. (2020) Song Bian, Weiwen Jiang, Qing Lu, Yiyu Shi, and Takashi Sato. 2020.  Nass: Optimizing secure inference via neural architecture search.  _arXiv preprint arXiv:2001.11854_ (2020).
  * Chi et al. (2016) Ping Chi, Shuangchen Li, Cong Xu, Tao Zhang, Jishen Zhao, Yongpan Liu, Yu Wang, and Yuan Xie. 2016\.  Prime: A novel processing-in-memory architecture for neural network computation in reram-based main memory.  _ACM SIGARCH Computer Architecture News_ 44, 3 (2016), 27–39.
  * Feinberg et al. (2018) Ben Feinberg, Shibo Wang, and Engin Ipek. 2018.  Making memristive neural network accelerators reliable. In _2018 IEEE International Symposium on High Performance Computer Architecture (HPCA)_. IEEE, 52–65.
  * Ielmini and Wong (2018) Daniele Ielmini and H-S Philip Wong. 2018.  In-memory computing with resistive switching devices.  _Nature Electronics_ 1, 6 (2018), 333–343.
  * Jiang et al. (2020a) Weiwen Jiang, Qiuwen Lou, Zheyu Yan, Lei Yang, Jingtong Hu, X Sharon Hu, and Yiyu Shi. 2020a.  Device-circuit-architecture co-exploration for computing-in-memory neural accelerators.  _IEEE Trans. Comput._ (2020).
  * Jiang et al. (2020b) Weiwen Jiang, Lei Yang, Sakyasingha Dasgupta, Jingtong Hu, and Yiyu Shi. 2020b.  Standing on the shoulders of giants: Hardware and neural architecture co-search with hot start.  _arXiv preprint arXiv:2007.09087_ (2020).
  * Jiang et al. (2020c) Weiwen Jiang, Lei Yang, Edwin H-M Sha, Qingfeng Zhuge, Shouzhen Gu, Sakyasingha Dasgupta, Yiyu Shi, and Jingtong Hu. 2020c.  Hardware/Software co-exploration of neural architectures.  _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ (2020).
  * Jiang et al. (2019) Weiwen Jiang, Xinyi Zhang, Edwin H-M Sha, Lei Yang, Qingfeng Zhuge, Yiyu Shi, and Jingtong Hu. 2019.  Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search. In _Proceedings of the 56th Annual Design Automation Conference 2019_. ACM, 5.
  * Krizhevsky et al. (2009) Alex Krizhevsky et al. 2009\.  Learning multiple layers of features from tiny images.  (2009).
  * LeCun et al. (1998) Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998.  Gradient-based learning applied to document recognition.  _Proc. IEEE_ 86, 11 (1998), 2278–2324.
  * Li et al. (2020) Yuhong Li, Cong Hao, Xiaofan Zhang, Xinheng Liu, Yao Chen, Jinjun Xiong, Wen-mei Hwu, and Deming Chen. 2020\.  EDD: Efficient Differentiable DNN Architecture and Implementation Co-search for Embedded AI Solutions.  _arXiv preprint arXiv:2005.02563_ (2020).
  * Liu et al. (2018) Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018\.  DARTS: Differentiable Architecture Search. In _International Conference on Learning Representations_.
  * Lu et al. (2019) Qing Lu, Weiwen Jiang, Xiaowei Xu, Yiyu Shi, and Jingtong Hu. 2019.  On Neural Architecture Search for Resource-Constrained Hardware Platforms. In _International Conference on Computer-Aided Design (ICCAD)_. ACM, 1.
  * Niu et al. (2012) Dimin Niu, Yang Xiao, and Yuan Xie. 2012.  Low power memristor-based ReRAM design with error correcting code. In _17th Asia and South Pacific Design Automation Conference_. IEEE, 79–84.
  * Shafiee et al. (2016) Ali Shafiee, Anirban Nag, Naveen Muralimanohar, Rajeev Balasubramonian, John Paul Strachan, Miao Hu, R Stanley Williams, and Vivek Srikumar. 2016.  ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars.  _ACM SIGARCH Computer Architecture News_ 44, 3 (2016), 14–26.
  * Sze et al. (2017) Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer. 2017\.  Efficient processing of deep neural networks: A tutorial and survey.  _Proc. IEEE_ 105, 12 (2017), 2295–2329.
  * Wu et al. (2019) Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. 2019\.  Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 10734–10742.
  * Xu et al. (2013) Cong Xu, Dimin Niu, Naveen Muralimanohar, Norman P Jouppi, and Yuan Xie. 2013.  Understanding the trade-offs in multi-level cell ReRAM memory design. In _2013 50th ACM/EDAC/IEEE Design Automation Conference (DAC)_. IEEE, 1–6.
  * Yan et al. (2020a) Xingang Yan, Weiwen Jiang, Yiyu Shi, and Cheng Zhuo. 2020a.  Ms-nas: Multi-scale neural architecture search for medical image segmentation. In _International Conference on Medical Image Computing and Computer-Assisted Intervention_. Springer, 388–397.
  * Yan et al. (2020b) Zheyu Yan, Yiyu Shi, Wang Liao, Masanori Hashimoto, Xichuan Zhou, and Cheng Zhuo. 2020b.  When Single Event Upset Meets Deep Neural Networks: Observations, Explorations, and Remedies. In _2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC)_. IEEE, 163–168.
  * Yang et al. (2020a) Lei Yang, Weiwen Jiang, Weichen Liu, HM Edwin, Yiyu Shi, and Jingtong Hu. 2020a.  Co-exploring neural architecture and network-on-chip design for real-time artificial intelligence. In _2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC)_. IEEE, 85–90.
  * Yang et al. (2020b) Lei Yang, Zheyu Yan, Meng Li, Hyoukjun Kwon, Liangzhen Lai, Tushar Krishna, Vikas Chandra, Weiwen Jiang, and Yiyu Shi. 2020b.  Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks.  _arXiv preprint arXiv:2002.04116_ (2020).
  * Zeng et al. (2020) Dewen Zeng, Weiwen Jiang, Tianchen Wang, Xiaowei Xu, Haiyun Yuan, Meiping Huang, Jian Zhuang, Jingtong Hu, and Yiyu Shi. 2020.  Towards Cardiac Intervention Assistance: Hardware-aware Neural Architecture Exploration for Real-Time 3D Cardiac Cine MRI Segmentation.  _arXiv preprint arXiv:2008.07071_ (2020).
  * Zhao et al. (2017) Meiran Zhao, Huaqiang Wu, Bin Gao, Qingtian Zhang, Wei Wu, Shan Wang, Yue Xi, Dong Wu, Ning Deng, Shimeng Yu, et al. 2017\.  Investigation of statistical retention of filamentary analog RRAM for neuromophic computing. In _2017 IEEE International Electron Devices Meeting (IEDM)_. IEEE, 39–4.
  * Zoph and Le (2017) Barret Zoph and Quoc V Le. 2017.  Neural architecture search with reinforcement learning. In _International Conference on Learning Representations (ICLR)_.
  * Zoph et al. (2018) Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. 2018.  Learning transferable architectures for scalable image recognition. In _IEEE conference on Computer Vision and Pattern Recognition (CVPR)_. 8697–8710.