# Learn to Unlearn: A Survey on Machine Unlearning

Youyang Qu, Xin Yuan, Ming Ding,
Wei Ni, Thierry Rakotoarivelo,
and David Smith,
Youyang Qu, Xin Yuan, Ming Ding, Wei Ni, Thierry Rakotoarivelo, and David
Smith are with Data61, Commonwealth Scientific and Industrial Research
Organization, Australia. Email: {youyang.qu, xin.yuan, ming.ding, wei.ni,
thierry.rakotoarivelo<EMAIL_ADDRESS>
###### Abstract

Machine Learning (ML) models contain private information, and implementing the
right to be forgotten is a challenging privacy issue in many data
applications. Machine unlearning has emerged as an alternative to remove
sensitive data from a trained model, but completely retraining ML models is
often not feasible. This survey provides a concise appraisal of Machine
Unlearning techniques, encompassing both exact and approximate methods,
probable attacks, and verification approaches. The survey compares the merits
and limitations each method and evaluates their performance using the
Deltagrad exact machine unlearning method. The survey also highlights
challenges like the pressing need for a robust model for non-IID deletion to
mitigate fairness issues. Overall, the survey provides a thorough synopsis of
machine unlearning techniques and applications, noting future research
directions in this evolving field. The survey aims to be a valuable resource
for researchers and practitioners seeking to provide privacy and equity in ML
systems.

###### Index Terms:

Machine Unlearning, Exact Unlearning, Approximate Unlearning, Verification,
Attacks

##  I Introduction

Nowadays, Machine Learning (ML) models are extensively deployed, whereas
immutable ones are scarce and unworkable. In most real-world scenarios, ML
models need to be retrained on slightly altered datasets, for example, when
data points have been removed or revised [1]. A vast array of exigent
applications has been identified, including privacy preservation, resilience,
bias reduction, and uncertainty quantification [2].

One notable mechanism could be the elimination of data points from the
training data to comply with items such as the right to be forgotten from
privacy laws and regulations like the GDPR [3, 4]. The establishment of ML
models that exclude specific data points is also essential in training bias-
corrected models, such as in jackknife resampling, which involves retraining
the model on all leave-one-out datasets. Moreover, training models on subsets
of data can be utilized for uncertainty quantification, such as constructing
statistically reliable prediction intervals via conformal prediction.
Retraining ML models are often time- and computationally-expensive, which
makes such process impractical or even infeasible [3, 5, 6].

Figure 1: Diagram of General Scenario for Machine Unlearning

Machine unlearning has emerged as an alternative approach to complete model
retraining, with two primary categories being actively researched, namely
exact machine unlearning and approximate machine unlearning, along with
associated new verification methods and related attacks. Besides, most machine
unlearning methods adopt Newton’s method as the loss function and build the
unlearn model upon it. We also conduct a mechanism study on it. Our objective
is to present a succinct yet methodical perspective of the current status of
machine unlearning research, which is shown in Fig. 1. The main contributions
of this article are:

  * •

A survey on existing leading research on exact machine unlearning and
approximate machine unlearning, with the resulting taxonomy presented in Table
I.

  * •

A discussion on the verification of machine unlearning as well as relevant
attacks.

  * •

A discussion on the adoption of Newton’s optimization method for convex loss
functions.

  * •

Some experimental results to show key insights into machine unlearning.

TABLE I: Taxonomy of Machine Unlearning Category |  Ref. |  Use Case |  Main Contributions |  Limitations
---|---|---|---|---
Exact Unlearning |  Cao et al. [3] |  Converting learning algorithms into a summation form for efficiently forgetting data lineage |  The idea of forgetting systems that can restore privacy, security, and usability by completely and quickly erasing data lineage A universal unlearning method that transforms learning algorithms into a summation form, facilitating efficient forgetting of data lineage Real-world data pollution attacks developed against systems and algorithms |  Not every machine learning algorithm can be converted to the summation form Data lineage at many levels of granularity is not considered
|  Bourtoule et al. [7] |  SISA: Data shard and incremental unlearning |  Consider non-uniform distributions of unlearning requests Introduce SISA, which stands for Sharded, Isolated, Sliced, and Aggregated training Demonstrate that for complex learning tasks, a combination of transfer learning and SISA training results in a slight decrease in accuracy (2̃ percentage points) with improved retraining time. |  Performance degradation in each constituent model when datasets are too small, or if the learning task is complex The unlearning through SISA is hard to verify
|  Wu et al. [2] |  Storing the original gradients, retaining the deleted data, and then integrating |  Designed to quickly retrain machine learning models based on stochastic gradient descent, in response to small changes in the data, such as the addition or deletion of a small number of points Supported by theoretical results that demonstrate its accuracy, with the error being smaller than the fraction of points removed, for both GD and SGD methods |  Extra storage resources required to store the history gradients Smaller mini-batch sizes in SGD is not supported ML models without strong convexity may fail to converge
|  Brophy et al. [8] |  Machine unlearning for random forests |  Present a novel variant of random forests called DaRE, which enables the swift removal of training instances Assess DaRE’s performance in handling sequences of deletions on 13 binary classification datasets from the real world Demonstrate that DaRE can remove data instances with a speed advantage of 2-4 orders, while incurring a predictive performance loss of less than 1% |  May leak privacy before and after deletion Only applies to Random Forests Lead to a slight degradation of predictive performance
Approx. Unlearning |  Guo et al. [5] |  Training a certified machine learning model considering the data removal in advance |  Present a method for removing data points from L2-regularized linear models, which are trained using a differentiable convex loss function, in a certified manner Utilize a Newton step to effectively remove the influence of the removed data point on the model parameters Proposes a technique for masking the residual by randomly perturbing the training loss to prevent adversaries from extracting information from the small residual |  Inverting the Hessian matrix required by the Newton update removal mechanism poses difficulties. Not applicable to models with non-convex losses A significant disparity exists between our data-dependent bound and the actual gradient residual norm
|  Golatkar et al. [9] |  Defining a forgetting Lagrangian to achieve selective forgetting for deep neural networks |  Define “selective forgetting” for neural network unlearning models Proposea a scrubbing procedure that removes information from the trained weights without accessing the raw training data or re-training from sketch Derive a computable upper bound to the amount of the retained information that is efficient to compute even for DNNs. |  It has been observed that if pretraining assumption is not considered while forgetting, even minor perturbations during the initial critical learning phase can result in significant variations in the final solution
|  Sekhari et al. [10] |  rigorous study of generalization in machine unlearning, where the goal is to perform well on previously unseen data points |  Explore the number of samples that can be unlearned while preserving performance Do not rely on the availability of training data during sample deletion, distinguishing it from prior work Show a strict separation between DP and machine unlearning |  Fail to provide dimension dependent information-theoretic lower bound on the deletion capacity Cannot deal with non-convex loss functions
|  Ginart et al. [11] |  Machine unlearning for K-means |  Introduced and Characterized the problem of efficient data deletion in machine learning Propose two different solutions for achieving deletion efficiency in k-means clustering, both of which offer theoretical guarantees and strong empirical performance synthesize 4 general engineering principles that can guide the design of deletion-efficient learning systems |  Only supports the deletion of a single data point at one time Only applies to K-means An assumption that there is only one model and one corresponding dataset
|  Nguyen et al. [6] |  Machine unlearning for variational Bayesian machine learning |  Extend machine unlearning to Variational Bayesian Unlearning Evaluate unlearning methods on Bayesian models, sparse Gaussian process, logistic regression |  New vulnerabilities of model fraud attacks Only applies to the variational inference frameworks of Bayesian machine learning
Verification & Attacks |  Sommer et al. [12] |  Probabilistic verification of machine unlearning |  Framework for machine unlearning verification Use data backdoors for verifying machine unlearning Assess the verification performance using multiple datasets |  Constraints on data samples Conflicting backdoor patterns Risk of backdoor injection Potential vulnerability to backdoor attack detection mechanisms
|  Chen et al. [13] |  Proposed member inference attacks on machine unlearning |  Introduce a novel membership inference attack in the machine unlearning setting that aims to detect if a target sample is present in the training set of the original model Quantify the privacy risks in machine unlearning by examining membership inference attacks Devise 4 defense methods to avoid the privacy risks arising from this attack |  Performance drops for SISA method Performance drops if the unlearned/updating samples represents more than 0.2%
|  Marchant et al. [14] |  Poisoning attacks on certified machine unlearning |  A novel poisoning attacks on machine unlearning Take into account different factors such as white-box and grey-box attacks, various perturbation geometries and bounds, balancing the attacker’s objective optimality with the computation time, and the feasibility of long-term attack execution |  Only targets on a particular category of machine unlearning models May not work well if off-the-shell anomaly detection models have been deployed

##  II Legislation Related to Machine Unlearning

In 2016, the European Union (EU) implemented the General Data Protection
Regulation (GDPR), widely recognized as the most stringent legislation
regarding data privacy and security globally [4]. This regulation enforces
obligations on organizations worldwide that target or collect data relating to
individuals within the EU. It recognizes novel data privacy rights to grant
individuals greater autonomy over their data, including the right to erasure
(i.e. right to be forgotten).

This right to erasure holds significant relevance to machine unlearning. More
specifically, this data right permits individuals to request that data
curators eliminate their individual data from the training dataset of machine
learning models, and potentially the trained models themselves, when certain
conditions are met, such as when one objects to the data processing and one’s
personal data are being processed for direct marketing purposes.

In 2018, California passed the California Consumer Privacy Act (CCPA)
111https://oag.ca.gov/privacy/ccpa, providing consumers with substantial
control over the personal information that businesses collect about them. It
defines personal information as data that identifies, relates to, describes,
is capable of being linked with, or could be linked to, a particular consumer
or household. The CCPA grants Californian consumers the entitlement to
discover what personal information businesses have collated on them, how it is
utilized and shared, the right to delete the collected personal information
(with certain exceptions), the right to decline the selling of their personal
information, and the right to non-discrimination for using their CCPA rights.

The right to delete data in the CCPA is the equivalent of the right to erasure
in the GDPR. Recently, California established the California Privacy Rights
Act (CPRA) 222https://oag.ca.gov/privacy/ccpa, also known as “CCPA 2.0”. This
Act extensively expands and modifies the CCPA, rendering it comparable to the
GDPR. The CPRA came into effect in January 2023.

##  III Exact Machine Unlearning

Exact machine learning methods aim at eliminating the influence of the
particular segment/proportion of data to be removed completely. Generally,
some degree of retraining is necessary. The major challenges encountered with
this approach involve retraining efficiency and the subsequent performance of
the retrained model.

A seminal work in this field was carried out by Cao et al. [3], who were the
first to introduce the concept of machine unlearning. In their research, they
proposed a heuristic approach to transforming a machine learning algorithm
into a summation format, which can effectively eliminate the impact of the
data lineage. Furthermore, they took into account the non-uniform distribution
of unlearning requests in real-world situations. However, not all machine
learning algorithms can be converted to the summation format, and the data
lineage with various levels of granularity may not always be considered.

To further advance exact machine unlearning techniques, Bourtoule et al. [7]
proposed another method called SISA, which improves on the generality and the
performance of the retrained model. SISA, which stands for “Sharded, Isolated,
Sliced, and Aggregated training”, achieves machine unlearning through
incremental unlearning on sharded data. The method involves grouping the data
into several isolated shards and further slicing the data in each shard.
Incremental learning is then carried out, and parameters are stored. When a
new unlearning request is made, SISA traces back to the corresponding data
slice and retrains from there. Since the other model parameters are already
stored, aggregation is easy and efficient. However, SISA may encounter
performance degradation in each constituent model when datasets are too small,
or the learning task is complex.

DeltaGrad [2] is an exact machine unlearning method that focuses on rapid
retraining. It achieves this by training the data to be deleted in reverse,
i.e., to maximize the loss instead of minimizing it in classic machine
learning scenarios. The obtained reversed model is then aggregated with the
raw model, which is stored in advance. With Deltagrad, the model’s performance
could be well-maintained when constrained by the proportion of data to be
deleted (1% in the Deltagrad paper). However, while DeltaGrad supports
stochastic gradient descent machine learning algorithms, it cannot handle
mini-batch sizes.

In lieu of broad-based machine unlearning techniques, targeted approaches are
being developed for specific machine learning algorithms. For example, Brophy
et al. [8] introduced “DaRE”, which is custom-designed for the random forest
algorithm. Despite a slight degradation in model performance, DaRE can rapidly
retrain and improve efficiency by 2 to 4 orders of magnitude compared to
retraining from scratch. However, DaRE is only applicable to the random forest
algorithm and cannot be applied to other machine learning algorithms.

In conclusion, there has been extensive research into exact machine unlearning
from various angles. Until now, most studies have targeted on fast retraining
strategies with little or ideally no sacrifice in model performance. In
theory, the deleted data can be verified through the retrained model since the
objective of exact machine unlearning is to physically eradicate the impact of
the removed data without disturbing the retrained model.

##  IV Approximate Machine Unlearning

While exact machine unlearning methods have made significant strides towards
achieving the right to erasure, researchers have identified new challenges
from two other perspectives. The first is that exact machine unlearning
inevitably results in model performance degradation, even if the degradation
is sometimes minor. The second is the potential for privacy leakage. For
example, if “Alice” is removed while the raw model differs from the retrained
model, an adversary could discern that the difference is due to the removal of
“Alice” and launch an inference attack to compromise privacy further.

The above two challenges can be mitigated by utilizing approximate machine
unlearning methods, which mask the difference between models before and after
data removal, while simultaneously optimizing model performance.

One of the prominent approximate machine unlearning methods is “certified-
removal” proposed by Guo et al. [5]. This paper presents a mechanism for
eliminating data points from L2-regularized linear models that have been
trained using a differentiable convex loss function. Their approach leverages
a Newton step on the model parameters to effectively remove the influence of
the data point being removed. Moreover, the residual is masked by randomly
perturbing the training loss to prevent potential privacy breaches. Despite
being a pioneering research, there are some limitations to the proposed
approach. Firstly, Newton’s optimization method necessitates the inversion of
the Hessian matrix, which can be a challenging task. Secondly, this approach
is not applicable to models with non-convex losses. Lastly, the data-dependent
bound still falls short of accurately estimating the gradient residual norm,
highlighting the need for further refinement and analysis.

To explore approximate machine unlearning for complex models, particularly
deep neural network (DNN) models, Golatkar et al. [9] introduced a forgetting
Lagrangian to achieve selective forgetting in DNNs. Expanding on this idea,
the authors developed a scrubbing technique that can erase information from
trained weights without requiring access to the original training data or
necessitating a complete network retraining. Furthermore, they developed a
computable upper bound to quantify the retained information, which can be
efficiently computed for Deep Neural Networks (DNNs). Nonetheless, when it
comes to forgetting without the pre-training assumption, studies have found
that even minor perturbations during the critical learning phase can result in
substantial variations in the final solution.

The aforementioned two representative research works used differential privacy
mechanisms to perturb the retrained model parameters, leading to the question
of whether machine unlearning can be considered a special case of differential
privacy in the machine learning domain. Sekhari et al. [10] addressed this
issue by conducting a thorough investigation of generalization in machine
unlearning, with the objective of achieving favorable performance on novel
data points. Unlike previous research, the algorithms employed in this study
do not necessitate the unlearning algorithm to have access to the training
data while deleting samples. This study introduces a clear distinction between
differential privacy and machine unlearning; however, it has a few drawbacks.
Specifically, it does not provide dimension-dependent information-theoretic
lower bounds on the deletion capacity and cannot handle non-convex loss
functions.

Notwithstanding the perturbation-based approaches, Nguyen et al. [6] have put
forward a machine unlearning technique for variational Bayesian machine
learning, which is a crucial subfield of machine learning. They have achieved
this by incorporating Bayesian theory to the model prior and posterior
knowledge and expanding machine unlearning to the Bayesian domain. While they
have utilized stochastic methods, the authors have approached this matter from
a probability theory standpoint rather than relying on the randomized noise
injection method used in differential privacy. This unlearning technique will
not impede model performance and is solely limited to this particular division
of machine learning models.

In the domain of approximate machine unlearning, there exist solutions
specifically designed for certain machine learning models. For instance, there
are machine unlearning methods developed by Ginart et al. [11] that are
tailored for K-means. These methods are aimed at achieving a balance between
efficiency and model performance. It is worth noting that these methods are
limited to removing only one record per round. Additionally, the authors
synthesized four overarching principles for designing efficient machine
unlearning methods. These principles are linearity, laziness, modularity, and
quantization.

To sum up, approximate machine unlearning places greater emphasis on
maintaining model performance while adding extra privacy protection to the
retrained model. Nonetheless, it also faces several challenges, such as the
difficulty in verifying the execution of machine unlearning methods. Indeed
merely from the observation of outputs, it is nearly impossible to discern
whether the unlearning process has taken place or not. Additionally, since
almost all approximate machine unlearning models begin with the model, rather
than the data, it raises concerns about whether this method aligns with the
“right to be forgotten” as defined in privacy laws and regulations.

##  V Verification and Attacks of Unlearning Algorithms

Mechanisms for systematic verification of erasure and attacks on machine
unlearning have emerged in parallel with the design of these new unlearning
methods. When conducting this survey, we only found one research contribution
related to the former [12], and a few other contributions related to the
latter [13, 14].

On the topic of the verification mechanism, Sommer et al. [12] have researched
probabilistic verification of machine unlearning. For the data to be deleted,
data owners intentionally inject a backdoor into the data before sending it to
the data user (e.g., Internet unicorn companies). After the data is claimed to
be deleted by the data user, the data owner can testify whether the data is
deleted by verifying the pre-injected backdoor. In addition to this, several
methodologies can help with the verification of machine unlearning, including
but not limited to data sampling verification, synthetic data verification,
and bias analysis. For example, analyzing the data for bias involves
evaluating the impact of data removal on the bias of the machine learning
model. This can involve measuring the fairness and accuracy of the model
before and after data removal, or analyzing the distribution of the data to
determine the impact of data removal on the representation of different
groups. The mechanism has been shown to be effective over various datasets and
networks. However, some limitations include constraints on data samples,
conflicting backdoor patterns, the risk of backdoor injection, and the
potential vulnerability to backdoor attack detection mechanisms.

Membership inference is a type of privacy-oriented attack that has been
proposed against ML models [15], and is also relevant to machine unlearning.
In their recent work, Chen et al. [13] introduced a new method for conducting
membership inference attacks in the context of machine unlearning. This
approach enables them to determine whether a given sample belongs to the
training set of the original model, shedding light on the unintended privacy
risks associated with machine unlearning. However, the success rates of the
attack drop if certain conditions are met, such as the deployment of the SISA
method and the need to delete more than 0.2% of data samples.

In addition to the attacks discussed above, poisoning attacks are a popular
form of attack in machine learning applications, including machine unlearning.
Marchant et al. [14] have devised an innovative poisoning attack that
considers a range of factors, such as white-box and grey-box attacks, diverse
perturbation geometries and limits, optimization of the attacker’s objective
in balance with computation time, and the viability of conducting the attack
over a longer duration. However, this attack is specific to a particular
category of machine unlearning models, namely certified-removal and its
variants. It may suffer from performance degradation if off-the-shelf anomaly
detection models have been deployed.

##  VI Further Discussion on Newton’s Method for Machine Unlearning

In recent studies on unlearning (both exact and approximate), the model
employed in research heavily relies on Newton’s method to optimize the
quadratic loss function [1] [2] [5]. Particularly, these unlearning methods
adopt the approach of “reverse learning” such as Deltagrad [2]. This Section
discusses this method in more detail.

In Fig. 2, we use a blue line to represent a cost function, and we aim to
reduce it to the optimal point. This process is similar to what we previously
discussed in Newton’s method of finding roots. In short, Newton’s method
employs the first-order calculation for root finding. However, in
optimization, Newton’s method relies on the second-order calculation, where
the Taylor series with a second-order approximation is used to estimate the
loss function, denoted by $f(x)$. The orange and red dashed lines in Fig. 2
exhibit such an approximation.

Let us assume that the solution is currently situated at $x_{0}$. To initiate
Newton’s method, we start by finding the second-order approximation of $f(x)$
in the vicinity of $x_{0}$ and then locate the position of its minimum value.
As depicted in Fig. 2, this point is represented by $x_{1}$, which can be
found by setting the first derivative to zero. We then obtain the next
approximation of $f(x)$ in the proximity of $x_{1}$, locate the minimum value
of this approximation, and repeat the process until we reach the optimal
point.

Figure 2: Newton’s method to find the minimum value of the loss function

##  VII Evaluation and Experimental Results

This section highlights the discoveries from our experiments on machine
unlearning. Specifically, our experiments are based on Deltagrad [2], which is
considered to be one of the most prominent and precise machine unlearning
techniques. As previously mentioned, the key approach employed in this study
is “reverse learning”. To elaborate further, Deltagrad amplifies the loss of
the excluded data samples in the model training, instead of using the
conventional method that minimizes the loss.

###  VII-A Evaluation Methodology

Our evaluation procedure of machine unlearning is as follows. Firstly, we
explore the relationship between the accuracy of the model and the percentage
of deleted data samples in the independent and identically distributed (IID)
scenario. Secondly, we investigate the same correlation in the non-IID
situation. Thirdly, we assess the time consumption associated with the
proportion of the deleted data samples.

For our experiments, we used the original
“Deltagrad”333https://github.com/thuwuyinjun/DeltaGrad implementation and
further extend it with new features to accommodate the evaluation of non-IID
deletion and accuracy of each class of different
datasets444https://github.com/DrQu89757/Extened_code_from_DeltaGrad. These
evaluations and related results were not considered in the original
“Deltagrad” article. All presented results are the mean values over 20 rounds
of experiments, with error bars showing the corresponding standard deviations.

(a) IID

(b) non-IID

Figure 3: Evaluation of accuracy in IID and non-IID settings

###  VII-B Experiment Environments and Settings

We conducted our experiments on a Windows-based machine equipped with an Intel
Xeon KVM<EMAIL_ADDRESS>and 16G of memory. Our model was a two-layer neural
network consisting of 300 hidden ReLU neurons, which we trained on the MNIST
dataset. The MNIST dataset comprises 10 labels, and contains 60,000 images for
training and 10,000 images for testing. Each image is composed of 28 × 28
features (pixels) and represents a single digit ranging from 0 to 9. We
applied L2 regularization with a rate of 0.001, and utilized a decaying
learning rate strategy. Specifically, we set the learning rate to 0.2 for the
first half of the training iterations and reduced it to 0.1 for the remaining
iterations. We also utilized deterministic gradient descent as our
optimization algorithm.

Regarding our hyperparameter configuration, we set $T_{0}=5$ as the period of
explicit gradient updates, and $j_{0}=10$ as the length of the initial “burn-
in”. Specifically, for the two-layer DNN, we set $T_{0}=2$, which is even
smaller, and we use the first quarter of the iterations as the “burn-in”. The
history size m is set to 2 for all of our experiments.

In the following experiments, we will assess the model’s performance by
varying the deletion ratios, which are set to 0.001, 0.005, 0.01, 0.05, 0.1,
0.2, and 0.3, respectively. Furthermore, we study the effect of machine
unlearning on data samples deleted in a non-IID manner.

###  VII-C Experimental Results

This subsection presents our primary evaluation results, including accuracy
evaluation in both IID and non-IID settings, as well as the inversion of time
consumption. The models examined in this evaluation are as follows:

  * •

Baseline Training: training a model using the whole dataset (in Red color).

  * •

Retraining from the sketch: retraining a model using the remaining dataset
after deletion (in green color).

  * •

Deltagrad: retraining a model using the deleted samples (in blue).

####  VII-C1 Evaluation of accuracy in an IID setting

In order to evaluate the accuracy of the model under an IID setting, we
randomly removed data samples using a uniform distribution. Fig. 3a
illustrates that the testing accuracy of all three models remains consistent
throughout this process. As a result, we can conclude that the baseline
model’s accuracy is not affected by this form of data deletion. However, for
the other two models, the degradation in testing accuracy is limited,
decreasing from approximately 0.8735 to 0.8725 (less than 0.1%).

####  VII-C2 Evaluation of accuracy in a non-IID setting

To evaluate the model accuracy in a non-IID setting, we delete more samples
with the label “4” than the other labels. In more detail, 30% of the deleted
samples are with the label “4”, while the rest 70% of the deleted samples have
a uniform distribution over the 9 labels without “4”. In this non-IID setting,
the performance degradation of two “unlearn” model is quite significant. As
shown in Fig. 3b, the accuracy degradation of Deltagrad is from around 0.8735
to 0.7921 (over 8%) when the deletion ratio increases to 30%. This is mainly
caused by the significant absence of label “4”. Noticeably, the classification
accuracy of Label “9” increases from 82.9% (deletion ratio: 0.001) to 88.3%
(deletion ratio: 0.3).

Figure 4: Evaluation of time consumption

####  VII-C3 Evaluation of time consumption

In Fig. 4, the time consumption v.s. deletion ratio in an IID setting is
displayed. In particular, to show the crossover point, we evaluate two more
deletion ratio values, which are 0.4 and 0.5. With the increase in the
deletion ratio from 0.001 to 0.3, the time consumption of “Re-training from
scratch” decreases from 42.19s to 29.15s. Meanwhile, the time consumption of
“Deltagrad” increases from 12.53s to 21.22s. At last, there will exist a
cross-over point. It is worth mentioning that the time consumption trend in a
non-IID setting is almost the same.

###  VII-D Insights

Based on our analysis and evaluation, we shed some insightful light on this
profound topic of machine unlearning as follows.

Training from scratch or not? Given the negligible degradation of accuracy
comparing training-from-scratch with Deltagrad in the IID setting, we can
conclude that the machine unlearning methods should be selected according to
other performance metrics such as computational complexity.

Non-IID machine unlearning is challenging. It has been widely acknowledged
that non-IID data distribution in distributed learning paradigms, such as
federated learning, leads to suboptimal model performance. In the context of
unlearning, the removal of non-IID samples also results in a tremendous
performance loss, which must be taken into consideration in practice.

Unlearning leads to Unfairness. When data samples are erased, particularly in
a non-IID manner, the classification accuracy of certain labels can
significantly degrade while the accuracy of others may improve. This
phenomenon would cause unfairness across different classes, which cannot be
revealed by traditional machine learning performance metrics such as average
accuracy.

Unlearning calls for novel and more generalized loss function formulation
methods. The majority of the present machine unlearning methods rely on
maximizing a quadratic loss function using Newton’s method. Nonetheless, in
practical scenarios, various forms of loss functions exist. Exploring the uses
of these alternatives may lead to the design of more adaptable machine
unlearning mechanisms in the future.

##  VIII Summary

In summary, machine unlearning holds great promise for addressing concerns of
privacy and equity in machine learning. In this survey article, we have
presented a comprehensive examination of current machine unlearning
techniques, including both exact and approximate unlearning methods, and
compared various techniques. The merits and shortcomings of each method have
been discussed, and our experimental evaluations have demonstrated that
machine unlearning can remove sensitive data or biases from a trained model
without significantly impairing its performance, particularly in IID settings.
However, there exist significant hurdles to overcome, such as developing
robust methods for non-IID sample deletion and evaluating the effectiveness of
machine unlearning techniques against potential attacks. We believe that
further research in this field will continue to advance the state-of-the-art
in machine unlearning, allowing for the development of more robust and
trustworthy machine-learning systems that meet the needs of diverse
stakeholders.

## References

  * [1] E. Chien, C. Pan, and O. Milenkovic, “Efficient model updates for approximate unlearning of graph-structured data,” in _International Conference on Learning Representations_ , 2023.
  * [2] Y. Wu, E. Dobriban, and S. Davidson, “Deltagrad: Rapid retraining of machine learning models,” in _International Conference on Machine Learning_. PMLR, 2020, pp. 10 355–10 366.
  * [3] Y. Cao and J. Yang, “Towards making systems forget with machine unlearning,” in _2015 IEEE Symposium on Security and Privacy_. IEEE, 2015, pp. 463–480.
  * [4] P. Voigt and A. Von dem Bussche, “The EU general data protection regulation (GDPR),” _A Practical Guide, 1st Ed., Cham: Springer International Publishing_ , vol. 10, no. 3152676, pp. 10–5555, 2017.
  * [5] C. Guo, T. Goldstein, A. Hannun, and L. Van Der Maaten, “Certified data removal from machine learning models,” in _Proceedings of the 37th International Conference on Machine Learning_ , 2020, pp. 3832–3842.
  * [6] Q. P. Nguyen, B. K. H. Low, and P. Jaillet, “Variational bayesian unlearning,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 16 025–16 036, 2020.
  * [7] L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, “Machine unlearning,” in _2021 IEEE Symposium on Security and Privacy (SP)_. IEEE, 2021, pp. 141–159.
  * [8] J. Brophy and D. Lowd, “Machine unlearning for random forests,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 1092–1104.
  * [9] A. Golatkar, A. Achille, and S. Soatto, “Eternal sunshine of the spotless net: Selective forgetting in deep networks,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 9304–9312.
  * [10] A. Sekhari, J. Acharya, G. Kamath, and A. T. Suresh, “Remember what you want to forget: Algorithms for machine unlearning,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 18 075–18 086, 2021.
  * [11] A. Ginart, M. Guan, G. Valiant, and J. Y. Zou, “Making ai forget you: Data deletion in machine learning,” _Advances in neural information processing systems_ , vol. 32, 2019.
  * [12] D. M. Sommer, L. Song, S. Wagh, and P. Mittal, “Athena: Probabilistic verification of machine unlearning,” _Proceedings on Privacy Enhancing Technologies_ , vol. 3, pp. 268–290, 2022.
  * [13] M. Chen, Z. Zhang, T. Wang, M. Backes, M. Humbert, and Y. Zhang, “When machine unlearning jeopardizes privacy,” in _Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security_ , 2021, pp. 896–911.
  * [14] N. G. Marchant, B. I. Rubinstein, and S. Alfeld, “Hard to forget: Poisoning attacks on certified machine unlearning,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 36, no. 7, 2022, pp. 7691–7700.
  * [15] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in _2017 IEEE Symposium on Security and Privacy (SP)_ , 2017, pp. 3–18.


# A machine-learning sleep-wake classification model using a reduced number of
features derived from photoplethysmography and activity signals

Douglas A. Almeida
Heart Institute (InCor)
Clinics Hospital University of
Sao Paulo Medical School
Sao Paulo - SP - Brazil
<EMAIL_ADDRESS>
&Felipe M. Dias
Heart Institute (InCor)
Clinics Hospital University of
Sao Paulo Medical School
Sao Paulo - SP - Brazil
<EMAIL_ADDRESS>
&Marcelo A. F. Toledo
Heart Institute (InCor)
Clinics Hospital University of
Sao Paulo Medical School
Sao Paulo - SP - Brazil
<EMAIL_ADDRESS>
&Diego A. C. Cardenas
Heart Institute (InCor)
Clinics Hospital University of
Sao Paulo Medical School
Sao Paulo - SP - Brazil
<EMAIL_ADDRESS>
&Filipe A. C. Oliveira
Heart Institute (InCor)
Clinics Hospital University of
Sao Paulo Medical School
Sao Paulo - SP - Brazil
<EMAIL_ADDRESS>
&Estela Ribeiro
Heart Institute (InCor)
Clinics Hospital University of
Sao Paulo Medical School
Sao Paulo - SP - Brazil
<EMAIL_ADDRESS>
&Jose E. Krieger
Heart Institute (InCor)
Clinics Hospital University of
Sao Paulo Medical School
Sao Paulo - SP - Brazil
<EMAIL_ADDRESS>
&Marco A. Gutierrez
Heart Institute (InCor)
Clinics Hospital University of
Sao Paulo Medical School
Sao Paulo - SP - Brazil
<EMAIL_ADDRESS>
###### Abstract

Sleep is a crucial aspect of our overall health and well-being. It plays a
vital role in regulating our mental and physical health, impacting our mood,
memory, and cognitive function to our physical resilience and immune system.
The classification of sleep stages is a mandatory step to assess sleep
quality, providing the metrics to estimate the quality of sleep and how well
our body is functioning during this essential period of rest.
Photoplethysmography (PPG) has been demonstrated to be an effective signal for
sleep stage inference, meaning it can be used on its own or in a combination
with others signals to determine sleep stage. This information is valuable in
identifying potential sleep issues and developing strategies to improve sleep
quality and overall health. In this work, we present a machine learning sleep-
wake classification model based on the eXtreme Gradient Boosting (XGBoost)
algorithm and features extracted from PPG signal and activity counts. The
performance of our method was comparable to current state-of-the-art methods
with a Sensitivity of 91.15 $\pm$ 1.16%, Specificity of 53.66 $\pm$ 1.12%,
F1-score of 83.88 $\pm$ 0.56%, and Kappa of 48.0 $\pm$ 0.86%. Our method
offers a significant improvement over other approaches as it uses a reduced
number of features, making it suitable for implementation in wearable devices
that have limited computational power.

_Keywords_ PPG signal $\cdot$ Wearable devices $\cdot$ Sleep stage $\cdot$
Sleep-wake classification.

##  1 Introduction

Sleep plays a vital role in maintaining good health for individuals of all
ages Ramar et al. (2021). Inadequate sleep of poor quality or duration can
lead to a host of chronic health problems, including cardiovascular diseases,
diabetes, and obesity Hoevenaar-Blom et al. (2011) Knutson and Van Cauter
(2008) Rahe et al. (2015). Moreover, sleep disorders are on the rise and have
been linked to a decrease in Gross Domestic Product (GDP) in some countries
Hillman et al. (2006) Streatfeild et al. (2021). These highlight the
importance of ensuring that individuals get adequate and high-quality sleep to
maintain their health and well-being.

Polysomnography (PSG) is the gold standard exam for evaluating human sleep
Krystal and Edinger (2008). The exam involves the simultaneous recording of
multiple electrophysiological signals during sleep, such as Electrocardiogram
(ECG), Photoplethysmography (PPG), Electroencephalogram (EEG), Electromyogram
(EMG), etc. Rundo and Downey III (2019). After registration, a specialist
analyzes the recorded signals and labels different sleep stages based on time
windows of 30 s. Nevertheless, PSG is an expensive and time-consuming
procedure and can also suffer from labeling errors by the specialist.
Additionally, PSG exams are performed in an unfamiliar environment, with
multiple electrodes attached, which can affect sleep quality.

With the rise of wearable devices, it is now possible to track several
physiological signals, such as heart rate and blood oxygen levels, more cost-
effectively and conveniently Prieto-Avalos et al. (2022). Wearable devices are
increasingly being used to track and evaluate sleep patterns by capturing PPG
signals, raw triaxis accelerometer signal (ACC), and activity-based signals
like Activity (ACT). PPG measures changes in the blood volume of vascular
tissues, allowing measurements of vital parameters, such as heart rate,
respiration rate, arterial oxygen saturation, and blood pressure Mejía-Mejía
et al. (2022). On the other hand, the ACT is derived from the ACC signal
processing and provides an estimation of rest and wakefulness periods to
assess sleep patterns Banfi et al. (2021). Besides the advantage of collecting
different signals in the same device, the compact hardware of wearable devices
ensures minimal discomfort and does not impact the quality of sleep.

Sleep consists of two main stages: rapid eye movement (REM) and non-rapid eye
movement (NREM). According to the guidelines provided by the American Academy
of Sleep Medicine (AASM) Iber (2007); Berry et al. (2012), sleep stages can be
classified into five distinct categories: (i) wakefulness (W), (ii) non-REM
stage 1 (N1), (iii) non-REM stage 2 (N2), (iv) non-REM stage 3 (N3), and (v)
REM stage (R). Accurately identifying and categorizing sleep stages is
essential for effectively monitoring sleep patterns. However, by establishing
only an accurate and reliable classification for sleep-wake stages, it is
possible to obtain data to calculate important sleep quality metrics, such as:
total sleep time; total wake time; sleep latency, that is the amount of time
between the beginning of bedtime to actually falling asleep; sleep efficiency,
calculated as total sleep time divided by total amount of time in bed; and
wakefulness after sleep onset (WASO), measured as total time of wakefulness
after sleep onset. Shrivastava et al. (2014)

While wearable devices cannot directly measure brain activity like EEG devices
used in sleep stage analysis , they can capture PPG signals, which can be
utilized to estimate Heart Rate (HR) Mejía-Mejía et al. (2022). HR is known to
decrease during the transition from wakefulness to non-REM stages Silvani
(2008). Additionally, there are indications that HR exhibits a slight decrease
during the transition from non-REM to REM sleep stages Silvani (2008); Habib
et al. (2023). Therefore, we hypothesize that only using the information of
the HR and the Heart Rate Variability (HRV), extracted from the PPG signals,
along with the ACT, derived from the ACC signals, is possible to accurately
classify sleep and wake stages.

Recent works in the field of sleep-wake classification proposed to use both
PPG and/or ACC signals. Most of them are based on features extracted of these
signals. Table 1 displays a summary of the state-of-the-art on this topic.
Fonseca et al. (2017) used a Bayesian Classifier to predict sleep-wake stages
based on a set of HRV features computed from interbeat intervals obtained from
PPG signals along with measures from ACC. Likewise, Eyal and Baharav (2017)
used a similar approach as Fonseca et al. (2017), without the measures from
ACC. Uçar et al. (2018) proposed to use k-Nearest Neighbors (KNN) and Support
Vector Machine (SVM) algorithms based on PPG and HRV features. Palotti et al.
(2019) compared a cohort of classification algorithms to perform the sleep-
wake classification based on classic machine learning or deep learning
techniques using only ACC signals. Likewise, Banfi et al. (2021) proposed to
use only raw ACC signals using Convolutional Neural Networks (CNN) for this
binary classification. Habib et al. (2023) proposed a CNN derived from PPG raw
signals of 10 subjects with sleep-disordered breathing, using a leave-one-out
strategy on the sleep-wake classification with data augmentation. Motin et al.
(2023) used the same dataset as Habib et al. (2023), extracting 72 features
from the PPG signals, instead of using the raw PPG, using three different
classifiers, KNN, SVM and a Random Forest (RF).

Table 1: Summary of state-of-the-art obtained results for the sleep-wake
classification.

| Accuracy | Sensitivity | Specificity | F1-score | Kappa | Dataset | Method
---|---|---|---|---|---|---|---
Fonseca et al. (2017) | 91.5 $\pm$ 5.1 | 58.2 $\pm$ 17.3 | 92.9 $\pm$ 2.0 | - | 0.55 $\pm$ 0.14 | Private (s = 101) |  | Bayesian classifier -
---
PPG features and ACC
Eyal and Baharav (2017) | 84.3 | 38.1 | 91.7 | - | 0.31 | Private (s = 88) |  | Bayesian classifier -
---
PPG features
Uçar et al. (2018) | 79.23 | 78.0 | 80.0 | 79.0 | 0.58 | Private (s = 10) |  | SVM -
---
PPG features
Uçar et al. (2018) | 79.36 | 77.0 | 81.0 | 79.0 | 0.59 | Private (s = 10) |  | KNN -
---
PPG features
Palotti et al. (2019) | 81.8 $\pm$ 1.0 | 90.40 $\pm$ 1.20 | 68.10 $\pm$ 1.90 | 84.30 $\pm$ 1.10 | - | MESA (s = 1,817) |  | Extra Trees -
---
370 ACC features
Palotti et al. (2019) | 83.1 $\pm$ 1.0 | 91.40 $\pm$ 1.10 | 69.90 $\pm$ 2.00 | 85.50 $\pm$ 1.00 | - | MESA (s = 1,817) |  | LSTM 100 -
---
ACC raw signal
Motin et al. (2019) | 72.36 | 70.64 | 74.22 | - | - | Private (s = 5) |  | Medium Gaussian SVM -
---
PPG 17 features
Motin et al. (2020) | 81.10 | 81.06 | 82.50 | 81.74 | - | Private (s = 10) |  | Cubic SVM -
---
PPG 22 features
Banfi et al. (2021) | - | 89.20 | 92.0 | 90.9 | 0.782 | Private (s = 81) |  | LightCNNA -
---
ACC raw features
Habib et al. (2023) | 94.18 $\pm$ 11.95 | 94.4 | - | 93.05 $\pm$ 13.77 | 0.864 $\pm$ 0.265 | Private (s = 10) |  | CNN -
---
PPG raw signal
Motin et al. (2023) | 83.75 $\pm$ 0.85 | 87.79 $\pm$ 1.10 | 73.63 $\pm$ 2.45 | 80.01 $\pm$ 1.88 | - | Private (s = 10) |  | KNN -
---
PPG 72 features
Motin et al. (2023) | 84.66 $\pm$ 0.99 | 87.41 $\pm$ 1.24 | 77.79 $\pm$ 0.93 | 82.32 $\pm$ 0.90 | - | Private (s = 10) |  | SVM -
---
PPG 72 features
Motin et al. (2023) | 85.22 $\pm$ 0.62 | 87.86 $\pm$ 1.48 | 77.67 $\pm$ 3.26 | 82.45 $\pm$ 1.62 | - | Private (s = 10) |  | RF -
---
PPG 72 features

In this work we present a method to classify sleep and wake stages, comparing
the results of three commonly used machine learning techniques: Logistic
Regression (LR), Random Forest (RF), and the eXtreme Gradient Boosting
(XGBoost). These algorithm are based on features extracted from PPG and ACT
signals with a reduced number of features for deployment feasibility on
wearable devices. Additionally, a stratified analysis was conducted
considering age and gender factors. Our method demonstrates advancement over
existing approaches by reducing the feature set, thereby enabling
implementation on computational-constrained wearable devices.

##  2 Materials and Methods

The proposed approach comprises five sequential steps: Data selection,
Preprocessing, Windowing, Feature Extraction, and Classification. To begin, we
describe the data utilized, specifically selecting subjects with both ACT and
PPG signals. We merge the sleep stages to generate a Wake and Sleep dataset,
forming the foundation for subsequent analysis. Subsequently, we present the
preprocessing and windowing procedures, detailing how the data is prepared for
further processing. Additionally, we outline the feature extraction process.
Furthermore, we describe our employed classifiers, providing information on
our experimental setup. A flow diagram summarizing these steps is shown in
Fig. 1.

Figure 1: Overview of the proposed method for sleep-wake classification.

###  2.1 Data Selection

The experiments were carried out on the MESA Sleep dataset, a subset of the
Multi-Ethnic Study of Atherosclerosis (MESA) dataset Zhang et al. (2018) Chen
et al. (2015). The MESA Sleep dataset contains data collected from 2,237
subjects, including overnight Polysomnography exams along with their
corresponding PPG signals, 7-day wrist-worn Actigraphy signals, sleep stage
labels for every 30 s windows, and sleep questionnaires. For this study, only
patients with both ACT and PPG signals were used, resulting in 1,831 patients.
The metadata information about the employed dataset is shown in Table 2.

Table 2: Metadata of the MESA dataset. Parameter | Value
---|---
Subjects | $2,237$
with PSG | $2,056$
with actigraphy | $2,158$
with PSG and actigraphy | $1,831$
Age | $69.6\pm 9.1$
Male subjects | $1,039$

This dataset provides sleep stage labels for every 30 s window. The labeling
process followed the AASM guidelines, which suggest five sleep stage classes,
as aforementioned: (i) wake (W), (ii) non-REM-1 (N1), (iii) non-REM-2 (N2),
(iv) non-REM-3 (N3), and (v) REM (R) Iber (2007). Since our goal is to detect
only sleep or wake stages, classes (ii), (iii), (iv), and (v) were grouped
into class “sleep” (S). For every 30 s window, the PPG signal was sampled at
256 Hz, and the corresponding ACT registered.

###  2.2 Preprocessing

We used the same preprocessing steps proposed by Kotzen et al. (2022)in this
study. The PPG data was filtered using a low-pass 8th-order Chebyshev Type II
filter at 8 Hz, followed by downsampling from 256 Hz to 34 Hz using linear
interpolation. Outlier values greater than or less than three standard
deviations from the mean were clipped, and the data was normalized using
z-score normalization.

###  2.3 Windowing

We partitioned the PPG signals into non-overlapping 30-second windows, as the
PSG exam provides a sleep stage label for every 30 seconds of recording time.
Due to the inherent variable recording lengths for each subject in the MESA
Sleep dataset, the number of windows differed among subjects. Figure 2
illustrates the windowing extraction step applied to the PPG signal of a
single subject.

Figure 2: Windowing and Feature Extraction process example.

###  2.4 Feature Extraction

Heart Rate (HR) and Heart Rate Variability (HRV) are widely recognized as
important features associated with sleep stages and disorders, being both
regulated by the sympathetic and parasympathetic nervous systems Stein and Pu
(2012). HR was calculated for every window by taking the mean time difference
between peaks, which were detected using the method proposed in Bishop and
Ercole (2018). HRV was calculated as the standard deviation of the time
difference between the peaks. Windows with HR greater than 180 beats per
minute or HRV outside of two standard deviations from the mean HRV of the
entire dataset were discharged. As consequence, all samples from four
participants were excluded in this phase, resulting in $s=1,827$ subjects. Our
model also used as input the activity value provided by the MESA-sleep
acquired with the Actiware-Sleep version 5.59 analysis software (Mini-Mitter
Co, Inc, Bend, OR).

Therefore, for each $i$ subject ($i=1,2,\dots,s$), the features collected in
each window were arranged as

$\displaystyle\mathbf{X}_{i}$ $\displaystyle=$
$\displaystyle\begin{bmatrix}x_{11}&x_{12}&x_{13}\\\ x_{21}&x_{22}&x_{23}\\\
\vdots&\vdots&\vdots\\\ x_{n1}&x_{n2}&x_{n3}\end{bmatrix},$ (1)

where $s$ is the number of subjects, $m$ is the number of features ($m=3$),
and $n$ is the number of windows for each subject. Overall, we have a total of
2,050,280 windows: 63.93% (1,310,690) are labeled as sleep and the remaining
36.07% (739,590) as wake.

###  2.5 Classification

Our sleep-wake classification was performed using three commonly used machine
learning techniques: Logistic Regression (LR) Stoltzfus (2011), Random Forest
(RF) Breiman (2001) and eXtreme Gradient Boosting (XGBoost) Chen and Guestrin
(2016). A 10-fold cross-validation approach was employed to prevent bias in
the training and testing split, with samples from the same patient grouped in
the same fold to avoid intra-patient bias. The results were compared with the
sleep-wake classification provided by the actigraphy used in the MESA Sleep
study and with a similar study using the same dataset Palotti et al. (2019).
Additionally, the results were compared with other similar studies.

The results with the proposed approach were compared to the literature by
using four different evaluation metrics commonly used in sleep-wake
classification studies: Accuracy (Ac), Sensitivity (Se), Specificity (Sp),
F1-score, and Cohen’s Kappa Coefficient (Kappa). These metrics were used to
quantify the accuracy and agreement of the sleep-wake classification results.

Experiments were performed using a Foxconn HPC M100-NHI with an 8-GPU cluster
of NVIDIA Tesla V100 16GB cards. The model was implemented in Python (3.8.10)
with the support of the libraries scikit-learn (1.1.3), XGBoost (1.6.1), and
scipy (1.8.1).

##  3 Results

Table 3 summarizes the overall performance of the proposed methods for the
sleep-wake classification task. Our algorithm that best performed (XGBoost)
achieved Ac, Se, Spe, F1-score, and Kappa of $77.62\pm 0.56\%$, $91.14\pm
1.15\%$, $53.66\pm 1.11\%$, $83.88\pm 0.56$ and $0.480\pm 0.008$,
respectively. Additionally, we used the same metrics to evaluate the
classification provided by the actigraphy used in the MESA Sleep dataset as
baseline. It obtained Se, Spe, F1-score, and Kappa of 50.49%, 94.75%, 64.08%,
and 0.478, respectively.

Table 3: Obtained results for the sleep-wake classification. | Accuracy | Sensitivity | Specificity | F1-score | Kappa
---|---|---|---|---|---
XGBoost | 77.62 $\pm$ 0.56 | 91.14 $\pm$ 1.15 | 53.66 $\pm$ 1.11 | 83.88 $\pm$ 0.56 | 0.480 $\pm$ 0.008
Logistic Regression | 74.62 $\pm$ 0.66 | 96.46 $\pm$ 0.33 | 35.91 $\pm$ 0.88 | 82.92 $\pm$ 0.55 | 0.370 $\pm$ 0.009
Random Forest | 73.80 $\pm$ 0.48 | 83.68 $\pm$ 0.93 | 56.30 $\pm$ 0.76 | 80.32 $\pm$ 0.54 | 0.413 $\pm$ 0.007
MESA Actigraphy | - | 50.49 | 94.75 | 64.08 | 0.478

To assess possible biases in our method, we also performed a stratified
analysis of our obtained results by age and gender. In Fig. 3, we show the
obtained F1-score of our approach in four age groups (54 – 65, 66 – 75, 76 –
85, and 86+) and two gender groups (male and female).

Figure 3: Stratified F1-score results for sleep-wake classification task for
age and gender.

##  4 Discussion

We developed a methodology with results comparable to the state-of-the-art
while maintaining low complexity. Our approach used only three features: two
obtained from the peak-to-peak interval from the photoplethysmography signal
and one from the actigraph value. The peak-to-peak derived features were
extracted even in adverse acquisition scenarios and the actigraph value was
easily extracted from the accelerometer. This is a major advantage compared to
other reports that extract a complex set of features that can be compromised
in noisy acquisitions. As shown in Table 3, apart from the Spe, our proposed
method achieved higher metric values than the MESA Sleep actigraphy. This
shows the gain of using PPG-derived physiological measures (HR and HRV).

Compared to the best method from Palotti et al. (2019), which also used the
MESA Sleep dataset, we obtained comparable Se ($91.14\pm 1.15\%$ vs $91.40\pm
1.10\%$) and F1-score ($83.88\pm 0.56\%$ vs $85.50\pm 1.00\%$); however, the
Spe value is lower ($53.66\pm 1.11\%$ vs $69.90\pm 2.00\%$). The lower Spe
might be due to the imbalance between the classes. Adopting a proper strategy
for dealing with unbalanced data might improve the observed results and need
further investigation.

It is not possible to make a direct comparison with other works, since all of
them are based on private datasets. Motin et al. (2023) achieved comparable
results as ours in terms of F1-score (see Table 1), however, they didn’t used
ACC signals and used a much larger set of PPG features than us. Moreover,
their work are based only on 10 distinct subjects. Even though their results
are promising, this reduced number of subjects may lead to limited
generalizability, along with biased learning since there is a high
interdependence among intra-subject heartbeats de Chazal et al. (2004); Costa
et al. (2023). Likewise, Habib et al. (2023) used a limited number of
subjects, and employed a CNN on PPG raw signal, achieving the best F1-score to
date, to the best of our knowledge. On the other hand, Banfi et al. (2021)
also achieved F1-score higher than 90% by using only raw ACC signals, but they
also have a limited number of subjects. However, these deep learning
approaches usually requires lots of data for training, computational
resources, are more complex than traditional machine learning models and often
have many hyperparameters.

In the stratified analysis shown in Fig. 3, the F1-score progressively
decreases with age. Also, there was consistently better mean F1-score for
females than for males, despite an increase in the standard deviation for both
genders. This indicates that the reduction in the number of older patients
affects both genders similarly. However, the cause of the higher mean F1-score
for females cannot be determined based on the available data, as it could be
due to a larger sample size or physiological differences between genders.
Further research may be necessary to determine the underlying reasons for
these results.

Furthermore, it was aforementioned that deep learning models could be
cumbersome for deployment in wearable devices. However, the computational
resource needed depends on a set of parameters that should be investigated.
Thus, future works should address this by comparing the performance and
computational resources of different models. Likewise, future works should
also be done on external datasets to validate the generalizability of our
proposed method.

Finally, the validation of wearable sleep monitoring devices would reduce the
cost of sleep disorder compared to traditional polysomnography exams. This
requires reliable and practical sleep stages prediction models easy to
implement and capable of being integrated into commercial devices in line with
the reported findings.

##  5 Conclusion

We provided evidence for a simple method to classify sleep-wake states using
only three features: HR, HRV, and ACT. Our method showed comparable results
with more complex methods and superior results than using only the ACT
measured by an actigraph, which is currently the wearable device of choice for
sleep monitoring.

## References

  * Ramar et al. [2021] Kannan Ramar, Raman K Malhotra, Kelly A Carden, Jennifer L Martin, Fariha Abbasi-Feinberg, R Nisha Aurora, Vishesh K Kapur, Eric J Olson, Carol L Rosen, James A Rowley, et al.  Sleep is essential to health: an american academy of sleep medicine position statement.  _Journal of Clinical Sleep Medicine_ , 17(10):2115–2119, 2021.
  * Hoevenaar-Blom et al. [2011] Marieke P Hoevenaar-Blom, Annemieke MW Spijkerman, Daan Kromhout, Julia F van den Berg, and WM3198203 Verschuren.  Sleep duration and sleep quality in relation to 12-year cardiovascular disease incidence: the morgen study.  _Sleep_ , 34(11):1487–1492, 2011.
  * Knutson and Van Cauter [2008] Kristen L Knutson and Eve Van Cauter.  Associations between sleep loss and increased risk of obesity and diabetes.  _Annals of the New York Academy of Sciences_ , 1129(1):287–304, 2008.
  * Rahe et al. [2015] Corinna Rahe, Maria Eszter Czira, Henning Teismann, and Klaus Berger.  Associations between poor sleep quality and different measures of obesity.  _Sleep medicine_ , 16(10):1225–1228, 2015.
  * Hillman et al. [2006] David R Hillman, Anita Scott Murphy, Ral Antic, and Lynne Pezzullo.  The economic cost of sleep disorders.  _Sleep_ , 29(3):299–305, 2006.
  * Streatfeild et al. [2021] Jared Streatfeild, Jackson Smith, Darren Mansfield, Lynne Pezzullo, and David Hillman.  The social and economic cost of sleep disorders.  _Sleep_ , 44(11):zsab132, 2021.
  * Krystal and Edinger [2008] Andrew D. Krystal and Jack D. Edinger.  Measuring sleep quality.  _Sleep Medicine_ , 9:S10–S17, 2008.  ISSN 1389-9457.  doi:10.1016/S1389-9457(08)70011-X.  The Art of Good Sleep Proceedings from the 5th International Sleep Disorders Forum: Novel Outcome Measures of Sleep, Sleep Loss and Insomnia.
  * Rundo and Downey III [2019] Jessica Vensel Rundo and Ralph Downey III.  Polysomnography.  _Handbook of clinical neurology_ , 160:381–392, 2019.
  * Prieto-Avalos et al. [2022] Guillermo Prieto-Avalos, Nancy Aracely Cruz-Ramos, Giner Alor-Hernández, José Luis Sánchez-Cervantes, Lisbeth Rodríguez-Mazahua, and Luis Rolando Guarneros-Nolasco.  Wearable devices for physical monitoring of heart: A review.  _Biosensors_ , 12(5):292, 2022.
  * Mejía-Mejía et al. [2022] Elisa Mejía-Mejía, John Allen, Karthik Budidha, Chadi El-Hajj, Panicos A. Kyriacou, and Peter H. Charlton.  4 - photoplethysmography signal processing and synthesis.  In John Allen and Panicos Kyriacou, editors, _Photoplethysmography_ , pages 69–146. Academic Press, 2022.  ISBN 978-0-12-823374-0.  doi:10.1016/B978-0-12-823374-0.00015-3.
  * Banfi et al. [2021] Tommaso Banfi, Nicolò Valigi, Marco di Galante, Paola d’Ascanio, Gastone Ciuti, and Ugo Faraguna.  Efficient embedded sleep wake classification for open-source actigraphy.  _Scientific reports_ , 11(1):1–12, 2021.
  * Iber [2007] Conrad Iber.  The aasm manual for the scoring of sleep and associated events: rules, terminology, and technical specification.  _(No Title)_ , 2007.
  * Berry et al. [2012] RB Berry, R Budhiraja, DJ Gottlieb, D Gozal, C Iber, VK Kapur, CL Marcus, R Mehra, S Parthasarathy, SF Quan, S Redline, KP Strohl, Ward SL Davidson, and MM Tangredi.  Rules for scoring respiratory events in sleep: update of the 2007 aasm manual for the scoring of sleep and associated events. deliberations of the sleep apnea definitions task force of the american academy of sleep medicine.  _J Clin Sleep Med_ , 8(5):597–619, 2012.  doi:10.5664/jcsm.2172.
  * Shrivastava et al. [2014] Deepak Shrivastava, Syung Jung, Mohsen Saadat, Roopa Sirohi, and Keri Crewson.  How to interpret the results of a sleep study.  _Journal of community hospital internal medicine perspectives_ , 4(5):24983, 2014.
  * Silvani [2008] Alessandro Silvani.  Physiological sleep-dependent changes in arterial blood pressure: Central autonomic commands and baroreflex control.  _Clinical and Experimental Pharmacology and Physiology_ , 35(9):987–994, 2008.  doi:10.1111/j.1440-1681.2008.04985.x.
  * Habib et al. [2023] Ahsan Habib, Mohammod Abdul Motin, Thomas Penzel, Marimuthu Palaniswami, John Yearwood, and Chandan Karmakar.  Performance of a convolutional neural network derived from ppg signal in classifying sleep stages.  _IEEE Transactions on Biomedical Engineering_ , 70(6):1717–1728, 2023.  doi:10.1109/TBME.2022.3219863.
  * Fonseca et al. [2017] Pedro Fonseca, Tim Weysen, Maaike S. Goelema, Els I.S. Møst, Mustafa Radha, Charlotte Lunsingh Scheurleer, Leonie van den Heuvel, and Ronald M. Aarts.  Validation of Photoplethysmography-Based Sleep Staging Compared With Polysomnography in Healthy Middle-Aged Adults.  _Sleep_ , 40(7):zsx097, 06 2017.  doi:10.1093/sleep/zsx097.
  * Eyal and Baharav [2017] Shuli Eyal and Anda Baharav.  Sleep insights from the finger tip: How photoplethysmography can help quantify sleep.  In _2017 Computing in Cardiology (CinC)_ , pages 1–4, 2017.  doi:10.22489/CinC.2017.274-197.
  * Uçar et al. [2018] Muhammed Kursad Uçar, Mehmet Recep Bozkurt, Cahit Bilgin, and Kemal Polat.  Automatic sleep staging in obstructive sleep apnea patients using photoplethysmography, heart rate variability signal and machine learning techniques.  _Neural Computing and Applications_ , 29:1–16, 2018.  doi:10.1007/s00521-016-2365-x.
  * Palotti et al. [2019] Joao Palotti, Raghvendra Mall, Michael Aupetit, Michael Rueschman, Meghna Singh, Aarti Sathyanarayana, Shahrad Taheri, and Luis Fernandez-Luque.  Benchmark on a large cohort for sleep-wake classification with machine learning techniques.  _NPJ digital medicine_ , 2(1):1–9, 2019.
  * Motin et al. [2023] Mohammod Abdul Motin, Chandan Karmakar, Marimuthu Palaniswami, Thomas Penzel, and Dinesh Kumar.  Multi-stage sleep classification using photoplethysmographic sensor.  _Royal Society Open Science_ , 10(4):221517, 2023\.  doi:10.1098/rsos.221517.
  * Motin et al. [2019] Mohammod Abdul Motin, Chandan Kumar Karmakar, Thomas Penzel, and Marimuthu Palaniswami.  Sleep-wake classification using statistical features extracted from photoplethysmographic signals.  In _2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)_ , pages 5564–5567. IEEE, 2019\.
  * Motin et al. [2020] Mohammod Abdul Motin, Chandan Karmakar, Marimuthu Palaniswami, and Thomas Penzel.  Photoplethysmographic-based automated sleep–wake classification using a support vector machine.  _Physiol. Meas._ , 41:075013, 2020.  doi:10.1088/1361-6579/ab9482.
  * Zhang et al. [2018] Guo-Qiang Zhang, Licong Cui, Remo Mueller, Shiqiang Tao, Matthew Kim, Michael Rueschman, Sara Mariani, Daniel Mobley, and Susan Redline.  The national sleep research resource: towards a sleep data commons.  _Journal of the American Medical Informatics Association_ , 25(10):1351–1358, 2018.
  * Chen et al. [2015] Xiaoli Chen, Rui Wang, Phyllis Zee, Pamela L Lutsey, Sogol Javaheri, Carmela Alcántara, Chandra L Jackson, Michelle A Williams, and Susan Redline.  Racial/ethnic differences in sleep disturbances: the multi-ethnic study of atherosclerosis (mesa).  _Sleep_ , 38(6):877–888, 2015.
  * Kotzen et al. [2022] Kevin Kotzen, Peter H Charlton, Sharon Salabi, Lea Amar, Amir Landesberg, and Joachim A Behar.  Sleepppg-net: a deep learning algorithm for robust sleep staging from continuous photoplethysmography.  _IEEE Journal of Biomedical and Health Informatics_ , 2022.
  * Stein and Pu [2012] Phyllis K. Stein and Yachuan Pu.  Heart rate variability, sleep and sleep disorders.  _Sleep Medicine Reviews_ , 16(1):47–66, 2012\.  ISSN 1087-0792.  doi:10.1016/j.smrv.2011.02.005.
  * Bishop and Ercole [2018] Steven M Bishop and Ari Ercole.  Multi-scale peak and trough detection optimised for periodic and quasi-periodic neuroscience data.  In _Intracranial Pressure & Neuromonitoring XVI_, pages 189–195. Springer, 2018.
  * Stoltzfus [2011] Jill C. Stoltzfus.  Logistic regression: A brief primer.  _Academic Emergency Medicine_ , 18(10):1099–1104, 2011.  doi:10.1111/j.1553-2712.2011.01185.x.
  * Breiman [2001] Leo Breiman.  Random forests.  _Machine Learning_ , 45:5–32, 2001.  doi:10.1023/A:1010933404324.
  * Chen and Guestrin [2016] Tianqi Chen and Carlos Guestrin.  XGBoost.  In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_. ACM, aug 2016.  doi:10.1145/2939672.2939785.
  * de Chazal et al. [2004] Philip de Chazal, M. O’Dwyer, and R.B. Reilly.  Automatic classification of heartbeats using ecg morphology and heartbeat interval features.  _IEEE Transactions on Biomedical Engineering_ , 51(7):1196–1206, 2004.  doi:10.1109/TBME.2004.827359.
  * Costa et al. [2023] Thiago Bulhões Da Silva Costa, Felipe Meneguitti Dias, Diego Armando Cardona Cardenas, Marcelo Arruda Fiuza De Toledo, Daniel Mário De Lima, Jose Eduardo Krieger, and Marco Antonio Gutierrez.  Blood pressure estimation from photoplethysmography by considering intra- and inter-subject variabilities: Guidelines for a fair assessment.  _IEEE Access_ , 11:57934–57950, 2023.  doi:10.1109/ACCESS.2023.3284458.