# Remote Blood Oxygen Estimation From Videos Using Neural Networks

Joshua Mathew  Student Member, IEEE, Xin Tian  Student Member, IEEE, Chau-Wai
Wong  Member, IEEE, Simon Ho, Donald K. Milton  and Min Wu  Fellow, IEEE J.
Mathew, and C.-W. Wong are with the Department of Electrical and Computer
Engineering and the Forensic Sciences Cluster, NC State University, Raleigh,
NC, 27695 USA (email: {jrmathew, chauwai.wong}@ncsu.edu).X. Tian, and M. Wu
are with the Department of Electrical and Computer Engineering, University of
Maryland, College Park, MD, 20742 USA (e-mail: {xtian17, minwu}@umd.edu). X.
Tian and J. Mathew equally contributed to this paper.S. Ho is with the
Physical Therapy and Rehabilitation Science, University of Maryland School of
Medicine, Baltimore, MD, 21201 USA (email: simon.ho@som.umaryland.edu).D. K.
Milton is with the Maryland Institute for Applied Environmental Health, School
of Public Health, University of Maryland, College Park, MD 20742, USA (e-mail:
dmilton@umd.edu).

###### Abstract

Peripheral blood oxygen saturation (SpO2) is an essential indicator of
respiratory functionality and has received increasing attention during the
COVID-19 pandemic. Clinical findings show that COVID-19 patients can have
significantly low SpO2 before any obvious symptoms. The prevalence of cameras
has motivated researchers to investigate methods for monitoring SpO2 using
videos. Most prior schemes involving smartphones are contact-based: They
require using a fingertip to cover the phone’s camera and the nearby light
source to capture re-emitted light from the illuminated tissue. In this paper,
we propose the first convolutional neural network based noncontact SpO2
estimation scheme using smartphone cameras. The scheme analyzes the videos of
a participant’s hand for physiological sensing, which is convenient and
comfortable for users and can protect their privacy and allow for keeping face
masks on. We design the neural network architectures inspired by the
optophysiological models for SpO2 measurement and demonstrate the
explainability by visualizing the weights for channel combination. Our
proposed models outperform the state-of-the-art model that is designed for
contact-based SpO2 measurement, showing the potential of the proposed method
to contribute to public health. We also analyze the impact of skin type and
the side of a hand on SpO2 estimation performance.

SpO2 monitoring, remote mobile sensing, convolutional neural networks, machine
learning.

##  1 Introduction

Peripheral blood oxygen saturation (SpO2) is an important physiological
parameter that represents the level of oxygenation in the blood and reflects
the adequacy of respiratory function [1]. The estimation and monitoring of
SpO2 are essential for the assessment of lung function and the treatment of
chronic pulmonary diseases. It has been reported that COVID-19 patients can
present with clinically significant hypoxemia or drop in blood oxygen
saturation, yet many do not exhibit respiratory symptoms [2, 3]. This
highlights the importance of early detection of changes in SpO2 to facilitate
timely management of asymptomatic patients with clinical deterioration.
Conventional SpO2 measurement methods rely on contact-based sensing, including
fingertip pulse oximetry and its variants in smartwatches and smartphones [4,
5, 6, 7]. These contact-based methods present the risk of cross contamination
between individuals using the same measurement device. An additional issue
with contact-based methods is limb perfusion, especially in the digits.
Circulation to the fingers and toes is often impaired in many cardiovascular
and pulmonary diseases, which complicates measurement of SpO2. Also, pulse
oximeters may not be widely available in marginalized communities and some
undeveloped countries [8].

In recent years, a growing number of studies have investigated SpO2
measurement using videos [9, 10, 11, 12, 13, 14, 15, 16, 17], which allows for
SpO2 estimation without contact. These video-based noncontact methods provide
a more comfortable and unobtrusive way to monitor SpO2 and have the potential
to be adopted in health screening and telehealth. In this paper, we introduce
explainable neural network models for feature extraction and SpO2 measurement
from contactless videos captured using consumer-grade smartphone cameras. To
the best of our knowledge, there is no prior work that remotely monitors SpO2
with regular RGB cameras using neural networks.

Figure 1:  Proposed SpO2 estimation method. Three color time series are
extracted from the skin area of a hand video and are then fed into an
optophysiology-inspired neural network for SpO2 prediction.

We propose using convolutional neural networks (CNN) for contactless SpO2
monitoring from videos captured by smartphone cameras. Fig. 1 is an overview
of the system design. First, the region of interest (ROI), including the palm
and the back of the hand, is extracted from the smartphone captured videos.
Second, the ROI is spatially averaged to produce R, G, and B time series.
Next, the three time series are fed into an optophysiology-inspired CNN for
SpO2 estimation. We consider the hand region in this work as a proof-of-
concept. Compared to using the face for SpO2 measurement as most of the prior
art did [14, 16], recording hand videos raises less privacy concern and is a
safer way for health condition screening and data collection during the
COVID-19 pandemic according to the mask-wearing guidelines. The contributions
of our work are summarized as follows:

  * •

To the best of our knowledge, this is the first work to use neural networks to
address the challenging problem of contactless SpO2 sensing using consumer-
grade RGB cameras.

  * •

Through a data-driven approach and visualization of the weights for the RGB
channel combinations, we demonstrate the explainability of our model and that
the choice of the color band learned by the neural network is consistent with
the suggested color bands used in the optophysiological methods.

  * •

We analyze the impact of the two sides of the hand and different skin tones on
the quality of SpO2 estimation.

  * •

We achieve more accurate SpO2 prediction with our optophysiologically inspired
neural network structures when compared to the state-of-the-art neural network
structure designed for this problem.

##  2 Background and Related Work

###  2.1 Blood Oxygen Saturation and the Ratio-of-Ratios Principle

Figure 2: Extinction coefficient curves of hemoglobin showing the absorption
properties at different wavelengths. The curves were plotted based on [7, 18].
The difference between oxygenated hemoglobin (HbO2) and deoxygenated
hemoglobin (Hb) at the red and blue wavelengths means that these color
channels contain useful information for SpO2 prediction by means of
optophysiological principles.

The protein molecule hemoglobin (Hb) in the blood carries oxygen from the
lungs to the tissues of the body. The level of blood oxygen saturation (SpO2)
represents the ratio of oxygenated hemoglobin (HbO2) to total hemoglobin and
indicates the adequacy of respiratory function [1]. The normal range of SpO2
is 95% to 100% [1]. SpO2 is an important indicator of the ability of the
respiratory system to meet metabolic demands. An abnormal drop in SpO2 can
serve as an early warning sign of inadequate oxygenation and clinical
deterioration [1]. A convenient and noninvasive way to continuously measure
SpO2 is pulse oximetry [4]. Pulse oximeters utilize the principle of the ratio
of ratios that was first proposed by Aoyagi in the early 1970s [4], and pulse
oximeters are commonly used today in hospitals, clinics, and homes. The ratio-
of-ratios method leverages the optical absorbance difference of Hb and HbO2 at
two wavelengths, which conventionally are at red and infrared wavelengths as
indicated in Fig. 2. For the commonly seen pulse oximeters, lights at the red
and infrared wavelengths are emitted through the fingertip. The transmitted
light, interacted and attenuated by the blood and tissue, and received by an
optical sensor, conveys information about pulsatile blood volume. The
pulsatile blood volume at the two wavelengths is further processed to produce
an SpO2 estimation.

###  2.2 Video Based SpO2 Measurement

With the prevalence of smartphones, researchers have investigated methods of
monitoring SpO2 using smartphones, most of which are contact-based and require
the fingertips to be pushed against the illuminated light source and the
built-in camera [5, 6, 7, 19, 20], so that the diffusely reflected light by
the fingertip is captured by the camera. In this setup, an adapted ratio-of-
ratios model is utilized with the red and blue (or green) channels of color
videos in lieu of the traditional narrowband red and infrared wavelengths.
Those contact-based measurements can cause a sense of burning after several
minutes of contact with the illuminated flashlight and is not suitable for
sensitive skin or prolonged use.

Researchers have also investigated contactless SpO2 measurement from videos.
Based on the setup of cameras and light sources, existing noncontact, video-
based SpO2 estimation methods can be grouped into two main categories. Methods
from the first category utilize monochromatic sensing similar to conventional
pulse oximetry. They use either high-end monochromatic cameras with selected
optical filters or controlled monochromatic light sources [9, 10, 11, 12]. The
other category uses consumer-grade RGB cameras, such as digital webcams and
smartphone cameras [16, 14, 15, 17]. All these video-based noncontact SpO2
estimation methods utilize the differences in the optophysiological
characteristics of oxygenated hemoglobin and deoxygenated hemoglobin. The
monochromatic light sources and sensors are selected to have accurate control
of the absorption effect of hemoglobin, while the more ubiquitous consumer-
grade cameras, including the low-cost webcams and smartphone cameras, have a
wider wavelength band and are more challenging for SpO2 sensing.

###  2.3 Deep Learning Aided Camera-based Physiological Monitoring

Deep learning has demonstrated promising performance in camera-based
physiological measurements, such as heart rate, breathing rate, and body
temperature [21, 22, 23, 24]. An end-to-end convolutional attention network
was proposed in [22] to estimate the blood volume pulse from face videos.
Frequency analysis is then conducted on the estimated pulse signal for heart
rate and breathing rate tracking. The study in [21] demonstrates that the
heart rate can be directly inferred using a convolutional network with
spatial-temporal representations of the face videos as its input. Mobile
applications have been developed to utilize CNNs to measure body temperature
from facial images [24].

Deep learning for SpO2 monitoring from videos is still in the early stage.
Ding et al. [7] recently proposed a convolutional neural network architecture
for contact-based SpO2 monitoring with smartphone cameras. Even though the
work in [7] showed better performance than the conventional ratio-of-ratios
method, their technique requires the users’ fingertips to be in contact with
the illuminated flashlight and camera, which not only may lead to a sense of
burning for a continuous period of time but also raises sanitation concerns,
especially if the sensing device is shared by multiple participants during
pandemics. Inspired by the optophysiological model for SpO2 measurement [25,
5, 10], we develop a deep learning architecture to monitor SpO2 in a
contactless way with regular RGB cameras, which has the potential to be
adopted in health screening and telehealth.

##  3 Proposed Method for SpO2 From Videos

We aim to estimate SpO2 levels using a hand video based on the fact that the
color of the skin changes subtly when red cells in the blood flow
carry/release oxygen. In our proposed method, we extract three color time-
series from the skin area of the hand video. We feed the extracted time series
to optophysiology-inspired neural networks designed to achieve more
explainable and accurate SpO2 predictions.

###  3.1 Extraction of Skin Color Signals

The physiological information related to SpO2 is embedded in the color of the
reflected/reemitted light from a person’s skin. Hence, a preprocessing step
that precisely extracts the color information from the skin area is crucial to
the design of an effective SpO2 estimation method. For each participant’s
video, we aim to extract the R, G, and B time series and refer to these 1-D
time series as _skin color signals_. We first need to locate the ROI of the
skin pixels from the video. We have found that it is most effective to
discriminate the skin pixels from the background along the Cr axis of the
YCbCr color space [26].

We use Otsu’s method [27] to determine a threshold that best separates the
skin pixels from the background by minimizing the variance within the skin and
non-skin classes. Once the ROI corresponding to the hand is located, the R, G,
and B time series are generated by spatially averaging over the values of skin
pixels for each frame of the video.

The skin color signals are split up into 10-second segments using a sliding
window with a step size/stride of 0.2 seconds to serve as the inputs for
neural networks. From an optophysiological perspective, the
reflected/reemitted light from the skin for the duration of one cycle of
heartbeat, i.e., 0.5–1 seconds for a heart rate of 60–120 bpm, should contain
almost the complete information necessary to estimate the instantaneous SpO2
[4]. In our system design, we use longer segments to add resilience against
sensing noise. Since the segment length is one order of magnitude longer than
the minimally required length to contain the SpO2 information, we can use a
fully-connected or convolutional structure to adequately capture the temporal
dependencies without resorting to a recurrent neural network structure.

###  3.2 Neural Network Architectures

Figure 3: Proposed network structures for predicting an SpO2 level from a
fixed-length segment of skin color signals. We highlight the differences among
the three model configurations instead of showing the exact model structures.
Model 1 combines the RGB channels before temporal feature extraction. Model 2
extracts the temporal features from each channel separately and fuses them
toward the end. Model 3 interleaves color channel mixing and temporal feature
extraction.

The previous neural network work for SpO2 prediction mainly explored
prediction, but not the model explainability [7].
Explainability/interpretability is highly desirable in many applications yet
often not sufficiently addressed, partly due to the black box nature of neural
networks. From a healthcare standpoint, explainability is a key factor that
should be taken into account at the beginning of the design of a system. To
extract features from the skin color signals and estimate SpO2, we propose
three physiologically motivated neural network structures. These structures
are inspired by domain knowledge-driven physiological sensing methods and
designed to be physically explainable. For heart rate sensing [28, 21] and
respiratory rate sensing [29, 30], the RGB skin color signals are often
combined first followed by temporal feature extraction, as is done in the
plane-orthogonal-to-skin (POS) algorithm [31]. In contrast, for conventional
SpO2 sensing methods such as the ratio-of-ratios [25], the temporal features
are extracted first and the color components are combined at the end. Our
proposed neural network structures explore different arrangements of channel
combination and temporal feature extraction. We want to systematically compare
the performance of our explainable model structures.

Color Channel Mixing Followed by Feature Extraction. In Model 1, shown as the
leftmost structure depicted in Fig. 3, we combine the color channels first
using several channel combination layers and then extract temporal features
using temporal convolution and max pooling. A channel combination layer first
linearly combines the $C_{\text{in}}$ input channels/vectors into
$C_{\text{out}}$ activation vectors and then applies a rectified linear unit
(ReLU) activation function to obtain the output channels/vectors.
Mathematically, the channel combination layer is described as follows:

$\mathbf{V}=\sigma(\mathbf{W}\mathbf{U}+\mathbf{b\mathbbm{1}}^{T}),$ (1)

where $\mathbf{U}\in\mathbb{R}^{C_{\text{in}}\times L}$ is the input comprised
of $C_{\text{in}}$ time series/vectors of length $L$. The initial channel
combination layer has an input of three channels with 300 points along the
time axis. $\mathbf{W}\in\mathbb{R}^{C_{\text{out}}\times C_{\text{in}}}$ is a
weight matrix, where each of the $C_{\text{out}}$ rows of the matrix is a
different linear combination for the input channels. A bias vector
$\mathbf{b}\in\mathbb{R}^{C_{\text{out}}}$ contains the bias terms for each of
the $C_{\text{out}}$ output channels, which ensures that each data point in
the created segment of length $L$ has the same intercept.
$\mathbbm{1}^{T}\in\mathbb{R}^{1\times L}$ is a row vector of all ones. The
nonlinear ReLU function $\mathbb{\sigma}(x)=\max(0,x)$ is applied elementwise
to the activation map/matrix. The output of the channel combination layer
$\mathbf{V}\in\mathbb{R}^{C_{\text{out}}\times L}$ contains $C_{\text{out}}$
channels of nonlinearly combined input channels.

The channel mixing section concatenates multiple channel combination layers
with decreasing channel counts to provide significant nonlinearity. The output
of the last channel combination layer has seven channels. After the channel
mixing, for temporal feature extraction, we utilize multiple convolutional and
max pooling layers with a downsampling factor of two to extract the temporal
features of the channel-mixed signals. When there are multiple filters in the
convolutional layer, there will also be some additional channel combining with
each filter outputting a channel-mixed signal. Finally, a single node is used
to represent the predicted SpO2 level. This model has three channel
combination layers, three feature extraction layers, and a total of 34K
trainable parameters.

Feature Extraction Followed by Color Channel Mixing. In Model 2, which is the
middle structure depicted in Fig. 3, we reverse the order of color channel
mixing and temporal feature extraction from that in Model 1. The three color
channels are separately fed for temporal feature extraction. The convolutional
layers learn different features unique to each channel. At the output of the
temporal feature extraction section, each color channel has been downsampled
to retain only the important temporal information. The color channels are then
mixed together in the same way as described for Model 1 before outputting the
SpO2 value. This model has three channel combination layers, 2 feature
extraction layers, and a total of 12K parameters.

Interleaving Feature Extraction and Channel Mixing. In our third model, we
explore the possibility of interleaving the color channel mixing and temporal
feature extraction steps. As illustrated by the rightmost structure depicted
in Fig. 3, the input is first put through a convolutional layer with many
filters and then passed to max pooling layers, resulting in feature extraction
along the time as well channel combinations through each filter. The number of
filters is reduced with each successive convolutional layer, gradually
decreasing the number of combined channels and downsampling the signal in the
time domain. This model has 4 layers and a total of 307K parameters.

Loss Function and Parameter Tuning. We use the root-mean-squared-error (RMSE)
as the loss function for all models. During training, we save the model
instance at the epoch that has the lowest validation loss. The neural network
inputs are scaled to have zero mean and unit variance to improve the numerical
stability of the learning. The parameters and hyperparameters of each model
structure were tuned using the HyperBand algorithm [32], which allows for
faster and more efficient search over a large parameter space than grid search
or random search. It does this by running random parameter configurations on a
specific schedule of iterations per configuration, and uses earlier results to
select candidates for longer runs. The parameters that are tuned include the
learning rate, the number of filters and kernel size for convolutional layers,
the number of nodes, the dropout probability, and whether to do batch
normalization after each convolutional layer.

##  4 Experimental Results

###  4.1 Dataset and Capturing Conditions

Figure 4: Fitzpatrick skin types [33].

Our proposed models were evaluated on a self-collected dataset. The dataset
consisted of hand video recordings and SpO2 data from 14 participants, of
which there were six males and eight females between the ages of 21 and 30.
Participants were asked to categorize their skin tone based on the Fitzpatrick
skin types [33] shown in Fig. 4. The distribution of the participants’ skin
types is as follows: Two participants of type II, eight participants of type
III, one participant of type IV, and three participants of type V. This
research was using protocol #1376735 approved by the University of Maryland
Institutional Review Board (IRB).

Our dataset consists of four recordings per participant for a total of 56
recordings. Each participant was asked to place his/her hands still on a table
to avoid hand motion. Their palm of the right hand and the back of the left
hand are facing the camera, as illustrated in Fig. 5. We refer to these two
hand-video capturing positions as _palm up (PU)_ and _palm down (PD)_ ,
respectively. Each participant was asked to follow the breathing protocol
outlined in Fig. LABEL:sub@fig:breathing. The participant breathes normally
for 30–40 seconds, exhales all the way, and then holds his/her breath for
30–40 seconds. This process is repeated three times for each recording. All
videos were recorded using an iPhone 7 Plus. The participant’s SpO2 was
simultaneously measured using a Contec CMS50E pulse oximeter clamped to the
left index finger of the hand. We use this pulse oximeter as the reference
measurement as it has been validated to be within $\pm 2\%$ of the true SpO2
level for the range of SpO2 levels in our dataset. The video frame rate is 30
fps and the sampling rate for the reference SpO2 measurements is 1 Hz. This
data capturing procedure was repeated twice for each participant with at least
15 minutes between sessions.

Figure 5: Illustration of two hand-video capturing positions. Left hand: palm
down (PD). Right hand: palm up (PU).

(a)

(b)

Figure 6: (a) Breathing protocol that participants were asked to follow,
including 3 cycles of normal breathing and breath holding. (b) Histogram of
SpO2 values in the collected dataset.

The reference SpO2 signal is interpolated to 5 sample points per second to
match the segment sampling rate using a smooth spline approximation [34]. Each
RGB segment and SpO2 value pair is fed into our models as a single data point,
the models output a single SpO2 estimate per segment. To evaluate a model on a
video recording, the model is sequentially fed with all RGB segments from the
recording to generate a time series of preliminarily predicted SpO2 values.
All predictions greater than 100% SpO2 are clipped to 100% based on the
physiological knowledge. A 10-second long moving average filter is applied to
generate a refined time series of predicted SpO2 values.

(a)

(b)

Figure 7: (a) Training vs. validation predictions. (b) Test predictions of
varying performance with reference SpO2. The higher the Pearson correlation,
the better the prediction captures the reference SpO2 trend. The lower the
MAE, the better the prediction captures the dips in SpO2.

###  4.2 Participant-Specific Results

To investigate how well the proposed models could learn to estimate a specific
individual’s SpO2 from his/her own data, we first conducted participant-
specific experiments, that is, we learn individualized models for each
participant.

Experimental Setting. Two recordings per participant were captured with at
least 15 minutes in between. One recording is used for training and validation
of the model and the remaining recording is for testing. An example of the
training and validation predictions curves is shown in Fig.
LABEL:sub@fig:train-val. Each recording contains three breathing cycles, for
each training/validation recording, the first two breathing cycles are taken
for training and the third cycle is used for validation. Splitting the
recordings into cycles instead of randomly sampling the 10-sec overlapping RGB
segments ensures that there are no overlapping segments of data between the
training and validation set. Example test prediction curves and their
correlation and mean-absolute-error (MAE) are shown for reference in Fig.
LABEL:sub@fig:plots. It should be noted that if the correlation is low, e.g.,
a constant temporal estimate, then the MAE and RMSE metrics are less
meaningful. For the participant-specific experiments, due to the small dataset
size, we augment the training and validation data by sampling with
replacement. This is an example of the bootstrapping data reuse strategy [35,
Chapter 5]. The oversampling also helps address the imbalance in SpO2 data
values that is shown in Fig. LABEL:sub@fig:spo2_distr.

In each experiment, the model structure and hyperparameters are first tuned
using the training and validation data. Once the model has been tuned, we
train multiple instances of the model using the best tuned hyperparameters.
Between each instance, we vary the random seed used for model weights
initialization and random oversampling. Each model instance is evaluated on
the training/validation recording, the model instance that achieves the
highest validation RMSE is selected for evaluation on the test recording. This
model is then evaluated on the test recording to obtain the final test
results.

Results. Table 1 shows the performance comparison of our proposed models with
the prior-art model from Ding  _et al._ [7]. To the best of our knowledge,
Ding  _et al._ ’s model is the only convolutional neural network structure
that has been tried for contact-based SpO2 estimation. Its structure is
similar to our Model 3 but with fewer layers. We also compare with the classic
ratio-of-ratios method proposed by Scully  _et al._ [5]. The performance is
measured in Pearson’s Correlation, mean absolute error (MAE), and root mean
square error (RMSE) and results of each condition are summarized in the median
and interquartile range (IQR). IQR quantifies the spread of an empirical
distribution of a set of data points by computing the difference between the
first quartile and the third quartile of the distribution.

Table 1: Performance comparison of each model structure for participant-
specific experiments. Results are given as the test median and IQR of all
participants.

| Hand | Correlation | MAE (%) | RMSE (%)
---|---|---|---|---
| Mode | Median | IQR | Median | IQR | Median | IQR
Model 1 | PD | 0.41 | 0.40 | 2.12 | 0.91 | 2.51 | 0.78
(Proposed) | PU | 0.39 | 0.37 | 2.16 | 1.80 | 2.70 | 2.09
Model 2 | PD | 0.46 | 0.44 | 2.09 | 1.32 | 2.52 | 1.63
(Proposed) | PU | 0.41 | 0.32 | 1.96 | 0.68 | 2.48 | 0.89
Model 3 | PD | 0.44 | 0.40 | 1.93 | 1.11 | 2.48 | 1.31
(Proposed) | PU | 0.41 | 0.46 | 1.81 | 1.83 | 2.43 | 2.44
Scully _et al._ [5] | PD | 0.08 | 0.37 | 1.94 | 0.92 | 2.22 | 0.77
PU | 0.19 | 0.24 | 2.01 | 0.80 | 2.36 | 0.78
Ding _et al._ [7] | PD | 0.38 | 0.39 | 3.25 | 2.85 | 3.83 | 3.24
PU | 0.34 | 0.56 | 3.40 | 3.16 | 4.58 | 3.12

Table 1 reveals that Model 2 achieves the best correlation in both PD and PU
cases, whereas Model 3 achieves the best MAE and a comparable correlation with
Model 2, suggesting that Model 2 and Model 3 are comparably the best in the
individualized learning. Even though the method proposed in Scully  _et al._
[5] achieves the best (lowest) RMSE, its correlations are the worst (lowest).
This suggests that the classic ratio-of-ratios method cannot track the trend
of SpO2 well using the contactless measurement by smartphone. All of our model
configurations outperform Ding  _et al._ [7]. For example, in the PU case for
Model 3, the correlation is improved from 0.34 to 0.41 and the MAE is lowered
from 3.40% to 1.81%. It is worth noting that the international standard for
clinically acceptable pulse oximeters allows an error of 4% [36], and our
estimation errors are all within this range.

(a)

(b)

Figure 8: Box plots comparing distributions of correlations for (a) lighter
vs. darker skin types, and (b) PD vs. PU for all skin types. The PD results
are better for darker skin tones in both the participant-specific and leave-
one-out cases.

There are two factors, including the skin type and the side of the hand, which
might influence the performance of SpO2 estimation. We therefore investigate
the following two questions: (1) Whether the different skin types matter in PU
or PD cases, and (2) whether the side of hand matters in lighter skin (types
II + III) or darker skin (types IV + V). The box plots in Fig. 8 summarize the
distributions of the test correlations from all the three proposed models in
PU and PD modes of (a) lighter-skin and darker-skin participants, and (b) all
participants.

Bayesian statistical test: We use Bayesian statistical tests to further
analyze the results in Fig. 8 by providing a probabilistic assessment of
whether the results from two groups being compared have the same mean [37, 38,
39, 40, 41]. We avoid using the popular $t$-test because it makes only a
binary decision due to its lack of direct information about the probability of
difference between group means of the given data [38]. In contrast, the
Bayesian statistical test computes the posterior distribution of difference
between the two group means to quantify its certainty of possible values [37].
The decision rule of the Bayesian statistical test for the null hypothesis
that the two groups have the same mean can be stated as follows given the
region of practical equivalence (ROPE) of zero difference [40]:

  * •

(Accepted): If the percentage of the posterior distribution of the group-mean
difference inside the ROPE is sufficiently high (e.g., greater than 95% [40]),
then the null hypothesis is accepted.

  * •

(Rejected): If the percentage of the posterior distribution of the group-mean
difference inside the ROPE is sufficiently low (e.g., less than 2.5% [40]),
then the null hypothesis is rejected.

  * •

(Undecided): When the null hypothesis is neither accepted nor rejected, the
percentage of the posterior distribution of the group-mean difference inside
the ROPE can be used to quantify the certainty that two group means are the
same. One example is shown in Fig. 9.

Figure 9: Posterior distribution of the difference of group means. This shows
an example of an undecided case of the Bayesian statistical test given that
the ROPE of zero difference is set to $[-0.03,0.03]$ and 33% of the posterior
distribution falls within the ROPE. The percentage of coverage can be used to
quantify the certainty that two groups have the same mean.

To conduct the Bayesian statistical tests, we use an R statistical package
named BEST [42]. To determine the ROPE on the difference between the means, we
use Cohen’s established convention that the ROPE of small standardized mean
difference is $[-0.1,0.1]$ [39, 41]. Given the standard deviation $0.3$ of our
data, the ROPE for the difference of means of our data is scaled to
$[-0.03,0.03]$.

To answer question (1) about the impact of the skin type on the prediction
performance, we focus on the left panel of Fig. LABEL:sub@fig:skin. For the PD
case, only 14% of the posterior distribution of the difference between the
means of the lighter and darker skin groups falls in the ROPE. For the PU
case, 23% of the posterior distribution falls in the ROPE. This suggests that
it is highly credible to conclude that the skin type makes a difference for
SpO2 prediction, and the difference is more certain to be observed when using
the back of the hand as the ROI compared to using the palm.

To answer question (2), we first focus on the left panel of Fig.
LABEL:sub@fig:pd_pu when participants of all skin colors are considered
together. 33% of the posterior distribution of the difference of means between
PU and PD cases falls in the ROPE. We then zoom into the darker skin group as
shown in the left panel of Fig. LABEL:sub@fig:skin, only 15% of the posterior
distribution of the difference of means between PD and PU cases falls in the
ROPE, whereas for the lighter skin group, 31% of the posterior distribution
falls in the ROPE. This implies that it is highly credible that the side of
the hand may have some impact on SpO2 prediction, especially when concerning
mainly the darker skin group.

###  4.3 Leave-One-Participant-Out Results

To investigate whether the features learned by the model from other
participants are generalizable to new participants whom it has not seen
before, we conduct leave-one-participant-out experiments. For each experiment,
when testing on a certain participant, we use all the other participant’s data
for training and leave the test participant’s data out. The recordings from
all the non-test participants are used for participant-wise cross-validation
to select the best model structure and hyperparameters. The selected model is
evaluated on the two recordings of the test participant, whose data was never
seen by the model during training.

Table 2 shows the performance comparison of each model in leave-one-
participant-out experiments. Model 1 achieved the best performance in terms of
correlation and achieved the best MAE and RMSE for the PU case. Similar to the
participant-specific case, the classic ratio-of-ratios method proposed in
Scully  _et al._ [5] achieved better MAE and RMSE results for the PD case but
the correlation result was low, suggesting that the model achieved low error
by simply predicting a nearly constant SpO2 near the middle of the SpO2 range.
The best performance of Model 1 in the leave-one-participant-out experiment
may imply that the features extracted after combining the color channels at
the beginning of the pipeline can be generalized better to unseen participants
than the features extracted before channel combination or through interleaving
as in Models 2 or 3.

In the participant-specific case, the model is specifically tailored to the
test individual, whereas the leave-one-participant-out case is more difficult
because the model needs to accommodate for the variation in the population. As
expected, in Fig. 8, we observe that the overall results from the leave-one-
participant-out experiments do not match those from the participant-specific
experiments. Because of the modest size of the dataset, the model has not seen
as diverse data as a larger and richer dataset would offer. The generalization
capability to new participants can be improved when more data is available.

Table 2: Performance comparison of each model structure in leave-one-
participant-out experiments. Results are given as the test median and IQR of
all participants.

| Hand | Correlation | MAE (%) | RMSE (%)
---|---|---|---|---
| Mode | Median | IQR | Median | IQR | Median | IQR
Model 1 | PD | 0.33 | 0.42 | 2.33 | 1.07 | 3.07 | 1.52
(Proposed) | PU | 0.46 | 0.36 | 1.97 | 0.80 | 2.32 | 0.87
Model 2 | PD | 0.15 | 0.50 | 2.43 | 0.94 | 3.35 | 1.11
(Proposed) | PU | 0.33 | 0.39 | 2.08 | 0.73 | 2.41 | 0.71
Model 3 | PD | 0.23 | 0.38 | 2.48 | 1.18 | 2.98 | 1.33
(Proposed) | PU | 0.27 | 0.31 | 2.02 | 1.03 | 2.54 | 1.28
Scully _et al._ [5] | PD | 0.05 | 0.43 | 2.08 | 0.65 | 2.44 | 1.14
PU | 0.01 | 0.54 | 2.08 | 0.60 | 2.43 | 1.20
Ding _et al._ [7] | PD | 0.11 | 0.56 | 3.19 | 1.61 | 3.76 | 1.52
PU | 0.26 | 0.42 | 2.43 | 1.22 | 2.85 | 1.51

We now revisit the two research questions raised in Section 4.2 under the
leave-one-participant-out scenario. First, we analyze the impact of skin type
given the same side of the hand. From the right panel of Fig.
LABEL:sub@fig:skin, in the PD case, only 0.04% of the posterior distribution
of the difference of means between lighter and darker skin groups is within
the ROPE, suggesting that the null hypothesis is rejected and the darker skin
group outperforms the lighter skin group. In the PU case, 18% of the posterior
distribution falls within the ROPE. This observation is consistent with the
participant-specific experiments that when using the back of the hand as the
ROI, the skin color is more credible to be a factor in the accuracy of SpO2
estimation than using the palm.

Second, we analyze the impact of the side of the hand for two skin color
groups. For the darker skin group shown in the right panel of Fig.
LABEL:sub@fig:skin, only 9% of the posterior distribution of the difference of
means of the PU and PD cases falls in the ROPE. This shows that there is a
high uncertainty in the estimate of zero difference, which is consistent with
the results from the participant-specific experiments. However, unlike the
participant-specific experiments, for the lighter skin group, 0.2% of the
posterior distribution of the difference of means between PU and PD cases
falls in the ROPE. This suggests that the null hypothesis is rejected and that
the PU outperforms the PD in the lighter skin group. As for the mixed group
illustrated in the right panel of Fig. LABEL:sub@fig:pd_pu, only 8% of the
posterior distribution of the difference of means falls in the ROPE,
suggesting that there is a high uncertainty to conclude that PU and PD cases
are comparable.

This different generalization capability in the PU and PD cases may be
attributed to the skin color difference between the palm and the back of the
hand. The color of the back of the hand tends to be darker than the color of
the palms and has larger color variation among participants due to different
degrees of sunlight exposure. In contrast, the color variation of the palms is
much milder among participants. Furthermore, in the participant-specific
experiments, the individualized models learn the traits of the skin type and
the side of the hand from each participant, whereas, in the leave-one-
participant-out experiments, the learned model must capture the general
characteristics of the population.

###  4.4 Ablation Studies

To justify the use of nonlinear channel combinations and convolutional layers
for temporal feature extraction in our proposed models, we conduct two
ablation studies comparing the performance of these model components to other
generic ones. We focus on the PU case to avoid the uncontrolled impact of such
factors as skin tone and hair. In the first ablation study, we compare
nonlinear to linear channel combination. We create a variant of Model 1 with
only a single linear channel combination layer with no activation function and
repeat the leave-one-participant-out experiments. In the second study, we
compare the performance of using convolutional layers for temporal feature
extraction to using fully-connected dense layers. We create this second
variant of Model 1 and repeat leave-one-participant-out experiments.

Table 3: Numerical results of the ablation studies for Model 1 (M1) in the
leave-one-participant-out mode. Comparisons among the proposed (nonlinear) M1,
modified M1 with only linear channel combinations, and modified M1 with fully
connected dense layers instead of convolutional layers are listed. Ablation
studies confirm that the nonlinear channel combinations and convolutional
layers improve model performance.

Method |  | $\rho$ | MAE(%) | RMSE(%)
---|---|---|---|---
Linear Ch. Comb. | Median | 0.46 | 2.14 | 2.66
\+ Conv. layer for Feat. Extra. | IQR | 0.38 | 0.73 | 0.93
Nonlinear Ch. Comb. | Median | 0.41 | 2.29 | 2.66
\+ Fully Connec. layer for Feat. Extra. | IQR | 0.39 | 0.63 | 0.70
Model 1 (Proposed): Nonlinear Ch. Comb. | Median | 0.46 | 1.97 | 2.32
\+ Conv. layer for Feat. Extra. | IQR | 0.36 | 0.80 | 0.87

Table 3 presents the medians and IQRs specified for numerical comparison of
the ablation study. First, we compare the first and the third rows in Table 3
for ablation study 1. Our proposed Model 1 achieves a better correlation with
a median of 0.46 and IQR of 0.36 and a better RMSE with a median of 2.32 and
IQR of 0.87 than its linear channel combination variant. Besides, Model 1
achieves a comparable MAE with a better median of 1.97 but a wider IQR of
0.80. The overall better performance of Model 1 suggests the necessity of
using the nonlinear channel combination method. Second, in ablation study 2,
we compare the second and the third rows in Table 3. We observe that Model 1
outperforms its second variant with fully connected layers for feature
extraction with better medians in terms of correlation (0.46 vs. 0.41), MAE
(1.97 vs. 2.29), and RMSE (2.32 vs. 2.66), and narrower IQR of correlation.
This suggests that convolutional layers are better than fully connected layers
for temporal feature extraction.

##  5 Discussion

###  5.1 Contact-based Dataset Testing

We also test our models on the publicly available dataset gathered by Nemcova
_et al._ for their SpO2 estimation work [19]. This dataset consists of
contact-based smartphone video recordings where a participant placed a finger
on the smartphone camera and was illuminated by the camera flashlight.
Participants were asked to breathe normally without following any
sophisticated breathing protocol. Each recording lasts about 10 to 20 seconds.
The subject for each recording is not identified, so subject-specific and
leave-one-participant-out experiments cannot be conducted. There is a single
reference SpO2 value associated with each recording. We used 14 recordings for
training and seven recordings for testing and compared them with the modified
ratio-of-ratios method proposed in their paper.

As shown in Table 4, Models 1 and 2 outperform the method used by Nemcova  _et
al._ on both the training and test recordings. Model 3 is not able to
generalize well from the training set to the test set, which may be due to the
small size of the dataset. It should be noted that because the participants
were not asked to follow any sophisticated breathing protocol, the dynamic
range of SpO2 values is narrow. These results show that our CNN Models 1 and 2
work well for contact-based video recordings in addition to contactless videos
recordings.

Table 4: Experimental results of proposed methods on the contact-based video SpO2 dataset from Nemcova _et al._ [19]. One SpO2 estimate was output per recording and MAE and RMSE were calculated across all recordings. Models 1 and 2 outperform the method proposed by Nemcova _et al._ , Model 3 was unable to generalize well to the test set. | MAE (%) | RMSE (%)
---|---|---
| Training | Test | Training | Test
Model 1 | 0.86 | 1.19 | 0.94 | 1.36
Model 2 | 0.50 | 1.28 | 0.59 | 1.64
Model 3 | 0.75 | 3.28 | 0.99 | 3.69
Nemcova _et al._ [19] | 2.05 | 2.18 | 2.24 | 2.36

###  5.2 Ability to Track SpO2 Change

Figure 10: Histograms of correlation values between reference SpO2 signals and
(a) randomly generated SpO2 signals, or (b) SpO2 signals predicted by neural
network Model 2. The correlation distribution for Model 2 is centered much
higher than the random guess, confirming Model 2’s capability to track SpO2.

By employing the standard machine learning methodology of training-validation-
test split in Section 4 to learn neural networks that perform well on unseen
data, we have already ensured the generalizability of our models [43, Chapter
11]. As further evidence that our models are capable of outputting meaningful
predictions, we compare SpO2 predictions from our learned models to randomly
generated SpO2 values. For each reference signal, a random prediction signal
was generated by choosing SpO2 values between the minimum and maximum values
from the reference signal and applying a moving average window in the same way
as is applied to the neural network predictions. Fig. 10(a) shows a histogram
of the correlations between the reference SpO2 signals and the randomly
generated predictions and Fig. 10(b) shows a histogram of correlations between
the reference SpO2 signals and the predictions generated by Model 2. It is
revealed that the neural network with a median correlation of $0.41$111It has
been shown in other applications that even low correlation coefficients can be
meaningful. For example, in photo response non-uniformity (PRNU) work, the
device used to take a photo can be predicted with correlation values below 0.1
[44]. outperforms random guessing with a median correlation of $-0.02$,
confirming Model 2’s capability to track SpO2.

###  5.3 Visualizations of RGB Combination Weights

(a)

(b)

(c)

(d)

Figure 11: Learned RGB channel weights. Plots (a) and (b) are the channel
weights learned by different model instances trained on the data of all study
participants together, projected onto the RB and RG planes in the RGB space.
Plots (c) and (d) are the RB and RG projections of the learned channel weights
for model instances trained on random subsets of the participants’ data. Each
point is color-coded according to the correlation $\rho$ achieved by the
instance.

To understand and explain what our physiologically inspired models have
learned, we conduct a separate investigation to visualize the learned weights
for the RGB channels. Our goal is to understand the best way to combine the
RGB channels for SpO2 prediction. Having an explainable model is important for
a physiological prediction task like this. Our neural network models can be
considered as nonlinear approximations of the hypothetically true function
that can extract the physiological features related to SpO2 buried in the RGB
videos. The ratio-of-ratios method, for example, is another such extractor
that combines the information from the different color channels at the end of
the pipeline. For this experiment, we use the modified version of Model 1 from
the ablation studies that has only a single linear channel combination at the
beginning. Seeing that using a single linear channel combination did not
significantly reduce model performance in the ablation studies, and
understanding that the linear component may dominate the Taylor expansion of a
nonlinear function, we use only linear combinations for this model to
facilitate more interpretable visualizations.

We have trained 100 different instances of the model on the first two cycles
from all the recordings and tested on the third cycle from all recordings. The
difference between each instance is that the weights are randomly initialized.
The weights for each channel learned by the model instances were visualized as
points representing the heads of the linear combination vector in RGB space.
Each point is colored according to the average test correlation achieved by
the model instance. Figs. LABEL:sub@fig:RB and LABEL:sub@fig:RG show the
projections of these points onto the RB and RG planes. The subfigures reveal
that the majority of the channel weights lay along certain lines in the RGB
space. For the weights on the line, the ratio of the blue channel weight to
the red channel weight is 0.87, the ratio of the green channel weight to red
channel weight is 0.18. It is clear that the red and blue channels are the
dominating factors for SpO2 prediction.

To further verify this result, we repeat this experiment under the following
setup: instead of using the data from all participants, for each model
instance, we randomly select seven participants and use their data for
training and testing. In this case, the difference between each model instance
is not only the initialized weights but also the random subset of participants
that the model was trained on. Fig. LABEL:sub@fig:RG_7 reveals that most of
the better-performing instances (with $\rho\geq 0.45$) have little
contribution from the green channel. In Fig. LABEL:sub@fig:RB_7, we again see
that most of the points lay on a line in the RB plane, the ratio of the blue
channel weight to the red channel weight for these points is 0.80.

These results are in accordance with the biophysical understanding of how
light is absorbed by hemoglobin in the blood. Recall that Fig. 2 reveals a
large difference between the extinction coefficients, or the amount of light
absorbed, by deoxygenated and oxygenated hemoglobin at the red wavelength.
There is a significantly smaller difference at the blue wavelength and almost
no difference at green. The amount of light absorbed influences the amount of
light reflected which can be measured through the camera. A larger difference
in extinction coefficients makes it easier to measure the ratio of light
absorbed by oxygenated vs. deoxygenated hemoglobin over time. This ratio
indicates the level of blood oxygen saturation. Therefore, from a
physiological perspective, it makes sense for the neural networks to give
larger weight to the red and then blue channels and give little to the green
channel. These visualizations indicate that the models are learning physically
meaningful features.

##  6 Conclusion and Future Work

In this paper, we have proposed the first CNN-based work to solve the
challenging problem of video-based remote SpO2 estimation. We have designed
three optophysiologically inspired neural network architectures. In both
participant-specific and leave-one-participant-out experiments, our models are
able to achieve better results than the state-of-the-art method. We have also
analyzed the effect of skin color and the side of the hand on SpO2 estimation
and have found that in the leave-one-participant-out experiments, the side of
the hand plays an important role, with better SpO2 estimation results achieved
in the palm-up case for the lighter-skin group. We have also shown the
explainability of our designed architectures by visualizing the weights for
the RGB channel combinations learned by the neural network, and have confirmed
that the choice of the color band learned by the neural network is consistent
with the established optophysiological methods.

In future work, we plan to investigate the impact of decreasing the
measurement duration since shorter measurement duration is generally preferred
in clinical applications. Another direction is to investigate the
effectiveness and performance of the proposed SpO2 estimation method in
clinical or physiological applications while the participants are in motion.

## References

  * [1] M. Nitzan, A. Romem, and R. Koppel, “Pulse oximetry: Fundamentals and technology update,” _Medical Devices_ , vol. 7, p. 231, 2014.
  * [2] J. Couzin-Frankel, “The Mystery of The Pandemic’s “Happy Hypoxia”,” _Science_ , 2020. [Online]. Available: https://science.sciencemag.org/content/368/6490/455
  * [3] N. Starr, D. Rebollo, Y. M. Asemu, L. Akalu, H. A. Mohammed, M. W. Menchamo, E. Melese, S. Bitew, I. Wilson, M. Tadesse _et al._ , “Pulse oximetry in low-resource settings during the COVID-19 pandemic,” _The Lancet Global Health_ , Sep. 2020.
  * [4] J. W. Severinghaus, “Takuo Aoyagi: Discovery of pulse oximetry,” _Anesthesia & Analgesia_, Dec. 2007.
  * [5] C. G. Scully, J. Lee, J. Meyer, A. M. Gorbach, D. Granquist-Fraser, Y. Mendelson, and K. H. Chon, “Physiological parameter monitoring from optical recordings with a mobile phone,” _IEEE Trans. on Biomedical Eng._ , Jul. 2011.
  * [6] Z. Lu, X. Chen, Z. Dong, Z. Zhao, and X. Zhang, “A prototype of reflection pulse oximeter designed for mobile healthcare,” _IEEE Journal of Biomedical and Health Informatics_ , Aug. 2015.
  * [7] X. Ding, D. Nassehi, and E. C. Larson, “Measuring oxygen saturation with smartphone cameras using convolutional neural networks,” _IEEE Journal of Biomedical and Health Informatics_ , Dec. 2018.
  * [8] L. J. Herbert and I. H. Wilson, “Pulse oximetry in low-resource settings,” _Breathe_ , vol. 9, no. 2, pp. 90–98, 2012. [Online]. Available: https://breathe.ersjournals.com/content/9/2/90
  * [9] L. Kong, Y. Zhao, L. Dong, Y. Jian, X. Jin, B. Li, Y. Feng, M. Liu, X. Liu, and H. Wu, “Non-contact detection of oxygen saturation based on visible light imaging device using ambient light,” _Opt. Exp._ , Jul. 2013.
  * [10] M. Van Gastel, S. Stuijk, and G. De Haan, “New Principle for Measuring Arterial Blood Oxygenation, Enabling Motion-Robust Remote Monitoring,” _Scientific Reports_ , Dec. 2016.
  * [11] D. Shao, C. Liu, F. Tsow, Y. Yang, Z. Du, R. Iriya, H. Yu, and N. Tao, “Noncontact Monitoring of Blood Oxygen Saturation Using Camera and Dual-wavelength Imaging System,” _IEEE Trans. Biomed. Eng_ , Sep. 2015.
  * [12] H.-Y. Tsai, K.-C. Huang, and J. A. Yeh, “No-contact oxygen saturation measuring technology for skin tissue and its application,” _IEEE Instrum. Meas. Magazine_ , Sep. 2016.
  * [13] M. van Gastel, W. Verkruysse, and G. de Haan, “Data-driven Calibration Estimation for Robust Remote Pulse-oximetry,” _Applied Sciences_ , Jan. 2019\.
  * [14] U. Bal, “Non-contact estimation of heart rate and oxygen saturation using ambient light,” _Biomedical Optics Express_ , Jan. 2015.
  * [15] G. Casalino, G. Castellano, and G. Zaza, “A mHealth solution for contact-less self-monitoring of blood oxygen saturation,” in _IEEE Symposium on Computers and Communications (ISCC)_ , Jul. 2020.
  * [16] L. Tarassenko, M. Villarroel, A. Guazzi, J. Jorge, D. Clifton, and C. Pugh, “Non-contact Video-based Vital Sign Monitoring Using Ambient Light and Auto-regressive Models,” _Physiol. Meas_ , Mar. 2014.
  * [17] Z. Sun, Q. He, Y. Li, W. Wang, and R. K. Wang, “Robust non-contact peripheral oxygenation saturation measurement using smartphone-enabled imaging photoplethysmography,” _Biomed. Opt. Exp._ , vol. 12, no. 3, pp. 1746–1760, Mar. 2021.
  * [18] “Optical Absorption of Hemoglobin,” https://omlc.org/spectra/hemoglobin/, Accessed: 2021-03-09.
  * [19] A. Nemcova, I. Jordanova, M. Varecka, R. Smiseka, L. Marsanova, L. Smital, and M. Vitek, “Monitoring of heart rate, blood oxygen saturation, and blood pressureusing a smartphone,” _Biomedical Signal Processing and Control_ , May 2020.
  * [20] F. Lamonaca, D. Carni, D. Grinaldi, A. Nastro, M. Riccio, and V. Spagnolo, “Blood oxygen saturation measurement by smartphone camera,” _IEEE Intl. Symposium on Medical Measurements and Applications (MeMeA) Proceedings_ , May 2015.
  * [21] X. Niu, S. Shan, H. Han, and X. Chen, “RhythmNet: End-to-end heart rate estimation from face via spatial-temporal representation,” _IEEE Trans. on Image Processing_ , Oct. 2019.
  * [22] W. Chen and D. McDuff, “DeepPhys: Video-based physiological measurement using convolutional attention networks,” in _The European Conference on Computer Vision (ECCV)_ , 2018, pp. 349–365.
  * [23] R. Špetlík, V. Franc, and J. Matas, “Visual heart rate estimation with convolutional neural network,” in _British Machine Vision Conf., Newcastle, UK_ , Sep. 2018.
  * [24] Y. Zheng, H. Wang, and Y. Hao, “Mobile application for monitoring body temperature from facial images using convolutional neural network and support vector machine,” _Mobile Multimedia/Image Processing, Security, and Applications_ , April 2020.
  * [25] J. G. Webster, _Design of Pulse Oximeters_. CRC Press, Oct. 1997.
  * [26] W. Burger and M. J. Burge, _Digital Image Processing - An Algorithmic Introduction using Java._ Springer, 2008\.
  * [27] N. Otsu, “A threshold Selection Method from Gray-level Histograms,” _IEEE Trans. Syst., Man, and Cybernet._ , Jan. 1979.
  * [28] Q. Zhu, C.-W. Wong, C.-H. Fu, and M. Wu, “Fitness heart rate measurement using face videos,” in _IEEE Int’l Conf. on Image Proc. (ICIP)_ , Sep. 2017.
  * [29] Y. Nam, B. A. Reyes, and K. H. Chon, “Estimation of respiratory rates using the built-in microphone of a smartphone or headset,” _IEEE Journal of Biomedical and Health Informatics_ , Sep. 2015.
  * [30] K. Sohn, F. M. Merchant, O. Sayadi, D. Puppala, R. Doddamani, A. Sahani, J. P. Singh, E. K. Heist, E. M. Isselbacher, and A. A. Armoundas, “A novel point-of-care smartphone based system for monitoring the cardiac and respiratory systems,” _Scientific Reports_ , Mar. 2017.
  * [31] W. Wang, A. C. den Brinker, S. Stuijk, and G. De Haan, “Algorithmic principles of remote PPG,” _IEEE Trans. on Biomedical Eng._ , Sep. 2016\.
  * [32] L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar, “Hyperband: A novel bandit-based approach to hyperparameter optimization,” _The Journal of Machine Learning Research_ , Apr. 2018.
  * [33] _Fitzpatrick skin phototype_ , Australian Radiation Protection and Nuclear Safety Agency.
  * [34] B. Green, P. J.; Silverman, _Nonparametric Regression and Generalized Linear Models_. Chapman and Hall, 1990\.
  * [35] G. James, D. Witten, T. Hastie, and R. Tibshirani, _An Introduction to Statistical Learning_. Springer, 2013\.
  * [36] _Particular requirements for basic safety and essential performance of pulse oximeter equipment_ , International Organization for Standardization, 2011\.
  * [37] J. K. Kruschke and T. M. Liddell, “The bayesian new statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a bayesian perspective,” _Psychonomic Bulletin & Review_, vol. 25, no. 1, pp. 178–206, 2018.
  * [38] J. K. Kruschke, “Bayesian estimation supersedes the t test.” _Journal of Experimental Psychology: General_ , vol. 142, no. 2, p. 573, 2013.
  * [39] J. K. Kruschke and T. M. Liddell, “Bayesian data analysis for newcomers,” _Psychonomic bulletin & review_, vol. 25, no. 1, pp. 155–177, 2018.
  * [40] J. K. Kruschke, “Bayesian analysis reporting guidelines,” _Nature Human Behaviour_ , pp. 1–10, 2021.
  * [41] ——, “Rejecting or accepting parameter values in bayesian estimation,” _Advances in Methods and Practices in Psychological Science_ , 2018.
  * [42] “R Package for BEST: Bayesian Estimation Supersedes the t-Test,” https://CRAN.R-project.org/package=BEST, Accessed: 2021-09-30.
  * [43] S. Shalev-Shwartz and S. Ben-David, _Understanding Machine Learning: From Theory to Algorithms_. Cambridge University Press, 2014.
  * [44] T. Baar, W. van Houten, and Z. Geradts, “Camera identification by grouping images from database, based on shared noise patterns,” 2012.