# The Total Solar Irradiance variability in the Evolutionary Timescale and its
Impact on the Mean Earth’s Surface Temperature.

N.T. Shukure Ethiopian Space Science and Technology Institute (ESSTI), Entoto
Observatory and Research Center (EORC),
Astronomy and Astrophysics Research and Development Department, P.O.Box 33679
Addis Ababa, Ethiopia.  Addis Ababa University P.O.Box 1176 Addis Ababa,
Ethiopia  College of Natural & computational Science, Dept. of Physics, Dilla
University, Box 419 Dilla, Ethiopia  S.B Tessema Ethiopian Space Science and
Technology Institute (ESSTI), Entoto Observatory and Research Center (EORC),
Astronomy and Astrophysics Research and Development Department, P.O.Box 33679
Addis Ababa, Ethiopia.  Addis Ababa University P.O.Box 1176 Addis Ababa,
Ethiopia  N. Gopalswamy Solar Physics Laboratory, NASA Goddard Space Flight
Center, Greenbelt, MD 20771, USA.

###### Abstract

The Sun is the primary source of energy for the Earth. The small changes in
total solar irradiance (TSI) can affect our climate in the longer timescale.
In the evolutionary timescale, the TSI varies by a large amount and hence its
influence on the Earth’s mean surface temperature (Ts) also increases
significantly. We develop a mass-loss dependent analytical model of TSI in the
evolutionary timescale and evaluated its influence on the Ts. We determined
the numerical solution of TSI for the next 8.23 Gyrs to be used as an input to
evaluate the Ts which formulated based on a zero-dimensional energy balance
model. We used the present-day albedo and bulk atmospheric emissivity of the
Earth and Mars as initial and final boundary conditions, respectively. We
found that the TSI increases by 10% in 1.42 Gyr, by 40% in about 3.4 Gyrs, and
by 120% in about 5.229 Gyrs from now, while the Ts shows an insignificant
change in 1.644 Gyrs and increases to 298.86 K in about 3.4 Gyrs. The Ts
attains the peak value of 2319.2 K as the Sun evolves to the red giant and
emits the enormous TSI of 7.93$\times 10^{6}Wm^{-2}$ in 7.676 Gys. At this
temperature Earth likely evolves to be a liquid planet. In our finding, the
absorbed and emitted flux equally increases and approaches the surface flux in
the main sequence, and they are nearly equal beyond the main sequence, while
the flux absorbed by the cloud shows opposite trend.

TThe Sun’s evolution, mass-loss, TSI variability, Earth’s mean surface
temperature.

##  1 Introduction

The Sun is the largest energy source to the Earth’s atmosphere (Kren, 2015;
Kren et al.,, 2017). The small changes in total solar irradiance (TSI), which
is also called solar constant ($S_{0}$), is the energy radiated by the Sun
that is incident on the Earth’s atmosphere. TSI varies by about 0.1% over the
solar-cycle timescale (Fröhlich, 2006; Kopp, 2016). It is varying with time
and its tiny changes in TSI affect the Earth’s climate in the longer timescale
(Eddy, 1976). This idea is strengthened by many other studies (Haigh, 2007;
Lean & Rind, 2008; Gray et al.,, 2010; Ineson, 2011; Ermolli et al.,, 2013;
Solanki et al.,, 2013; Kopp, 2016). The space-borne instruments have been
measuring the TSI since 1978. Its average value is 1361 $Wm^{-2}$ at 1 au
(Kopp & Lean, 2011).
In the evolutionary timescale, the Sun’s luminosity changes by a large
amount.The term ”evolution” refers a star’s change over the course of time.
Currently, the solar luminosity is increasing by 0.009% per million years
(Hecht, 1994). At this rate, the TSI increases by 0.1% in 10 million years.
Following this rate, TSI is expected to increase by 10% in 1 Gyr at which time
the Earth will become uninhabitable. In the same way, the luminosity will
increase by 40% in about 3.5 Gyrs (Hecht, 1994; Kopp, 2016). As Kopp (2016)
indicated, the TSI increases as the solar luminosity increases in the
evolutionary timescale. However, there is a limitation of estimation of the
TSI in the evolutionary timescale. Kaplan (1981) estimated the TSI to be
varying from 1.1$S_{0}$ to 1.4$S_{0}$. This variation is in about 1.1 to 3.5
Gyrs as indicated by Sackmann (1993); Hecht (1994). However, to study the
influence of the TSI on the Earth’s mean surface temperature ($T_{s}$) during
Sun’s lifetime, modeling of the TSI in relation to the evolutionary timescale
is needed.
O’Malley-James (2013) used time-dependent luminosity to study the TSI
variability and found its influence on $T_{s}$ in the evolutionary timescale.
However, the solar mass-loss has an influence on the solar luminosity on this
timescale. Moreover, their work is limited to the next 3 Gyrs, although
further study is needed to understand the variability of an extended time
interval. Schröde et al., (2001) presented a model of $T_{s}$ in which they
include the effect of solar mass variation, and predict the $T_{s}$ of
different planets including Earth over the lifetime of the Sun. They conclude
that we can survive although the maximum ratio of the future to the present
temperature is about 6.6. However, in their modeling, they assume that albedo
($\alpha$) is constant, and neglect the effect of bulk emissivity
($\varepsilon$). But, $\alpha$ is expected to be changing, perhaps decreases
in the future. Therefore, in our model, we incorporate the climate parameters:
non-constant $\alpha$ and $\varepsilon$. Moreover, in our modeling, we include
the solar mass-loss in both TSI and $T_{s}$ models.
The aim of this work is to model the TSI variability and $T_{s}$ in the
evolutionary timescale and to study the link between the two parameters.
In this study, we developed an analytical model of the TSI variability and
$T_{s}$ in the evolutionary timescale. The base of our formulation is the
models developed by Shukure et al., (2019) and N. T. Shukure et al. (2021,
submitted to APJ). They developed a mass-loss dependent solar luminosity in
the evolutionary timescale. To calculate the solar mass-loss, they first
calculate the solar mass reduction every 1 million years for the next 8.23
Gyrs. Then, they calculate the mass-loss at each interval of 1 million years
by subtracting every newly reduced solar mass from the present-day solar mass.
Following this method, we calculate the mass-loss to model TSI in the
evolutionary timescale. Using the newly formulated TSI model and zero-
dimensional energy balance model (EBM), we formulate $T_{s}$. We used the
numerical method to solve the newly formulated models by setting some boundary
conditions to $\varepsilon$ and $\alpha$ that will be explained in detail in
the method part.

##  2 Method

###  2.1 Mathematical Formulation

####  2.1.1 The Solar Mass-loss

In modeling TSI in evolutionary timescale, considering the solar mass-loss
effect is mandatory. The solar mass reduction and solar mass-loss are modeled
by Shukure et al., (2019):

$M(t)=M_{\odot}e^{-\delta t},$ (1)

where $M_{\odot}$ is the present-day solar mass, $\delta$ determines the rate
of decay (Langtangen, 2016; Strang & Herman, 2021) of the total mass of the
star. It is defined as the ratio of a star’s mass-loss rate ($\dot{M}$) to the
star’s mass which is measured in per year ($yr^{-1}$) N. T. Shukure et al.
(2021, submitted to APJ). Eq. 1 estimates the variation of the original mass
of the Sun with time and t is the time needed to change the original mass of
the star. For $M(t=0)=M_{\odot}$ which is original mass of the Sun. However,
the solar mass-loss $(\Delta{M})$ is the difference between original mass $\&$
the remained mass $(M(t))$.

$\Delta{M}=M_{\odot}-M(t).$ (2)

Eq. 2 estimates the solar $\Delta{M}$ at any time t.

####  2.1.2 Luminosity-mass Loss Relation

The $\Delta{M}$ dependent luminosity that is modeled by N. T. Shukure et al.
(2021, submitted to APJ) is:

$L(\Delta{M})=\frac{L_{\odot}}{1+\frac{2}{5}\left[1+\frac{1}{\delta
t_{\odot}}ln\left(1-\frac{\Delta{M}}{M_{\odot}}\right)\right]-\frac{2}{5}},$
(3)

where, $L_{\odot}=3.85\times 10^{26}\>W$ is the present-day solar luminosity,
and $t_{\odot}$ = 4.57 Gyr is the present-day solar age (Dwivedi2010makes;
Feulner, 2012). Eq. 3 used as an input to model TSI in the evolutionary
timescale.

###  2.2 Modeling Total solar irradiance in the evolutionary timescale

The TSI for the present-day solar constant is (O’Malley-James, 2013):

$S_{0}=\frac{L_{\odot}}{4\pi Kd^{2}_{E}},$ (4)

where k is a constant (k $\approx$ 1) and $d_{E}$ is the Earth-Sun distance.
Eq. 4 estimates the amount of energy which cross an area $A=1m^{2}$ (one
square meter). For our new model of luminosity in the evolutionary timescale,
Eq. 4 can be re written as:

$S_{evolution}=\frac{L(\Delta{M})}{4\pi Kd^{2}_{E}}.$ (5)

Re writing Eq. 5 in terms of Eq. 3 become

$S_{evolution}=\frac{S_{0}}{1+\frac{2}{5}\left[1+\frac{1}{\delta
t_{\odot}}ln\left(1-\frac{\Delta{M}}{M_{\odot}}\right)\right]-\frac{2}{5}}.$
(6)

Eq. 6 estimates the TSI variability in the evolutionary timescale as solar
mass-loss significantly increases.

###  2.3  Earth’s Mean Surface Temperature Variation

The temperature of any planet varies as the Sun evolves. Our main interest in
this section is to model $T_{s}$. In the modeling, we excluded the greenhouse
effect on the $T_{s}$ variation. The base of our modeling is the EBM. The
energy flux that crosses a unit area is estimated by Eq. 4. If we consider the
region of our Earth that faces the Sun as a circular area, the total amount of
energy flux ($\phi$) that intercept the Earth’s circular area is the product
of the flux through the unit area ($S_{0}$) and the area of the circle:

$\phi_{intercept}=S_{0}\pi R^{2}_{E},$ (7)

where $R_{E}$ is the radius of Earth. However, the total surface area of Earth
$(A_{E})$ is

$A_{E}=4\pi R_{E}^{2}.$ (8)

The average energy absorbed by the Earth is the ratio of the total energy
absorbed by the Earth to the surface area of the Earth and it is written as:

$\phi_{absorbed}=\frac{S_{0}}{4}.$ (9)

Eq. 9 can be the flux absorbed by a completely dark planet whose albedo is
zero. However, planet Earth has an albedo different from zero. Hence, the
absorbed $\phi$ by Earth is defined as Mcguffie & Hendersonsellers (1987):

$\phi_{absorbed}=\frac{S_{0}}{4}(1-\alpha),$ (10)

where $\alpha$ is the temperature-dependent Earth’s albedo taken to be varying
in the interval $0\leq\alpha\leq 1$. Energy emitted by Earth’s surface is
estimated by the Stefan-Boltzmann law:

$\phi_{s}=\sigma T_{s}^{4},$ (11)

where $\phi_{s}$ is the flux that escapes from the surface of the Earth,
$\sigma$ is Stefan-Boltzmann constant and $T_{s}$ is the Earth’s mean surface
temperature. However, Eq. 11 needs a correction for the cloud absorption.The
flux absorbed by clouds is:

$\phi_{c}=\varepsilon\sigma T_{s}^{4},$ (12)

where $\phi_{c}$ flux absorbed by cloud, $\varepsilon$ is dimensionless bulk
emissivity that can be in the interval $0\leq\varepsilon<1$ (Swift, 2013). The
flux radiated out of the atmosphere is :

$\phi_{emitted}=\left(1-\varepsilon\right)\sigma T_{s}^{4}.$ (13)

When the energy of a planet is at thermal equilibrium, the absorbed and
emitted $\phi$ by Earth become equals.

$\phi_{absorbed}=\phi_{emitted}$ (14)

Substituting Eq. 10 and 13 in to Eq. 14 one can calculate the $T_{s}$ as:

$T_{s}=\left(\frac{S_{0}(1-\alpha)}{4\left(1-\varepsilon\right)\sigma}\right)^{\frac{1}{4}}.$
(15)

Eq. 15 estimates the expected present-day Earth’s mean temperature. In the
evolutionary timescale, the TSI is expected to be changing (i.e.,
$S_{0}\rightarrow S_{evolution}$) and hence, it influences $T_{s}$. Therefore,
the expected $T_{s}$ in the evolutionary timescale can be estimated by
rewriting Eq. 15 in terms of $S_{evolution}$.

$T_{s}=\left(\frac{S_{evolution}(1-\alpha)}{4\left(1-\varepsilon\right)\sigma}\right)^{\frac{1}{4}},$
(16)

where $T_{s}$ changes as the Sun evolves. Since the Earth’s surface covered by
ice will reduce in the future, which will result in the decrease of Earth’s
albedo. Substituting Eq. 6 in to 16, $T_{evolution}$ is estimated as:

$T_{evolution}=\left(\frac{(1-\alpha)S_{0}}{4\left(1-\varepsilon\right)\sigma\left(1+\frac{2}{5}\left[1+\frac{1}{\delta
t_{\odot}}ln\left(1-\frac{\Delta{M}}{M_{\odot}}\right)\right]-\frac{2}{5}\right)}\right)^{\frac{1}{4}}$
(17)

Eq. 17 predicts the $T_{s}$ of our planet in the evolutionary timescale.

###  2.4 Numerical Computations of the Model

In the numerical analysis, we assumed that the present-day position of the
Earth’s orbit is not changing as the Sun evolves. We numerically solved Eq. 6
by setting $\delta$ to be varying as $10^{-13}yr^{-1}\leq\delta\leq 6\times
10^{-11}yr^{-1}$ based on the present-day value of
$\dot{M}=10^{-13}M_{\odot}yr^{-1}$ reported by Feulner (2012). We also assumed
value of $\dot{M}=6\times 10^{-11}M_{\odot}yr^{-1}$ during red giant phase and
we used the solar observables to be $S_{0}=1361Wm^{-2}$ (Schmutz et al.,,
2013), $M_{\odot}=1.99\times 10^{30}$ (Kaplan & Union, 1981) and the present-
day solar age as initial time $t_{\odot}=4.57\times 10^{9}$ (Bahcall et al.,,
1995; Feulner, 2012). The solar mass reduction is calculated for about 8.23
Gyrs from now. The new mass of the Sun, M(t) is recorded at the intervals of
every $1Myrs$ based on Eq. 1 to have a total of 8231 numerical values. The
total numerical values are used for further analysis.
similarly, the solar $\Delta{M}$ in the same time interval is calculated using
Eq. 2. Each of the new numerical values is subtracted from the present-day
mass of the Sun in each time step ($1Myrs$). The new values of $\Delta{M}$ are
used as an input to calculate the TSI in the same time intervals.
To solve Eq. 17 we set some boundary conditions of $\alpha$ and $\varepsilon$.
We set the present-day values of the Earth’s $\alpha$ and $\varepsilon$ to be
0.3 and 0.407, respectively, as initial boundary conditions. By assuming the
future Earth’s atmosphere to be the present-day atmosphere of Mars, we set the
final boundary condition to be 0.250 and 0.0038, respectively, for $\alpha$
and $\varepsilon$ (Swift, 2013).
We numerically solved Eq. 17 for the next 8.23 Gyrs by using the numerical
solution of TSI as an input. Our program determines the $T_{s}$ every 1
million years. We then calculate $\phi_{absorbed}$, $\phi_{s}$, $\phi_{c}$,
and $\phi_{emitted}$ based on Eq. 10, 11, 12 , and 13, respectively, by using
the $T_{s}$ solution as an input.

##  3 Result and Discussion

###  3.1 Total solar irradiance variability in the evolutionary timescale

Figure 1 shows the TSI variability as the Sun evolves. Figure 1a indicates
that the TSI increases to 1448.1 $Wm^{-2}$ in the first 1 Gyr from now,
representing an increase by 6.4% with respect to the present-day value. At
about 1.42 Gyr from now, the TSI changes to 1497.1 $Wm^{-2}$ which is about
10% increase compared to the present value. At this value of TSI, the Earth
will lose water (Hecht, 1994). However according to our model, the 10%
variability achieved at about 1.42 Gyrs that lags by 0.32 Gyrs as compared
with Hecht’s finding.
In 3.374 Gyrs, the TSI increases to 1905.4 $Wm^{-2}$, and it is about 40%
increment. This is the Kaplan (1981) flux at which an ocean evaporates
entirely. This value achieved at about 3.5 Gyrs as indicated by Sackmann
(1993); Hecht (1994). In our model, the 40% increment of the TSI occurs before
3.5 Gyrs.
At the end of the time in the main sequence, TSI increases to 2994.2 $Wm^{-2}$
at about 5.23 Gyrs from now. As compared to $S_{0}$ it increased by about
120%. Beyond the main sequence, the TSI increases enormously. Figure 1b shows
the peak value of 7.93$\times 10^{6}$ $Wm^{-2}$ at about an age of 7.676 Gyrs
from now. This age is the age at which the Sun approaches the final stage of
RGB (Sackmann, 1993). N. T. Shukure et al. (2021, submitted to APJ) indicated
that the present-day Sun evolves into the white dwarf of mass 0.46 $M_{\odot}$
in the next 8.23 Gyrs. At this age, the dwarf will have the TSI of
-1.13$\times 10^{4}$ $Wm^{-2}$. The negative sign indicates the TSI is below
the present-day value of the Sun.

Figure 1: Evolution of TSI. (a) on the main sequence. (b) beyond the main
sequence produced based on Eq.6; the red line is to indicate before the Sun
loses its hydrogen envelope and cyan line after the Sun loses its hydrogen
envelope.

###  3.2  Earth’s Mean Surface Temperature Variation in Evolutionary Timescale

It is known that the present-day $T_{s}$ is 288 K (Schröde et al.,, 2001;
Mcguffie & Hendersonsellers, 1987). This temperature is expected to change
during the Sun’s evolution. Figure 2a shows the $T_{s}$ variation as the Sun
evolves in the main sequence and Figure 2b for beyond main sequence produced
based on Eq.17. The equation predicts the present-day $T_{s}$ to be 290.16 K.
This value is close to the experimental mean value.
We assumed that $\alpha$ and $\varepsilon$ are decreasing up to the present-
day values of Mars. Our basic assumption is that in the future, $\alpha$ and
$\varepsilon$ of Earth are decreasing. But it is difficult to estimate the
final limit. We select the present-day value of Mars’s $\alpha$ and
$\varepsilon$. This is because the present-day atmosphere of Mars has no right
conditions. The right condition for human life is in the temperature range
between 270 K and 300 K on the Earth. According to the model of Schröde et
al., (2001), the range of temperature ratio and times for which human life
might survive on Mars is 1.29 - 1.43 and 11.6 - 11.7 billion years,
respectively. This indicates that about 7 billion years are needed to warm
Mars. However, human life might be extinct from the Earth before Mars is warm
enough: oceans would evaporate entirely in 3.5 Gyrs from now as estimated by
Sackmann (1993) and in 3.374 Gyrs by N. T. Shukure et al. (2021, submitted to
APJ). Hence, its $\alpha$ and $\varepsilon$ assumed to be decreasing.
Therefore, we select its initial value as our final boundary condition by
assuming that the parameters decrease in the selected time boundary. The Eq.17
predicts that the $T_{s}$ does not show a significant change in the absence of
greenhouse effect for the next 1.644 Gyrs. This shows the influence of TSI in
the evolutionary timescale on $T_{s}$ is not significant although the TSI
increased to 1528.1 $Wm^{-2}$. Including the effluence of human activities and
the greenhouse effect my change the magnitude of temperature in this interval
which results in the Earth losing its water (Kaplan, 1981).
Beyond 1.644 Gyrs from now, the temperature of Earth is observed to be rising
significantly (see Figure 2a). This is perhaps due to the increment of
moisture in the Earth’s atmosphere that induces an extra temperature which
influences the $T_{s}$ (O’Malley-James, 2013). At about 3.374 Gyrs, when TSI
increases to 1905.4 $Wm^{-2}$ (40% increment from present-day value ), part of
this energy absorbed by the Earth’s atmosphere raises the $T_{s}$ to 298.86 K.
According Kaplan (1981), the oceans would evaporate entirely when the TSI
become 1905.4 $Wm^{-2}$. $T_{s}$ rises by 3% as predicted by Eq.17. The
equation also predicts the $T_{s}$ to be 326.72 K in 5.23 Gyr from now when
the Sun completes its main sequence life. This happens when the TSI becomes
2994.2 $Wm^{-2}$ (increases by 120%).
If we consider the assumption of Schröde et al., (2001), that all forms of
life will extinct when $T_{s}$ has reached 380 K. Eq.17 predicts that this
will happen at the age of 7.13 Gyrs (see Figure 2b). This shows our equation
predicts this phenomenon happens 0.1 Gyrs before the Schröder et al.,
prediction. This is because, in our model, we included $\varepsilon$ and
$\alpha$ parameters and let them vary. However, humans will have to leave much
sooner, similar to the present-day situation, perhaps before the model
temperature reaches 291.9 K in 2.2 Gyrs, as our model predicts. Figure 2b also
shows that $T_{s}$ attains a peak value of 2319.2 K, where the ratio is 7.7 as
the Sun evolves to the red giant phase. This value is much greater than the
melting points of rocks and minerals as investigated by Eppelbaum et al.,
(2014). This finding indicates the bad news that our planet Earth will have
the probability of disappearance or have a chance of evolving into the liquid
planet due to extreme temperature increment in the next 7.676 Gyrs during
Sun’s red giant stage. However, beyond this age the good news is since the
Sun’s energy output observed to be decreasing, the $T_{s}$ also decreases and
becomes 311.34 K at the end of 8.23 Gyrs from now. The decrease in the
temperature of the planet may have allowed our planet to revert to a solid
state.

Figure 2:  $T_{s}$ relative to its present temperature which is produced based
on Eq. 17. (a) The predicted variation for the next 5.23 Gyrs of the Sun’s
life, up to the end of the main sequence. (b) On the expansion of the Sun on
the RGB . The maximum ratio is about 7.99.

###  3.3 Earth’s flux variation surface to top of atmosphere

####  3.3.1 Variation in the main sequence

So far we discussed the $T_{s}$ variation due to solar evolution. Since the
magnitude of an energy flux entering into Earth’s atmosphere is expected to
vary with the altitude of our atmosphere, we estimate the energy flux at
different levels by assuming that $T_{s}$ is independent of the atmospheric
altitude.
Figure 3a shows the energy flux variation at different levels of Earth’s
atmosphere. The $\phi_{absorbed}$ and $\phi_{emitted}$ that are produced based
on Eq. 10 and 13, respectively, are equal in the figure. Their present-day
value is 238.41 $Wm^{-2}$ that shows the energy balance of the system. The two
values increase as the Sun evolves in the main sequence, and approach the
values of $\phi_{s}$ whose present-day value is 401.91 $Wm^{-2}$ based on Eq.
11. This value is shown to be greater than $\phi_{absorbed}$ since it is
dependent only on the $T_{s}$ of the Earth. The $\phi_{s}$ does not show
significant change for the next 2.83 Gyrs. If the $T_{s}$ varies according to
the prediction of our model, the life on the Earth’s surface would have a
probability of surviving up to 2.83 Gyrs from now. This result is close to
that of O’Malley-James (2013), which predicts the maximum lifetime for the
life on our planet is 2.8 Gyrs from now. The $\phi_{c}$ is found to be
decreasing from the present-day values of 163.58 $Wm^{-2}$. This is because we
assumed the cloud emissivity to be decreasing until it becomes equal to the
present-day value of Mars. The values of the fluxes at different time
intervals are illustrated in Table 1.

Table 1:  The sample of the impute variables to calculate the TSI and mean Earth’s Surface temperature variation and their sample result.  Age | TSI | $T_{s}$ | $\phi_{absorbed}$ | $\phi_{c}$ | $\phi_{s}$ | $\phi_{emitted}$ | $\alpha$ | $\varepsilon$
---|---|---|---|---|---|---|---|---
$(Gyrs)$ | $(Wm^{-2})$ | $(K)$ | $(Wm^{-2})$ | $(Wm^{-2})$ | $(Wm^{-2})$ | $(Wm^{-2})$ | – | –
4.570 | 1361.908 | 290.160 | 238.334 | 163.578 | 401.912 | 238.334 | 0.300 | 0.407
5.990 | 1497.552 | 289.889 | 265.301 | 135.113 | 400.414 | 265.301 | 0.291 | 0.337
7.944 | 1905.465 | 298.908 | 343.221 | 109.400 | 452.621 | 343.221 | 0.280 | 0.242
9.800 | 2995.261 | 326.615 | 547.963 | 97.287 | 645.251 | 547.963 | 0.268 | 0.150
11.700 | 12089.574 | 452.832 | 2246.597 | 137.543 | 2384.139 | 2246.597 | 0.257 | 0.058
12.246 | 7925489.4 | 2277.903 | 1479360.5 | 47234.825 | 1526595.300 | 1479360.500 | 0.253 | 0.031
12.247 | -34065380.000 | 2319.196 | -6358646.800 | -202694.910 | -6561341.700 | -6358646.800 | 0.253 | 0.031
12.800 | -11308.673 | 311.247 | -2120.376 | -8.088 | -2128.464 | -2120.376 | 0.250 | 0.004

####  3.3.2 Variation beyond the main sequence

The variation of various energy fluxes in the Earth’s atmosphere beyond the
main sequence is shown in Figure 3b. Unlike in the main sequence, all fluxes
except $\phi_{c}$ have almost the same values. For example, at about 5.43
Gyrs, $\phi_{absorbed}$, $\phi_{emitted}$, and $\phi_{s}$ has the values of
590.97 $Wm^{-2}$, 590.97 $Wm^{-2}$, and 687.96 $Wm^{-2}$, respectively.
However, $\phi_{c}$ have the value of 96.99 $Wm^{-2}$. At about 12.25 Gyrs,
when the Sun gets its peak expansion, all the fluxes attain their peak values
as indicated in the figure. Additional values of the fluxes at different time
intervals are shown in the Table 1.

Figure 3: The Earth’s flux variations from surface to top of the atmosphere.
Panel (a) during the main sequence and (b) beyond the main sequence.

##  4 Conclusion

We modeled the TSI variability in the evolutionary timescale. It shows a
significant influence on the Earth’s mean surface temperature. The following
are the conclusions from the above analysis.

  1. 1.

The TSI varies by 6.4% in 1 Gyr increasing from 1361 $Wm^{-2}$ to 1448.1
$Wm^{-2}$

  2. 2.

The TSI increases to 1497.1 $Wm^{-2}$ (by 10%) from a present-day value in
1.42 Gyr from now.

  3. 3.

The TSI increases by 40% at the end of 3.4 Gyrs from now.

  4. 4.

The model also predicts the TSI variability when the Sun is at the end of the
main sequence ton be about 2994.2 $Wm^{-2}$ at about 5.23 Gyrs from now.

  5. 5.

The Earth’s mean surface temperature does not change significantly with a
value of 290.16 K at 1.644 Gyrs from now.

  6. 6.

At about 3.4 Gyrs, when TSI increases to 1905.4 $Wm^{-2}$, the Earth’s surface
temperature becomes 298.86 K.

  7. 7.

The maximum Earth’s temperature is predicted when the Sun becomes an RGB to be
2319.2 K turning Earth into a liquid planet.

  8. 8.

The absorbed flux and emitted flux observed to be increasing equally and
approach the surface flux at the end of the main sequence, while the flux
absorbed by clouds is found to be decreasing.

  9. 9.

Beyond the main sequence, the surface flux, absorbed flux and emitted flux
become nearly equal attaining a peak value at about 7.676 Gyrs from now.
However, the flux absorbed by the clouds decreases relative to others,
although it peaks at the same points as other fluxes.

## Acknowledgments

N.T.S thanks Entoto Observatory and Research Center (EORC) for hosting his
research. This research was partially supported by the International Science
Program (ISP) under the East African Astrophysics Research Network (grant code
IPPS/AFRO5). N.T.S also thanks Dilla University for covering the living
expenses in Ethiopia NG is supported by NASA’s Living With a Star Program.

## References

  * Bahcall et al., (1995) Bahcall, J. N., Pinsonneault, M., & Wasserburg, G. 1995, Reviews of Modern Physics, 67, 781,
doi: 10.1103/revmodphys.67.781

  * Dwivedi (2010) Dwivedi, B. 2010.
$http://nopr.niscair.res.in/handle/123456789/10583$

  * Eddy (1976) Eddy, J. A. 1976, Science, 192, 1189,
doi: 10.1126/science.192.4245.1189

  * Eppelbaum et al., (2014) Eppelbaum, L., Kutasov, I., & Pilchin, A. 2014, in Applied geothermics (Springer), 99–149.
${https://doi.org/10.1007/978-3-642-34023-92}$

  * Ermolli et al., (2013) Ermolli, I., Matthes, K., Dudok de Wit, T., et al. 2013, Atmospheric Chemistry and Physics, 13, 3945. https://doi.org/10.5194/acp-13-3945-2013
  * Feulner (2012) Feulner, G. 2012, Reviews of Geophysics, 50.
https://doi.org/10.1029/2011RG000375

  * Fröhlich (2006) Fröhlich, C. 2006, Solar Variability and Planetary Climates, 53\.
https://doi.org/10.1007/978-0-387-48341-2 5

  * Gray et al., (2010) Gray, L. J., Beer, J., Geller, M., et al. 2010, Reviews of Geophysics, 48.
https://doi.org/10.1029/2009RG000282

  * Haigh (2007) Haigh, J. D. 2007, Living reviews in solar physics, 4, 1.
https://doi.org/10.12942/lrsp-2007-2

  * Hecht (1994) Hecht, J. 1994, Fiery future for planet Earth, NEW SCIENTIST PUBL EXPEDITING INC 200 MEACHAM AVE, ELMONT, NY 11003.
https://www.newscientist.com/article/
mg14219191900sciencefieryfutureforplanetearth/

  * Ineson (2011) Ineson, S., Scaife, A. A., Knight, J. R., et al. 2011, Nature Geoscience, 4, 753.
https://doi.org/10.1038/ngeo1282

  * Kaplan & Union (1981) Kaplan, G., & Union, I. A. 1981, The IAU Resolutions on Astronomical Constants, Time Scales, and the Fundamental Reference Frame, United States Naval Observatory circular (U.S. Naval Observatory).
https://books.google.com.et/books?id=8UnHaahPyicC

  * Kaplan (1981) Kasting, J. F. 1988, Icarus, 74, 472.
https://doi.org/10.1016/0019-1035(88)90116-9

  * Kopp (2016) Kopp, G., 2016, Journal of space weather and space climate, 6, A30. doi:10.1051/swsc/2016025
  * Kopp & Lean (2011) Kopp, G., & Lean, J. L. 2011, Geophysical Research Letters, 38.
https://doi.org/10.1029/2010GL045777

  * Kren (2015) Kren, A. C. 2015, PhD thesis, University of Colorado at Boulder.
https://ui.adsabs.harvard.edu/abs/2015PhDT……..78K

  * Kren et al., (2017) Kren, A. C., Pilewskie, P., & Coddington, O. 2017, Journal of Space Weather and Space Climate, 7, A10,
doi: 10.1051/swsc/2017007

  * Lean & Rind (2008) Lean, J. L., & Rind, D. H. 2008, Geophysical Research Letters, 35.
https://doi.org/10.1029/2008GL034864

  * Mcguffie & Hendersonsellers (1987) Mcguffie, K., & Hendersonsellers, A. 1987,
doi: 10.5860/choice.35-2776

  * O’Malley-James (2013) O’Malley-James, J. T., Greaves, J. S., Raven, J. A., & Cockell, C. S. 2013, International Journal of Astrobiology, 12, 99.
https://doi.org/10.1017/S147355041200047X

  * Langtangen (2016) petter Langtangen, H. 2016, Finite difference computing with exponential decay models (Springer Nature),
doi: 10.1007/978-3-319-29439-1

  * Sackmann (1993) Sackmann, I., Boothroyd, A. I., Kraemer, K. E., et al. 1993, The Astrophysical Journal, 418, 457.
http://adsabs.harvard.edu/abs/1993ApJ…418..457S

  * Schmutz et al., (2013) Schmutz, W., Fehlmann, A., Finsterle, W., Kopp, G., & Thuillier, G. 2013in , American Institute of Physics,
624–627. https://doi.org/10.1063/1.4804847

  * Schröde et al., (2001) Schröder, P., Smith, R., & Apps, K. 2001, Astronomy & Geophysics, 42, 6.
https://doi.org/10.1046/j.1468-4004.2001.42626.x

  * Shukure et al., (2019) Tilahun Shukure, N., Tessema, S., & Mengistu, E. (2019). Mass-loss varying luminosity and its implication to the solar evolution. Proceedings of the International Astronomical Union, 15(S356), 403-404. doi:10.1017/S1743921320003580
  * Solanki et al., (2013) Solanki, S. K., Krivova, N. A., & Haigh, J. D. 2013, Annual Review of Astronomy and Astrophysics, 51.
https://doi.org/10.1146/annurev-astro-082812-141007

  * Strang & Herman (2021) Strang, G., & Herman, E. . 2021, Exponential Growth and Decay, Massachusetts Institute of Technology (Strang) & University of Wisconsin-Stevens Point (Herman).
https://math.libretexts.org/@go/page/2526

  * Swift (2013) Swift, L. 2018, Climate, 6, 52.
https://doi.org/10.3390/cli6020052


#  TransfoRhythm: A Transformer Architecture Conductive to Blood Pressure
Estimation via Solo PPG Signal Capturing

Amir Arjomand  Department of Computer Science, University of New Brunswick,
Fredericton, Canada  Amin Boudesh  Concordia Institute for Information Systems
Engineering, Concordia University, Montreal, QC, Canada  Farnoush Bayatmakou
Concordia Institute for Information Systems Engineering, Concordia University,
Montreal, QC, Canada  Kenneth B. Kent  Department of Computer Science,
University of New Brunswick, Fredericton, Canada  Arash Mohammadi  Concordia
Institute for Information Systems Engineering, Concordia University, Montreal,
QC, Canada

###### Abstract

Recent statistics indicate that approximately $1.3$ billion individuals
worldwide suffer from hypertension, a leading cause of premature death
globally. Blood pressure (BP) serves as a critical health indicator for
accurate and timely diagnosis and/or treatment of hypertension. Traditional BP
measurement methods rely on cuff-based approaches, which lack real-time,
continuous, and reliable BP estimates, crucial for timely diagnosis/treatment
of hypertension. Driven by recent advancements in Artificial Intelligence (AI)
and Deep Neural Networks (DNNs), there has been a surge of interest in
developing data-driven and cuff-less BP estimation solutions. In this context,
current literature predominantly focuses on coupling Electrocardiography (ECG)
and Photoplethysmography (PPG) sensors, though this approach is constrained by
reliance on multiple sensor types. An alternative, utilizing standalone PPG
signals, presents challenges due to the absence of auxiliary sensors (ECG),
requiring the use of morphological features while addressing motion artifacts
and high-frequency noise. To address these issues, the paper introduces the
TransfoRhythm framework, a Transformer-based DNN architecture built upon the
recently released physiological database, MIMIC-IV. Leveraging Multi-Head
Attention (MHA) mechanism, TransfoRhythm identifies dependencies and
similarities across data segments, forming a robust framework for cuff-less BP
estimation solely using PPG signals. To our knowledge, this paper represents
the first study to apply the MIMIC IV dataset for cuff-less BP estimation, and
TransfoRhythm is the first MHA-based model trained via MIMIC IV for BP
prediction. Performance evaluation through comprehensive experiments
demonstrates TransfoRhythm’s superiority over its state-of-the-art
counterparts. Specifically, TransfoRhythm achieves highly accurate results
with Root Mean Square Error (RMSE) of [$1.84,1.42$] and Mean Absolute Error
(MAE) of [$1.50,1.17$] for systolic and diastolic blood pressures,
respectively.

##  1 Introduction

High blood pressure is a serious symptom of hypertensive disease that can lead
to disability and even death [1]. Based on an American Heart Association
estimation, about $41.4$% of US people will be struggling with hypertension by
$2030$ [2]. There are various approaches to measure Blood Pressure (BP), among
which, cuff-based measurement [3] (where a cuff is employed to temporarily
stop the blood flow to obtain the extremum BP values) is the common
traditional method used both in hospitals and for in-home care. Such an
approach, however, carries the following drawbacks: (i) Cuff devices are not
readily available in low-resource settings, leading to the lack of awareness
and control of hypertensive conditions; (ii) Repeated cuff inflation and
deflation can disrupt gold-standard hypertension diagnosis and hypotension
surveillance, resulting in under-utilization and infrequent measurements, and;
(iii) Widely used cuff-based devices do not provide continuous BP measurement,
limiting immediate detection of hypotension and real-time therapy titration
[4]. Alternative methods of BP measurement, such as cuff-less monitoring, have
been explored to provide continuous monitoring and reduce the potential for
measurement errors caused by cuff-based methods. Cuffless BP measurement can
facilitate regular monitoring, enable long-term control, and provide precise
evaluation and diagnosis of hypertension. Additionally, it can allow for
hypotension management and therapy through seamless continuous monitoring [4].
Another advantage of continuous BP monitoring, which is provided by cuffless
methods, is the potential to monitor nocturnal BP patterns. Nocturnal blood
pressure can be a more informative risk factor compared to daytime blood
pressure in predicting the incidence of Cardiovascular Disease (CVD). Both
hypertensive and general populations are at risk of developing CVD if there is
a significant surge or dip (more than $10-20$%) in BP during nighttime, which
may result in stroke, heart failure, and even death [5]. Cuffless approaches
use indirect methods such as pulse wave velocity, pulse transit time, pulse
wave analysis, volume clamping, and applanation tonometry to estimate blood
pressure [6].

The history of cuff-less BP estimation arises from arterial stiffness
measurement through Pulse Wave Velocity (PWV) [7, 8, 9, 10]. In $1981$, Geddes
and Voelz tried, for the first time, to analyze Pulse Transit Time (PTT) and
BP on ten dogs and reported a meaningful correlation between PTT and diastolic
BP [11]. Recent cuffless measurement techniques, typically, employ biological
signals such as Photoplethysmography (PPG), Ballistocardiography (BCG), and
Electrocardiography (ECG) to estimate BP. PPG is obtained based on the
principle that the absorption of light by blood changes as it passes through
the blood vessels. A PPG sensor placed on the skin measures the changes in
light absorption caused by the pulsatile blood flow, which is then used to
estimate BP. BCG is obtained based on the principle that the heart’s
contraction generates a force that is transmitted through the body, causing a
measurable movement. A BCG sensor placed on a person’s seat measures the
movements caused by the heart’s contraction and can be used for BP estimation.
ECG measures heart’s electrical activities, which can be used to derive
various parameters related to the heart’s function, including BP [12].
Capitalizing on recent advancements of Machine Learning (ML) models, there has
been a surge of interest in ML-based cuff-less measurement of BP from ECG and
PPG signals. Generally speaking, ML models, in particular, Deep Learning (DL)
architectures, showed significant potentials in extracting discriminative
features from ECG/PPG signals, estimating volumetric change of blood flow, and
predicting relative systolic and diastolic BP, referred to as SBP and DBP. For
instance, Baker et. al [13] developed an integrated Convolutional Neural
Network (CNN) and Long Short-Term Memory (LSTM) model, fed by PPG and ECG
signals. The model was trained based on data from $84$ individuals. The
reported Mean Absolute Error (MAE) values are $4.41$ and $2.91$ for the SBP
and DBP. Another relevant study [1] developed a Temporal Convolutional Network
(TCN) architecture using multi-lead ECG and PPG sensors. The model is trained
based on data of $293$ individuals obtained from the MIMIC [I, III] datasets,
and showed compelling results. The Root Mean Square Error (RMSE) associated
with the SBP is reported as $3.03$, while that of the DBP is reported as
$1.58$. Huang et. al [14] proposed a modified architecture of MLP-Mixer [15]
with a novel pre-processing technique (referred to as the MFMC). The resulting
SBP-RMSE is $5.10$ and DBP-RMSE is $3.13$ on $3,000$ records of the MIMIC II
[16] dataset.

Although BP estimation via PPG and ECG sensors is promising, it suffers from
some critical drawbacks such as reliance on two types of sensors, high power
consumption, need for sensor re-calibration, and dependence on keeping a fixed
distance between sensors. These issues can be handled by relying on stand-
alone PPG signals captured from the skin surface. The PPG technique is an
optical solution designed to measure volumetric variations of blood
circulation and can provide invaluable information about the underlying
cardiovascular system. Its principal mechanism is based on the emission and
reflection of light in the near-infrared wavelength (800nm $\sim$ 2500nm) from
arteries’ blood flow. Due to the easement use of PPG sensors and its potential
for BP estimation, it has been extensively utilized for development of
advanced processing/learning models [17, 18, 19, 20]. In this context, when it
comes to feature extraction from PPG signals, two types of data
representations are commonly used (as the appropriate feed for downstream
processing/learning modules), i.e., raw PPG signals (time series input) [14,
21, 13, 22], and feature-based vectors [23, 24, 25, 26, 27]. Such
representations can be extracted from the original PPG signal, multiple-order
signal derivative, and Pulse Wave Decomposition (PWD) of the PPG. In the
modeling step, to achieve accurate BP estimation, recent studies have employed
various methodologies ranging from traditional ML models, such as Support
Vector Regression (SVR) and Random Forest [27, 28, 29], to Deep Neural Network
(DNN) architectures [1, 30] such as LSTM networks, and Gated Recurrent Unit
(GRU) models with attention mechanisms [22, 26, 31]. Despite these studies,
standalone PPG-based BP estimation is still a critically challenging task as
by removing the auxiliary sensor (ECG), one needs to particularly consider
morphological features, and pay attention to vulnerability to motion artifacts
and high-frequency noise [32].

To address the above mentioned challenges, the paper introduces the
TransfoRhythm framework, an attention-based DNN architecture built upon the
recently released physiological database, MIMIC-IV dataset [33]. To our
knowledge, the paper represents the first study to apply the MIMIC IV dataset
for BP prediction. At the core of the TransfoRhythm framework is a Multi-Head
Attention (MHA) mechanism, making it the first MHA-based DNN model trained via
MIMIC IV for the application at hand. More specifically, to the best of our
knowledge, only handful of recent studies [37, 34, 35] tapped into potentials
of attention-mechanism for BP estimation via PPG signals. In brief, Reference
[35] developed a MHA model using the UCL dataset [36]. This study implemented
two different models, a transformer-based architecture, and a Frequency-Domain
Linear/Non-linear Regression model (FD-L/NL), with the latter providing better
performance. Similarly, Reference [37] proposed a transformer-based model
refereed to as AriaNet, while authors in Reference [34] focused on self-
supervised transfer learning based on transformers. Different from these
recent studies, the TransfoRhythm is built on the regressive time series
transformer network, which is crafted to learn from time series data while
preserving sequence order and allowing parallel computation during training
and inference. To expand the feature vectors’ dimensionality, an embedding
feature, similar to those used in Natural Language Processing (NLP) tasks, is
employed. Positional encoding is then added to the extracted features to
create the input data matrix, which is fed into a model block via the MHA
mechanism. The output is refined in a position-wise block, which is then
processed by time compressor and flattening blocks to shorten the time frame
and calculate estimated systolic and diastolic BP values. The obtained results
validate the effectiveness of such a modeling framework and highlight its
potential for clinical implementation.

##  2 Methodology

Figure 1: This block diagram depicts the streamlined process for estimating
systolic and diastolic blood pressures. Features are extracted from the
dataset (top left), combined with positional encoding, and form an embedded
input matrix. This matrix is then inputted into the model block, which
integrates a multi-head attention mechanism. The output of the model is
further refined in the position-wise block. Finally, the Time compressor and
flattening blocks are applied to reduce the time frame and yield the estimated
systolic and diastolic blood pressure values.

This section focuses on material and methodology, whereas in the first part
(the dataset formation subsection) we describe the preprocessing and feature
extraction approaches. In the second part, the network architecture subsection
provides an overview of the utilized modeling process. In brief and according
to the workflow of our study, shown in Fig. 1, after dataset formation and
preprocessing, the overall dataset is compiled based on $12$ final features.
Next, the positional encoding values are added to the flattened values of the
dataset, resulting in the construction of the embedded input matrix, which is
used to feed the model. In the model part, a multi-head attention algorithm is
employed. The position-wise process is applied to the output of the multi-head
attention block and finally, by reducing the time frame and flattening the
position-wise output, the values related to systolic and diastolic blood
pressures are obtained.

###  2.1 Dataset Formation

Figure 2: Time distribution of records depicted in the graph. The $x-axis$
represents the length of records, ranging from a few minutes to a maximum of
17 hours. The darker shades of blue indicate a higher number of observations.
The majority of patients have records with a time duration of less than 1.5
hours.

The Medical Information Mart for Intensive Care (MIMIC)-IV Waveform Dataset is
employed, which is available on the Physio-Net repository [38] to analyze the
version of waveform recordings from bedside monitoring devices in modern
Intensive Care Units (ICUs). MIMIC-IV includes ECG, PPG, and invasive ABP
signals and is the latest and most recently released Dataset of its kind
following a joint effort from Beth Israel Deaconess Medical Center (BIDMC) and
Massachusetts Institute of Technology (MIT). The recording process was
conducted at a sampling rate of $62.4$ Hz. To ensure patient confidentiality,
all $198$ individuals in the dataset were completely de-identified. Fig. 2
illustrates the duration of the recorded signals and their corresponding
iterations. The majority of patients were monitored for less than $1.5$ hours,
while some patients had multiple recordings lasting over $15$ hours. Although
previous studies have employed MIMIC-II and MIMIC-III datasets for blood
pressure estimation, to the best of our knowledge, this study stands out as
the first utilization of the released MIMIC-IV v2.0 dataset for this purpose.

####  2.1.1 Preprocessing

Figure 3: A comparison of the signal before and after preprocessing. The photo
demonstrates the exceptional noise filtering achieved, resulting in a clean
and smooth signal. Furthermore, the peaks and feet are accurately preserved,
showcasing the effectiveness of the preprocessing techniques employed.

The preprocessing stage is a crucial step to enhance the quality of the
underlying signals and remove potential noise and/or artifact sources that may
have been introduced during data acquisition. In this study, in the
preprocessing phase, data cleaning and signal filtering were performed. In the
data cleaning step, signals with frequent outrange amplitudes and records that
did not meet the minimum time duration (less than 15 minutes) as well as
certain frames of long records that contained disqualified signals, such as
flattened-line or extremely noisy segments were detected, and removed from the
records. After the data cleaning step, a total of $360$ purified records were
identified. In the second step of the preprocessing phase, i.e., the signal
filtering step, a $5^{\text{th}}$ order Butterworth filter in the frequency
range of $0.7$ to $10$ Hz and a $5^{\text{th}}$ order Moving Average Filter
(MAF) were applied to remove overshoots and fluctuations. More specifically,
the Butterworth Infinite Impulse Response (IIR) bandpass filter was employed
to remove wandering baseline and high-frequency noise, such as power-line
interference ($50$ $60$ Hz). The MAF involves taking the average of a subset
of data points and using it to smooth out the underlying signals. The
combination of the Butterworth filter and MAF resulted in a highly filtered
and smooth PPG signal. Some statistics associated with the result of the
preprocessing phase are shown in Fig. 3.

####  2.1.2 Feature Extraction

The first step in the feature extraction phase involves identifying and
separating the cycles of each frame to obtain meaningful data. In this regard,
we employ a robust algorithm based on a one-dimensional array that locates
peak and foot (valley) points of the cycles in the PPG and the corresponding
second derivative of the PPG (SDPPG) signal inside each frame. This algorithm
is parameterized by an adjustable peak-to-peak distance, which allows the
detection of varying heights and amplitudes of the peaks. Subsequently,
according to these identified foot and peak points, PPG and its corresponding
SDPPG cycles were extracted. Additionally, before starting feature extraction,
each frame was evaluated to be used or to be skipped based on an adaptive
threshold for peak detection, which sets the threshold value based on the
average amplitude of the signal. This approach ensures that the algorithm is
sensitive enough to detect peaks of varying heights and amplitudes, while also
being robust to noise and other signal distortions. Based on the correlation
matrix between the features, a set of $12$ features was selected. The
definition and description of these incorporated features are shown in Table
1, furthermore, visual illustrations of PPG and SDPPG signals representing
selected features’ specifications are shown in Fig. 4. Furthermore, to address
potential bias and ensure a well-balanced dataset for detecting hypertension
and hypotension events, a dynamic random frame generator is utilized. This
generator automatically increases the number of observations with systolic
blood pressure (SBP) exceeding 140 mmHg or falling below 80 mmHg. This step is
crucial in achieving dataset balance and preventing overfitting of the model.

Figure 4: Characterization of features on the signal. In the PPG signal, the peak is defined as the maximum value of a PPG wave, while d.notch is defined as the minimum value between two consecutive peaks. In the SD PPG signal, amp peak refers to the maximum amplitude, while amp foot refers to the minimum amplitude between two consecutive waves. Table 1: Definition of the selected features. Feature | Symbol | Description
---|---|---
PPG_Cycle_Duration | TD1 = Tend - Tstart | Total Duration
PPG_Rise_Half_Peak | Trhp | The duration between start to the half peak amplitude (rise mode)
PPG _Peak To_ Notch | TD2 = Tdn-Tp | The duration between peak to dicrotic notch
PPG_Rise_Peak | Tp | The duration between the start to the peak
PPG_Fall_Half | Tfh | The duration between peak to half of its amplitude (fall mode)
PPG_Fall_Peak | TD3 = Tend - Tp | The duration between peak to foot (end)
PPG_BPM_Frame | PBF | Cycles number per frame
PPG_ Integration | PPGI = (Green Area) | Under Curve(cycle) Area
SDPPG_Fall_Half_Foot | SDPPG_Tfhf | The duration between the start to the half of its amplitude (fall mode)
SDPPG_Extermum_Amp | SDPPG_AMP | SDPPG Maximum Amplitude
SDPPG_ Integration | SDPPGI = (Blue Area) | SDPPG Upper Curve Area
SDPPG_Foot_Peak | TD4 = Tpeak - Tfoot | SDPPG Duration between Foot to Peak

###  2.2 Network Architecture

The proposed TransfoRhythm framework is developed based on the regressive time
series transformer network, which is designed to learn from time series data
while maintaining sequence order and enabling parallel computation during the
training and inference phases. To enhance the dimensionality of the feature
vectors, an embedding feature, previously implemented in Natural Language
Processing (NLP) tasks [39] is utilized. The initial structure of the features
is defined by Stacked Frames (SF) based on shuffled frames, with $20,000$
frames ($N_{i}$) and a time frame sequence of $48$, which is given by

$SF=\left[F_{T}^{L,1},F_{T}^{L,2},\dots,F_{T}^{L,n}\right],$ (1)

where $n$ refers to the number of frames, $F_{T}^{L,i}$ represents an
individual frame that spans a shuffled frame with the input feature length of
$L$ ($L_{in}=12$). Furthermore, $T$ refers to the time frame sequence and is
equal to $48$. To extract the extended features, a $1$D convolution layer is
applied to the stacked frames as follows

$ST_{Extended}(N_{i},L_{out})=Bias(L_{out})+\sum\limits_{k=0}^{L_{in}}weight(L_{out},k)input(N_{i},k),$
(2)

where $ST_{Extended}\in\mathbb{R}^{N_{i}\times L_{out}\times T}$,
$weight\in\mathbb{R}^{L_{out}\times L_{in}\times KS}$, and
$input\in\mathbb{R}^{N_{i}\times L_{in}\times T}$. Eq. 2 expands the feature
length from $12$ ($L_{in}$) to 128 ($L_{out}$) with a kernel size ($KS$) of 1.

####  2.2.1 Multi-Head Attention Mechanism

The self-attention mechanism is employed to capture the sequence dependencies
and features carried on each frame. This mechanism employs three main
matrices, i.e., Key, Queue, and Value, which are obtained by multiplying the
input embedding matrix with learnable weights $W_{K}$, $W_{Q}$, and $W_{V}$,
respectively. The resulting matrices are then used to compute a set of
attention scores. To capture additional information about in-frame sequence
dependency, multiple learnable sets (Heads) are stacked and concatenated, in
the current study, we utilize a multi-head attention mechanism with $14$ heads
for BP estimation. Additionally, the model incorporates a linear layer in the
last section of multi-head attention to introduce non-linearity. This approach
enables the model to capture complex relationships and dependencies among
input features, resulting in better BP predictions.

Despite the self-attention mechanism being a key component of the transformer
architecture, it does not incorporate information about the relative positions
of tokens in the input sequence. To address this limitation, a technique
called Positional Encoding (PE) was introduced that encodes the position of
each element in the input sequence by adding a sinusoidal function of
different frequencies to the input embedding [32]. The PE array is added, in
an element-wise fashion, to the $ST_{Extended}$ array, resulting in an
Embedded input with a feature length of $48$ and a time frame sequence length
of $128$. The incorporated approach enables the transformer architecture to
capture both the content and positional information of the input sequence.

Position-Wise Feed-Forward Neural Network: The next component of the attention
module is the position-wise feed-forward layer that employs a $1$D convolution
layer with a ReLU activation function to help the model learn better
representations for each position in the input sequence. This layer enables
the model to capture context-specific information and understand the
relationship between different positions in the sequence [40]. Furthermore,
generalization techniques such as the normalization layer and dropouts are
implemented to improve the model’s performance and avoid overfitting.

Time Frame Compressor and Flattening: In the initial stages of
experimentation, we encountered computational challenges related to the
extensive computational overhead on local hardware. We identified that the
final flattening layer, responsible for converting the high-dimensional output
into a one-dimensional structure, posed a bottleneck. To overcome this, we
employed a Time Frame Compressor unit that effectively reduces dimensionality
while preserving temporal signals. Specifically, this non-learnable
transformer shortens the temporal order into a more condensed representation.
Thereafter, in the last stage of the model, we employ a flattened layer with
non-learnable weights to transform the temporally compressed signals to
systolic and diastolic blood pressure values through a ReLU function.

###  2.3 Evaluation Metrics

To evaluate the effectiveness of the proposed TransfoRhythm framework, we have
identified three key evaluation metrics, i.e., R-squared ($R^{2}$), Mean
Absolute Error ($MAE$), and Root Mean Square Error ($RMSE$), given by

$\displaystyle R^{2}$
$\displaystyle=\frac{\sum\limits_{i=1}^{n}\left(T_{i}-P\right)^{2}}{\sum\limits_{i=1}^{m}\left(T_{i}-\bar{T}\right)^{2}},$
(3) $\displaystyle MAE$
$\displaystyle=\frac{1}{n}\sum\limits_{i=1}^{n}|T_{i}-P_{i}|,$ (4)
$\displaystyle RMSE$
$\displaystyle=\sqrt{\frac{1}{n}\sum\limits_{i=1}^{n}(T_{i}-P_{i})^{2}}.$ (5)

where $T_{i}$ refers to the target and $P_{i}$ is the predicted value. In
brief, Metric $R^{2}$ serves as an indicator of the extent to which the
variance in the dependent variable can be attributed to the independent
variables. The other two metrics ($MAE$ and $RMSE$) quantify the magnitude
and/or percentage of errors between actual and predicted values.

##  3 Results

Table 2: Values of the considered hyperparameters  Hyperparameters | Type and Value
---|---
Dropout | 0.15
Optimization Algorithm | Adam
Learning Rate | 1E-4 (with dynamic wight decay)
Activation Function | ReLU
Loss Function | MSE
Batch Size | 128
Epoch | 400

To evaluate different aspects and potentials of the proposed TransfoRhythm
framework, we conducted a series of experiments involving various
hyperparameters, as well as modifications to the model architecture. The final
set of hyperparameters used in our experiments is presented in Table 2. More
specifically, to evaluate performance of the proposed model, we employed a
5-fold cross-validation approach. This involved dividing the data into five
subsets, with $80\%$ allocated for training and $20\%$ for the validation
dataset. The utilization of the cross-validation method provided validation on
the entire dataset and effectively mitigated the risk of overfitting, ensuring
the model’s generalizability and reliability. The proposed TransfoRhythm
framework was subjected to $500$ epochs of training for each fold on a high-
end local server equipped with an Intel $12700$ CPU, $16$ gigabytes of RAM,
and an Nvidia $3060$ GPU with $12$ gigabytes of VRAM. Although the final model
was selected based on the best metric of the cross-validation process, the
reported results were obtained by calculating the mean values of the
performance metrics across the five folds. This highlights the model’s
capability to generalize its performance across the entire dataset. We
obtained mean $R^{2}$ values of $[0.991,0.991]$, mean MAE values of
$[1.50,1.17]$, and mean RMSE values of $[1.84,1.42]$ for the SBP and DBP,
respectively. Further details can be found in Table 5.

###  3.1 Validation against AAMI and BHS Standards

Table 3: Evaluation of the Proposed Model with AAMI and BHS Standards. |  | Records | Mean Error (mmHg) | STD (mmHg) | Cumulative error percentage
---|---|---|---|---|---
|  | <5 mmHg | <10 mmHg | <15 mmHg
TransfoRhythm Architecture | SBP | 360 | 0.138 | 1.93 | 98% | 100% | 100%
| DBP | 360 | -0.166 | 1.58 | 98% | 100% | 100%
AAMI Standard Requirement |  | >85 | <5 | <8 | - | - | -
BHS Standard Requirement | Grade A | - | - | - | 60% | 85% | 95%
Grade B | - | - | - | 50% | 75% | 90%
Grade C | - | - | - | 40% | 65% | 85%

To ensure that the proposed TransfoRhythm architecture meets the international
standards for medical device regulation, we evaluated our findings using the
standards of the Association for the Advancement of Medical Instrumentation
(AAMI) and The British Hypertension Society (BHS). To comply with AAMI
standards, certain statistical conditions must be met, including a sample size
of at least $85$ individuals aged $12$ years or older, a Mean Error (ME) of
less than $5$ mmHg, and a Standard Deviation (SD) of the error less than $8$
mmHg. Additionally, BHS grades accuracy based on the cumulative percentage of
error. Table 3 provides a comparison between our results and these standard
metrics. As presented in Table 3 the proposed model in this study has
successfully met the AAMI standard. Moreover, the model has also achieved
“Grade A” in the BHS standard. These results confirm the reliability and
outstanding performance of the proposed model based on the prediction error
values.

###  3.2 Error Investigation

Figure 5: Comparison of true values and prediction. The blue dots represent
the predicted values, while the gold line represents the optimal prediction.
(a) shows the results for SBP, and; (b) shows the results for DBP.

Comparison between the predicted and true values is an essential step in
evaluating the performance of the TransfoRhythm model as it provides insight
into how well the model can be generalized to unseen data and also helps to
identify the functional range of the model. Fig. 5 represents a comparison of
true and predicted SBP and DBP values. Fig. 5 effectively showcases the
model’s ability to estimate these blood pressure measurements and provides a
clear understanding of its predictive accuracy.

###  3.3 Statistical Assessment via Bland-Altman

Figure 6: Bland–Altman plots for the SBP and DBP. The Bland-Altman plot
illustrates the agreement between true and predicted values. The plot depicts
the $95\%$ statistical limits of agreement, corresponding to 1.96 standard
deviations (SD) of the mean. The mean difference or bias observed is 0.12 and
-0.31 for SBP and DBP, respectively.

The Bland-Altman diagram is a statistical method for assessing the agreement
between two measurement methods by plotting their difference from their mean.
In Fig. 6, we present the Bland-Altman plot for the TransfoRhythm model. The
plot shows a mean difference of $0.12$ mmHg with a $95\%$ limit of agreement
ranging from $-3.21$ to $3.46$ mmHg for SBP and a mean difference of $-0.31$
mmHg with a $95\%$ limit of agreement ranging from $-5.37$ to $4.74$ mmHg for
DBP.

###  3.4 Comparisons with Benchmark Models

This section focuses on validating our results by comparing the performance of
our proposed model with benchmark algorithms from previous studies. Given the
lack of prior work utilizing the MIMIC IV dataset for BP prediction, we
implemented benchmark models from other studies that analyzed earlier versions
of the MIMIC dataset. These benchmark models are employed to establish a
baseline performance for blood pressure estimation in the context of the MIMIC
IV dataset. The selection of benchmark models was based on specific criteria.
Firstly, we considered models that exclusively utilized PPG signals.
Furthermore, due to the impracticality of conducting extensive feature
engineering for each model, we specifically chose models that were trained on
raw data. Additionally, we selected models with open-access codes with
satisfying results in their respective papers. The benchmark models enclosed
various architecture families. From CNN-based networks, we selected ResNet1D
and AlexNet1D, which were originally implemented and evaluated on MIMIC II and
MIMIC-III datasets [41]. Additionally, we included a U-Net model that had been
evaluated on the MIMIC III dataset [42]. In terms of Recurrent Neural Networks
(RNN), we defined a BiLSTM model for our benchmark set, which aimed to address
the absence of suitable RNN-based models meeting our selection criteria. The
BiLSTM model begins with a 1D Convolutional layer for feature extraction,
followed by a Max pooling layer to enhance pattern detection. Four
Bidirectional LSTM layers then capture contextual understanding by considering
past and future states. To prevent overfitting, four Dropout layers are placed
after the LSTM layers. Finally, two dense layers are used to perform the final
regression task and map features to the desired output format. By containing
the BiLSTM model, we aimed to gain insights into RNN-based approaches for
blood pressure prediction and expand the scope of evaluation by comparing our
model performance with other benchmark models. Lastly, we incorporated a
hybrid architecture combining CNN and RNN layers, which utilized the raw PPG
signal along with its first and second derivatives of the MIMIC III dataset as
input features [43]. It is essential to note that the pre-processing methods
employed in the benchmark studies were customized for their specific MIMIC
dataset version. As the structure of the MIMIC IV dataset differs from the
earlier versions, we utilized our pre-processing pipelines. Additionally,
since the referenced studies employed a $10-second$ window size, we maintained
consistency by adopting the same window length for the data.

Table 4 presents the quantitative evaluation metrics obtained by implementing
the benchmark models on the MIMIC IV dataset. Each benchmark model was trained
using different sets of hyperparameters, and the most accurate model was
identified and reported. The results demonstrate the superior performance of
our proposed model compared to the benchmark algorithms across all the
evaluation metrics. This notable performance confirms the efficacy of our
approach in accurately estimating blood pressure solely from PPG signals.

Table 4: Comparison of the proposed model with benchmark models on Mimic IV dataset Architecture Type |  Model | MAE SBP | MAE DBP | RMSE SBP | RMSE DBP
---|---|---|---|---|---
CNN |  ResNet_1D | 2.28 | 1.06 | 5.55 | 3.21
AlexNet_1D | 3.09 | 1.59 | 7.98 | 4.60
U-Net | 2.76 | 1.56 | 7.47 | 4.70
RNN |  Bi-LSTM | 3.86 | 1.9 | 7.98 | 4.19
Hybrid |  Hybrid CNN-RNN Network | 2.79 | 1.96 | 5.62 | 3.91
MHA |  TransfoRhythm | 1.50 | 1.17 | 1.84 | 1.42

##  4 Discussion and Conclusion

Long-term BP measurement could not be carried out using cuff-based solutions,
because of its discrete nature and the inaccuracy caused by factors such as
the cuff size, placement, and the patient’s movement. Consequently, there has
been a surge of recent interest in using DL techniques to estimate BP based on
ECG and/or PPG data [44]. In this context, the TransfoRhythm framework is
developed in this study for continuous BP estimation. The TransfoRhythm takes
as input $12$ features extracted from the MIMIC IV raw and derivatives PPG
waveforms. It is worth noting that despite many studies on the MIMIC datasets,
to the best of our knowledge, this work is the first investigation of BP
prediction implemented on the MIMIC IV waveform version dataset. At the core
of the TransfoRhythm framework is a Multi-Head Attention (MHA)-based
mechanism, which is a neural network architecture with several advantages over
its traditional DL methods for time series analysis [45]. Several studies have
shown that MHA can outperform conventional DL methods in various time series
prediction tasks. For example, MHA-based networks have been shown to
outperform RNNs in predicting fault diagnosis [46], Energy Consumption
assessment [47], and speech enhancement [48]. Additionally, MHA has been shown
to improve prediction accuracy in stock price prediction [49], and traffic
flow prediction [50]. When it comes to BP estimation, conventional DL methods
are, typically, developed based on RNN and/or CNN architectures. While these
methods have shown promising results, they have limitations, such as
difficulty in capturing long-term dependencies or invariance to temporal
shifts in the input [51] [48]. The primary advantage of MHA is its ability to
attend to various input sequence segments, enabling the model to learn
complicated relationships between various observations in the sequence and
improve representations of the input. It, therefore, can capture both short-
term and long-term dependencies in time series data, where detailed relations
between distinct observations might be difficult to capture by conventional DL
methods [52] [53]. Attention-based networks have also the potential to enhance
forecasting accuracy and improve the robustness of time series analysis by
handling noise and perturbations [53]. Another merit of the MHA is its ability
to handle variable-length inputs without using reduction methods, making it
possible to learn more accurate representations of the input. In conventional
DL methods, padding or truncation is often used to ensure all input sequences
have the same length. This could lead to information loss and reduced model
performance [47]. Finally, the Attention mechanism can also be used to improve
the interpretability of the model in diverse academic contexts[54]. Attention
scores can provide insight into the specific segments of the input sequence
that the model focuses on when making predictions [55]. This can be extremely
practical in BP studies, in which identifying important observations can
provide valuable insight.

Table 5: Comparison of studies in BP estimation in terms of methodology, database, features, and metrics (NR means no result reported) Model Architecture |  Signal |  Datasets |  Model Injection Data Type |  MAE SBP |  MAE DBP |  RMSE SBP |  RMSE DBP |  STD SBP |  STD DBP
---|---|---|---|---|---|---|---|---|---
MLPlstm-BP (Based on MLP Mixer) [14] |  ECG + PPG |  MIMIC II |  Raw Waveform (multi-filter to multi-channel) |  3.52 |  2.13 |  5.1 |  3.13 |  5.09 |  3.07
Temporal Convolution Network (TCN) [1] |  ECG + PPG |  MIMIC I + MIMIC III |  Feature Extraction |  2.59 |  1.33 |  3.03 |  1.58 |  3.58 |  1.97
CNN + LSTM[13] |  ECG + PPG |  MIMIC III |  Raw Waveform Concatenation |  4.41 |  2.91 |  NR |  NR |  6.11 |  4.23
NABNet (BiConvLSTM)[22] |  ECG + PPG |  MIMIC III |  Raw Waveform (multi-channel) |  2.63 |  1.09 |  NR |  NR |  2.96 |  1.05
Bi-LSTM[56] |  ECG + PPG |  MIMIC II |  Feature Extraction |  2.51 |  1.38 |  3.22 |  1.78 |  NR |  NR
Deep Autoencoder[21] |  PPG |  MIMIC II |  Raw Waveform Concatenation |  5.42 |  3.14 |  NR |  NR |  6.64 |  3.74
1D_CNN + 2D_CNN[44] |  PPG |  MIMIC II |  Raw Waveform + Image |  3.05 |  1.58 |  NR |  NR |  5.26 |  2.6
CNN (Transfer Learning)[32] |  PPG |  MIMIC II |  Image (Transformed PPG) |  6.17 |  3.66 |  8.46 |  5.36 |  8.46 |  5.36
Bi-GRU + Attention[31] |  PPG |  MIMIC II |  Feature Extraction (22 features) |  2.58 |  1.26 |  NR |  NR |  3.35 |  1.63
BiLSTM + LSTM + Attention[26] |  PPG |  MIMIC II |  Feature Extraction |  4.51 |  2.6 |  NR |  NR |  7.81 |  4.41
GRU[24] |  PPG |  MIMIC II |  Feature Extraction (7 features) |  3.25 |  1.43 |  NR |  NR |  4.76 |  1.77
DNN[25] |  PPG |  MIMIC II |  Feature Extraction (32 features) |  3.21 |  2.23 |  4.64 |  3.3 |  NR |  NR
U-Net[42] |  PPG |  MIMIC III |  Raw Waveform |  5.73 |  3.45 |  NR |  NR |  9.16 |  6.14
Feed Forward ANN[57] |  PPG |  MIMIC II + UCI |  Feature Extraction (21 features) |  7.41 |  3.32 |  NR |  NR |  10.40 |  4.89
CNN+LSTM[58] |  PPG |  MIMIC III |  Feature Extraction |  3.79 |  2.89 |  NR |  NR |  7.89 |  5.34
ArterialNet Transformer[37] |  PPG |  MIMIC III |  raw waveform |  4.15 |  3.17 |  5.26 |  4.01 |  1.32 |  1.37
Transformer[35] |  PPG |  UCI |  raw waveform |  19.0 |  9.0 |  NR |  NR |  NR |  NR
FD L/NL regression[35] |  PPG |  UCI |  Frequency-domain waveform |  11.87 |  8.01 |  NR |  NR |  NR |  NR
STP (Self-supervised transformer)[34] |  PPG |  MIMIC III |  raw waveform |  3.37 |  2.48 |  NR |  NR |  4.21 |  2.76
TransfoRhythm (This Study) |  PPG |  MIMIC IV (360 record) |  Feature Extraction (12 features) |  1.50 |  1.17 |  1.84 |  1.42 |  1.84 |  1.42
|  |  |  |  |  |  |  |  |

Our results, obtained through different evaluation criteria, demonstrate the
supremacy of the proposed TransfoRhythm framework for estimating BP. In terms
of MAE and STD, the model obtained an interval of $[1.50,1.84]$ mmHg for DBP
and $and[1.17,1.42]$ mmHg for SBP. These findings fully satisfy the AAMI SP10
and BHS standard requirements. The lower MAE and RMSE values achieved by the
proposed model highlight its potential for accurate and reliable blood
pressure estimation in real-world applications. To further evaluate the
performance of the proposed model, the following two assessment stages are
considered.

  * •

Benchmarks on MIMIC IV Dataset: In the first stage, the model’s performance on
the MIMIC IV dataset is compared to that of benchmark models trained on the
same dataset. The results are shown in Table 4. This comparative evaluation
ensures a fair assessment of the model’s performance and provides confidence
in its accuracy and reliability for BP estimation. Among the evaluated models,
the proposed TransfoRhythm outperforms its counterparts, which could be
attributed to its internal MHA mechanism. By selectively attending to
important features, the MHA mechanism allows the TransfoRhythm to effectively
capture the relevant temporal dependencies and spatial relationships present
in the PPG signals. Furthermore, it allows the model to focus on informative
patterns, leading to more accurate BP estimations. In the benchmark models,
the ResNet1D model demonstrates the best performance (second best compared to
the TransfoRhythm). ResNet1D is a variant of the Residual Neural Network
(ResNet) that utilizes one-dimensional convolutional layers to capture spatial
features from the PPG signals. By employing residual connections, ResNet1D can
effectively address the vanishing gradient problem and facilitate the flow of
gradients during training. Following ResNet1D, the hybrid [43] and the $U-Net$
models exhibit competitive performance. The hybrid architecture combines the
strengths of both CNN and RNN allowing it to capture both spatial and temporal
dependencies in the PPG signals. The U-Net architecture is modified to
effectively capture temporal dependencies and extract meaningful features from
sequential data. The AlexNet1D model performs slightly worse than ResNet1D and
the hybrid model, achieving an SBP MAE of $3.09$, a DBP MAE of $1.59$, an SBP
RMSE of $7.98$, and a DBP RMSE of $4.6$. AlexNet1D employs 1D convolutional
layers followed by fully connected layers to extract features from the PPG
signals. Another evaluated benchmark model utilizes Bi-LSTM layers to capture
temporal dependencies in the PPG signals. Bi-LSTM processes the input sequence
in both forward and backward directions, effectively encoding past and future
information. Although Bi-LSTM performs reasonably well, it does not achieve
the same level of accuracy as our proposed model, ResNet1D, the hybrid model,
$U-Net$, and AlexNet1D models.

  * •

Comparisons with the State-of-The-Art Models: In the second stage of
performance evaluation, the model’s functionality is investigated by comparing
it to related state-of-the-art studies. Table 5 illustrates the comparisons,
which include model architecture, input signal type, dataset, feature
injection model, and metrics. The main objective is to provide a comprehensive
evaluation of the model’s performance, independent of dataset consideration.
This analysis allows for a thorough assessment of the model’s functionality
and its relative standing compared to other advanced approaches in the field
of BP estimation. Regarding datasets, most of the similar works employed MIMIC
II or MIMIC III datasets. MIMIC-IV, which was used in this study in comparison
with other available MIMIC datasets includes more PPG data. Meanwhile, MIMIC-
IV includes more recent data, which may be more representative of current
clinical practices, and has undergone rigorous quality control processes to
ensure that the data is accurate and reliable [38]. Regarding the model
selection, first, we should note that multiple studies applied traditional ML
methods such as SVM [59], RF [60], and XGBOOST [61]. These are not mentioned
in Table 5, as despite achieving acceptable results in some scenarios, DL
methods tend to perform considerably better given the time series nature of
ECG and PPG signals. Some other studies, such as [32], demonstrate good
performance in BP prediction using transfer learning without RNN layers. In
another type of modeling, RNN networks, especially LSTM and GRU, have been
used to extract two sets of features, e.g. [24]. The importance of employing
hybrid networks to extract temporal and recursive features of the signal was
indicated by multiple studies [31, 26, 29]. For instance, Malayeri et al. used
a hybrid CNN-based network and gathered the recursive features of the signal
with a 2D CNN structure Instead of RNN networks [44]. Furthermore, a
combination of RNN and attention layers was presented by [31, 26], and the
importance of attention usage was shown.

Despite the state-of-the-art performance achieved by the proposed
TransfoRhythm framework, below are some restrictions and potential future
research directions for this study. In terms of generalization, we used a
portion of the MIMIC IV dataset for the training and validation phases, and
all findings are based on this data. In other words, the TransfoRhythm is the
first BP estimation model developed solely based on PPG signals from the MIMIC
IV dataset. It would be interesting to compare the model with new developments
in the MIMIC IV dataset. Furthermore, to improve the model’s generalizability,
it would be interesting to extend it by incorporating other data sources. A
final fruitful future research direction is to go beyond the utilization of a
pure attention model, capitalizing on the recent progression of attention-
based networks.

## References

  * [1] Zabihi, S. _et al._ Bp-net: Cuff-less and non-invasive blood pressure estimation via a generic deep convolutional architecture.  _Biomedical Signal Processing and Control_ 78, 103850 (2022).
  * [2] Heidenreich, P. A. _et al._ Forecasting the future of cardiovascular disease in the united states: a policy statement from the american heart association.  _Circulation_ 123, 933–944 (2011).
  * [3] Lewis, P. S. & British and Irish Hypertension Society’s Blood Pressure Measurement Working Party.  Oscillometric measurement of blood pressure: a simplified explanation. a technical note on behalf of the british and irish hypertension society.  _Journal of human hypertension_ 33, 349–351 (2019).
  * [4] Mukkamala, R., Stergiou, G. S. & Avolio, A. P.  Cuffless blood pressure measurement.  _Annual Review of Biomedical Engineering_ 24, 203–230 (2022).
  * [5] Kario, K. _et al._ Effect of esaxerenone on nocturnal blood pressure and natriuretic peptide in different dipping phenotypes.  _Hypertension Research_ 45, 97–105 (2022).
  * [6] Hu, J.-R. _et al._ Validating cuffless continuous blood pressure monitoring devices.  _Cardiovascular Digital Health Journal_ (2023).
  * [7] Shao, J. _et al._ An optimization study of estimating blood pressure models based on pulse arrival time for continuous monitoring.  _Journal of Healthcare Engineering_ 2020 (2020).
  * [8] Yousefian, P. _et al._ Pulse transit time-pulse wave analysis fusion based on wearable wrist ballistocardiogram for cuff-less blood pressure trend tracking.  _IEEE Access_ 8, 138077–138087 (2020).
  * [9] Esmaili, A., Kachuee, M. & Shabany, M.  Nonlinear cuffless blood pressure estimation of healthy subjects using pulse transit time and arrival time.  _IEEE Transactions on Instrumentation and Measurement_ 66, 3299–3308 (2017).
  * [10] Pielmus, A.-G. _et al._ Surrogate based continuous noninvasive blood pressure measurement.  _Biomedical Engineering/Biomedizinische Technik_ 66, 231–245 (2021).
  * [11] Geddes, L., Voelz, M., Babbs, C., Bourland, J. & Tacker, W.  Pulse transit time as an indicator of arterial blood pressure.  _psychophysiology_ 18, 71–74 (1981).
  * [12] Mishra, B. & Thakkar, N.  Cuffless blood pressure monitoring using ptt and pwv methods.  In _2017 International Conference on Recent Innovations in Signal processing and Embedded Systems (RISE)_ , 395–401 (IEEE, 2017).
  * [13] Baker, S., Xiang, W. & Atkinson, I.  A hybrid neural network for continuous and non-invasive estimation of blood pressure from raw electrocardiogram and photoplethysmogram waveforms.  _Computer Methods and Programs in Biomedicine_ 207, 106191 (2021).
  * [14] Huang, B., Chen, W., Lin, C.-L., Juang, C.-F. & Wang, J.  Mlp-bp: A novel framework for cuffless blood pressure measurement with ppg and ecg signals based on mlp-mixer neural networks.  _Biomedical Signal Processing and Control_ 73, 103404 (2022).
  * [15] Tolstikhin, I. O. _et al._ Mlp-mixer: An all-mlp architecture for vision.  _Advances in neural information processing systems_ 34, 24261–24272 (2021).
  * [16] Goldberger, A. L. _et al._ Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals.  _circulation_ 101, e215–e220 (2000).
  * [17] Le, T. _et al._ Continuous non-invasive blood pressure monitoring: a methodological review on measurement techniques.  _IEEE Access_ 8, 212478–212498 (2020).
  * [18] Welykholowa, K. _et al._ Multimodal photoplethysmography-based approaches for improved detection of hypertension.  _Journal of Clinical Medicine_ 9, 1203 (2020).
  * [19] Bassiouni, M. M., Hegazy, I., Rizk, N., El-Dahshan, E.-S. A. & Salem, A. M.  Combination of ecg and ppg signals for smart healthcare systems: Techniques, applications, and challenges.  In _2021 Tenth International Conference on Intelligent Computing and Information Systems (ICICIS)_ , 448–455 (IEEE, 2021).
  * [20] Chao, P. C.-P. _et al._ The machine learnings leading the cuffless ppg blood pressure sensors into the next stage.  _IEEE Sensors Journal_ 21, 12498–12510 (2021).
  * [21] Qin, K., Huang, W. & Zhang, T.  Deep generative model with domain adversarial training for predicting arterial blood pressure waveform from photoplethysmogram signal.  _Biomedical Signal Processing and Control_ 70, 102972 (2021).
  * [22] Mahmud, S. _et al._ Nabnet: a nested attention-guided biconvlstm network for a robust prediction of blood pressure components from reconstructed arterial blood pressure waveforms using ppg and ecg signals.  _Biomedical Signal Processing and Control_ 79, 104247 (2023).
  * [23] Hassani, A. & Foruzan, A. H.  Improved ppg-based estimation of the blood pressure using latent space features.  _Signal, Image and Video Processing_ 13, 1141–1147 (2019).
  * [24] El Hajj, C. & Kyriacou, P. A.  Cuffless and continuous blood pressure estimation from ppg signals using recurrent neural networks.  In _2020 42nd annual international conference of the IEEE engineering in medicine & biology society (EMBC)_, 4269–4272 (IEEE, 2020).
  * [25] Hsu, Y.-C., Li, Y.-H., Chang, C.-C. & Harfiya, L. N.  Generalized deep neural network model for cuffless blood pressure estimation with photoplethysmogram signal only.  _Sensors_ 20, 5668 (2020).
  * [26] El-Hajj, C. & Kyriacou, P. A.  Cuffless blood pressure estimation from ppg signals and its derivatives using deep learning models.  _Biomedical Signal Processing and Control_ 70, 102984 (2021).
  * [27] Gupta, S., Singh, A., Sharma, A. & Tripathy, R. K.  Higher order derivative-based integrated model for cuff-less blood pressure estimation and stratification using ppg signals.  _IEEE Sensors Journal_ 22, 22030–22039 (2022).
  * [28] Liu, M., Po, L.-M. & Fu, H.  Cuffless blood pressure estimation based on photoplethysmography signal and its second derivative.  _International Journal of Computer Theory and Engineering_ 9, 202 (2017).
  * [29] Rastegar, S., Gholam Hosseini, H. & Lowe, A.  Hybrid cnn-svr blood pressure estimation model using ecg and ppg signals.  _Sensors_ 23, 1259 (2023).
  * [30] Paviglianiti, A., Randazzo, V., Villata, S., Cirrincione, G. & Pasero, E.  A comparison of deep learning techniques for arterial blood pressure prediction.  _Cognitive Computation_ 14, 1689–1710 (2022).
  * [31] El-Hajj, C. & Kyriacou, P. A.  Deep learning models for cuffless blood pressure monitoring from ppg signals using attention mechanism.  _Biomedical Signal Processing and Control_ 65, 102301 (2021).
  * [32] Wang, W., Mohseni, P., Kilgore, K. L. & Najafizadeh, L.  Cuff-less blood pressure estimation from photoplethysmography via visibility graph and transfer learning.  _IEEE Journal of Biomedical and Health Informatics_ 26, 2075–2085 (2021).
  * [33] Johnson, A. _et al._ Mimic-iv.  _PhysioNet. Available online at: https://physionet.org/content/mimic-iv-ed/2.0/ (accessed August, 2023)_ (2020).
  * [34] Ma, C. _et al._ Stp: Self-supervised transfer learning based on transformer for noninvasive blood pressure estimation using photoplethysmography.  _Expert Systems with Applications_ 123809 (2024).
  * [35] Tahir, M. A. _et al._ Cuff-less arterial blood pressure waveform synthesis from single-site ppg using transformer & frequency-domain learning.  _arXiv preprint arXiv:2401.05452_ (2024).
  * [36] Reiss, A., Indlekofer, I. & Schmidt, P.  PPG-DaLiA.  UCI Machine Learning Repository (2019).  DOI: https://doi.org/10.24432/C53890.
  * [37] Huang, S., Jafari, R. & Mortazavi, B. J.  Arterialnet: Arterial blood pressure reconstruction.  In _2023 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI)_ , 1–4 (IEEE, 2023).
  * [38] Moody, B. _et al._ Mimic-iv waveform database (version 0.1.0), DOI: 10.13026/a2mw-f949 (2022).
  * [39] Kalyan, K. S. & Sangeetha, S.  Secnlp: A survey of embeddings in clinical natural language processing.  _Journal of biomedical informatics_ 101, 103323 (2020).
  * [40] Vaswani, A. _et al._ Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * [41] Schrumpf, F., Frenzel, P., Aust, C., Osterhoff, G. & Fuchs, M.  Assessment of non-invasive blood pressure prediction from ppg and rppg signals using deep learning.  _Sensors_ 21, 6022 (2021).
  * [42] Ibtehaz, N. _et al._ Ppg2abp: Translating photoplethysmogram (ppg) signals to arterial blood pressure (abp) waveforms.  _Bioengineering_ 9, 692 (2022).
  * [43] Slapničar, G., Mlakar, N. & Luštrek, M.  Blood pressure estimation from photoplethysmogram using a spectro-temporal deep neural network.  _Sensors_ 19, 3420 (2019).
  * [44] Malayeri, A. B. & Khodabakhshi, M. B.  Concatenated convolutional neural network model for cuffless blood pressure estimation using fuzzy recurrence properties of photoplethysmogram signals.  _Scientific Reports_ 12, 6633 (2022).
  * [45] Wu, F., Fan, A., Baevski, A., Dauphin, Y. N. & Auli, M.  Pay less attention with lightweight and dynamic convolutions.  _arXiv preprint arXiv:1901.10430_ (2019).
  * [46] Jiang, L., Li, X., Wu, L. & Li, Y.  Bearing fault diagnosis method based on a multi-head graph attention network.  _Measurement Science and Technology_ 33, 075012 (2022).
  * [47] Bu, S.-J. & Cho, S.-B.  Time series forecasting with multi-headed attention-based deep learning for residential energy consumption.  _Energies_ 13, 4722 (2020).
  * [48] Nicolson, A. & Paliwal, K. K.  Masked multi-head self-attention for causal speech enhancement.  _Speech Communication_ 125, 80–96 (2020).
  * [49] Chen, Z., Jiaze, E., Zhang, X., Sheng, H. & Cheng, X.  Multi-task time series forecasting with shared attention.  In _2020 International Conference on Data Mining Workshops (ICDMW)_ , 917–925 (IEEE, 2020).
  * [50] Reza, S., Ferreira, M. C., Machado, J. J. & Tavares, J. M. R.  A multi-head attention-based transformer model for traffic flow forecasting with a comparative analysis to recurrent neural networks.  _Expert Systems with Applications_ 202, 117275 (2022).
  * [51] Blohm, M., Jagfeld, G., Sood, E., Yu, X. & Vu, N. T.  Comparing attention-based convolutional and recurrent neural networks: Success and limitations in machine reading comprehension.  _arXiv preprint arXiv:1808.08744_ (2018).
  * [52] Li, Y., Du, M. & He, S.  Attention-based sequence-to-sequence model for time series imputation.  _Entropy_ 24, 1798 (2022).
  * [53] Niu, P., Zhou, T., Wang, X., Sun, L. & Jin, R.  Attention as robust representation for time series forecasting.  _arXiv preprint arXiv:2402.05370_ (2024).
  * [54] Li, A., Xiao, F., Zhang, C. & Fan, C.  Attention-based interpretable neural network for building cooling load prediction.  _Applied Energy_ 299, 117238 (2021).
  * [55] Ahmed, S. _et al._ Transformers in time-series analysis: A tutorial.  _arXiv preprint arXiv:2205.01138_ (2022).
  * [56] Chen, H., Lyu, L., Zeng, Z., Jin, Y. & Zhang, Y.  Beat-to-beat continuous blood pressure estimation with optimal feature set of ppg and ecg signals using deep recurrent neural networks.  _Vessel Plus_ 7 (2023).
  * [57] Samimi, H. & Dajani, H. R.  A ppg-based calibration-free cuffless blood pressure estimation method using cardiovascular dynamics.  _Sensors_ 23, 4145 (2023).
  * [58] Mahardika T, N. Q., Fuadah, Y. N., Jeong, D. U. & Lim, K. M.  Ppg signals-based blood-pressure estimation using grid search in hyperparameter optimization of cnn–lstm.  _Diagnostics_ 13, 2566 (2023).
  * [59] Zhang, Y. & Feng, Z.  A svm method for continuous blood pressure estimation from a ppg signal.  In _Proceedings of the 9th international conference on machine learning and computing_ , 128–132 (2017).
  * [60] He, R. _et al._ Beat-to-beat ambulatory blood pressure estimation based on random forest.  In _2016 IEEE 13th international conference on wearable and implantable body sensor networks (BSN)_ , 194–198 (IEEE, 2016).
  * [61] Che, X., Li, M., Kang, W., Lai, F. & Wang, J.  Continuous blood pressure estimation from two-channel ppg parameters by xgboost.  In _2019 IEEE International Conference on Robotics and Biomimetics (ROBIO)_ , 2707–2712 (IEEE, 2019).

## Acknowledgments

This Project was partially supported by the Natural Sciences and Engineering
Research Council (NSERC) of Canada through the NSERC Discovery Grant
RGPIN-2023-05654.

## Data Availability

The utilized dataset is publicly available through the following link:
https://physionet.org/content/mimic4wdb/0.1.0/ [38].

## Author Contributions Statement

A.A. and A.B. implemented the deep/machine learning models and performed
evaluations; A.A and A.B. together with F.B. drafted the manuscript; F.B. and
A.M. contributed to the analysis and interpretation and edited the manuscript;
All authors reviewed the manuscript.

## Additional Information

Competing Interests: Authors declare no competing interests.