# CARTOS: A Charging-Aware Real-Time Operating System for Intermittent
Batteryless Devices

Mohsen Karimi, Yidi Wang, Hyoseung Kim  University of California, Riverside
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Mohsen Karimi, Yidi
Wang, Youngbin Kim, Yoojin Lim, Hyoseung Kim This work is supported in part by
grants from IITP (2021-0-00360), NSF (1943265), and USDA/NIFA
(2020-51181-32198).M. Karimi, Y. Wang, and H. Kim are with the Department of
Electrical and Computer Engineering, the University of California Riverside,
Riverside, CA 92521, USA (email<EMAIL_ADDRESS><EMAIL_ADDRESS>hyoseung@ucr.edu). Y. Kim and Y. Lim are with the Electronics and
Telecommunications Research Institute, Daejeon 34129, South Korea (email:
<EMAIL_ADDRESS>yoojin.lim@etri.re.kr).

###### Abstract

This paper presents CARTOS, a charging-aware real-time operating system
designed to enhance the functionality of intermittently-powered batteryless
devices (IPDs) for various Internet of Things (IoT) applications. While IPDs
offer significant advantages such as extended lifespan and operability in
extreme environments, they pose unique challenges, including the need to
ensure forward progress of program execution amidst variable energy
availability and maintaining reliable real-time time behavior during power
disruptions. To address these challenges, CARTOS introduces a mixed-preemption
scheduling model that classifies tasks into computational and peripheral
tasks, and ensures their efficient and timely execution by adopting just-in-
time checkpointing for divisible computation tasks and uninterrupted execution
for indivisible peripheral tasks. CARTOS also supports processing chains of
tasks with precedence constraints and adapts its scheduling in response to
environmental changes to offer continuous execution under diverse conditions.
CARTOS is implemented with new APIs and components added to FreeRTOS but is
designed for portability to other embedded RTOSs. Through real hardware
experiments and simulations, CARTOS exhibits superior performance over state-
of-the-art methods, demonstrating that it can serve as a practical platform
for developing resilient, real-time sensing applications on IPDs.

##  I Introduction

Batteryless systems, often called intermittently-powered devices (IPDs), have
the potential to revolutionize a variety of applications in Internet-of-Things
(IoT), such as smart healthcare, agriculture, and building monitoring, where
the use of batteries is undesirable or incurs significant maintenance costs.
These systems harvest energy from ambient sources such as sunlight, heat, and
radio signals, charge a small energy buffer, and execute intermittently for a
short period of time whenever the energy is available. Since no batteries are
needed, they have a much longer lifespan than battery-powered systems and can
be deployed in harsh environments where batteries cannot operate properly,
e.g., due to cold and hot temperatures [1, 2, 3]. This can also bring other
benefits to system design such as smaller form factors and reduced
manufacturing costs.

The aforementioned benefits, however, come with several technical challenges.
First, given the variability in energy availability from ambient sources, IPDs
must ensure the forward progress of program execution across power failures.
One of the most effective strategies is checkpointing [4, 5, 6, 7], a method
that involves inserting additional code snippets to store the state of a
running program in non-volatile memory (NVM). When power is restored, the
system can recover from the latest stored state, ensuring continuous progress.
However, such static checkpointing introduces a high overhead and can
significantly impact the performance of intermittent systems compared to
conventional battery-powered embedded systems. Just-In-Time (JIT)
checkpointing [8, 9, 10] mitigates this issue by making checkpoints
dynamically only when the power goes low. However, it is difficult to be used
for tasks with peripheral operations, e.g., sensor access and flash writing,
because they cannot be paused and resumed at an arbitrary point, i.e.,
execution needs to be done atomically.

Secondly, IPDs should provide accurate time behavior even in the presence of
intermittent power disruptions. This is particularly important for periodic
sensing applications, which are one of the most in-demand areas of IoT. While
many battery-powered IoT devices have been developed using conventional real-
time operating systems (RTOS) for microcontrollers, such as FreeRTOS [11],
$\mu$C/OS [12], RTEMS [13], to implement timely operations, they cannot offer
forward progress guarantees in IPDs. For instance, a power failure during the
execution of a sensor-reading or data-logging task can result in incomplete
task execution or the loss or corruption of data. Therefore, it is crucial for
the RTOS to consider the power consumption of tasks and the energy
availability of the system. Furthermore, providing analytical support to
understand how these devices will operate under diverse real-world
circumstances is of great importance.

In this paper, we propose CARTOS, a charging-aware real-time operating system,
as a practical platform to develop energy-resilient and reliable real-time
sensing applications on IPDs. To achieve forward progress guarantees without
losing efficiency, CARTOS classifies tasks into two types, computation tasks
and peripheral tasks, and introduces a mixed-preemption scheduling model.
Computation tasks are suspendable and resumable at any point of execution;
hence, CARTOS schedules them preemptively and uses JIT checkpointing to store
their states. On the other hand, since peripheral tasks are not JIT-
checkpointable, CARTOS checks before their execution if the system has enough
energy to complete them, and it schedules them non-preemptively to prevent
interruption. Upon this model, CARTOS supports processing chains of tasks,
i.e., tasks within an application have precedence constraints, which is common
in practical sensing applications. CARTOS also adapts task scheduling based on
environmental changes and ensures correct and timely execution under various
conditions. Based on our system design, we provide analysis to test the timing
behavior of processing chains of computation and peripheral tasks before
deployment.

CARTOS is implemented by introducing new APIs and components to FreeRTOS,
which is one of the most widely used embedded RTOS in real-world applications
such as Amazon AWS IoT. However, as the design of CARTOS is not specific to
FreeRTOS, it is portable to other RTOS as well. From our experiments on real
hardware and simulation, CARTOS demonstrated several advantages over the
state-of-the-art methods.

Section II provides an overview of previous work and establishes essential
background information. In Section III, we introduce the proposed framework.
Section IV presents the system model, the proposed mixed-preemption scheduling
approach, and offers a detailed scheduling analysis. Section V presents the
results of our evaluation. Lastly, Section VI summarizes and concludes this
paper.

##  II Background and Related Work

In this section, we give detailed background on IPD hardware characteristics
and discuss state-of-the-art methods developed in the literature. We also
review other studies related to energy-harvesting and IPD-based applications.

###  II-A IPD Hardware and Operation Phases

Figure 1: A typical hardware design of IPDs

A typical hardware design of IPDs is illustrated in Fig. 1. The harvesting
unit collects energy from the surrounding environment. It could be a solar
cell, RFID receiver, or piezoelectric transducer. The power management unit
(PMU) is responsible for converting and regulating the energy harvested by the
energy harvester into a stable voltage that can be used to power the device’s
components. It also manages the device’s power consumption to ensure that it
does not exceed the available energy. Energy storage is usually a
supercapacitor and is responsible for storing the collected electrical energy
so that it can be used later when it is needed. The micro-controller unit
(MCU) executes the device’s software instructions and controls the device’s
various components. Memory stores the program code and data that the MCU uses
to operate the device. Most IPDs have at least two types of memory: (i)
volatile main memory equipped as part of MCU, e.g., SRAM, and (ii) nonvolatile
memory to store execution progress during power loss, e.g., FRAM and MRAM.
IPDs also often include sensors and peripherals, such as temperature,
humidity, light sensors and wireless Bluetooth communication modules, to
monitor the device’s environment, perform certain actions, and report any
findings to the user.

Figure 2: Charging and discharging cycles of an IPD

The operation of an IPD can be broken down into the following three phases:

  1. 1.

Power-Off Phase: In this initial phase, the device remains entirely powered
off. It actively collects energy from the environment using an energy
harvester and subsequently stores the gathered energy in a capacitor.

  2. 2.

Operation Phase: Once the capacitor attains a sufficient voltage level, the
PMU provides a consistent voltage supply to the device. This, in turn, enables
the MCU to execute processing tasks and capture sensor data. The device
continues in this phase until it reaches the predefined power-off threshold
(going back to the power-off phase) or low-voltage threshold (moving on to the
standby phase discussed next) depending on the design. Many prior IPDs [14,
15, 4] use only the power-off threshold for simplicity and they operate as in
Fig. 2. The MCU starts processing and sensing activities when the capacitor
voltage reaches a specific power-on threshold, and it continues until the
voltage remains above the power-off threshold, at which point the device
powers off. It is worth noting that charging can still happen while the system
actively runs tasks in the operation phase; however, as the discharging rate
is typically much higher than the charging rate, the capacitor voltage goes
down over time.

  3. 3.

Standby Phase: In this phase, energy-intensive components like the MCU,
sensors, and radio units are either powered down or transitioned into sleep
mode. Only the timekeeping ability, e.g., MCU’s clock unit in low-power mode
or external programmable RTC, remains active as it plays a crucial role in
maintaining the precise time behavior of tasks [16, 17, 18, 19]. The device
remains in this state until the capacitor accumulates enough charge to return
to the operation phase. This approach can be particularly useful in situations
where the energy-harvesting rate is sporadic, as it allows the device to
conserve energy and maintain the notion of time during periods of low power
availability.

###  II-B Forward Progress Mechanisms

There has been extensive research on checkpointing to ensure that programs
running on IPDs make forward progress and maintain data consistency across
power failures. Static checkpointing [4, 5, 6, 20, 21, 22] inserts
checkpointing code at the fixed locations of the program, which are determined
by either the compiler support or manual programmer efforts. When a checkpoint
is reached, the system saves the program’s state and relevant data to non-
volatile storage, allowing the program to restart from that point in case of a
power failure. Instead of finding checkpoint locations after writing the
program code, new programming models have been studied to create checkpoints
as part of the program’s execution flow [6, 23, 24, 25, 7, 14, 15]. In such
models, the programmer needs to decompose a program logic into a collection of
idempotent, atomic blocks with explicit input and output data.111They call
such atomic blocks tasks in their work, but we avoid using their term since
tasks in our paper refer to threads in conventional RTOS. Then, the input and
output data of each task are recorded in NVM, which serves as checkpoints,
e.g., data channels between tasks [14, 7]. However, this adds a significant
burden to programmers to restructure their programs, which is particularly
difficult for long-running computations, e.g., algorithm logic. In addition,
programmers have to make sure that each atomic block is small enough to run
within one charging cycle.

Just-In-Time (JIT) checkpointing [10, 8, 26, 9], on the other hand, makes
checkpoints at runtime by capturing the current state of the system, including
MCU register values and volatile data. To do so, the system needs to monitor
the remaining energy and initiate JIT checkpointing when the low-voltage
threshold is reached. This can bring performance benefits over static
checkpointing by reducing the frequency of checkpointing and simplifying the
programming of long-running computations [26, 8]. However, since peripheral
operations cannot resume after reboot by recovering just the MCU’s state, JIT
checkpointing cannot be used for them [10].

TABLE I: Comparison of previous work Methods | JIT checkpointing | Peripherals | Long computations | Real-time scheduling | Energy adaptive | Analysis
---|---|---|---|---|---|---
Mementos[6] | ✗ | ✗ | ✓ | ✗ | ✓ | ✗
Chinchilla[4] | ✗ | ✗ | ✓ | ✗ | ✓ | ✗
Capybara[24] | ✗ | ✓ | ✗ | ✗ | ✗ | ✗
Quickrecall[8] | ✓ | ✗ | ✓ | ✗ | ✗ | ✗
Samoyed[10] | ✓ | ✓ | ✓ | ✗ | ✓ | ✗
InK[23] | ✗ | ✗ | ✗ | ✓ | ✓ | ✗
CatNap[9] | ✓ | ✓ | ✓ | ✗ | ✓ | ✗
Celebi[17] | ✗ | ✗ | ✓ | ✓ | ✓ | ✗
Rtag[16] | ✗ | ✓ | ✗ | ✓ | ✗ | ✓
Karimi et al.[27] | ✗ | ✓ | ✗ | ✓ | ✓ | ✓
CARTOS (this work) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓

###  II-C IPD Task Scheduling

Task scheduling for batteryless devices can be categorized into two main
types: real-time and reactive scheduling. Real-time scheduling focuses on
executing periodic tasks with predictable charging behavior. These approaches
aim to meet the deadlines of a specific set of tasks by assuming a certain
amount of energy supply[16, 27, 17]. By doing so, they can guarantee tasks to
be scheduled in a predictable manner and achieve consistent performance as
long as the assumed energy model holds. On the other hand, reactive scheduling
focuses on minimizing the overall response times of event-driven tasks [9,
23]. It starts task execution when the device’s energy reaches the predefined
power-on threshold and stops when it drops below the power-off threshold.
These approaches are usually more flexible and may perform better with
changing environmental conditions than the real-time approaches, but cannot
ensure predictability and consistent performance due to their best-effort
nature.

###  II-D Related Work

Table I presents a comprehensive comparison of various methods designed to
address task execution on IPDs. As we already characterized prior work by
category, we will discuss specific details of some of the work and additional
related work below.

Among static checkpointing approaches, Mementos [6] proposes a method to
automatically insert checkpointing code to the program at compile time. To
reduce the cost of checkpointing, it also proposes to estimate the remaining
energy and perform checkpointing only when the energy goes below a certain
threshold. Chinchilla [4] also takes a similar approach. Capybara [24] relies
on the programming model support for forward progress, and introduces a
hardware-software co-design solution that includes a set of capacitors to
handle tasks with different energy requirements.

Quickrecall [8] is a JIT checkpointing system but does not consider peripheral
tasks. Samoyed [10] addresses this issue by introducing the undo-logging
mechanism, which basically adds static checkpoints for peripheral operations.
However, this approach is applicable only to the peripherals that “do not have
internal non-volatile state and are arbitrarily restartable” [10] because of
the idempotency requirement.

For real-time scheduling on IPDs, Celebi [17] proposes a scheduler that
optimizes task scheduling based on known charging patterns. However, this
method is primarily designed for independent computational tasks and may not
handle peripheral tasks correctly because the scheduler does not ensure the
atomicity and idempotency of task execution, e.g., the scheduler can stop a
task at an arbitrary point. Karimi at al. [16] introduce a real-time scheduler
designed to atomically execute independent periodic tasks with various energy
requirements and provide a formal analysis of task schedulability for a fixed
charging rate of the system. The same authors [27] present an energy-adaptive
scheduler to minimize the age of information. While these approaches
contribute to enabling real-time operations on IPDs, none of them are
sufficient to handle real-world applications that consist of a chain of
peripheral access and long-running computations.

There are also other previous studies that focus on running neural networks on
IPDs [28, 18], enabling multiple energy buffers [29], energy-aware memory
mapping [30], communication on IPDs [31], and zero-power timekeeping methods
[32]. They contribute to expanding the application scope of IPDs and our work
does not conflict with them.

##  III CARTOS Framework

Figure 3: System Framework

This section presents our proposed CARTOS framework. Fig. 3 depicts the
overview of the framework. CARTOS targets typical IPD hardware as discussed in
Sec. V-A (left upper corner of the figure) and introduces additional circuitry
to monitor and control the power of the entire board (right upper corner).
CARTOS includes an adaptive real-time scheduler that is built upon our mixed-
preemption scheduling model to support applications with a chain of peripheral
and long-running computational tasks. CARTOS also includes a neural network-
based energy (charging rate) predictor to adapt to changing environmental
conditions. In the rest of this section, we will first describe the hardware
setup for CARTOS and then present the major software components of our
framework.

###  III-A Hardware Setup

CARTOS targets an IPD that is equipped with an MCU capable of running a
conventional RTOS such as FreeRTOS. The IPD is expected to have both volatile
memory (SRAM) and non-volatile memory (NVM). Although we used an ARM Cortex-M4
MCU in our implementation, there is no other restriction imposed on MCUs by
CARTOS.

While different energy sources can be used, we primarily consider solar energy
as a harvesting source. Hence, we assume the IPD has a solar harvesting unit
that collects energy from the sun and converts it into usable electrical
energy. The harvested energy is then stored in a capacitor to drive the IPD.

In order to utilize all three operational phases discussed in Sec. II-A and
support our scheduling behavior, CARTOS assumes that IPD hardware provides the
ability to (i) monitor the current charging rate and the remaining energy
level of the capacitor, and (ii) cut off power supply to the processing part
of the board, e.g., MCU, NVM, sensors, etc., for a requested interval. Item
(i) is the voltage monitoring circuits and item (ii) is the energy controller
in Fig. 3. Those can be integrated with other hardware components on the same
board or can be realized in a separate board, as we did in our implementation
(see Sec. V-A).

Figure 4: An example of multiple chains with different tasks

###  III-B Task Chains and Mixed Preemption Model

CARTOS supports periodic applications that consist of a processing chain of
tasks, which is a common design pattern found in sensing and intelligent
applications [33, 34, 14, 35, 36]. Fig 4 illustrates two processing chains,
one comprising four tasks and the other with three tasks. Each task
$\tau_{i}^{j}$ on a chain $i$ can be either a peripheral or computation
task.222Detailed chain and task parameters needed for analysis will be
provided in Sec. IV. The chain can start execution when its first task is
released (e.g., the periodic sensing task $\tau_{1}^{1}$ on chain 1) and the
subsequent tasks are eligible to run only when their preceding task finishes
(e.g., the action task $\tau_{1}^{4}$ becomes “ready” after the completion of
the processing task $\tau_{1}^{3}$). We assume that the first task of a chain
is a periodic task, triggered by either a timer or an external event. Hence,
all other tasks of the same chain follow the same periodic pattern with
precedence constraints among them.

####  III-B1 Mixed-Preemption Model

Recall that processing chains can consist of a mixture of peripheral and
computation tasks. Peripheral tasks need to be executed atomically, especially
when the corresponding hardware maintains internal state [10]. To ensure the
atomic execution of peripheral tasks, CARTOS schedules peripheral tasks non-
preemptively, i.e., once selected for scheduling, it will continue with no
preemption from other higher-priority tasks. However, this is not enough for
atomicity. The adaptive scheduler of CARTOS (Sec. III-C) waits until enough
energy is charged to complete the execution of a peripheral task for this
period before scheduling that task. This guarantees that the system does not
turn off at least until the completion of the peripheral task, thereby
achieving correctness. On the other hand, computational tasks are scheduled
preemptively, just like other RTOS, and CARTOS manages their forward progress
by JIT checkpointing. To summarize, peripheral and computational tasks in
CARTOS have the following characteristics:

  * •

Peripheral tasks: atomic, non-preemptable (i.e., not suspendable while it is
actively executing; unable to JIT checkpoint)

  * •

Computational tasks: non-atomic, preemptable (i.e., suspendable at any time;
can be JIT checkpointed)

This mixed-preemption model is the basis for our scheduler presented in the
next section. The type of a task (atomic or non-atomic) is configurable when
the user creates a task. For ease of presentation, we will use peripheral and
atomic tasks interchangeably. The same applies to computational and non-atomic
tasks.

Figure 5: Periodic task example in CARTOS

####  III-B2 Support for Periodic Tasks and Chains

CARTOS offers APIs to assist the programming and execution of periodic tasks.
Users can define tasks as periodic and specify their required periods and
release offset based on the application’s demands. Specifically, the APIs are:

  * •

setTaskPeriod(period, offset): This makes the calling task run periodically
after the given initial release offset. The second parameter is optional.

  * •

waitForNextPeriod(): This tells the kernel that the calling task has finished
its execution for the current period. The task will be suspended immediately
and will be released (ready state) at the start of the next period.

Fig. 5 depicts the example of a periodic task (task1Runner) in CARTOS.
setTaskPeriod() is called at the beginning of the task’s code, declaring that
it is a periodic task. The task has a while-loop where each iteration
represents a job for each period. When the task reaches the end of execution
in the current period, it calls waitForNextPeriod(). Then, the scheduler
removes it from the ready queue and sets the system timer to insert the task
back to the ready queue at the start of the next period.

For chained task execution, CARTOS also allows the user to create chain
objects in the kernel and assign periodic tasks to specific chains. While
chain objects are not schedulable entities, they help control the execution
order of tasks within a chain. Once tasks are assigned to a chain, these tasks
are removed from the ready queue and made inactive. Chain execution commences
with the release of the first task, which is triggered by either a timer or an
external event as we explained earlier. When the first task finishes its job
through the invocation of waitForNextPeriod(), the subsequent task in the
chain is set ready and becomes eligible for execution by the scheduler. This
continues until the last task of the chain. Consequently, if all tasks within
the chain can complete execution within one chain period (equivalent to the
period of the first task), only one task from the chain is ready at any given
time. This approach enforces the precedence constraints within a chain while
allowing our scheduler to follow the task-level priority-based scheduling
policy.

###  III-C Adaptive Real-Time Scheduler

The real-time scheduler of CARTOS is designed to utilize efficiently available
energy while supporting both peripheral (atomic) and computational (non-
atomic) tasks. In doing so, it prioritizes the execution of higher-priority
tasks that are ready over those with lower priority, a crucial feature for
RTOS to meet timing requirements. By adhering to the priority-based scheduling
policy, CARTOS minimizes the risk of missing important deadlines, thereby
enhancing the system’s overall performance and reliability.

Figure 6: Scheduler operation flowchart

Fig 6 illustrates the operation flow of the scheduler from the device boot-up.
Upon boot-up, the scheduler checks for the presence of any valid JIT
checkpoint in NVM. If a checkpoint is found, the device restores tasks from
this checkpoint and updates the system time and timer events, including those
of waitForNextPeriod() from periodic tasks.

For scheduling tasks, the scheduler first finds the highest-priority task
$\tau_{h}$ from the ready queue. It is worth noting that the scheduler does
not need to consider the execution order of tasks within chains here, because
such precedence constraints are controlled by the chain support explained in
the previous section. Then, the scheduler checks if the highest-priority task
$\tau_{h}$ is atomic (peripheral) or not and the scheduling behavior changes
according to the task type.

If the task $\tau_{h}$ is non-atomic (computational), the scheduler executes
$\tau_{h}$ for one tick of time (scheduling time quantum; 1 ms in our
implementation) and repeats the above procedure to find the highest-priority
ready task again for preemptiveness. Note that this is the case for tick-based
RTOS. In the case of a tickless system, $\tau_{h}$ can continue to run until
it voluntarily relinquishes the CPU or any higher-priority task than
$\tau_{h}$ becomes ready.

If $\tau_{h}$ is atomic (peripheral), the scheduler checks the required energy
for $\tau_{h}$’s job execution based on the profiled data, and compares it
with the current system energy level, measured through the voltage level of
the capacitor (more details on this will be given in the next subsection). If
the system’s available energy is higher than the amount required to ensure the
uninterrupted execution of the given atomic task, the scheduler dispatches
this task and the task will continue until its job is completed, i.e., non-
preemptive execution. Otherwise, it proactively enforces a power cycle to
harvest more energy.333Precisely speaking, the device transitions to the
standby phase using the energy controller unit discussed in Sec. III-A. In
doing so, it first estimates the required harvesting time, $\Delta t$,
considering the predicted charging rate and the current voltage level, and
then computes the next device wake-up time from the standby phase by taking
into account both the estimated harvesting time and the future release times
of other higher-priority periodic tasks. This is done to ensure that higher-
priority tasks are not missed during charging. For example, if a high-priority
task $\tau_{1}$ needs to be released at time $t_{1}$ while the device is being
charged for a low-priority task $\tau_{2}$, the charging process should be
stopped so that the high-priority task $\tau_{1}$ is scheduled. Therefore, the
next wake-up time from the standby phase is obtained by taking the minimum
between the required harvesting time and the earliest future release time of
higher-priority tasks than $\tau_{h}$. Subsequently, the device is powered
down (after necessary JIT checkpoint operations), and the atomic task
$\tau_{h}$ as well as other tasks are scheduled when the device is powered on.
It should be noted that such a proactive power-cycling sequence happens before
reaching the low-voltage threshold. When the device reaches a low-voltage
threshold, the device power cycling is managed by the JIT checkpointing
service presented next.

###  III-D JIT Checkpointing Service

In CARTOS, JIT checkpointing is implemented as a system service task. This
service monitors if the capacitor voltage level reaches the low-voltage
threshold, which can be implemented as an external interrupt or periodic
polling of the voltage monitoring circuits. The JIT service task is launched
as part of the boot-up sequence and executes with the highest priority in the
system to execute as soon as possible when needed.

The low-voltage threshold can be triggered at any time during non-atomic
preemptive task execution (recall that our scheduler executes atomic tasks
only when the device has sufficient energy to complete their jobs). Then, the
JIT service task enforces a new power cycle in a similar procedure as stated
before. Specifically, it is in charge of performing JIT checkpointing
operations. It iterates over all non-atomic tasks in the system, and for each
non-atomic task, it stores the task’s context (task control block), stack, and
SRAM data areas to NVM so that those can be restored to SRAM upon the next
power-on. The JIT service also stores the current system time and timer events
in NVM in order to ensure the correct execution of timer-based operations
across power cycles. Once the device powers up, the boot-up procedure restores
non-atomic tasks and updates the system time and timer events. The time update
can be done in two ways. If there is an external RTC, e.g., as part of the
energy controller, it can simply update the system time based on it.
Otherwise, it can estimate the current time by adding the latest checkpointed
system time and the power-up time that was computed before power cycling (Sec.
III-C and Fig. 6). We use the latter approach in our implementation.

###  III-E Charging Management

While JIT checkpointing facilitates the proper execution of non-atomic tasks,
charging management is vital to ensure the execution of atomic tasks. To
address this, we assign a voltage threshold $V^{j}_{i}$ to each atomic task
$\tau^{j}_{i}$. This threshold represents the minimum voltage that the system
requires before starting the execution of task $\tau^{j}_{i}$’s job, thereby
ensuring that the task can be completed before reaching the low-voltage
threshold used for JIT checkpointing. The calculation for $V^{j}_{i}$ is as
follows:

$V^{j}_{i}=\sqrt{\frac{2Q^{j}_{i}W_{S}+\mathbb{C}V_{min}^{2}}{\mathbb{C}}}$
(1)

where

  * •

$C^{j}_{i}$: Worst-case execution time of a job of the task $\tau_{i}^{j}$

  * •

$Q^{j}_{i}$: Charging time of the task, which is the amount of time required
to charge to perform $C^{j}_{i}$ of execution.

  * •

$W_{S}$: Energy-harvesting rate of the system

  * •

$\mathbb{C}$: Capacitor size in Farad

  * •

$V_{min}$: Low-voltage threshold that triggers JIT checkpointing and power
cycling

Eq. (1) is derived from the well-known calculation on the energy changes in a
capacitor when it receives a fixed power of $W$ from time $t_{1}$ to $t_{2}$,
given by

$\Delta
E=\frac{1}{2}\mathbb{C}\left(V_{2}^{2}-V_{1}^{2}\right)=W\left(t_{2}-t_{1}\right)$
(2)

To derive Eq. (1) from the above equation, $(t_{2}-t_{1})$ is substituted with
$Q^{j}_{i}$, $W$ is substituted with $W_{s}$, $V_{2}$ is substituted with
$V^{j}_{i}$, and $V_{1}$ is substituted with $V_{min}$.

In Eq. (1), the values of $C^{j}_{i}$ and $Q^{j}_{i}$ are specific to each
task, while $V_{min}$ and $\mathbb{C}$ are the system parameters given by the
user at design time. The charging rate of the system, $W_{S}$, is updated at
runtime using an energy predictor as the environment changes. Additionally,
after each job execution, the value of the worst-case execution time
$C^{j}_{i}$ is profiled and updated in the task’s task control block (TCB). If
there is a change in $C^{j}_{i}$ or $W_{S}$, the value of $Q^{j}_{i}$ for that
task is also updated at runtime. The calculation for updating $Q^{j}_{i}$ is
as follows:

$Q^{j}_{i}=\frac{\left(W^{j}_{i}-W_{S}\right)\times C^{j}_{i}}{W_{S}}$ (3)

where $W^{j}_{i}$ is the discharging rate of $\tau^{j}_{i}$.

When scheduling an atomic task $\tau^{j}_{i}$, the scheduler compares current
voltage level of the system, ${V_{current}}$, with $V^{j}_{i}$. If the current
voltage level exceeds $V^{j}_{i}$, then the scheduler starts executing the
task until its completion. Otherwise, the required harvesting time $\Delta t$
is calculated by the following equation and a power cycle is scheduled:

$\Delta
t=\frac{\mathbb{C}\left({V^{j}_{i}}^{2}-{V_{current}}^{2}\right)}{2\cdot
W_{S}}$ (4)

The energy harvesting rate, $W_{S}$, can change over time depending on
environmental conditions. As the required harvesting time and the next device
wake-up time rely on this, it is crucial to obtain a precise value of $W_{S}$
for efficient operations because the over-estimation makes the device turn on
unnecessarily late and the under-estimation lets the device turn on and off
multiple times until it reaches the required charge level. To address this
issue, CARTOS employs a machine learning-based energy prediction model
proposed in [27]. This model is a simple yet viable approach for MCU-level
devices, providing a reliable prediction of solar harvesting rates by
leveraging the temporal dependencies of time-series voltage data. In our
implementation, we integrated this model into CARTOS and used a 30-minute
interval for our prediction window.

##  IV Mixed-Preemption Processing Chain Analysis

This section gives a schedulability analysis of processing chains running in
CARTOS. The ability to check if chains will be able to meet their deadlines is
important since such information can be used to predict the performance of the
system before deployment and provides an opportunity to further optimize
hardware and software designs. In the following, we describe the system model
and assumptions made for analysis purposes and provide our analysis for chains
of tasks.

###  IV-A System Model

We consider a system with $n$ processing chains. A chain $i$ is modeled as
$\Gamma_{i}:(C_{i},T_{i},D_{i},Q_{i})$, where $C_{i}$, $T_{i}$, and $D_{i}$
are the worst-case execution time, period, deadline, and charging time of one
instance (job) of the chain. Each chain $\Gamma_{i}$ is comprised of $m_{i}$
tasks. Each task is either atomic (non-preemtable) or non-atomic
(preemptable), following the mixed-preemption model of CARTOS. A task $j$ of a
chain $\Gamma_{i}$ is denoted as
$\tau^{j}_{i}:(C^{j}_{i},T^{j}_{i},D^{j}_{i},Q^{j}_{i})$, where $\forall j\leq
m_{i}~{}|~{}T^{j}_{i}=T_{i},C_{i}=\sum_{j=1}^{m_{i}}C^{j}_{i},Q_{i}=\sum_{j=1}^{m_{i}}Q^{j}_{i}$,
i.e., all the tasks within the same chain share the same period as their chain
and the execution time of the chain is the sum of the execution of all tasks
in that chain. Tasks in a processing chain are ordered in sequence. Hence, for
a processing chain to complete, all of its tasks should complete their
execution in that order, i.e. $\tau^{1}_{i}$ should be the first task to be
executed and then $\tau^{2}_{i}$ and so on.

Each chain $\Gamma_{i}$ has a priority of $\pi_{i}$, e.g., $\pi_{h}>\pi_{i}$
means $\Gamma_{h}$ is a higher-priority chain than $\Gamma_{i}$. Tasks within
a chain share the same task-level priority and they are executed in order.

We use $R_{i}$ to denote the worst-case response time of a chain $\Gamma_{i}$.
A system with $n$ processing chains is said to be schedulable when the
response times of all the chains are smaller than or equal to their deadlines,
i.e., $\forall i\leq n~{}|~{}R_{i}\leq D_{i}$.

In real-world scenarios, the energy harvesting rate of the system and the
charging time of tasks can vary with the availability of energy resources.
However, especially for the solar energy we consider in this work, the changes
are relatively slow (e.g., over several tens minutes and hours) and do not
typically happen within a chain period or the hyperperiod of chains.
Therefore, a fixed energy harvesting rate is considered for analysis purposes
while our runtime system can still adapt to such changes.

###  IV-B Schedulability Analysis

Recall our mixed-preemption scheduling model. Atomic tasks are non-preemptibly
scheduled, so once started, they cannot be preempted by any other tasks. On
the other hand, non-atomic tasks are preemptable by higher-priority tasks
during their execution. To analyze the worst-case response time of a chain
comprised of such tasks, we extend two existing methods: the task-level mixed
preemption analysis without energy constraint [37], and the non-preemptive IPD
task scheduling analysis [16], both of which are exact analysis. Our extension
takes into account the charging requirements of tasks and the execution order
within chains, as follows.

The level-$i$ active period plays an important role in computing the worst-
case response time when non-preemptiveness is involved. As such, we first
revise the existing calculation of the level-$i$ active period for a task to
our processing chain model. The level-$i$ active period of a chain
$\Gamma_{i}$ with the charging time $Q_{i}$ can be computed recurrently by:

$L_{i}^{\left(s\right)}=B_{i}+\sum_{h:\pi_{h}\geq\pi_{i}}\left\lceil\frac{L_{i}^{\left(s-1\right)}}{T_{h}}\right\rceil\left(C_{h}+Q_{h}\right)$
(5)

where $B_{i}$ is the blocking time due to a non-preemptive task from a lower-
priority chain and $Q_{h}$ is the charging time of $\Gamma_{h}$. $B_{i}$ can
be obtained by

$B_{i}=\max_{\forall l,j\mid\pi_{l}<\pi_{i}\land
A^{j}_{l}=true}\left\\{C^{j}_{l}\right\\}$ (6)

where $A_{l}^{j}$ indicates the atomicity of a task $\tau_{l}^{j}$, i.e.,
$A_{l}^{j}=true$ if $\tau_{l}^{j}$ is atomic (non-preemptable) and
$A_{l}^{j}=false$ if $\tau_{l}^{j}$ is non-atomic (preemptable). The iteration
of Eq. (5) starts with $L_{i}^{\left(0\right)}=B_{i}+C_{i}$ and stops when
$L_{i}^{\left(s\right)}=L_{i}^{\left(s-1\right)}$ (converged) or
$L_{i}^{\left(s\right)}\geq T_{H}$ where $T_{H}$ is the hyperperiod of all
chains in the system (failed). Here, the charging time $Q_{h}$ is added to the
execution time $C_{h}$ because CARTOS can effectively delay execution by up to
this amount when the charge is insufficient (recall our scheduler and JIT
checkpointing service in Sec. III).

The use of the level-$i$ active period, $L_{i}$, allows us to find the worst-
case response time of a chain $\Gamma_{i}$ by considering the response times
of all jobs of $\Gamma_{i}$ within $L_{i}$. Specifically, the chain response
time $R_{i}$ can be obtained by

$R_{i}=\max_{k\leq K_{i}}\left\\{F_{i,k}-(k-1)T_{i}\right\\}$ (7)

where $K_{i}$ is the number of jobs of $\Gamma_{i}$ within $L_{i}$, i.e.,
$K_{i}=\left\lceil\frac{L_{i}}{T_{i}}\right\rceil$, and $F_{i,k}$ is the
finishing time of the $k^{th}$ job of $\Gamma_{i}$ in $L_{i}$. Since tasks
within a chain are executed sequentially, $F_{i,k}$ is equal to the finishing
time of the last task $\tau_{i,k}^{m_{i}}$ of $\Gamma_{i}$, i.e.,

$F_{i,k}=F_{i,k}^{m_{i}}$ (8)

If the last task $\tau_{i,k}^{m_{i}}$ is atomic, due to non-preemptiveness,
its latest finishing time $F_{i,k}^{m_{i}}$ is calculated by

$F_{i,k}=S_{i,k}^{m_{i}}+C_{i}^{m_{i}}$ (9)

where $S_{i,k}^{m_{i}}$ is the start time of $\tau_{i,k}^{m_{i}}$’s execution
and we discuss this later.

If the task $\tau_{i,k}^{m_{i}}$ is non-atomic, $F_{i,k}$ is obtained by the
following recurrence, taking into account preemption caused by higher-priority
chains during the start and finish time of the task $\tau_{i,k}^{m_{i}}$:

${F_{i,k}}^{\left(s\right)}=S_{i,k}^{m_{i}}+C^{m_{i}}_{i}\\\
+\sum_{h:\pi_{h}>\pi_{i}}\left(\left\lceil\frac{{F_{i,k}}^{\left(s-1\right)}}{T_{h}}\right\rceil-\left(\left\lfloor\frac{S_{i,k}^{m_{i}}}{T_{h}}\right\rfloor+1\right)\right)(C_{h}+Q_{h})$
(10)

The iteration in Eq. (10) can start with
${F_{i,k}}^{(0)}=S_{i,k}^{m_{i}}+C^{m_{i}}_{i}$.

To find the start time of task $\tau_{i,k}^{m_{i}}$, $S_{i,k}^{m_{i}}$, we
should consider the following factors: (i) blocking caused by lower priority
chains, (ii) preemption caused by higher priority chains, (iii) execution time
of previous jobs of the chain $\Gamma_{i}$, (iv) executions of all other tasks
in the chain $\Gamma_{i}$ that should be serviced before the task
$\tau_{i,k}^{m_{i}}$, and (v) all the overheads caused by charging. Note that
the job of the chain $\Gamma_{i}$ can be blocked only once during its entire
job execution due to the priority-based scheduling of CARTOS. Therefore, for
the task $\tau_{i,k}^{m_{i}}$, the start time $S_{i,k}^{m_{i}}$ can be
computed by the following recurrence:

${S_{i,k}^{m_{i}}}^{(s)}=B_{i}+\left(k-1\right)C_{i}+\sum_{p=1}^{m_{i}-1}C_{i}^{p}\\\
+\sum_{h:\pi_{h}>\pi_{i}}\left(\left\lfloor\frac{{S^{m_{i}}_{i,k}}^{(s-1)}}{T_{h}}\right\rfloor+1\right)C_{h}+{\nu_{i,k}}^{(s)}$
(11)

where the starting condition is

${S_{i,k}^{m_{i}}}^{(0)}=(k-1)\cdot T_{i}+B_{i}+\sum_{p=1}^{m_{i}-1}C_{i}^{p}$
(12)

In Eq. (11), ${\nu_{i,k}}^{(s)}$ is the total amount of time required for $k$
jobs of the chain $\Gamma_{i}$ to acquire their own portions of charge. This
can be calculated at each iteration by considering the charging time of
higher-priority chain jobs:

${\nu_{i,k}}^{(s)}=k\times
Q_{i}+\sum_{h:\pi_{h}>\pi_{i}}\left(\left\lfloor\frac{{S^{m_{i}}_{i,k}}^{(s-1)}}{T_{h}}\right\rfloor+1\right)Q_{h}$
(13)

From the above, we can compute the worst-case response time of each chain and
test the schedulability of the system.

##  V Evaluation

We first demonstrate the effectiveness of CARTOS by using our implementation
on a real system. Then, we perform schedulability experiments to evaluate how
well our scheduling framework performs compared to the state-of-the-art under
various conditions.

Figure 7: The hardware setup for evaluation

###  V-A Hardware Setup

Fig. 7 depicts the hardware setup for our experiments. This includes a
processing board equipped with an STM32L496ZG MCU (ARM Cortex-M4) and an
external nonvolatile MRAM module. To supply power, we make use of an energy
harvesting board comprising a 0.47F capacitor, LTC3588 energy harvester, and a
2W solar panel. Furthermore, an energy controller board featuring an ultra-
low-power MPS430 MCU is used to control the power supply to the processing
board. We have also integrated and configured a camera within the system to
facilitate applications related to image processing. In our setup, the power-
on and power-off thresholds are 4.8V and 3.8V, respectively. The low-voltage
threshold is set to 4.0V and the difference between the power-off and low-
voltage thresholds is to ensure enough energy for JIT checkpointing.

The processing board interacts with the energy controller board using serial
communication. The interaction is mainly to request power cut-off for a given
amount of time. To control the power connection between the energy harvesting
board and the processing board, MOSFET transistors are used as a switch. This
switch is placed between the regulated power from the energy harvesting board
and the processing board and is controlled by the energy controller.

###  V-B Real System Experiments

We consider two experimental scenarios to demonstrate the performance of
CARTOS against the standard JIT checkpointing method that allows preemptive
scheduling and the state-of-the-art (SOTA) real-time IPD scheduler [16] that
executes all tasks non-preemptively. In both methods, priority-based
scheduling is used for fairness of comparison.

The first scenario is designed to evaluate the ability to handle peripheral
tasks with long execution time. The system is configured to run two chains.
The first chain contains three tasks running with high priority, where the
first and last tasks are non-preemptable (peripheral) and the second task is
preemptable (computational). The second chain has one non-preemptable task
$\tau_{2}$ with extended execution time and running with low priority. The
task parameters are detailed in Table II.

TABLE II: Task parameters for first scenario | $C~{}(ms)$ | $T~{}(s)$ | $D~{}(s)$ | $\pi$ | Preemptable
---|---|---|---|---|---
$\tau^{1}_{1}$ | 3000 | 60 | 59.2 | 2 | N
$\tau^{2}_{1}$ | 500 | 60 | 59.7 | 2 | Y
$\tau^{3}_{1}$ | 300 | 60 | 60 | 2 | N
$\tau_{2}$ | 3500 | 120 | 120 | 1 | N

Fig. 8(a) illustrates the execution behavior of tasks in the first scenario
under the JIT checkpointing method. The release offset of all tasks is set to
zero, resulting in all jobs arriving at time $t=0$. After the execution of the
first three high-priority jobs ($\tau_{1}^{1}$, $\tau_{1}^{2}$,
$\tau_{1}^{3}$), the execution of the chain 2 task $\tau_{2}$, starts.
However, during the execution of $\tau_{2}$, the device’s energy level drops
to the low-voltage threshold, causing it to shut down the device. Upon the
next boot-up, new jobs from high-priority tasks are released and subsequently
executed. As $\tau_{2}$ is a non-preemptive atomic task, it restarts its
execution from the beginning, repeating the same execution pattern until
$\tau_{2}$ misses its deadline at $t=120$s.

Conversely, as illustrated in Fig. 8(b), CARTOS does not begin the execution
of $\tau_{2}$ when the first three jobs of high-priority tasks finish.
Instead, it finds that the charging level is insufficient to complete
$\tau_{2}$’s execution, and initiates power cycling just enough to charge the
demand of $\tau_{2}$. Consequently, the device wakes up at $t=20$s, and
$\tau_{2}$ executes successfully without interruption. These results show that
JIT checkpointing lacks the ability to accommodate non-preemptable tasks and
can lead to energy wastage.

(a) JIT Checkpointing

(b) CARTOS

Figure 8: Task execution behavior of the proposed method compared to JIT
checkpointing

The second scenario involves a long computational (preemptive) task. The
system has two single-task chains: the high-priority chain $\Gamma_{1}$ has a
non-preemptive task $\tau_{1}$ with a shorter execution time and the low-
priority chain $\Gamma_{2}$ has a preemptive task $\tau_{2}$ with a longer
execution time. $\Gamma_{1}$ has a release offset of 10ms, while $\Gamma_{2}$
has a release offset of zero. The task parameters are detailed in Table III.

TABLE III: Task parameters for second scenario | $C~{}(ms)$ | $T~{}(s)$ | $D~{}(s)$ | $\pi$ | Preemptable
---|---|---|---|---|---
$\tau_{1}$ | 3000 | 40 | 40 | 2 | N
$\tau_{2}$ | 5000 | 200 | 200 | 1 | Y

Fig. 9(a) shows the results of the SOTA real-time IPD scheduler [16]. Due to
the non-zero release offset of the high-priority task $\tau_{1}$, the low-
priority task $\tau_{2}$ starts first at $t=0$. The non-preemptive scheduling
behavior of [16] makes this task continue its execution until completion. At
time 5 when $\tau_{2}$ finishes, the device shuts down to accumulate enough
energy to run $\tau_{1}$. This makes the first job of the high-priority task
$\tau_{1}$ complete its execution after $t=40$s, resulting in a deadline miss
and subsequently skipping its second job.

In the case of CARTOS, both tasks meet their deadlines as shown in Fig. 9(b).
Even though the low-priority task $\tau_{2}$ begins execution at $t=0$, it is
soon preempted by the high-priority task $\tau_{1}$, allowing $\tau_{1}$ to
meet its deadline. The execution of the job of $\tau_{2}$ spans across
multiple power cycles and it finishes at around $t=163.5$s, which is before
its deadline. This result demonstrates the effectiveness of our mixed-
preemption scheduling especially when long-running computational tasks are
involved.

(a) SOTA [16]

(b) CARTOS

Figure 9: Task execution behavior of the proposed method compared to the [16]
method

###  V-C Analytical Experiments

In this section, we compare the schedulability of the SOTA real-time IPD
scheduler [16] and our work. The experiments are conducted in MATLAB on a
workstation equipped with an Intel 4.2GHz Core i7 CPU with 16GB of RAM. Since
the SOTA method does not support the analysis of processing chains, we only
generated single-task chains for comparison. Therefore, we refer to the
generated chains as tasks in the rest of this section.

To determine the execution time of each task for our experiments, we choose
the period randomly from 1s to 60s, find the task’s utilization using the
UUniFast method [38], and then multiply this utilization by the task’s period
and round the result to the nearest positive value with one decimal point
accuracy. In simpler terms, we determine the task execution time as
$C_{i}=\max(\lfloor 10\cdot T_{i}\cdot U_{i}\rceil/10,0.1)$.

We begin by assessing the impact of discharging rates on schedulability. This
experiment involves categorizing tasks into two distinct groups: those with
high energy demands and those with low energy demands. For the high-energy
demand tasks, the discharging rate is selected randomly from the range of 8 to
10, while for the low-energy demand tasks, it is chosen from 1 to 3. We
introduce variations in the proportion of low-energy demand tasks, ranging
from 0 to 100%. For each configuration of the low task demand ratio, we create
1000 tasksets and calculate the average schedulability rate across these sets.
In this experiment, the number of tasks in a taskset is set to 5 and the
utilization of the taskset is randomly selected from a range of 0.1 to 0.9.
The atomicity of each task (i.e., if the task is preemptable or non-
preemptable) is also selected randomly. The priorities of tasks for each
taskset are assigned based on the Rate-Monotonic method (i.e., tasks with
lower periods get higher priority). The charging rate is fixed to 3, and
implicit deadlines are considered, i.e., $D_{i}=T_{i}$.

Fig. 10 shows that CARTOS outperforms SOTA by up to 20% of schedulability.
This is mainly due to the unnecessary blocking time caused by lower-priority
tasks in SOTA which treats all tasks non-preemptable even if some are fully
computational tasks. The gap between the two methods increases with the number
of low-energy demand tasks because the negative impact of the blocking time
outweighs the energy constraint.

Figure 10: Scheduler performance with various discharging rates

Next, we evaluate the impact of utilization on taskset schedulability. In this
experiment, the taskset utilization is chosen from 0.1 to 0.9 with 0.1 steps.
For each taskset utilization, 1000 tasksets are generated and the average
schedulability ratio of the total 1000 tasksets is reported. The number of
tasks, period, and discharging rate of each task are chosen randomly from 3 to
8, from 1s to 60s, and from 1 to 10, respectively. We use the same method as
the previous experiment to calculate the execution time for each of the tasks
within a taskset.

Fig. 11 shows the results. The schedulability improvement of CARTOS over SOTA
tends to increase with the taskset utilization although the gap is smaller
than in the previous experiment. From these results, we conclude that the
energy constraint is indeed the major factor contributing to the
schedulability performance in IPDs and the benefit of our mixed-preemption
approach will be tremendous when the system has long-running computational
tasks along with peripheral tasks.

Figure 11: Scheduler performance with various utilization rates

##  VI Conclusion

In this paper, we introduced CARTOS, a charging-aware real-time operating
system for intermittently-powered betteryless devices. CARTOS leverages a
mixed-preemption scheduling model to facilitate correct and efficient
execution of periodic sensing applications, comprised of a chain of
computational and peripheral tasks. For computational tasks, it schedules non-
preemptively while ensuring forward progress through just-in-time (JIT)
checkpointing. For peripheral tasks, it guarantees atomicity by scheduling
them non-preemptively and charging sufficient energy to complete their jobs
before execution. Furthermore, CARTOS includes energy management that computes
the required charting time and utilizes a machine-learning-based energy
predictor to update the harvesting rate to adapt to changing environmental
conditions. We evaluated CARTOS through real system experiments as well as
schedulability experiments. Our results showed that CARTOS can successfully
schedule periodic chains of tasks, while state-of-the-art fails to do so.
Furthermore, CARTOS outperformed state-of-the-art by up to 20% in
schedulability across various experimental settings.

For future work, we plan to expand the capabilities of CARTOS to accommodate a
more diverse set of applications. This encompasses task synchronization and
locking primitives, integrating with multi-core microcontrollers, and managing
multiple energy buffers, all within the context of real-time operating systems
and batteryless devices.

## References

  * [1] B. Denby and B. Lucia, “Orbital edge computing: Nanosatellite constellations as a new class of computer system,” in _Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems_ , 2020, pp. 939–954.
  * [2] J. Choi, L. Kittinger, Q. Liu, and C. Jung, “Compiler-directed high-performance intermittent computation with power failure immunity,” in _2022 IEEE 28th Real-Time and Embedded Technology and Applications Symposium (RTAS)_. IEEE, 2022, pp. 40–54.
  * [3] J. Hester and J. Sorber, “The future of sensing is batteryless, intermittent, and awesome,” in _Proceedings of the 15th ACM conference on embedded network sensor systems_ , 2017, pp. 1–6.
  * [4] K. Maeng and B. Lucia, “Adaptive Dynamic Checkpointing for Safe Efficient Intermittent Computing,” in _OSDI_ , 2018.
  * [5] A. Mirhoseini, E. M. Songhori, and F. Koushanfar, “Idetic: A High-level Synthesis Approach for Enabling Long Computations on Transiently-powered ASICs,” in _2013 IEEE International Conference on Pervasive Computing and Communications (PerCom)_ , 2013, pp. 216–224.
  * [6] B. Ransford, J. Sorber, and K. Fu, “Mementos: System Support for Long-running Computation on RFID-scale Devices,” in _ASPLOS_ , 2011. [Online]. Available: http://doi.acm.org/10.1145/1950365.1950386
  * [7] J. Hester, K. Storer, and J. Sorber, “Timely Execution on Intermittently Powered Batteryless Sensors,” in _SenSys_ , 2018.
  * [8] H. Jayakumar _et al._ , “QUICKRECALL: A low overhead HW/SW approach for enabling computations across power cycles in transiently powered computers,” in _IEEE International Conference on VLSI Design_ , 2014.
  * [9] K. Maeng and B. Lucia, “Adaptive Low-Overhead Scheduling for Periodic and Reactive Intermittent Execution,” in _PLDI_ , 2020.
  * [10] ——, “Supporting Peripherals in Intermittent Systems with Just-in-Time Checkpoints,” in _Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation_ , ser. PLDI 2019. New York, NY, USA: Association for Computing Machinery, 2019, p. 1101–1116. [Online]. Available: https://doi.org/10.1145/3314221.3314613
  * [11] FreeRTOS, “FreeRTOS - Market leading RTOS (Real Time Operating System) for embedded systems with Internet of Things extensions.” [Online]. Available: https://freertos.org/
  * [12] “$\mu$C/OS RTOS and Stacks.” [Online]. Available: https://weston-embedded.com/micrium/17-micrium/58-micrium-kernels
  * [13] “RTEMS Real Time Operating System (RTOS) — Real-Time and real Free RTOS.” [Online]. Available: https://www.rtems.org/
  * [14] A. Colin and B. Lucia, “Chain: tasks and channels for reliable intermittent programs,” in _Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications_ , 2016, pp. 514–530.
  * [15] K. Maeng, A. Colin, and B. Lucia, “Alpaca: Intermittent Execution without Checkpoints,” _Proc. ACM Program. Lang._ , vol. 1, no. OOPSLA, oct 2017.
  * [16] M. Karimi, H. Choi, Y. Wang, Y. Xiang, and H. Kim, “Real-Time Task Scheduling on Intermittently Powered Batteryless Devices,” _IEEE Internet of Things Journal_ , vol. 8, no. 17, pp. 13 328–13 342, 2021.
  * [17] B. Islam and S. Nirjon, “”scheduling computational and energy harvesting tasks in deadline-aware intermittent systems”,” in _IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS)_ , 2020.
  * [18] ——, “Zygarde: Time-Sensitive On-Device Deep Inference and Adaptation on Intermittently-Powered Systems,” _Proc. ACM Interact. Mob. Wearable Ubiquitous Technol._ , vol. 4, no. 3, 2020.
  * [19] M. Karimi and H. Kim, “Energy Scheduling for Task Execution on Intermittently-Powered Devices,” _ACM SIGBED Review_ , vol. 17, no. 1, pp. 36–41, 2020.
  * [20] V. Kortbeek, K. S. Yildirim, A. Bakar, J. Sorber, J. Hester, and P. Pawełczak, “Time-sensitive intermittent computing meets legacy software,” in _Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems_ , 2020, pp. 85–99.
  * [21] N. A. Bhatti and L. Mottola, “”harvos: Efficient code instrumentation for transiently-powered embedded sensing”,” in _Proceedings of the 16th ACM/IEEE International Conference on Information Processing in Sensor Networks_ , 2017, pp. 209–219.
  * [22] Y. Kim, Y. Lim, and C. Lim, “Liveness-aware checkpointing of arrays for efficient intermittent computing,” in _2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2023, pp. 1–6.
  * [23] K. S. Yildirim, A. Y. Majid, D. Patoukas, K. Schaper, P. Pawelczak, and J. Hester, “InK: Reactive Kernel for Tiny Batteryless Sensors,” in _SenSys_ , 2018.
  * [24] A. Colin, E. Ruppel, and B. Lucia, “A Reconfigurable Energy Storage Architecture for Energy-harvesting Devices,” in _ACM SIGPLAN Notices_ , vol. 53, 2018.
  * [25] A. Colin and B. Lucia, “Termination Checking and Task Decomposition for Task-Based Intermittent Programs,” in _Proceedings of the 27th International Conference on Compiler Construction_ , ser. CC 2018. New York, NY, USA: Association for Computing Machinery, 2018, p. 116–127.
  * [26] D. Balsamo, A. S. Weddell, G. V. Merrett, B. M. Al-Hashimi, D. Brunelli, and L. Benini, “Hibernus: Sustaining Computation During Intermittent Supply for Energy-Harvesting Systems,” _IEEE Embedded Systems Letters_ , vol. 7, no. 1, pp. 15–18, 2015.
  * [27] M. Karimi, Y. Wang, and H. Kim, “Energy-Adaptive Real-time Sensing for Batteryless Devices,” in _2022 IEEE 28th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA)_ , 2022, pp. 205–211.
  * [28] H. R. Mendis, C.-K. Kang, and P.-c. Hsiu, “Intermittent-Aware Neural Architecture Search,” _ACM Trans. Embed. Comput. Syst._ , vol. 20, no. 5s, sep 2021. [Online]. Available: https://doi.org/10.1145/3476995
  * [29] J. Hester, L. Sitanayah, and J. Sorber, “Tragedy of the Coulombs: Federating Energy Storage for Tiny, Intermittently-Powered Sensors,” in _Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems_ , ser. SenSys ’15. New York, NY, USA: Association for Computing Machinery, 2015, p. 5–16. [Online]. Available: https://doi.org/10.1145/2809695.2809707
  * [30] H. Jayakumar, A. Raha, J. R. Stevens, and V. Raghunathan, “Energy-Aware Memory Mapping for Hybrid FRAM-SRAM MCUs in Intermittently-Powered IoT Devices,” _ACM Trans. Embed. Comput. Syst._ , vol. 16, no. 3, apr 2017. [Online]. Available: https://doi.org/10.1145/2983628
  * [31] K. Wardega, W. Li, H. Kim, Y. Wu, Z. Jia, and J. Hu, “Opportunistic communication with latency guarantees for intermittently-powered devices,” in _2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2022, pp. 1093–1098.
  * [32] J. Hester, N. Tobias, A. Rahmati, L. Sitanayah, D. Holcomb, K. Fu, W. P. Burleson, and J. Sorber, “Persistent Clocks for Batteryless Sensing Devices,” _ACM Trans. Embed. Comput. Syst._ , vol. 15, no. 4, aug 2016. [Online]. Available: https://doi.org/10.1145/2903140
  * [33] H. Choi, M. Karimi, and H. Kim, “Chain-Based Fixed-Priority Scheduling of Loosely-Dependent Tasks,” in _2020 IEEE 38th International Conference on Computer Design (ICCD)_ , 2020, pp. 631–639.
  * [34] H. Sobhani, H. Choi, and H. Kim, “Timing Analysis and Priority-driven Enhancements of ROS 2 Multi-threaded Executors,” in _2023 IEEE 29th Real-Time and Embedded Technology and Applications Symposium (RTAS)_. IEEE, 2023, pp. 106–118.
  * [35] H. Kim, S. Yi, W. Jung, and H. Cha, “A decentralized approach for monitoring timing constraints of event flows,” in _2010 31st IEEE Real-Time Systems Symposium_. IEEE, 2010, pp. 327–336.
  * [36] C. Delgado and J. Famaey, “Optimal energy-aware task scheduling for batteryless iot devices,” _IEEE Transactions on Emerging Topics in Computing_ , vol. 10, no. 3, pp. 1374–1387, 2021.
  * [37] J. Regehr, “Scheduling tasks with mixed preemption relations for robustness to timing faults,” in _23rd IEEE Real-Time Systems Symposium, 2002. RTSS 2002._ IEEE, 2002, pp. 315–326.
  * [38] E. Bini and G. C. Buttazzo, “Measuring the performance of schedulability tests,” _Real-Time Systems_ , vol. 30, no. 1-2, pp. 129–154, 2005.