# Adaptive User-Oriented Direct Load-Control of Residential Flexible Devices

Davide Frazzetto  0000-0002-5809-8050 Aalborg University<EMAIL_ADDRESS>,
Bijay Neupane  0000-0001-8559-2794 Aalborg University<EMAIL_ADDRESS>,  Torben
Bach Pedersen  0000-0002-1615-777X Aalborg University<EMAIL_ADDRESS>and
Thomas Dyhre Nielsen  0000-0002-4823-6341 Aalborg University<EMAIL_ADDRESS>
###### Abstract.

Demand Response (DR) schemes are effective tools to maintain a dynamic balance
in energy markets with higher integration of fluctuating renewable energy
sources. DR schemes can be used to harness residential devices’ flexibility
and to utilize it to achieve social and financial objectives. However,
existing DR schemes suffer from low user participation as they fail at taking
into account the users’ requirements. First, DR schemes are highly demanding
for the users, as users need to provide direct information, e.g. via surveys,
on their energy consumption preferences. Second, the user utility models based
on these surveys are hard-coded and do not adapt over time. Third, the
existing scheduling techniques require the users to input their energy
requirements on a daily basis. As an alternative, this paper proposes a DR
scheme for user-oriented direct load-control of residential appliances
operations. Instead of relying on user surveys to evaluate the user utility,
we propose an online data-driven approach for estimating user utility
functions, purely based on available load consumption data, that adaptively
models the users’ preference over time. Our scheme is based on a day-ahead
scheduling technique that transparently prescribes the users with optimal
device operation schedules that take into account both financial benefits and
user-perceived quality of service. To model day-ahead user energy demand and
flexibility, we propose a probabilistic approach for generating flexibility
models under uncertainty. Results on both real-world and simulated datasets
show that our DR scheme can provide significant financial benefits while
preserving the user-perceived quality of service.

##  1\. Introduction

The uncertainty in the power supply due to fluctuating Renewable Energy
Sources (RES) has severe implications (financial and others) for energy market
players. Traditional solutions such as the curtailment and the use of costly
auxiliary services that market players utilize for compensating deviations
between supply and demand lead to loss of revenue. ††To appear in the
Proceedings of the e-Energy 2018, ninth ACM International Conference on Future
Energy Systems (ACM e-Energy 2018)  Several smart grid projects have already
addressed this problem, aiming at mitigating the effects of imbalances due to
the integration of RES, by proposing Demand Response (DR) strategies for load
shifting control (Paetz et al., 2013; Kobus et al., 2015; Labeeuw et al.,
2015). In this regard, the concept of utilizing the flexibility in household
energy demand to dynamically balance the available RES with the energy load is
one of the most promising. The goal is to capture the shiftable portion of
energy as flexible consumption descriptions, so-called flex-offers (proposed
in MIRABEL(mir, 2013; Boehm et al., 2012; Šikšnys et al., 2015; Šikšnys and
Pedersen, 2016; Valsomatzis et al., 2016, 2015)), at the individual device
level to obtain the highest resolution for flexibility control (Neupane et
al., 2017). In (Neupane et al., 2014; Neupane et al., 2017; Neupane et al.,
2015) it has been shown that residential household devices such as
dishwashers, washing machines, refrigerators, electric heating, heat pumps,
and electric vehicles present opportunities for high flexibility at low (user)
cost. In this paper we focus on wet devices, such as dishwashers and washing
machines, as they alone account for approximately 30$\%$ of the total energy
consumption in residential households (Neupane et al., 2014).

Recent studies have already researched how to exploit the flexibility
potential of household devices for different objectives, e.g., increase smart
grid capacity, maximize use of RES, or decrease energy costs, via Demand Side
Management (DSM) programs based on user incentive design or dynamic
pricing(Deng et al., 2015). The former explores the possibility to shift the
time and amount of load consumption to reduce the load at peak hours (Klaassen
et al., 2016; Paterakis et al., 2015; Adika and Wang, 2014; Kobus et al.,
2015). In the latter (Li et al., 2011; Ma et al., 2016; Astaneh et al., 2015),
instead of directly controlling the devices, the proposed programs indirectly
encourage the users to change their energy usage schedules according to a
dynamic pricing mechanism, as users are prone to use less electricity when the
prices are higher. Most of the proposed demand shifting based DR schemes
require the availability of accurate real-time and predicted future
information on energy requirements, e.g., next 24 hours estimated energy
needs, maximum device flexibility, device usage preferences/priorities, and
manual device operation scheduling. This type of information has to be
provided directly by the user via surveys or smart applications and has been
shown to be too taxing for most users. In a user survey regarding home heating
automation (Jensen et al., 2016) it was reported that ”[Users] do not want to
sit and regulate the heating every night”, and in (Yang et al., 2014) the
authors discuss that because of time consuming user interaction, users do not
use manual heating schedules and fail to reassess existing control patterns.
As a consequence, it has been reported that users tend to drop out of more
complex DSM programs in favor of default non-automated flat rate energy plans—
a tendency denoted as user response fatigue (Kim and Shcherbakova, 2011).
Since energy consumption has historically been a passive purchase routine, the
more interaction the user is required to perform, the higher the chances the
users will abandon DSM programs.

Regarding the estimation of day-ahead device-level load, there are a number of
ongoing research projects looking at the possibility of utilizing device-level
load data for dynamic DR (Gottwalt et al., 2017; Develder et al., 2016; James
et al., 2016); even so, an efficient method for predicting device usage
patterns and the associated energy demand is still missing as the
unpredictability of the user behavior creates challenges in achieving higher
accuracy for device-level demand forecasting. Further, predictive models for
estimating flexibility in device usage, in order to devise an effective
schedule for demand deferral, have been only partially addressed or still
remains unexplored. In (Reinhardt et al., 2014) the authors narrow the scope
to predicting the deactivation times of currently operating devices. This
approach conflicts with the prediction horizon requirement of flexibility
based DR, which requires beforehand flexibility information for efficient
scheduling of supply and demand. Flexibility analysis based on the predicted
demand has already been the focus of earlier work (Zhou et al., 2016; Humeau
et al., 2013). However, the proposed models aim at predicting aggregated
household demands, rather than individual devices. Again, we emphasize that
identification of device-level flexibility is paramount both to provide
efficient load shifting DR schemes and to understand the end-user device usage
behavior.

With respect to modeling and evaluation of user flexibility and scheduling of
device flexible demand, we argue that direct load-control strategies should
consider the user perceived loss of quality of service induced by a forced
shifting in time of a device operation. Most of the existing research
(Vardakas et al., 2015; Deng et al., 2015; Ma et al., 2016; Tsui and Chan,
2012; Setlhaolo et al., 2014; Samadi et al., 2012; Fahrioglu and Alvarado,
2001; Adika and Wang, 2014) has focused on hard-coded approaches, independent
from the specific user-device behavior. Furthermore, current research has
solely focused on modeling the level of satisfaction obtained by the user as a
function of the amount of energy consumption, rather than the time of
consumption, which is necessary to apply time-based scheduling techniques. In
(Jensen et al., 2018) the authors perform a qualitative user study that
analyzes the wet-devices’ usage behavior under a load shifting program. The
study concludes that the users were not only able to shift their consumption
to obtain financial benefits, but were also very willing to adapt to the new
conditions. Nevertheless, the study involves heavy user-machine interaction,
with no consideration for the risk of user response fatigue.

As an alternative, in this paper we propose a data-driven model for online
estimation of the user utility as a function of device usage patterns. In
order to minimize the burden of user-machine interaction, we propose solutions
for estimating day-ahead device-level load consumption, modeling user
flexibility, evaluating user preference for device operation, and prescribing
a demand schedule that satisfies the user requirements. The contribution of
the paper can be summarized as:

  1. (1)

We present data-driven models for estimating user utility which significantly
reduce the requirement of demanding user interaction and the threat of user
response fatigue.

  2. (2)

We propose a novel user-oriented direct-load scheme for scheduling of
predicted flexible demand that considers both social and financial aspects of
demand shifting.

  3. (3)

We present a novel method for modeling device-level flexibility under
uncertainty.

  4. (4)

We perform experiments on both real-world and synthetic device load datasets,
and on a real energy market dataset.

  5. (5)

The experimental results show that our technique yields to an optimal trade-
off between financial benefits and user-perceived quality of service, with up
to $32\%$ and $38\%$ savings in the Spot and Regulation Market.

The remainder of the paper is organized as follows. Section 2 discusses the
flexibility concept and the proposed DR scheme for scheduling of flexible
demand. Section 3 presents our approach for device-level activity prediction
and how to model flexibility and uncertainty as probabilistic flex-offers.
Section 4 describes adaptive user flexibility estimation model. Section 5
presents our scheme for user-oriented flexible demand scheduling. Section 6
presents experimental results and analysis. Finally, Section 7 concludes the
paper and provides directions for future work.

##  2\. Direct Load-Control of Flexible Devices

A DR scheme for direct-load control of flexible household devices involves
multiple phases. Here we describe the workflow of our proposed DR scheme, and
the modeling of flexibility into flex-offers.

###  2.1. DR Scheme Workflow

Our proposed DR scheme focuses on day-ahead demand deferral of flexible high-
demand wet-devices. We choose a day ahead approach as it gives a better
opportunity to avoid inbalances in the energy market (Deng et al., 2015).
Starting with a practical example of our proposed scheme we consider the
scheduling of a dishwasher $\mathcal{D}$, and through the example we will
introduce the major concepts of our proposed scheme that will be further
examined in the next sections.

Daily, we predict which devices will be operated during the next day, the
amount of demand needed by the operation, and the time in which this demand
will be generated. The dishwasher $\mathcal{D}$ is predicted to operate at
hour $h$ of day $d$. In this case, its flexibility is extracted and a schedule
for deferring its time of operation (within the defined flexibility interval)
is produced such that it fulfills market and user requirements. Now, the user
is informed directly via a mobile app, for example, of the maximum time for
preparing the dishwasher ready to operate to comply with the proposed
schedule. A user can always verify the proposed schedule using an application
and at any time override the schedule if so desired. In this case, the user
receives no financial benefit. If the user accepts the schedule and prepares
it for the operation (fills it with dishes and marks it ready) no further
interaction is required and the device operation takes place as scheduled
unless interrupted by the user as described above. If the schedule is carried
out successfully, the user receives a financial reward that depends on the
amount of benefit the device flexibility provides to the energy market.

Our DR scheme is based on financial incentive-based scheduling of device
operations while ensuring that users still have control over their devices.
Compared to the related work (Klaassen et al., 2016; Paterakis et al., 2015;
Samadi et al., 2012; Ma et al., 2016; Labeeuw et al., 2015) in which the users
are required to directly provide information on their device and scheduling
preferences, our proposed method requires fewer and simpler user interactions
(e.g., only passive feedback from the device or smartphone via push
notifications) and manual overriding of a proposed schedule. It is important
to note that complete elimination of user interaction is not only difficult to
achieve but also undesirable as it has been shown that users want to feel in
control of their devices and that feedback can help them understand the
effects of home automation (Yang et al., 2014; Raptis et al., 2017).
Nevertheless, reducing the burden of taxing user interaction can lower the
risk of user response fatigue. In case of more traditional devices that cannot
be automated, our proposed method can suggest the user optimal times of
operation for a device. For example, in case of traditional dishwasher, the
only required user interaction is to set the suggested delay for the device’s
operation at the moment of filling it with dishes.

###  2.2. Flexible Operation Modeling

In our DR scheme, we consider a smart device whose operations can be
externally controlled. The process describing the operation of a device
includes various steps. For example, if we consider the operation of a
dishwasher, the process starts with a user preparing and consuming her meal,
then loading the dishwasher with the used dishes, and finally activating the
device to perform the cleaning task. Hence, we can abstract the usage of the
device into the prepare process, leading to the need of the device, e.g.,
using the dishes, the ready process, started by the ready action, e.g. loading
the device and setting it ready to operate, and the activation process,
started by the activate action, leading to the actual operation of the device.
Our DR scheme is based on two assumptions on the relationship between these
processes:

###### User Assumption 1 (Device Usage Independence).

The timestamps of the _prepare_ and ready processes of a device are
independent of the timestamp of performing an _activate_ action on the same
device.

This assumption captures a realistic user behavior of performing a task based
on their requirements and preferences rather than based on external influence,
i.e., users would not change their daily routine to cope with the market-
induced schedule. For example, users always prefer to prepare their meals
irrespectively of the time for cleaning the dishes. Based on this assumption,
we can consider the normal user behavior, i.e., the prepare and ready
processes, to not be affected by the scheduling of the device.

###### User Assumption 2 (Schedule Delay).

A user would accept a device to be activated after it is ready for operation,
but not before.

We assume that a user would not easily change their preference of performing
the prepare process, e.g., change meal time to comply with our schedule.
Therefore, we have no control over the prepare and ready actions, and the
scheduler can only influence the time of the activate process without
interfering with the user-preferred time of the other two processes.

Figure 1. a) Non-flexible/non-controlled and b) flexible/controlled scenario.

Figure 2. Flex-offer model for an activation/deactivation event of a device

Figure 2 shows a comparison between the operation of a device in a non-
flexible/non-scheduled scenario (a), and in our proposed DR scheme (b). On the
one hand, in (a) each pair of ready and activate actions coincide, e.g., when
a user loads and immediately starts a dishwasher, as the device provides no
flexibility. On the other hand, in (b), an activate action can be delayed
within its flexibility interval, i.e., as long as it does not interfere with
the prepare process that triggers the need of the device.

Our DR scheme relies on the availability of the timestamps of ready actions.
In historical data, as the one we have used in this work, ready actions are
represented by the timestamps of the device activation (assuming non-direct
load-control) and in an online setup, the ready actions represent the
timestamps when user loads and marks ready their devices following normal
behavior. Since the prepare process is user specific and may occur anytime
between two ready actions, we will only consider ready and activate actions
for the proposed flexibility-based DR scheme.

###  2.3. Flexibility as Flex-Offers

We define the flexibility as the potential to amend the energy profile and
demand deferral potential of a device, and we represent this flexibility as a
so-called flex-offer. The demand deferral potential, i.e. time flexibility,
represents the range between the _earliest start time_ ($t_{es}$) and the
_latest start time_ ($t_{ls}$) for an operation (activate action) of a device.
For example, if we consider a single usage of a dishwasher, the time
flexibility represents the earliest and latest time the dishwasher can be
activated in order to fulfill a user’s task. In more general terms, a flex-
offer can define the boundaries within which future energy demands from a
device can be scheduled.

###### Definition 2.1.

_A flex-offer $f$ is a tuple $\mathit{f=([t_{es},t_{ls}],\rho)}$, where
[$\mathit{t_{es}}\mathit{t_{ls}}$] is the time interval during which the
activate action can be shifted, $\rho=\mathit{\left\langle
s_{1},\dots,s_{d}\right\rangle}$ is the energy profile for the activation,
$s_{i}=\mathit{[\mathit{e_{min},e_{max}}]}$ is a continuous range defined by
the minimum $\mathit{e_{min}}$ and maximum $\mathit{e_{max}}$ energy
boundaries, and $\mathit{d}$ is the number of slices in $\rho$. _

The latest end time of the device operation is calculated as
$t_{ls}=t_{ls}+d$. Although a flex-offer represents both time and amount
flexibility, the paper mainly focuses only on the time dimension of flex-
offers, as our target wet-devices do not allow for amount deferral, but only
for time shifting, such that $e_{min}=e_{max}$. In Figure 2 we show an example
of flex-offer with $t_{es}=$ 10:00 and $t_{ls}=$ 20:00, in which we have a 10
hours of time flexibility range for rescheduling the device activate event.

To conclude, a higher integration of RES into the grid system and
electrification of user appliances (heat pump, electric vehicles) increases
uncertainty in demand and supply, creating greater DR challenges for market
players such as Balance Responsible Parties (BRPs) and Distribution System
Operators (DSOs). Flex-offers can be a valuable asset to the energy market
players, where BRPs could utilize flex-offers to schedule demands that
minimize their market deviations. Similarly, DSOs can use the flex-offer
scheduling options to analyze the distribution of flexible loads within their
grid system and assess the possibility of deferring expensive grid upgrades.
The financial advantage obtained by the market players can then be shared with
their customers. This way, end-users provide flexibilities to the market
players, who can exploit it for financial benefits and share some portion of
the benefits to the end-users. Further, One of the main objectives of modeling
flexibility as a flex-offer is to have a generalized object capable of
capturing and modeling flexibility from a variety of different devices.

##  3\. Prediction of Flex-Offers

In this section, we discuss the process of predicting device load consumption
to generate the device’s flexibility. We first describe the workflow of
predicting device activity, and then we define an extension of flex-offers
that takes uncertainty into account.

###  3.1. Device Activity Prediction

The first step in the generation of flexibility is to predict the device’s
activity, in terms of future ready actions. To do so, we first need to
identify in which day the device will be operated, the time of the ready
action during that day, and the time until the next ready action. The data we
utilize for this process is load consumption time series at the device level.
This type of data can be obtained directly from the device, e.g. with Smart
Plugs, or by disaggregating Smart Meter data into individual device time
series (Hosseini et al., 2017).

First, we transform the load time series into an event time series describing
only the time of ready actions. This is done by replacing the load consumption
with the initial timestamp of each device operation. To identify device
operations while maintaining the associated load/duration information, we
abstract a device operation into a device signature, representing a specific
device operation in terms of both duration and energy consumption for time
point of operation. A device signature is defined as:

(1) $\sigma=[e_{1},e_{2},\ldots,e_{k}]$

where $e_{i}$ is the load demand per hour, and $k$ the (average) operation
length in hours. Given the event time series, the activations are used to
extract the average device operation duration and load demand per hour of
operation. First, the energy demand per hour $e_{i}$ is estimated by averaging
the hourly energy demand over all the activations in the event time series.
Second, the operation length $k$ is extracted as the ceiling of the average
operation duration of each activation in the event time series.

Figure 3. Comparison of device signature and real activations in Hours (left)
and energy consumption (right).

In Figure 3 we show how the extracted device signatures resemble the actual
device operations, in terms of the difference in duration between real
activations and signature (left) and difference of total operation demand
between real activations and signature (right) for all the devices in our
datasets. It is possible to see that, given the short inter-quartile range in
both plots, most the operations do not significantly differ from the device
signature, allowing us to use the device signature as a mean to simplify the
device load forecasting, without compromising the quality of the predictions.
Although some outliers stretch beyond the upper and lower whiskers, they are
less than $5\%$ of the total number of activations.

Second, we use the activation time series to perform a day-level
classification of whether the device will be operated during the next day. If
a day is predicted to present an activation with a probability higher than
$50\%$ (more information on the prediction models used in Section 6.1), we
also perform two hour-level predictions of the hour of the next two
consecutive ready actions during that (or the next) day, in order to generate
the earliest start time, first ready action, and latest end time, second ready
action, of a flex-offer.

Let $T_{es}$ and $T_{le}$ be two random variables representing the time of the
earliest start time and latest end time, respectively. Given that load time
series are usually discrete, with the granularity of the sampling frequency or
of the chosen aggregation level, e.g., 1 hour, we can also assume $T_{es}$ and
$T_{le}$ to be discrete.

We first predict the probability of the time of the earliest start time
$P(T_{es}|e)$, given an evidence set $e$, e.g., calendar information, day,
month, week, etc. Then we predict

(2) $P(T_{le},T_{es}|e)=P(T_{le}|T_{es},e)P(T_{es}|e),$

as the probability of the time of the earliest start time and the latest end
time, conditioned by the probability of $T_{es}$. Similarly, the probability
$P(T_{le})$ can be defined as

(3) $P(T_{le}|e)=\sum_{T_{es}}P(T_{le}|T_{es},e)P(T_{es}|e).$

###  3.2. Probabilistic Flex-Offer

A flex-offer can be used to model the device’s flexibility in terms of
earliest start time and latest end time, as described in Section 2.3. In our
case, when modeling future flexibility from predicted device activity,
earliest start time and latest start time are represented by the prediction
over the two random variables $T_{es}$ and $T_{le}$. The definition of flex-
offer given in Section 2.3 can only represent the range between two individual
time points, rather than random variables. To overcome this limitation, we
propose an extension to the standard flex-offer defined as a Probabilistic
flex-offer.

###### Definition 3.1.

_A probabilistic flex-offer $f$ is a tuple
$\mathit{f=\langle[T_{es},T_{le}],}$ $\mathit{\rho\rangle}$, where $T_{es}$
and $T_{le}$ are two discrete random variables describing the earliest start
time and the latest end time respectively, and $\rho$ is, as in a standard
flex-offer , the energy profile of the activation. $[T_{es},T_{le}]$ defines a
set of $|T_{es}|\times|T_{le}|$ possible flexibility intervals in which the
activate action can be shifted, described by the tuple
$\langle[t_{es},t_{le}-|\rho|],P(T_{es}=t_{es},T_{le}=t_{le})\rangle$, where
$t_{es}\in T_{es}$, $t_{le}\in T_{le}$, $|\rho|$ is the length of the
operation, and $P(T_{es}=t_{es},T_{le}=t_{le})$ is the interval probability
defined in Eq. 2._

A probabilistic flex-offer considers not just a single flexibility interval
between the earliest and latest start times, but all the intervals
$[t_{es},t_{ls}]$, with $t_{ls}=t_{le}-|\rho|$ between the time points
described by the distributions over earliest start time and latest end time.
Each of these intervals is also associated with the probability of such an
interval to be correct, i.e. to accurately describe the actual interval
between two consecutive device ready actions.

Figure 4 shows an example of the result of the prediction process for
$P(T_{es}|e),P(T_{le}|e)$, showing the (truncated normal) probability
distributions, with a granularity of 1 hour, of the random variables over the
time ranges [7,13] and [17,23]. One of the possible flexibility intervals is
[11,19]. In Section 5 we will describe how to make use of probabilistic flex-
offers in order to more accurately schedule flexible demand.

Figure 4. Probabilistic flex-offer, the example shows one of the possible
intervals between $T_{es}$ and $T_{ls}=T_{le}-|\rho|$, in this case [11,19].

##  4\. Adaptive User Utility Modeling

In this section, we describe our proposed model for data-driven user utility
modeling. We describe how to model the acceptance of a user towards a schedule
as _user utility_ , a combination of financial benefits and user flexibility.
Then, we show how to adaptively estimate the user flexibility directly from
device-level load consumption data.

###  4.1. User Utility

In our proposed DR scheme, device schedules need to be (at least implicitly)
approved by the owners. If the proposed schedules do not fulfill the users’
requirements, users can actively diverge from them, resulting in a loss of
(financial) opportunities for all the involved parties. So far we have modeled
flexibility as a flex-offer. However, the flexibility interval described in
such a model is an over-estimation of the actual user flexibility, as it only
describes the time between which the device is needed ready to operate. As an
example, a user might not need to utilize the dishwasher until the day after,
but the result of the dishwasher operation, i.e. clean dishes, is probably
needed before that time. Therefore, understanding and respecting the user
flexibility beyond what modeled so far in a flex-offer becomes paramount.

Each user is assumed to take decisions independently from the other users, the
energy market, or the smart grid requirements. Specifically, residential users
may follow a purely self-interest based utility in their energy consumption.
This utility can be modeled as a combination of financial interests, e.g.
lower energy prices, and device interests, the value of the operation of a
device at a certain time of the day (Samadi et al., 2012). Hence, each user
may possess a different utility function, resulting from different
combinations of these two factors. As an example, a user might be more
interested in the financial interests than in the device interests, thus not
being concerned about the time of operation of her dishwasher as long as she
is obtaining the highest financial benefit from the device operation.
Alternatively, the same user might have a different utility if the scheduled
device is a washing machine. Therefore, user utility is a function of the
specific user-device pair, the time of the schedule, and the financial
interests resulting from such a schedule.

Following this intuition, we start by introducing an assumption of our
proposed user utility model:

###### User Assumption 3 (User Utility).

Users are flexible in regards to their devices being rescheduled in return for
financial benefits, as long as the schedule matches their preferred device
behavior. Their acceptance of the device schedule is positively correlated to
their financial interests and negatively correlated to the amount of delay
induced by the schedule.

Intuitively, a user would, all else being equal, maintain control over the
usage of the devices, i.e., to not reschedule the activations. However,
according to our user utility assumption, we can represent the scheduling of a
device activation as a trade-off between financial interests, providing higher
flexibility, and device interests, constraining to lower flexibility. We will,
therefore, define the expected User Utility, for a user $u$ and a device $d$,
whose operation $o$ starting at time $t_{es}$ has been scheduled to time $t$,
as

(4) $\begin{split}\mathbb{E}[U_{u,d}(t|t_{es},t_{le},A)]&=G(A=a,t,t_{es})\cdot
P(A=a|t,t_{es},t_{le})\\\ &+G(A=r,t,t_{es})\cdot
P(A=r|t,t_{es},t_{le})\end{split}$

where $G(A=a,t,t_{es})$ is the financial benefit obtainable when the proposed
schedule is accepted, $A=a$, by delaying the operation of $o$ from $t_{es}$ to
$t$, while $G(A=r,t,t_{es})$ is the financial benefit that would have been
obtained by the rejected schedule, $A=r$. $P(A|t,t_{es},t_{le})$ is the
probability that the user would either accept, $P(A=a)$, or reject, $P(A=r)$,
a schedule, with regards to the delay $t-t_{es}$ in the interval
$[t_{es},t_{le}]$, and user available flexibility. We can see the financial
benefit $G(A,t,t_{es})$ as the savings obtainable by the lower energy price of
the operation starting at $t$, compared to the original start time $t_{e}s$,
$G(t,t_{es})=Price(o,t)-Price(o,t_{e}s)$

When the proposed schedule is rejected we assume that the earliest start time
is kept as operation starting time, i.e. $t=t_{es}$ and the user receives no
financial benefit, thus $G(A=r,t,t_{es})=Price(o,t_{es})-Price(o,t_{es})=0$.
Consequently, $G(A=r,t,t_{es})\cdot P(A=r|t,t_{es},t_{le})=0$, and Eq. 4
simplifies to

(5) $\mathbb{E}[U_{u,d}(t|t_{es},t_{le},A)]=G(A=a,t,t_{es})\cdot
P(A=a|t,t_{es},t_{le})$

Here, $G(A,t,t_{es})$ depends on the financial benefits, that will be
described in Section 5.1, while $P(A|t_{es},t_{le})$ depends on the acceptance
of the user, which we model as User Flexibility.

###  4.2. User Flexibility

User flexibility can be described as the degree of acceptance of a user for a
given device schedule, or as the probability that the user does not preempt
the schedule proposed to a device operation. The objective is to understand
the user preference in order to maximize the probability
$P(A=a|t,t_{es},t_{le})$, and consequently the expected utility of Equation 5.
To do so, we resort to utilizing a model based on exponential distributions,
whose natural interpretation is such that they describe the probability
distribution of the time intervals between events in a stochastic process in
which the events, in our case the $ready$ actions, occur at a constant average
rate.

Let $T$ be a random variable describing the distance between two $ready$
events of a device $d$, and $t-t_{es}$ be the delay imposed on a device
operation $o$ by a schedule. Let us assume $t_{es}\leq t\leq t_{le}-|\rho|$,
then $P(A=a|t,t_{es},t_{le})$ simplifies into $P(A=a|t,t_{es})$.

Therefore,

(6) $P(A=r|t,t_{es})=P(T\leq t-t_{es}),$

where $P(T\leq t-t_{es})$ is the cumulative distribution function of the
probability that a user will need the device ready before the proposed time
$t$. Alternatively,

(7) $P(A=a|t,t_{es})=1-P(T\leq t-t_{es}),$

represents the probability that a user accepts the schedule delay. This way,
we can model user flexibility functions as the cumulative distribution
function $P(A=a|t,t_{es})$, starting by defining intuitive properties that
user flexibility functions must possess.

###### Property 4.1.

User flexibility functions are not increasing

(8) $\frac{\partial(1-P(T\leq t-t_{es}))}{\partial t}\leq 0,$

representing the partial derivative of the cumulative distribution function
with respect to the schedule delay $t$. It described how the larger the delay
$t$ induced by schedule, the smaller the probability the user would accept
such a delay.

###### Property 4.2.

When the delay $t-t_{es}$ is zero, the user acceptance is maximum

(9) $1-P(T\leq t_{es}-t_{es})>1-P(T\leq t^{\prime}-t_{es}),\quad T\geq
0,t^{\prime}>t_{es},$

meaning that $1-P(T\leq 0)=1$.

###### Property 4.3.

The probability a user accepts a schedule when the delay $t-t_{es}$ becomes
larger tends to zero

(10) $\lim_{(t-t_{es})\to\infty}1-P(T\leq t-t_{es})=0.$

Meaning that a user loses all the utility of the device if the activation is
scheduled too far from the original activation time. In our case, the
exponential distribution function possesses the 3 properties, confirming as a
good choice for describing user flexibility.

###  4.3. Adaptive Estimation of User Flexibility

The current methods for modeling user utility functions already described in
Section 1 rely on direct user interaction, such as surveys or smart
applications, for the estimation of the user utility functions. However,
direct information from the users is often unavailable, as surveys are
expensive to collect on a large scale or burden the user experience with
unwanted interaction overhead, as already shown by a large number of users
dropping out from DSM programs due to user response fatigue (Kim and
Shcherbakova, 2011). Further, user behavior can be temporary and may change
over time, invaliding survey data.

To address this issue, we propose a data-driven user flexibility model purely
based on device-level load consumption data. The objective is to estimate the
function $1-P(T\leq t-t_{es})$ defined in Eq. 7, for which we first need to
estimate the distribution of $T$, i.e. the distribution of time between two
ready events.

Let $T$ be the random variable describing the distance (in hours) between two
ready actions, following an exponential distribution. Then the cumulative
distribution function of $T$ can be defined as

(11) $F(t^{\prime},\lambda)=\left\\{\begin{array}[]{@{}ll@{}}1-\lambda
e^{-\lambda t^{\prime}},&\text{if}\ t^{\prime}\geq 0\\\
0,&\text{otherwise}\end{array}\right.$

where $t^{\prime}\in T$ is a time interval, and $\lambda$ is the rate
parameter, describing how often two activations are separated by an interval
of $t^{\prime}$. In our implementation, we generalize the $\lambda$ parameter
to be a function of multiple factors, such as day of the week, month, and
season. This way, we estimate a set of user flexibility functions rather than
a single one, thus taking into account the possibly different behavior of the
user in different time periods. For our purposes, since the user flexibility
is defined as $1-P(T\leq t-t_{es})$, we estimate the inverse of the cumulative
distribution function from the distribution of $T$, as shown in Figure 5.

Figure 5. Inverse CDF—exponential distribution of the time intervals between
ready actions of a device.

To learn the parameter $\lambda$ of the exponential distribution we use a
standard Stochastic Gradient Descent method, by fitting the inverse of the
cumulative distribution function of $T$ to the exponential distribution
function. This approach can be applied both in an offline setting, when
historical device data is available or in an online setting, where the user
flexibility function updates every time a new user feedback is available.

For each iteration $i$ of the stochastic gradient descent algorithm, the
parameter $\lambda$ is updated as in the following

(12) $\lambda=\lambda-\mu\bigtriangledown Q_{i}(\lambda),$

where $\lambda$ is the parameter to be estimated by calculating the gradient
of the objective function $Q_{i}(\lambda)$, and $\mu$ is the learning rate,
i.e. how fast the parameter is changed in each iteration. $Q_{i}$ is
associated with the i-th observation. In an online setting, a new observation
is given by the user feedback of either accepting or rejecting of the proposed
schedule delay $t-t_{es}$. In case of acceptance, the observation is the
scheduling delay $t-t_{es}$. Otherwise, the time of manual activation of the
device by the user $t_{m}$, with $t_{es}\leq t_{m}<t$.

We choose a standard convex objective function, least-squares error, that
together with an adaptive learning rate, guarantees the convergence of the
algorithm. Finally, to account for periods of times in which the user
flexibility changes more drastically, e.g. different seasons, the $\lambda$
and $\mu$ parameters can be periodically re-initialized, e.g. every month,
season, etc.

##  5\. Scheduling Device Operations

The scheduling of flexible demand of a device is based on the User Utility, a
combination of three factors. The first and the second are the financial gain
that can be achieved in the spot and the regulation markets respectively by
scheduling the device operation. The third is the User Flexibility, that we
have analyzed in the previous Section. With the User Utility, we reflect the
trade-off between maximizing financial gain and minimizing loss of user-
perceived quality of service and thus propose a schedule that optimizes the
combination of these factors. The next sections describe the financial factors
in detail, and formally define the scheduling objective.

###  5.1. Financial Model

The adaptation of the flex-offer concept and demand scheduling depends on the
financial value that flexibility can generate to all the involved parties,
i.e., the savings for market players and rewards to the consumers. The market
players can obtain saving by utilizing the demand flexibility to confront
their challenges, such as to correct their deviations in the spot market or to
avoid grid constraints, etc. However, some percentage of savings from the
market players has to be transferred to the flexibility providers
(consumers/prosumers) to maintain their commitment to the market. In this
paper, we evaluate the savings from flexibility in two different markets.
First, we evaluate the savings that can be obtained in the spot market by
scheduling demands to a low energy price area. Second, we evaluate the savings
that can be obtained by schedule demand in a way that reduces the energy
traded on the regulation (intra-day) market. The decision on timestamp to
shift the flexible demand depends on a combination of savings obtained in both
the markets (discussed in the next sections) and also the acceptance of the
new schedule (discussed in Section 4).

###  5.2. Spot Market Savings

Spot market savings describes the total financial savings of energy demands
and the corresponding flex-offers at the spot market for the predicted device
activation. To maximize this factor, a device operation is scheduled such that
the cost of purchasing the energy required for a device operation is
minimized. Let $\mathit{Spot=[\sigma^{s}(t_{es}),\dots,\sigma^{s}(t_{le})]}$
represent the hourly spot prices between the earliest start time and the
latest end time. The energy cost for each timestamp of the device operation is
calculated as the product of energy demand and the respective spot price given
by

(13) $S(x)=\sum_{i=0}^{|\rho|-1}e_{i}\cdot\sigma^{s}(x+i),$

where $|\rho|$ is the duration of the device operation $o$ starting at a
timestamp $x$ and $e_{i}$ is the demand for each operating time unit.
Therefore, the savings in spot market cost obtained by scheduling the device
activate action from $t_{es}$ to $t$ is given by

(14) $\begin{split}\Delta S&=S(t)-S(t_{es})\\\
&=\sum_{i=0}^{|\rho|-1}e_{i}\cdot\sigma^{s}(t+i)-\sum_{i=0}^{|\rho|-1}e_{i}\cdot\sigma^{s}(t_{es}+i).\end{split}$

###  5.3. Regulation Market Savings

Regulation market savings describes the total financial savings in the
regulation market that can be achieved by scheduling the device’s demand for
the predicted device activation. To maximize this factor we schedule the
operations such that the total energy traded in the regulation market is
decreased, eventually reducing the financial loss from demand and supply
imbalance.

Let $V=\langle v_{u/d}(t_{es}),\dots,v_{u/d}(t_{le})\rangle$ and
$Reg=\langle\sigma^{r}_{u/d}(t_{es}),\dots,\sigma^{r}_{u/d}(t_{le})\rangle$
represent regulation volumes and prices between EST and LET, where
$\mathit{v_{u/d}(x)}$ denotes the nonzero elements of regulating volume, i.e.,
up- or down-regulation and $\mathit{\sigma^{r}_{u/d}(x)}$ is the predicted up-
regulating power price $\sigma^{r}_{u}(x)$ in case of up-regulation and the
predicted down-regulating power price $\sigma^{r}_{d}(x)$ in case of down-
regulation. For each hour in $V$, the loss due to the market imbalance is
computed as a product of the regulation volume times the price difference
between regulating and the spot price. Hence, the total regulation cost for an
operation starting at $t_{es}$ is calculated as:

(15) $\displaystyle
R(t_{es})=\sum_{i=1}^{|\rho|-1}v_{u/d}(t_{es}+i)\cdot|\sigma^{r}_{u/d}(t_{es}+i)-\sigma^{s}(t_{es}+i)|$

where $\sigma^{r}_{u/d}(t_{es}+i)$ and $\sigma^{s}(t_{es}+i)$ are regulation
price and spot price at time $t_{es}+i$, respectively.

Given the regulation volume and the predicted device ready event and
associated demand, the market generates a device schedule, inducing a delay
$t-t_{es}$, that minimizes the regulation volumes. Let the new expected
regulation volumes be

$\bar{V}=\\{\overline{v_{u/d}(t+i)},\forall i\,|\,\overline{v_{u/d}(t+i)}\leq
v_{u/d}(t+i)\\},$

where the overbar denotes the change in regulation volume due to shifting of
flexible demand.

Finally, the savings in the regulation market cost obtained by scheduling the
device activate action from $t_{es}$ to $t$ is given by

(16) $\begin{split}\Delta R&=R(t)-R(t_{es})\\\
&=\sum_{i=1}^{|\rho|-1}\overline{v_{u/d}}(t+i)\cdot|\sigma^{r}_{u/d}(t+i)-\sigma^{s}(t+i)|\\\
&-\sum_{i=1}^{|\rho|-1}v_{u/d}(t_{es}+i)\cdot|\sigma^{r}_{u/d}(t_{es}+i)-\sigma^{s}(t_{es}+i)|\end{split}$

###  5.4. User Utility based Scheduling

The schedule of an operation $o$ described by a probabilistic flex-offer $f$
maps the initial time of the activate action $t_{es}$ to a new timestamp $t$,
inducing a delay of $t-t_{es}$ in the device operation. The new timestamp is
included between the probabilistic time flexibility interval $[T_{es},T_{le}]$
of the flex-offer, remembering also that the latest start time is defined as
$t_{ls}=t_{le}-|\rho|$.

Now, we can complete the definition of User Utility given in Eq. 5 by
replacing $G(A,t,t_{es})$ with the sum of Spot and Regulation market savings,
of Equations 14 and 16 respectively:

(17) $\mathbb{E}[U_{u,d}(t|t_{es},t_{le},A)]=(\Delta S+\Delta R)\cdot
P(A=a|t,t_{es},t_{le}).$

However, the scheduling is also dependent on $P(t_{es},t_{le}|e)$, i.e., that
the flexibility interval $[t_{es},t_{le}]\in[T_{es},T_{le}]$ is correct. To
take into account the uncertainty of the flexibility intervals, the objective
function for the scheduling of an operation $o$ can be defined as

(18)
$\mathbb{E}[t]=\sum_{t_{es},t_{le}}\mathbb{E}[U_{u,d}(t|t_{es},t_{le},A)]\cdot
P(t_{es},t_{le}|e)$

where $P(t_{es},t_{le}|e)$ is the probability of the flex-offer interval
$[t_{es},t_{le}]$ defined in Eq. 2. We remind the reader that the $T_{es}$ and
$T_{le}$ are discrete random variables, thus Equation 18 calculates a
summation over the discrete values in $[t_{es},t_{le}]$ rather than an
integration.

Therefore, the scheduling function for the operation $o$ selects the $t$ that
maximizes the expected utility $\mathbb{E}[t]$ such that:

(19) $\text{Sched(o)}=\operatorname*{argmax}_{t}\mathbb{E}[t].$

##  6\. Experiments

We performed a number of experiments to analyze the effectiveness of the
proposed DR program. First, we show the evaluation of the flex-offer
scheduling process. Second, we show how our proposed adaptive user flexibility
estimation can help preserving user comfort. Finally, we present the impact
that the prediction of probabilistic flex-offers has on the scheduling
process.

###  6.1. Experimental Setup Description

We utilize a real-world energy consumption time series at the device-level.
The time series are associated with 11 devices (6 washing-machines, 5
dishwashers), collected from 11 different households, which are available from
the open dataset Intrepid (INT, 2012). The logged granularity of the time
series varies depending on the device, ranging from 1 minute to 1 hour, and
spanning for 1 year. In this paper, we consider scheduling of flexible demand
at an hourly resolution. Hence we aggregate, by averaging, the time series
accordingly. We pre-process the time series, and use them for the generation
of probabilistic flex-offers, as discussed in Sec 3.1.

The device datasets are highly unbalanced, with an average number of
activations events ranging between $1\%$ and $7\%$ of the entire time series,
while the average number of activations per day is approximately $0.7$. Also,
load consumption time series at a device level present a high degree of
_irregularity_ , due to the dynamic behavior of the user, and the device usage
can vary due to external factors not included in the data. Hence, to evaluate
and compare the proposed scheduling methods for various user-specific
attributes such as family size, occupation, etc., we utilize a synthetic
dataset generated with the Genetx tool described in (Hoogsteen et al., 2016).
The dataset is composed of 26 devices—13 dishwashers and 13 washing machines,
each belonging to a different category of households, e.g., single worker, a
family with children, etc. Each of these devices is modeled with a dynamic
amount of flexibility, depending on the time of operation and household
category. Both the Intrepid and the Genetx datasets present similar device
usage characteristics, with comparable time between ready events for the
devices. For evaluation purposes, we split both datasets into training
($80\%$) and test set ($20\%$). The forecasting model and the user flexibility
model are trained on the training data. On the test set we perform a
prequential evaluation, with each step being a device schedule proposal, with
both the user flexibility models and the forecasting models updated at each
step, to align to a possible real application scenario.

For the financial evaluation, we use the Danish energy market dataset for DK1
price zone obtained from Energinet111
www.energinet.dk/EN/El/Engrosmarked/Udtraek-afmarkedsdata/Sider/default.aspx.
To avoid dependency of the experiments on this specific energy market dataset,
we perform multiple runs of each experiment, randomizing the energy dataset at
each iteration, finally averaging the results.

For forecasting device activity, we used a Naive Bayesian model for the day-
level prediction, and a Linear Regression model for the hour-level prediction.
We chose these two generic models as they fit the two respective problems
without specific tuning: a Naive Bayesian model for classification of which
day presents activity and a Linear Regression model for the hour of activity
prediction respectively. As this paper does not focus directly on the
challenge of Short-Term Load Forecasting at a device level, we decided to
apply the two simple models as baselines representatives of the existing
forecasting models. Both models were trained on features directly extracted
from event time series, the time such as day of the week, week, month,
weekend/weekday, and season.

###  6.2. Evaluation of Device Operation Scheduling

The acceptance of a schedule and the utility to the market players are
affected by both the accuracy of the probabilistic flex-offer (at describing
the time of device flexibility) and the modeling of the user flexibility. With
regards to the user flexibility model, the stricter the model is, i.e., a
model that always attempts to satisfy the user acceptance, the higher the
acceptance. However, at the same time, a stricter model also provides less
flexibility to the market, resulting in lower utility to both users and market
players. Therefore, we emphasize that for an efficient implementation of
flexibility-based markets, a scheduler should always perform a trade-off
between user acceptance and financial benefits.

Figure 8 shows a summary of the device operation scheduling results on both
the Intrepid and Genetx datasets. The figure shows a positive utility, in
terms of decrease in energy price, for the proposed device scheduling model.
Specifically, for the Intrepid dataset with an acceptance rate of the proposed
schedules of $64\%$, we can achieve a savings of $32\%$ and $38\%$ in spot and
regulation markets, respectively. On the other hand, in the Genetx dataset we
can achieve a higher acceptance of 79$\%$, leading to higher spot and
regulation Market cost reduction of 42$\%$ an 39$\%$, respectively. This is
due to the higher prediction accuracy that can be achieved in the (more
regularized) synthetic dataset, as we further explain in Section 6.3.

Figure 6. Scheduling results on Intrepid and Genetx datasets.

Figure 7. Effects of user flexibility models on acceptance.

Figure 8. Effects of exponential models on the absolute financial savings.

To analyze the relationship between user acceptance and financial benefits, we
show a comparison of two different user flexibility models on the scheduling
objective function shown in Eq. 17. The first is a simple uniform model that
does not distinguish between different schedules, i.e. $\forall
t,t_{es},t_{le}\quad P(A=a|t,t_{es},t_{le})=1$. Then, we apply our proposed
user flexibility model with different learning rates $\mu$ for Eq. 12 that
were found empirically, controlling how fast does our flexibility model adapts
to each new user feedback.

Fig. 8 shows the effects of the different user flexibility models on the
scheduling acceptance. The adaptive flexibility model obtains a higher
acceptance, from $48\%$ for the learning rate $\mu=0.04$, up to $64\%$ for the
learning rate $\mu=0.16$. The uniform model achieves the lowest acceptance of
$36\%$. This comes at the cost of lower financial benefits. If we do not
consider the user acceptance, i.e., every schedule is accepted, the uniform
model obtains the highest amount of savings. Fig. 8 shows the percentage of
savings, compared to savings of the uniform model, for the adaptive
flexibility models. However, the higher savings for the uniform model comes at
the cost of lower user acceptance, which entails a larger risk of user
response fatigue. On the other hand, the adaptive user-flexibility model
trade-off the savings with the acceptability ensuring efficient implementation
of the proposed DR scheme.

The results show that the more the flexibility model adheres to the user
preference, the higher the acceptance. Moreover, although our flexibility
model obtains overall lower financial benefits, a DR schema that does not take
into account the user preference will ultimately fail at being accepted, in
the long term denying all benefits to the involved parties. Ultimately, the
trade-off between the user preference over financial benefits and flexibility
can be learned online, when user feedback, in terms of their scheduling
acceptance/rejection, is available.

###  6.3. Evaluation of Flexoffer Prediction

The effectiveness of the device operation scheduling highly depends on the
quality of the predicted device activities and the generated flex-offers. If
the flex-offer models a device flexibility interval that is distant, in terms
of time or duration, from the real flexibility interval, the chances the
resulting schedule be rejected will increase. Hence, the acceptability of the
proposed schedule and the associated utility is influenced by underlying
prediction models.

In Table 1 we start by showing the results of the prediction phase. Here we
show the quality of the predictions using our proposed combination of day-
level and hour-level prediction, defined as 2-levels prediction. Moreover, we
also show the quality of the predictions obtained with a 1-level prediction,
performed by applying only the hour-level prediction (with linear regression)
and selecting the 10$\%$ activations with the associated highest probability,
yielding approximately the same number of predicted activations as the
2-levels prediction. In both datasets the 2-levels model outperforms the
simple 1-level model, leading to higher accuracy in classifying which days
will present an activation and lower error in the prediction of the hour of
activation. Additionally, in the Genetx dataset, we obtained an overall better
prediction quality that leads to the higher scheduling results shown in Figure
8, confirming a correlation between predictions and scheduling results.

Table 1. Prediction measures for day-level prediction (Day Accuracy) and hour-level prediction (RMSE). | 2-levels Prediction | 1-level Prediction
---|---|---
Day Acc. | Hour RMSE | Day Acc. | Hour RMSE
Intrepid | 66% | 4.1 | 48% | 6.88
Genetx | 80% | 3.36 | 54% | 6.78

Figure 9. Discounted effect of predictions on acceptance.

Figure 10. Discounted effect of predictions on savings.

Figure 11. Comparison of standard flex-offers and probabilistic flex-offers.

To evaluate the exact effect of the prediction models on the final scheduling
results, we compare both financial savings and acceptance in two different
scenarios. The first is the predicted scenario, where day and hour of the
earliest start time of a flex-offer is predicted by our prediction models. The
second is the ideal scenario, where we assume we assume the start time of a
flex-offer is already known. In both scenarios, we introduce time flexibility
by manually setting the latest start time of each flex-offer between 0 and 22
hours after the earliest start time. Again, in our experiments, we have used
two simple forecasting models, Naive Bayesian and Linear regression model for
day-level and hour-level predictions, respectively. We want to emphasize that
short-term load forecasting is a largely researched field, and work on highly
accurate forecasting model is outside of the scope of this paper.

Figure 11, compares the user acceptability of the prescribed schedule for
ideal and predicted scenarios at various time flexibility. The figure
demonstrates that at a lower time-flexibility the user acceptability for the
predicted scenario is very low compared to the ideal scenario. However, for a
time-flexibility of more than 4 hours, the user acceptability are almost
comparable for both the scenarios.

This behavior is mainly due to the fact that at lower time-flexibilities the
scheduling is not able to rectify the prediction error resulting in the lower
acceptability. For e.g., a prediction error of -3 hours can be rectified if
the device has $\geq$ 3 hours of time-flexibility, but not with $<$ 3 hours of
time-flexibility. Therefore, we can see a significant improvement in user
acceptability with the increase in time-flexibility. However, after a certain
threshold (5 hours in our case) the acceptability gradually decreases because
of the larger flexibility a scheduler might overshoot the actual flexibility
duration. For example, with 10 hours of time flexibility and +5 hours of
prediction error, there is a high chance that a scheduler schedule a flexible
demand after the user preferred timestamp of the next $ready$ action,
resulting in the rejection of the proposed schedule.

Similarly, Figure 11 illustrates the percentage of the ideal savings that can
be achieved by the predicted scenario. From the figure, we can see that even
with the lower time flexibilities the predicted scenario can achieve up to 90%
of the savings of the ideal scenario. However, for higher time-flexibilities
savings is at a range of 40% of the ideal scenario. As before, this is mainly
because for higher time-flexibility, user acceptability decreases reducing the
savings.

Finally, Figure 11 shows the scheduling results, evaluated on the Genetx
dataset, of using our probabilistic flex-offer compared to standard a flex-
offer model. As it can be seen, modeling flexibility uncertainty improves the
scheduling effectiveness by increasing the user acceptance, and therefore
higher the financial benefits. In an uncertain scenario, standard flex-offers
miss useful device flexibility, leading to the lower probability of scheduling
acceptance.

###  6.4. Discussion

First, we showed how our approach can provide users with effective device
operation schedules and how it can reduce the energy cost of at least $30\%$
in both Spot and Regulation market, while maintaining a scheduling acceptance
of at least $60\%$ on a real dataset.

Second, we demonstrated the importance of trade-off between quality of
service, scheduling acceptance, and financial savings in order to successfully
deploy a DR scheme. Our flexibility estimation model can increase user
acceptance from less than $40\%$ up to $60\%$ while keeping a financial saving
that is approximately $50\%$ of the one where no user comfort is taken into
account, improving the customization of the DR scheme to specific users.

Third, the high stochasticity in the user behavior can lead to incorrect
predictions of the user energy needs. However, generic prediction models can
lead to scheduling acceptance and financial savings that are close to the
optimal, when the user provides sufficient flexibility. The higher prediction
and scheduling results obtained in the synthetic dataset Genetx hint to a
correlation between prediction and scheduling quality that allow an even
higher margin of improvement if more sophisticated prediction models are used.
In this highly stochastic scenario, our proposed probabilistic flex-offer for
modeling flexibility under uncertainty can bring an improvement, with regards
to the scheduling effectiveness, over traditional flexibility models, such as
standard flex-offers.

##  7\. Conclusions

We presented a novel DR scheme for user-oriented direct load-control of
residential appliance operations. Instead of relying on user surveys and
demanding interaction to evaluate the user utility, we proposed a data-driven
approach for estimating user utility functions, purely based on available load
consumption data. Moreover, we presented an online technique for learning the
user utility functions, which adaptively model the users’ preference over
time. Furthermore, our scheme is based on a scheduling scheme that
transparently prescribes the users with optimal device operation schedules
that take into account both financial benefits and user comfort, in order to
reduce the threat of user response fatigue.

The experimental results, performed on two different datasets, consistently
showed that our approach can successfully reduce energy costs while preserving
user comfort. On a real dataset, our approach yields savings of approximately
$32\%$ in the Spot Market, and $38\%$ in the Regulation Market, with $64\%$ of
user acceptance of the proposed schedules. Further, the results demonstrated
that the proposed adaptive learning model could capture user flexibility with
an acceptable accuracy generating positive utility to all involved players.
Additionally, even under the stochasticity, the predicted scenario was able to
achieve $90\%$ of the savings in the ideal scenario. Finally, our proposed
probabilistic flex-offer for modeling flexibility under uncertainty can bring
an improvement, with regards to the scheduling effectiveness, over traditional
flexibility models.

Future work will establish statistical models to evaluate the scheduling of
multiple devices and households. Also, the generic models applied in this
paper, although already successful, allow for a margin of improvement. In this
regard, we will explore more robust prediction techniques for device-level
load forecasting by exploiting the information of multiple residential devices
(e.g., kitchen appliances, HVAC, lights, etc.) to analyze patterns the user
behavior, in order to improve both the prediction results and the estimation
of the user flexibility. Additionally, we will investigate the use of
flexibility-oriented prediction error measures that align the prediction of
device activity to the scheduling of flexible demand.

###### Acknowledgements.

This work was supported in part by the GoFLEX project funded under the Horizon
2020 program.

## References

  * (1)
  * INT (2012) 2012\.  Source of Dataset: INTrEPID, INTelligent systems for Energy Prosumer buildIngs at District level, funded by the European Commission under FP7, Grant Agreement N. 317983.  http://www.fp7-intrepid.eu/. (2012).
  * mir (2013) 2013\.  The MIRABEL Project, 2013.  http://www.mirabel-project.eu.. (2013).
  * Adika and Wang (2014) Christopher O Adika and Lingfeng Wang. 2014.  Autonomous appliance scheduling for household energy management.  IEEE transactions on smart grid 5, 2 (2014), 673–682.
  * Astaneh et al. (2015) Mostafa F Astaneh, Bishnu P Bhattarai, Birgitte Bak-Jensen, Weihao Hu, Jayakrishnan R Pillai, and Zhe Chen. 2015.  A novel technique to enhance demand responsiveness: An EV based test case. In Power and Energy Engineering Conference (APPEEC), 2015 IEEE PES Asia-Pacific. IEEE, 1–5.
  * Boehm et al. (2012) Matthias Boehm, Lars Dannecker, Andreas Doms, Erik Dovgan, Bogdan Filipič, Ulrike Fischer, Wolfgang Lehner, Torben Bach Pedersen, Yoann Pitarch, Laurynas Šikšnys, and Tea Tušar. 2012.  Data Management in the MIRABEL Smart Grid System. In Proceedings of the 2012 Joint EDBT/ICDT Workshops (EDBT-ICDT ’12). 95–102.
  * Deng et al. (2015) Ruilong Deng, Zaiyue Yang, Mo-Yuen Chow, and Jiming Chen. 2015\.  A survey on demand response in smart grids: Mathematical models and approaches.  IEEE Transactions on Industrial Informatics 11, 3 (2015), 570–582.
  * Develder et al. (2016) Chris Develder, Nasrin Sadeghianpourhamami, Matthias Strobbe, and Nazir Refa. 2016.  Quantifying flexibility in EV charging as DR potential: Analysis of two real-world data sets. In Smart Grid Communications (SmartGridComm), 2016 IEEE International Conference on. IEEE, 600–605.
  * Fahrioglu and Alvarado (2001) Murat Fahrioglu and Fernando L Alvarado. 2001.  Using utility information to calibrate customer demand management behavior models.  IEEE transactions on power systems 16, 2 (2001), 317–322.
  * Gottwalt et al. (2017) Sebastian Gottwalt, Johannes Gärttner, Hartmut Schmeck, and Christof Weinhardt. 2017.  Modeling and valuation of residential demand flexibility for renewable energy integration.  IEEE Transactions on Smart Grid 8, 6 (2017), 2565–2574.
  * Hoogsteen et al. (2016) Gerwin Hoogsteen, Albert Molderink, Johann L Hurink, and Gerard JM Smit. 2016.  Generation of flexible domestic load profiles to evaluate demand side management approaches. In Energy Conference (ENERGYCON), 2016 IEEE International. IEEE, 1–6.
  * Hosseini et al. (2017) Sayed Saeed Hosseini, Kodjo Agbossou, Sousso Kelouwani, and Alben Cardenas. 2017.  Non-intrusive load monitoring through home energy management systems: A comprehensive review.  Renewable and Sustainable Energy Reviews 79 (2017), 1266–1274.
  * Humeau et al. (2013) Samuel Humeau, Tri Kurniawan Wijaya, Matteo Vasirani, and Karl Aberer. 2013.  Electricity load forecasting for residential customers: Exploiting aggregation and correlation between households. In Sustainable Internet and ICT for Sustainability (SustainIT), 2013. IEEE, 1–6.
  * James et al. (2016) JQ James, Junhao Lin, Albert YS Lam, and Victor OK Li. 2016\.  Maximizing aggregator profit through energy trading by coordinated electric vehicle charging. In Smart Grid Communications (SmartGridComm), 2016 IEEE International Conference on. IEEE, 497–502.
  * Jensen et al. (2016) Rikke Hagensby Jensen, Jesper Kjeldskov, and Mikael B Skov. 2016\.  HeatDial: Beyond User Scheduling in Eco-Interaction. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction. ACM, 74.
  * Jensen et al. (2018) Rikke Hagensby Jensen, Dimitrios Raptis, Jesper Kjeldskov, and Mikael B Skov. 2018.  Washing with the Wind: A Study of Scripting towards Sustainability. In Proceedings of the 2018 Conference on Designing Interactive Systems.
  * Kim and Shcherbakova (2011) Jin-Ho Kim and Anastasia Shcherbakova. 2011.  Common failures of demand response.  Energy 36, 2 (2011), 873–880.
  * Klaassen et al. (2016) Elke Klaassen, Charlotte Kobus, Jasper Frunt, and Han Slootweg. 2016.  Load shifting potential of the washing machine and tumble dryer. In Energy Conference (ENERGYCON), 2016 IEEE International. IEEE, 1–6.
  * Kobus et al. (2015) Charlotte BA Kobus, Elke AM Klaassen, Ruth Mugge, and Jan PL Schoormans. 2015.  A real-life assessment on the effect of smart appliances for shifting households electricity demand.  Applied Energy 147 (2015), 335–343.
  * Labeeuw et al. (2015) Wouter Labeeuw, Jeroen Stragier, and Geert Deconinck. 2015\.  Potential of active demand reduction with residential wet appliances: A case study for Belgium.  IEEE Transactions on Smart Grid 6, 1 (2015), 315–323.
  * Li et al. (2011) Na Li, Lijun Chen, and Steven H Low. 2011.  Optimal demand response based on utility maximization in power networks. In Power and Energy Society General Meeting, 2011 IEEE. IEEE, 1–8.
  * Ma et al. (2016) Jinghuan Ma, He Henry Chen, Lingyang Song, and Yonghui Li. 2016\.  Residential load scheduling in smart grid: A cost efficiency perspective.  IEEE Transactions on Smart Grid 7, 2 (2016), 771–784.
  * Neupane et al. (2014) Bijay Neupane, Torben Bach Pedersen, and Bo Thiesson. 2014\.  Towards flexibility detection in device-level energy consumption. In International Workshop on Data Analytics for Renewable Energy Integration. Springer, 1–16.
  * Neupane et al. (2015) Bijay Neupane, Torben Bach Pedersen, and Bo Thiesson. 2015\.  Evaluating the Value of Flexibility in Energy Regulation Markets. In Proceedings of the 2015 ACM Sixth International Conference on Future Energy Systems. ACM, 131–140.
  * Neupane et al. (2017) Bijay Neupane, Laurynas Šikšnys, and Torben Bach Pedersen. 2017\.  Generation and Evaluation of Flex-Offers from Flexible Electrical Devices. In Proceedings of the Eighth International Conference on Future Energy Systems. ACM, 143–156.
  * Paetz et al. (2013) Alexandra-Gwyn Paetz, Thomas Kaschub, Patrick Jochem, and Wolf Fichtner. 2013.  Load-shifting potentials in households including electric mobility-A comparison of user behaviour with modelling results. In European Energy Market (EEM), 2013 10th International Conference on the. IEEE, 1–7.
  * Paterakis et al. (2015) Nikolaos G Paterakis, Ozan Erdinc, Anastasios G Bakirtzis, and João PS Catalão. 2015.  Optimal household appliances scheduling under day-ahead pricing and load-shaping demand response strategies.  IEEE Transactions on Industrial Informatics 11, 6 (2015), 1509–1519.
  * Raptis et al. (2017) Dimitrios Raptis, Rikke Hagensby Jensen, Jesper Kjeldskov, and Mikael B Skov. 2017.  Aesthetic, Functional and Conceptual Provocation in Research Through Design. In Proceedings of the 2017 Conference on Designing Interactive Systems. ACM, 29–41.
  * Reinhardt et al. (2014) Andreas Reinhardt, Delphine Christin, and Salil S Kanhere. 2014.  Can smart plugs predict electric power consumption?: a case study. In Proceedings of the 11th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 257–266.
  * Samadi et al. (2012) Pedram Samadi, Hamed Mohsenian-Rad, Robert Schober, and Vincent WS Wong. 2012.  Advanced demand side management for the future smart grid using mechanism design.  IEEE Transactions on Smart Grid 3, 3 (2012), 1170–1180.
  * Setlhaolo et al. (2014) Ditiro Setlhaolo, Xiaohua Xia, and Jiangfeng Zhang. 2014\.  Optimal scheduling of household appliances for demand response.  Electric Power Systems Research 116 (2014), 24–28.
  * Šikšnys et al. (2015) Laurynas Šikšnys, Emmanouil Valsomatzis, Katja Hose, and Torben Bach Pedersen. 2015.  Aggregating and disaggregating flexibility objects.  IEEE Transactions on Knowledge and Data Engineering 27, 11 (2015), 2893–2906.
  * Tsui and Chan (2012) Kai Man Tsui and Shing-Chow Chan. 2012.  Demand response optimization for smart home scheduling under real-time pricing.  IEEE Transactions on Smart Grid 3, 4 (2012), 1812–1821.
  * Valsomatzis et al. (2015) Emmanouil Valsomatzis, Katja Hose, Torben Bach Pedersen, and Laurynas Siksnys. 2015.  Measuring and Comparing Energy Flexibilities. In EDBT/ICDT Workshops. 78–85.
  * Valsomatzis et al. (2016) Emmanouil Valsomatzis, Torben Bach Pedersen, Alberto Abelló, and Katja Hose. 2016.  Aggregating energy flexibilities under constraints. In Smart Grid Communications (SmartGridComm), 2016 IEEE International Conference on. IEEE, 484–490.
  * Vardakas et al. (2015) John S Vardakas, Nizar Zorba, and Christos V Verikoukis. 2015\.  A survey on demand response programs in smart grids: Pricing methods and optimization algorithms.  IEEE Communications Surveys & Tutorials 17, 1 (2015), 152–178.
  * Šikšnys and Pedersen (2016) Laurynas Šikšnys and Torben Bach Pedersen. 2016.  Dependency-based FlexOffers: Scalable Management of Flexible Loads with Dependencies. In Proceedings of the Seventh International Conference on Future Energy Systems (e-Energy ’16). 1–13.
  * Yang et al. (2014) Rayoung Yang, Mark W Newman, and Jodi Forlizzi. 2014\.  Making sustainability sustainable: challenges in the design of eco-interaction technologies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 823–832.
  * Zhou et al. (2016) Datong Zhou, Maximilian Balandat, and Claire Tomlin. 2016\.  Residential demand response targeting using machine learning with observational data. In Decision and Control (CDC), 2016 IEEE 55th Conference on. IEEE, 6663–6668.