# Modeling and Control of Multi-Energy Dynamical Systems: Hidden Paths to
Decarbonization

Marija Ilić
Laboratory for Information and Decision Systems (LIDS)  M.I.T.
Cambridge, MA United States
<EMAIL_ADDRESS>Rupamathi Jaddivada
Research Laboratory of Electronics (RLE) Partial funding provided by the US
National Science Foundation M.I.T
Cambridge, MA United States
<EMAIL_ADDRESS>
###### Abstract

This paper points out some key drawbacks of today’s modeling and control
underlying hierarchical electric power system operations and planning as the
hidden roadblocks on the way to decarbonization. We suggest that these can be
overcome by enhancing today’s information exchange and control. This can be
done by revealing and utilising inherent structure-preserving features of
complex physical systems, and, based on this, by establishing multi-layered
energy modeling. Each module (component, control area, non-utility-owned
entities) can be characterized in terms of its interaction variable, and
higher level models can be used to understand the interaction dynamics between
different modules. Once the structure is understood, we propose nonlinear
energy control for these modules which supports feed-forward self-adaptation
to ensure feasible interconnected system. Based on these technology agnostic
structures it becomes possible to expand today’s Balancing Authorities (BA) to
multi-layered interactive intelligent Balancing Authorities (iBAs) and to
introduce protocols for flexible utilization of diverse technologies over
broad ranges of temporal and spatial conditions.

###### Index Terms:

Energy modeling; Hierarchical modeling of electric power systems; Hierarchical
control of electric power systems; Interaction variable; Nonlinear distributed
energy control; Standards/protocols for stable dynamics; Non-wire solutions;
Inverter-based resources (IBRs).

##  I Introduction

This paper concerns fundamental modeling and control of multi-energy dynamical
systems whose performance objectives are becoming rapidly diverse and more
complex than there has been the case in the past. As new investments are being
planned, mainly for distributed energy resources (DERs) to replace large
polluting plants, it is necessary to enable their on-line utilisation. In
Section II we describe in some detail some key power balancing and delivery
problems facing the emerging electric power grid operations. In Section III we
propose basic enhancements necessary for enhanced operations and higher
utilisation of resources in a reliable and stable way. In order to manage
overly complex and system- and technology-specific interconnection rules and
approval processes, we review in Section IV inherent structures of today’s
electric power systems, and summarise in Section V structure-preserving
modeling for managing complex systems with multiple administrative entities.
We recall the notion of interaction variable introduced some time ago, and its
fundamental role in establishing multi-layered transparent models of different
spatial and temporal granularity. As an example, we consider small signal
coordinated frequency stabilisation and regulation for systems with
fluctuating intermittent resources in SectionVI. The rates at which these
frequency oscillations occur are basically electro-mechanical in nature, and,
as such, can be modelled and controlled assuming no voltage dynamics. However,
we next recognise in Section VII that the increasing presence of inverter-
based resources (IBRs) creates new problems such as very fast electronically-
induced oscillations caused by the interactions between the inverter
controllers.

The main contribution in this paper is presented in Section VIII in which we
propose a unifying structure-preserving multi-layered energy modeling of
coupled electro-mechanical and electromagnetic dynamics, and its inherent
structure-preserving properties. A modular aggregate model for representing
energy system dynamics is relevant for assessing and controlling interactions
between subsystems, and it is based on a recently introduced generalised
interaction variable as a means of specifying the ability of subsystem to
inject power at certain rates. Notably, we stress the key role of rate of
change of instantaneous reactive power [1], its modeling and the validity for
general non-sinusoidal voltages and currents. The second major contribution is
described in Section IX: Using multi-layered energy modeling we conceptualise
a unifying technology-agnostic energy control of interactions between the
components which results in a linear closed-loop aggregate energy interactive
model. This nonlinear energy control has a transparent and explainable
physical meaning, and it lends itself to provable performance of large-scale
system dynamics over broad ranges of input and topology changes. This is
particularly important during extreme events, faults and loss of large
generation in bulk power systems. Similar control is needed in small
microgrids subject to wind gusts and large weather-related variations in solar
radiance. Since the control does not require real-reactive power decoupling
nor linearisation it becomes effective during large changes of operating
conditions, an extremely difficult task. We suggest three steps toward
generalising today’s area control error (ACE) as the basis for reliable and
efficient emerging systems operation.

##  II Fundamental challenges to today’s hierarchical modeling and control

As new resources are beginning to replace large-scale controllable power
plants, the industry is facing many major operations and planning challenges.
In deregulated industry many mandates require utilities to connect new
resources in an open-access manner, and the interconnection process is time
consuming. Even in the regulated industry it has become difficult to use
today’s worst case scenario reliability and security tests in systems with
highly intermittent resources. More generally, today’s deterministic rules do
not lend themselves well to operating under uncertainties [2]. Important for
purposes of this paper is that the changing systems are beginning to
experience dynamical problems, such as low frequency oscillations [3] as well
as fast control-induced sub synchronous stability (CISS) problems [4, 5, 6,
7]. This situation brings up many technical challenges of interest in this
paper, in addition to regulatory and financial. Replacing large polluting
power plants with smaller-scale distributed intermittent resources requires
new investment in grid wires as well as in non-wire solutions. Non-wire
solutions are mainly “smarts” including various types of FACTS, HVDC, and
controllers placed on both conventional power plants (governors, AVRs, PSSs)
and inverter-based resources (IBRs), which comprise power-electronically
controlled solar and wind power plants, batteries and even clusters of EVs. It
quickly becomes obvious that the emerging systems are going to have many more
primary controllers dispersed throughout the system, and their design and
tuning represents a major technical challenge, summarized next.

##  III Necessary enhancements of today’s operations

Advanced bulk power systems (BPS) are operated in a hierarchical manner, by
performing feed-forward power scheduling to supply predictable system load
(tertiary control); frequency (and in some systems voltage) is regulated in
response to relatively slow deviations from schedules by the Balancing
Authorities (BAs) (secondary control); power plants have local primary
controllers, governors, AVRs and some times PSSs, and are expected to
stabilize fast voltage and frequency fluctuations to their set values given by
the higher level controllers [8, 9].

However, today’s hierarchical control needs to be enhanced to be effective in
the emerging systems with high penetration of intermittent resources. Tertiary
level computer applications need to be such that they enable participation by
all controllable resources, and not only by large power plants, as it is
mainly the case today. Also, it is necessary to schedule all controllable
variables (voltage set points, adjustable demand) in addition to real power
dispatch. This is needed because it is often necessary to optimize voltage
dispatch in order to support power delivery over large electrical distances.
While voltage is often thought of as a highly local problem, computer tools
like AC Optimal Power Flow (AC OPF) are needed to find voltage binding
constraints which make the AC power flow infeasible and to advise which
voltage and other adjustments are needed to make the AC power flow feasible
within acceptable operating constraints [10]. Also, tertiary level scheduling
tools should have adaptive performance metrics to seamlessly find a
combination of actions (real power, voltage, relaxing line flow limits, load
shedding) so that a feasible operation is found for anticipated uncontrolled
system demand. These functions are critical in systems where power flow
analyses and/or real power generation dispatch are insufficient to enable
feasibility. Examples of these are striking, it was shown that 1 GW more hydro
power can be delivered from Niagara Mohawk to NYC on a hot summer day by
combining voltage and real power dispatch [11]. These functions are critically
important during $(N-k)$ where $k>>2$ contingencies typical of weather-related
extreme events such as hurricanes in Puerto Rico [12] and, more recently, in
ERCOT [13]. Having such resource allocation tools is very important for
managing systems with high penetration of intermittent resources as they
support less conservative preventive and corrective, flexible reserves and
ensure their delivery, namely feasible AC power flow. Finally, tertiary level
SCADA should be expanded to interactively exchange information with lower
level grid users whose ability and willingness to participate in grid services
may vary. Much the same way, as conventional power plants have ramp rate-
limited ability, the less conventional resources can and should be integrated
into tertiary level by relying on their information about power and rate of
change of power. At the tertiary level this information is needed in a feed-
forward manner, prior to scheduling. A seemingly minor, but important is the
differentiation between the system operator using ramp-rate limit as a hard
constraint when scheduling, on one side, and optimising use of diverse
resources within power and rate of change of power ranges determined by the
resources themselves in a model predictive manner, on the other [14].

Figure 1: Basic function of secondary control: Adjusting reference set point
to produce given power,assuming nominal frequency Figure 2: Sketch of a
generator-turbine-governor (G-T-G) and its droop model

The actual implementation of these commands is carried out by the governor
changing the set point for its “droop” $P_{G}^{ref}[K]$ in a feed-forward
manner ($K$ represents the timescales relevant for tertiary control).
Secondary level control is generally used to map power commands given by the
tertiary level into the set points to which controllable equipment responds
and/or to in a feedback manner adjust frequency set points so that incremental
power is produced and the system frequency deviation within a pre-specified
tolerance of $\epsilon$ is allowed. However, the line between tertiary and
secondary level governor control is often gray. In systems which do not have
automatic generation control (AGC), only “power sharing” commands are
implemented this way. Regulation of frequency deviations resulting from
relatively slow power excursions around the scheduled value is done manually
in such systems. In systems with AGC, the PI governor control responding to
frequency deviations, and/or area control error $(ACE)$ in multi-BA systems,
adjusts the set point for governor controller $\omega_{G}^{ref}[k]$ ($k$
denotes discrete time samples relevant for secondary control) according to the
three way droop relation. Shown in Figure 1 is a sketch of these droops
defining relations between frequency deviations from nominal frequency
$\omega_{0}$, $\Delta\omega_{G}[k]=\omega_{G}[k]-\omega_{0}$ and power
increments $\Delta P_{G}[k]=P_{G}[k]-P_{G}^{ref}[K]$ for given
$\omega_{G}^{ref}[k]$. AGC basically computes set point
$\Delta\omega_{G}^{ref}[k]=\omega_{G}^{ref}[k+1]-\omega_{G}^{ref}[k]$ so that
$\Delta\omega_{G}[k]<\epsilon$ [9]. To avoid a major confusion in recent
literature concerning the notion of a droop in systems with IBRs, in
particular, we point out that power sharing takes place in a feed forward way
for predictable bounds on power deviations, while the AGC takes place in a
feedback manner at the faster rate. Moreover, the three way droop used for AGC
is a quasi-static concept which is derived by assuming that derivatives are
zero as illustrated in Figure 2. The power sharing droop can be set
independently in a slower feed forward way to shape the frequency response to
anticipated power by the tertiary level. This clarification is written because
many references in the literature assume one or the other functionality of
secondary level control. There are several major issues regarding the role of
secondary level control. The main is that power sharing function assumes that
governor can control power generated over broad ranges of power commands given
by the tertiary level. The second major issue is that the gains in governor
controller are derived using a linearised turbine-generator model around pre-
selected operating point. The third major issue is that secondary level
assumes that primary controller stabilises and regulates output variables
(frequency) to its set point, and, that it can control power generated at the
right rate. All these assumptions are major causes of frequency problems in
systems with intermittent resources and require further studies as discussed
in Section IV.

Finally, and independent from the version of secondary control in place, a
combination of primary and secondary control should guarantee that commands
given by the tertiary level are implementable, namely stable, feasible and
robust with respect to parameter uncertainties. In this paper we are
particularly concerned with the huge issue that state-of-the-art of today’s
primary controllers can generally not do this, and propose new controllers.
Broadly speaking, they can not stabilise system dynamics in response to
continuously varying fast power imbalances. It is generally hard to control
power/rate of change of power while maintaining voltage and frequency within
the operating limits. Later part of this paper describes this in some detail.

##  IV Structure-preserving modeling of frequency dynamics in transformed
state space

In order to overcome extreme perceived complexity when attempting to improve
today’s hierarchical control so that it supports the on-going changes, we
propose to consider the structure-preserving modeling of electro-mechanical
dynamics relevant for assessing frequency stability, briefly summarised next.
Consider without loss of generality a two-area electric power system shown in
Figure 3 with highly varying solar power in one of the areas.

Figure 3: A sketch of two-area interconnected power system

It is known that the linearized electro-mechanical dynamics $x_{LC,i}$ of each
module $i$ ( resource, load, BA, interconnected system) can be written in
terms of its internal local state variables $x_{LC,i}$, and the real power
exchanged with the rest of the system $P_{G,i}$. As an illustration, shown in
Figure 2 is a conventional generator with its governor control. Its states
$x_{LC,i}$ are shown in Figure 2, and their dynamics are

$\displaystyle{{\dot{x}}_{LC,i}}=\underbrace{\left[{\begin{array}[]{*{20}{c}}0&{{\omega_{0}}}&0&0\\\
0&{\frac{{{K_{d}}}}{M}}&{\frac{1}{M}}&0\\\
0&0&{-\frac{1}{{{T_{t}}}}}&{\frac{{{K_{t}}}}{{{T_{t}}}}}\\\
0&0&0&{-\frac{r}{{{T_{g}}}}}\end{array}}\right]}_{{A_{LC}}}{x_{LC,i}}+\underbrace{\left[{\begin{array}[]{*{20}{c}}0\\\
{\frac{1}{M}}\\\ {\;0}\\\ 0\end{array}}\right]}_{{c_{M}}}{P_{G,i}}$ (9)

It can be seen that this model takes on the standard state space form in which
local state variables
$x_{LC,i}=[\theta_{G,i}~{}~{}\omega_{G.i}~{}~{}P_{T,i}~{}~{}a_{i}]^{T}$ define
the dynamics as a function of power generated $P_{G,i}$; notably they do not
depend on $\theta_{G,i}$ explicitly [15]. Power generated and its rate of
change must balance local load deviation and any other local disturbance as
follows

$\dot{P}_{G,i}=\dot{P}_{L,i}$ (10)

When Equations (9) and (10) are combined they result in dynamical model in
transformed state space of a stand alone generator $i$

$\displaystyle\underbrace{\left[{\begin{array}[]{*{20}{c}}{{{\dot{x}}_{LC,i}}}\\\
{{{\dot{P}}_{G,i}}}\end{array}}\right]}_{\dot{x}_{i}^{new}}=\underbrace{\left[{\begin{array}[]{*{20}{c}}{{A_{LC}}}&{{C_{M}}}\\\
{{K_{P}}}&0\end{array}}\right]}_{A_{i}}x_{i}^{new}+\underbrace{\left[{\begin{array}[]{*{20}{c}}0\\\
1\end{array}}\right]}_{D_{i}}{{{\dot{P}}_{L,i}}}$ (17)

where

$x_{i}^{new}=[\omega_{G}~{}~{}P_{T}~{}~{}a~{}~{}P_{G}]^{T}$ (18)

is the new state variable in transformed state space.

Consider now a BA $J,J\in\\{1,2\\}$ of two area system shown in Figure 3. When
generators are interconnected, the model of their local states Eqn. (9)
remains the same as for the stand-alone generator case and takes on the form

$\dot{x}_{LC}=A_{LC}x_{LC}+C_{M}P_{G}$ (19)

and the conservation of power becomes

$\dot{P}_{G}=K_{P}\omega_{G}+\dot{F}_{e}+D_{P}\dot{P}_{L}$ (20)

where $P_{G}$, $\omega_{G}$, $F_{e}$ and $P_{L}$ are vectors of power
generated by all generators in the area, net tie-line power flow coming into
the area and loads within the area [15]. Matrix $K_{P}$ is structurally
singular as a direct result of conservation of energy in the area. Equations
(19) and (20) combinedly result in a decentralised model of BA in transformed
state space [8]

$\displaystyle\left[{\begin{array}[]{*{20}{c}}{{{\dot{x}}_{LC}}}\\\
{{{\dot{P}}_{G}}}\end{array}}\right]=\left[{\begin{array}[]{*{20}{c}}{{A_{LC}}}&{{C_{M}}}\\\
{{K_{P}}}&0\end{array}}\right]\left[{\begin{array}[]{*{20}{c}}{{x_{LC}}}\\\
{{P_{G}}}\end{array}}\right]+\left[{\begin{array}[]{*{20}{c}}0\\\
{{{\dot{F}}_{e}}}\end{array}}\right]+\left[{\begin{array}[]{*{20}{c}}0\\\
{{{\dot{P}}_{L}}}\end{array}}\right]$ (31)

Once this structure is understood, it becomes fairly straightforward to
formalize objectives of primary and secondary level generation control to both
stabilize and regulate frequency in systems whose disturbances are
continuously varying loads and/or intermittent resources. We observe that
today’s separation of primary control for frequency stabilization, on one
side, and secondary control for frequency regulation, on the other, can no
longer be done by assuming that secondary control is quasi-static and that, as
such, can be modeled using quasi-static frequency regulation droop-based
models and by neglecting the grid effects [16, 17]. Instead, continuous-time
models are needed to stabilize and regulate frequency in systems with ever-
fluctuating power imbalances [18].

Figure 4: Emerging diverse technologies

Second, quite relevant for the changing systems is that the same structure-
preserving modeling is technology agnostic; shown in Figure 4 are vastly
different technologies, all of which lend themselves to the same structural
modeling.

##  V Structure-preserving modeling of systems comprising multiple entities

Figure 5: Emerging portfolios of intelligent Balancing Authorities (iBAs)

To further formalise modeling in the changing industry, we suggest that one of
the major challenges is the lack of specifications of new technologies in
terms of variables that make it possible to integrate them the same way as
other resources. Shown in Figure 5 is a sketch of the emerging architectures,
in which often non-utility-owned entities wish to connect to the legacy system
comprising coarse BAs [19]. These new entities are “nested” within the legacy
BAs, and often have their own control and decision-making “smarts”. Such
emerging examples are: portfolios of wind farms and storage (See Figure 6);
interconnected microgrids embedded in today’s distribution grids with and/or
without own storage; clusters of EVs; neighbourhoods with controlled water
pumps, HVACs, and water heaters; clusters of smart buildings; utility scale
solar plants; and many more.

Figure 6: Emerging portfolios of intelligent Balancing Authorities (iBAs)

It is highly unrealistic to require that each of these new technologies
reveals exactly their internal dynamical models and control. To overcome this
problem, we propose that these enities can be viewed as technology-agnostic
iBAs which can be characterized in terms of their interaction variables
(intVar). This notion of intVar was introduced first for linearized systems
for frequency stabilization and regulation and has two unique properties key
to modeling, assessing and controlling interactions between different
entities. In short, an intVar is a direct consequence of energy conservation
by each iBA and, when modeling only linearized electro-mechanicial
interactions can be shown to have the following [8, 9]:

  * •

Property 1: intVar $z_{i}$ associated with component $i$ is a function of its
own internal state variables written in transformed power state space, Eqn.
(31)

  * •

Property 2: When disconnected from the rest of the system it remains constant
$\frac{dz_{i}}{dt}=0$.

The existence of interaction variable associated with any physical module
(generator $Gi$, BA $J$) whose structure-based model in transformed state
space was introduced above directly follows from the fact that its system
matrix is structurally singular. For the case of BA $J$ $K_{P}$ in its
transformed state space model (31) matrix $K_{P}$ is structurally singular,
leading to structurally singular matrix in its transfomred state space given
in Eqn. (31).

Therefore, there exists a transformation $T_{BA}$ such that

$T_{BA}A_{BA}=0$ (32)

where $A_{BA}$ is system matrix in BA model (31). Then, by pre-multiplying
this model by such transformation it follows that

$\dot{z}_{BA}=T_{BA}\dot{F}_{e}+T_{BA}D_{P}\dot{P}_{L}$ (33)

The interaction variable is defined as

$z_{BA}=T_{BA}x_{BA}^{new}$ (34)

is the integral of power imbalance created by deviations in net tie line flow
power and internal load power deviations. Similarly, it can be shown that
$T_{Gi}$ exists such that

$T_{Gi}A_{Gi}=0$ (35)

where $A_{Gi}$ is the matrix in the generator model given in Equation (9). Its
interaction variable can be shown to be

$z_{Gi}=T_{Gi}x_{Gi}^{new}$ (36)

The interaction variable has a straightforward physical interpretation. It is
directly a result of first conservation law of energy which says that the
dynamics of stored energy $E_{i}$ in a module $i$ is a result of net power
injected into it $P_{i}$ minus its internal thermal loss
$\frac{E_{i}}{\tau_{i}}$, as follows

$\dot{E}_{i}=-\frac{E_{i}}{\tau_{i}}+\dot{z}_{i}$ (37)

In the case of linearized electro-mechanical dynamics $\dot{z}_{i}=P_{i}$,
where $P_{i}$ is real power out of module $i$ and $\tau_{i}$ is time constant
representing thermal losses.

Figure 7: Interaction variables and power generated by individual plants

Shown in Figure 7 are interaction variables and internal generator power
variables for the two area system shown in Figure 3 when solar power radiance
is zero. Only electro-mechanical dynamics can be modeled for assessing inter-
area power and frequency oscillations. We generalize the notion of intVars in
Section VII that can capture the electro-magnetic fast oscillations and the
effects of their inverter cotnrol.

##  VI Structure-based small signal frequency stability assessment

Representing small signal frequency dynamics of very large-scale multi-BA with
nested iBA systems as in Figure 5, is almost an impossible task in the
changing industry. The resources and loads can have very different
manufacturers data, and include many diverse types, hydro, coal, gas, wind and
their diverse non-standardized governor controllers. Also, many entities do
not like to share their internal technology. To overcome this problem, we
propose here that sources of low frequency oscillations caused by poor tuning
of governors, for example, can be detected in a highly structure-preserving
transparent manner by exchanging only minimal information in terms of their
intVars, a method based on [20]. The method is based on conceptualising the
system shown in Figure 3 as a large-scale interconnected system comprising two
subsystems, shown in Figure 3. A vector Lyapunov function
$v(x)=[v_{1}(x_{1})~{}v_{2}(x_{2})]^{T}$ has Lyapunov functions of each
subsystem $v_{1}(x_{1})=x_{1}H_{1}x_{1}^{T}$ and
$v_{2}(x_{2})=x_{2}H_{2}x_{2}^{T}$. Matrices $H_{1}$ and $H_{2}$ are computed
by solving Riccati equations in each subsystem

$\displaystyle A_{1}H_{1}+A_{1}^{T}H_{1}$ $\displaystyle=$ $\displaystyle-
G_{1}$ (38) $\displaystyle A_{2}H_{2}+A_{2}^{T}H_{2}$ $\displaystyle=$
$\displaystyle-G_{2}$ (39)

where $G_{1}$ and $G_{2}$ are positive definite matrices. It can be shown that
the vector Lyapunov function has negative derivative if matrix $W$ with its
entries defined as

$\displaystyle w_{ij}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}\frac{\lambda_{m}(G_{i})}{\lambda_{M}(H_{i})}+\lambda_{M}^{\frac{1}{2}}(A_{ij}^{T}A_{ij})~{}~{}i=j$
(40) $\displaystyle w_{ij}$ $\displaystyle=$
$\displaystyle\lambda_{M}^{\frac{1}{2}}(A_{ij}^{T}A_{ij})~{}~{}i\neq j$ (41)

is a Metzler matrix and it is diagonally dominant [21]. Here $\lambda_{M}$ and
$\lambda_{m}$ are maximum and minimum eigenvalues, respectively.

To check this condition, each subsystem computes in a decentralized manner the
minimal and maximum eigenvalues and tests whether the interconnected system is
made unstable because of very strong effects of interconnection matrices
$A_{ij}$. This method is computationally very effective because it only
requires computing extreme eigenvalues by the subsystems themselves, and
checking conditions of the very low order matrix $W$ whose dimension is
determined by the number of subsystems. This method, in addition to being
scalable, indicates relative effects of local stability and interactions on
global stability of the interconnected system. The same method can be used to
design control of subsystems as well as the control of FACTS devices which can
affect the strength of interactions on tie lines interconnecting subsystems.
For numerical examples illustrating use of these conditions on the two area
system shown in Figure 3, see [20].

##  VII Examples of IBR-and fault-caused electromagnetic voltage oscillations

The electro-magnetic voltage oscillations have typically been a problem during
sudden fast faults in BPS. However, they are also becoming an important
challenge in systems with fast fluctuating intermittent resources, due to,
wind gusts and hard-to-predict variations in solar radiance. There has been a
major concern about “loss of inertia” and the need for faster-responding
controllers to low frequency voltage oscillations in such systems. At present
there is a major effort under way to control wind and solar power fluctuations
with their own fast power-electronically-switched controllers and these
resources are referred to as the IBRs [22, 23]. There are also many power-
electronically switched controllers placed in grid components, such as HVDC
technologies, STATCOMs, Static Var Compensators (SVCs), Thyristor Controlled
Series Capacitors (TCSCs) and, more recently, even electronically controlled
series inductors [24].

While system planners continue to analyse likely operating problems in these
newly emerging systems by relying on well-established feasibility, small
signal stability and transient stability analysis tools, it is becoming
increasingly clear that both enhanced modeling and new controllers are needed.
The currently used software often does not even identify transient
instability, and, as a result, can not assess potential benefits of deploying
fast switching controllers. The industry is aware of these issues and it often
resorts to using a very detailed electromagnetic transients (EMT) modeling to
set and test protection during faults, for example. These tests are
excessively time consuming and system-specific. Because of their complexity,
they are not scalable and, most critically, can not be used to assess
interactions between an IBR tested and other dynamically varying IBRs and grid
controllers placed elsewhere in the system. The tests are typically done using
static Thevenin equivalent of the rest of the system and applying a
combination of most likely and/or most critical step power changes at the IBR
location [25]. System measurements detect interactions between, for example,
an IBR wind power plant and connecting weak TCSC-compensated transmission
line, but there are no effective software tools for simulation of this
“fighting” of IBR controllers known as control induced sub-synchronous
stability (CISS) problems [4]. In this paper we propose that these
oscillations caused by the IBRs and the smarts on the grid interacting can be
modeled to a large extent using structure-preserving modeling described
earlier in this paper by introducing a generalised interaction variable. In
addition to modeling electro-mechanical stored energy the generalised energy
modeling represents fast electro-magnetic dynamics as well. This is introduced
in the next Section VIII. In this section we briefly illustrate the
electromagnetic energy and its dependence on the controller logic used.
Notably, these examples utilize TVP modeling of all components, including line
currents and voltages and do not require detailed EMT modeling.

Figure 8: Fault in a BPS with small inertia wind power plant Figure 9:
Dependence of electro-magnetic dynamics during faults on FACTS control:
Increment of accumulated energy cause by a fault - the key role of FACTS
control Figure 10: Nonlinear control of electromagnetic energy and frequencies

Shown in Figure 8 is a sketch of a BPS with one large inertia power plant,
small inertia wind power plant sending power to load via TCSC-compensated
transmission line [26]. During short circuit at bus 3, load is lost and
electro-mechanical energy in generators begins to increase. Prior to this,
electro-magnetic energy stored in generators increases and it reduces critical
clearing time determined by the electro-mechanical dynamics. Shown in Figures
9 and 10 is fast accumulated energy in the small inertia generator without
TCSC, and it is contrasted with the same electro-magnetic energy transferred
to the TCSC. This dynamics and the effect of fast controller cannot be modeled
unless dynamics of wires and TCSC are accounted for. At the same time, it is
possible to have a lumped-parameter TVP-based model for designing effective
control logic without requiring excessively detailed EMT modeling. Later in
Section IX we describe energy control design used here. The purpose of showing
this example is to demonstrate that during fast sudden changes it becomes
necessary to model electro-magnetic dynamics and not just the electro-
mechanical dynamics.

Figure 11: Small electric power grid in Flores

Shown in Figure 11 is a sketch of a small Flores power system in Azores
Islands, Portugal [27].

Figure 12: SVC control of short wind gust in Flores electric power grid: (a)
Total accumulated energy and (b) total accumulated electromagnetic energy in a
system controlled by different controller Figure 13: SVC control of short wind
gust in Flores electric power grid: Mechanical frequency of all generators in
the system during a short-term high- magnitude wind perturbation: (a) dashed
(without control on the SVC), (b) solid (with control on the SVC) Figure 14:
Long and short term wind gust in Flores electric power grid

An SVC can absorb a sudden power burst by storing electro-magnetic energy
fast, as shown in Figure 12 and 13, for a sudden wind gust. A typical wind
gust is shown in Figure 14 of short and long duration, respectively.

Similarly, consider a flywheel storage system connected to Flores island shown
in Figure 15 (conceptual block diagram shown in Figure 16). The flywheel can
compensate for a longer-duration wind gust as shown in Figure 17.

Figure 15: Full diagram connecting the flywheel to Flores Figure 16: Interface
between the power grid and the flywheel Figure 17: Flywheel control of long
wind gust in Flores electric power grid

The TVP-based energy models can generally be used to design energy controllers
which counteract oscillations caused by such sudden disturbances, without
requiring EMT modeling. Given inherent multi-layered energy modeling in terms
of interaction variables which represent the dynamics of electromagnetic
interactions in addition to the dynamics of electro-mechanical interactions
described in Section VI earlier in this paper, it becomes possible to control
CISS problems even in large-scale changing electric power systems.

##  VIII Unifying structure-preserving multi-layered energy modeling

Shown in Figure 18 is a sketch of legacy system with the interconnected
IBRs.The legacy system comprises conventional power plants serving slowly
varying system load. The IBRs, solar in this example system, send highly
fluctuating power injections and are equipped with fast power-electronically
switching inverters. Shown in Figure 19 is internal design of a typical solar
PV with small battery. Inverter controllers are not standardised, but for the
purposes of this paper they can be thought of black boxes in which some energy
is stored, and that they interact with the rest of the system by balancing
energy over certain time, by balancing instantaneous power at the
interconnections according to the first law of thermodynamics, conservation of
energy. However, since the rates of change of instantaneous power are vastly
different, it becomes necessary to also balance power at the right rates to
avoid acceleration and related oscillations. It was recently proposed to
generalize the interaction variable previously used for assessing and
controlling electro-mechanical dynamics, as described in the section above.

Figure 18: IBRs interconnected to a legacy BPS; fast IBR generation serving
slow system load Figure 19: Internal design of a typical solar PV with a
battery

Such generalized interaction variable is defined as [28, 29].

$\dot{z}^{out}_{i}=[P^{out}_{i}~{}~{}\dot{Q}^{out}_{i}]^{T}$ (42)

It has the same structural properties as the interaction variable
characterizing electro-mechanical dynamics introduced above in Eqn. (37),
except that it depends on both its states and rates of change of internal
states. As such, it is instrumental in modeling interactions between different
modules without making decoupling assumptions or having to linearize nonlinear
models. In particular, it was shown that the dynamics of aggregate variables,
both energy $E_{i}$ and its rate of change $p_{i}=\frac{dE_{i}}{dt}$ take on a
general technology-agnostic form

$\displaystyle\dot{E}_{i}$ $\displaystyle=$
$\displaystyle-\frac{E_{i}}{\tau_{i}}+P^{out}_{i}+P^{u}_{i}=p_{i}$ (43)
$\displaystyle\dot{p}_{i}$ $\displaystyle=$ $\displaystyle
4E_{t,i}-\dot{Q}^{out}_{i}-\dot{Q}^{u}_{i}$ (44)

The interaction variables must obey both conservation of instantaneous power
and the rate of change of generalized instantaneous power. Basically, both
instantaneous power out of module $i$ and rate of change of instantaneous
reactive power should balance with the net instantaneous power and the net
rate of change of instantaneous reactive power being sent from the
neighbouring modules, respectively, as follows

$\displaystyle P^{out}_{i}$ $\displaystyle=$ $\displaystyle P^{in}_{j}$ (45)
$\displaystyle\dot{Q}^{out}_{i}$ $\displaystyle=$
$\displaystyle\dot{Q}^{in}_{j}$ (46)

The aggregate interactive model defining rate of change of energy and rate of
change of reactive power is given in Eqn. (43) and (44) and is subject to
conservation laws of rate of change of interaction variable defined in Eqn.
(Eqn. 42) as in Eqn. (45) and Eqn. (46).

###  VIII-A An example of structure preserving solar PV serving constant power
load

Shown in Figure 20 is a conceptual sketch of solar PV with a battery [30].
Internal states are inductor current and capacitor voltage and inverter-
controlled solar source is shown as a controllable DC source. There are
several different ways of controlling this source, ranging from two-loop
proportional voltage-current controller; or PID controller; or energy based
control which stabilizes aggregate variables by aligning the interaction
variables between the solar PV and power load.

Figure 20: Sketch of an inverter controlled solar battery serving power load

In Section IX we will describe in detail the effects of primary controllers on
closed loop dynamics of inverter-controlled solar PV with a battery. For
purposes of illustrating further energy modeling and control we consider
without loss of generality its representation as a voltage controlled source
connected via RL circuit, with the objective of serving constant power load as
shown in Fig. 21.

Figure 21: Interconnected system comprising interactions between RLC circuit
with controllable voltage source ($\Sigma_{1}$) and a load with given power
specifications ($\Sigma_{2}$)

This circuit can be thought of as comprising two interacting modules
$\Sigma_{1}$ and $\Sigma_{2}$ . The dynamics of internal state variable,
inductor current, takes on the form

$\frac{di_{L1}}{dt}=-\frac{R_{1}}{L_{1}}i_{L1}+\frac{1}{L_{1}}u_{1}-\frac{1}{L_{1}}v_{1}$
(47)

Its energy model takes on the form

$\displaystyle\dot{E}_{1}$ $\displaystyle=$
$\displaystyle-\frac{E_{1}}{\tau_{1}}+P^{out}_{1}+P^{u}_{1}$ (48)
$\displaystyle\dot{p}_{1}$ $\displaystyle=$ $\displaystyle
4E_{t,1}-\dot{Q}^{out}_{1}+\dot{Q}^{u}_{1}$ (49)

where $E_{1}=\frac{1}{2}Li_{L1}^{2}$, $\frac{E_{1}}{\tau_{1}}=Ri_{L1}^{2}$,
$P^{u}_{1}$ and $\dot{Q}^{u}_{1}$ are interaction variables between the
controllable source and the RL circuit within the module $\Sigma_{1}$. The
term $E_{t,1}=\frac{1}{2}L(\frac{di}{dt})^{2}$ is the stored energy in tangent
space. The aggregate variables of this module are
$x_{z,1}=[E_{1}~{}~{}p_{1}]^{T}$, and their dynamics takes on the form

$\dot{x}_{z,1}=A_{z,1}x_{z,1}+B_{z}u_{z,1}+B_{t}E_{t,1}+B_{z}\dot{z}_{1}^{out}$
(50)

Here, $B_{t}=\left[0,4\right]^{T},B_{z}=\left[1,-1\right]^{T}$ for any
component and matrix
$A_{z,1}=\left[{\begin{array}[]{*{20}{c}}{-1/{\tau_{1}}}&0\\\
0&0\end{array}}\right]$ depend only on the time constant $\tau_{1}$.

Notably, when aggregate control is selected to be
$u_{z,1}=[P^{u}_{1}~{}~{}\dot{Q}^{u}_{1}]^{T}$, it can be designed to align
the rate of change of interaction variable $\dot{z}_{1}^{out}$ and rate of
change of interaction variable $\dot{z}_{2}^{out}$ from $\Sigma_{2}$. The
closed loop aggregate dynamics is linear and can be shaped in a provable
manner. This control design is illustrated next.

##  IX Unifying energy control of interaction dynamics

It follows from the above derivations that a technology-agnostic energy
controller which aligns rate of change of interaction variables so that the
conservation of power and rate of change of reactive power are observed
according to Eqns. (45) and (46). This is achieved by having the control in
energy space $u_{z,1}$ respond to the the output variable that takes the form

$y_{z,1}=C_{t,1}E_{t,1}+C_{z,1}x_{z,1}+D_{z,1}(\dot{z}^{out}_{1}-\dot{z}_{1}^{ref})$
(51)

where $\dot{z}_{1}^{ref}=\dot{z}_{2}^{out}$. The energy control can be
designed to both cancel error dynamics and the term $E_{t,1}$. Alternatively,
the term $E_{t,1}$ can be upper-bounded and the sliding mode control
implementation can be done [30].

This technology-agnostic control $u_{z,1}$ needs to be mapped into physically-
implementable control signal. In the case of $\Sigma_{1}$ the physical control
signal is $u_{1}=v_{1}$. The implementation of this physical control signal is
typically power-electronically controlled switch. This map is non-unique and
one example of mapping is shown in Eqn. (52) resulting in a dynamical control.

$\frac{du_{1}}{dt}=\frac{u_{1}}{i_{L1}}\frac{di_{L1}}{dt}-\frac{\dot{Q}_{1}^{u}}{i_{L1}}$
(52)

###  IX-A Dependence of system response on controller logic when serving
constant power load

As one attempts to control systems with IBRs, it is necessary to assess
performance of today’s state-of-the-art controllers with the energy
controllers described in this paper. The actual expression for energy control
differs depending on whether the objective is to:

  * •

Single timescale energy control: Align power controlled by the IBR after its
stored energy settles.

  * •

Two timescale energy control: Align both instantaneous power and rate of
change of instantaneous reactive power controlled by the IBR and required by
the load.

To illustrate this, consider without loss of generality a step power change
required by the load from the IBR sketched in Figure 20. This scenario is
rather difficult because constant power load is known to create negative
incremental resistance problem which inherently leads to voltage collapse
and/or other types of unacceptable dynamics in IBRs controlled by conventional
controllers. Shown in Figure 22(a) is the response of such controller. It can
be seen that typically-used two loops control will have problems serving
constant power load as shown in Figure 22(a), voltage collapsing after several
seconds. To the contrary, the response of a conventional proportional-
derivative (PD) controller remain stable, as shown in the plots in Figure
22(b). This plot highlight the key relevance of dynamic controllers.

(a) PI two loop controller resulting in voltage collapse

(b) PD controller resulting in stable response

Figure 22: Closed loop response to step power load change

.

Figure 23: Closed loop response to step power load change: Single timescale
nonlinear energy controller; Upper plots show the state trajectories. Lower
plots show the evolution of the difference in the outgoing and incoming
interaction variables.  Figure 24: Closed loop response to step power load
change: Two-timescale nonlinear energy controller; Lower plots show the
evolution of the difference in the outgoing and incoming interaction
variables. Figure 25: Closed loop response to small intermittent fluctuations:
Conventional feedback control Figure 26: Closed loop response to small
intermittent fluctuations: Constant gain derivative control responding to
voltage Figure 27: Closed loop response to small intermittent fluctuations:
Single timescale nonlinear energy controller; Lower plots show the evolution
of the difference in the outgoing and incoming interaction variables.  Figure
28: Closed loop response to small intermittent fluctuations: Two-timescale
nonlinear energy controller; Lower plots show the evolution of the difference
in the outgoing and incoming interaction variables.

Shown next in Figure 23 is the closed loop dynamic response for the same
scenario with the energy controllers when both electro-mechanical and electro-
magnetic dynamics are controlled as a single time scale process, and when the
only control objective is to alignment of instantaneous power after energy
dynamics settles ($\frac{dE}{dt}=0$). It can be seen that even this nonlinear
controller is not fast enough to stabilize system response and to avoid
voltage collapse.

However, when the energy controller is designed with the objective to align
both instantaneous power and rate of change of instantaneous reactive power at
the interfaces of IBR and constant power load, and time scale separation
between these processes is carefully accounted for by using slow reduced order
model to stabilise electro-mechanical dynamics and fast reduced order model is
used to stabilise fast reactive power rate of change, shown in Figure 24 is
closed loop stable response.

###  IX-B Dependence of system response on controller logic when serving
intermittent power load

A similar comparison of the effects of iBR controllers on closed loop response
can be shown in response to fluctuating power disturbances seen by the IBR.
Shown in Figure 25 is the closed loop response to small intermittent
fluctuations. It can be seen that conventional feedback is too slow to cancel
the fast imbalances, resulting in small persistent oscillations in voltage.
The response obtained with dynamical PD controller cancels the persistent
oscillations, but results in slower response as shown in Fig. 26.

The response obtained with single timescale energy control is shown in Fig.
27. It can be seen that even this nonlinear controller is not able to suppress
fast imbalances. Finally, the responses obtained with two timescale energy
control is shown in Fig. 28, which results in perfect voltage regulation. We
further see that the instantaneous and reactive power imbalances settle at
zero.

##  X Concluding recommendations: Structure-preserving standards/protocols for
reliable and efficient multi-energy systems

In closing, this paper takes a broad look at the state-of-the-art hierarchical
modeling and control for bulk power systems. It identifies several hidden
roadblocks to using this paradigm in support of large-scale decarbonization.
Fundamental challenges are described first. It is proposed next to utilize
structure-preserving modeling of complex power grids which has been introduced
by our many collaborators and former doctoral students both at Carnegie Mellon
University and, more recently at M.I.T. These concepts are explained in the
context of emerging low frequency oscillations, using the notion of
interaction variable. Then a recent generalization of interaction variables is
introduced and it is suggested that energy modeling which captures both
instantaneous power and instantaneous rate of change of reactive power be
considered as the basis for multi-layered modeling of the changing electric
energy systems. An aggregate model of technology-agnostic module is derived in
terms of stored energy and its rate of change, and it is used for nonlinear
energy control. A proof-of-concept energy control which avoids voltage
collapse and significantly reduces otherwise persistent oscillations created
by intermittent IBRs is presented using a DC voltage controlled source which
supplies time varying power. Based on these findings, further research is
needed toward a modular feed-forward distributed information exchange in
support of self-adaptation of subsystems to ensure feasible interconnected
system.

We propose the interaction-based energy modeling to be used as the basis for
structure preserving interacting standards/protocols that ensure stable
operations in the changing systems comprising multiple layers of intelligent
Balancing Authorities (iBAs) with well-defined interfaces to the rest of the
system. We suggest three steps toward generalising today’s area control error
(ACE) as the basis for reliable and efficient emerging systems operation. They
are as follows:

  * •

Generalise today’s coarse balancing authorities into intelligent Balancing
Authorities (iBAs).

  * •

Enhance today’s AGC standard Area control error (ACE) into generalized
interaction variables as the measure of any iBAs performance.

  * •

Work toward next generation end-to-end SCADA supported by the information
exchange proposed.

These three steps form the foundations of interactive operations in future
energy systems so that they can be modelled, controlled and operated without
excessive complexity.

## Acknowledgement

The first author greatly appreciates partial support from the NSF EAGER
Project number 2002570 entitled “EAGER: Fundamentals of Modeling and Control
for the Evolving Electric Power System Architectures”. The first author fondly
recalls Dr Narain Hingorani’s comments at the CURENT lunch meeting, in which
stated that “it is all about energy dynamics” and not about “low inertia”.
This comment has helped us pursue what is presented in this paper. The first
author is also very grateful for many years of collaborations with students
and colleagues, those whose papers are referred to and all others, who have
helped with concepts presented here.

## References

  * [1] J. Wyatt and M. Ilic, “Time-domain reactive power concepts for nonlinear, nonsinusoidal or nonperiodic networks,” in _IEEE international symposium on circuits and systems_. IEEE, 1990, pp. 387–390.
  * [2] “Performance-based energy resource feedback, optimization, and risk management,” https://arpa-e.energy.gov/technologies/programs/perform, accessed: 2022-05-11.
  * [3] K. Uhlen, S. Elenius, I. Norheim, J. Jyrinsalo, J. Elovaara, and E. Lakervi, “Application of linear analysis for stability improvements in the nordic power transmission system,” in _2003 IEEE Power Engineering Society General Meeting (IEEE Cat. No. 03CH37491)_ , vol. 4. IEEE, 2003, pp. 2097–2103.
  * [4] J. Adams, V. A. Pappu, and A. Dixit, “Ercot experience screening for sub-synchronous control interaction in the vicinity of series capacitor banks,” in _2012 IEEE Power and Energy Society General Meeting_. IEEE, 2012, pp. 1–5.
  * [5] M. J. Hossain, H. R. Pota, M. A. Mahmud, and R. A. Ramos, “Investigation of the impacts of large-scale wind power penetration on the angle and voltage stability of power systems,” _IEEE Systems journal_ , vol. 6, no. 1, pp. 76–84, 2011.
  * [6] F. Li, A. Z. Liu, and G. R. Swenson, “Characteristics of instabilities in the mesopause region over maui, hawaii,” _Journal of Geophysical Research: Atmospheres_ , vol. 110, no. D9, 2005.
  * [7] E. Troester, “New german grid codes for connecting pv systems to the medium voltage power grid,” in _2nd International workshop on concentrating photovoltaic power plants: optical design, production, grid connection_ , 2009, pp. 1–4.
  * [8] M. D. Ilić and S. Liu, _Hierarchical power systems control: its value in a changing industry_. Springer, 1996\.
  * [9] M. D. Ilic and J. Zaborszky, _Dynamics and control of large electric power systems_. Wiley New York, 2000.
  * [10] S. Cvijić, M. Ilić, E. Allen, and J. Lang, “Using extended ac optimal power flow for effective decision making,” in _2018 IEEE PES Innovative Smart Grid Technologies Conference Europe (ISGT-Europe)_. IEEE, 2018, pp. 1–6.
  * [11] M. Ilic and J. Lang, “Final report entitled voltage dispatch and pricing in support of efficient real power dispatch,” 2012.
  * [12] M. Ilic, R. S. Ulerio, E. Corbett, E. Austin, M. Shatz, and E. Limpaecher, “A framework for evaluating electric power grid improvements in puerto rico,” 2020\.
  * [13] A. Nejat, L. Solitare, E. Pettitt, and H. Mohsenian-Rad, “Equitable community resilience: The case of winter storm uri in texas,” _arXiv preprint arXiv:2201.06652_ , 2022.
  * [14] M. Ilic, R. Jaddivada, X. Miao, and N. Popli, “Toward multi-layered mpc for complex electric energy systems,” in _Handbook of Model Predictive Control_. Springer, 2019, pp. 625–663.
  * [15] M. Ilic and X. Liu, “A simple structural apporach to modeling and control of the inter-area dynamics of the large electric pwoer systems: Part ii nonlinear models of frequency and voltage dynamics,” in _Proceedings of the North American Power Conference (NAPS) IEEE_ , 1994.
  * [16] M. Ilic, X. Liu, B. Eidson, C. Vialas, and M. Athans, “A structure-based modeling and control of electric power systems,” _Automatica_ , vol. 33, no. 4, pp. 515–531, 1997.
  * [17] P. S. Kundur, N. J. Balu, and M. G. Lauby, “Power system dynamics and stability,” _Power system stability and control_ , vol. 3, 2017.
  * [18] M. D. Ilić and Q. Liu, “Toward sensing, communications and control architectures for frequency regulation in systems with highly variable resources,” in _Control and optimization methods for electric smart grids_. Springer, 2012, pp. 3–33.
  * [19] M. D. Ilić, “Dynamic monitoring and decision systems for enabling sustainable energy services,” _Proceedings of the IEEE_ , vol. 99, no. 1, pp. 58–79, 2010.
  * [20] Q. Liu, “Large-scale systems framework for coordinated frequency control of electric power systems,” Ph.D. dissertation, Carnegie Mellon University, 2013\.
  * [21] A. Zecevic and D. D. Siljak, _Control of complex systems: Structural constraints and uncertainty_. Springer Science & Business Media, 2010.
  * [22] M. Ilic and et. al, “Ieee tf report on pes simulation methods, models, and analysis techniques to represent the behavior of bulk power system connected inverter-based resources,” (Under Preparation) 2022.
  * [23] Y. Lin, J. H. Eto, B. B. Johnson, J. D. Flicker, R. H. Lasseter, H. N. Villegas Pico, G.-S. Seo, B. J. Pierre, and A. Ellis, “Research roadmap on grid-forming inverters,” National Renewable Energy Lab.(NREL), Golden, CO (United States), Tech. Rep., 2020.
  * [24] “Smart wires - reimagine the grid,” https://www.smartwires.com/, accessed: 2022-05-11.
  * [25] P. Godin, M. Fischer, H. Röttgers, A. Mendonca, and S. Engelken, “Wind power plant level testing of inertial response with optimised recovery behaviour,” _IET Renewable Power Generation_ , vol. 13, no. 5, pp. 676–683, 2019.
  * [26] M. Cvetković and M. D. Ilić, “Ectropy-based nonlinear control of facts for transient stabilization,” _IEEE Transactions on Power Systems_ , vol. 29, no. 6, pp. 3012–3020, 2014.
  * [27] M. Ilic, L. Xie, and Q. Liu, _Engineering IT-enabled sustainable electricity services: the tale of two low-cost green Azores Islands_. Springer Science & Business Media, 2013, vol. 30.
  * [28] M. D. Ilić and R. Jaddivada, “Multi-layered interactive energy space modeling for near-optimal electrification of terrestrial, shipboard and aircraft systems,” _Annual Reviews in Control_ , vol. 45, pp. 52–75, 2018\.
  * [29] M. D. Ilic and R. Jaddivada, “Fundamental modeling and conditions for realizable and efficient energy systems,” in _2018 IEEE Conference on Decision and Control (CDC)_. IEEE, 2018, pp. 5694–5701.
  * [30] R. Jaddivada and M. D. Ilic, “A feasible and stable distributed interactive control design in energy state space,” in _2021 60th IEEE Conference on Decision and Control (CDC)_. IEEE, 2021, pp. 4950–4957.


# Efficient Curriculum based Continual Learning with Informative Subset
Selection for Remote Sensing Scene Classification

S Divakar Bhat  Biplab Banerjee  Subhasis Chaudhuri  Avik Bhattacharya
Department of Electrical Engineering, Indian Institute of Technology Bombay,
Mumbai, Maharashtra 400076 India.  Centre of Studies in Resources Engineering,
Indian Institute of Technology Bombay, Mumbai, Maharashtra 400076 India.

###### Abstract

We tackle the problem of class incremental learning (CIL) in the realm of
land-cover classification from optical remote sensing (RS) images in this
paper. The paradigm of CIL has recently gained much prominence given the fact
that data are generally obtained in a sequential manner for real-world
phenomenon. However, CIL has not been extensively considered yet in the domain
of RS irrespective of the fact that the satellites tend to discover new
classes at different geographical locations temporally. With this motivation,
we propose a novel CIL framework inspired by the recent success of replay-
memory based approaches and tackling two of their shortcomings. In order to
reduce the effect of catastrophic forgetting of the old classes when a new
stream arrives, we learn a curriculum of the new classes based on their
similarity with the old classes. This is found to limit the degree of
forgetting substantially. Next while constructing the replay memory, instead
of randomly selecting samples from the old streams, we propose a sample
selection strategy which ensures the selection of highly confident samples so
as to reduce the effects of noise. We observe a sharp improvement in the CIL
performance with the proposed components. Experimental results on the
benchmark NWPU-RESISC45, PatternNet, and EuroSAT datasets confirm that our
method offers improved stability-plasticity trade-off than the literature.

††journal: arxiv

##  1 Introduction

Remote sensing image analysis has been witnessing a surge in the development
of innovative solutions to many advanced problems with considerable momentum
in the recent past. Reinforcing the notion of data-driven feature learning,
deep convolutional neural networks can be attributed the credit for a
significant portion of this success. This, along with the large number of
satellites being deployed with its highly advanced imaging technologies
capable of accumulating abundant data [1] has resulted in a peak in the
research happening in the field.

Figure 1: Comparison of the conventional learning method and the proposed
continual learning scheme. (a) shows a typical deep learning scenario in which
the entire dataset is supposed to be available during training. (b) depicts
the proposed method for learning incrementally by combining a curriculum
learning strategy with continuous learning integrated with the informative
subset selection technique.

Unlike the conventional setting, where the entire information is present
during the training phase, the land cover classes are acquired sequentially by
the earth observation sensors. Therefore, the data thus collected is prone to
limitations in temporal, spatial, and spectral resolution. For instance, a
satellite will continuously send raw image data from across the world,
creating a rich reservoir of information that could be used for many
applications. However, this requires these samples to be annotated
meticulously and adequately. The gradual accumulation of these vast amounts of
multi-temporal data resulting from continuous inflow from the sensors thus
demands a framework to learn incrementally.

However, it is observed that training a deep learning framework on dynamically
varying streams of incoming data is particularly complex and demanding. This
is due to the fact that while the network tends to forget previously acquired
information as a result of its sensitivity to quick updates, it also shows
drastically poor reactivity to slow and gradual updates [2]. This is the well-
known phenomenon of the stability-plasticity dilemma [3], which is a
significant constraint for developing real-time artificial learning systems.
This dramatic decrease in the performance of the model under consideration
when it is trained over a continually growing dynamic distribution of data is
termed as catastrophic forgetting [4]. The most naive approach that can be
adopted to solve this issue is to train the network from scratch every time a
new set of data is introduced. However, as expected, this will account for a
massive overhead of retaining the whole set of data acquired continually by
earth observation sensors from across the globe. Therefore, it is necessary to
develop novel learning paradigms that can handle catastrophic forgetting, thus
facilitating on the go training of the deep neural network.

As discussed before, despite the advancements made in feature-based learning
in remote sensing, the perpetual flow of data from satellites pose severe
challenges in terms of memory resource requirements, the need for proper
annotation of this vast collection of image data, etc. for making the most out
of these rich satellite image data reservoirs. Moreover, not much work has
been done in this direction in remote sensing despite the clear possibilities
to explore. Also, it is not practically feasible to always expect the
availability of having all the class information beforehand. And contrary to
the expectation from any efficient learning systems, none of the current
methods exploit the similarity between the incoming stream of data with that
of the previously acquired information. Furthermore, existing methodologies
that employ methods like the rehearsal-based approach do not consider how the
selection of old samples could affect the extent of interference the network
is subjected to. Memory replay-based methods explicitly hold a steady ground
as most suitable for continual learning in satellite images, mainly due to the
presence of rich semantic concepts in satellite images that a GAN-based method
will fail to generate precisely.

To address these problems, we propose a novel approach to continual learning
by employing curriculum learning and an intelligent selection of informative
samples for rehearsal-based learning. Introduced in [5], curriculum learning
is an efficient learning paradigm that proposes to present data to the model
in a more meaningful manner based on the complexity of the concepts. This is
reinforced by [6, 7] which points out that when a task is decomposed into sub-
tasks from easy to complex, animals tend to learn faster. We also employ an
entropy-based efficient selection of informative samples inspired from works
in active learning like [8], thus aiming to make the most out of the available
budget for sample retention. Commonly used techniques for sample selection for
computer vision problems is discussed in [9] and have been successfully
implemented for tracking [10], object detection [11], etc. In this work, we
propose a novel entropy-based efficient selection of representative samples
for replay coupled with a curriculum-based continual learning approach for
remote sensing scene classification. Unlike existing replay-based works in
vision like [12] and remote sensing works like [13], we propose an end-to-end
model and do not use any extra network for sample selection. The proposed work
consists of the following contributions:

  * 1.

We tackle the problem of CIL in the context of land-cover classification from
optical remote sensing data.

  * 2.

Our method is developed on the premise of rehearsal-based memory replay.
However, we propose two novel components within the framework to better tackle
the issues related to stability and plasticity. A novel continual learning
approach for remote sensing image classification using a curriculum learning
technique. We also present a pseudo-teacher-student-based approach for
continual learning using the curriculum.

  * 3.

We perform thorough experiments with three optical datasets to demonstrate how
a combination of proposed efficient sample selection technique and curriculum
learning improves the performance of a continual learning network for
satellite image classification.

We have provided a brief overview of the proposed approach in the Figure 1
motivating the relevance of the work for real time applications like remote
sensing.

Figure 2: The complete pipeline of the proposed framework. The incoming novel
classes are organised into a curriculum for the collection of tasks $t$. Based
on the curriculum produced, the network $\Theta_{t}$ learns the features from
a new stream of data. To transfer previously learned information, we rely on
knowledge distillation from $\Theta_{t-1}$.

##  2 Related Works

There have been many approaches that has been proposed to alleviate the issue
of catastrophic forgetting both using classical and modern methods in feature
based learning. Notable of these works are [14, 15] which proposed to use SVM
and RBF based models respectively for training in incremental setting. While,
in [16], it was proposed to use hierarchically growing random forest-based
approach.

In [17] and [18], the authors employ an elastic weight consolidation term
which was used to assign importance to neurons corresponding both old and new
tasks and impose a quadratic penalty on the change between previous and
current network. This importance weight computation was made online in [19].
While the authors align the predictions based on the current task in [20].

The approach used in [12] suggests to train at a given instant by using a
small fraction of retained samples from previous class along with the new
samples. While [20] proposes to use knowledge distillation [21] for
facilitating proper transfer of knowledge from the previous tasks to to the
updated network. Whereas in [22, 23] recommends an approach where most
representative samples are used in combination with the knowledge distillation
based approach to mitigate forgetting. In [24] the authors propose a
generative approach where the samples corresponding to old classes are
synthesised. Most recent works in the field like in [25] the authors propose a
replay based approach where only the most conflicted samples are retrieved.
While, performance improvement is observed in [26] by greedily storing
previous samples and retraining on these during testing. Whereas an expansion-
based approach is employed in [27] which is built on the principles of
Bayesian nonparametric.

There have been only a handful of works that explored continual learning in
the field of remote sensing. These few existing techniques like that in [13]
relies on a separate module for the retrieval of samples for replay. While the
authors explore incremental learning for the task of semantic segmentation in
[23] with only relatively smaller number of new classes per stream. In [28],
continual learning is employed for end-to-end semantic segmentation for global
mapping of buildings from VHR satellite images. Whereas the authors have
introduced a large scale remote sensing benchmark dataset for continual
learning in [29]. Despite many works that explore continual learning, none
investigate the efficacy of a combination of curriculum learning and
informative subset selection for better performance. In this perspective, we
propose a versatile framework for continual learning in remote sensing scene
classification.

##  3 Methodology

The complete overview of the proposed framework is illustrated in the Figure
2, where it consists of mainly three core modules, a curriculum based pseudo-
teacher-student network, a curriculum generator and a informative subset
selector. In this section we first formulate the problem definition, and then
introduce each section of the framework in detail, respectively.

###  3.1 Problem Definition and Notations

Consider a model $\Theta_{t-1}$ which is to be trained with an incoming set of
tasks indexed as $t$ with $k$ novel classes per stream which can be denoted as
$\mathcal{D}_{train}^{t}=\bigcup_{i=1}^{k}\\{(\mathcal{X}_{(t-1)k+i}^{j},\mathcal{Y}_{(t-1)k+i}^{j})\\}_{j=1}^{N}$,
with $N$ number of samples per class in the given incoming set of tasks. The
ordered pair $(\mathcal{X}_{(t-1)k+i}^{j},\mathcal{Y}_{(t-1)k+i}^{j})$ the
$j^{th}$ represents the sample and corresponding ground truth for the $i^{th}$
class in set of task $t$. Our objective is to train the model on the new
stream of data without forgetting the previously acquired information
resulting in the updated model $\Theta_{t}$.

For the discussions in this paper, we will use the variable $t$ to denote the
current set of tasks or stream, $\mathcal{D}_{train}^{t}$ will denote the new
set of tasks, while $\mathcal{D}_{prev}^{t}$ will denote the data retained
from the previous tasks. i.e,
$\mathcal{D}_{prev}^{t}=\bigcup_{w=1}^{k(t-1)}\\{\mathcal{X}_{w_{prev}}^{j}\\}_{j=1}^{m.N}$
where $w$ indexes the previously learned classes, and $m$ is the fraction of
data retained. For ease of representation, we may drop the indices to
represent the sets of sample and ground truth belonging to
$\mathcal{D}_{train}^{t}$ and $\mathcal{D}_{prev}^{t}$ as
$\\{\mathcal{X}_{train}^{t},\mathcal{Y}_{train}^{t}\\}$ and
$\\{\mathcal{X}_{prev}^{t},\mathcal{Y}_{prev}^{t}\\}$ respectively.

###  3.2 Overview

We organise the novel classes to be learned based on the curriculum created
for each incoming set of tasks $t>1$. The curriculum for a collection of
tasks, say $t$, is created using a curriculum generation algorithm based on
previously learned information from the old model, $\Theta_{t-1}$, as shown in
section 3.3. Then, when training on the new classes, we use a combination of
replay and information distillation techniques to ensure minimal interference
with previously learned features. We also use an insightful subset selection
algorithm to prune data in order to make the most of the memory budget, as
discussed in section 3.5.Algorithm depicting the overall process is added in
the supplementary.

###  3.3 Curriculum Generation

In this work we propose a curriculum based continual learning framework
building on the idea of learning efficiently using a curriculum based on
similarity of concepts. For this we introduce a strategy for devising
curriculum based on the similarity between the concepts to be learned. The
basic idea is to learn the concepts similar to the previously learned
information. We formulate a cosine similarity based feature ranking for the
novel set of tasks $t$, with respect to the features learned till the previous
set of tasks $t-1$. Solely for the purpose of curriculum generation and to
execute equation 1, we retain the extracted features corresponding to $N$
samples, from each of $k$ classes of the previous set of task $t-1$. Note that
no previous data is retained nor these extracted features are used for any
sort of training.

Consider the model $\Theta_{t-1}$ trained incrementally on the set of tasks
till $t-1$. Now we extract the features corresponding to the set of tasks $t$
and $t-1$ and obtain the representative prototype per class denoted by
$\mu_{l}^{q}$ for both these set of tasks as shown in Equation 1.

$\mu_{l}^{q}=\frac{1}{N}\sum_{j=1}^{N}\Theta_{t-1}(x_{j}^{l,q})$ (1)

Where, $l\in\\{t-1,t\\}$ denoting the previous and current set of tasks, $q$
is used to index the classes in each set of task with each class having a
total of $N$ samples.

With these per class prototypes in hand, we now compute the curriculum for set
of tasks $t$ as,

$C_{t}=\operatorname*{arg\,max}_{c}\\{[s_{r1},s_{r2},\dots,s_{rc},\dots,s_{rk}]\\}_{r=1}^{k}$
(2)

where $s_{rc}$ denotes an element of the similarity matrix $S$, denoting the
similarity measure between the class prototypes of the class $r$ from the
previous set of task $t-1$ with that of the $k$ classes from the current set
of tasks. This similarity matrix $S$ is computed as shown in Equation 3,

$s_{rc}=\frac{\mu_{r}^{T}\mu_{c}}{\left\|\mu_{r}\right\|\left\|\mu_{c}\right\|}$
(3)

The argmax function in Equation 2 gives us the index $r$ indicating closest
category to the previous set of tasks $t-1$ among the current set of tasks
$t$. Thus a $k$ dimensional vector $C_{t}$ gives us the curriculum to train
the model $\Theta_{t}$ incrementally as discussed in the section 3.4.

###  3.4 Curriculum based Continual Learning

The training of the model $\Theta$ can be split into two phases with the
initial phase of training, for $t=1$ performed over a random set of $k$
classes which will form the first set of tasks. For this phase we train the
model using the traditional multi-class cross entropy loss function as can be
seen in Equation 4.

After this initial training phase we have the continual learning phase with
the new set of incoming tasks i.e, $t=2,t=3,\dots$ and so on. For each set of
incoming task, say $t$ we will generate the curriculum as explained in section
3.3. Based on the curriculum $C_{t}$ thus obtained we train on the new set of
tasks using a combination of multi class cross-entropy and contrastive loss
based regularizer imposed over the distilled logits from the teacher model
corresponding to previously learned categories as shown in Equation 6. The
contrastive distillation based regularization, $\mathcal{R}$ is applied only
to the output of the old classification layers which corresponds to the
previous classes. While the multi-class cross-entropy loss, $\mathcal{L_{C}}$
is employed upon all the classification layers corresponding to both old and
new classes.

$\mathcal{L_{C}}=-\frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{k}p_{ij}\log q_{ij}$
(4)
$\mathcal{R}=-\frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{P}\log\frac{\exp(p_{ij}^{\prime}.q_{ij}^{\prime})}{\sum_{j}\exp(p_{ij}^{\prime}.q_{ij}^{\prime})}$
(5) $\mathcal{L}_{T}=\mathcal{L_{C}}+\mathcal{R}$ (6)

$N$ is the number of samples, and $P$ denotes the total number of previous
classes. $p_{ij}^{\prime}$ is the ground truth and $q_{ij}^{\prime}$ denotes
the softmax output for the $i^{th}$ sample of the $j^{th}$. For proper
transfer of the already learned features of previous tasks we employ the
knowledge distillation technique as,

$p_{i}^{\prime}=\frac{\exp{(z_{i}/T)}}{\sum_{j}\exp{(z_{j}/T)}}$ (7)

where, $p_{i}$ is obtained by performing distillation over the logit $z_{i}$
using $T$ which is called the temperature parameter. The value of $T$ is kept
equal to $2$ for optimal performance as suggested in the work [21].

This is then followed by a representative subset selection stage as explained
in the section 3.5. Also every phase of learning continually over the incoming
set of new tasks is followed by a fine-tuning stage as followed in [22] and
likes, employed to remove the bias due to class imbalance.

###  3.5 Informative Subset Selection

After each incremental training stage explained in section 3.4, followed by
the fine-tuning stage. We select a pool of $\epsilon\times N$ samples from
each category in the current set of tasks for representing the corresponding
classes in the subsequent learning stages. We try to incorporate the idea of
teacher based sample selection for informative subset selection, goal of which
is to maximize the efficiency of the continual learning process using this
limited budget of retained samples.

We start by extracting the features
$\bigcup_{i=1}^{k}\\{\Theta_{t}(\mathcal{X}^{ij})\\}_{j=1}^{N}$ corresponding
to the $N$ samples of each class in the current set of novel tasks. The
extracted features is then subjected to k-means clustering. We obtain the
cluster centroids $\\{\Gamma^{l}\\}_{l=1}^{k}$ for the clusters
$\\{K_{l}\\}_{l=1}^{k}$. Now, we compute the probability of each feature in
the given class to belong to the corresponding centroid by calculating the
feature wise squared distance from the centroid as shown in Equation 8.

$p(\Theta_{t}(\mathcal{X}^{ij})\in
K_{l})=\frac{\exp{(-\left\|\Theta_{t}(\mathcal{X}^{ij})-\Gamma^{j}\right\|_{2}^{2})}}{\sum_{v=1}^{k}\exp{(-\left\|\Theta_{t}(\mathcal{X}^{ij})-\Gamma^{v}\right\|_{2}^{2})}}$
(8)

This is used to get the probability vector $\mathcal{P}$ as shown in Equation
9

$\mathcal{P}(\Theta_{t}(\mathcal{X}^{ij}))=[p(\Theta_{t}(\mathcal{X}^{ij})\in
K_{1}),\dots,p(\Theta_{t}(\mathcal{X}^{ij})\in K_{k})]$ (9)

Now we compute the entropy of each image with respect to the cluster centroid
of their corresponding class as,

$\mathcal{H}(\mathcal{P}_{ij})=-\sum_{i=1}^{k}p(\Theta_{t}(\mathcal{X}^{ij})\in
K_{l})\log(p(\Theta_{t}(\mathcal{X}^{ij})\in K_{l}))$ (10)

Based on the computed entropy per sample in each cluster we, select a subset
by pruning $N(1-\epsilon)$ samples which is farthest from the cluster center.
Thus the informative subset $\mathcal{S}_{t}$ of the current set of tasks $t$
can be represented as,

$\mathcal{S}_{t}\subsetneq\bigcup_{i=1}^{k}\\{\Theta_{t}(\mathcal{X}^{ij})\\}_{j=1}^{N}$
(11)

such that, $\left|S_{t}\right|=\epsilon kN$.

| Dataset
---|---
Method | NWPU | EuroSat | PatternNet
LwF [20] | 30.17 | 45.67 | 29.79
EwC [17] | 20.50 | 24.00 | 30.39
E2E [22] | 29.40 | 78.69 | 27.08
DR [30] | 22.09 | - | -
iCaRL [12] | 37.20 | 76.81 | 60.97
RS [13] | 36.54 | - | 57.38
MIR [25] | 27.95 | 53.42 | 53.82
GDUMB [26] | 39.68 | 77.90 | 59.02
CN-DPM [27] | 20.58 | 65.47 | 37.40
Proposed | 85.73 | 83.80 | 93.67
w/o curriculum | 75.40 | 76.67 | 84.75
w/o subset selection | 78.90 | 78.41 | 89.63
Table 1: Comparison of accuracy (in %) on NWPU-RESISC45, EuroSat and
PatternNet dataset as shown, using existing approaches. (’-’ denotes the
result is not reported)

Joint* Proposed w/o curriculum and ISS w/o curriculum w/o ISS Stream Accuracy
Accuracy Forgetting Accuracy Forgetting Accuracy Accuracy 1 92.76 – 93.14 –
91.62 81.33 2 93.65 1.52 69.02 11.05 69.90 76.00 3 86.41 0.21 78.51 1.08 78.35
89.90 4 89.75 0.38 81.16 1.28 82.99 87.03 5 87.59 89.10 1.05 77.02 7.906 78.83
82.53 6 86.25 1.25 77.47 6.703 78.15 78.23 7 82.90 1.96 75.48 7.602 73.67
76.72 8 79.30 4.03 71.34 11.02 72.47 72.64 9 78.48 2.61 68.41 12.75 68.87
68.19

Table 2: Ablation results on the proposed framework in the presence and
absence of curriculum and subset selection as obtained for the NWPU-RESISC45
dataset. All the reported accuracy and forgetting values are in %. ISS stands
for Informative Subset Selection. * not a result obtained on incremental
learning setting

##  4 Experiments

In this section we discuss about the experiments conducted to evaluate the
proposed approach. In the following sections we discuss about the datasets
used and also on some selected comparisons. Finally we show some ablation
studies and discuss on the same.

###  4.1 Datasets and Implementation details

We show our results mainly on NWPU-RESISC45 [31], developed by Northwestern
Polytechnical University as a publicly accessible benchmark for Remote Sensing
Image Scene Classification (RESISC) (NWPU). This dataset contains 31,500
images divided into 45 categories. Other datasets which we conduct experiments
on include PatternNet [32], a high-resolution remote sensing image dataset
collected using Google Earth imagery or the Google Map API for satellite image
retrieval, divided into 38 classes and RGB version of EuroSat [33] dataset
based on Sentinel-2 satellite images containing 27000 labelled and geo-
referenced samples divided into ten classes. As the pipeline’s foundation, we
use the pre-trained VGG-16 network. For initialising, we use [34] and after
each convolutional layer, we use batch-norm [35].

We use the Adam optimizer for optimization, with a learning rate of $1\times
10^{-6}$ and a weight decay of $1\times 10^{-4}$. Each input image is resized
to $224\times 224$ pixels and the number of training epochs is kept at 40 for
each set of tasks. We extract a 128 dimension feature vector, and hence we add
a 128-dimension fully connected penultimate layer to the architecture. The
value of $\epsilon$ is set at $0.3$ and we use the Adam optimizer with a
learning rate of $1\times 10^{-7}$ for fine-tuning for 30 epochs. The
experiments were conducted using a single Nvidia GeForce GTX 1080 Ti GPU.

To evaluate the performance of our model, we report the average accuracy
starting from the incremental stage, training time taken in hours and also the
forgetting measure as described in [36].

###  4.2 Comparison with existing works

The results of the proposed solution were compared to the results of various
algorithms in the following experiments: 1\. The implementation as done in
[20] is referred to as LwF. 2\. EWC refers to the work done in [17] with
elastic weight consolidation, and 3. E2E refers to the end-to-end incremental
learning work proposed in [22]. 4\. The work suggested in [12] is referred to
as iCaRL and 5. RS denotes the replay based work in [13]. Please notice that
for reference, the terms DR, MIR, GDUMB, and CN-DPM are used, which refer to
the works proposed in [30], [25], [26], and [27], respectively.

To reproduce these findings, we depend on the extensive work presented in [37]
and the code given here111https://github.com/RaptorMai/online-continual-
learning. The results for EWC are obtained using the code contained at
222https://github.com/xialeiliu/RotateNetworks while the code from this url
333https://github.com/kibok90/iccv20 is used to get E2E and LwF.

For the NWPU-RESISC45, EuroSat, and PatternNet datasets, the Table 1 displays
the results for a fixed incremental step size of five classes, two classes,
and six classes per set of task, respectively. Our method outperforms the
current algorithms by a margin of around $45\%$ and $18.33\%$ for the NWPU-
RESISC45 and EuroSat datasets, respectively, and by $32.7\%$ on the PatternNet
dataset, from the closest performing algorithm based on these findings.

figureAccuracy of different per class memory sizes for NWPU-RESISC45 dataset
for a step size of 9 classes.

Figure 3:  Convergence time Vs stream plot for comparing multiple methods. The
result is shown for a step size of 5 on NWPU dataset.

###  4.3 Ablation Studies

In this paper, we propose an efficient continual learning system for satellite
image classification. By promoting the learning process and assisting it in
achieving a better optimum, we expect the curriculum learning approach to
reduce the time it takes to converge. The proposed subset selection for sample
retention aims to utilize the sample retention budget we have effectively. The
suggested solution is examined in the following sections when the curriculum
is omitted, subset selection is missing, or a mix of these.

Step size Accuracy (in %) Time taken (in hrs) Our Approach w/o curriculum Our
Approach w/o curriculum 5 85.73 75.40 32.05 39.29 9 76.17 63.31 20.17 30.75

Table 3: Accuracy and total time consumed for different step sizes for the
NWPU-RESISC45 dataset for proposed approach and without a curriculum.

Convergence Time Comparison To demonstrate the improved convergence time, we
plot the time taken per stream to converge for both the curriculum-based and
curriculum-less approaches. The time required is calculated based on the 40
epochs of training and includes the curriculum development and fine-tuning
stages for consistency.

In contrast to other methods, it is clear that our system has the fastest
convergence time. Compared to DR [30]’s the highest time of 182.09 hours and
39.29 hours when the curriculum is excluded from our system, the total time
for the whole process is around 32.05 hours. These findings are also
consistent across different step sizes as presented in Table 3, where for a
step size of 9, the proposed approach is quicker and converges in 20.17 hours
while it takes 30.75 hours when the curriculum is removed.

Accuracy and extent of forgetting Table 3 shows the average accuracy obtained
for the proposed framework and curriculum-less learning for different
incremental step sizes. From the Table 1 and 2, we observe that the proposed
framework outperforms those without curriculum by a margin of around $10\%$
and with the subset selection removed by a margin of $7\%$ for the step size
considered. Also, it is to be noted that the extent of forgetting is
relatively minimal for the proposed approach, as shown in Figure 4b. We also
see that the joint training accuracy is higher, as anticipated by a small
margin. However, it should be remembered that this is an outcome of training
the network with all classes together in a conventional manner and not how
real-world systems operate.

(a)

(b)

Figure 4: Results for accuracy and forgetting at each stream for the NWPU-
RESISC45 dataset. The results are shown for both with and without curriculum
approach.

Effect of memory budget This section looks at how the model’s output changes
when the per-class memory budget changes. It is clear from Figure 3 that the
output degrades as the number of samples retained per class is reduced. This
pattern is in line with what one would expect as the number of samples used to
memorize previous classes decreases. However, note that the extent of this
impact is mitigated with use of subset selection algorithm.

Nonetheless, for the NWPU dataset, the efficiency decline for the proposed
approach with change in per-class memory is by a margin of $29.81\%$ while the
amount of decrease in samples retained ranges from $5-30\%$, accounting for a
$25\%$ variation in memory usage.

##  5 Conclusions

In this paper, we propose a novel CIL framework for tackling the problem of
incremental land-cover classification from optical remote sensing images. Our
method is developed on the premise of rehearsal-based memory replay, where a
subset of samples from the old classes is preserved and replayed periodically.
However, there are two inherent problems of such an approach which we address
here. First, we limit the catastrophic forgetting by using curriculum learning
on the novel classes by exploiting their similarities with the old classes.
Next, we propose an improved replay memory by selecting a highly reliable
subset from the old classes. We experimentally find that the proposed
modifications offer an improved stability-plasticity trade-off for CIL with
respect to a number of baseline techniques from the literature. We are
currently interested in extending our model for non-rehearsal-based memory
replay models and considering the possibility of having unknown novel classes
in each episode.

## References

  * [1] A. S. Belward, J. O. Skøien, Who launched what, when and why; trends in global land-cover observation capacity from civilian earth observation satellites, ISPRS Journal of Photogrammetry and Remote Sensing 103 (2015) 115 – 128, global Land Cover Mapping and Monitoring.  doi:https://doi.org/10.1016/j.isprsjprs.2014.03.009.
URL http://www.sciencedirect.com/science/article/pii/S0924271614000720

  * [2] A. Gepperth, B. Hammer, Incremental learning algorithms and applications, in: European symposium on artificial neural networks (ESANN), 2016.
  * [3] M. Mermillod, A. Bugaiska, P. Bonin, The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects, Frontiers in psychology 4 (2013) 504.
  * [4] R. M. French, Catastrophic forgetting in connectionist networks, Trends in cognitive sciences 3 (4) (1999) 128–135.
  * [5] Y. Bengio, J. Louradour, R. Collobert, J. Weston, Curriculum learning, in: Proceedings of the 26th annual international conference on machine learning, ACM, 2009, pp. 41–48.
  * [6] B. F. Skinner, Reinforcement today., American Psychologist 13 (3) (1958) 94.
  * [7] K. A. Krueger, P. Dayan, Flexible shaping: How learning in small steps helps, Cognition 110 (3) (2009) 380–394.
  * [8] S. Paul, J. H. Bappy, A. K. Roy-Chowdhury, Efficient selection of informative and diverse training samples with applications in scene classification, in: 2016 IEEE International Conference on Image Processing (ICIP), IEEE, 2016, pp. 494–498.
  * [9] B. Settles, Active learning literature survey.
  * [10] C. Vondrick, D. Ramanan, Video annotation and tracking with active learning, Advances in Neural Information Processing Systems 24 (2011) 28–36.
  * [11] S. Vijayanarasimhan, K. Grauman, Large-scale live active learning: Training object detectors with crawled data and crowds, International journal of computer vision 108 (1) (2014) 97–114.
  * [12] S.-A. Rebuffi, A. Kolesnikov, G. Sperl, C. H. Lampert, icarl: Incremental classifier and representation learning, in: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017, pp. 2001–2010.
  * [13] N. Ammour, Y. Bazi, H. Alhichri, N. Alajlan, Continual learning approach for remote sensing scene classification, IEEE Geoscience and Remote Sensing Letters.
  * [14] Y.-M. Wen, B.-L. Lu, Incremental learning of support vector machines by classifier combining, in: Z.-H. Zhou, H. Li, Q. Yang (Eds.), Advances in Knowledge Discovery and Data Mining, Springer Berlin Heidelberg, Berlin, Heidelberg, 2007, pp. 904–911.
  * [15] L. Bruzzone, D. F. Prieto, An incremental-learning neural network for the classification of remote-sensing images, Pattern Recognition Letters 20 (11-13) (1999) 1241–1248.
  * [16] B. Lakshminarayanan, D. M. Roy, Y. W. Teh, Mondrian forests: Efficient online random forests, in: Advances in neural information processing systems, 2014, pp. 3140–3148.
  * [17] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al., Overcoming catastrophic forgetting in neural networks, Proceedings of the national academy of sciences 114 (13) (2017) 3521–3526.
  * [18] X. Liu, M. Masana, L. Herranz, J. Van de Weijer, A. M. Lopez, A. D. Bagdanov, Rotate your networks: Better weight consolidation and less catastrophic forgetting, in: 2018 24th International Conference on Pattern Recognition (ICPR), IEEE, 2018, pp. 2262–2268.
  * [19] F. Zenke, B. Poole, S. Ganguli, Continual learning through synaptic intelligence, Proceedings of machine learning research 70 (2017) 3987.
  * [20] Z. Li, D. Hoiem, Learning without forgetting, IEEE transactions on pattern analysis and machine intelligence 40 (12) (2017) 2935–2947.
  * [21] G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, arXiv preprint arXiv:1503.02531.
  * [22] F. M. Castro, M. J. Marín-Jiménez, N. Guil, C. Schmid, K. Alahari, End-to-end incremental learning, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 233–248.
  * [23] O. Tasar, Y. Tarabalka, P. Alliez, Incremental learning for semantic segmentation of large-scale remote sensing data, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12 (9) (2019) 3524–3537.
  * [24] Y. Wu, Y. Chen, L. Wang, Y. Ye, Z. Liu, Y. Guo, Z. Zhang, Y. Fu, Incremental classifier learning with generative adversarial networks, arXiv preprint arXiv:1802.00853.
  * [25] R. Aljundi, E. Belilovsky, T. Tuytelaars, L. Charlin, M. Caccia, M. Lin, L. Page-Caccia, Online continual learning with maximal interfered retrieval, in: H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, R. Garnett (Eds.), Advances in Neural Information Processing Systems 32, Curran Associates, Inc., 2019, pp. 11849–11860.
URL http://papers.nips.cc/paper/9357-online-continual-learning-with-maximal-
interfered-retrieval.pdf

  * [26] A. Prabhu, P. Torr, P. Dokania, Gdumb: A simple approach that questions our progress in continual learning, in: The European Conference on Computer Vision (ECCV), 2020.
  * [27] S. Lee, J. Ha, D. Zhang, G. Kim, A neural dirichlet process mixture model for task-free continual learning, arXiv preprint arXiv:2001.00689.
  * [28] N. Yang, H. Tang, Geoboost: an incremental deep learning approach toward global mapping of buildings from vhr remote sensing images, Remote Sensing 12 (11) (2020) 1794.
  * [29] H. Li, H. Jiang, X. Gu, J. Peng, W. Li, L. Hong, C. Tao, Clrs: Continual learning benchmark for remote sensing image scene classification, Sensors 20 (4) (2020) 1226.
  * [30] S. Hou, X. Pan, C. C. Loy, Z. Wang, D. Lin, Lifelong learning via progressive distillation and retrospection, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 437–452.
  * [31] G. Cheng, J. Han, X. Lu, Remote sensing image scene classification: Benchmark and state of the art, Proceedings of the IEEE 105 (10) (2017) 1865–1883.
  * [32] W. Zhou, S. Newsam, C. Li, Z. Shao, Patternnet: A benchmark dataset for performance evaluation of remote sensing image retrieval, ISPRS journal of photogrammetry and remote sensing 145 (2018) 197–209.
  * [33] P. Helber, B. Bischke, A. Dengel, D. Borth, Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.
  * [34] X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Proceedings of the thirteenth international conference on artificial intelligence and statistics, 2010, pp. 249–256.
  * [35] S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, arXiv preprint arXiv:1502.03167.
  * [36] A. Chaudhry, P. K. Dokania, T. Ajanthan, P. H. Torr, Riemannian walk for incremental learning: Understanding forgetting and intransigence, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 532–547.
  * [37] Z. Mai, R. Li, J. Jeong, D. Quispe, H. Kim, S. Sanner, Online continual learning in image classification: An empirical survey, arXiv preprint arXiv:2101.10423.