# Investigating Sanity Checks for Saliency Maps with Image and Text
Classification

Narine Kokhlikyan Vivek Miglani Bilal Alsallakh Miguel Martin Orion Reblitz-
Richardson
Facebook AI
<EMAIL_ADDRESS>
###### Abstract

Saliency maps have shown to be both useful and misleading for explaining model
predictions especially in the context of images. In this paper, we perform
sanity checks for text modality and show that the conclusions made for image
do not directly transfer to text. We also analyze the effects of the input
multiplier in certain saliency maps using similarity scores, max-sensitivity
and infidelity evaluation metrics. Our observations reveal that the input
multiplier carries input’s structural patterns in explanation maps, thus
leading to similar results regardless of the choice of model parameters. We
also show that the smoothness of a Neural Network (NN) function can affect the
quality of saliency-based explanations. Our investigations reveal that
replacing ReLUs with Softplus and MaxPool with smoother variants such as
LogSumExp (LSE) can lead to explanations that are more reliable based on the
infidelity evaluation metric.

##  1 Introduction

Gradient-based explanation algorithms, also known as saliency maps, help us to
explain model predictions (Ancona et al., 2018) w.r.t. input features and
internal layers. Despite their popularity, Adebayo et al. (2018) and Sixt et
al. (2020) show that some gradient-based methods produce explanations that are
misleading and invariant to data and parameter randomization. Moreover,
Dombrowski et al. (2019) show that the explanations can be easily manipulated
by nearly imperceptible perturbations. The latter is tightly connected to the
non-smooth nature of explanation functions which suggests that replacing ReLU
activations with Softplus increases robustness of explanations. Although
gradient-based methods have been studied thoroughly, their evaluation has been
primarily performed for Computer Vision (CV) models, thus their effect on
other modalities remains largely unexplored. Accordingly, our contributions
are structured as follows:

  * •

First, we investigate the applicability of saliency maps in the natural
language domain and focus on text classification. We show that saliency maps
are more sensitive to parameter randomization when applied to text and less
sensitive when applied to image.

  * •

Second, we further analyse how the input multiplier affects explanation
evaluation metrics and similarity scores using sanity checks both for images
and text.

  * •

Finally, we show that the smoothness of the NN function is related not only to
the quality of model explanations but also to the infidelity (Yeh et al.,
2019) evaluation metric. We show that the substitution of ReLU with Softplus
and MaxPool with LSE makes explanations not only robust but also more
trustworthy based on the infidelity evaluation metric.

##  2 Related methods and techniques

Adebayo et al. (2018) propose parameter and data randomization tests, also
known as sanity checks, in order to assess the quality of explanation methods.
Parameter randomization implies gradual cascade-like randomization of all
model weights starting from the top (logit) moving to the bottom layers of the
network. Data randomization implies that explanations are computed for a model
that is trained on randomly perturbed ground truth labels. Adebayo et al.
(2018) show that most gradient-based explanation methods can be seen as edge
detectors since they highlight image contours and are invariant to parameter
and data randomization. Based on that Adebayo et al. (2018) claim that
gradient-based methods are inadequate tools for model explanation. They also
mention that parameter and data invariance is likely to be related to the
input multiplier in some gradient-based methods; however, they do not measure
input multiplier’s quantitative impact.

We show that certain gradient-based methods such as Integrated Gradients (IG),
reported to be invariant to parameter and data randomization, do not
necessarily act as edge detectors. For the sake of brevity, we report data
randomization results in appendices A and B. We perform thorough experiments
and show that the observations made for image modality do not necessarily
transfer to modalities such as text. We perform our analysis for most
explanation methods mentioned in Adebayo et al. (2018). For brevity, we focus
on the IG algorithm because it has both local and global variants (Ancona et
al., 2018). We report experimental results for other methods in Appendix A. We
define an input $\displaystyle x\in{\mathbb{R}}^{N}$, a baseline
$\displaystyle x_{0}\in{\mathbb{R}}^{N}$, a NN model represented by a
continuous function $\displaystyle
F:{\mathbb{R}}^{N}\rightarrow{\mathbb{R}}^{C}$, where $C$ is the number of
classes and an explanation function
$\displaystyle\Phi:\mathcal{F}\times{\mathbb{R}}^{N}\rightarrow{\mathbb{R}}^{N}$.
Given $\alpha\in[0,1]$ as a scaling factor along the straight line between
$x_{0}$ and $x$, global IG along the $i^{th}$ dimension of the input $x$ is
defined as follows:

$\Phi({F,x^{i}})=(x^{i}-x^{i}_{0})\cdot\int_{0}^{1}\frac{\partial
F(x_{0}^{i}+\alpha\cdot(x^{i}-x_{0}^{i}))d\alpha}{dx^{i}}$ (1)

The local definition, in contrast, does not include $x^{i}-x_{0}^{i}$ as a
multiplier. This multiplier is used to magnify the sensitivities by a constant
factor and is independent of model parameters. This explains why global IG
fails sanity checks when applied to images (Adebayo et al., 2018). The same
observations are valid for InputXGradient (Shrikumar et al., 2017b), DeepLift
(Shrikumar et al., 2017a), and Gradient SHAP as they incorporate the input
multipliers. Certain algorithms such as Guided BackProp and Guided GradCAM,
which modify the gradients during back-propagation, do not pass randomization
tests for different reasons (Sixt et al., 2020), despite being local.

Since visual evaluation of explanations is subjective and misleading, we
compute quantitative evaluation metrics such as infidelity, max-sensitivity
and similarity scores between explanation maps. Infidelity is a generalization
of the completeness axiom supported by algorithms such as IG which states that
the sum of the explanations is equal to the differences of a NN function at
its input and baseline. On the other hand, max-sensitivity measures the
sensitivity of model explanations to subtle input perturbations. Lower scores
for both metrics indicate more trustworthy explanations.

In terms of similarity metrics, we use the popular Structural Similarity Index
(SSIM) (Wang et al., 2004) for image and Cosine Similarity (CosSim) for text.
Adebayo et al. (2018) show that SSIM computed on explanations for original and
randomized models retains significant similarity. Therefore we use this metric
also for local explanations and compare the results with the global variants.
We chose CosSim for text because it is is often used to quantify text
similarities. Further metrics could be used instead, such as correlation
metrics and Euclidean distance (see Appendix B).

##  3 Sanity Checks For Image vs Text

We compare sanity checks for images and text when model parameters are
randomized in cascading manner. We show that local variants of IG for images
are sensitive to model parameter randomization while the global variants are
not, due to the input multiplier. In case of text, both global and local IG
explanations are sensitive to parameter randomization. Global IG for text is
still sensitive to parameter randomization because in contrary to image we sum
the explanations along the embedding dimension in order to obtain token-level
human interpretable importance scores. The effect of the multipliers becomes
insignificant due to small variance and are likely to cancel out in the
summation.

In our CV experiments we use Inception model and randomize model parameters
gradually, layer-wise in a cascading manner, from top to bottom. In Figure 1
we can see that starting from third layer saliency maps look random for the
local but not for the global variant. We also observe that the SSIM score for
the local variant is almost half of what we observe for the global variant.
This indicates how much structural information $x-x_{0}$ carries. The boxplot
distribution of SSIM scores is computed based on 10 random experiments per
layer randomization. It is also interesting to observe that the infidelity
score increases significantly starting from the second layer, indicating that
the explanations are less trustworthy. This is likely because randomizing
model parameters makes NN functions less smooth, depending on the chosen
randomization technique. For these experiments, we use xavier randomization.
The gradients, in turn, become more noisy and integral approximation becomes
less accurate. To alleviate this issue we either increase the number of
integral approximation steps or smooth the NN function. In Figure 2 we observe
that once we replace ReLUs with SoftPlus, the NN function for selected five
input dimensions becomes more smooth and infidelity score drops significantly.
We observe a similar phenomenon when MaxPool is replaced with a smooth variant
such as LSE. Smoothing NN functions also helps to reduce gradient noise and
sharp fluctuations, leading to more reliable explanations.

For our text experiments we fine-tuned a BERT (Devlin et al., 2019)
classification model using Hugging Face (Wolf et al., 2020) with its default
parameter setup on SST2 dataset (Socher et al., 2013) and used $L_{2}$ norm
for normalizing the explanations. Figure 3 visualizes which tokens are
correlated with the positive sentiment in green and with the negative in red.

Figure 1: Model explanations (at the top) and SSIM scores (in the bottom)
using local and global variants of IG for Inception model when its parameters
are randomized in a cascading manner.

Figure 2: NN function plots for five selected input dimensions with different
combinations of smooth and non-smooth activation variants and their relation
to infidelity and max-sensitivity metrics.

Figure 3: IG explanations and related similarity and evaluation metrics for
sentiment analysis model when layer weights are randomized in cascading
manner. See Appendix B for more details.

In contrast to the image use case, we note that the global variant of IG does
not carry a strong signal of the input structural patterns along the cascading
randomization axis. This is related to the fact that a) embedding dimensions
do not carry strong structural patterns as images do and b) since token’s
embedding dimensions are not human-understandable, we summarize importance
values along those dimensions. Positive and negative scores cancel each other
out, which results in explanations that are more parameter-sensitive. It is
also important to observe that both infidelity and max-sensitivity scores
remain low or even decrease which indicates that the explanations are more
trustworthy and the NN function is more smooth. In terms of CosSim score, we
see that the medians of both variants of IG are close to zero which suggests
that the explanations become dissimilar to the original explanations after
randomization. In this experiment the distribution of CosSim score was
computed for 10 different randomization trials per layer and the perturbations
for text are performed in embedding dimension.

In all our experiments the perturbations for infidelity metric and local IG
are sampled uniformly from a normal distribution
$\displaystyle{\mathbb{N}}(0,0.03)$ and for max-sensitivity from a
$L_{\infty}$ ball with a radius of 0.02. We used five perturbed samples and
$L_{2}$ norm for max-sensitivity. The perturbations for global IG are
represented as $x-x_{0}$.

##  4 Conclusion

We show that IG returns different qualitative results depending on the model
type and application domain. According to our experiments, saliency maps are
more sensitive to parameter randomization for text compared with image
classification models. We also show that the input multiplier used in many
saliency maps carries strong structural patterns especially for images. We
recommend that users compare their explanations with and without this
multiplier in order to understand the magnitude of these structural effects.
We also show that the smoothness of a NN function can affect the infidelity
and the quality of explanations and that replacing ReLUs with Softplus and
MaxPools with LSE significantly improves the infidelity of explanations. These
findings can help model developers to better understand how saliency maps work
when implementing Responsible AI frameworks for their models. As a future
direction it would be interesting to explore feature pruning tasks and the
effects of feature and neuron correlations and interactions on saliency maps.

## References

  * Adebayo et al. (2018) Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim.  Sanity checks for saliency maps.  In _Advances in Neural Information Processing Systems_ , volume 31, pp. 9505–9515, 2018.
  * Ancona et al. (2018) Marco B Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross.  Towards better understanding of gradient-based attribution methods for deep neural networks.  In _ICLR_ , 2018.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pp. 4171–4186, 2019.
  * Dombrowski et al. (2019) Ann-Kathrin Dombrowski, Maximillian Alber, Christopher Anders, Marcel Ackermann, Klaus-Robert Müller, and Pan Kessel.  Explanations can be manipulated and geometry is to blame.  In _Advances in Neural Information Processing Systems_ , volume 32, pp. 13589–13600. Curran Associates, Inc., 2019.
  * Shrikumar et al. (2017a) Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje.  Learning important features through propagating activation differences.  In _Proceedings of the 34th International Conference on Machine Learning_ , pp. 3145–3153, 2017a.
  * Shrikumar et al. (2017b) Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje.  Not just a black box: Learning important features through propagating activation differences, 2017b.
  * Sixt et al. (2020) Leon Sixt, Maximilian Granz, and Tim Landgraf.  When explanations lie: Why many modified BP attributions fail.  In _International Conference on Machine Learning_ , pp. 9046–9057. PMLR, 2020.
  * Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts.  Recursive deep models for semantic compositionality over a sentiment treebank.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pp. 1631–1642. Association for Computational Linguistics, 2013.
  * Wang et al. (2004) Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli.  Image quality assessment: from error visibility to structural similarity.  _IEEE Transactions on Image Processing_ , 13(4):600–612, 2004.  doi: 10.1109/TIP.2003.819861.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pp. 38–45, Online, October 2020. Association for Computational Linguistics.  URL https://www.aclweb.org/anthology/2020.emnlp-demos.6.
  * Yeh et al. (2019) Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Sai Suggala, David I. Inouye, and Pradeep D. Ravikumar.  On the (in)fidelity and sensitivity of explanations.  In _NeurIPS_ , 2019.

##  Appendix A Sanity Checks for Image

In this section we describe additional experiments performed for image
classification. In Figure 6 we visualize saliency maps for eight different
local explanation methods. IG, IG w/SmoothGrad (IG_SG), DeepLift, DeepLift
SHAP and Gradient SHAP are computed without $x$ or $x-x_{0}$ multiplier. As we
can see, all methods that do not modify gradients during back-propagation
including DeepLift and DeepLift SHAP are sensitive to parameter randomization.
We also observe that infidelity score increases significantly starting from
the second or third layer of cascading randomization, whereas max-sensitivity
remains almost unchanged, for all sensitivity methods. One possible
explanation for this could be the fact that parameter randomization makes NN
function less smooth which increases the amount of noise in the gradients and
infidelity score. Note that this behaviour depends on the randomization
technique. In this experiment, we used Xavier random initialization of model
weights.

Another way of measuring the quality of our explanations is to look into the
correlation of saliency maps before and after randomization. Besides SSIM we
also measure Spearman correlation rank both for local and global explanations.
Figure 4 visualizes Spearman rank-order correlation score when the model is
randomized in cascading manner. As also discussed in Adebayo et al. (2018),
Spearman correlation gets close to zero after the first randomization attempt
of the last layer and it remains that way until the entire network is fully
randomized. We observe that behaviour both for local and global explanations.

Figure 4: The distribution of Spearman rank correlation scores for local and
global explanations of IG with Inception model.

In order to convince ourselves that the evaluation and similarity metrics
yield similar results for most examples, we run our experiments for a subset
of ten examples. We average the infidelity 1, max-sensitivity 1 and SSIM 5
metrics across ten samples and show that their relative distributions are
similar to those for one sample.

Figure 5: SSIM score averaged across 10 samples for cascading-manner parameter randomization sanity check. Table 1: Infidelity and max-sensitivity scores averaged across ten input samples when the layers are randomized in cascading manner for Inception model. Layer | Infid. Global | Infid. Local | Max-Sens. Global | Max-Sens. Local
---|---|---|---|---
Original | 2.8433 | 0.0001 | 0.2890 | 0.2621
fc | 0.0643 | 1.9918e-06 | 0.2890 | 0.2832
Mixed_7 | 1233.5145 | 0.2014 | 0.3567 | 0.3328
Mixed_6 | 12719875.9301 | 17915.7583 | 0.3835 | 0.3552
Mixed_5 | 826694.3726 | 14482.7274 | 0.3966 | 0.3618
Conv2d_4a_3x3 | 118765.0183 | 2443.3645 | 0.4428 | 0.3971
Conv2d_3b_1x1 | 22535.3965 | 1019.3469 | 0.4531 | 0.4084
Conv2d_2b_3x3 | 6237.2998 | 213.2799 | 0.3865 | 0.3403
Conv2d_2a_3x3 | 139539.6044 | 3071.9756 | 0.3926 | 0.3580
Conv2d_1a_3x3 | 1360343.0962 | 3497.0469 | 0.3176 | 0.2951

Figure 6: Local explanations, infidelity and max-sensitivity scores for eight
different saliency maps when model parameters are randomized in cascading
manner.

Data randomization is another sanity check that we performed on MNIST dataset.
In Figure 7 we can see how saliency maps change for IG and IG_SG (with
SmoothGrad) for the model that was trained on randomized labels. As we can
see, global variants of explanation methods are sensitive to input, whereas
local variants carry almost no structural patterns from the input and look
random.

Figure 7: Local and global IG explanations with and without SmoothGrad (SG),
before and after label randomization on MNIST dataset.

##  Appendix B Sanity Checks for Text

Similar to image modality, we performed data randomization tests for text as
well. In Figure 10 we visualize saliency maps for local and global variants of
IG for all layers of Bert classification model. We can see that starting from
randomization of the first layer the saliency maps look arbitrary and do not
carry structural patters from input both for global and local variants of IG.

In addition to cosine similarities, we also compute euclidean distances
between the explanation maps for the original model and after parameter
randomization. In Figure 8 we can see that explanations become dissimilar
right after randomizing the first layer and that both global and local
variants of IG have dissimilarity patterns.

Furthermore, we computed Spearman rank correlation between saliency maps of
local and global explanations 11. In contrary to image, the variance of
correlation scores is larger here, however the median is still close to zero.
This means that after randomization saliency maps become fairly dissimilar.

In addition to that, we performed data randomization sanity check for text as
well. In Figure 12 we can see highlighted explanations using IG for the
original model and the model that was trained using randomized data labels. As
we can see the explanations for those too models look pretty different and the
quality of explanations doesn’t seem to be changed much based on infidelity
and max-sensitivity metrics.

Figure 8: Euclidean distance between explanations for the original model and
after randomizing the weights in cascading manner.

Similar to image, we would like to examine how the evaluation and similarity
metrics change for a subset of ten samples. Figures 2 and 9 visualize the
distribution of infidelity, max-sensitivity and cosine similarity metrics. As
we can see, all those metrics change with the patterns similar to those that
we obtained for one sample.

Figure 9: Cosine similarity score averaged across 10 samples for cascading-manner parameter randomization sanity check. Table 2: Infidelity and max-sensitivity scores averaged across ten input samples when the layers are randomized in cascading manner for Bert Classification model. Layer | Infid. Global | Infid. Local | Max-Sens. Global | Max-Sens. Local
---|---|---|---|---
Original | 8.63892e-17 | 4.70457e-17 | 0.33020 | 0.03452
classifier | 6.49653e-17 | 7.19617e-17 | 0.31056 | 0.03374
bert.pooler | 1.56992e-17 | 1.94419e-17 | 0.31914 | 0.03371
bert.encoder.layer.11 | 3.76434e-17 | 4.45455e-17 | 0.33087 | 0.03442
bert.encoder.layer.10 | 9.04224e-18 | 1.719544e-17 | 0.31933 | 0.03463
bert.encoder.layer.9 | 4.40619e-17 | 2.42904e-17 | 0.31151 | 0.03369
bert.encoder.layer.8 | 3.63641e-17 | 3.348883e-17 | 0.33557 | 0.03355
bert.encoder.layer.7 | 1.11239e-17 | 4.37865e-17 | 0.30347 | 0.03474
bert.encoder.layer.6 | 3.40005e-17 | 4.78741e-17 | 0.30498 | 0.03478
bert.encoder.layer.5 | 2.25947e-17 | 3.35171e-17 | 0.29744 | 0.03399
bert.encoder.layer.4 | 1.98625e-17 | 3.09084e-17 | 0.29910 | 0.03434
bert.encoder.layer.3 | 3.530162e-17 | 4.18133e-17 | 0.2714 | 0.03469
bert.encoder.layer.2 | 1.74339e-17 | 3.06048e-17 | 0.31196 | 0.03423
bert.encoder.layer.1 | 7.82793e-18 | 2.21003e-17 | 0.28099 | 0.03310

Figure 10: Highlighted local and global IG explanations, corresponding
infidelity and max-sensitivity scores and SSIM between the explanations of the
original model and the model after randomizing the weights in cascading
manner.

Figure 11: The distribution of Spearman rank correlation scores for local and
global explanations of IG with Sentiment Analysis Bert model.

Figure 12: Model explanations for the original model and the model that was
trained on randomly shuffled data labels.


# Under-frequency Load Shedding for Power Reserve Management in Islanded
Microgrids

Bei Xu, Victor Paduani, Qi Xiao, Lidong Song, David Lubkeman, and Ning Lu This
research is supported by the U.S. Department of Energy’s Office of Energy
Efficiency and Renewable Energy (EERE) under the Solar Energy Technologies
Office Award Number DE-EE0008770. The authors are with the Department of
Electrical and Computer Engineering, North Carolina State University, Raleigh,
NC, 27606 USA. E-mails<EMAIL_ADDRESS>and nlu2@ncsu.edu).

###### Abstract

This paper introduces under-frequency load shedding (UFLS) schemes specially
designed to fulfill the power reserve requirements in islanded microgrids
(MGs), where only one grid-forming resource is available for frequency
regulation. When the power consumption of the MG exceeds a pre-defined
threshold, the MG frequency will be lowered to various setpoints, thereby
triggering UFLS for different levels of load reduction. Three types of
controllable devices are considered for executing UFLS: sectionalizers, smart
meters, and controllable appliances. To avoid unnecessary UFLS activation,
various time delay settings are analyzed, allowing short-lived power spikes
caused by events like motor startups or cold-load pickups to be disregarded.
We tested the proposed UFLS schemes on a modified IEEE 123-bus system on the
OPAL-RT eMEGASIM platform. Simulation results verify the efficacy of the
proposed approaches in restoring power reserves, maintaining phase power
balance, and effectively handling short-lived power fluctuations. Furthermore,
in comparison to sectionalizer-based UFLS, using smart meters or controllable
loads for UFLS allows for a more accurate per-phase load shedding in a
progressive manner. As a result, it leads to better balanced three-phase
voltage and serves more loads.

###### Index Terms:

Battery energy storage system (BESS), demand response (DR), frequency control,
grid-forming, microgrid, phase balancing, power reserve, under-frequency load
shedding (UFLS)

##  I Introduction

When operating islanded microgrids (MGs), grid-forming (GFM) sources are
required to establish and maintain voltage and frequency while meeting power
reserve requirement (PRR) at all times to effectively manage sudden load
increases. These GFM functions play a pivotal role in ensuring reliable and
stable MG operation[1, 2].

In a small MG, the absence of load diversity leads to frequent large load
fluctuations, making the fulfillment of PRR vital for ensuring a reliable and
stable MG operation [3]. Nevertheless, depending solely on GFM resources for
maintaining power reserves in an islanded MG poses various challenges,
particularly when there is only one GFM resource available. First, variable
renewable generation resources, such as photovoltaic (PV) and wind, often
exhibit considerable fluctuations in their power outputs. As a result, the GFM
resource must rapidly adapt its output to accommodate these unpredictable
power variations and ensure an adequate power reserve margin. However, this
frequent adjustment can lead to increased wear-and-tear[4] and a degradation
of the GFM’s capabilities to regulate frequency and voltage. Second, MG
reconfiguration can cause significant power surges due to cold-load pickup and
motor startup events, further complicating the GFM capability for meeting PRR
while regulating large voltage and frequency changes. Thus, to ensure a more
stable and resilient MG operation, it is essential to explore alternative
approaches and complementary resources for meeting PRR, rather than relying
solely on a single GFM resource.

By integrating advanced forecasting algorithms and utilizing battery energy
storage systems (BESS), grid-following (GFL) wind and solar farms can
effectively contribute to power reserves[5, 6, 7]. Thus, in MGs with multiple
GFM resources, an option exists to alleviate overloading by switching the
overloaded resources from GFM to GFL mode[8, 9]. Another promising approach is
to aggregate distributed energy resources (DERs) into a virtual power plant
(VPP), allowing the VPP to serve as a source of power reserves[10, 11].
However, note implementing the aforementioned methods requires the presence of
multiple GFM or GFL resources within an islanded MG. Therefore, in a MG with
just a single GFM resource, and where distributed generation resources like
rooftop PV systems lack GFM capabilities, demand response (DR) emerges as the
sole viable solution.

Over the last decade, significant advancements in communication and control
technologies have rendered DR a viable and effective resource for providing
grid regulation and load following services [12, 13, 14]. By promptly
responding to price or load balancing signals, controllable loads can be
efficiently activated and deactivated, allowing for real-time adjustments to
electricity consumption [15, 16]. However, fulfilling PRR demands
exceptionally rapid responses from DR resources. Achieving this level of
responsiveness necessitates the real-time transmission of price or control
signals to hundreds or even thousands of controllable loads, which heavily
relies on highly reliable communication networks. Unfortunately, this can
become impractical during prolonged outages.

TABLE I: Comparison of Existing and Proposed Under-Frequency Load Shedding
Schemes

Method Object Operation Condition Triggered by UFLS Execution Control
Mechanism 3-Phase Imbalance Power Surge Traditional UFLS Recover system
frequency Emergency response Large frequency drops due to outages UFLS
relays[17, 18, 19] Autonomous[17, 18, 19] No No Controllable loads[20, 21, 22,
2] Centralized[20, 21, 2] Decentralized[22] Proposed Method Keep power reserve
margin Normal operation Low power reserve Sectionalizers Autonomous No Yes
Smart meters; Controllable appliances Yes

Therefore, in this paper, we introduce a pioneering method for delivering
power reserve through under-frequency load shedding (UFLS). The key innovation
of this approach lies in using a power reserve threshold as the triggering
mechanism while employing modulated system frequency as control signals to
enable autonomous DR for meeting PRR. The primary advantage of this approach
is its capability of operating an islanded MG during prolonged outages without
relying on reliable communication networks. Using frequency as the control
signal, the method is immune to communication failures, which greatly
increases the MG’s robustness. The second contribution of the paper is the
introduction of an intricate autonomous UFLS mechanism for achieving three
critical objectives: 1) uniformly distributed device tripping delays for
progressively shedding loads when meeting the Power-to-Remain-Ratio
requirement to minimize the amount of unserved loads and ensure efficient
response, 2) a ramp rate checking for avoiding inadvertent load shedding in
response to short-lived power surges to prevent unnecessary load
interruptions, and 3) uniformly distributed device recovery delays to avoid
large power fluctuations caused by simultaneously turning on DR resources. By
implementing this sophisticated delay mechanism, our approach further enhances
the reliability and precision of UFLS, making it a robust solution for power
reserve management in islanded MGs.

A comprehensive comparison between conventional UFLS and the proposed UFLS can
be found in Table I. Traditionally, UFLS is used as an emergency response
mechanism to prevent frequency collapse in large-scale power systems[23]. In
the main grid, system frequency declines when demand surpasses generation
during outages[24]. When the system frequency drops below a predetermined
level, UFLS will be triggered to quickly disconnect a large amount of loads to
regain balance between the demand and supply[18, 20].

In contrast, the proposed UFLS method prioritizes the preservation of power
reserves in an islanded MG during regular operation to meet the PRR.
Consequently, UFLS activation is based on a power threshold determined by the
PRR, rather than relying on frequency drops caused by insufficient generation.
Furthermore, we modulate the system frequency as universal control signals to
activate distributed controllable devices within the MG, such as
sectionalizers, smart meters, or appliances, for autonomous power reserve
management. To guarantee the stable and efficient management of power
reserves, the UFLS mechanism incorporates ramp rate checking and uniformly
distributed tripping and recovery delays, which can prevent unnecessary
triggering during motor start-ups, and synchronized activation of DR devices.
Moreover, employing per-phase UFLS enables the effective correction of three-
phase imbalances, which is a novel application of UFLS further showcasing the
versatility and potential of UFLS in addressing MG power management
challenges.

The subsequent sections of the paper are arranged as follows: Section II
introduces the methodology, Section III presents the simulation results, and
Section IV concludes the paper while suggesting directions for future
research.

##  II Methodology

In this section, we first present a simplification made to the conventional
BESS GFM control structure to facilitate modulating system frequency for
providing a MG-wise UFLS control signal that enables autonomous UFLS response.
Then, we introduce three UFLS schemes: sectionalizer based, smart meters
based, and appliance-based.

###  II-A Modification to the BESS-based GFM Control

The MG test system utilized in this study is depicted in Fig. 1(a). The sole
GFM resource is a BESS, comprising a three-phase inverter, an LC filter, and a
Y-Yg isolation transformer that connects the BESS to the main grid. A
grounding transformer is added to mitigate the zero-sequence components for
keeping the voltage at the point of common coupling (PCC), $v_{\mathrm{pcc}}$,
balanced when serving highly unbalanced single-phase loads [25]. It is
essential to highlight that the proposed UFLS mechanism is employed to fulfill
PRR in an islanded MG with just one GFM resource.

Figure 1: (a) BESS circuit topology and conventional BESS GFM control
structure, and (b) The proposed, simplified BESS control structure.

The conventional BESS-based GFM control often applies the droop-based
hierarchical approach, originally introduced by Guerrero _et al._[26], as
illustrated in Fig. 1(a). This control structure consists of inner current and
voltage control loops, primary (droop) control, and secondary control. When
operating in off-grid mode, the GFM BESS inverter functions as a voltage
source with two main objectives: regulating the PCC voltage
($v_{\mathrm{pcc}}$) and establishing the system frequency ($f$).

The inner current and voltage controller play a vital role in ensuring the
regulation of frequency and inverter output voltage ($v_{\mathrm{o}}$). The
droop control is responsible for adjusting the frequency reference ($f^{*}$)
and the voltage reference amplitude ($V^{*}$) based on the active and reactive
powers ($P$ and $Q$) using the well-known P/Q droop method. Meanwhile, the
secondary control is aimed at restoring the frequency and voltage. This
control algorithm enables power sharing regulation among multiple GFM
resources and facilitates a seamless transition between off-grid mode and
grid-connected mode without the need to modify the control structure.

In a MG with only one GFM resource, where control duty sharing among resources
is unnecessary, the control structure can be significantly simplified. As
illustrated in Fig. 1(b), the secondary voltage controller is still required
to regulate the voltage magnitude at PCC. However, in the absence of multiple
GFM resources, the droop and secondary control become redundant, and the
frequency reference can be directly sent to the inner voltage controller. This
simplification not only reduces the complexity of the control system but also
minimizes the hardware cost required for control. Most importantly, it enables
faster dynamic response with reduced voltage and frequency transients. As
shown in Fig. 2, the frequency reference no longer changes within the inner
control loop, resulting in significantly smaller frequency deviation and much
quicker response times. We consider this simplification of the BESS frequency
control.

###  II-B UFLS Execution Devices

As depicted in Fig. 3, in an islanded MG, three controllable devices can be
used to implement UFLS: sectionalizers (e.g., S1-S6), which can turn on/off an
entire load group (LG); smart meters (e.g., SM1-SM6), capable of turning
on/off an entire building/house; and controllable appliances (e.g., APP1 and
APP2).

A summary of the advantages and disadvantages of conventional UFLS and the
three proposed UFLS schemes has been provided in Table I. In practice,
controlling circuit breakers offers two primary advantages: simple
implementation and reliable execution. Nonetheless, the UFLS approach suffers
from a drawback: its non-selective, on/off-based load shedding often leads to
over-shedding of loads or causes significant power surges during the recovery
phase.

Figure 2: Frequency response comparison (a) Conventional BESS control
structure (b) Simplified, proposed BESS control structure.

Figure 3: Configuration of an islanded MG powered by a single GFM BESS.

On the other hand, the smart meter and appliance-based UFLS schemes offer the
advantages of performing progressive, per-phase load shedding, allowing for
selective, voluntary participation from the customers. Consequently, UFLS can
not only provide power reserves but also reduce three-phase imbalances. The
main drawback of these two methods is that integrating frequency control chips
into a large amount of smart meters or appliances introduces additional costs.
Additionally, well-designed autonomous UFLS coordination schemes are required
for preventing synchronized turning off/on of controllable loads during UFLS
execution or when restoring the shed loads.

###  II-C Coordination between UFLS and the Embedded Controller

Fig. 4 demonstrates how the UFLS controller installed on an appliance actively
monitors the appliance’s on/off status ($s$) and executes switching on/off
commands ($u$) in synchronization with the appliance’s embedded function. This
mechanism plays a crucial role in ensuring the safe operation of the
appliance. For instance, when a water heater thermostat activates the
appliance to heat up the water, a UFLS “off” command can turn off the water
heater and reduce electricity consumption. Conversely, if the water heater is
already in the “off” state, the UFLS “on” command cannot turn it on, because
the temperature in the tank is already within the allowable range. This
setting can prevent overheating of the water when executing UFLS commands. On
the other hand, if the water heater thermostat activates the appliance due to
low water temperature, the water heater will not turn on until the UFLS “off”
period is over, ensuring meeting UFLS requirements.

Figure 4: Coordination between the UFLS control and the embedded, native
control functions on a controllable load.

###  II-D Device-level UFLS Triggering and Recovery Mechanism

As depicted in Fig. 5, a UFLS device is characterized by four vital control
parameters: the triggering frequency threshold ($f_{\mathrm{TH}}$), the
tripping delay ($\tau_{1}$), fixed recovery delay ($\tau_{2}$), and the random
recovery delay ($\tau_{\mathrm{rand}}$). The flow chart of the detailed
device-level UFLS control process is shown in Fig. 6.

Upon activation of the device ($s=\mathrm{True}$), the device-level UFLS
controller will go through the initialization process, assigning initial
values to each control parameter and resetting timers. If the device command
is “on” ($u=\mathrm{True}$), the system frequency $f$ is measured continuously
at each time step ($\Delta t_{\mathrm{step}}$). If the measured frequency
falls within the range of $f_{\mathrm{TH}}-0.05$ to $f_{\mathrm{TH}}+0.05$, a
tripping timer ($t_{1}$) is initiated. Once the predefined turn-off delay
$\tau_{1}$ is reached, the UFLS device will be deactivated and set back to
“off” status. It is worth noting that the frequency deadband of 0.05 Hz is
utilized to accommodate any frequency detection errors and is determined by
the accuracy of the device’s frequency sensor.

Once the UFLS device is deactivated, it remains in the “off” state until the
designated recovery time ($\tau_{2}$) has elapsed. To ensure that all shed
UFLS loads are not simultaneously turned on, the recovery timer ($t_{2}$)
duration is determined by the sum of the fixed restoration time ($\tau_{2}$)
and the individual restoration delay ($\tau_{\mathrm{rand}}$). Note that
$\tau_{2}$ can be a constant for all UFLS devices but $\tau_{\mathrm{rand}}$
is randomly selected by each UFLS device. For instance, if $\tau_{2}$ is set
to 15 minutes, a UFLS device with a randomized delay of
$\tau_{\mathrm{rand}}=1$ will be off for 16 minute, while another UFLS device
with $\tau_{\mathrm{rand}}=2$ will be off for 17 minutes. This approach
prevents synchronized activation and ensures a more distributed recovery of
UFLS devices.

####  II-D1 Per-phase Triggering Frequency

One of the main contributions of this paper is that we introduce per-phase
UFLS triggering frequency for the first time. As shown in Fig. 7(a), for
controllable appliances on phase $a$, $b$, and $c$, we assigned three unique
triggering frequency thresholds. Thus, by lowering the system frequency to
$f_{\mathrm{A}}$, $f_{\mathrm{B}}$, or $f_{\mathrm{C}}$, we can selectively
trigger controllable resources based on phase $a$, $b$, or $c$. Thus, if the
phase $a$ power exceeds the power reserve threshold, we can shed phase $a$
load only. This per-phase UFLS is highly efficient when meeting PRR and can
also be used to solve phase imbalance issues. For a three-phase UFLS load, if
$f$ falls into any of the three frequency detection bands, the UFLS mechanism
will be triggered. This is because 3-phase loads can reduce the power
consumption of all phases without exacerbating phase imbalance issues.

When sectionalizers are used for UFLS, an entire section of the feeder section
is deactivated. Given that these feeder sections typically operate on a
3-phase basis, sectionalizer-based UFLS schemes inherently lack the capacity
for load shedding on a per-phase basis. Consequently, to enable a sequential
shedding process using sectionalizers, individual frequency setpoints are
designated to each sectionalizer in a descending order, as depicted in Fig.
7(b).

Figure 5: An illustration of the UFLS response time delay, fixed recovery
time, and random recovery delay during an UFLS event.

Figure 6: Flow chart of the device-level UFLS scheme.

Conventional UFLS frequency thresholds are set to be 0.25 to 0.3 Hz apart so
two UFLS groups will not be triggered simultaneously [21]. In an islanded MG,
the distance between the frequency setpoints depends on the maximum frequency
deviations caused by typical step load changes. For example, if frequency
deviations can be limited within 0.1 Hz, then we can select the frequency
setpoints for phase $a$, $b$, and $c$ loads to be: $f_{\mathrm{A}}$=59.85 Hz,
$f_{\mathrm{B}}$=59.55 Hz, $f_{\mathrm{C}}$=59.25 Hz, respectively. The
frequency setpoints used in sectionalizer-based method should have the same
frequency difference. In the sectionalizer-based method, the frequency
setpoints can maintain this identical frequency difference, reflecting the
same precision.

Figure 7: An illustration of the UFLS parameter setups (a) Phase-based
triggering frequency setting; (b) load-group-based triggering frequency
setting; (c) Tripping delay selection; (d) UFLS recovery delay selection.

####  II-D2 UFLS Tripping Delay

Once an UFLS device detects that the system frequency falls within
$f_{\mathrm{TH}}\pm 0.05$ Hz, it will select $\tau_{1}$ from a uniform
distribution ($\sim U(0,\tau_{1}^{\mathrm{max}})$) by:

$\vspace{-3pt}\tau_{1}\sim U(0,\tau_{1}^{\mathrm{max}}),\quad\text{for}\quad
i=1,2,\ldots,n\vspace{-3pt}$ (1)

where $\tau_{1}^{\mathrm{max}}$ is the maximum tripping delay, $n$ is the
number of participant device.

Tripping delay is the key parameter for progressively shedding loads to meet
PRR. As shown in Fig. 7(c), the upper bound of the range
$\tau_{1}^{\mathrm{max}}$ is contingent on the lowest frequency setpoint
$f_{\mathrm{min}}=min(f_{\mathrm{A}}~{}f_{\mathrm{B}}~{}f_{\mathrm{C}})$. NERC
reliability standard PRC-024-2 [27] and IEEE standard 1547-2018[28] outline
the frequency ride-through performance characteristic for generator’s
protective relay settings. To avoid tripping DERs (e.g., rooftop PV systems),
$\tau_{1}^{\mathrm{max}}$ should be set to a value less than the low-frequency
duration associated with the lowest frequency setpoint, as specified in the
standards:

$\tau_{1}^{\mathrm{max}}<10^{(1.7373*f_{\mathrm{min}}-100.116)}$ (2)

####  II-D3 UFLS Recovery Delay

After a UFLS device is switched off, it will remain off for a duration of
$\tau_{2}+\tau_{\mathrm{rand}}$. The fixed recovery delay, $\tau_{2}$, can be
set to match the duration of a typical DR event, such as 15 minutes. As
illustrated in Fig. 7(d), to prevent synchronized turning-on of all UFLS
devices, a random time delay $\tau_{\mathrm{rand}}$ is chosen from a uniform
distribution:

$\tau_{\mathrm{rand}}\sim
U(0,\tau_{\mathrm{rand}}^{\mathrm{max}}),\quad\text{for}\quad
i=1,2,\ldots,n\vspace{-3pt}$ (3)

where $\tau_{\mathrm{rand}}^{\mathrm{max}}$ is the maximum individual recovery
delay.

Figure 8: Flow chart of BESS frequency regulation mechanism for meeting PRR
(a) BESS frequency regulation mechanism; (b) Power reserve control triggering
mechanism; (c) UFLS reference frequency setting mechanism.

###  II-E Frequency Regulation Mechanism for Meeting PRR

In this paper, we assume that there is only one GFM BESS in the islanded MG,
and it is solely responsible for system frequency regulation. This allows us
to use the simplified frequency control structure depicted in Fig. 1(b) to
modulate the system frequency as the UFLS control signal. However, it’s
important to acknowledge that if there are multiple GFM resources regulating
frequency, coordination among the GFM devices must be considered. We recognize
this as a potential area for future research.

The BESS frequency control mechanism displayed in Fig. 8 comprises three
essential processes: initialization, power reserve control triggering
mechanism, and UFLS reference frequency setting mechanism. The initialization
mechanism is used for assigning initial values for all controllable parameters
and resetting timers.

####  II-E1 Power Reserve Control Triggering Mechanism

In normal operation, the BESS monitors the per-phase upper power reserve
threshold ($S_{\mathrm{th}}^{\mathrm{up}}$) calculated by:

$\vspace{-3pt}S_{\mathrm{th}}^{\mathrm{up}}=1-S_{\mathrm{PR}}\vspace{-3pt}$
(4)

where $S_{\mathrm{PR}}$ is the PRR. Note that the per unit values are used for
all $S$ values.

Low-power-reserve Detection: If any of the three-phase power reserve limits
exceed the threshold, i.e.,
$S_{\mathrm{A/B/C}}>S_{\mathrm{th}}^{\mathrm{up}}$, the low-power-reserve flag
($lpr$) is set to “True” and the BESS activates the triggering timer
($t_{\mathrm{trigger}}$). The time delay
($\tau^{\mathrm{trigger}}_{\mathrm{th}}$) is used to determine when to
activate the UFLS trigger flag $tr$ so that frequency setpoint $f^{*}$ can be
dropped to corresponding UFLS triggering threshold. Then, the frequency change
can be detected by the frequency chips on sectionalizers, smart meters, or
appliance controllers to activate autonomous device level UFLS following the
device level UFLS mechanism shown in Fig. 6.

Avoid Power Surges: To prevent unnecessary responses to short-lived power
surges (such as motor startups), we compare the Rate of Change of Load Power
(ROCOLP) ($\Delta S_{\mathrm{BESS}}$) with the ROCOLP threshold ($\Delta
S_{\mathrm{th}}$). If the change in the BESS capacity, denoted as $\Delta
S_{\mathrm{BESS}}$, is less than the threshold $\Delta S_{\mathrm{th}}$, the
trigger timer ($t_{\mathrm{trigger}}$) will be set with a threshold of
$\tau_{\mathrm{th}}^{\mathrm{trigger}}$. Typically,
$\tau_{\mathrm{th}}^{\mathrm{trigger}}=\tau_{\mathrm{normal}}^{\mathrm{trigger}}$
can be chosen to be 1 to 3 cycles (approximately 0.02-0.06 seconds) to allow
the measuring of a steady-state power value [29]. If the change in the BESS
capacity, denoted as $\Delta S_{\mathrm{BESS}}$, exceeds or is equal to the
threshold $\Delta S_{\mathrm{th}}$, the parameter $t_{\mathrm{trigger}}$ will
be set with a threshold of
$\tau_{\mathrm{th}}^{\mathrm{trigger}}=\tau_{\mathrm{motor}}^{\mathrm{trigger}}$.
In this scenario, the UFLS trigger flag $tr$ will only activate if the phase
power $S_{\mathrm{A/B/C}}$ remains above $S_{\mathrm{th}}^{\mathrm{up}}$ for a
duration longer than $\tau_{\mathrm{motor}}^{\mathrm{trigger}}$. This designed
triggering delay will ensure that short-lived power spikes do not mistakenly
trigger the UFLS process.

Power Reserve Recovered: The UFLS process continues until all phase powers
fall below the specified low power threshold, denoted as
$S_{\mathrm{A,B,C}}<S_{\mathrm{th}}^{\mathrm{low}}$. This condition indicates
that the power reserve has been restored, and the previously shed loads can
now be safely restored. To prevent power oscillation caused by frequent
activation of UFLS, it is advisable to set the value of
$S_{\mathrm{th}}^{\mathrm{low}}$ approximately 0.05 to 0.1 p.u. lower than
$S_{\mathrm{th}}^{\mathrm{up}}$. The UFLS recovery timer ($t_{\mathrm{rec}}$)
can be set with a threshold of $\tau_{\mathrm{th}}^{\mathrm{rec}}$. Once this
threshold is reached, the UFLS trigger flag will be reset, and all parameters
will undergo re-initialization.

####  II-E2 UFLS Frequency Reference Selection Mechanism

Once the UFLS trigger flag $tr$ is set to be “True”, the GFM controller will
select $f^{*}$. Note that $f^{*}$ is the frequency reference signal sent to
the BESS GFM controller shown in Fig. 1(b) for adjusting the system frequency
to its setpoint ($f_{\mathrm{set}}$).

Frequency Ramp Limits: In normal operation, the Rate of change of frequency
(ROCOF) limit is typically set to a value varying from 0.1 Hz/s to 0.5 Hz/s
[30, 31] for ensuring that the frequency remains stable within a narrow range
around its nominal value. Thus, when the ramp-up and ramp-down slopes of
$f^{*}$ should be set as $f_{\mathrm{ramp}}=(0.5$ Hz/s) $\times\Delta
t_{\mathrm{step}}$ to comply with the normal operation requirements, where
$\Delta t_{\mathrm{step}}$ is the execution time step of GFM BESS controller.

UFLS Phase Selection and Transition: Once the UFLS is triggered
($tr=\mathrm{True}$), the BESS controller will initially identify the phase
$x$ ($x\in{\mathrm{A},\mathrm{B},\mathrm{C}}$) with the highest power that
exceeds the upper power threshold ($S_{\mathrm{th}}^{\mathrm{up}}$),
specifically satisfying $S_{x}>S_{\mathrm{th}}^{\mathrm{up}}$ and
$S_{x}=\max(S_{\mathrm{A}},S_{\mathrm{B}},S_{\mathrm{C}})$. Subsequently, the
controller will compare the phase frequency $f_{x}$ at time $t$ with that of
$t-1$, i.e., assessing whether $f_{x}^{(t)}$ is equal to $f_{x}^{(t-1)}$, to
determine if a phase shift for UFLS is necessary. For instance, assuming phase
$b$ has the highest power exceeding the threshold, causing
$f_{\mathrm{set}}=f_{\mathrm{B}}$ at time $t-1$. If at time $t$, phase $a$
exhibits the highest power, a phase shift occurs, leading to a change in
$f_{\mathrm{set}}$ from $f_{\mathrm{B}}$ to $f_{\mathrm{A}}$.

However, it’s important to note that the BESS will not immediately make
$f_{\mathrm{set}}=f_{\mathrm{A}}$ once the phase shift is detected. Instead, a
frequency setpoint change timer ($t_{\mathrm{f}}$) will be activated. This
ensures that a frequency setpoint $f_{\mathrm{set}}$ will only be altered
after a timer delay ($\tau_{\mathrm{th}}^{\mathrm{f}}$), thus avoiding
responding to short-lived power fluctuations.

Frequency Reference Selection: Once the $f_{\mathrm{set}}$ is determined, the
GFM BESS frequency reference $f^{*}$ will be adjusted gradually to converge
towards the frequency setpoint $f_{\mathrm{set}}$ at a controlled ramp rate of
$f_{\mathrm{ramp}}$. After the UFLS event concludes, $f^{*}$ will be reverted
back to its original value of 60 Hz, as illustrated in Fig. 8.

####  II-E3 An Illustration of the Frequency Regulation Process

In Fig. 9, the upper diagram displays the BESS phase apparent power, while the
lower diagram illustrates the changes in the GFM BESS reference frequency
$f^{*}$. Initially, both phase $a$ and phase $c$ exhibit power outputs that
surpass the power reserve threshold, denoted as
$S_{\mathrm{A}},S_{\mathrm{C}}>S_{\mathrm{th}}^{\mathrm{up}}$. Consequently,
the UFLS is triggered following a time delay of
$\tau_{\mathrm{th}}^{\mathrm{trigger}}$.

Due to the highest power output in phase $a$, the reference frequency is set
as $f^{*}=f_{\mathrm{A}}$. This initiates the load shedding process for
devices connected to phase $a$, leading to a reduction in the BESS power
output in this phase. Subsequently, when the phase with the highest power
output shifts from $a$ to $c$, the reference frequency adjusts accordingly
after a time delay of $\tau_{\mathrm{th}}^{\mathrm{f}}$, ensuring a smooth
transition. The updated frequency setpoint is then set to
$f^{*}=f_{\mathrm{C}}$, triggering the shedding of phase $c$ loads. This
iterative process continues until the power in all phases falls below
$S_{\mathrm{th}}^{\mathrm{low}}$, causing the frequency to ramp up back to 60
Hz.

By this procedural sequence, the proposed UFLS scheme effectively orchestrates
load shedding based on phase power thresholds, ultimately achieving 3-phase
balance regulation.

Figure 9: An illustration of the BESS 3-phase power output variations and
corresponding BESS reference frequency adjustment for triggering a UFLS event
to meet PRR.

##  III Simulation Results

To assess the performance of the proposed UFLS scheme, a MG testbed based on
the IEEE 123-bus topology was developed on the OPAL-RT platform. As depicted
in Fig. 3, a 3-MVA GFM BESS with the topology shown in Fig. 1 is connected to
Node 149 at the feeder head. Note the BESS serves as the sole GFM resource
within the MG. To evaluate the UFLS during a motor start-up event, a 400 kVA
three-phase motor load is connected to Node 47 in LG2. The parameters used in
the simulation are listed in Table II. The GFM BESS model is simulated at the
microsecond level in eMEGASIM. The ZIP load models, motor load model, and the
unbalanced IEEE 123-bus network model are used to emulate the operation of
realistic distribution feeders, which are simulated in ePHASORSIM at the
milisecond level. For more detailed BESS model parameters and simulation
setups, please refer to [25, 32].

A one-hour test involving the MG’s blackstart process has been conducted to
verify the effectiveness of the proposed UFLS approaches for managing MG power
reserves. Load groups LG1, LG2, LG3, LG4, and LG5 will be sequentially powered
up by activating switches S1, S2, S3, S4, and S5 at specific times:
$t=0.3,50,100,150,200$ seconds. Note that S6 remains deactivated to prevent
the formation of loops within the MG. A set of load profiles capable of
triggering UFLS events has been chosen. We focus on comparing two UFLS
mechanisms: the sectionalizer-based and the appliance-based mechanisms. As the
smart-meter based method produces outcomes very similar to those of the
appliance-based method, we omit the comparison due to space constraints.

TABLE II: UFLS Control Parameters Triggering frequency ($[f_{\mathrm{A}}~{}f_{\mathrm{B}}~{}f_{\mathrm{C}}]$) | [59.85 59.55 59.25] Hz
---|---
Max. tripping delay ($\tau_{\mathrm{1}}^{\mathrm{max}}$) | 10 $s$
Fixed recovery delay ($\tau_{\mathrm{2}}$) | 15 minutes
Max. individual recovery delay ($\tau_{\mathrm{rand}}^{\mathrm{max}}$) | 3 minutes
Device controller simulation time step ($\Delta t_{\mathrm{step}}$) | 10 $ms$
BESS controller simulation time step ($\Delta t_{\mathrm{step}}$) | 100 $\mu s$
Upper power ($S_{\mathrm{th}}^{\mathrm{up}}$) | 0.9 p.u.
Low power ($S_{\mathrm{th}}^{\mathrm{low}}$) | 0.87 p.u.
ROCOLP ($\Delta S_{\mathrm{th}}$) | 0.5 p.u.
Triggering timers ($\tau_{\mathrm{normal}}^{\mathrm{trigger}}$ and $\tau_{\mathrm{motor}}^{\mathrm{trigger}}$) | 0.02 $s$, 10 $s$
Recovery timer ($\tau_{\mathrm{th}}^{\mathrm{rec}}$) | 0.02 $s$
Frequency change timer ($\tau_{\mathrm{th}}^{\mathrm{f}}$) | 1 $s$
Frequency ramp rate ($f_{\mathrm{ramp}}$) | 0.5 Hz/s

###  III-A Case 1: Sectionalizer-based UFLS

In case 1, we assess the performance of power reserve management by shedding
LGs via sectionalizers. Note that in this case, UFLS devices can be either
sectionalizers or breakers. Different from appliance or smart meter based
approaches, the switching on/off of sectionalizers will result in an entire
3-phase feeder section to loss power supply.

We configure the trigger frequencies for LG1, LG2, LG3, LG4, and LG5 as 59.85,
59.55, 59.25, 58.95, and 58.65 Hz, respectively. The tripping delay
($\tau_{\mathrm{1}}$) is 0.02 s. The fixed restoration delay is
$\tau_{\mathrm{2}}=$15 minutes and there is no random recovery delay, i.e.,
$\tau_{\mathrm{rand}}=0$.

The dynamic responses of the phase power output of the BESS, system frequency,
sectionalizer status, PCC voltage, and unbalance factors during this one-hour
period are shown in Fig. 10. As depicted in Fig. 10(a), the BESS phase power
exceeds 1.0 p.u. at 450 s and persists until 2200 s when the power reserve
control is not implemented. This scenario poses a significant risk of a
complete shutdown of the MG. On the other hand, if the power reserve control
is implemented, a low-power-reserve flag ($lpr$) will be sent to the BESS
controller, indicating that the power reserve upper limit
($S_{\mathrm{th}}^{\mathrm{up}}=0.9$) has been exceeded and the BESS frequency
regulation scheme will be triggered to shed loads and restore the power
reserve.

From Fig. 10, four UFLS events can be observed during the one-hour MG
blackstart process for meeting PRR. Table III summarizes the cause and actions
taken during the four UFLS events. The $1^{\mathrm{st}}$ UFLS event is
triggered at approximately $t=330$ s, resulting in the shedding of LG5. As the
loads continue to increase, a $2^{\mathrm{nd}}$ UFLS event is soon triggered
at $t=410$ s, causing the shedding of LG4. The loads then remain under 0.8
p.u. after that. After the 15-minute recovery delay elapses, LG4 and LG5 are
switched on again. The sudden increase of power consumption immediately
triggers the $3^{\mathrm{rd}}$ UFLS event at $t=1310$ s. The $4^{\mathrm{th}}$
UFLS occurs at $t=2210$ s, and is also caused by the re-connection of LG4 and
LG5. Finally, at $t=3110$ s, the power reserve can be satisfied after LG4 and
LG5 are reconnected.

TABLE III: UFLS events in case 1 Events | Time |  Cause |  Actions
---|---|---|---
$1^{\mathrm{st}}$ | 330 s |  High power demand |  shed LG5
$2^{\mathrm{nd}}$ | 410 s |  High power demand |  shed LG4
$3^{\mathrm{rd}}$ | 1310 s |  Restore LG4&LG5 |  Shed LG5 then LG4
$4^{\mathrm{th}}$ | 2210 s |  Restore LG4&LG5 |  Shed LG5 then LG4

Figure 10: Simulation results of the sectionalizers-based UFLS case. (a) BESS
3-phase power outputs; (b) System frequency; (c) Sectionalizer on/off status;
(d) PCC voltage; (e) Phase-imbalance metrics: PUF and VUF.

To evaluate phase imbalance, we use two metrics: the power unbalance factor
(PUF) and voltage unbalance factor (VUF) defined in standards [33, 25]. As
shown in Fig. 10 (c), there is a violation in both PUF and VUF during the load
shedding process. This shows that one of the drawbacks of the sectionalizer-
based UFLS method is that it is incapable of regulating phase imbalance and
can inadvertently exacerbate imbalance issues.

As shown in the zoom-in plot of the blackstart process (0 to 250 s) in Fig.
11, a three-phase motor connected to LG2 is energized at $t=50$ s. To avoid
undesirable response to short-lived power surges caused by motor start-up or
cold-load pickup processes, the ROCOLP threshold is set to $\Delta
S_{\mathrm{th}}=$ 0.5 p.u. From the results, we can see that although the BESS
power output in phase $a$ is larger than 0.9 p.u. during 50-56 s, the step
change of BESS power ($\Delta S_{\mathrm{BESS}}$) is larger than the threshold
$\Delta S_{\mathrm{th}}=$ 0.5 p.u., hence the UFLS is not triggered. In this
case, the UFLS is successfully avoided because the duration of the high power
period is less than the motor triggered timer threshold
$\tau_{\mathrm{motor}}^{\mathrm{trigger}}=10$ s.

Figure 11: PCC voltage, system frequency, and BESS 3-phase power outputs
during the blackstart period.

Figure 12: BESS 3-phase power outputs, system frequency, sectionalizers S4 and
S5 on/of status during the third UFLS event.

Figure 12 displays a complete UFLS event occuring during seconds 1309 to 1314.
Initially, LG4 and LG5 are shed at the $1^{\mathrm{st}}$ and $2^{\mathrm{nd}}$
UFLS event, as shown in Fig. 10. After the recovery time of 15 minutes, shed
loads in LG4 and LG5 are reconnected at 1309.8 s. Subsequently, since the BESS
power output exceeds the upper power threshold
($S_{\mathrm{th}}^{\mathrm{up}}=0.9$ p.u), another UFLS event is triggered at
$t=1310$ s. The system frequency starts to decline with a rate of 0.5 Hz/s
until it reaches 59.85 Hz, after which, switch S5 is turned off. However,
since phase $a$ power remains over 0.87 p.u., which is higher than the
restoration threshold ($S_{\mathrm{th}}^{\mathrm{low}}=0.87$ p.u.), the system
frequency drops again to 59.55 Hz to shed LG4. Once LG4 is shed, the BESS
power drops below 0.87 p.u.. Thus, the system frequency is restored to 60 Hz,
effectively ending the UFLS event. All shed loads will be restored after a
recovery time of $\tau_{\mathrm{rec}}=15$ minutes.

The results show that the major drawback of the sectionalizer-based approach
is that the disconnection of an entire LG may over-shed loads when meeting
PRR. Moreover, the restoration of the shed LGs after 15-minute restoration
timer may immediately trigger another low-power-reserve event. For instance,
in Fig. 10, another UFLS event occurs around $t=2200$ s due to the restoration
of LG4 and LG5.

###  III-B Case 2: Appliances-based UFLS

Case 2 models the appliance-based UFLS scheme. The simulation results in
comparison to the sectionalizer case are shown in Fig. 13. From the results,
we can highlight the following observations:

  * •

The BESS power output gradually decreases and recovers when fulfilling PRR, as
opposed to the abrupt drop or immediate rebound observed in the sectionalizer-
based scenario.

  * •

Using appliance-based UFLS will lead to a higher number of UFLS events when
compared to the sectionalizer case. This is because shedding controllable
appliances across all LGs allows non-controllable loads to continue
increasing. Note that preventing the disconnection of entire LGs can ensure
equitable load shedding among LGs.

  * •

By examining the PUF and VUF plots, we can clearly see that per-phase UFLS has
effectively alleviated the phase imbalance issue, in contrast to the
sectionalizer case where the issue was exacerbated. This enhancement
contributes to improved BESS control stability and MG operational stability,
as excessive phase imbalance has the potential to trigger a complete shutdown
of the entire MG.

Figure 13: Simulation results with appliance-level UFLS (a) BESS phase power;
(b) System frequency; (c) PCC voltage; (d) Unbalance factor PUF and VUF.

Figure 14: Simulation results of an UFLS event with appliance-level UFLS.

Fig. 14 presents a zoom-in plot of an UFLS event from 360 s to 370 s. The
frequency setpoint is adapted according to the phase exhibiting the highest
power output, ensuring that the load with the greatest power is shed, thereby
contributing to effective three-phase imbalance mitigation. At 360.2 s, UFLS
is activated when the phase $c$ power exceeds the high power threshold,
$S_{\mathrm{C}}>S_{\mathrm{th}}^{\mathrm{up}}=$ 0.9 p.u.. Since phase $c$
bears the highest load, the system frequency drops to the phase $c$ frequency
setpoint stage, $f_{\mathrm{C}}=$ 59.25 Hz, with a 0.5 Hz/s ramp rate,
shedding loads and hence decreasing the BESS phase $c$ output power. When the
phase with the highest power output switches from phase $c$ to phase $a$, the
frequency will remains unchanged for a duration of $\tau_{\mathrm{f}}=1s$ to
validate the transition of the phase with maximum power output. Afterwards,
the frequency is adjusted to phase $a$ setpoint, $f_{\mathrm{A}}=$ 59.85 Hz,
to shed phase $a$ loads. This process persists until the power outputs across
all three phases are lower than the low power threshold,
$S_{\mathrm{th}}^{\mathrm{low}}=$ 0.87 p.u.. Upon reaching this state, the
frequency ramps up to its nominal value of 60 Hz.

TABLE IV: UFLS performance comparison: three-phase v.s. per-phase |  3-phase |  Per phase
---|---|---
PUF (compares to non-UFLS) |  increase 0.1 |  reduce 0.03
VUF (compares to non-UFLS) |  increase 1.1% |  reduce 0.2%
Max. Deviation of $f$ (Hz) |  0.45 |  0.75
Max. Deviation of $V_{\mathrm{pcc}}$ (p.u.) |  0.03 |  0.002
Load Served (MWh) |  1.8 |  2.54
Number of UFLS device |  5 |  95

Figure 15: UFLS performance comparison: load served in each load group.

To summarize, the simulation results show that both developed methods can
effectively maintain power reserves for BESS and improve power system
stability. However, The appliance-level UFLS method can reduce the shedding of
excessive loads when compared to the sectionalizers-level UFLS method.
Furthermore, the appliance-level UFLS scheme can better regulate the three-
phase imbalance and avoid the potential imbalance issues caused by UFLS.

Table IV presents a summary of the observed metrics pertaining to the two
proposed UFLS methodologies. The appliance-based method demonstrates superior
performance by reducing both PUF and VUF, exhibiting lower frequency and
voltage deviations, and enabling the shedding of fewer loads to meet the PRR.
As indicated in Fig. 15, a greater number of loads within LG4 and LG5 are
being supplied. This illustrates that the per-phase based UFLS scheme
effectively distributes the shedding of controllable loads across all LGs,
preventing the repetitive shedding of LGs situated at the far end of the
feeder (i.e., LG5 and LG4). This methodology ensures an equitable allocation
of resources, promoting fairness in the provision of energy. However, this
strategy does necessitate a larger number of controlled execution devices,
thereby adding to the overall complexity of control.

Sectionalizers-level UFLS scheme can quickly shed whole loads in an area.
However, it is non-selective, and may shed more loads than necessary. For the
appliance-level UFLS scheme, the load shedding is selective and gradual. So it
will cause smaller disturbances to the system. The disadvantage is that it
requires controller chips to be installed in controllable loads (e.g. HVAC
units). With the help of internet of things (IoT), the future grid can be
composed of “smart devices” which include embedded UFLS functions.

##  IV Conclusion

This paper introduces a new UFLS scheme tailored for power reserve management
in an islanded MG with a single GFM resource. The novel UFLS mechanism employs
power reserve thresholds as event triggers and system frequency as control
signals, achieving autonomous UFLS during regular operations. This reduces
reliance on communication networks, crucial during extended main grid outages.
By comparing per-phase UFLS to sectionalizer-based UFLS, we demonstrate the
benefits in preventing excessive load shedding, addressing power imbalances,
and ensuring equity when meeting PRR. The addition of triggering and recovery
delays mitigates oscillations caused by transient events and synchronized load
switching actions. Our future work will be focused on extending the proposed
UFLS approach to broader grid services and MGs with multiple GFM resources.

## References

  * [1] S. Som, S. Chakrabarti, S. R. Sahoo, A. Ghosh, and X. Liang, “Bess reserve-based frequency support during emergency in islanded residential microgrids,” _IEEE Transactions on Sustainable Energy_ , 2023.
  * [2] M. Karimi, P. Wall, H. Mokhlis, and V. Terzija, “A new centralized adaptive underfrequency load shedding controller for microgrids based on a distribution state estimator,” _IEEE Transactions on Power Delivery_ , vol. 32, no. 1, pp. 370–380, 2016.
  * [3] Z. Li and Y. Xu, “Temporally-coordinated optimal operation of a multi-energy microgrid under diverse uncertainties,” _Applied energy_ , vol. 240, pp. 719–729, 2019.
  * [4] X. Ke, N. Lu, and C. Jin, “Control and size energy storage systems for managing energy imbalance of variable generation resources,” _IEEE Transactions on Sustainable Energy_ , vol. 6, no. 1, pp. 70–78, 2014.
  * [5] J. Chang, Y. Du, E. G. Lim, H. Wen, X. Li, and L. Jiang, “Coordinated frequency regulation using solar forecasting based virtual inertia control for islanded microgrids,” _IEEE Transactions on Sustainable Energy_ , vol. 12, no. 4, pp. 2393–2403, 2021.
  * [6] X. Li, D. Hui, and X. Lai, “Battery energy storage station (bess)-based smoothing control of photovoltaic (pv) and wind power generation fluctuations,” _IEEE transactions on sustainable energy_ , vol. 4, no. 2, pp. 464–473, 2013.
  * [7] U. R. Nair, M. Sandelic, A. Sangwongwanich, T. Dragičević, R. Costa-Castello, and F. Blaabjerg, “An analysis of multi objective energy scheduling in pv-bess system under prediction uncertainty,” _IEEE Transactions on Energy Conversion_ , vol. 36, no. 3, pp. 2276–2286, 2021.
  * [8] M. Rezkallah, S. Singh, B. Singh, A. Chandra, H. Ibrahim, and M. Ghandour, “Implementation of two-level coordinated control for seamless transfer in standalone microgrid,” _IEEE Transactions on Industry Applications_ , vol. 57, no. 1, pp. 1057–1068, 2020.
  * [9] S. Bayhan and H. Abu-Rub, “A simple control technique for distributed generations in grid-connected and islanded modes,” in _2018 IEEE 27th International Symposium on Industrial Electronics (ISIE)_. IEEE, 2018, pp. 1237–1242.
  * [10] J. Naughton, H. Wang, M. Cantoni, and P. Mancarella, “Co-optimizing virtual power plant services under uncertainty: A robust scheduling and receding horizon dispatch approach,” _IEEE Transactions on Power Systems_ , vol. 36, no. 5, pp. 3960–3972, 2021.
  * [11] J. Lee and D. Won, “Optimal operation strategy of virtual power plant considering real-time dispatch uncertainty of distributed energy resource aggregation,” _IEEE Access_ , vol. 9, pp. 56 965–56 983, 2021.
  * [12] N. Lu and Y. Zhang, “Design considerations of a centralized load controller using thermostatically controlled appliances for continuous regulation reserves,” _IEEE Transactions on Smart Grid_ , vol. 4, no. 2, pp. 914–921, 2012.
  * [13] X. Wu, J. He, Y. Xu, J. Lu, N. Lu, and X. Wang, “Hierarchical control of residential hvac units for primary frequency regulation,” _IEEE Transactions on Smart Grid_ , vol. 9, no. 4, pp. 3844–3856, 2017.
  * [14] T. Sun, J. Lu, Z. Li, D. L. Lubkeman, and N. Lu, “Modeling combined heat and power systems for microgrid applications,” _IEEE Transactions on Smart Grid_ , vol. 9, no. 5, pp. 4172–4180, 2017.
  * [15] B. Biegel, P. Andersen, J. Stoustrup, L. H. Hansen, and A. Birke, “Sustainable reserve power from demand response and fluctuating production—two danish demonstrations,” _Proceedings of the IEEE_ , vol. 104, no. 4, pp. 780–788, 2016.
  * [16] L. Junhui, B. Hongkun, L. Hujun, Y. Qinchen, D. Fangzhao, and Z. Wenjie, “Optimal decomposition of flexible load resource reserve target based on power demand response,” in _2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA)_. IEEE, 2021, pp. 1046–1049.
  * [17] B. Potel, V. Debusschere, F. Cadoux, and L. de Alvaro Garcia, “Under-frequency load shedding schemes characteristics and performance criteria,” in _2017 IEEE Manchester PowerTech_. IEEE, 2017, pp. 1–6.
  * [18] S. Gordon, C. McGarry, J. Tait, and K. Bell, “Impact of low inertia and high distributed generation on the effectiveness of under frequency load shedding schemes,” _IEEE Transactions on Power Delivery_ , vol. 37, no. 5, pp. 3752–3761, 2021.
  * [19] S. S. Banijamali and T. Amraee, “Semi-adaptive setting of under frequency load shedding relays considering credible generation outage scenarios,” _IEEE Transactions on Power Delivery_ , vol. 34, no. 3, pp. 1098–1108, 2018\.
  * [20] Q. Zhou, Z. Li, Q. Wu, and M. Shahidehpour, “Two-stage load shedding for secondary control in hierarchical operation of islanded microgrids,” _IEEE Transactions on Smart Grid_ , vol. 10, no. 3, pp. 3103–3111, 2018.
  * [21] C. Li, Y. Wu, Y. Sun, H. Zhang, Y. Liu, Y. Liu, and V. Terzija, “Continuous under-frequency load shedding scheme for power system adaptive frequency control,” _IEEE Transactions on Power Systems_ , vol. 35, no. 2, pp. 950–961, 2019.
  * [22] W. Gu, W. Liu, J. Zhu, B. Zhao, Z. Wu, Z. Luo, and J. Yu, “Adaptive decentralized under-frequency load shedding for islanded smart distribution networks,” _IEEE Transactions on Sustainable Energy_ , vol. 5, no. 3, pp. 886–895, 2014.
  * [23] Y. Wen, C. Chung, and X. Ye, “Enhancing frequency stability of asynchronous grids interconnected with hvdc links,” _IEEE Transactions on Power Systems_ , vol. 33, no. 2, pp. 1800–1810, 2017.
  * [24] P. Ferraro, E. Crisostomi, R. Shorten, and F. Milano, “Stochastic frequency control of grid-connected microgrids,” _IEEE Transactions on Power Systems_ , vol. 33, no. 5, pp. 5704–5713, 2018.
  * [25] B. Xu, V. Paduani, H. Yu, D. Lubkeman, and N. Lu, “A novel grid-forming voltage control strategy for supplying unbalanced microgrid loads using inverter-based resources,” in _2022 IEEE Power & Energy Society General Meeting (PESGM)_. IEEE, 2022, pp. 1–5.
  * [26] J. M. Guerrero, J. C. Vasquez, J. Matas, L. G. De Vicuña, and M. Castilla, “Hierarchical control of droop-controlled ac and dc microgrids—a general approach toward standardization,” _IEEE Transactions on industrial electronics_ , vol. 58, no. 1, pp. 158–172, 2010.
  * [27] V. P. R. Settings, “NERC Std. PRC-024-2.” _The North American Electric Reliability Corporation (NERC): Atlanta, GA, USA_ , 2018.
  * [28] I. S. Association, “IEEE Standard for interconnection and interoperability of distributed energy resources with associated electric power systems interfaces,” _IEEE Std 1547-2018_ , pp. 1–138, 2018.
  * [29] R. Setiabudy, H. B. Hartono, and B. Budiyanto, “Analysis characteristics of on/off grid tie inverter and implementation in microgrid,” _TELKOMNIKA (Telecommunication Computing Electronics and Control)_ , vol. 11, no. 3, pp. 441–450, 2013.
  * [30] C. Broderick, “Rate of change of frequency (rocof) withstand capability,” _ENTSOE, Brussels, Belgium, Tech. Rep_ , 2018.
  * [31] G. Chown, J. G. Wright, R. P. Van Heerden, and M. Coker, “System inertia and rate of change of frequency (rocof) with increasing non-synchronous renewable energy penetration,” 2017.
  * [32] V. Paduani, B. Xu, D. Lubkeman, and N. Lu, “Novel real-time EMT-TS modeling architecture for feeder blackstart simulations,” in _2022 IEEE Power & Energy Society General Meeting (PESGM)_. IEEE, 2022, pp. 1–5.
  * [33] F. Shahnia, P. J. Wolfs, and A. Ghosh, “Voltage unbalance reduction in low voltage feeders by dynamic switching of residential customers among three phases,” _IEEE Transactions on Smart Grid_ , vol. 5, no. 3, pp. 1318–1327, 2014.