11institutetext: Department of Electrical Engineering, Video Coding &
Architectures, Eindhoven University of Technology, Eindhoven, The Netherlands
%\institute{Anonymous␣Institute␣\\\␣\email{**<EMAIL_ADDRESS>22institutetext: Department of Biomedical Engineering, Medical Image Analysis,
Eindhoven University of Technology, Eindhoven, The Netherlands
33institutetext: Department of Surgery, University Medical Center Utrecht,
Utrecht, The Netherlands 44institutetext: Department of Oncological Urology,
University Medical Center Utrecht, Utrecht, The Netherlands 55institutetext:
Department of Urology, Catharina Hospital, Eindhoven, The Netherlands

# Exploring the Effect of Dataset Diversity in Self-Supervised Learning for
Surgical Computer Vision

Tim J.M. Jaspers Corresponding Author11 0009-0001-8306-5058 Ronald L.P.D. de
Jong  0009-0005-7806-4340 22 Yasmina Al Khalil 22 Tijn Zeelenberg 11 Carolus
H.J. Kusters 11 0009-0004-3114-3888 Yiping Li 22 Romy C. van Jaarsveld 33
Franciscus H.A. Bakker 4455 Jelle P. Ruurda  Willem M. Brinkman 3344 Peter H.
N. De With 11 Fons van der Sommen 11 0000-0002-3593-2356

###### Abstract

Over the past decade, computer vision applications in minimally invasive
surgery have rapidly increased. Despite this growth, the impact of surgical
computer vision remains limited compared to other medical fields like
pathology and radiology, primarily due to the scarcity of representative
annotated data. Whereas transfer learning from large annotated datasets such
as ImageNet has been conventionally the norm to achieve high-performing
models, recent advancements in self-supervised learning (SSL) have
demonstrated superior performance. In medical image analysis, in-domain SSL
pretraining has already been shown to outperform ImageNet-based
initialization. Although unlabeled data in the field of surgical computer
vision is abundant, the diversity within this data is limited. This study
investigates the role of dataset diversity in SSL for surgical computer
vision, comparing procedure-specific datasets against a more heterogeneous
general surgical dataset across three different downstream surgical
applications. The obtained results show that using solely procedure-specific
data can lead to substantial improvements of 13.8%, 9.5%, and 36.8% compared
to ImageNet pretraining. However, extending this data with more heterogeneous
surgical data further increases performance by an additional 5.0%, 5.2%, and
2.5%, suggesting that increasing diversity within SSL data is beneficial for
model performance. The code and pretrained model weights are made publicly
available at https://github.com/TimJaspers0801/SurgeNet.

###### Keywords:

Self-supervised learning Surgical computer vision Transfer learning Data
diversity

##  1 Introduction

Over the last decade, computer vision applications in minimally invasive
surgery have significantly expanded, encompassing tasks such as anatomy
recognition [20, 4, 2], surgical phase recognition [21, 30], and surgical
training [10]. Despite this growth, computer vision has not yet achieved an
impact in surgery, as compared to other fields like pathology or radiology,
where advanced applications are already approaching market introduction [18].
It has been concluded that the surgical field is still in its infancy [5, 18],
primarily due to the lack of representative annotated data [18].

In the broader context of computer vision, transfer learning has emerged as a
powerful strategy for leveraging related datasets. Transferring pretrained
weights can significantly reduce the amount of labeled data required to
achieve high-performing models. Conventionally, large annotated datasets such
as ImageNet [8] have been the cornerstone of transfer learning, providing
state-of-the-art results for many years. However, recent advancements have
demonstrated that self-supervised learning (SSL) methods, which learn generic
representations from unannotated data, can outperform supervised pretraining
by providing better model initialization [6]. This stems from SSL’s ability to
capture a wider variety of features from diverse data, reducing the risk of
overfitting to specific labeled datasets and enhancing the model’s
generalization capabilities.

In medical image analysis, in-domain self-supervised pretraining has offered
superior performance compared to ImageNet-based initialization [27, 12]. A
large-scale benchmark study has explored the feasibility of SSL methods within
the surgical domain, offering valuable insights into methods, training
settings, and frame sampling rates [22]. Other recent research has focused on
scaling up SSL pretraining for surgical computer vision to achieve state-of-
the-art performance [27, 12]. The study by Alapatt et al. [1] is particularly
relevant to our work, since it highlights critical considerations regarding
the composition of pretraining data.

The proposed research aims to delve deeper into the effects of dataset
diversity in SSL for surgical computer vision. Although unlabeled data is
abundantly available within this field, as most videos are recorded at over 25
frames per second (fps) and often last several hours, the diversity within a
single video is limited. Large-scale pretraining requires substantial
computational resources and is therefore often expensive. This study seeks to
improve understanding of the optimal dataset composition for pretraining,
aiming to produce more effective and generic model weights for surgical
computer vision applications.

##  2 Methodology

###  2.1 Self-Supervised learning datasets

To analyze the effects of SSL for surgical computer vision applications, we
construct SurgeNet, a comprehensive dataset comprising over 2,636,790 frames
from more than seven different surgical procedures. SurgeNet includes both
public and private datasets, offering a diverse collection of surgical video
data. An overview of SurgeNet summarizing its contents can be found in Table
1. Most datasets are publicly accessible, however, to enhance diversity/size
and to create meaningful procedure-specific datasets SurgeNet is further
enlarged in specific ways. We extend the public datasets with two private
datasets: one from the Antoni van Leeuwenhoek Hospital (AvL) Amsterdam, while
the second dataset originates from the University Medical Center Utrecht
(UMCU), the Netherlands. All frames are extracted at a rate of 1 fps from the
videos, with black edges removed.

From SurgeNet, we derive three procedure-specific subsets. The SurgeNet-CHOLEC
dataset contains all frames extracted from laparoscopic cholecystectomy
procedures, resulting in a total of 250,655 unique frames. The robot-assisted
minimally invasive esophagectomy procedure-specific (SurgeNet-RAMIE) dataset
comprises 377,287 frames. Similarly, the SurgeNet-RARP dataset includes
382,416 frames from robot-assisted prostatectomy procedures. Each of these
subsets is used to analyze the impact of procedure-specific data on SSL
pretraining. Lastly, we also randomly create SurgeNetSmall including 10% of
the SurgeNet dataset, making it comparable in terms of size to the procedure-
specific datasets.

Procedure- | Dataset | Procedure | Incl. frames | Public
---|---|---|---|---
specific subset
CHOLEC |  Cholec80 [24] | Laparoscopic Cholecystectomy | 179,164 | Yes
HeiChole [19] | Laparoscopic Cholecystectomy | 53,427 | Yes
hSDB-Chole [28] | Laparoscopic Cholecystectomy | 18,064 | Yes
RAMIE | RAMIE-UMCU | RA Esophagectomy | 377,287 | No
RARP |  ESAD [3] | RA Prostatectomy | 47,282 | Yes
PSI-AVA [25] | RA Prostatectomy | 73,618 | Yes
RARP-AvL | RA Prostatectomy | 261,516 | No
Others |  DSAD [7] | RA Rectal Resection/Extirpation | 14,623 | Yes
GLENDA [16] | Gynecologic Laparoscopy | 25,682 | Yes
LapGyn4 (v1.2) [17] | Gynecologic Laparoscopy | 59,616 | Yes
MultiBypass140 [15] | Laparoscopic Gastric bypass surgery | 749,419 | Yes
hSDB-Gastric [28] | RA Gastrectomy | 35,576 | Yes
SurgLoc2022 [31] | 11 different RA porcine procedures | 741,516 | Yes
SurgeNet | All of the above | All of the above | 2,636,790 | Part
Table 1: Composition of SurgeNet. SurgeNet comprises over 2.6 million frames
derived from more than seven distinct surgical procedures. A part of SurgeNet
is publicly available (over 1.9M frames).

###  2.2 Downstream datasets

The experiments in this study are conducted on three different downstream
datasets, each corresponding to a specific surgical procedure: laparoscopic
cholecystectomy (LC), robot-assisted radical prostatectomy (RARP), and robot-
assisted minimally invasive esophagectomy (RAMIE). All three datasets focus on
anatomy segmentation, of which the specifications are summarized in Table 2.
It should be noted that all videos of patients included in the test set are
excluded from SurgeNet. For the CholecSeg8k dataset, all patients present in
the downstream training set are also included in the SurgeNet-pertaining
dataset. In the case of the RAMIE downstream dataset, 22 of the 50 patient
videos are also included in the SurgeNet-pretraining dataset. There is no
overlap between the patients in the RARP downstream dataset and RAPR videos in
the the SurgeNet dataset.

The first dataset is the public CholecSeg8k dataset [13], derived from the
Cholec80 dataset [24], which consists of 80 videos of LC surgeries performed
by 13 surgeons. The CholecSeg8k dataset includes a selected subset of these
videos, annotated with semantic segmentation masks. Specifically, 80 frames
from 101 video fragments were annotated, totaling 8,080 frames. Among these,
6,800 (84%) were used for training and 1,280 (16%) were selected for testing.
Initially, the dataset contained 13 classes: background, abdominal wall,
blood, connective tissue, cystic duct, fat, gallbladder, gastrointestinal
tract, grasper, hepatic vein, L-hook electrocautery, liver, and liver
ligament. For this research, classes with low prevalence were excluded to
ensure a more robust and reliable analysis. The excluded classes are blood,
cystic duct, hepatic vein, and liver ligament.

The second dataset consists of 869 frames from 31 distinct patients undergoing
RAMIE, with 749 frames (86%) allocated for training and 120 frames (14%)
reserved for testing. These frames were labeled by two students with a medical
background, and two research fellows in surgery or medical imaging under the
supervision of an expert surgeon. In total, 12 distinct classes were
annotated, including four classes for surgical tools: forceps, hook, suction &
irrigation, and vessel sealer. The other eight classes represent vital
anatomical structures during RAMIE: airways (including the trachea, left main
bronchus, and right main bronchus), aorta, azygos vein & vena cava, esophagus,
nerves, pericardium, right lung, and thoracic duct. The airways were grouped
into a single class, due to their similar appearance and the difficulty in
defining exact boundaries between the trachea and bronchi. Similarly, the
azygos vein and vena cava were treated as one class for the same reason.

The third dataset focuses on anatomy recognition during RARP procedures. The
total dataset consists of 282 frames from 114 patients. The training set
comprised 84 procedures from distinct patients, totaling 252 (89%) annotated
frames for model training. All videos were sourced from a single-center cohort
at the AvL in Amsterdam. Annotation covered four structures: the prostate,
urethra, catheter, and ligated dorsal venous plexus. For the test set, one
frame from 30 different patient videos was annotated. Training annotations
were performed by a research fellow in the field of surgery under the
supervision of an expert surgeon, while the expert surgeon solely completed
test annotations. Further details on this dataset can be found in a previous
publication [2].

Downstream Dataset | Training | Training | Test | Test | Structures included
---|---|---|---|---|---
patients | frames | patients | frames
CholecSeg8k [13] | 13 | 6,800 | 4 | 1,280 | 8
RAMIE | 27 | 749 | 4 | 120 | 12
RARP | 84 | 252 | 30 | 30 | 4
Table 2: Descriptions of the downstream datasets. The datasets exhibit
diversity in terms of patient inclusion, annotated frames, and the number of
structures, offering a representative sample of surgical downstream datasets
that could benefit from self-supervised learning on SurgeNet.

###  2.3 Model architecture and Training configurations

Architecture: This study emphasizes the impact of data on SSL, therefore we
select a state-of-the-art architecture in the natural computer vision domain.
Conventional architectures like ResNet [11] and EfficientNet [23] are CNN-
based, but recent advancements have favored the Vision Transformer [9].
However, due to the low-resolution outputs, the vanilla Vision Transformer has
difficulties regarding various dense prediction tasks [26]. Therefore, we
adopt the CAFormer-S18 [29] as the encoder for this study. The CAFormer-S18
builds upon the abstract MetaFormer approach, where the token mixer is not
predefined, while other components remain consistent with Transformers.
CAFormer-S18 incorporates convolution blocks in the first two stages and
attention blocks in the last two stages, achieving state-of-the-art
performance on natural image datasets like ImageNet, with 26M parameters.
Concerning the downstream application, the CAFormer-S18 encoder architecture
is integrated into a feature pyramid network (FPN)[14].

Pretraining configurations: As SSL objective, we use the well-known framework
proposed by Caron et al. [6], called “Self-Distillation with NO Labels”
(DINO). This method employs distillation-based techniques to enable efficient
learning with smaller batch sizes, thereby reducing the demand for extensive
computational resources. We closely follow the original implementation.
Notably, we initiate pretraining from ImageNet-initialized weights as
recommended by Ramesh et al. [22], and then align with the concept of fine-
tuning natural computer vision foundation models for medical applications. The
pretraining of the CaFormer-S18 architecture is performed on four A100 GPUs
(NVIDIA Corp., CA, USA), each with 40GB of VRAM, using a maximum feasible
batch size of 544 for 50 epochs. Note that due to computational constraints,
the CAFormer-S18 network on the complete SurgeNet is only trained for 25
epochs.

Downstream training settings:  The downstream training process involves a
five-fold cross-validation at the patient level using the training data. All
training parameters are kept constant for each experiment. Frames are resized
to 256$\times$256 pixels using bicubic interpolation. The cross-entropy loss
is used, while the Adam optimizer is employed with a learning rate of
1$\times$10-5. The learning rate is halved after 10 epochs without a decrease
in the validation loss. All models are trained on a GeForce RTX 2080 Ti GPU
(NVIDIA Corp., CA, USA) with a batch size of 16 and an early stopping
criterion of 15 epochs. Limited data augmentation techniques are used,
including horizontal and vertical flipping, and rotation, each with a
probability of 50%.

Figure 1: Segmentation results (Dice scores) on the three downstream datasets
from the fivefold cross-validation. The performance is given for both fine-
tuning the entire segmentation model and the model with a frozen encoder.

##  3 Results and Discussion

Figure 1 presents the results of SSL pretraining on the SurgeNet and its
procedure-specific datasets across the three downstream tasks. The encoder is
evaluated with both frozen and trainable weights during downstream training.
For LC, RAMIE, and RARP, procedure-specific pretraining results in
segmentation improvements of 13.8%, 9.5%, and 36.8%, respectively, compared to
ImageNet initialization with trainable encoder weights. Against ImageNet with
frozen weights, improvements are 19.7%, 25.6%, and 49.4%. These findings are
in line with the results of Alapatt et al. [1], who also observe substantial
improvements using procedure-specific pretraining datasets. Additionally, our
results indicate that training on procedure-specific datasets provides
superior performance compared to SurgeNet-Small, despite their comparable
sizes.

Moreover, this study indicates that incorporating extra, more heterogeneous
data during pretraining further enhances segmentation performance compared to
procedure-specific training only. More specifically, training on SurgeNet
results in a further improvement of 5.0%, 5.2%, and 2.5% for LC, RAMIE, and
RARP, respectively, when encoder weights are trainable, albeit at the expense
of a longer training time. Additionally, the results show that SurgeNet
pretraining benefits from fine-tuning the encoder in contrast to procedure-
specific pretraining, where on both the CholecSeg8k and RARP downstream
datasets the highest performance is achieved using a frozen encoder.

Furthermore, the disparity between ImageNet and SurgeNet initialization is
most pronounced on the RARP dataset, which has the smallest amount of labeled
training data. This underscores SurgeNet’s effectiveness in small-dataset
scenarios. However, even on datasets with relatively large labeled data, such
as CholecSeg8k, the segmentation model still benefits substantially from
SurgeNet pretraining.

Figure 2: Visual examples from the three downstream test sets. The first two
rows display samples from the CholecSeg8k test set, the subsequent two rows
show samples from the RAMIE test set, and the final two rows present samples
from the RARP test set.

In Figure 2, visual results for all three downstream datasets are displayed.
The differences between SurgeNet and ImageNet are particularly pronounced in
the anatomy classes, while the smallest differences are observed with the
segmentation of surgical tools. Small structures e.g. the nerve (third row,
yellow) and the ligated plexus (last two rows, white) are completely missed by
the ImageNet-initialized model. However, both the SurgeNet-specific and
SurgeNet-initialized models can accurately detect these important anatomical
structures.

Figure 3: 2D-TSNE visualization of learned representations of 1000 randomly
sampled images per surgical procedures of SurgeNet and ImageNet.

When we consider the learned representations, the 2D-TSNE visualization in
Figure 3 indicates that the SurgeNet pretrained CAFormer-S18 can cluster
images from the same procedure without any supervision during pretraining.
This suggests that the encoder has learned meaningful representations
capturing procedure-specific information. Additionally, this capability
highlights the potential of the pretrained model to generalize well to unseen
data and enhance performance in downstream tasks. These findings underscore
the effectiveness of using a heterogeneous and comprehensive dataset like
SurgeNet for pretraining, since it enables the model to develop a robust
understanding of surgical contexts.

##  4 Conclusions and Future work

This study demonstrates the benefits of increasing data diversity for self-
supervised pretraining in surgical computer vision. We construct a large-scale
dataset, SurgeNet, containing over 2.6 million surgical video frames from more
than seven procedures. When testing on three segmentation downstream datasets,
each representing a different surgical procedure, we observe substantial
improvements with pretraining on the SurgeNets’ procedure-specific datasets,
compared to ImageNet by 13.8%, 9.5%, and 36.8%. Incorporating extra data from
various procedures by pertaining on the complete SurgeNet, further enhances
the results with an additional 5.0%, 5.2%, and 2.5%. The largest performance
gains are observed with the smallest downstream datasets, indicating that
pretraining on SurgeNet is especially beneficial in small-data scenarios.
Additionally, visual examples show the biggest differences on the smallest and
often hardest structures to segment. Future work should investigate if
pretraining on SurgeNet can advance other applications, like surgical phase or
action recognition, where next to temporal feature extraction, spatial feature
extraction is also crucial. Although our results show that the model
pretrained with SurgeNet can cluster different surgical procedures without
supervision, it remains to be seen if pretraining with SurgeNet captures more
complex relationships, such as temporal cues between frames. The obtained
SurgeNet pretrained weights provide a superior alternative to ImageNet
pretrained models for surgical computer vision and are made publicly available
at https://github.com/TimJaspers0801/SurgeNet.

####  4.0.1 Acknowledgements

We thank SURF (www.surf.nl) for the support in using the National
Supercomputer Snellius.

## References

  * [1] Alapatt, D., Murali, A., Srivastav, V., Mascagni, P., Consortium, A., Padoy, N.: Jumpstarting surgical computer vision (2023)
  * [2] Bakker, F.H.A., de Nijs, J.V., Jaspers, T., et al.: Estimating surgical urethral length on intraoperative robot-assisted prostatectomy images using artificial intelligence anatomy recognition. Journal of Endourology 0(0), null (0). https://doi.org/10.1089/end.2023.0697, pMID: 38613819
  * [3] Bawa, V.S., Singh, G., KapingA, F., et al.: The saras endoscopic surgeon action detection (esad) dataset: Challenges and methods (2021)
  * [4] den Boer, R.B., Jaspers, T.J.M., de Jongh, C., et al.: Deep learning-based recognition of key anatomical structures during robot-assisted minimally invasive esophagectomy. Surgical Endoscopy 37(7), 5164–5175 (July 2023). https://doi.org/10.1007/s00464-023-09990-z
  * [5] den Boer, R.B., de Jongh, C., Huijbers, W.T.E., et al.: Computer-aided anatomy recognition in intrathoracic and -abdominal surgery: a systematic review. Surgical Endoscopy 36(12), 8737–8752 (December 2022). https://doi.org/10.1007/s00464-022-09421-5
  * [6] Caron, M., Touvron, H., et al.: Emerging properties in self-supervised vision transformers. In: Proceedings of the International Conference on Computer Vision (ICCV) (2021)
  * [7] Carstens, M., Rinner, F.M., Bodenstedt, S., et al.: The dresden surgical anatomy dataset for abdominal organ segmentation in surgical data science. Scientific Data 10(1), 3 (2023). https://doi.org/10.1038/s41597-022-01719-2, https://doi.org/10.1038/s41597-022-01719-2
  * [8] Deng, J., Dong, W., Socher, R., et al.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848
  * [9] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale (2021)
  * [10] Hashimoto, D.A., Rosman, G., Volkov, M., Rus, D.L., Meireles, O.R.: Artificial intelligence for intraoperative video analysis: Machine learning’s role in surgical education. Journal of the American College of Surgeons 225(4, Supplement 1), S171 (2017). https://doi.org/https://doi.org/10.1016/j.jamcollsurg.2017.07.387, scientific Forum Abstracts: 2017 Clinical Congress
  * [11] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015)
  * [12] Hirsch, R., Caron, M., Cohen, R., et al.: Self-supervised learning for endoscopic video analysis. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. pp. 569–578. Springer Nature Switzerland, Cham (2023)
  * [13] Hong, W.Y., Kao, C.L., Kuo, Y.H., et al.: Cholecseg8k: A semantic segmentation dataset for laparoscopic cholecystectomy based on cholec80 (2020)
  * [14] Kirillov, A., Girshick, R., He, K., Dollar, P.: Panoptic feature pyramid networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)
  * [15] Lavanchy, J.L., Ramesh, S., Dall’Alba, D., et al.: Challenges in multi-centric generalization: phase and step recognition in roux-en-y gastric bypass surgery. International Journal of Computer Assisted Radiology and Surgery (May 18 2024). https://doi.org/10.1007/s11548-024-03166-3
  * [16] Leibetseder, A., Kletz, S., Schoeffmann, K., Keckstein, S., Keckstein, J.: Glenda: Gynecologic laparoscopy endometriosis dataset. In: Ro, Y.M., Cheng, W.H., Kim, J., Chu, W.T., Cui, P., Choi, J.W., Hu, M.C., De Neve, W. (eds.) MultiMedia Modeling. pp. 439–450. Springer International Publishing, Cham (2020)
  * [17] Leibetseder, A., Petscharnig, S., Primus, M.J., et al.: Lapgyn4: a dataset for 4 automatic content analysis problems in the domain of laparoscopic gynecology. In: Proceedings of the 9th ACM Multimedia Systems Conference. pp. 357–362 (2018)
  * [18] Maier-Hein, L., Eisenmann, M., Sarikaya, D., et al.: Surgical data science – from concepts toward clinical translation. Medical Image Analysis 76, 102306 (2022). https://doi.org/https://doi.org/10.1016/j.media.2021.102306
  * [19] Maier-Hein, L., Wagner, M., Ross, T., et al.: Heidelberg colorectal data set for surgical data science in the sensor operating room (2021)
  * [20] Mascagni, P., Vardazaryan, A., Alapatt, D., et al.: Artificial intelligence for surgical safety: Automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning. Annals of Surgery 275(5) (May 2022)
  * [21] Padoy, N., Blum, T., Ahmadi, S.A., Feussner, H., Berger, M.O., Navab, N.: Statistical modeling and recognition of surgical workflow. Medical Image Analysis 16(3), 632–641 (2012). https://doi.org/https://doi.org/10.1016/j.media.2010.10.001, computer Assisted Interventions
  * [22] Ramesh, S., Srivastav, V., Alapatt, D., et al.: Dissecting self-supervised learning methods for surgical computer vision. Medical Image Analysis 88, 102844 (2023). https://doi.org/https://doi.org/10.1016/j.media.2023.102844
  * [23] Tan, M., Le, Q.V.: Efficientnet: Rethinking model scaling for convolutional neural networks (2020)
  * [24] Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., de Mathelin, M., Padoy, N.: Endonet: A deep architecture for recognition tasks on laparoscopic videos. IEEE Transactions on Medical Imaging 36(1), 86–97 (2017). https://doi.org/10.1109/TMI.2016.2593957
  * [25] Valderrama, N., Ruiz Puentes, P., Hernández, I., et al.: Towards holistic surgical scene understanding. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. pp. 442–452. Springer Nature Switzerland, Cham (2022)
  * [26] Wang, W., Xie, E., Li, X., Fan, D.P., Song, K., Liang, D., Lu, T., Luo, P., Shao, L.: Pyramid vision transformer: A versatile backbone for dense prediction without convolutions (2021)
  * [27] Wang, Z., Liu, C., et al.: Foundation model for endoscopy video analysis via large-scale self-supervised pre-train. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 101–111. Springer (2023)
  * [28] Yoon, J., Lee, J., Heo, S., et al.: hsdb-instrument: Instrument localization database for laparoscopic and robotic surgeries. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. pp. 393–402. Springer International Publishing, Cham (2021)
  * [29] Yu, W., Si, C., Zhou, P., et al.: Metaformer baselines for vision. IEEE Transactions on Pattern Analysis and Machine Intelligence 46(2), 896–912 (Feb 2024). https://doi.org/10.1109/tpami.2023.3329173
  * [30] Zhang, Y., Bano, S., Page, A.S., Deprest, J., Stoyanov, D., Vasconcelos, F.: Retrieval of surgical phase transitions using reinforcement learning. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. pp. 497–506. Springer Nature Switzerland, Cham (2022)
  * [31] Zia, A., Bhattacharyya, K., Liu, X., et al.: Surgical tool classification and localization: results and methods from the miccai 2022 surgtoolloc challenge (2023)

##  5 Supplementary Materials

Figure 4: 400 randomly sampled images from SurgeNet. Figure 5: Per-class
performance evaluation on three downstream datasets. The top left radar plot
displays the results for the CholecSeg8k test set. The top right plot presents
the results for the RAMIE test set. The bottom plot illustrates the results
for the RARP dataset. The differences are most pronounced for the smaller
anatomical structures.


# A Hybrid Optimization and Deep Learning Algorithm for Cyber-resilient DER
Control

Mohammad Panahazari1, Matthew Koscak1, Jianhua Zhang1, Daqing Hou1, Jing Wang2
and David Wenzhong Gao3 1Department of Electrical and Computer Engineering,
Clarkson University, Potsdam, NY 13699, USA
2National Renewable Energy Laboratory (NREL), Golden, CO 80401, USA
3Department of Electrical and Computer Engineering, University of Denver,
Denver, CO 80208, USA
Emails: {panaham, koscakmm, jzhang<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

With the proliferation of distributed energy resources (DERs) in the
distribution grid, it is a challenge to effectively control a large number of
DERs resilient to the communication and security disruptions, as well as to
provide the online grid services, such as voltage regulation and virtual power
plant (VPP) dispatch. To this end, a hybrid feedback-based optimization
algorithm along with deep learning forecasting technique is proposed to
specifically address the cyber-related issues. The online decentralized
feedback-based DER optimization control requires timely, accurate voltage
measurement from the grid. However, in practice such information may not be
received by the control center or even be corrupted. Therefore, the long
short-term memory (LSTM) deep learning algorithm is employed to forecast
delayed/missed/attacked messages with high accuracy. The IEEE 37-node feeder
with high penetration of PV systems is used to validate the efficiency of the
proposed hybrid algorithm. The results show that 1) the LSTM-forecasted lost
voltage can effectively improve the performance of the DER control algorithm
in the practical cyber-physical architecture; and 2) the LSTM forecasting
strategy outperforms other strategies of using previous message and skipping
dual parameter update.

###### Index Terms:

orithm, distributed energy resources (DERs), DER control, LSTM, Deep learning.

cyber-resilient alg

##  I Introduction

The distribution grid is undergoing 1) proliferation of distributed energy
resources (DERs) including utility-level DERs and behind-the-meter (BTM) DERs,
2) more and faster data streaming from sensor networks, 3) underpinning data-
driven methods, and 4) local energy market design. This creates the open
research question that how does the future development of the synchronized
sampling data and data analytics technology may contribute to the grid
visibility, and reliable and resilient operation of the integrated grid.
Especially, geographically dispersed DERs can be coordinated at scale with two
basic core functions: a) DER production scheduling, dispatch of active and
reactive power to address stochastic and dynamic challenges; b) DER ancillary
services provision, including frequency and voltage regulation [1]. However,
coordinating a large number of DERs heavily depend on access to reliable and
secure data, sensing, communications and computing at multiple operational
timescales spanning milliseconds to hours[2]. Therefore, as a typical cyber-
physical system, the development of the DER management systems (DERMS) and
scalable cyber-resilient DER monitoring and control algorithms for the
distribution grid with proliferation of heterogenous grid-edge resources still
remains unsolved.

Figure 1: Cyber-resilient DER Control Architecture of Hybrid Feedback-based
Optimization and Deep Learning Algorithm

The existing research work related to the DER coordination are focusing on 1)
DERMS platform[3, 4], 2) optimal voltage regulation of virtual power plant
(VPP) [5, 6], and communications architectures for DER coordination[7, 2, 8].
However, very little attention has been paid to perhaps development of
scalable cyber-physical DER control algorithms resilient to asynchronous data
flow resulting from real communication networks. Therefore, the novel cyber-
resilient DER control algorithms are in a critical need to address
communication and security issues.

To fill in this gap, this study further proposes a hybrid feedback-based
optimization and deep learning algorithm for DER control at the grid edge with
incentive for utilizing more sampling grid data and underpinning data-driven
methods; and providing the guideline to the DERMS deployment. This work is
based on the existing optimal regulation of virtual power plant (VPP)
algorithm [5] and cyber-physical DER control algorithm [9]. The challenge of
development of cyber-resilient DER control algorithms is the way to handle
delayed or lost voltage measurements. To this end, the long short-term memory
(LSTM) deep learning algorithm is employed to forecast delayed/missed messages
with high accuracy, which is a main contribution of this paper.

##  II Problem Recap of DER Control

A DER penetrated distribution feeder with $N+1$ nodes,
$\mathcal{N}\cup\\{0\\},\mathcal{N}:=\\{1,...,N\\}$ is considered. The feeder
head is denoted as Node $0$. Let define the $N$-dimensional phasor voltage
vector as $\mathbf{v}:=[V_{1},...,V_{N}]^{T}\in\mathbb{C}^{N}$. $P_{0}$ and
$Q_{0}$ denote the active and reactive powers at the feeder, and $P_{l,n}$ and
$Q_{l,n}$ are the load at the $n$th node. Let
$\mathcal{G}:=\\{1,...,G\\}\subseteq\mathcal{N}$ be a set of nodes equipped
with DERs, and $P_{i}$ and $Q_{i}$ are the DER powers at Node
$i\in\mathcal{G}$. For each PV system with the capacity $S_{i}$,
$\mathcal{Y}_{i}=\\{(P_{i},Q_{i}):0\leq P_{i}\leq
P_{i}^{av},P_{i}^{2}+Q_{i}^{2}\leq S_{i}^{2}\\}\subset\mathbb{R}^{2}$ denotes
the feasible range of $P_{i},Q_{i}$, and $P_{i}^{av}$ be the available power.
The injection power at nodes $\mathcal{N}$ is denoted as
$\mathbf{s}_{inj}:=[S_{1},...,S_{N}]\in\mathcal{C}^{N}$, where
$S_{i}=-P_{l,i}-jQ_{l,i}$ for $i\in\mathcal{G}\backslash\mathcal{N}$, and
$S_{i}=P_{i}-P_{l,i}+j(Q_{i}-Q_{l,i})$ for $i\in\mathcal{G}$. Denoting
$\mathbf{v}_{nom}$ as the equilibrium point of the nominal-voltage vector, the
”LinDisFlow” approach is employed to achieve the approximate linear power flow
equations, where $|\mathbf{v}|$ and $P_{0},Q_{0}$ are the functions of real
and reactive injection power:

$\centering\begin{split}|\mathbf{v}|&\approx\mathbf{Ap_{inj}}+\mathbf{Bq_{inj}}+\mathbf{c},\\\
[P_{0},Q_{0}]^{T}&\approx\mathbf{Mp_{inj}}+\mathbf{Nq_{inj}}+\mathbf{o};\end{split}\@add@centering$
(1)

where $\mathbf{p_{inj}}:=\Re\\{\mathbf{s}_{inj}\\}$,
$\mathbf{q_{inj}}:=\Im\\{\mathbf{s}_{inj}\\}$. And suitable linearization
methods for the AC power-flow equations can be employed to achieve the model
parameters $\mathbf{A}\in\mathbb{R}^{N\times
N},\mathbf{B}\in\mathbb{R}^{N\times N},\mathbf{M}\in\mathbb{R}^{2\times
N},\mathbf{N}\in\mathbb{R}^{2\times
N},\mathbf{c}\in\mathbb{R}^{N},\mathbf{o}\in\mathbb{R}^{2}$ [5].

Each DER dispatch happens in a discrete-time fashion. For each time instant
$t_{k},k\in\mathbb{N}$, Let functions $f_{i}^{t_{k}}(\cdot)$ capture different
objectives from different DER owners and the utility, and $P_{0,set}^{t_{k}}$
be the setpoint at the feader head. Denote
$\mathcal{M}:=\\{1,...,M\\}\subset\mathcal{N}$ as a set of nodes where vlotage
measurements are available and the voltage regulation within
$[V^{min},V^{max}]$ is required at each node. Then, the DER dispatch problem
is formulated into a time-varying optimization problem with the operational
objectives and constraints at $t_{k}$, as below:

$\displaystyle\min_{P_{i},Q_{i}}$
$\displaystyle\sum_{i\in\mathcal{G}}{f_{i}^{t_{k}}(P_{i},Q_{i})}$ (2) s.t.
$\displaystyle P_{i},Q_{i}\in\mathcal{Y}_{i}^{t_{k}}(2a)$ $\displaystyle
P_{0}^{t_{k}}(P_{i},Q_{i})-P_{0,set}^{t_{k}}\leq E^{t_{k}}(2b)$
$\displaystyle-(P_{0}^{t_{k}}(P_{i},Q_{i})-P_{0,set}^{t_{k}})\leq
E^{t_{k}}(2c)$ $\displaystyle V^{min}-|V_{n}^{t_{k}}|(P_{i},Q_{i})\leq
0,\forall{n}\in\mathcal{M}\vspace{5mm}(2d)$
$\displaystyle|V_{n}^{t_{k}}|(P_{i},Q_{i})-V^{max}\leq
0,\forall{n}\in\mathcal{M}\vspace{5mm}(2e)$

Lagrangian multipliers $\lambda^{t_{k}}$ and $\zeta^{t_{k}}$ are associated
with the setpoints tracking constraints (2b)-(2c). And the dual variables
$\boldsymbol{\gamma}^{t_{k}}:=[\gamma_{1}^{t_{k}},...,\gamma_{M}^{t_{k}}]^{T}$
and $\boldsymbol{\mu}^{t_{k}}:=[\mu_{1}^{t_{k}},...,\mu_{M}^{t_{k}}]^{T}$ are
associated with the voltage regulation constraints(2d) - (2e). Then, the DER
contorl algorithm is reformulated to the lagrangian equation with
$\mathbf{d}:=\\{\boldsymbol{\gamma},\boldsymbol{\mu},\lambda,\zeta\\}$, as
below,

$\begin{split}&\mathcal{L}^{t_{k}}(\mathbf{p},\mathbf{q},\mathbf{d}):=\sum_{i\in\mathcal{G}}f_{i}^{t_{k}}(P_{i},Q_{i})\\\
&+\sum_{n\in\mathcal{M}}[\gamma_{n}(V^{min}-|V_{n}^{t_{k}}|(P_{i},Q_{i}))\\\
&+\mu_{n}(|V_{n}^{t_{k}}|(P_{i},Q_{i})-V^{max})]\\\
&+\lambda[P_{0}^{t_{k}}(P_{i},Q_{i})-P_{0,set}^{t_{k}}-E^{t_{k}}]\\\
&+\zeta[P_{0,set}^{t_{k}}-P_{0}^{t_{k}}(P_{i},Q_{i})-E^{t_{k}}]\\\
&+\frac{\nu}{2}\sum_{i\in\mathcal{G}}(P_{i}^{2},Q_{i}^{2})-\frac{\epsilon}{2}\|\mathbf{d}\|^{2}_{2},\forall{i\in\mathcal{G}},\forall{n\in\mathcal{M}}\end{split}$
(3)

where $\mathbf{p}:=[P_{1},...,P_{G}]^{T}$,
$\mathbf{q}:=[Q_{1},...,Q_{G}]^{T}$, the tracking error $E^{t_{k}}>0$, and
$\nu$ and $\epsilon$ be regularization coefficients.

##  III Hybrid Optimization and Deep Learning Algorithm for Cyber-resilient
DER Control

To solve the DER control problem described in (3) considering data loss and
network issues, a new cyber-resilient algorithm is proposed in this section.

###  III-A Distributed DER Control

The distributed architecture will improve the reliability of the DERs control
at scale. The hierarchical and distributed control framework proposed in [5,
10] consists of three main steps, shown in Fig. 1: Step 1 collecting voltage
magnitude measurements from each node $n\in\mathcal{M}$ and measurement of
$\widehat{P}_{0}^{t_{k}}$ from the head to the control center (e.g., the DERMS
software); Step 2 updating dual parameter set
$\mathbf{d}^{t_{k+1}}=[\gamma_{n}^{t_{k+1}},\mu_{n}^{t_{k+1}},\lambda^{t_{k+1}},\zeta^{t_{k+1}}]$
as follows and then broadcasting it to each DER controller/node:

$\begin{split}\gamma_{n}^{t_{k+1}}&=proj_{\mathbb{R}_{+}}\left\\{\gamma_{n}^{t_{k}}+\alpha(V^{min}-|\widehat{V}_{n}^{t_{k}}|-\epsilon\gamma_{n}^{t_{k}})\right\\},\\\
\mu_{n}^{t_{k+1}}&=proj_{\mathbb{R}_{+}}\left\\{\mu_{n}^{t_{k}}+\alpha(|\widehat{V}_{n}^{t_{k}}|-V^{max}-\epsilon\mu_{n}^{t_{k}})\right\\},\\\
\lambda^{t_{k+1}}&=proj_{\mathbb{R}_{+}}\left\\{\lambda^{t_{k}}+\alpha(\widehat{P}_{0}^{t_{k}}-P_{0,set}^{t_{k}}-E^{t_{k}}-\epsilon\lambda^{t_{k}})\right\\},\\\
\zeta^{t_{k+1}}&=proj_{\mathbb{R}_{+}}\left\\{\zeta^{t_{k}}+\alpha(P_{0,set}^{t_{k}}-\widehat{P}_{0}^{t_{k}}-E^{t_{k}}-\epsilon\zeta^{t_{k}})\right\\};\\\
\end{split}$ (4)

Step 3 calculating and updating new $P_{i}^{t_{k+1}},Q_{i}^{t_{k+1}}$ at each
DER agent as follow, after receiving
$\widehat{P}_{i}^{t_{k}},\widehat{Q}_{i}^{t_{k}}$ locally and
$\mathbf{d}^{t_{k+1}}$ remotely from control center:

$\begin{split}[P_{i}^{t_{k+1}},Q_{i}^{t_{k+1}}]^{T}&=proj_{\mathcal{Y}_{i}^{t_{k}}}\\{[P_{i}^{t_{k}},Q_{i}^{t_{k}}]^{T}\\\
&-\alpha\nabla_{[P_{i},Q_{i}]}\mathcal{L}^{t_{k}}(\mathbf{p},\mathbf{q},\mathbf{d})|_{\widehat{P}_{i}^{t_{k}},\widehat{Q}_{i}^{t_{k}},\mathbf{d}^{t_{k+1}}}\\};\end{split}$
(5)

In the cyber-physical system, Step 1 and Step 2 is implemented in the control
center located in the feeder head, and Step 3 is conducted in the individual
DER control agent.

###  III-B Sensitivity Analysis of Delayed Messages

As illustrated in Fig. 1, there are two main data streams for the hierarchical
DER control algorithm. The upstream collects the nodal voltage measurement and
the downstream sends updated dual variables to DER controllers. In our
previous research work in [9], we developed two strategies to deal with
delayed messages in both uplink and downlink. The first strategy is to use
previous measurement of a delayed/missed message to continue the DER control
procedure, and the another strategy is to skip the updating of dual parameters
or new dispatched power for corresponding delayed messages. Along with these
two strategies, we validated the impact of individual communication
uplink/downlink situation on the control algorithm performance, based on the
metric of the feeder head’s power setpoint tracking error. The sensitivity
analysis results show that more voltage measurement delayed in the uplink will
degrade the algorithm performance more dramatically for both strategies,
compared to delayed downlink dual variables. Fig.2 shows such sensitivity
observation of using previous message with different message loss rates.

Figure 2: Tracking Error-based Sensitivity analysis

###  III-C LSTM Network

The above sensitivity analysis results indicate that it is critically needed
to develop a more intelligent and effective method to deal with the
delayed/lost voltage magnitude measurement massages. Currently. data-driven
methods have obtained a great success in anomaly detection and missed features
and data estimation [11]. In addition, considering time series nature of
collected voltage magnitude measurements, the state of art long-short term
memory (LSTM) network is proposed to effectively estimate the delayed/lost
data. The LSTM is an extended and advanced version of traditional recurrent
neural networks (RNNs) [12].

The LSTM network depicted in Fig. III-Clstm, contains cell states, input gate,
output gate, and forget gate. The cell state $c(t)$ is the key concept of the
LSTM model and it keeps important parts of historical data. The input gate
decides to select parts of the input which is relevant to the current state of
the system and allows them to pass through the gate. This procedure is
implemented by considering the previous output $h_{t-1}$ and the current input
$x_{t}$ together, as below:

$i_{t}=\sigma(W_{i}.[h_{t-1},x_{t}]+b_{i}),$ (6)

where $i_{t}$, $\sigma$, $W_{i}$ and $b_{i}$ are output of input gate, sigmoid
function, weight matrix and bias vector of input gate, respectively. The
output of sigmoid function is in the range of (0,1). The value close to 1
means that the input is more relevant to the current cell state, while the
value close to 0 means there is a few coherency between input and current cell
state. Then, to filter the desired part of input, the $\tanh$ layer is used to
create a vector of new candidate values, $\tilde{C_{t}}$, which will be used
to create the new cell state. The $\tilde{C_{t}}$ can be found as follow:

$\tilde{C_{t}}=\tanh(W_{C}.[h_{t-1},x_{t}]+b_{C}),$ (7)

where $W_{C}$ and $b_{C}$ are weight matrix and bias vector of input layer.
The forget gate decides what part of the previous state should be forgotten.
The procedure is similar to the input gate:

$f_{t}=\sigma(W_{f}.[h_{t-1},x_{t}]+b_{f}),$ (8)

where $W_{f}$ and $b_{f}$ are weight matrix and bias vector of the forget
layer. The forget gate is equipped with a sigmoid function to choose parts of
previous step that remain in the cell state. Combining new data came from the
input layer and remained data from the previous cell state, the new cell state
can be calculated as below:

$C_{t}=f_{t}*C_{t-1}+i_{t}*\tilde{C_{t}}.$ (9)

The output gate decides what should be reported as output. This output is
based on the new cell state, as shown:

$\begin{split}o_{t}=\sigma(W_{o}[h_{t-1},x_{t}]+b_{o}),\\\
h_{t}=o_{t}*\tanh{C_{t}}.\end{split}$ (10)

This LSTM network will be used to forecast the delayed or missed voltage
measurement messages in the uplink.

Figure 3: LSTM Network Structure

lstm

###  III-D Hybrid Cyber-resilient DER Control Algorithm

The key concept of the hybrid cyber-resilient DER control algorithm is to
employ the deep learning forecast technique to resilient the cyber issues,
such as delayed message, lost message, as well as attacked message. Thus, the
proposed algorithm will integrate the LSTM-based delayed message forecast
model into the original optimization-based DER control framework. This LSTM
forecast model consists of four components: Data collector and validator,
Historical data, LSTM network training, and Online voltage forecast, shown in
Fig.1. At each iteration, the Data collector and validator module collects the
voltage measurements and validates if the measurement arrives within the
threshhold. All received messages are stored into the Historical data block
for the training purpose. An optimization technique and a back-propagation
through the time are employed for the LSTM network training, and this offline
LSTM network can be trained in a periodic way to have updated and more
accurate network parameters, which are passed to the Online voltage forecast
module periodically. Once the Online voltage forecast module is informed that
there is a message delayed, it conducts the forecast the delayed message in
real-time to ensure the DER control algorithm running properly.

Algorithm 1 Hybrid Optimization and Deep Learning Algorithm for Cyber-
resilient DER Control

1:procedure DERMS($\nu,\epsilon,\alpha$)

2: initialization: $t_{k}=1,d^{*},V^{min},V^{max},n\in\mathcal{M}$

3: repeat

4: update $E^{t_{k}}$

5: wait

6: receive the setpoint: $\widehat{P}_{0,set}^{t_{k}}$

7: receive measurements: $|\widehat{V}_{n}^{t_{k}}|$,
$\widehat{P}_{0}^{t_{k}}$,

8: until timer $\geq d^{*}$ or all measurements received

9: if $|\widehat{V}_{n}^{t_{k}}|$ received within $d^{*}$ then

10: update $\mathbf{d}^{t_{k+1}}$ by (4)

11: else

12: call LSTM forecast model to estimate $|\widehat{V}_{n}^{t_{k}}|$

13: wait

14: receive estimated $|\widehat{V}_{n}^{t_{k}}|$ to update
$\mathbf{d}^{t_{k+1}}$

15: end if

16: broadcast $\mathbf{d}^{t_{k+1}}$ to all DERs at grid edge

17: $t_{k}=t_{k}+1$

18:end procedure

19:procedure Local DER agent $i$

20: initialization: $t_{k}=1,\mathcal{Y}_{i}^{t_{k}}$

21: repeat

22: receive $\widehat{P}_{i}^{t_{k}},\widehat{Q}_{i}^{t_{k}}$

23: wait

24: until receive $\mathbf{d}^{t_{k+1}}$

25: update $P_{i}^{t_{k+1}},Q_{i}^{t_{k+1}}$ by (5)

26: dispatch $P_{i}^{t_{k+1}},Q_{i}^{t_{k+1}}$ to the DER device

27: $t_{k}=t_{k}+1$

28: send $|\widehat{V}_{n}^{t_{k}}|$ to the DERMS

29:end procedure

30:procedure Local non-DER grid edge $n$

31: initialization: $t_{k}=1$

32: while do

33: send $|\widehat{V}_{n}^{t_{k}}|$ to the DERMS

34: $t_{k}=t_{k}+1$

35: end while

36:end procedure

37:procedure LSTM
Estimator($\tilde{V_{n}}^{t_{k-len}},...,\tilde{V_{n}}^{t_{k-1}}$)

38: normalize input voltage vector to per unit value

39: predict using historical voltages:
[$\tilde{V_{n}}^{t_{k-len}},...,\tilde{V_{n}}^{t_{k-1}}$]

40: de-normalize predicted voltage value

41: report predicted voltage value: $|\widehat{V}_{n}^{t_{k}}|$

42:end procedure

We define a deadline or delay threshold, namely $d^{*}>0$ in milliseconds, for
the uplink message. In a normal operating condition, the Data collector and
validator module collects and validates nodal voltages and the DER control
module generates dual variables to update DERs’ setpoints, shown in Fig. 1).
If any local voltage measurement $|\widehat{V}_{n}^{t_{k}}|$ does not arrive
at the DERMS within time $d^{*}$, the LSTM forecast model is to predict the
delayed voltage $\tilde{V_{n}}^{t_{k}}$ by using previous voltages
$\tilde{V_{n}}^{t_{k-len}},...,\tilde{V_{n}}^{t_{k-1}}$ of Node $n$ to
continue computing the dual parameters
$\lambda_{n}^{t_{k+1}},\mu_{n}^{t_{k+1}}$, where $len$ is the length of
historical data at Node $n$, that work as the input of the LSTM forecast
model. The resulting hybrid DER control algorithm is described in detail in
Algorithm 1.

(a) Profile of Loads and PV Generation (b) Using Previous Values (c) Skipping
(d) LSTM Forecast

(e) Voltage without DER Control (f) Using Previous Values  (g) Skipping  (h)
LSTM Forecast

Figure 4: Profile of Loads and PV Generation, Voltage Regulation and Setpoint
Tracking Performance of Cyber-resilient DER Control Algorithm for Using
Previous value (b,c), Skipping update(c,g) and LSTM forecast (d,h)

##  IV Validation and Results

To validate the proposed hybrid algorithm, we consider a modified single phase
IEEE 37-node test feeder and please refer to [9] for the detailed
configuration data, and the topology is shown in Fig. 5. The generation
profile data is generated based on the real solar radiation data of
Sacramento, CA on August 15, 2012 from the NREL Measurement and
Instrumentation Data Center (MIDC) with a granularity of 1 second after
processing and capacity of 50kW, shown in Fig. 4(a) too. Other parameters are
set as
$V^{min}=0.95,V^{max}=1.05,\nu=10^{-3},\epsilon=10^{-4},E^{t_{k}}=0.001$,and
the step size $\alpha=0.1$. And the PV system optimization objective (3) is
set as
$f_{i}^{t_{k}}(P_{i},Q_{i})=c_{p}(P_{av,i}^{t_{k}}-P_{i}^{t_{k}})^{2}+c_{q}(Q_{i}^{t_{k}})^{2}$,
where $c_{p}=3,c_{q}=1,i\in\mathcal{G}$. We consider the setpoints
$P_{0,set}^{t_{k}}$ from 12:00 to 14:00, consists of 5-minute economic
dispatch commands, 1-minute automatic generator control setpoints, ramp
signals and constant commands of 65 minutes, depicted in red line, shown in
Fig. 4(b).
The LSTM network is implemented by using the Keras library. To generate the
training data set of voltage values, the randomly generated $P_{0}$ setpoint
curves are used to run the algorithm in the ideal cyber network. The look-back
time window size is set to 10 to train the LSTM model for each node and the
root mean square error (RMSE) is adopted as the loss function to optimize
trained model.
To validate the performance of the proposed hybrid optimization and deep
learning cyber-resilient DER control algorithm, we conduct the comparison with
other two commonly-used strategies for the delayed messages: 1) using previous
voltage measurement to update dual parameters, and 2) skipping the update of
the corresponding dual parameters. The delay model described in [9] is applied
to generate delays in the uplink. Setting $d^{*}=6.675$ ms will lead to 1% of
messages being delayed. To better show the impact of delayed messages on the
performance of DER control algorithm, the communication delay model has been
applied only from 12:30 to 13:30. We implemented IEEE-37 test case in OpenDSS
and the cyber-physical DER control along with two above-mentioned strategies
in Matlab and the LSTM based voltage forecast model in Python with a
granularity of 1 second.
Testing the trained LSTM model approves the high accuracy of LSTM in
predicting missed voltage values. The RMSE for predicting missed voltages for
our test case is 0.00065 kV. The tracking and voltage regulation performance
is shown in Fig. 4. From Fig. 4(b) and (f), we have this observation: the
strategy of using previous message for delayed measurements can not be
successful in keeping setpoint tracking and voltage regulation convergence.
Even after removing asynchrony of communication, the algorithm is not able to
track the setpoint. The performance of the skipping strategy, shown in Fig.
4(c) and (g), indicates that the skipping strategy outperforms the strategy of
using previous message, although the total performance of this strategy is not
acceptable in practice. Fig. 4(d) and (h) shows that the LSTM forecast
strategy can track $P_{0,set}^{t_{k}}$ with the RMSE value of 3.685 kW and
regulate nodal voltages properly, and it obviously has the best and acceptable
performance among three strategies with 1% delay rate.

Figure 5: IEEE 37-node Feeder with PV System and Load Deployment

##  V Conclusion

In this paper, we developed a hybrid feedback-based optimization and deep
Learning algorithm for cyber-resilient DER control to enhance the resiliency
of the DERMS system to all kinds of cyber issues, such as the delayed/lost
voltage measurements. The well-trained LSTM forecast model can estimate the
delayed voltage data with high accuracy. The experiment result shows that the
proposed algorithm obviously outperforms both using previous message and
skipping strategies for the delayed messages.

## Acknowledgement

Matthew Koscak was partially supported by NSF OAC-1852102.

## References

  * [1] “Ieee guide for distributed energy resources management systems (derms) functional specification,” _IEEE Std 2030.11_ , pp. 1–61, 2021.
  * [2] M. Zajc, M. Kolenc, and N. Suljanovic, “Virtual power plant communication system architecture,” _Smart Power Distribution Systems_ , pp. 231–250, 2018\.
  * [3] “Understanding derms,” July 2018.
  * [4] “Electric program investment charge (epic) 2.02 - distributed energy resource management system,” Jan. 2019.
  * [5] E. Dall’Anese, S. S. Guggilam, A. Simonetto, Y. C. Chen, and S. V. Dhople, “Optimal regulation of virtual power plants,” _IEEE Transactions on Power Systems_ , vol. 33, no. 2, pp. 1868–1881, 2018.
  * [6] X. Zhou, Z. Liu, W. Wang, C. Zhao, F. Ding, and L. Chen, “Hierarchical distributed voltage regulation in networked autonomous grids,” in _2019 American Control Conference (ACC)_ , 2019, pp. 5563–5569.
  * [7] F. Heimgaertner and M. Menth, “Distributed controller communication in virtual power plants using smart meter gateways,” in _2018 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC)_ , 2018, pp. 1–6.
  * [8] J. Zhang, A. Hasandka, S. Alam, T. Elgindy, A. Florita, and B. Hodge, “Analysis of hybrid smart grid communication network designs for distributed energy resources coordination,” in _IEEE Power Energy Society Innovative Smart Grid Technologies Conference (ISGT)_ , 2019, pp. 1–5.
  * [9] H. Gan, J. Zhang, J. Wang, D. Hou, Y. Jiang, and D. W. Gao, “Cyber physical grid-interactive distributed energy resources control for VPP dispatch and regulation,” in _IEEE PES Innovative Smart Grid Technologies Europe, ISGT Europe 2021, Espoo, Finland, October 18-21, 2021_. IEEE, 2021, pp. 1–5.
  * [10] A. Bernstein and E. Dall’Anese, “Real-time feedback-based optimization of distribution grids: A unified approach,” _IEEE Transactions on Control of Network Systems_ , vol. 6, no. 3, pp. 1197–1209, 2019.
  * [11] I. Mitiche, T. McGrail, P. Boreham, A. Nesbitt, and G. Morison, “Data-driven anomaly detection in high-voltage transformer bushings with lstm auto-encoder,” _Sensors_ , vol. 21, no. 21, 2021.
  * [12] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” _Neural Computation_ , vol. 9, no. 8, pp. 1735–1780, 11 1997.