# Online learning for robust voltage control under uncertain grid topology

Christopher Yeh , Jing Yu , Yuanyuan Shi ,
Adam Wierman C. Yeh, J. Yu, and A. Wierman are with the Department of
Computing and Mathematical Sciences, California Institute of Technology,
Pasadena, California, U.S.A. (email: cyeh@caltech.edu)Y. Shi is with the
Department of Electrical and Computer Engineering, University of California,
San Diego, La Jolla, California, U.S.A.

###### Abstract

Voltage control generally requires accurate information about the grid’s
topology in order to guarantee network stability. However, accurate topology
identification is challenging for existing methods, especially as the grid is
subject to increasingly frequent reconfiguration due to the adoption of
renewable energy. In this work, we combine a nested convex body chasing
algorithm with a robust predictive controller to achieve provably finite-time
convergence to safe voltage limits in the online setting where there is
uncertainty in both the network topology as well as load and generation
variations. In an online fashion, our algorithm narrows down the set of
possible grid models that are consistent with observations and adjusts
reactive power generation accordingly to keep voltages within desired safety
limits. Our approach can also incorporate existing partial knowledge of the
network to improve voltage control performance. We demonstrate the
effectiveness of our approach in a case study on a Southern California Edison
56-bus distribution system. Our experiments show that in practical settings,
the controller is indeed able to narrow the set of consistent topologies
quickly enough to make control decisions that ensure stability in both
linearized and realistic non-linear models of the distribution grid.

###### Index Terms:

voltage control, distribution grid, convex body chasing, robust control

##  I Introduction

Operators of electricity distribution grids must maintain voltages at each bus
within certain operating limits, as deviations from such limits may damage
electrical equipment and cause power outages [1, 2]. This “voltage control” or
“voltage regulation” problem has been well-studied, e.g., [3, 4, 5] and the
references therein. Voltage control devices and algorithms aim to guarantee
grid stability and minimize the costs associated with control inputs. While
classic voltage regulation devices such as tap-changing transformers are
effective in dealing with _slow_ voltage variations [6, 7], increasing
penetration of renewables leads to faster variations, and a growing body of
literature has focused on inverter-based controllers that can respond quickly
by adjusting their active and reactive power set-points. Most of these works
cast voltage control as an optimization problem and then propose different
centralized or decentralized algorithms depending on the communication
infrastructure.

Typically, voltage control algorithms assume exact knowledge of the underlying
grid topology. This includes centralized controllers such as algorithms based
on model predictive control (MPC) which optimize control decisions for a
short-term horizon. [8] uses MPC to manage distributed generation and energy
storage systems, whereas [9] proposes a robust MPC controller that is robust
to uncertainty in the forecasts of future loads and solar generation.

However, the exact grid topology and line parameters are often not known, and
using existing voltage control algorithms with incorrect grid information may
lead to problems with grid stability [10, 11]. For example, parts of the grid
may undergo reconfiguration due to load balancing or unplanned maintenance, as
frequently as every hour of the day [12, 13, 14, 15]. This problem is
exacerbated by the increasing integration of distributed energy resources
(DERs), such as photovoltaic (PV) and storage devices. Especially in
distribution grids, where DERs are not owned or operated by the electricity
utility, the grid operator may lack up-to-date information about the grid
topology [16]. While a grid operator can install sensors to help identify the
current network topology, unless such sensors are densely deployed (at great
cost), uncertainty about the topology remains. Thus, distribution grid
operators cannot expect to operate with perfect topology information and the
design of voltage control algorithms robust to unknown grid topology is
crucial.

There are several families of existing algorithms that do not require knowing
the network topology: decentralized controllers, model-free controllers, and
controllers that first try to infer the network topology. While decentralized
voltage control algorithms are generally efficient to implement, such
controllers lack voltage stability guarantees when the load is time-varying
[17, 18, 4, 19, 20]. Likewise, model-free controllers based on deep
reinforcement learning do not require knowing the network topology, but they
generally have no performance or voltage stability guarantees and are
therefore not suitable for safety-critical infrastructure [21, 22, 23, 24,
25]. Some recent works [26, 27, 28] have proposed methods for introducing
stability guarantees for model-free deep reinforcement learning approaches.
Their main tool is Lyapunov stability theory, from which a structural
constraint for stable controllers is derived, and policy optimization with the
constraint is performed. However, their stability guarantees are only valid
over an infinite time horizon, and achieving good performance with deep
reinforcement learning generally requires large amounts of historical training
data. In contrast, our proposed framework jointly learns the system model
(consistent with data) and stable controller in an online fashion, achieving a
finite-mistake guarantee and good performance without relying on historical
data.

Another standard approach for handling uncertainty about network topology is
to first estimate the topology and line parameters using a form of system
identification with data and then apply a standard voltage control algorithm
using the identified network topology. There is a growing literature of such
data-driven methods, e.g., [29, 16, 10, 11, 30, 31, 32, 33, 34, 35, 36, 37]. A
common approach is to leverage least squares for system model estimation. The
estimation and therefore control guarantees depend on statistical modeling of
measurement noise (e.g., Gaussian). In contrast, we leverage online learning
in order to be robust against any bounded disturbances, such as modeling
errors and adversarial noise. While least squares-based algorithms focus on
asymptotic estimation convergence, e.g. [38, 39], we present a finite mistake
guarantee that is crucial for safe transient system behavior.

Another prominent approach is to use graphical models for topology
reconstruction [40], via maximum likelihood methods while enforcing other
structural restrictions like low-rank and sparsity. However, these methods
that first perform some form of system identification have drawbacks. First,
the estimated topology and/or system dynamics may be imperfect [41], and
applying standard voltage control algorithms using these imperfect estimates
may still lead to system instability. Second, these methods either assume
access to historical data or require acquiring data online over hundreds of
time steps, during which the stability of the system is ignored [16, 40]. In
contrast, our proposed approach does not perform system identification
separately from control; the joint operation of our robust controller with the
system dynamics estimation gives rise to our stability guarantee.

###  I-A Contributions

We propose a new approach for voltage control over an uncertain grid topology
that does not perform system identification and voltage control separately.
Instead, our approach robustly learns to stabilize voltage within the desired
limits directly, _without prior knowledge of the topology and without needing
to precisely learn the topology_. Our approach takes ideas from online nested
convex body chasing (NCBC) [42] and robust predictive control and combines
them using a new learning framework [43] to design a voltage control
algorithm. Intuitively, we use a NCBC algorithm to track the set of topologies
that are consistent with the observed voltage measurements—as more
measurements are taken, the set of consistent topologies shrinks (and so the
sets are nested). As these measurements are taken, a form of robust predictive
control is used for voltage control, where the robustness guarantee is used to
ensure that the uncertainty about the topology can be handled. Our main result
(Theorem 1) provides a finite error stability bound for the overall
controller, which is summarized in Algorithm 1. This represents the first
voltage control algorithm that is provably robust to large uncertainty about
network topology.

This paper supersedes the results of the preliminary version of this work [44]
in the following aspects:

  1. 1.

We improve the analysis of [44], which assumes no uncertainty in the maximum
load/generation variability, to both reduce the mistake bound by a factor of 2
and also improve empirical voltage control performance.

  2. 2.

We extend our approach to handle uncertainty in the maximum variability of
load and generation entities in the grid, and we show that in the limiting
case of 0 uncertainty, our result coincides with the improved analysis
mentioned in (1).

  3. 3.

We perform case studies of the proposed algorithm on the Southern California
Edison (SCE) 56-bus distribution system [45] with a more realistic nonlinear
power flow model with partial control and partial observation. Even though the
design of our method is based on a linear approximation to the power flow
model, our method still performs well for the nonlinear system.

  4. 4.

We demonstrate how to incorporate existing partial knowledge of the grid
topology and network line parameters into the algorithm. We show that
incorporating such prior knowledge can improve the performance of our
algorithm.

##  II Model

We study voltage control on an unknown grid topology. We consider a radial
(tree-structured) power distribution network represented as a connected
directed graph $G=(\mathcal{N},\mathcal{E})$, where
$\mathcal{N}=\\{0,1,2,\dotsc,n\\}$ is the set of buses (nodes) and
$\mathcal{E}\subset\mathcal{N}\times\mathcal{N}$ is the set of lines (directed
edges). Let the network be rooted at bus 0 (the substation or slack bus), and
let other buses be branch buses. Let $\mathcal{C}\subseteq\mathcal{N}$ denote
the subset of buses with controllable reactive power injection. Because the
network is radial and rooted at bus 0, there is a unique path
$\mathcal{P}_{i}$ from bus 0 to any other bus $i$. For branch buses, let
$v\in\mathbb{R}^{n}$ be their squared voltage magnitudes and $p+\mathbf{i}q$
be their complex power injection, where $p\in\mathbb{R}^{n}$ (units
$\mathrm{W}$) is the net active power injection, and $q\in\mathbb{R}^{n}$
(units $\operatorname{Var}$) is the net reactive power injection. The DistFlow
branch equations [46] for a distribution grid are as follows, for all
$j\in\mathcal{N}$ and $(i,j)\in\mathcal{E}$:

$\displaystyle-p_{j}$
$\displaystyle=P_{ij}-r_{ij}{l_{ij}}-\sum_{k:(j,k)\in\mathcal{E}}P_{jk}$ (1a)
$\displaystyle-q_{j}$
$\displaystyle=Q_{ij}-x_{ij}{l_{ij}}-\sum_{k:(j,k)\in\mathcal{E}}Q_{jk}$ (1b)
$\displaystyle v_{j}$
$\displaystyle=v_{i}-2(r_{ij}P_{ij}+x_{ij}Q_{ij})+(r_{ij}^{2}+x_{ij}^{2})l_{ij}$
(1c) $\displaystyle l_{ij}$
$\displaystyle=\frac{P_{ij}^{2}+Q_{ij}^{2}}{v_{i}}$ (1d)

where $P_{ij}$ and $Q_{ij}$ represent the active power and reactive power flow
on line $(i,j)$, and $r_{ij},x_{ij}>0$ are the real-valued line resistance and
reactance (units $\mathrm{\SIUnitSymbolOhm}$). Equations 1a and 1b represent
the real and reactive power conservation at bus $j$, and (1c) represents the
voltage drop from bus $i$ to bus $j$.

Assuming the branch power losses ($r_{ij}l_{ij}$, $x_{ij}l_{ij}$) are
negligible yields the simplified DistFlow equations [47], which can be
rearranged into

$v=R^{\star}p+X^{\star}q+v^{0}\mathbf{1}_{n}$ (2)

where $v^{0}\in\mathbb{R}^{n}$ is the known, constant squared voltage
magnitude at the substation, and $R^{\star},X^{\star}\in\mathbb{S}^{n}$ are
computed from the network topology and line parameters

$R^{\star}_{ij}:=2\sum_{\mathclap{(h,k)\in\mathcal{P}_{i}\cap\mathcal{P}_{j}}}r_{hk},\quad
X^{\star}_{ij}:=2\sum_{\mathclap{(h,k)\in\mathcal{P}_{i}\cap\mathcal{P}_{j}}}x_{hk},\quad
i,j\in[n]$ (3)

with $[n]:=\\{1,\dotsc,n\\}$ [18]. ($\mathbb{S}^{n}$ is the set of symmetric
$n\times n$ matrices.) $R^{\star},X^{\star}$ are positive definite with
nonnegative entries [48], and the largest entry of each row of these matrices
is along the diagonal, since

$X^{\star}_{ij}=2\sum_{\mathclap{(h,k)\in\mathcal{P}_{i}\cap\mathcal{P}_{j}}}x_{hk}\leq
2\sum_{\mathclap{(h,k)\in\mathcal{P}_{i}}}x_{hk}=X^{\star}_{ii}$ (4)

and likewise for $R^{\star}_{ij}\leq R^{\star}_{ii}$.

We assume that the active power injection $p$ is exogenous but that reactive
power at each bus can be decomposed as $q=q^{c}+q^{e}$, where $q^{c}$ is the
“controllable” component and $q^{e}$ is the “exogenous” (i.e., uncontrollable)
component. Following [18], we define
$v^{\text{par}}=R^{\star}p+X^{\star}q^{e}+v^{0}\mathbf{1}_{n}\in\mathbb{R}^{n}$
(“par” stands for “partial”) representing the exogenous effects on voltage.
Then, $v=X^{\star}q^{c}+v^{\text{par}}$, which can be modeled as a discrete-
time linear system

$v(t+1)=X^{\star}q^{c}(t)+v^{\text{par}}(t).$ (5)

Substituting $u(t)=q^{c}(t)-q^{c}(t-1)$ (change in controllable reactive power
injection) and $w(t)=v^{\text{par}}(t)-v^{\text{par}}(t-1)$ (change in
exogenous noise) yields the linear dynamical system

$v(t+1)=v(t)+X^{\star}u(t)+w(t).$ (6)

The voltage control problem [45] is to drive the squared voltage magnitudes of
each bus from an initial state $v(1)\in\mathbb{R}^{n}$ into a given multi-
dimensional interval $[\underline{v},\overline{v}]\subset\mathbb{R}^{n}$; it
is possible that $v(1)$ does not start within the interval due to some large
initial disturbance. For all $t\geq 2$, the voltage control algorithm aims to
maintain $v(t)$ within $[\underline{v},\overline{v}]$, ideally as close as
possible to a “nominal” value $v^{\text{nom}}\in[\underline{v},\overline{v}]$,
typically $v^{\text{nom}}=(\underline{v}+\overline{v})/2$. The cost for
deviating from $v^{\text{nom}}$ is measured by $\left\lVert
v(t)-v^{\text{nom}}\right\rVert_{P_{v}}^{2}$ for some positive semidefinite
matrix $P_{v}$, where $\left\lVert x\right\rVert_{A}^{2}:=x^{\top}Ax$.

At each time step, buses may change their reactive power injection $q^{c}(t)$
in order to regulate the voltage close to $v^{\text{nom}}$. The reactive power
injection (including $q^{c}(0)$) is limited within a given bound
$[\underline{q},\overline{q}]\subset\mathbb{R}^{n}$. Buses not in
$\mathcal{C}$ do not have any ability to control the reactive power injection:
$\forall i\not\in\mathcal{C}.\ \underline{q}_{i}=\overline{q}_{i}=0$. We do
not place any hard “ramp constraints” on $u(t)$. However, we impose a
quadratic ramping cost $\left\lVert u(t)\right\rVert_{P_{u}}^{2}$ where
$P_{u}$ is a positive semidefinite matrix.

In summary, the voltage control problem is to determine an online sequence of
reactive power injections $q^{c}(1),q^{c}(2),\dotsc$ to drive voltages $v(t)$
to a desired interval $[\underline{v},\overline{v}]$ while minimizing voltage
violation and control costs $\left\lVert
v(t)-v^{\text{nom}}\right\rVert_{P_{v}}^{2}+\left\lVert
u(t)\right\rVert_{P_{u}}^{2}$. In this work, we solve the voltage control
problem in the setting where $X^{\star}$ is unknown.

##  III Robust Online Voltage Control

In this section we introduce our robust online voltage control algorithm
(Algorithm 1) and its performance bound (Theorem 1), which is the main result
of this paper.

Figure 1: Online robust voltage control framework

###  III-A Algorithm

As shown in Figure 1, the algorithm has two main components: a consistent
model chasing algorithm SEL (Algorithm 1, step 2) and a robust control oracle
$\Pi$ (Algorithm 1, step 3). SEL and $\Pi$ are combined by adapting ideas from
[43].

The model chasing algorithm SEL selects a consistent model for the robust
control oracle $\Pi$ out of all plausible models that are consistent with the
online observations and prior knowledge of the grid. The selection may use any
competitive NCBC algorithm, which is the online problem of choosing a sequence
of points within sequentially nested convex sets, with the aim of minimizing
the sum of distances between the chosen points [42]. In our experiments, we
use a simple projection-based NCBC algorithm, detailed in Section V.

The robust control oracle $\Pi$ is a novel robust predictive controller
(Theorem 3). The robustness guarantee of $\Pi$ is necessary for the analysis
which integrates SEL with $\Pi$ to provide the finite mistake guarantee of the
overall algorithm. We remark that other choices for either component are
possible, as long as they provide the guarantees needed in the analysis in
Section IV.

Intuitively, SEL and $\Pi$ are combined in a way such that SEL always reduces
the uncertainty about the unknown model whenever $\Pi$ outputs an action that
causes a voltage limit violation. This means that $\Pi$ cannot take too many
“bad” actions before the system uncertainty is small.

###  III-B Assumptions

Before presenting the main results, we introduce three assumptions that
underlie our analysis and discuss why they are both needed and practical.

######  Assumption 1.

The change in noise is bounded as

$\forall t:\quad\left\lVert w(t)\right\rVert_{\infty}\leq\eta^{\star},$

where $w(t)=v^{\text{par}}(t)-v^{\text{par}}(t-1)$.
$\eta^{\star}\in[0,\overline{\eta}]$ is a constant (possibly unknown), while
$\overline{\eta}$ is a known upper-bound.

This first assumption is standard and bounds the noise in the dynamics. It
represents realistic behavior in power systems where the active and exogenous
reactive power injections do not vary dramatically between time steps, as can
be seen by expanding $w(t)$:

$\displaystyle w(t)$ $\displaystyle=v^{\text{par}}(t)-v^{\text{par}}(t-1)$
$\displaystyle=R^{\star}(p(t)-p(t-1))+X^{\star}(q^{e}(t)-q^{e}(t-1)).$

For example, if the net active and exogenous reactive power injection is the
same at time steps $t$ and $t-1$, then $w(t)=0$.

An unknown $\eta^{\star}$ indicates uncertainty in the maximum variability of
the exogenous power injections. Unlike [44] which assumes a fixed $\eta$, our
inclusion of both an unknown $\eta^{\star}$ and a known upper-bound
$\overline{\eta}$ allows more flexibility in our algorithmic design and the
incorporation of prior knowledge.

######  Assumption 2.

The true model $X^{\star}$ lies within a known compact, convex uncertainty set
$\mathcal{X}\subset\mathbb{S}_{+}^{n}\cap\mathbb{R}_{+}^{n\times n}$.
($\mathbb{S}_{+}^{n}$ is the set of $n\times n$ positive semidefinite
matrices, and $\mathbb{R}^{n\times n}_{+}$ is the set of $n\times n$ matrices
with nonnegative entries.)

Our second assumption bounds the uncertainty about the network topology and
line parameters. It ensures that the unknown true model parameters $X^{\star}$
belong to a compact, convex set $\mathcal{X}$, which is a minimal assumption
necessary for proving an analytic guarantee.
$P_{1}=\mathcal{X}\times[0,\overline{\eta}]$ forms the initial “consistent
set” (see Definition 2) for our consistent model chasing algorithm SEL.

This assumption is realistic, as a grid operator should have at least some
prior knowledge about the distribution grid topology and the range of possible
line parameters, even if they do not have the exact values. In cases where the
grid has multiple possible topologies due to switches, $\mathcal{X}$ could be
set to the convex hull of the corresponding $X$ matrices.

######  Definition 1 ($\left\lVert\cdot\right\rVert_{\triangle}$ and
$\left\lVert\cdot\right\rVert_{\triangle,\delta}$).

For any matrix $X\in\mathbb{S}^{n}$ and scalars $\eta,\delta\geq 0$, define

$\displaystyle\left\lVert X\right\rVert_{\triangle}$
$\displaystyle:=\left\lVert\mathrm{upper\text{-}triangle}(X)\right\rVert_{2}=\sqrt{\sum_{i=1}^{n}\sum_{j=i}^{n}X_{ij}^{2}}$
$\displaystyle\left\lVert(X,\eta)\right\rVert_{\triangle,\delta}$
$\displaystyle:=\sqrt{\delta^{2}\eta^{2}+\sum_{i=1}^{n}\sum_{j=i}^{n}X_{ij}^{2}}=\sqrt{\delta^{2}\eta^{2}+\left\lVert
X\right\rVert_{\triangle}^{2}}.$

For any sets $\mathcal{X}\subseteq\mathbb{S}^{n}$ and $A\subseteq\mathbb{R}$,
we define diameters $\operatorname{diam}(\mathcal{X})$ and
$\operatorname{diam}(\mathcal{X}\times A)$ with respect to the norms
$\left\lVert\cdot\right\rVert_{\triangle}$ and
$\left\lVert\cdot\right\rVert_{\triangle,\delta}$, respectively.

These norms isometrically map our parameter space to Euclidean space, enabling
us to take advantage of known results on NCBC within Euclidean space. For the
norm $\left\lVert\cdot\right\rVert_{\triangle,\delta}$, the hyperparameter
$\delta$ trades off the weight between $X$ and $\eta$ in the norm. The choice
of $\delta$ is discussed in Section V.

In practice, we consider uncertainty sets of the form

$\mathcal{X}_{\alpha}=\left\\{X\in\mathbb{S}_{+}^{n}\cap\mathbb{R}_{+}^{n\times
n}\ \middle|\ \begin{aligned}
&\|X-X^{\star}\|_{\triangle}\leq\alpha\left\lVert
X^{\star}\right\rVert_{\triangle},\\\ &\forall i,j\in[n]:X_{ij}\leq
X_{ii}\end{aligned}\right\\}$

with $\operatorname{diam}(\mathcal{X}_{\alpha})=2\alpha\left\lVert
X^{\star}\right\rVert_{\triangle}$. A larger $\alpha$ yields a larger
uncertainty set. From Section II (e.g., (4)), we know that
$X^{\star}\in\mathcal{X}_{\alpha}$.

Furthermore, we can incorporate partial knowledge we may have of the network
topology and/or line parameters by adding constraints to the description of
$\mathcal{X}$. For example, if we know that the lowest common ancestor between
buses $i,j$ in the network is bus $k$, then we can add the following
constraint on $X$, which is a consequence of (3):

$X_{ij}=\begin{cases}0,&k=0\\\ X_{kk},&\text{otherwise}.\end{cases}$ (7)

If we additionally know the values for some line parameters $x_{ij}$, we may
be able to further constrain some entries of $X$, again by applying (3).

######  Assumption 3.

There exists a compact, convex set
$\mathcal{V}^{\text{par}}\subset\mathbb{R}^{n}$ such that $\forall t\geq 0:\
v^{\text{par}}(t)\in\mathcal{V}^{\text{par}}$. Furthermore, for some known
$\epsilon>0$,

$\displaystyle\forall
v^{\text{par}}\in\mathcal{V}^{\text{par}},\,X\in\mathcal{X}.$
$\displaystyle\exists q^{c}\in[\underline{q},\overline{q}]\text{ s.t.
}Xq^{c}+v^{\text{par}}\in[\underline{v}+(\overline{\eta}+\epsilon)\mathbf{1},\,\overline{v}-(\overline{\eta}+\epsilon)\mathbf{1}].$

Our final assumption is about the existence of feasible control actions for
the robust control oracle. This assumption can be interpreted as either a
bound on the noise, or a requirement that the controllable reactive power
injection be flexible enough to satisfy the demand of any admissible noise. It
represents the reasonable assumption that a grid operator should have
installed enough controllable reactive power injection capability to perform
voltage control. Intuitively, the $\overline{\eta}$ padding is required for
robustness to the noise $w(t)$, while the $\epsilon$ padding is required for
robustness to model uncertainty (i.e., uncertainty about $X^{\star}$).

Algorithm 1 Online Robust Voltage Controller

Inputs

  * •

desired nominal squared voltage magnitude: $v^{\text{nom}}\in\mathbb{R}^{n}$

  * •

limits on the squared voltage magnitude:
$[\underline{v},\overline{v}]\subset\mathbb{R}^{n}$

  * •

limits on the reactive power injection:
$[\underline{q},\overline{q}]\subset\mathbb{R}^{n}$

  * •

initial state: $v(1),q^{c}(0)\in\mathbb{R}^{n}$

  * •

state and action cost matrices: $P_{v},P_{u}\in\mathbb{S}^{n}_{+}$

  * •

compact convex uncertainty set for the model parameter:
$\mathcal{X}\subset\mathbb{S}_{+}^{n}\cap\mathbb{R}^{n\times n}_{+}$

  * •

compact convex uncertainty set for exogenous voltage quantities:
$\mathcal{V}^{\text{par}}\subset\mathbb{R}^{n}$

  * •

upper bound for noise: $\overline{\eta}>0$

  * •

robustness padding: $\epsilon>0$

  * •

weight for slack variable: $\beta>0$

  * •

weight for noise accuracy: $\delta>0$

Procedure

  1. 1.

Initialize an empty trajectory $D_{0}=[\,]$. Set $t=1$.

  2. 2.

Query the model chasing algorithm for a new consistent parameter estimate:
$(\hat{X}_{t},\hat{\eta}_{t})\leftarrow\texttt{SEL}[D_{t}]$.

$\displaystyle\texttt{SEL}[D_{t}]:=\mathrm{NCBC}(P_{t},\hat{X}_{t-1},\hat{\eta}_{t-1})$
(8a) $\displaystyle P_{t}:=\left\\{(\hat{X},\hat{\eta})\ \middle|\
\begin{aligned} &\hat{X}\in\mathcal{X},\ \hat{\eta}\in[0,\overline{\eta}]\\\
&\forall(v_{i},v_{i+1},u_{i},q^{c}_{i})\in D_{t}:\\\ &\ \
\|v_{i+1}-v_{i}-\hat{X}_{t}u_{i}\|_{\infty}\leq\hat{\eta}\\\ &\ \
v_{i+1}-\hat{X}q_{i}^{c}\in\mathcal{V}^{\text{par}}\end{aligned}\right\\}$
(8b)

  3. 3.

Query the robust control oracle for the next control action:
$u(t)\leftarrow\texttt{$\Pi$}_{\hat{X}_{t},\hat{\eta}_{t}}(v(t))$.

$\displaystyle\texttt{$\Pi$}_{\hat{X}_{t},\hat{\eta}_{t}}:\ \min_{u,\xi}\ $
$\displaystyle\left\lVert\hat{v}^{\prime}-v^{\text{nom}}\right\rVert_{P_{v}}^{2}+\left\lVert
u\right\rVert_{P_{u}}^{2}+\beta\xi^{2}$ (9a) s.t. $\displaystyle
u\in\mathbb{R}^{n},\ \xi\in\mathbb{R}_{+}$ (9b)
$\displaystyle\underline{q}\preceq q^{c}(t-1)+u\preceq\overline{q}$ (9c)
$\displaystyle\hat{v}^{\prime}=v(t)+\hat{X}_{t}u$ (9d) $\displaystyle
k=\hat{\eta}_{t}+\rho\left(\frac{1}{\delta}+\left\lVert
u\right\rVert_{2}\right)$ (9e)
$\displaystyle\underline{v}+(k-\xi)\mathbf{1}\preceq\hat{v}^{\prime}\preceq\overline{v}-(k-\xi)\mathbf{1}$
(9f)

where $\rho=\delta\epsilon/(1+\delta\|\overline{q}-\underline{q}\|_{2})$.

  4. 4.

Apply the control action $u(t)$. Observe the system transition to
$v(t+1)=v(t)+X^{\star}u(t)+w(t)$ and $q^{c}(t)=q^{c}(t-1)+u(t)$.

  5. 5.

Append $(v(t),v(t+1),u(t),q^{c}(t))$ to the trajectory:

$D_{t}=\left[(v(i),v(i+1),u(i),q^{c}(i))\right]_{i=1}^{t}.$

  6. 6.

Increment $t\leftarrow t+1$. Repeat from Step (2).

###  III-C Main result

We now state our main result, which is a finite-error bound for Algorithm 1.

######  Theorem 1 (Main Result).

Under Assumptions 1, 2 and 3, Algorithm 1 ensures that the voltage limits will
be violated at most
$\frac{2\gamma(m)}{\rho}\operatorname{diam}(\mathcal{X}\times[0,\overline{\eta}])+1$
times, where
$\rho=\frac{\delta\epsilon}{1+\delta\|\overline{q}-\underline{q}\|_{2}}$ and
$\gamma(m)$ is the competitive ratio of the NCBC algorithm in $m$-dimensional
Euclidean space, where $m=1+\frac{n(n+1)}{2}$.

If $\eta^{\star}$ is known, then the voltage limits will be violated at most
$\frac{2\gamma(m)}{\rho}\operatorname{diam}(\mathcal{X})+1$ times, where
$\rho=\frac{\epsilon}{\|\overline{q}-\underline{q}\|_{2}}$ and
$m=\frac{n(n+1)}{2}$.

To the best of our knowledge, this result is the first provable stability
bound for voltage control in a setting where the network topology is unknown.
It highlights that Algorithm 1 can ensure stability even after _unknown_
changes to the network topology, e.g., due to maintenance, failures, etc.,
without the need to perform system identification while remaining robust to
any bounded and potentially adversarial perturbations satisfying Assumptions 1
and 3.

Intuitively, this result guarantees that the model chasing algorithm SEL will
learn a “good enough” model for control quickly. When the robust controller
$\Pi$ makes a mistake, the model chasing algorithm will learn from that
mistake and significantly reduce the set of consistent models. Because the
initial set of consistent models is bounded, and this set shrinks a
significant amount after each mistake, the total number of mistakes is
bounded. Note that this finite mistake bound implies finite-time convergence
to safe voltage limits without an explicit finite-time bound.

To interpret the error bounds in Theorem 1, we notice that they are
proportional to the diameter of the parameter space and the competitive ratio
$\gamma(m)$ of the NCBC algorithm, and inversely proportional to the oracle
robustness margin $\rho$. Because of computational tractability concerns, our
experiments implement SEL with a greedy projection-based NCBC algorithm with
$\gamma_{\text{proj}}(m)=\pi(m-1)m^{m/2}$ [42], rather than the state-of-the-
art Steiner point method which can achieve $\gamma_{\text{Steiner}}(m)=m/2$
[49]. As our case studies show, in practice the projection-based NCBC
algorithm performs much better than the worst-case bound. We note that any
other NCBC algorithm with a finite competitive ratio can be used in (8a) in
Algorithm 1. Investigating whether widely-used estimation methods, like least
squares, have a finite competitive ratio would be an interesting avenue for
future research.

Note that for Theorem 1 to hold, the optimization problem for the robust
control oracle $\Pi$ should first be solved without the slack variable $\xi$
in Algorithm 1. This ensures that if $(\hat{X}_{t},\hat{\eta}_{t})$ is
sufficiently close enough to the true model, then the algorithm will not make
a mistake. In the case that $\Pi$ is infeasible initially (e.g., when the
initial model estimate is far from the true model), it should be solved again
with a slack variable, which ensures feasibility. However, solving $\Pi$ twice
is unnecessary in practice, and so we have written Algorithm 1 to reflect its
practical implementation.

We outline a proof of Theorem 1 in the next section. We want to highlight one
piece of that proof that is of independent interest. In particular, a major
step in the proof is to provide a feasibility guarantee for the robust control
oracle component $\Pi$ of the algorithm, which is done in Theorem 3.

##  IV Proofs

We now prove our main result Theorem 1. Our proof builds on and adapts the
approach of [43], which outlines a general framework for integrating model
chasing and robust control. To explain the general framework, we first
consider a discrete-time nonlinear dynamical system

$x_{t+1}=f_{*}(x_{t},u_{t})+w_{t},\qquad x_{0}\text{
given},\qquad(f_{*},\mathbf{w})\in\mathcal{F},$

where $x\in\mathcal{S}\subseteq\mathbb{R}^{n}$ is the system state and
$u\in\mathcal{U}\subseteq\mathbb{R}^{m}$ is the control input. The unknown
function $f_{*}$ and disturbance sequence
$\mathbf{w}\in\ell^{\infty}(\mathbb{Z}_{+};\mathbb{R}^{n})$ belong to an
uncertainty set $\mathcal{F}$, and the disturbance is bounded as
$\left\lVert\mathbf{w}\right\rVert_{\infty}\leq\overline{\eta}$. Assume that
$\mathcal{F}$ has a _compact parametrization_ $(\mathbb{T},\mathsf{K},d)$,
where $\mathbb{T}:\mathsf{K}\to\wp(\mathcal{F})$ is a mapping from a parameter
space $\mathsf{K}$ to a set of functions and disturbances such that
$\mathcal{F}\subseteq\bigcup_{\theta\in\mathsf{K}}\mathbb{T}[\theta]$.
$\wp(\mathcal{F})$ denotes the powerset of $\mathcal{F}$. Let $d$ denote a
metric on $\mathsf{K}$, so $(\mathsf{K},d)$ is a compact metric space.

The control objective is specified as a sequence of indicator “goal” functions
$\mathcal{G}=(\mathcal{G}_{0},\mathcal{G}_{1},\dotsc)$. Each
$\mathcal{G}_{t}:\mathcal{X}\times\mathcal{U}\to\\{0,1\\}$ encodes a desired
condition per time step $t$:

$\mathcal{G}_{t}(x_{t},u_{t})=\mathbf{1}[\text{$x_{t}$, $u_{t}$ violate
desired condition at time $t$}].$

The main result of [43] specifies a set of sufficient conditions for a finite-
mistake guarantee—i.e.,
$\sum_{t=0}^{\infty}\mathcal{G}_{t}(x_{t},u_{t})<\infty$. These conditions
decouple online robust control into separate online learning and robust
control components. The online learning component requires a consistent model
chasing algorithm SEL, which takes as input the current observed trajectory
$D_{t}=[(x_{i},x_{i+1},u_{i})]_{i=1}^{t}$ and outputs an estimated parameter
$\theta_{t}\in\mathsf{K}$ which must be _consistent_ with $D_{t}$.

######  Definition 2 (Consistent Parameter).

We say $\theta\in\mathsf{K}$ is consistent with $D_{t}$ if there exists
$(f,\mathbf{w})\in\mathbb{T}[\theta]$ such that

$\forall(x_{t},x_{t+1},u_{t})\in D_{t}:\ x_{t+1}=f(x_{t},u_{t})+w_{t}.$

Let $P_{t}$ denote the set of all parameters consistent with $D_{t}$; $P_{t}$
is called the _consistent set_. We say SEL is $\gamma$-competitive if
$\sum_{t=1}^{\infty}d(\theta_{t},\theta_{t-1})\leq\gamma\max_{\theta\in\mathsf{K}}d(P_{\infty},\theta)$
holds for a fixed constant $\gamma>0$, which we call the _competitive ratio_.

The robust control component requires a control oracle $\Pi$, which given the
current state $x_{t}$ and a parameter $\theta_{t}$, outputs a control action
$u_{t}=\texttt{$\Pi$}_{\theta_{t}}(x_{t})$ that is robust for all systems that
are close to $\theta_{t}$. In particular, we call a control oracle _$\rho$
-robust_ for control objective $\mathcal{G}$, if all trajectories in
$S^{\Pi}[\rho;\theta]$ achieve $\mathcal{G}$ after finitely many mistakes.
$S^{\Pi}[\rho;\theta]$ is defined as the set of all possible trajectories
generated by $\texttt{$\Pi$}_{\hat{\theta}}$ for all $\hat{\theta}$ such that
$d(\theta,\hat{\theta})\leq\rho$:

$S^{\Pi}[\rho;\theta]=\left\\{\begin{aligned}
&D_{\infty}=[(x_{t},x_{t+1},u_{t})]_{t=1}^{\infty}:\\\ &\quad
u_{t}=\Pi_{\hat{\theta}}(x_{t})\\\ &\quad
x_{t+1}=f(x_{t},u_{t})+w_{t}\end{aligned}\,\middle|\,\begin{aligned}
&(f,\mathbf{w})\in\mathbb{T}[\theta],\\\
&d(\hat{\theta},\theta)\leq\rho\end{aligned}\right\\}$ (10)

Due to the page limit, we refer readers to [43] for a more detailed discussion
of consistent model chasing algorithms and $\rho$-robust control oracles. As a
summary, if SEL chases consistent models and $\Pi$ is a robust oracle for
$\mathcal{G}$, then the resulting $A_{\texttt{$\Pi$}}(\texttt{SEL})$ algorithm
achieves a finite mistake guarantee, which is stated in the following.

######  Theorem 2.

[43, Theorem 2.5] Assume that SEL chases consistent models and $\Pi$ is a
robust oracle for objective $\mathcal{G}$. Then for any starting point $x_{0}$
and trajectory $[(x_{t},u_{t})]_{t=0}^{\infty}$ generated by
$\mathcal{A}_{\texttt{$\Pi$}}(\texttt{SEL})$ (illustrated in Figure 1), the
following mistake guarantees hold: (i) If $\Pi$ is robust, then
$\sum_{t=0}^{\infty}\mathcal{G}_{t}(x_{t},u_{t})<\infty$; (ii) If $\Pi$ is
uniformly $\rho$-robust and SEL is $\gamma$-competitive, then

$\sum_{t=0}^{\infty}\mathcal{G}_{t}(x_{t},u_{t})<\max\left\\{1,M_{\rho}^{\texttt{$\Pi$}}\right\\}\left(\frac{2\gamma}{\rho}\operatorname{diam}(\mathsf{K})+1\right)$

where $M_{\rho}^{\texttt{$\Pi$}}$ denotes the worst case total mistakes of the
$\rho$-robust control oracle $\Pi$.

To apply Theorem 2 to prove Theorem 1, we need to prove that (i) the proposed
algorithm (8) chases consistent models and has a bounded competitive ratio,
and (ii) the proposed robust algorithm in (12) is a $\rho$-robust control
oracle, for bounded disturbance in the system topology. In particular, the
correspondence of the definitions is as follows. We have $\theta=(X,\eta)$,
and

$\displaystyle\mathsf{K}=\mathcal{X}\times[0,\overline{\eta}],\quad
v(1),q^{c}(0)\text{ given}$ $\displaystyle
d((X,\eta),(X^{\prime},\eta^{\prime}))=\|(X,\eta)-(X^{\prime},\eta^{\prime})\|_{\triangle,\delta}$
$\displaystyle\mathbb{T}[(X,\eta)]=\left\\{(f,\mathbf{w})\ \middle|\
\begin{aligned} &f(v,u)=v+Xu,\
\left\lVert\mathbf{w}\right\rVert_{\infty}\leq\eta,\\\ &\forall t\geq
0:\widehat{v^{\text{par}}_{0}}+\sum_{\tau=1}^{t}w(\tau)\in\mathcal{V}^{\text{par}}\\\
&\text{where}\
\widehat{v^{\text{par}}_{0}}:=v(1)-Xq^{c}(0)\end{aligned}\right\\}$
$\displaystyle\mathcal{F}=\bigcup_{(X,\eta)\in\mathcal{X}\times[0,\overline{\eta}]}T[(X,\eta)]$
$\displaystyle\mathcal{G}_{t}(v(t))=\mathbf{1}[v(t)\in[\underline{v},\overline{v}]].$

We begin by proving that the set $P_{t}$ defined in (11) in Algorithm 1 is
consistent with the trajectory $D_{t}$.

######  Lemma 1 (SEL is consistent).

Suppose $D_{T}$ is a trajectory of voltage measurements and control actions
taken up to time $T$:

$D_{T}=\left[(v(t),v(t+1),u(t),q^{c}(t))\right]_{t=1}^{T}.$

The set

$P_{T}:=\left\\{(\hat{X},\hat{\eta})\ \middle|\ \begin{aligned}
&\hat{X}\in\mathcal{X},\ \hat{\eta}\in[0,\overline{\eta}],\\\
&\forall(v(t),v({t+1}),u(t),q^{c}(t))\in D_{T}:\\\
&\quad\|v({t+1})-v(t)-\hat{X}u(t)\|_{\infty}\leq\hat{\eta}\\\ &\quad
v({t+1})-\hat{X}q^{c}(t)\in\mathcal{V}^{\text{par}}\end{aligned}\right\\}$
(11)

is a consistent set for $D_{T}$, i.e., $(\hat{X},\hat{\eta})$ is consistent
(Definition 2) if and only if $(\hat{X},\hat{\eta})\in P_{T}$.

###### Proof.

Consider any $(\hat{X},\hat{\eta})\in P_{T}$. For $t\in[T]$, define

$\hat{f}(v,u):=v+\hat{X}u,\qquad\hat{w}(t):=v({t+1})-v(t)-\hat{X}u(t)$

so $\|\hat{w}(t)\|\leq\hat{\eta}$ and $v(t+1)=\hat{f}(v(t),u(t))+\hat{w}(t)$.
Define $\widehat{v^{\text{par}}_{0}}:=v(1)-\hat{X}q^{c}(0)$, so for all $t\geq
0$,

$\widehat{v^{\text{par}}_{0}}+\sum_{\tau=1}^{t}w(\tau)=v(t+1)-\hat{X}q^{c}(t)\in\mathcal{V}^{\text{par}}.$

Thus, $(\hat{f},\hat{w})\in\mathbb{T}[(\hat{X},\hat{\eta})]$, so
$(\hat{X},\hat{\eta})$ is consistent with $D_{T}$.

Conversely, suppose $(\hat{X},\hat{\eta})$ is consistent with $D_{T}$, which
implies the existence of $\hat{f}(v,u):=v+\hat{X}u$ and $\mathbf{\hat{w}}$
satisfying $\left\lVert\mathbf{\hat{w}}\right\rVert_{\infty}\leq\hat{\eta}$
such that $v(t+1)=\hat{f}(v(t),u(t))+\hat{w}(t)$. Rearranging yields
$\hat{w}(t)=v(t+1)-v(t)-\hat{X}u(t)$, so $(\hat{X},\hat{\eta})$ satisfies the
norm constraint in (11). Now define

$\forall t\geq 0:\
\widehat{v}^{\text{par}}(t):=v(t+1)-\hat{X}q^{c}(t)=\widehat{v^{\text{par}}_{0}}+\sum_{\tau=1}^{t}\hat{w}(t)$

so $\widehat{v}^{\text{par}}(t)\in\mathcal{V}^{\text{par}}$ satisfies the
remaining constraint in (11). ∎

Observe that each $P_{t}$ is a closed, bounded, and convex set. Furthermore,
$P_{t}$ is non-empty, since $(X^{\star},\eta^{\star})\in P_{t}$. Intuitively,
$P_{t}$ is the smallest set containing all parameters that could generate the
observed trajectory $D_{t}$ along with a corresponding admissible sequence of
noise compatible with Assumptions 1, 2 and 3.

The consistent sets are nested $P_{t}\subseteq P_{t-1}$, and we use our
particular choice of norm $\left\lVert\cdot\right\rVert_{\triangle,\delta}$ to
establish a linear bijection between
$(\mathbb{S}^{n}\times\mathbb{R},\left\lVert\cdot\right\rVert_{\triangle,\delta})$
and Euclidean space $(\mathbb{R}^{m},\left\lVert\cdot\right\rVert_{2})$. This
allows us to take advantage of any $\gamma(m)$-competitive NCBC algorithm in
Euclidean space [42, 49], where $m$ is the dimension of the space, to prove
that SEL is $\gamma(m)$-competitive. This is formalized in the following
lemma.

######  Lemma 2 (SEL is competitive).

If the NCBC algorithm used in SEL has competitive ratio $\gamma(m)$, then SEL
is $\gamma(m)$-competitive.

###### Proof.

The proof is similar to [44, Lemma 2], except that learning $\hat{\eta}$ adds
an additional dimension to the parameter space. That is, there exists a norm-
preserving linear bijection between
$(\mathbb{S}^{n}\times\mathbb{R},\left\lVert\cdot\right\rVert_{\triangle,\delta})$
and Euclidean space $(\mathbb{R}^{m},\left\lVert\cdot\right\rVert_{2})$. ∎

Finally, we show that our controller $\Pi$ is $\rho$-robust. In particular, we
prove that $\texttt{$\Pi$}_{\hat{X}}$ makes no mistakes ($M_{\rho}^{\Pi}=0$)
given consistent parameters $(\hat{X},\hat{\eta})\in P_{t}$.

######  Theorem 3 ($\Pi$ is $\rho$-robust).

Under Assumptions 1, 2 and 3, suppose $(\hat{X},\hat{\eta})\in P_{t}$, where
$P_{t}$ is given in (11) for $t\geq 1$. Define
$\rho=\frac{\delta\epsilon}{1+\delta\|\overline{q}-\underline{q}\|_{2}}$.
Then, the following optimization problem is feasible:

$\displaystyle\min_{u\in\mathbb{R}^{n}}\quad$
$\displaystyle\left\lVert\hat{v}^{\prime}-v^{\text{nom}}\right\rVert_{P_{v}}^{2}+\left\lVert
u\right\rVert_{P_{u}}^{2}$ (12a) s.t. $\displaystyle\underline{q}\preceq
q^{c}(t-1)+u\preceq\overline{q}$ (12b)
$\displaystyle\hat{v}^{\prime}=v(t)+\hat{X}u$ (12c) $\displaystyle
k=\hat{\eta}+\rho\left(\frac{1}{\delta}+\left\lVert u\right\rVert_{2}\right)$
(12d)
$\displaystyle\underline{v}+k\mathbf{1}\preceq\hat{v}^{\prime}\preceq\overline{v}-k\mathbf{1}.$
(12e)

Further, the solution of (12), $u(t)$, guarantees voltage stability for all
$(X,\eta)\in\mathcal{X}\times[0,\overline{\eta}]$ such that
$\|(X,\eta)-(\hat{X},\hat{\eta})\|_{\triangle,\delta}\leq\rho$. That is,
$v(t)+Xu(t)+w(t)\in[\underline{v},\overline{v}]$ for all $w(t)$ such that
$\left\lVert w(t)\right\rVert_{\infty}\leq\eta$.

Observe that (12) corresponds to (9) in Algorithm 1 with the slack variable
set to zero. We note that the robustness margin $\rho$ decreases as
$[\underline{q},\overline{q}]$ increase. The intuitive reason is that the
voltage is more sensitive to changes in $\hat{X}$ when the range of possible
$u$’s expands. Therefore, a fixed voltage buffer of $\epsilon$ in constraints
(9e) and (12d) affords less robustness to changes in $\hat{X}$ as
$[\underline{q},\overline{q}]$ gets larger.

###### Proof of Theorem 3.

First, we will show that the following two conditions are sufficient for
feasibility of the optimization problem and $\rho$-robustness for the solution
$u$.

  * •

Feasibility: $k\leq\overline{\eta}+\epsilon$

  * •

Robustness: $k\geq\hat{\eta}+\rho\sqrt{\frac{1}{\delta^{2}}+\left\lVert
u\right\rVert_{2}^{2}}$

Then, we will show that our choices of $k$ and $\rho$ satisfy these sufficient
conditions.

To derive the sufficient condition for feasibility, define

$\widehat{v}^{\text{par}}(t-1):=v(t)-\hat{X}q^{c}(t-1)$

as the conjectured noise when we assume the underlying parameter is $\hat{X}$.
Since $\hat{X}\in P_{t}$ and $P_{t}\subseteq P_{t-1}$, we have
$\widehat{v}^{\text{par}}(t-1)\in\mathcal{V}^{\text{par}}$. Then, by
Assumption 3, there exists $q^{c}\in[\underline{q},\overline{q}]$ such that

$\underline{v}+(\overline{\eta}+\epsilon)\mathbf{1}\preceq\hat{X}q^{c}+\widehat{v}^{\text{par}}(t-1)\preceq\overline{v}-(\overline{\eta}+\epsilon)\mathbf{1}.$

Set $u=q^{c}-q^{c}(t-1)$ (which satisfies (12b)) and define

$\displaystyle\hat{v}^{\prime}(u)$
$\displaystyle:=v(t)+\hat{X}u=v(t)+\hat{X}[q^{c}-q^{c}(t-1)]$
$\displaystyle=\hat{X}q^{c}+\widehat{v}^{\text{par}}(t-1).$

Recalling (6), we can interpret $\hat{v}^{\prime}(u)$ as the one-step voltage
prediction (without disturbance) under the model $\hat{X}$ given control
action $u$ and the current voltage $v(t)$. We thus have

$\underline{v}+(\overline{\eta}+\epsilon)\mathbf{1}\preceq\hat{v}^{\prime}(u)\preceq\overline{v}-(\overline{\eta}+\epsilon)\mathbf{1}.$
(13)

Therefore, as long as $k\leq\overline{\eta}+\epsilon$, $u$ will satisfy
constraint (12e).

Next, we derive the sufficient condition for robustness. Let $u$ be a solution
of (12), so it satisfies (12e). Let
$(X,\eta)\in\mathcal{X}\times[0,\overline{\eta}]$ be arbitrary parameters
satisfying $\|(X,\eta)-(\hat{X},\hat{\eta})\|_{\triangle,\delta}\leq\rho$.
Define $\rho_{X}:=\|X-\hat{X}\|_{\triangle}$. By Lemma 3,

$-\rho_{X}\left\lVert
u\right\rVert_{2}\mathbf{1}\preceq(X-\hat{X})u\preceq\rho_{X}\left\lVert
u\right\rVert_{2}\mathbf{1}.$ (14)

Furthermore, suppose

$-\eta\mathbf{1}\preceq w(t)\preceq\eta\mathbf{1}.$ (15)

Adding together the 3 inequalities (12e), (14), (15) yields

$\displaystyle\underline{v}+(k-\rho_{X}\left\lVert
u\right\rVert_{2}-\eta)\mathbf{1}$ $\displaystyle\preceq v(t)+Xu+w(t)$
$\displaystyle\preceq\overline{v}-(k-\rho_{X}\left\lVert
u\right\rVert_{2}-\eta)\mathbf{1}.$

Clearly, if $k-\rho_{X}\left\lVert u\right\rVert_{2}-\eta\geq 0$, then the
desired robustness condition is satisfied. Since

$\|(X,\eta)-(\hat{X},\hat{\eta})\|_{\triangle,\delta}^{2}=\rho_{X}^{2}+\delta^{2}\left\lvert\eta-\hat{\eta}\right\rvert^{2}\leq\rho^{2},$

we have
$\left\lvert\eta-\hat{\eta}\right\rvert\leq\frac{1}{\delta}\sqrt{\rho^{2}-\rho_{X}^{2}}$.
This implies
$\eta\leq\hat{\eta}+\frac{1}{\delta}\sqrt{\rho^{2}-\rho_{X}^{2}}$. Therefore,
we can express the robustness condition in terms of $\hat{\eta}$:

$k\geq\hat{\eta}+\frac{1}{\delta}\sqrt{\rho^{2}-\rho_{X}^{2}}+\rho_{X}\left\lVert
u\right\rVert_{2}=:f(\rho_{X}).$

For $\rho>0$, $f(\rho_{X})$ is strictly concave and twice-differentiable and
therefore achieves its maximum when $f^{\prime}(\rho_{X})=0$. This maximum
value is $\hat{\eta}+\rho\sqrt{\frac{1}{\delta^{2}}+\left\lVert
u\right\rVert_{2}^{2}}$. Thus, if $k$ is at least this value, then we achieve
robustness.

Finally, we show that our choices of $k$ and $\rho$ satisfy the sufficient
conditions. Since $a+b\geq\sqrt{a^{2}+b^{2}}$ for all $a,b\geq 0$, our choice
of $k$ satisfies the robustness condition:

$k=\hat{\eta}+\rho\left(\frac{1}{\delta}+\left\lVert
u\right\rVert_{2}\right)\geq\hat{\eta}+\rho\sqrt{\frac{1}{\delta^{2}}+\left\lVert
u\right\rVert_{2}^{2}}.$

Note that while setting
$k=\hat{\eta}+\rho\sqrt{\frac{1}{\delta^{2}}+\left\lVert
u\right\rVert_{2}^{2}}$ would also satisfy the robustness condition, this
expression would make (12) a nonconvex optimization problem.

The remaining step is to satisfy the feasibility condition. We must choose
$\rho$ such that $\hat{\eta}+\rho\left(\frac{1}{\delta}+\left\lVert
u\right\rVert_{2}\right)\leq\overline{\eta}+\epsilon$. Since
$\hat{\eta}\leq\overline{\eta}$, it suffices to find $\rho$ such that
$\rho\left(\frac{1}{\delta}+\left\lVert u\right\rVert_{2}\right)\leq\epsilon$.
As $\left\lVert u\right\rVert_{2}\leq\|\overline{q}-\underline{q}\|_{2}$,
setting
$\rho=\frac{\delta\epsilon}{1+\delta\left\lVert\overline{q}-\underline{q}\right\rVert_{2}}$
satisfies the inequality. ∎

In the case where $\eta^{\star}$ is known, a similar proof shows that
$k=\eta^{\star}+\rho\left\lVert u\right\rVert_{2}$ and
$\rho=\frac{\epsilon}{\|\overline{q}-\underline{q}\|_{2}}$ satisfy feasibility
and robustness. (This can be seen as the $\delta\to\infty$ limiting case of
Theorem 3 such that consistent model chasing only updates $\hat{X}$ and keeps
$\hat{\eta}=\eta^{\star}$ fixed.)

######  Lemma 3.

For all $A\in\mathbb{S}^{n}$, $b\in\mathbb{R}^{n}$, and
$\alpha\in\mathbb{R}_{+}$,

$\left\lVert
A\right\rVert_{\triangle}\leq\alpha\quad\implies\quad-\alpha\left\lVert
b\right\rVert_{2}\mathbf{1}\preceq Ab\preceq\alpha\left\lVert
b\right\rVert_{2}\mathbf{1}.$

###### Proof.

See [44]. ∎

Finally, combining Theorem 3 with Lemma 2 and applying Theorem 2 completes the
proof of Theorem 1.

##  V Case Study

We demonstrate the effectiveness of Algorithm 1 using a case study based on a
single-phase 56-bus network ($n=55$) from the Southern California Edison (SCE)
utility, with line parameters $r_{ij},x_{ij}$ from [45, Table 1]. Even though
our algorithm only has guarantees for the linear power flow model (2), we show
that our algorithm works well on both the linear model and the more realistic
nonlinear DistFlow model (1).

(a)

(b)

Figure 2: Voltage profile of 7 buses without control, simulated with (a)
linear dynamics (2) and (b) nonlinear balanced AC dynamics (1).

###  V-A Experimental Setup

Following [20], we adapt real-world load and PV data from [50] for the 56-bus
network by adding power injection (scaled by the PV generation) at buses
$\mathcal{C}=\\{2,4,7,8,9,10,11,12,13,14,15,16,19,20,23,25,26,32\\}$.
Exogenous active and reactive power injection measurements are taken at each
bus at 6-second intervals over a 24-hour period. Figure 2 plots these values
for several buses to illustrate the setting considered. We assume that
controllers with reactive power injection capacity are available at every
node. The network parameters used in our experiments are:

  * •

nominal squared voltage magnitude at the substation
$v^{0}=v^{\text{nom}}=(12$\mathrm{k}\mathrm{V}$)^{2}$

  * •

squared voltage magnitude limits
$[\underline{v},\overline{v}]=[0.95,1.05]\text{pu}=[11.4^{2},12.6^{2}]$\mathrm{k}\mathrm{V}$^{2}$

  * •

reactive power injection limits
$[\underline{q},\overline{q}]=[-0.24,0.24]\text{MVar}$

  * •

state and input cost matrices $P_{v}=0.1I$, $P_{u}=10I$

  * •

initial state $v(1)=Rp(0)+Xq^{e}(0)+v^{0}\mathbf{1}$, $q^{c}(0)=\mathbf{0}$

In comparison to previous papers in the voltage control literature, our
reactive power injection limits $[\underline{q},\overline{q}]$ are slightly
more generous than $\pm 0.2$ MVar used in, e.g., [20]. We choose $\pm 0.24$
MVar because even a controller with perfect knowledge of the future would need
reactive power injection capabilities of at least $\pm 0.238$ MVar in order to
maintain $v(t)\in[\underline{v},\overline{v}]$ (if
$\underline{q}=-\overline{q}$) under linear dynamics (2).

We set $\overline{\eta}=10$, which upper-bounds the maximum change in
exogenous noise observed in our data, which is $\approx 8.6$:

$\eta^{\star}=\max_{t}\left\lVert
R^{\star}(p(t)-p(t-1))+X^{\star}(q^{e}(t)-q^{e}(t-1))\right\rVert_{\infty}.$

We fix $\epsilon=0.1$. In order to satisfy the requirement in Assumption 3
that
$v(t)\in[\underline{v}+(\overline{\eta}+\epsilon),\overline{v}-(\overline{\eta}+\epsilon)]$,
the reactive power injection capabilities must exceed $\pm 0.528$ MVar. As we
show in experiments with only $\pm 0.24$ MVar range of control, though,
Assumption 3 does not need to be fully satisfied in order for our method to
still provide strong empirical results.

For the robust controller $\Pi$, we set slack variable weight $\beta=100$ and
$\mathcal{V}^{\text{par}}=[\underline{v^{\text{par}}},\overline{v}^{\text{par}}]$
to be a rectangle around the true noise. Under linearized system dynamics,
$v^{\text{par}}(t)$ is calculated as described in Section II, and then we set

$\forall i\in[n]:\
\underline{v^{\text{par}}}_{i}=\min_{t}v^{\text{par}}_{i}(t),\quad\overline{v}^{\text{par}}_{i}=\max_{t}v^{\text{par}}_{i}(t).$
(16)

Under nonlinear system dynamics, we approximate $v^{\text{par}}(t)$ as the
nodal squared voltage magnitudes when $q^{c}(t)=0$ (as shown in Figure 2), and
we add $0.5$\mathrm{k}\mathrm{V}$^{2}$ padding which empirically suffices as a
convex outer approximation of $\mathcal{V}^{\text{par}}$:

$\underline{v^{\text{par}}}_{i}=\min_{t}v^{\text{par}}_{i}(t)-0.5,\quad\overline{v}^{\text{par}}_{i}=\max_{t}v^{\text{par}}_{i}(t)+0.5.$

As mentioned previously, we use a greedy projection-based NCBC algorithm [42]
in SEL that minimizes the movement distance
$\|(\hat{X}_{t},\hat{\eta}_{t})-(\hat{X}_{t-1},\hat{\eta}_{t-1})\|_{\triangle,\delta}$
between nested convex sets $P_{t}\subseteq P_{t-1}$:

$\displaystyle\mathrm{NCBC}_{\text{proj}}(P_{t},\hat{X}_{t-1},\hat{\eta}_{t-1})$
(17) $\displaystyle:=\operatorname*{arg\,min}_{(X,\eta)\in
P_{t}}\|(X,\eta)-(\hat{X}_{t-1},\hat{\eta}_{t-1})\|_{\triangle,\delta}.$

This achieves competitive ratio $\gamma_{\text{proj}}(m)=\pi(m-1)m^{m/2}$.

To keep the optimization problem (8) computationally tractable for consistent
model chasing, our implementation does not use the full trajectory $D$ as in
the constraints of the consistent set (11). Instead, we include the 20 latest
observations and 80 more observations sampled uniformly at random
$(v(t),v(t+1),u(t),q^{c}(t))\sim D$. This provides a computationally tractable
approximation of the uncertainty set. In our experiments on linear system
dynamics, we found that $\hat{X}_{t}$ selected using this approximation was
always in the consistent set defined by the full trajectory $D$, when allowing
for small numerical inaccuracies introduced by the CVXPY optimization solver.

Unless otherwise stated, we initialize $\hat{\eta}_{1}=0$. We initialize
$\hat{X}_{1}$ by adding noise to the true $X^{\star}$ in two ways. First, we
scale each line impedance $x_{ij}$ by a random factor
$\sigma_{ij}\stackrel{{\scriptstyle\mathrm{iid}}}{{\sim}}\text{Uniform}[0,2]$.
Second, we randomly permute the bus ordering, so $\hat{X}_{1}$ corresponds to
a permuted grid topology. Finally, we project $\hat{X}_{1}$ into the
uncertainty set $\mathcal{X}_{\alpha}$, with $\alpha=1$.

Except for the experiments shown in Figure 5, we fix $\delta=20$ which
empirically strikes a balance between minimizing the modeling error
$\|\hat{X}_{t}-X^{\star}\|_{\triangle}$ and overfitting noise.

(a)

(b)

(c)

(d)

(e)

Figure 3: (a)-(d) Voltage profiles of 7 different buses simulated under linear
system dynamics (2). Dotted black lines indicate voltage limits
$[\underline{v},\overline{v}]$. (a) $\Pi$+SEL initialized with random
$\hat{X}\in\mathcal{X}_{\alpha}$. (b) like (a) but the topology for buses 1-14
is known. (c) like (a) but the topology and line parameters for buses 1-14 are
known. (d) like (a) but $\hat{X}=X^{\star}$ is fixed and known so only
$\hat{\eta}$ is learned (e) Convergence of $\hat{X}_{t}$ towards true
$X^{\star}$ (solid lines, left axis) and estimated $\hat{\eta}$ (dotted lines,
right axis). Notice that even when $\|\hat{X}_{t}-X^{\star}\|_{\triangle}$
does not reach 0, the controller still performs quite well.

(a)

(b)

(c)

(d)

(e)

Figure 4: Parallels Figure 3. Voltage profiles of 7 different buses simulated under balanced nonlinear AC power flow (1). TABLE I: Performance of our method simulated under linear system dynamics (top) and nonlinear system dynamics (bottom). See Section V-B for descriptions of information provided. A “mistake” refers to any time step where any bus’ voltage violated the limits $[\underline{v},\overline{v}]$. “Avg. violation” refers to the average absolute squared-voltage violation: $\mathrm{mean}_{i\in[n],t\in[T]:v_{i}(t)\not\in[\underline{v}_{i},\overline{v}_{i}]}\max(v_{i}(t)-\overline{v}_{i},\,\underline{v}_{i}-v_{i}(t)))$. “Max violation” is like “avg. violation” but replaces the $\mathrm{mean}$ with a $\max$. Results given show the mean and std-dev over 4 different initializations of $\hat{X}_{1}$. Info provided | # mistakes | avg. violation | max violation
---|---|---|---
Unknown | 662.2 $\pm$ 435.1 | 0.43 $\pm$ 0.16 | 4.40 $\pm$ 2.59
Topo-14 | 917.0 $\pm$ 155.2 | 0.34 $\pm$ 0.12 | 4.93 $\pm$ 2.19
Lines-14 | 1085.8 $\pm$ 186.6 | 0.57 $\pm$ 0.29 | 2.55 $\pm$ 1.09
Known | 88.0 | 0.07 | 0.12
Unknown | 16.0 $\pm$ 15.8 | 0.68 $\pm$ 0.56 | 2.74 $\pm$ 2.39
Topo-14 | 0.5 $\pm$ 0.6 | 2.21 $\pm$ 2.56 | 2.90 $\pm$ 3.38
Lines-14 | 0.5 $\pm$ 0.6 | 1.01 $\pm$ 1.20 | 1.45 $\pm$ 1.73
Known | 0.0 | 0.00 | 0.00

###  V-B Experimental Results

Our experimental results demonstrate the ability of Algorithm 1 to stabilize
the system without knowledge of the network topology, providing good voltage
control performance even though it still has significant uncertainty about the
topology at the end of the experiments. We test our algorithm under both the
linearized system dynamics (5) as well as the more realistic nonlinear
balanced AC power flow setting (1) simulated using Pandapower [51]. The convex
optimization problems for SEL and $\Pi$ are solved with CVXPY [52] using the
MOSEK solver [53]. Code for our simulations are available on
GitHub.111https://github.com/chrisyeh96/voltctrl

(a)

(b)

Figure 5: Effect of varying $\delta$ on consistent model chasing. As in Figure
3(e), convergence of $\hat{X}_{t}$ towards $X^{\star}$ is plotted in solid
lines (left axis), and estimated $\hat{\eta}$ is plotted in dotted lines
(right axis). In blue are results where we fix $\hat{\eta}=\eta^{\star}=8.65$
and $\delta$ has no effect. (a) linear dynamics (b) nonlinear dynamics.

(a)

(b)

Figure 6: Balanced nonlinear AC power flow simulation of the voltage profiles
under different algorithms with partial control and observation. The dark
colors plot the mean voltages across 4 random initializations of $\hat{X}_{1}$
and the light shading plots $\pm 1$ standard deviation. (a) bus 18 (b) bus 30.

#### Linearized power flow with full control

Our first set of experiments, shown in Figure 3 and Table I (top), tests our
algorithm’s performance on the SCE-56 bus network under linearized system
dynamics (5). Different amounts of network information are provided to the
consistent model chasing algorithm SEL via the initial consistent set
$\mathcal{X}_{\alpha}$, ranging from no information (“unknown,” Figure 3(a)),
information about the edges among the first 14 buses but not the line
impedances (“topo-14,” Figure 3(b)), information about the edges and line
impedances among the first 14 buses (“lines-14,” Figure 3(c)), and complete
information about the network (“known,” Figure 3(d)). Because the buses in the
SCE 56-bus network are numbered in a topological ordering, the “topo-14”
setting adds constraints of the form (7) for all of the first 14 buses, and
the “lines-14” setting constrains all $X\in\mathcal{X}_{\alpha}$ such that
$X_{ij}=X^{\star}_{ij}$ for all $i,j\in\\{1,\dotsc,14\\}$.

As shown in Figure 3(e), incorporating more prior knowledge about the network
into the initial uncertainty set reduces the model estimation error
$\|\hat{X}-X^{\star}\|_{\triangle}$. Furthermore, the model estimation error
decreases the most dramatically when the voltage violations are the largest.
However, we note that lower model estimation error does not always result in
fewer mistakes in our experiments.

#### Nonlinear power flow with full control

Our second set of experiments test our online controller on the standard
balanced AC power flow model (1). As in the linearized power flow experiments,
we compare Algorithm 1’s performance across varying levels of prior
information (Figure 4 and Table I, bottom). Even though the controller is
designed under the assumption of linearized voltage dynamics, our algorithm
still performs well in the nonlinear simulation. The performance improves
progressively, with less voltage violation and smaller overall deviation from
the desired steady state voltage as it is provided more information.

#### Nonlinear power flow with partial observation and partial control

We also test our proposed online controller in the partial observation and
partial control setting. In Figure 6, we withhold voltage observations and
control authority from buses $i\in\\{8,18,21,30,39,45,54\\}$ by setting
$q^{c}_{i}(t)=0$ for all $t$. We simulate the voltage profiles across 4 random
initializations of $\hat{X}_{1}$ and plot the mean and $\pm 1$ standard
deviation. Despite the more challenging setting, the performance of Algorithm
1 remains strong. We again observe in Figure 6 that adding prior topology and
line parameter information marginally improves the performance of Algorithm 1.

#### Varying $\delta$

In Figure 5, we demonstrate the effect of varying $\delta$ on the performance
of our algorithm. From a theoretical perspective, Theorem 1 shows that our
algorithm achieves a finite mistake bound for every $\delta>0$, and this bound
is minimized by taking $\delta$ to be very large. What happens when using a
large $\delta$, though, is that the model chasing algorithm may overfit to
noise until a time when the noise is too large, forcing the algorithm to
increase the noise bound (e.g., around the 16h mark in Figure 5). This leads
to inconsistent performance in the short term, albeit with perhaps better
worst-case performance. In contrast, a smaller $\delta$ allows more of the
network uncertainty to be captured in a larger noise $\hat{\eta}$ term at the
cost of learning a less accurate $\hat{X}$, but the decrease in modeling error
$\|\hat{X}_{t}-X^{\star}\|_{\triangle}$ becomes monotonic.

In practice, $\delta$ should be treated as a prior “confidence” about how
close the initial guess of $\hat{\eta}$ is to $\eta^{\star}$. $\delta$ should
be larger when there is greater confidence that $\hat{\eta}$ is close to the
true $\eta^{\star}$.

#### Detecting topology changes

Finally, we consider the challenge of responding to a change in the
distribution grid topology in real-time. If the topology changes from one
radial grid to another due to switches, new observed data may render the
consistent set empty. That is, when consistent model chasing (17) becomes
infeasible, we are assured that the topology has changed. At this point, we
may reset the algorithm by discarding the observed trajectory $D_{t}$ and
reinitializing consistent parameter estimates from the original consistent set
$P_{1}$. Figure 7 demonstrates this on linear system dynamics, where we
introduce a topology change at the 12h mark. We replace lines $33\to 40$ and
$46\to 48$ with new lines $1\to 40$ and $10\to 48$, which maintains a radial
distribution grid.

Figure 7: Demonstration of the detection of a topology change under linear
system dynamics. Convergence of $\hat{X}_{t}$ towards $X^{\star}$ is plotted
in solid lines (left axis), where $X^{\star}$ changes at the 12h mark. The
topology change triggers a reset of the consistent model chasing algorithm.
Estimated $\hat{\eta}$ is plotted in dotted lines (right axis).

##  VI Conclusion

This paper provides the first controller that establishes a finite-mistake
guarantee for voltage control in a setting with uncertainty in both the grid
topology and load and generation variations. We showed that our proposed
algorithm is able to learn a model of the grid dynamics in an online fashion
and provably (under linearized voltage dynamics) converge to a stable
controller. Further, simulated experiments on a 56-bus distribution grid
demonstrate the effectiveness of our algorithm even under more realistic
nonlinear dynamics. We demonstrated how to incorporate prior knowledge about
the network topology and line parameters to improve performance, while also
extending our algorithm to the partial observability and partial
controllability setting which may better reflect real-world scenarios.

As the current algorithm is centralized, future works may consider
decentralized approaches to topology-robust voltage control in order to enable
faster real-time control with ideas from [54]. Another direction is to extend
the current algorithm to the time-varying topology setting with techniques
from works such as [55]. Further studies may also explore loosening the radial
topology assumption and test our algorithm on unbalanced 3-phase AC grids to
accommodate a wider range of distribution grids. This would be a challenging,
but important, extension. Finally, an interesting algorithmic extension is to
consider computationally efficient convex body chasing algorithms with better
competitive ratios. Existing methods based on Steiner point [42, 49] achieve
nearly-optimal competitive ratio but are computationally inefficient in high
dimension settings such as voltage control, so designing efficient approximate
Steiner point algorithms could potentially lead to significant performance
improvements.

## Acknowledgments

We thank Dimitar Ho for useful conversations. This work was supported by the
Caltech Resnick Sustainability Institute, two Caltech/Amazon AWS AI4Science
fellowships, and National Science Foundation grants CNS-2146814, CNS-2106403,
CPS-2136197, ECCS-2200692, and NGSDI-2105648.

## References

  * [1] “Global Survey of Regulatory Approaches for Power Quality and Reliability,” EPRI, Palo Alto, CA, USA, Tech. Rep. 1008589, Mar. 2005.
  * [2] H. Haes Alhelou, M. E. Hamedani-Golshan, T. C. Njenda, and P. Siano, “A survey on power system blackout and cascading events: Research motivations and challenges,” _Energies_ , vol. 12, no. 4, 2019.
  * [3] Y. G. Rebours, D. S. Kirschen, M. Trotignon, and S. Rossignol, “A survey of frequency and voltage control ancillary services—part i: Technical features,” _IEEE Trans. Power Syst._ , vol. 22, no. 1, 2007.
  * [4] H. Zhu and H. J. Liu, “Fast local voltage control under limited reactive power: Optimality and stability analysis,” _IEEE Trans. Power Syst._ , vol. 31, no. 5, 2015.
  * [5] D. K. Molzahn, F. Dörfler, H. Sandberg, S. H. Low, S. Chakrabarti, R. Baldick, and J. Lavaei, “A survey of distributed optimization and control algorithms for electric power systems,” _IEEE Trans. Smart Grid_ , vol. 8, no. 6, 2017.
  * [6] T. Senjyu, Y. Miyazato, A. Yona, N. Urasaki, and T. Funabashi, “Optimal distribution voltage control and coordination with distributed generation,” _IEEE Trans. Power Del._ , vol. 23, no. 2, 2008.
  * [7] C. Gao and M. A. Redfern, “A review of voltage control techniques of networks with distributed generations using on-load tap changer transformers,” in _45th International Universities Power Engineering Conference UPEC2010_. IEEE, 2010.
  * [8] Y. Guo, Q. Wu, H. Gao, X. Chen, J. Østergaard, and H. Xin, “MPC-Based Coordinated Voltage Regulation for Distribution Networks With Distributed Generation and Energy Storage System,” _IEEE Trans. Sustainable Energy_ , vol. 10, no. 4, Oct. 2019.
  * [9] S. Maharjan, A. M. Khambadkone, and J. C. H. Peng, “Robust constrained model predictive voltage control in active distribution networks,” _IEEE Trans. Sustainable Energy_ , vol. 12, no. 1, Jun. 2021.
  * [10] S. Park, D. Deka, and M. Chcrtkov, “Exact topology and parameter estimation in distribution grids with minimal observability,” in _2018 Power Syst. Comput. Conf._ IEEE, 2018.
  * [11] H. Li, Y. Weng, Y. Liao, B. Keel, and K. E. Brown, “Robust Hidden Topology Identification in Distribution Systems,” _arXiv preprint arXiv:1902.01365_ , Feb. 2019.
  * [12] P. Dehghanian and M. Kezunovic, “Probabilistic impact of transmission line switching on power system operating states,” in _2016 IEEE PES Transmiss. Distrib. Conf. Exhib._ IEEE, 2016, pp. 1–5.
  * [13] Z. Li, S. Jazebi, and F. De Leon, “Determination of the optimal switching frequency for distribution system reconfiguration,” _IEEE Trans. Power Del._ , vol. 32, no. 4, pp. 2060–2069, 2016.
  * [14] D. Deka, S. Backhaus, and M. Chertkov, “Structure learning in power distribution networks,” _IEEE Trans. Control Netw. Syst._ , vol. 5, no. 3, pp. 1061–1074, Sep. 2018.
  * [15] S. Esmaeili, A. Anvari-Moghaddam, S. Jadid, and J. M. Guerrero, “Optimal simultaneous day-ahead scheduling and hourly reconfiguration of distribution systems considering responsive loads,” _International Journal of Electrical Power & Energy Systems_, vol. 104, 2019.
  * [16] Y. Liao, Y. Weng, M. Wu, and R. Rajagopal, “Distribution grid topology reconstruction: An information theoretic approach,” in _2015 North American Power Symposium (NAPS)_. IEEE, Oct. 2015.
  * [17] S. Bolognani, R. Carli, G. Cavraro, and S. Zampieri, “Distributed reactive power feedback control for voltage regulation and loss minimization,” _IEEE Trans. Autom. Control_ , vol. 60, no. 4, Apr. 2015.
  * [18] N. Li, G. Qu, and M. Dahleh, “Real-time decentralized voltage control in distribution networks,” in _Proc. 52nd Annu. Allerton Conf. Commun. Control Comput._ IEEE, Sep. 2014.
  * [19] Z. Tang, D. J. Hill, and T. Liu, “Fast distributed reactive power control for voltage regulation in distribution networks,” _IEEE Trans. Power Syst._ , vol. 34, no. 1, 2019.
  * [20] G. Qu and N. Li, “Optimal distributed feedback voltage control under limited reactive power,” _IEEE Trans. Power Syst._ , vol. 35, no. 1, Jan. 2020.
  * [21] J. Duan, D. Shi, R. Diao, H. Li, Z. Wang, B. Zhang, D. Bian, and Z. Yi, “Deep-reinforcement-learning-based autonomous voltage control for power grid operations,” _IEEE Trans. Power Syst._ , vol. 35, no. 1, Jan. 2020.
  * [22] S. Wang, J. Duan, D. Shi, C. Xu, H. Li, R. Diao, and Z. Wang, “A data-driven multi-agent autonomous voltage control framework using deep reinforcement learning,” _IEEE Trans. Power Syst._ , vol. 35, no. 6, 2020.
  * [23] H. Xu, A. D. Domínguez-García, and P. W. Sauer, “Optimal tap setting of voltage regulation transformers using batch reinforcement learning,” _IEEE Trans. Power Syst._ , vol. 35, no. 3, May 2020.
  * [24] Y. Gao, W. Wang, and N. Yu, “Consensus multi-agent reinforcement learning for volt-var control in power distribution networks,” _IEEE Trans. Smart Grid_ , vol. 12, no. 4, 2021.
  * [25] X. Sun and J. Qiu, “Two-stage volt/var control in active distribution networks with multi-agent deep reinforcement learning method,” _IEEE Trans. Smart Grid_ , vol. 12, no. 4, 2021.
  * [26] Y. Shi, G. Qu, S. Low, A. Anandkumar, and A. Wierman, “Stability constrained reinforcement learning for real-time voltage control,” in _2022 American Control Conference (ACC)_ , 2022, pp. 2715–2721.
  * [27] J. Feng, Y. Shi, G. Qu, S. H. Low, A. Anandkumar, and A. Wierman, “Stability constrained reinforcement learning for real-time voltage control in distribution systems,” _arXiv preprint arXiv:2209.07669_ , 2022.
  * [28] W. Cui, J. Li, and B. Zhang, “Decentralized safe reinforcement learning for inverter-based voltage control,” _Electr. Power Syst. Res._ , vol. 211, 2022\.
  * [29] V. Kekatos, G. B. Giannakis, and R. Baldick, “Grid topology identification using electricity prices,” in _2014 IEEE Power Eng. Soc. Gen. Meet._ IEEE, 2014.
  * [30] E. Fabbiani, P. Nahata, G. De Nicolao, and G. Ferrari-Trecate, “Identification of ac distribution networks with recursive least squares and optimal design of experiment,” _IEEE Trans. Control Syst. Technol._ , vol. 30, no. 4, pp. 1750–1757, 2021.
  * [31] J.-S. Brouillon, E. Fabbiani, P. Nahata, K. Moffat, F. Dörfler, and G. Ferrari-Trecate, “Bayesian error-in-variables models for the identification of distribution grids,” _IEEE Trans. Smart Grid_ , vol. 14, no. 2, pp. 1289–1299, 2022.
  * [32] G. Cavraro and V. Kekatos, “Graph algorithms for topology identification using power grid probing,” _IEEE Control Syst. Lett._ , vol. 2, no. 4, pp. 689–694, 2018.
  * [33] O. Ardakanian, V. W. Wong, R. Dobbe, S. H. Low, A. von Meier, C. J. Tomlin, and Y. Yuan, “On identification of distribution grids,” _IEEE Trans. on Control of Network Systems_ , vol. 6, no. 3, pp. 950–960, 2019.
  * [34] H. Xu, A. D. Domínguez-García, V. V. Veeravalli, and P. W. Sauer, “Data-driven voltage regulation in radial power distribution systems,” _IEEE Trans. Power Syst._ , vol. 35, no. 3, pp. 2133–2143, 2019.
  * [35] S. Nowak, Y. C. Chen, and L. Wang, “Measurement-based optimal der dispatch with a recursively estimated sensitivity model,” _IEEE Trans. Power Systems_ , vol. 35, no. 6, pp. 4792–4802, 2020.
  * [36] D. Deka, V. Kekatos, and G. Cavraro, “Learning distribution grid topologies: A tutorial,” _IEEE Trans. Smart Grid_ , 2023.
  * [37] Y. Chen, Y. Shi, and B. Zhang, “Data-driven optimal voltage regulation using input convex neural networks,” _Electr. Power Syst. Res._ , vol. 189, Dec. 2020.
  * [38] J.-S. Brouillon, K. Moffat, F. Dörfler, and G. Ferrari-Trecate, “Robust online joint state/input/parameter estimation of linear systems,” in _2022 IEEE 61st Conference on Decision and Control (CDC)_. IEEE, 2022, pp. 2153–2158.
  * [39] J. Yu, Y. Weng, and R. Rajagopal, “Patopa: A data-driven parameter and topology joint estimation framework in distribution grids,” _IEEE Trans. Power Syst._ , vol. 33, no. 4, pp. 4335–4347, 2017.
  * [40] D. Deka, S. Backhaus, and M. Chertkov, “Estimating distribution grid topologies: A graphical learning based approach,” in _2016 Power Syst. Comput. Conf._ Genoa, Italy: IEEE, Jun. 2016.
  * [41] Y. Sharon, A. M. Annaswamy, A. L. Motto, and A. Chakraborty, “Topology identification in distribution network with limited measurements,” in _2012 IEEE PES Innovative Smart Grid Technologies (ISGT)_ , 2012.
  * [42] C. Argue, S. Bubeck, M. B. Cohen, A. Gupta, and Y. T. Lee, “A nearly-linear bound for chasing nested convex bodies,” in _Proc. 30th Annu. ACM-SIAM Symposium on Discrete Algorithms_ , 2019.
  * [43] D. Ho, H. Le, J. Doyle, and Y. Yue, “Online robust control of nonlinear systems with large uncertainty,” in _Proc. 24th Int. Conf. Artif. Intell. Stat. (AISTATS)_ , vol. 130. PMLR, Apr. 2021.
  * [44] C. Yeh, J. Yu, Y. Shi, and A. Wierman, “Robust online voltage control with an unknown grid topology,” in _Proc. 13th ACM Int. Conf. Future Energy Syst._ , 2022, pp. 240–250.
  * [45] M. Farivar, R. Neal, C. Clarke, and S. Low, “Optimal inverter VAR control in distribution systems with high PV penetration,” in _2012 IEEE Power Eng. Soc. Gen. Meet._ USA: IEEE, 2012.
  * [46] M. E. Baran and F. F. Wu, “Optimal capacitor placement on radial distribution systems,” _IEEE Trans. Power Del._ , vol. 4, no. 1, 1989.
  * [47] ——, “Optimal sizing of capacitors placed on a radial distribution system,” _IEEE Trans. Power Del._ , vol. 4, no. 1, 1989.
  * [48] M. Farivar, L. Chen, and S. Low, “Equilibrium and dynamics of local voltage control in distribution systems,” in _Proc. IEEE Conf. Decision Control_. USA: IEEE, Dec. 2013.
  * [49] S. Bubeck, B. Klartag, Y. T. Lee, Y. Li, and M. Sellke, “Chasing nested convex bodies nearly optimally,” in _Proc. 31st Annu. ACM-SIAM Symposium on Discrete Algorithms_. USA: SIAM, 2020.
  * [50] A. Bernstein and E. Dall’Anese, “Real-time feedback-based optimization of distribution grids: A unified approach,” _IEEE Trans. on Control of Network Systems_ , vol. 6, no. 3, pp. 1197–1209, 2019.
  * [51] L. Thurner, A. Scheidler, F. Schäfer, J. Menke, J. Dollichon, F. Meier, S. Meinecke, and M. Braun, “pandapower — An Open-Source Python Tool for Convenient Modeling, Analysis, and Optimization of Electric Power Systems,” _IEEE Trans. Power Syst._ , vol. 33, no. 6, Nov 2018.
  * [52] S. Diamond and S. Boyd, “CVXPY: A Python-embedded modeling language for convex optimization,” _JMLR_ , vol. 17, no. 83, 2016.
  * [53] M. ApS, _MOSEK Optimizer API for Python 10.0.46_ , 2023. [Online]. Available: https://docs.mosek.com/10.0/pythonapi/index.html
  * [54] J. Yu, D. Ho, and A. Wierman, “Online adversarial stabilization of unknown networked systems,” _Proceedings of the ACM on Measurement and Analysis of Computing Systems_ , vol. 7, no. 1, 2023.
  * [55] J. Yu, V. Gupta, and A. Wierman, “Online adversarial stabilization of unknown linear time-varying systems,” in _2023 62nd IEEE Conference on Decision and Control (CDC)_. IEEE, 2023, pp. 8320–8327.

|  Christopher Yeh (Student Member, IEEE) is a PhD candidate in Computing and
Mathematical Sciences at Caltech, advised by Yisong Yue and Adam Wierman. He
received B.S. and M.S. degrees in Computer Science from Stanford University in
2019, and a M.M.S. degree in Global Affairs from Schwarzman College, Tsinghua
University, in 2020. His research interests lie at the intersection of machine
learning and environmental sustainability, with a focus on uncertainty
quantification for decision making. He is supported by a Amazon/Caltech AWS
AI4Science Fellowship and is recognized as a Caltech Resnick Sustainability
Institute Scholar.
---|---
|  Jing Yu (Student Member, IEEE) is a PhD candidate in Control and Dynamical
Systems at Caltech, advised by John Doyle and Adam Wierman. She is broadly
interested in the interplay between control theory and machine learning, with
a focus on online decision making and distributed algorithms for large-scale
sustainable energy systems. She is the recipient of the Amazon/Caltech
AI4Science Fellowship and a best paper finalist award for ACM e-energy.
---|---
|  Yuanyuan Shi (Member, IEEE) is an Assistant Professor of Electrical and
Computer Engineering at the University of California, San Diego. She received
her Ph.D. in Electrical Engineering, masters in Electrical Engineering and
Statistics, all from the University of Washington, Seattle, in 2020. From 2020
to 2021, she was a Postdoctoral Scholar with the Department of Computing and
Mathematical Sciences, Caltech. Her research interests include machine
learning, dynamical systems, and control, with applications to sustainable
power and energy systems.
---|---
|  Adam Wierman (Senior Member, IEEE) is the Carl F Braun Professor in the
Department of Computing and Mathematical Sciences at Caltech. He received his
Ph.D., M.Sc., and B.Sc. in Computer Science from Carnegie Mellon University.
Adam’s research strives to make the networked systems that govern our world
sustainable and resilient. He is best known for his work spearheading the
design of algorithms for sustainable data centers and his work on heavy-tails,
including co-authoring a book on “The Fundamentals of Heavy Tails”.
---|---