# Power Grid Parameter Estimation Without Phase Measurements: Theory and
Empirical Validation

Jean-Sébastien Brouillon
Giancarlo Ferrari Trecate  Institute of Mechanical Engineering
Ecole Polytechnique Fédérale de Lausanne
Lausanne, Switzerland
{jean-sebastien.brouillon<EMAIL_ADDRESS>Keith Moffat
Florian Dörfler  Department of Information Technology and Electrical
Engineering
ETH Zürich
Zürich, Switzerland
{kmoffat<EMAIL_ADDRESS>
###### Abstract

Reliable integration and operation of renewable distributed energy resources
requires accurate distribution grid models. However, obtaining precise models
is often prohibitively expensive, given their large scale and the ongoing
nature of grid operations. To address this challenge, considerable efforts
have been devoted to harnessing abundant consumption data for automatic model
inference. The primary result of the paper is that, while the impedance of a
line or a network can be estimated without synchronized phase angle
measurements in a consistent way, the admittance cannot. Furthermore, a
detailed statistical analysis is presented, quantifying the expected
estimation errors of four prevalent admittance estimation methods. Such errors
constitute fundamental model inference limitations that cannot be resolved
with more data. These findings are empirically validated using synthetic data
and real measurements from the town of Walenstadt, Switzerland, confirming the
theory. The results contribute to our understanding of grid estimation
limitations and uncertainties, offering guidance for both practitioners and
researchers in the pursuit of more reliable and cost-effective solutions.

###### Index Terms:

Distribution Grid, Parameter estimation, Smart meters, Network identification

This research is supported by the Swiss National Science Foundation under the
NCCR Automation (grant agreement 51NF40_180545).

##  I Introduction

The deployment of sensors and machine learning techniques on power grids has
opened a new set of power system monitoring applications. Among them, line
parameter and topology estimation may play a crucial role for deploying smart
energy resources [1, 2, 3, 4, 5, 6, 7]. This is specially needed at the
distribution level, as Distribution System Operators (DSOs) often lack
accurate models of their Distribution Grids (DGs)111We focus on distribution
network applications in this paper, however the analysis applies to
transmission networks as well.. Many recent studies have shown that physical
models of a distribution grid can be estimated from synchronized voltage and
current measurements [8, 9, 10, 11, 12]. However, micro synchrophasor
measurement units ($\mu$PMUs) remain expensive and therefore mostly absent in
DGs.

Sensor costs motivate grid analysis that does not rely on accurate phase angle
measurements, as voltage and current magnitude sensors are considerably
cheaper than $\mu$PMU sensors. Smart meter sensors [13] are becoming
ubiquitous for measuring power consumption. The load—coverage in European DGs
is already above 70% [14]. Thus, the question whether it is possible to
accurately estimate grid impedance or admittance without synchronized phase
angle measurements is of practical importance. This paper investigates the
feasibility of impedance and admittance estimation from voltage and current
magnitude and power angle measurements only.

Given that phase angles are typically small in DGs, one might consider
applying the methods used for synchronized measurements with a zero angle.
However, recent research demonstrates that such an approach results in an
inconsistent estimate [15], even when employing Error-in-Variables (EIV)
methods, e.g., the Total Least Squares (TLS). Consistency is critical when
estimating DG parameters because the stability of the voltage operating point
leads to a low Signal-to-Noise Ratio (SNR), which can only be mitigated by
collecting a large volume of data.

Prior research has improved the accuracy of power system parameter estimation
from smart meter measurements. The authors of [16] attempted to simultaneously
estimating phase angles and parameters, which reduced the estimation error in
some cases but did not provide consistency guarantees. A similar approach is
employed in [17], where $\mu$PMUs are added at some nodes to mitigate
inconsistencies, albeit not eliminating them entirely. A more recent
development [18] focuses on canceling the phase variable out of the current
flow equations. This reduction yields consistent estimates of X/R ratios and
certain transformations of conductance and susceptance, but not of these
quantities themselves. Alternatively, some studies such as [19] have chosen to
estimate impedance rather than admittance, but inverting the impedance
estimates also results in accuracy issues [12].

In this paper, we aim to enhance our understanding of existing estimation
methods by quantifying their inherent biases, i.e., their expected estimation
errors. Our contribution can be summarized in three main aspects. We first
present a detailed statistical model for common DG sensors and utilize it to
express the biases associated with each estimation method in a single-line
identification setup. Notably, we reveal that while each method effectively
addresses certain biases from previous approaches, no admittance estimation
method achieves a perfect bias cancellation, for which synchronized phasor
measurements are required. Second, we show how the same reasoning can be
applied to admittance matrix estimation problems when the topology is unknown.
Third, to validate our findings and underscore the practical challenges of the
problem, we conduct an experiment using real DG data, thereby demonstrating
the applicability of the methods in the real world.

###  I-A Preliminaries and Notation

Complex numbers are defined as $z=a+jb$, where $a,b\in\mymathbb{R}^{2}$ and
$j^{2}=-1$. The complex conjugate of $z\in\mymathbb{C}$ is denoted by
$z^{\star}$. The pseudo-inverse $A^{\dagger}$ of a matrix
$A\in\mymathbb{C}^{m\times n}$ is obtained by inverting its non-zero singular
values, i.e., if $A=U\textnormal{diag}([s,0,\dots,0])(V^{\star})^{\top}$ then
$A^{\dagger}=U\textnormal{diag}([1/s,0,\dots,0])(V^{\star})^{\top}$. If $A$ is
square and invertible, then $A^{\dagger}$ is equal to $A^{-1}$ the inverse of
$A$. The noisy measurement of a variable $x$ is denoted by $\tilde{x}$. In a
regression model $\tilde{z}=A\tilde{x}+\epsilon$, we call $\tilde{x}$ and
$\tilde{z}$ the Right-Hand Side (RHS) and Left-Hand Side (LHS) variables,
respectively. All variables have an implicit time-dependence, except the
parameters of the network and probability distributions. This dependence is
implicit. The stacked vector of $N+1$ measurements of a variable $x$ is
denoted by $[\tilde{x}]_{t=t_{0}}^{t_{N}}$

##  II System model

###  II-A Distribution Network

We model the a power grid as a graph $\mathcal{G}(\mathcal{V},\mathcal{E})$
with $n$ nodes $\mathcal{V}=\\{1,\dots,n\\}$ and edges
$\mathcal{E}\subseteq\mathcal{V}\times\mathcal{V}$. Each node $h=1,\dots,n$ is
injecting a current $i_{h}$ at a voltage $v_{h}$. Consumer nodes are modeled
with a negative injection. Nodal AC voltages can be expressed as phasors
$v_{h}=|v_{h}|e^{j\theta_{h}}$, where $\theta_{h}$ is the phase difference
with node $1$. The nodal power angle $\phi_{h}$ is the phase difference
between the current injection $i_{h}$ and the voltage $v_{h}$ (i.e., the arc
cosine of the power factor). Additionally, we define all the quantities for
each line $h\rightarrow k$ using a double subscript notation ”$hk$”. Thus, the
voltage drop, current flow, and power angle of a line from $h$ to $k$ are
$v_{hk}$, $i_{hk}$, and $\phi_{hk}$, respectively. For convenience, we define
$\phi_{hk}$ as the angle between $i_{hk}$ and $v_{h}$.

######  Assumption 1.

The voltage phase angle differences are small, i.e., $|\theta_{h}|\ll 1$.

Comparatively small amounts of power is transmitted on DGs, which are also
quite resistive. This means that Assumption 1 is quite mild. Similar to the
voltages, nodal currents are also phasors
$i_{h}=|i_{h}|e^{j\theta_{h}-j\phi_{h}}$, where the nodal power angle
$\phi_{h}$ is not small in general.

We use a lumped-$\pi$ circuit to model each electrical connection in
$\mathcal{E}$ [20], where the lines are modeled as an inductor and a resistor
in series, and the shunts as capacitors. This gives the line and shunt
admittances $y_{hk}=g_{hk}+jb_{hk}$ and $y_{hh,s}$, respectively. If two nodes
$h$ and $k$ are not connected, we use $y_{hk}=0$. We assume that there are no
phase-shifting transformers, and that the existing transformers have a
constant ratio over the duration of the experiment. This means that the
voltages and currents can be re-scaled to model the transformers as simple
line admittances.

All parameters are collected in the admittance matrix $Y=G+jB$ with $G$ and
$B$ in $\mymathbb{R}^{n\times n}$. $Y$ is defined by its non-diagonal elements
$Y_{hk}=y_{hk},\forall h,k\in\mathcal{V}^{2}$ and its diagonal ones
$Y_{kk}=-\sum_{k=1}^{n}y_{hk}+y_{hh,s},\forall h\in\mathcal{V}$. For
convenience, we also define the X/R ratio $\rho_{hk}=-\frac{b_{hk}}{g_{hk}}$
of each line222Inverting a complex number does not invert the ratio between
its real and imaginary parts..

###  II-B Measuring Devices

Smart meters provide the measurements $|\tilde{v}_{h}|$ and $|\tilde{i}_{h}|$
of the amplitudes $|v_{h}|$ and $|i_{h}|$ of the voltage and current and the
measurement $\tilde{\phi}_{h}$ of the power angle $\phi_{h}$. Moreover, such
sensors can provide the line flow measurements $\tilde{i}_{hk}$ and
$\tilde{\phi}_{hk}$ if they are placed on a specific line rather than a node.

######  Assumption 2.

The noise on the current magnitude, voltage magnitude, and power angle
measurements is independent, Gaussian, and centered on zero.

Although the regulations of commercial smart meters require a given maximum
admissible error, which would imply that the noise follow a truncated Gaussian
[21], the resulting interval is usually large enough to approximate the noise
distribution as Gaussian.

######  Assumption 3.

The error on power angle measurement is small.

With Assumption 3, the noise on the current phasor is almost perfectly
Gaussian in Cartesian coordinates. However, the real and imaginary parts of
the noise on current measurements are not independent if the power angle is
non-zero.

######  Remark 1.

The active and reactive powers are not a linear combination of the measured
quantities and are subject to the dependence between the real and imaginary
parts of the current noise. Their uncertainty is therefore neither Gaussian-
distributed nor independent, which is a common assumption in the literature.

For clarity, we define the noise on the difference of voltage magnitudes
between two nodes and its distribution as

$\displaystyle\delta_{hk}^{v}:=(|\tilde{v}_{h}|-|\tilde{v}_{k}|)-(|v_{h}|-|v_{k}|)\sim\mathcal{N}(0,\sigma_{hk}^{v}).$
(1a) We chose the lower-case symbol $\sigma_{hk}^{v}$ for the variance to
emphasize the fact that it is scalar. We also define the noise on the line
current flow $|\tilde{i}_{hk}|e^{-j\tilde{\phi}_{hk}}$ measured at node $h$ as
$\displaystyle\\!\left[\begin{matrix}\delta_{hk}^{\Re}\\\
\delta_{hk}^{\Im}\end{matrix}\right]\\!\\!:=\\!\\!\left[\begin{matrix}|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})\\!-\\!|i_{hk}|\cos(\phi_{hk})\\\
|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk})\\!-\\!|i_{hk}|\sin(\phi_{hk})\end{matrix}\right]\\!\sim\mathcal{N}(0,\Sigma_{hk}^{i}),\\!$
(1b)

where $\Sigma_{hk}^{i}$ is a $2$-by-$2$ symmetric matrix containing the
variances $\sigma_{hk}^{\Re}=\textnormal{var}[\delta_{hk}^{\Re}]$ and
$\sigma_{hk}^{\Im}=\textnormal{var}[\delta_{hk}^{\Im}]$ of both parts of the
noise on $i_{hk}$ in the diagonal elements, and their covariance
$\textnormal{cov}[\delta_{hk}^{\Re},\delta_{hk}^{\Im}]$ in the off-diagonal
elements. Finally, we consider the the final measurements are synchronized
block-averages of the instantaneous readings, and that the average over the
estimation window is subtracted from each block to center the data.

##  III Line parameter estimation

We first study the estimation problem for a single line before addressing
network estimation in Section VI.

###  III-A Impedance Estimation

The parameters $z_{hk}=y_{hk}^{-1}$ of a line $h\rightarrow k$ relate the
current flow $i_{hk}$ to the difference of voltages $v_{h}-v_{k}$ at each end
of the line according to the following relationship

$\displaystyle|v_{h}|e^{j\theta_{h}}-|v_{k}|e^{j\theta_{k}}$
$\displaystyle=z_{hk}|i_{hk}|e^{j(\theta_{h}-\phi_{hk})},$ (2)

which can be written in real numbers and linearized using
$e^{-j\theta_{hk}}\approx 1-j\theta_{hk}$ from Assumption 1 as

$\displaystyle|v_{h}|-|v_{k}|$
$\displaystyle=r_{hk}|i_{hk}|\cos(\phi_{hk})+x_{hk}|i_{hk}|\sin(\phi_{hk}),$
(3a) $\displaystyle|v_{k}|\theta_{hk}$
$\displaystyle=-r_{hk}|i_{hk}|\sin(\phi_{hk})+x_{hk}|i_{hk}|\cos(\phi_{hk}).$
(3b)

As shown in [15], (3) is a very close approximation of (2) when the currents
and voltages are exact.

The phase $\theta_{hk}$ is not measured so one must use (3a) to fit the
parameters $r_{hk}$ and $x_{hk}$. With noisy measurements, the regression is
performed on the model

$\displaystyle\\!\\!|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|$
$\displaystyle=r_{hk}|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})+x_{hk}|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk})+\\!\epsilon_{hk},\\!$
(4)

where
$\epsilon_{hk}=\delta_{hk}^{v}-r_{hk}\delta_{hk}^{\Re}-x_{hk}\delta_{hk}^{\Im}$
embeds the uncertainty.

######  Remark 2.

Under Assumption 1, (3b) $\approx 0$ so one can also use (3b) as a regularizer
with strength $\lambda$, by adding $-\lambda
r_{hk}|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk})\\!+\\!\lambda
x_{hk}|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})\approx 0$ to the regression.
This can be done by augmenting the voltage and current data matrices with
$[0]_{t=t_{0}}^{t_{N}}$ and
$[-\lambda|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk}),\lambda|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})]_{t=t_{0}}^{t_{N}}$,
respectively.

###  III-B Admittance Estimation

Similar to the impedance, the admittance relates the current flow $i_{hk}$ to
the difference of voltages $v_{h}-v_{k}$ at each end of the line, but in the
inverse way. This gives following relationship

$\displaystyle|i_{hk}|e^{-j\phi_{hk}}$
$\displaystyle=y_{hk}\left(|v_{h}|-|v_{k}|e^{-j\theta_{hk}}\right).$ (5)

As for (3), using the truncated expansion $e^{-j\theta_{hk}}\approx
1-j\theta_{hk}$ yields

$\displaystyle|i_{hk}|\cos(\phi_{hk})$
$\displaystyle=g_{hk}(|v_{h}|-|v_{k}|)-b_{hk}|v_{k}|\theta_{hk},$ (6a)
$\displaystyle|i_{hk}|\sin(\phi_{hk})$
$\displaystyle=-g_{hk}|v_{k}|\theta_{hk}-b_{hk}(|v_{h}|-|v_{k}|).$ (6b)

One can observe that the unobserved phase angle $\theta_{hk}$ is not simple
from (6) as it is done in (3). Assumption 1 hints that one could handle the
missing phase measurements by replacing them with zero. However, the phase
often has a key role in power transmission in practice so assuming it to be
exactly zero is often too constraining. Nevertheless, for very resistive
grids, inferring $g_{hk}$ and $b_{hk}$ in the following regression model
sometimes yields good results.

$\displaystyle\\!\\!\\!|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})\\!$
$\displaystyle=g_{hk}(|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|)+\mu_{hk}\\!-\\!b_{hk}|v_{k}|\theta_{hk},\\!$
(7a) $\displaystyle\\!\\!\\!|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk})\\!$
$\displaystyle=-b_{hk}(|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|)\underbrace{+\,\nu_{hk}\\!-\\!g_{hk}|v_{k}|\theta_{hk}}_{\textnormal{considered
as noise}},\\!$ (7b)

where $\mu_{hk}=\delta_{hk}^{\Re}-g_{hk}\delta_{hk}^{v}$ and
$\nu_{hk}=\delta_{hk}^{\Im}+b_{hk}\delta_{hk}^{v}$.

To conclude this section, we observe that the admittance regression model (7)
actively enforces a small phase angle $\theta_{hk}$, in contrast with the
impedance regression (4).

##  IV Impedance Estimation Bias and Variance

This section characterizes the bias and variance of single line impedance
estimation. As described in Appendix A, fitting (4) using the Ordinary Least
Squares (OLS) results in biased estimates of $x_{hk}$ and $r_{hk}$. This bias
can be eliminated by using Total Least Squares (TLS). However, if there are
correlations between the LHS and the RHS of the regression model, the TLS
estimates can also be biased. Nevertheless, for the noise model (1), the TLS
are a close approximation of the Maximum Likelihood Estimator (MLE), which can
only be computed exactly if the phase is known.

######  Lemma 1.

If a regression model $\tilde{z}=A(\tilde{x}-\epsilon_{x})+\epsilon_{z}$,
where $\epsilon_{x},\epsilon_{x}\sim\mathcal{N}(0,\Sigma)$333The RHS and LHS
variables must be normalized to have the same noise variance. is fitted to $D$
datasets $\tilde{x}_{d},\tilde{z}_{d}$, the bias of the TLS is given by

$\displaystyle\\!E[\hat{A}]\\!-\\!A=$
$\displaystyle\\!\left(\sum_{d=1}^{D}\textnormal{var}[\tilde{x}_{d}]-\textnormal{var}[\epsilon_{xd}]\\!\right)^{\\!\\!\\!-1}$
(8)
$\displaystyle\cdot\\!\\!\left(\sum_{d=1}^{D}\textnormal{cov}[\epsilon_{xd},\epsilon_{zd}]+\textnormal{cov}[\epsilon_{xd},\tilde{z}_{d}]+\textnormal{cov}[\tilde{x}_{d},\epsilon_{zd}]\\!\right)\\!\\!.$

###### Proof.

The proof is derived in Appendix A-B. ∎

Applying Lemma 1 with $D=1$ to the model (4) with the noise statistics (1)
gives the bias

$\displaystyle
E[\hat{r}_{hk}]-r_{hk}=(\textnormal{var}[|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})]-\textnormal{var}[\delta_{hk}^{\Re}])^{-1}$
$\displaystyle\\!\cdot\\!(\textnormal{cov}[\delta_{hk}^{\Re},|\tilde{v}_{h}|\\!\\!-\\!\\!|\tilde{v}_{k}|]\\!+\\!\textnormal{cov}[|\tilde{i}_{hk}|\\!\cos(\tilde{\phi}_{hk}),\delta_{hk}^{v}]\\!-\\!\textnormal{cov}[\delta_{hk}^{\Re},\delta_{hk}^{v}]),$

and similarly for $x_{hk}$. All three covariances are zero in (1) so we
conclude that the TLS estimate of the impedance is unbiased. Moreover, the
variance of the impedance estimate can be derived from [22]
as444$\textnormal{var}[\hat{r}_{hk},\hat{x}_{hk}]$ depends on the exact
parameters $r_{hk}$ and $x_{hk}$ and can, in practice, only be estimated if
the SNR is sufficiently high [22].

$\displaystyle\\!\\!\textnormal{var}\left[\begin{matrix}\hat{r}_{hk}\\\
\hat{x}_{hk}\end{matrix}\right]$
$\displaystyle=\frac{\sigma_{hk}^{v}\\!+\\!\|[r_{hk},x_{hk}]\|^{2}_{\Sigma_{hk}^{i}}}{N}\textnormal{var}\\!\left[\begin{matrix}|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})\\\
|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk})\end{matrix}\right]^{-1}\\!\\!\\!\\!,$
(9)

where
$\|[r_{hk},x_{hk}]\|^{2}_{\Sigma_{hk}^{i}}\\!\\!=[r_{hk},x_{hk}]\Sigma_{hk}^{i}[r_{hk},x_{hk}]^{\top}$.
Equation (9) shows the consistency555Consistent means that it converges with
probability $1$ to the exact parameters as $N\rightarrow\infty$. Efficient
means that its variance is equal to the Camer-Rao lower bound[23]. of the TLS
estimate because $\textnormal{var}[\hat{r}_{hk},\hat{x}_{hk}]\rightarrow 0$ as
$N\rightarrow\infty$.

##  V Admittance estimation bias

This section characterizes the biases of several admittance estimation methods
from the literature for a single line. We focus on the limit case where the
sample size is very large, because the amount of data is not a big limitation
for the estimation problem. This means that although the variance can be
derived from [22] (similar to (9)), we do not include it in this study because
it decays to zero as $N\rightarrow\infty$ and is therefore less limiting than
the biases.

The analysis demonstrates that four known methods for estimating the
admittance from smart meter measurements produce biased estimates.
Nevertheless, we show in Section V-D that, surprisingly, the biases decay with
$N\rightarrow\infty$ only if one inverts the impedance estimate. Thus, if one
is interested in the admittance of a line from a large data set, it may be
better to estimate the impedance and then invert the estimate.

###  V-A Omitted-Phase Bias

We start by quantifying the bias that appears when fitting the model (7) using
the TLS. The magnitude difference $|\tilde{v}_{h}|-|\tilde{v}_{k}|$ is often
correlated with the phase shift $\theta_{hk}$, which means that a bias may
appears if it is incorporated as noise, as explained in Section V-C. Using
Lemma 1 with $D=1$ and the model (7a), we can express the resulting omitted-
phase bias666More commonly known as omitted-variable bias. with

$\displaystyle E[\hat{g}_{hk}]$ $\displaystyle\\!-\\!g_{hk}=$
$\displaystyle\;b_{hk}(\textnormal{var}[|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|]\\!-\\!\sigma_{hk}^{v})^{-1}\textnormal{cov}[|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|,\\!|v_{k}|\theta_{hk}],$

and similarly for $\hat{b}_{hk}$.

In order to avoid the omitted-phase bias, the authors of [16, 17] estimate
both the phase $\theta_{hk}$ and the parameters $g_{hk}$ and $b_{hk}$
iteratively in a joint problem. This means using the estimates $\hat{g}_{hk}$
and $\hat{b}_{hk}$ to find the estimate $\hat{e}_{hk}^{\theta}$ of
$|v_{k}|\theta_{hk}$, which can in be plugged in (7) to update $\hat{g}_{hk}$
and $\hat{b}_{hk}$. Any method to compute $\hat{e}_{hk}^{\theta}$ will aim at
satisfying (7). Rearranging the terms, vectorizing and plugging in the
estimates, this means

$\displaystyle\hat{e}_{hk}^{\theta}\\!\\!\left[\begin{matrix}\hat{b}_{hk}\\\
\hat{g}_{hk}\end{matrix}\right]\\!\\!$
$\displaystyle=\\!(|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|\\!-\\!\delta_{hk}^{v})\\!\\!\left[\begin{matrix}\hat{g}_{hk}\\\
-\hat{b}_{hk}\end{matrix}\right]\\!\\!-\\!|\tilde{i}_{hk}|\\!\\!\left[\begin{matrix}\cos(\tilde{\phi}_{hk})\\\
\sin(\tilde{\phi}_{hk})\end{matrix}\right]\\!\\!+\\!\\!\left[\begin{matrix}\delta_{hk}^{\Re}\\\
\delta_{hk}^{\Im}\end{matrix}\right]\\!\\!.$ (10)

The estimate $\hat{e}_{hk}^{\theta}$ can then be used in (7) as

$\displaystyle\\!\\!\\!|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})\\!$
$\displaystyle=[g_{hk},b_{hk}][(|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|),\\!-\hat{e}_{hk}^{\theta}]^{\top}\\!+\mu_{hk},\\!$
(11a) $\displaystyle\\!\\!\\!|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk})\\!$
$\displaystyle=-[g_{hk},b_{hk}][\hat{e}_{hk}^{\theta},(|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|)]^{\top}\\!+\nu_{hk},\\!$
(11b)

While estimating $\theta_{hk}$ avoids the omitted-phase bias, it leads to a
real and imaginary simultaneity bias777The simultaneity bias problem is well
known in statistics. It originates from the lack of causality between
variables that occur simultaneously [24], i.e., variations in voltage and
current flow., which is presented in the next section.

###  V-B Real and Imaginary Simultaneity

A simultaneity bias appears when LHS variables in one equation appear on the
RHS of another equation, hence creating correlation between the noise on the
LHS and the variables in the RHS. To show that this correlation exists, we
consider that the estimates are good, i.e.,
$[\hat{e}_{hk}^{\theta},\hat{g}_{hk},\hat{b}_{hk}]\approx[|v_{k}|\theta_{hk},g_{hk},b_{hk}]$.
Because $[\hat{g}_{hk},\hat{b}_{hk}][g_{hk},b_{hk}]^{\dagger}\approx 1$ by
construction, one obtains

$\displaystyle\textnormal{cov}\\!\big{[}[\delta_{hk}^{\Re},\delta_{hk}^{\Im}],\hat{e}^{\theta}_{hk}\big{]}\\!$
$\displaystyle\approx\textnormal{cov}\\!\big{[}[\delta_{hk}^{\Re},\delta_{hk}^{\Im}],\hat{e}^{\theta}_{hk}[\hat{g}_{hk},\hat{b}_{hk}]\big{]}[g_{hk},b_{hk}]^{\dagger}.$

The expression (10) shows that $\hat{e}_{hk}^{\theta}$ is correlated to the
noises $\delta_{hk}^{\Re}$ and $\delta_{hk}^{\Im}$. Hence,

$\displaystyle\textnormal{cov}\\!\big{[}[\delta_{hk}^{\Re},\delta_{hk}^{\Im}],\hat{e}^{\theta}_{hk}[\hat{g}_{hk},\hat{b}_{hk}]\big{]}=\textnormal{var}\\!\\!\left[\begin{matrix}\delta_{hk}^{\Re}\\\
\delta_{hk}^{\Im}\end{matrix}\right]\\!,$

which means that

$\displaystyle\textnormal{cov}\\!\big{[}[\delta_{hk}^{\Re},\delta_{hk}^{\Im}],\hat{e}^{\theta}_{hk}\big{]}\\!$
$\displaystyle\approx\Sigma_{hk}^{i}[g_{hk},b_{hk}]^{\dagger}\\!.$

Hence, using Lemma 1 for (11) with $D=2$ and the datasets
$[|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|,-\hat{e}^{\theta}_{hk}],|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})$
and
$[-\hat{e}^{\theta}_{hk},-|\tilde{v}_{h}|\\!+\\!|\tilde{v}_{k}|],|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk})$
yields the real and imaginary simultaneity bias

$\displaystyle
E[\hat{b}_{hk},\hat{g}_{hk}]^{\top}\\!-\\![b_{hk},g_{hk}]^{\top}\approx$
$\displaystyle-\\!\\!\Bigg{(}\\!\underbrace{\\!\\!\textnormal{var}\\!\\!\left[\begin{matrix}\\!|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|\\!\\\
-\hat{e}^{\theta}_{hk}\end{matrix}\right]}_{\textnormal{from
\eqref{eq_simple_ls_regression_noisy_with_phase_est_a}}}\\!\\!+\\!\underbrace{\textnormal{var}\\!\\!\left[\begin{matrix}\hat{e}^{\theta}_{hk}\\\
\\!|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|\\!\end{matrix}\right]}_{\textnormal{from
\eqref{eq_simple_ls_regression_noisy_with_phase_est_b}}}\\!\\!-\\!\\!\left[\begin{matrix}\\!s_{hk}^{e}&\\!\\!\\!\\!0\\\
0&\\!\\!\\!\\!s_{hk}^{e}\end{matrix}\right]\\!\\!\\!\Bigg{)}^{\\!\\!\\!\\!-1}\\!\Sigma_{hk}^{i}[g_{hk},b_{hk}]^{\dagger}\\!,$

where
$s_{hk}^{e}=\textnormal{var}[\hat{e}^{\theta}_{hk}\\!-\\!|\tilde{v}_{k}|\theta_{hk}]+\sigma^{v}_{hk}$.

The solution given in the literature [24] to the simultaneity bias is to study
the reduced regression model, i.e. plugging one equation into the other,
substituting the exact unknown variable rather than its estimate. This has
been done in [18], where the authors reduce the equations to remove the
unobserved phase $\theta_{hk}$. We proceed similarly here, i.e., reducing
$|\tilde{v}_{k}|\theta_{hk}$ by plugging (7b) into (7a), which gives the
reduced regression model

$\displaystyle|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})$
$\displaystyle=-\rho_{hk}|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk})$ (12)
$\displaystyle\\!\\!\\!\\!\\!\\!\\!+g_{hk}(1\\!+\\!\rho_{hk}^{2})(|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|-\delta_{hk}^{v})+\rho_{hk}\delta_{hk}^{\Im}+\delta_{hk}^{\Re}.$

This regression gives the estimates of $\rho_{hk}$ and
$g_{hk}(1\\!+\\!\rho_{hk}^{2})$. The latter can then be divided by
$(1+\hat{\rho}_{hk}^{2})$ to obtain an estimate of $g_{hk}$. A similar process
can be followed for $b_{hk}$ by plugging (7a) into (7b).

###  V-C Endogeneity of $\delta_{hk}^{\Re}$ and $\delta_{hk}^{\Im}$

We investigate the bias of TLS estimates when using the regression model (12).
As explained in Section II-B, there is a correlation between
$\delta_{hk}^{\Re}$ and $\delta_{hk}^{\Im}$ when the power angle is not zero.
Hence, from Lemma 1, the estimate of $\hat{\rho}_{hk}$ has the following
endogeneity bias

$\displaystyle E[\hat{\rho}_{hk}]\\!-\\!\rho_{hk}=[1,0]\,\cdot$
$\displaystyle\quad\quad-\\!\\!\left(\\!\\!\textnormal{var}\\!\left[\begin{matrix}|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk})\\\
|\tilde{v}_{h}|\\!-\\!|\tilde{v}_{k}|\end{matrix}\right]\\!\\!-\\!\\!\left[\begin{matrix}\sigma_{hk}^{\Im}&\\!\\!\\!\\!0\\\
0&\\!\\!\\!\\!\sigma_{hk}^{v}\end{matrix}\right]\\!\right)^{\\!\\!\\!-1}\\!\\!\left[\begin{matrix}\textnormal{cov}[\delta_{hk}^{\Re},\delta_{hk}^{\Im}]\\\
0\end{matrix}\right]\\!\\!,$

when estimated from (12).

The removal of the endogeneity bias can be done by dividing (12) by
$g_{hk}(1\\!+\\!\rho_{hk}^{2})$ and putting $|\tilde{v}_{h}|-|\tilde{v}_{k}|$
on the LHS. Thus, the dependence of $\delta_{hk}^{\Re}$ and
$\delta_{hk}^{\Im}$ does not cross sides. Surprisingly, adjusting (12) in this
way produces (4), an impedance estimate, because

$\displaystyle r_{hk}$
$\displaystyle=\frac{g_{hk}^{-1}}{(1\\!+\\!\rho_{hk}^{2})}=\left(g_{hk}\\!+\\!b_{hk}^{2}g_{hk}^{-1}\right)^{-1}\\!\\!,$
(13a) $\displaystyle x_{hk}$
$\displaystyle=\rho_{hk}r_{hk}=-\left(b_{hk}\\!+\\!g_{hk}^{2}b_{hk}^{-1}\right)^{-1}\\!\\!.$
(13b)

###  V-D Impedance Inversion Bias

If the goal is to estimate the admittance values, rather than the impedance
values, the problem is still not fully solved as one must invert (13) to
obtain the estimates of $g_{hk}$ and $b_{hk}$888In Section V-B, inverting
$(1+\hat{\rho}_{hk}^{2})$ also leads to a bias because $\rho_{hk}$ is not
known exactly.. This yields

$\displaystyle g_{hk}$
$\displaystyle=\left(r_{hk}\\!+\\!x_{hk}^{2}r_{hk}^{-1}\right)^{-1},$ (14a)
$\displaystyle b_{hk}$
$\displaystyle=-\left(x_{hk}\\!+\\!r_{hk}^{2}x_{hk}^{-1}\right)^{-1}.$ (14b)

We can quantify the quality of the admittance estimate using bounds on
$E[|\hat{g}_{hk}|]$ and $E[|\hat{b}_{hk}|]$, as the signs are known999From the
line model in Section II-A, $r_{hk}$ and $x_{hk}$ are positive. One can
enforce the signs of the conductance and susceptance by setting
$\hat{g}_{hk}=0$ if the result was negative and $\hat{b}_{hk}=0$ if it was
positive.. To do so, we rewrite (14) as

$\displaystyle|g_{hk}|$
$\displaystyle=r_{hk}(x_{hk}^{2}+r_{hk}^{2})^{-1}+x_{hk}\cdot 0,$
$\displaystyle|b_{hk}|$ $\displaystyle=r_{hk}\cdot
0+x_{hk}(x_{hk}^{2}+r_{hk}^{2})^{-1}.$

Plugging the functions $f_{1}(r,x)=(x^{2}+r^{2})^{-1}$ and $f_{2}(r,x)=0$,
which are convex on the positive quadrant, into a sharp101010the equality
holds for at least one realization of $\hat{r}_{hk}$ and $\hat{x}_{hk}$
Jensen-like inequality [25, Theorem 2.1] gives the bounds111111Upper bounds
were recently discovered for univariate functions [26], however there are no
upper bounds yet for bivariate problems such as (14)

$\displaystyle E[|\hat{g}_{hk}|]\\!$
$\displaystyle\geq\\!E[\hat{r}_{hk}]f_{1}\\!\\!\left(\\!\frac{\textnormal{var}[\hat{r}_{hk}]}{E[\hat{r}_{hk}]}\\!+\\!E[\hat{r}_{hk}],\\!\frac{\textnormal{cov}[\hat{x}_{hk},\\!\hat{r}_{hk}]}{E[\hat{r}_{hk}]}\\!+\\!E[\hat{x}_{hk}]\\!\\!\right)\\!\\!,$
$\displaystyle E[|\hat{b}_{hk}|]\\!$
$\displaystyle\geq\\!E[\hat{x}_{hk}]f_{1}\\!\\!\left(\\!\frac{\textnormal{cov}[\hat{x}_{hk},\\!\hat{r}_{hk}]}{E[\hat{x}_{hk}]}\\!+\\!E[\hat{r}_{hk}],\\!\frac{\textnormal{var}[\hat{x}_{hk}]}{E[\hat{x}_{hk}]}\\!+\\!E[\hat{x}_{hk}]\\!\\!\right)\\!\\!,$

where the variance of the estimates
$\textnormal{var}[\hat{r}_{hk},\hat{x}_{hk}]$ is given by (9). Using the
definition of $f_{1}(r,x)=(r^{2}+x^{2})^{-1}$ and noting that
$E[\hat{r}_{hk}]=r_{hk}$ and $E[\hat{x}_{hk}]=x_{hk}$, one gets

$\displaystyle\\!\\!\\!\\!E[|\hat{g}_{hk}|]\\!$
$\displaystyle\geq\\!\frac{r_{hk}^{3}}{(\textnormal{var}[\hat{r}_{hk}]\\!+\\!r_{hk}^{2})^{2}\\!+\\!(\textnormal{cov}[\hat{x}_{hk},\hat{r}_{hk}]\\!+\\!r_{hk}x_{hk})^{2}},\\!\\!\\!$
(15a) $\displaystyle\\!\\!\\!\\!E[|\hat{b}_{hk}|]\\!$
$\displaystyle\geq\\!\\!\frac{x_{hk}^{3}}{(\textnormal{var}[\hat{x}_{hk}]\\!+\\!x_{hk}^{2})^{2}\\!+\\!(\textnormal{cov}[\hat{x}_{hk},\hat{r}_{hk}]\\!+\\!r_{hk}x_{hk})^{2}}.\\!\\!\\!$
(15b)

Because of the consistency of the TLS estimate,
$\textnormal{var}[\hat{r}_{hk},\hat{x}_{hk}]\rightarrow 0$ as
$N\rightarrow\infty$. This means that $\hat{g}_{hk}$ and $\hat{b}_{hk}$ can be
asymptotically unbiased because the RHS of (15) tends to (14). However, with a
finite number of samples, $\textnormal{var}[\hat{r}_{hk},\hat{x}_{hk}]$ can
remain quite high, which may heavily bias the admittance estimate.

##  VI Network identification

Often, DSOs are interested in estimating the model for their full network.
Ideally, the model could be estimated from smart meter injection measurements
throughout the network. When a grid is radial and the topology is known, the
network estimation problem for the full network can be decomposed into
individual line parameter estimation problems for each line using Kirchoff’s
Current Law. However, this is not possible for mesh networks or radial
networks with unknown topologies. Thus, it is desirable to have a method for
estimating the grid parameters for a full network121212The “full” network that
is estimated consists of the Kron-reduced network connecting just the nodes at
which the injections are measured. Note, the Kron-reduction is different than
the statistical reductions that are used elsewhere in this paper. The
unmeasured injections are treated as noise [27, 12]. from just smart meter
voltage and injection measurements.

Estimating a network from just smart meter voltage and injection measurements
is a much more challenging task than estimating just a single line, however,
and thus requires the following additional approximation:

$\displaystyle i_{h}\approx|i_{h}|e^{-j\phi_{h}}\hskip 26.0pt$
$\displaystyle\textnormal{: neglect current phase shifts}.$

Such approximations are quite accurate under Assumption 1, as shown in [18].
With the aforementioned approximations, the Kirchhoff law at node $h$ is given
by
$|i_{h}|e^{-j\phi_{h}}=\sum_{k=1}^{n}y_{hk}(|v_{h}|(1-j\theta_{h})-|v_{k}|(1-j\theta_{k}))$.
In matrix form and for all $h\in\mathcal{V}$, this gives

$\displaystyle I^{\Re}$ $\displaystyle=(G|V|-B(|V|\Theta)),$ (16a)
$\displaystyle I^{\Im}$ $\displaystyle=-(B|V|+G(|V|\Theta)),$ (16b)

where $I^{\Re}=[|i_{h}|\cos(\phi_{h})]_{h=1}^{n}$,
$I^{\Re}=[|i_{h}|\sin(\phi_{h})]_{h=1}^{n}$, $|V|=[|v_{h}|]_{h=1}^{n}$, and
$\Theta=[|\theta_{h}|]_{h=1}^{n}$. The parameter estimation problem consists
in finding $G\in\mymathbb{R}^{n\times n}$ and $B\in\mymathbb{R}^{n\times n}$
from noisy measurements of the current magnitudes, voltage magnitudes and
power angles.

###  VI-A Reduced Regression Model

Similar to [28], we use pseudo-inverses to avoid the problems caused by the
singularity of $Y$ in the absence of shunt elements. Recall that if $Y$ is
invertible, its pseudo-inverse is equal to its inverse. We first note that
$BG^{\dagger}G=B$ because the matrices have the same null space131313For this
reason, the equality also holds for $GB^{\dagger}B=G$ and other combinations
of these matrices, as well as their sums and products. defined by the vector
of all ones and are both rank $n-1$. Hence, multiplying (16b) by
$BG^{\dagger}$ on both sides, one can rewrite (16) as

$\displaystyle B(|V|\theta)$ $\displaystyle=(G|V|-I^{\Re}),$ $\displaystyle
B(|V|\theta)$ $\displaystyle=-(BG^{\dagger}B|V|+BG^{\dagger}I^{\Im}).$

Combining the two equations yields the reduced model

$\displaystyle(G+BG^{\dagger}B)|V|$
$\displaystyle=I^{\Re}-BG^{\dagger}I^{\Im}.$ (17)

This model was first presented in [18], where the authors regress $|V|$ and
$I^{\Im}$ onto $I^{\Re}$ to obtain
$G=(G+BG^{\dagger}B)(GG^{\dagger}+(BG^{\dagger})^{2})^{\dagger}$, and
similarly for $B$. However, as explained in Section V-C, the regression (17)
suffers from a correlation between $I^{\Re}$ and $I^{\Im}$, which makes the
estimation of $BG^{\dagger}$ very challenging.

###  VI-B Equivalent Impedance

In order to avoid the bias caused by the correlation between $I^{\Re}$ and
$I^{\Im}$, we define the matrix $O$ such that $O_{hh}=\frac{n-1}{n}$ and
$O_{hk}=\frac{1}{n}$. This matrix has the same null space as $G$ and $B$ and
has rank $n-1$. Moreover, the matrix $(G+BG^{\dagger}B)$ also has the same
null space and rank, which means that
$O(G+BG^{\dagger}B)^{\dagger}(G+BG^{\dagger}B)=O$. Moreover, by construction
$O(G+BG^{\dagger}B)^{\dagger}=(G+BG^{\dagger}B)^{\dagger}$. We can therefore
multiply both sides of (17) by $O(G+BG^{\dagger}B)^{\dagger}$ to obtain

$\displaystyle O|V|$
$\displaystyle=(G+BG^{\dagger}B)^{\dagger}I^{\Re}-(G+BG^{\dagger}B)^{\dagger}BG^{\dagger}I^{\Im}.$

Finally, because
$(G+BG^{\dagger}B)^{\dagger}BG^{\dagger}=(B+GB^{\dagger}G)^{\dagger}$, similar
to (4) and (13), one obtains the regression

$\displaystyle O|V|$ $\displaystyle=RI^{\Re}+XI^{\Im},$ (18)

where

$\displaystyle\begin{array}[]{ll}\,R=(G+BG^{\dagger}B)^{\dagger},\\\
X=-(B+GB^{\dagger}G)^{\dagger},\end{array}\Leftrightarrow\begin{array}[]{ll}G=(R+XR^{\dagger}X)^{\dagger},\\\
B=-(X+RX^{\dagger}R)^{\dagger}.\end{array}$ (23)

Estimating $R$ and $X$ using the TLS is consistent, unbiased and efficient for
the same reasons as in the single line problem. Similar to Section V-D, the
pseudo-inversion of the estimates of $R$ and $X$ lead to an asymptotic bias.
It can also be bounded using [25, Theorem 2.1]. The exact expressions are very
long and are outside of the scope of this paper. Nevertheless, we show the
accuracy of the estimates numerically in the next section.

##  VII Unbalanced three-phases identification

The parameter estimation of unbalanced three-phase networks can be done using
the same methods as for the single-phase ones by stacking the voltages and
current matrices as $V_{|}=[V_{k}]_{k=a,b,c}\in\mymathbb{R}^{3n}$ and
similarly for $I^{\Re}_{|}$ and $I^{\Im}_{|}$. However, this greatly reduces
the SNR as the phases are often heavily correlated with one another. In this
case, the voltage difference between the phases is too small to be detected.
There are two main ways to simplify the problem:

  1. (i)

Assume that all self-admittances are equal and that all mutual admittances are
equal.

  2. (ii)

Assume that all self-admittances are equal and that all mutual admittances are
zero.

The simplification (i) still tries to estimate the mutual admittances and is
adapted for heavily unbalanced loads. It is equivalent to the assumption that
the network infrastructure is balanced, while the loads are not. This
assumption is similar to neglecting the negative and zero sequences and holds
quite well if all phases follow the same path, even if they are not transposed
[29]. The resulting regression can be written as

$\displaystyle O|V_{abc}|$
$\displaystyle=R_{L}I^{\Re}_{abc}+X_{L}I^{\Im}_{abc}+R_{M}$ (24)
$\displaystyle\quad\quad+(I^{\Re}_{cab}+I^{\Re}_{bca})+X_{M}(I^{\Im}_{cab}+I^{\Im}_{bca})$

where $V_{abc}=[V_{a},V_{b},V_{c}]\in\mymathbb{R}^{n\times 3}$ and similarly
for $I^{\Re}_{abc}$, $I^{\Re}_{cab}$, $I^{\Re}_{bca}$, $I^{\Im}_{abc}$,
$I^{\Im}_{cab}$, and $I^{\Im}_{bca}$.

The simplification (ii) holds for balanced enough systems, when the influence
of mutual components cancels out. The regression (18) can be used with
$V_{abc}$, $I^{\Re}_{abc}$, and $I^{\Im}_{abc}$ in place of $V$, $I^{\Re}$,
and $I^{\Im}$, respectively. The number of parameters to estimate is reduced,
therefore increasing the data-efficiency. However, it may not be applicable in
networks with large single-phase loads.

##  VIII Experiments

We validate our results on the 16kV distribution grid of Walenstadt,
Switzerland. Not all the nodes are observed/have smart meter measurements. We
first use the line from the substation to Schlittriet to test the single-line
identification methods. We then experiment with network identification using
the Brüsis-to-Freihof six-node sub-network. A groundtruth model of the network
is not available so we assess the accuracy of the estimation methods using a
combination of intuition, cross-validation, and synthetic data, as explained
below.

SchlittrietBrüsisTscherlachSonnentalTöbeliGeneratorFreihofSub-networkRest of
the network Figure 1: Graph of the region of interest in the Walenstadt
network. The lines that will be identified are in blue. The unobserved nodes
are in red.

###  VIII-A Single Line Estimation

####  VIII-A1 Real-World Data Single Line Estimation

The line of interest connecting ”Schlittriet” to the substation (see Fig. 1)
has a resistance of approximately $0.1\Omega$.141414The linear resistivity is
$114$m$\Omega/$km and the line is $880$m long. The reactance is much harder to
compute as it depends on the environment of the line [30]. Its reactance is
however unknown. We first use the raw data into all four methods described in
Section V to obtain the results in Fig. 2. In the rest of the section, we use
the sub-figure indices (a)-(d) of Fig. 2 to denote the corresponding methods.

7321$0$$0$$0$$0$$0$$0$$0$$0$$0$g
[S]7321732173217321$0$$0$$0$$0$$0$$0$$0$$0$$0$(a)b
[S]7321(b)7321(c)7321(d)sample size [days] Figure 2: Estimates of the
conductance and susceptance of a single line using (a) the admittance
regression (7), (b) the joint estimation using (10), (c) the reduced model
(12), (d) the impedance regression (4) with real data. For each method, the
points mark the estimates using each data subset and the line their average.
The true conductance $b$ and susceptance $g$ of the line is not known, but
$R\approx 0.1\Omega$.

We assess the consistency of the estimates with $R=0.1\Omega$, i.e., our only
ground truth, by inverting the 7-days estimate using (13). We observe that
(a), (c), and (d) are consistent with $R=0.1\Omega$, but (b) yields
$y=0.003-0.46j$ S, which corresponds to an incorrect $R=0.01\Omega$. Moreover,
(b) and (c) estimates a much larger X/R ratio than all three other methods.

Because the true conductance and susceptance of the line is not known, we use
cross validation in order to determine the accuracy of the methods. We split
the data into 6 days for estimating the parameters, and 1 day to validate the
line flow power predictions that use the parameters. Table I shows that (a)
provides a good estimate for only $P$, (b) provides good estimates for only
$Q$, and (c) does not provide a good estimate. Thus, only (d) could
approximate both $P$ and $Q$ correctly.

| method | P [kW] | Q [kVA]
---|---|---|---
| measured | 107.49 | 5.19
(a) | admittance (7) | 138.69 | 13.34
(b) | joint (10) | 0.044 | 6.98
(c) | reduced (12) | 15.62 | 47.08
(d) | impedance (4) | 140.34 | 6.63
TABLE I: Measured and predicted average power transmission in the estimated
line for the last day of the data.

####  VIII-A2 Synthetic Data Single Line Estimation

In order to understand better the very wide range of estimates, we generate
synthetic data by computing the voltages that would perfectly match the
measured power for all 7 days and $y=10-0.5j$ S. We then add an increasing
amount of Gaussian noise to observe how the estimates behave and compute the
expected biases from Section V.

0.050.10.20.5125$0$$5$$10$$15$g
[S]0.050.10.20.51250.050.10.20.51250.050.10.20.51250.050.10.20.5125$-10$$-5$$0$$5$$10$(a)b
[S]0.050.10.20.5125(b)0.050.10.20.5125(c)0.050.10.20.5125(d)noise std. [A],
[°], [V] Figure 3: Estimates of the conductance and susceptance of a single
line using the four methods (a)-(d) with synthetic data. For each method, the
points mark the estimates affected by one of the 50 realizations of the noise
generated for each noise level. The line in color shows their average and the
black line shows the predicted bias. In case (d) the black line is a lower
bound on the bias.

Fig. 3 shows that all methods perform much better with synthetic data.
However, the $g$ estimates in (b) and (c) get smaller with more noise and the
$b$ estimate in (a) is too large by a factor of $2$. Both of these errors can
also be observed in Fig. 2. Moreover, the $b$ estimate in (c) changes sign and
becomes larger. We also observe that the bias predictions become worse with
large amounts of noise, probably because Assumption 3 does not hold anymore.
Moreover, the approximation of $E[b]$ for (b) given in Section V-B is not very
close, and the lower bound (15) for (d) is very loose in this particular case.

###  VIII-B Network Estimation

####  VIII-B1 Real-World Data Network Estimation

We use V, P and Q measurements from the sub-network indicated in Fig. 1 in
order to estimate its admittance matrix. The singular values of
$[V,I^{\Re},I^{\Im}]$ reveal a SNR of approximately 1, which makes the
estimation challenging. In order to avoid numerical stability issues151515In
high SNR scenarios, the TLS may fail to estimate the EIV correctly and remove
parts of the signal instead. While the OLS is biased, it is more robust than
the TLS., we test the TLS-based methods with OLS as well. The estimation with
all 7 days of data yields the following results for each method.

  1. (a)

from [15] diverges with both OLS and TLS, yielding a quasi-infinite admittance
estimate.

  2. (b)

from [16] converges to zero.

  3. (c)

from [18] diverges with the TLS but results in the matrix shown in Fig. 4 (c).

  4. (d)

using (18) yields the matrix shown in Fig. 4 (d) with the TLS and a similar
one with the OLS.

(c)  (d)

Figure 4: Heatmaps of the magitudes of the admittance matrix estimates given
by the methods (c) and (d).

The resistance of the lines should be similar to the one in Section VIII-A
given their length and cross-section. We note that only (d) is consistent with
this observation. While the topology is not recognizable from Fig. 4 (d), the
estimate may be useful for control and state estimation purposes. If it is
not, the the data must be improved by installing more accurate smart meter
sensors, PMUs, or lineflow measurements.

##  IX Conclusion

Estimating electrical grid parameters using unsychronized measuring devices is
challenging. This paper quantifies this challenge by describing the non-zero
expected admittance estimation error for four admittance estimation methods in
the literature, as well as the variance of impedance estimation. Impedance
estimation can be conducted without bias using TLS. The results in this paper
warrant further impedance and admittance estimation experiments with real
smart meter data, and experiments that combine smart meter data with data from
other types of sensors such as uPMUs and lineflow measurements.

## Acknowledgement

We thank the Swiss National Science Foundation for supporting this research
under the NCCR Automation project (grant number 51NF40 180545). We would also
like to thank Benjamin Sawicki and WEW (the DSO of Walenstadt) for providing
the data required by our experiments. Additionally, we thank Lisa Laurent and
Paul Dené for the eye-opening results that they found during their Master
projects.

## References

  * [1] “Quadrennial energy review report,” IEEE Power and Energy Society, Tech. Rep., September 2015.
  * [2] “Developing innovative technologies to enhance reliability, ensure resilience, and increase flexibility,” US Department of Energy, Tech. Rep., March 2017.
  * [3] K. Morrissey, “Fundamental research challenges for distribution state estimation to enable high-performing grids,” Smarter Grid Solutions, Tech. Rep., May 2018.
  * [4] L. Schenato, G. Barchi, D. Macii, R. Arghandeh, K. Poolla, and A. Von Meier, “Bayesian linear state estimation using smart meters and pmus measurements in distribution grids,” in _2014 IEEE International Conference on Smart Grid Communications (SmartGridComm)_ , 2014, pp. 572–577.
  * [5] A. Iovine, T. Rigaut, G. Damm, E. De Santis, and M. D. Di Benedetto, “Power management for a dc microgrid integrating renewables and storages,” _Control Engineering Practice_ , vol. 85, pp. 59–79, 2019.
  * [6] “Flexible export limits-final response and proposed actions-july 2023,” Australian Energy Regulator, Tech. Rep., 2023.
  * [7] M. Z. Liu and L. N. Ochoa, “Project edge–knowledge sharing report: Electrical model validation and operating envelope calculation,” Tech. Rep., 2023.
  * [8] O. Ardakanian, V. W. S. Wong, R. Dobbe, S. H. Low, A. von Meier, C. J. Tomlin, and Y. Yuan, “On identification of distribution grids,” _IEEE Transactions on Control of Network Systems_ , vol. 6, no. 3, pp. 950–960, 2019.
  * [9] E. Fabbiani, P. Nahata, G. D. Nicolao, and G. Ferrari-Trecate, “Identification of ac networks via online learning,” _IEEE Transactions on Control Systems Technology_ , 2021.
  * [10] R. Arghandeh, M. Gahr, A. von Meier, G. Cavraro, M. Ruh, and G. Andersson, “Topology detection in microgrids with micro-synchrophasors,” in _2015 IEEE Power & Energy Society General Meeting_. IEEE, 2015, pp. 1–5.
  * [11] A. Wehenkel, A. Mukhopadhyay, J.-Y. Le Boudec, and M. Paolone, “Parameter estimation of three-phase untransposed short transmission lines from synchrophasor measurements,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 69, no. 9, pp. 6143–6154, 2020.
  * [12] K. Moffat, M. Bariya, and A. Von Meier, “Unsupervised impedance and topology estimation of distribution networks—limitations and tools,” _IEEE Transactions on Smart Grid_ , vol. 11, no. 1, pp. 846–856, 2020.
  * [13] K. Weranga, S. Kumarawadu, and D. Chandima, _Smart metering design and applications_. Springer, 2014.
  * [14] S. Vitiello, N. Andreadou, M. Ardelean, and G. Fulli, “Smart metering roll-out in europe: Where do we stand? cost benefit analyses in the clean energy package and research trends in the green deal,” _Energies_ , vol. 15, no. 7, p. 2340, 2022.
  * [15] L. Laurent, J.-S. Brouillon, and G. Ferrari-Trecate, “Maximum likelihood estimation of distribution grid topology and parameters from smart meter data,” in _2023 IEEE PES Grid Edge Technologies Conference & Exposition (Grid Edge)_. IEEE, 2023, pp. 1–5.
  * [16] J. Zhang, Y. Wang, Y. Weng, and N. Zhang, “Topology identification and line parameter estimation for non-pmu distribution network: A numerical method,” _IEEE Transactions on Smart Grid_ , vol. 11, no. 5, pp. 4440–4453, 2020.
  * [17] V. L. Srinivas and J. Wu, “Topology and parameter identification of distribution network using smart meter and $\mu$pmu measurements,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 71, pp. 1–14, 2022.
  * [18] J. Zhang, P. Wang, and N. Zhang, “Distribution network admittance matrix estimation with linear regression,” _IEEE Transactions on Power Systems_ , vol. 36, no. 5, pp. 4896–4899, 2021.
  * [19] M. Vanin, F. Geth, R. D’hulst, and D. Van Hertem, “Combined unbalanced distribution system state and line impedance matrix estimation,” _International Journal of Electrical Power & Energy Systems_, vol. 151, p. 109155, 2023.
  * [20] P. Kundur, _Power System Stability and Control_. CRC Press New York, NY, USA, 2007.
  * [21] Q. Cetina, R. A. J. Roscoe, and P. S. Wright, “Challenges for smart electricity meters due to dynamic power quality conditions of the grid: A review,” in _2017 IEEE International Workshop on Applied Measurements for Power Systems (AMPS)_. IEEE, 2017, pp. 1–6.
  * [22] J. Crassidis and Y. Cheng, “Error-covariance analysis of the total least squares problem,” _Journal of Guidance, Control, and Dynamics_ , vol. 37, 07 2014.
  * [23] G. Casella and R. Berger, _Statistical Inference_. Duxbury Press, 01 2002.
  * [24] J. M. Wooldridge, _Introductory econometrics: A modern approach_. Cengage learning, 2015.
  * [25] R. A. Agnew and J. E. Pecaric, “Generalized multivariate jensen-type inequality.” _JIPAM. Journal of Inequalities in Pure & Applied Mathematics [electronic only]_, vol. 7, no. 4, pp. Paper–No, 2006.
  * [26] J. Liao and A. Berg, “Sharpening jensen’s inequality,” _The American Statistician_ , 2018.
  * [27] F. Dörfler, J. W. Simpson-Porco, and F. Bullo, “Electrical networks and algebraic graph theory: Models, properties, and applications,” _Proceedings of the IEEE_ , vol. 106, no. 5, pp. 977–1005, 2018.
  * [28] O. Stanojev, L. Werner, S. Low, and G. Hug, “Tractable identification of electric distribution networks,” _arXiv preprint arXiv:2304.01615_ , 2023\.
  * [29] J.-S. Brouillon, E. Fabbiani, P. Nahata, K. Moffat, F. Dörfler, and G. Ferrari-Trecate, “Bayesian error-in-variables models for the identification of distribution grids,” _IEEE Transactions on Smart Grid_ , vol. 14, no. 2, pp. 1289–1299, 2022.
  * [30] J. R. Carson, “Electromagnetic theory and the foundations of electric circuit theory 1,” _Bell System Technical Journal_ , vol. 6, no. 1, pp. 1–17, 1927\.
  * [31] I. Markovsky and S. Van Huffel, “Overview of total least-squares methods,” _Signal processing_ , vol. 87, no. 10, pp. 2283–2302, 2007.
  * [32] S. Rhode, F. Bleimund, and F. Gauterin, “Recursive generalized total least squares with noise covariance estimation,” _IFAC proceedings volumes_ , vol. 47, no. 3, pp. 4637–4643, 2014.

##  Appendix A Dilution Bias and Total Least Squares

We explore different methods to fit regression models such as (4) or (7a). In
order to present the theory in various cases, we consider a general regression
model $\tilde{z}=Ax+\epsilon$. For example, in (4) $z$ represents the voltage
drop and $x$ represents the current. The matrix $A$ contains the parameters
and the noise term $\epsilon$ is such that $E[\epsilon]=0$.

###  A-A Dilution Bias

The OLS are a very common method for parameter estimation, which estimates $A$
as

$\displaystyle\hat{A}=\textnormal{var}[x]^{-1}\textnormal{cov}[x,\tilde{z}].$
(25)

This estimate is exact if (i) the variance and covariance estimates are exact
and (ii) $\textnormal{cov}[x,\epsilon]=0$, which means
$\textnormal{cov}[x,\tilde{z}]=A\textnormal{var}[x]$. The condition (i) is
usually almost satisfied with very large amounts of data, as the uncertainty
on variance scales with the inverse of the number of samples.

The condition (ii) is not satisfied in (4) and (7a), as the noises on
$|\tilde{v}_{h}|-|\tilde{v}_{k}|$, $|\tilde{i}_{hk}|\cos(\tilde{\phi}_{hk})$,
and $|\tilde{i}_{hk}|\sin(\tilde{\phi}_{hk})$ are collected in
$\epsilon_{hk}$, thus creating correlations. Independent uncertainty in the
measurement $\tilde{x}$ of $x$, then
$\textnormal{var}[\tilde{x}]=\textnormal{var}[x]+\textnormal{var}[\tilde{x}-x]$,
so $\textnormal{var}[\tilde{x}]^{-1}\prec\textnormal{var}[x]^{-1}$, which
leads to a dilution bias in the OLS [31].

###  A-B Total Least Squares

The TLS estimate the parameters $A$ and the exact variance
$\textnormal{var}[x]$ simultaneously, assuming that
$\epsilon=\epsilon_{z}+\epsilon_{x}\sim\mathcal{N}(0,\Sigma_{z})+\mathcal{N}(0,A\Sigma_{z}A^{\top})$,
i.e., that the right and left hand side variables are i.i.d., Gaussian and
centered on zero. The variance $\textnormal{var}[x]$ is computed as
$\textnormal{var}[\tilde{x}]-\textnormal{var}[\epsilon_{x}]$, where
$\textnormal{var}[\epsilon_{x}]$ is obtained using a low-rank approximation of
$[x+\epsilon_{x},z+\epsilon_{z}]$ [31]161616The TLS can also be computed
online using its recursive formulation [32]. In this case, (25) becomes

$\displaystyle\hat{A}=(\textnormal{var}[\tilde{x}]-\textnormal{var}[\epsilon_{x}])^{-1}\textnormal{cov}[\tilde{x},\tilde{z}].$
(26)

This estimate is unbiased if and only if
$\textnormal{cov}[\tilde{x},\tilde{z}]=\textnormal{cov}[x,z]+\textnormal{cov}[\epsilon_{x},\tilde{z}]+\textnormal{cov}[\tilde{x},\epsilon_{z}]-\textnormal{cov}[\epsilon_{x},\epsilon_{z}]=\textnormal{cov}[x,z]$.
Otherwise the bias is given by
$(\textnormal{var}[\tilde{x}]-\textnormal{var}[\epsilon_{x}])^{-1}(\textnormal{cov}[\tilde{x},\tilde{z}]-\textnormal{cov}[x,z])$,
which is equal to (8) when $D=1$.

###  A-C Multiple Datasets

If $D$ datasets $\tilde{x}_{d},\tilde{z}_{d}$ of equal size such that
$z_{d}=Ax_{d}$ for $d=1,\dots,D$ are combined, (25) becomes
$\hat{A}=\big{(}\sum_{d=1}^{D}\textnormal{var}[x_{d}]\big{)}^{-1}\sum_{d=1}^{D}\textnormal{cov}[x_{d},\tilde{z}_{d}]$.
Moreover, similar to (26), the TLS estimate uses
$\textnormal{var}[x_{d}]=\textnormal{var}[\tilde{x}_{d}]-\textnormal{var}[\tilde{x}_{d}-x_{d}]$
when the noise is independent. Hence the bias is
$\big{(}\sum_{d=1}^{D}\textnormal{var}[x_{d}]-\textnormal{var}[\tilde{x}_{d}-x_{d}]\big{)}^{-1}\sum_{d=1}^{D}(\textnormal{cov}[\tilde{x}_{d},\tilde{z}_{d}]-\textnormal{cov}[x_{d},z_{d}])$,
which is equal to (8).


*Funding:This work was partly funded by DFG Grant 389792660 as part of TRR 248 (Foundations of Perspicuous Software Systems), the Cluster of Excellence EXC 2050/1 (CeTI, project ID 390696704, as part of Germany’s Excellence Strategy), and  the DFG projects BA-1679/11-1 and BA-1679/12-1.

[1]
C. Baier, N. Coenen, B. Finkbeiner, F. Funke, S. Jantsch, and J. Siber.
Causality-based Game Solving.
In Proc. of the 33rd Intern. Conf. on Computer Aided
Verification (CAV) (to appear), 2021.

[2]
C. Baier, C. Dubslaff, F. Funke, S. Jantsch, R. Majumdar, J. Piribauer, and
R. Ziemek.
From verification to causality-based explications (invited talk).
In N. Bansal, E. Merelli, and J. Worrell, editors, 48th
International Colloquium on Automata, Languages, and Programming, (ICALP),
volume 198 of LIPIcs, pages 1:1–1:20. Schloss Dagstuhl -
Leibniz-Zentrum für Informatik, 2021.

[3]
C. Baier, C. Dubslaff, F. Funke, S. Jantsch, J. Piribauer, and R. Ziemek.
Operational causality – necessarily sufficient and sufficiently
In N. Jansen, M. Stoelinga, and P. van den Bos, editors, A
Journey from Process Algebra via Timed Automata to Model Learning : Essays
Dedicated to Frits Vaandrager on the Occasion of His 60th Birthday, pages
27–45. Springer Nature Switzerland, Cham, 2022.

[4]
C. Baier, F. Funke, and R. Majumdar.
A game-theoretic account of responsibility allocation.
In Z.-H. Zhou, editor, Proceedings of the Thirtieth
International Joint Conference on Artificial Intelligence, IJCAI-21, pages
1773–1779. International Joint Conferences on Artificial Intelligence
Organization, 8 2021.
Main Track.

[5]
C. Baier and J.-P. Katoen.
Principles of Model Checking.
MIT Press, 2008.

[6]
T. Ball, M. Naik, and S. K. Rajamani.
From symptom to cause: Localizing errors in counterexample traces.
SIGPLAN Not., 38(1):97–105, 2003.

[7]
I. Beer, S. Ben-David, H. Chockler, A. Orni, and R. J. Trefler.
Explaining counterexamples using causality.
Formal Methods in System Design, 40(1):20–40, 2012.

[8]
H. Chockler.
Causality and responsibility for formal verification and beyond.
In G. Gößler and O. Sokolsky, editors, Proceedings
First Workshop on Causal Reasoning for Embedded and safety-critical Systems
Technologies, CREST@ETAPS 2016, Eindhoven, The Netherlands, 8th April 2016,
volume 224 of EPTCS, pages 1–8, 2016.

[9]
E. M. Clarke, O. Grumberg, K. L. McMillan, and X. Zhao.
Efficient Generation of Counterexamples and Witnesses in Symbolic
Model Checking.
In Proc. of the 32nd Annual ACM/IEEE Design Automation Conf.
(DAC), pages 427–432, New York, NY, USA, 1995. ACM.

[10]
E. M. Clarke, O. Grumberg, and D. Peled.
Model Checking.
MIT Press, 1999.

[11]
N. Coenen, B. Finkbeiner, H. Frenkel, C. Hahn, N. Metzger, and J. Siber.
Temporal causality in reactive systems.
In 20th International Symposium on Automated Technology for
Verification and Analysis, ATVA, pages 25–28, 2022.

[12]
M. Delfour and J. Zolesio.
Shapes and Geometries: Metrics, Analysis, Differential Calculus,
and Optimization, Second Edition.
Advances in Design and Control. Society for Industrial and Applied
Mathematics (SIAM, 3600 Market Street, Floor 6, Philadelphia, PA 19104),

[13]
E. Grädel, W. Thomas, and T. Wilke, editors.
Automata Logics, and Infinite Games: A Guide to Current
Springer-Verlag, Berlin, Heidelberg, 2002.

[14]
A. Groce, S. Chaki, D. Kroening, and O. Strichman.
Error explanation with distance metrics.
International Journal on Software Tools for Technology
Transfer, 8(3):229–247, 2006.

[15]
A. Groce and W. Visser.
What went wrong: Explaining counterexamples.
In T. Ball and S. K. Rajamani, editors, Model Checking
Software, pages 121–136, Berlin, Heidelberg, 2003. Springer Berlin

[16]
J. Y. Halpern.
A Modification of the Halpern-Pearl Definition of Causality.
In Proc. of the 24th Intern. Joint Conf. on AI (IJCAI), pages
3022–3033. AAAI Press, 2015.

[17]
J. Y. Halpern and J. Pearl.
Causes and Explanations: A Structural-Model Approach: Part i:
In Proc. of the 17th Conf. on Uncertainty in AI (UAI), pages
194–202. Morgan Kaufmann Publishers Inc., 2001.

[18]
J. Y. Halpern and J. Pearl.
Causes and Explanations: A Structural-Model Approach. Part I:
The British Journal for the Philosophy of Science,
56(4):843–887, 2005.

[19]
J. Y. Halpern and J. Pearl.
Causes and Explanations: A Structural-Model Approach. Part II:
The British Journal for the Philosophy of Science,
56(4):889–911, 2005.

[20]
D. Hume.
A Treatise of Human Nature.
John Noon, 1739.

[21]
D. Hume.
An Enquiry Concerning Human Understanding.
London, 1748.

[22]
R. M. Karp.
Reducibility among Combinatorial Problems, pages 85–103.
Springer US, Boston, MA, 1972.

[23]
L. Khachiyan, E. Boros, K. Borys, K. Elbassioni, V. Gurvich, G. Rudolf, and
J. Zhao.
On short paths interdiction problems: Total and node-wise limited
Theory of Computing Systems, 10 2007.

[24]
F. Leitner-Fischer and S. Leue.
Causality Checking for Complex System Models.
In Proc. of the 14th Intern. Conf. on Verification, Model
Checking, and Abstract Interpretation (VMCAI), pages 248–267, 2013.

[25]
V. I. Levenshtein.
Binary codes capable of correcting deletions, insertions, and
Soviet physics doklady, 10(8):707–710, 1966.

[26]
D. Lewis.
Journal of Philosophy, 70(17):556–567, 1973.

[27]
D. K. Lewis.
Cambridge, MA, USA: Blackwell, 1973.

[28]
J. L. Mackie.
Causes and conditions.
American Philosophical Quarterly, 2(4):245–264, 1965.

[29]
Z. Manna and A. Pnueli.
The Temporal Logic of Reactive and Concurrent Systems: Safety.
Springer-Verlag, 1995.

[30]
K. S. Namjoshi.
Certifying model checkers.
In 13th International Conference on Computer Aided Verification
(CAV), volume 2102 of Lecture Notes in Computer Science, pages 2–13.
Springer, 2001.

[31]
J. Pearl.
Cambridge University Press, 2 edition, 2009.

[32]
J. Peters, D. Janzing, and B. Schölkopf.
Elements of Causal Inference: Foundations and Learning
MIT Press, Cambridge, MA, USA, 2017.

[33]
M. Renieres and S. P. Reiss.
Fault localization with nearest neighbor queries.
In Proc. of the 18th IEEE Intern. Conf. on Automated Software
Engineering (ASE), pages 30–39, 2003.

[34]
A. Schrijver.
Combinatorial Optimization - Polyhedra and Efficiency.
Springer, 2003.

[35]
K. U. Schulz and S. Mihov.
Fast string correction with levenshtein automata.
International Journal on Document Analysis and Recognition,
5(1):67–85, 2002.

[36]
R. C. Stalnaker.
A theory of conditionals.
In W. L. Harper, R. Stalnaker, and G. Pearce, editors, IFS. The
University of Western Ontario Series in Philosophy of Science, volume 15,
pages 41–55, Dordrecht, 1968. Springer.

[37]
I. van de Poel.
The Relation Between Forward-Looking and Backward-Looking
Responsibility, pages 37–52.
Springer Netherlands, Dordrecht, 2011.

[38]
C. Wang, Z. Yang, F. Ivancic, and A. Gupta.
Whodunit? Causal Analysis for Counterexamples.
In Proc. of the 4th Intern. Symp. on Automated Technology for
Verification and Analysis (ATVA), pages 82–95, 2006.

[39]
S. Wang, A. Ayoub, B. Kim, G. Gößler, O. Sokolsky, and I. Lee.
A causality analysis framework for component-based real-time systems.
In Proceedings of the 4th International Conference on Runtime
Verification (RV), pages 285–303, 2013.

[40]
V. Yazdanpanah and M. Dastani.
Distant group responsibility in multi-agent systems.
In M. Baldoni, A. K. Chopra, T. C. Son, K. Hirayama, and P. Torroni,
editors, PRIMA 2016: Princiles and Practice of Multi-Agent Systems -
19th International Conference, Phuket, Thailand, August 22-26, 2016,
Proceedings, volume 9862 of Lecture Notes in Computer Science, pages
261–278. Springer, 2016.

[41]
V. Yazdanpanah, M. Dastani, W. Jamroga, N. Alechina, and B. Logan.
Strategic responsibility under imperfect information.
In E. Elkind, M. Veloso, N. Agmon, and M. E. Taylor, editors, Proceedings of the 18th International Conference on Autonomous Agents and
MultiAgent Systems, AAMAS '19, Montreal, QC, Canada, May 13-17, 2019,
pages 592–600. International Foundation for Autonomous Agents and Multiagent
Systems, 2019.

[42]
A. Zeller.
Isolating Cause-Effect Chains from Computer Programs.
In Proc. of the 10th ACM SIGSOFT Symp. on Foundations of
Software Engineering (FSE), pages 1–10, New York, NY, USA, 2002. ACM.