# Parameter Estimation in Ill-conditioned Low-inertia Power Systems

Rajasekhar Anguluri, Lalitha Sankar, and Oliver Kosut This work is funded in
part by the NSF under grant OAC-1934766. All the authors are with the School
of Electrical, Computer, and Energy Engineering, Arizona State University,
Tempe, AZ, 85281 USA (e-mail: {rangulur,lalithasankar,okosut}@asu.edu).

###### Abstract

This paper examines model parameter estimation in dynamic power systems whose
governing electro-mechanical equations are ill-conditioned or singular. This
ill-conditioning is because of converter-interfaced power systems generators’
zero or small inertia contribution. Consequently, the overall system inertia
decreases, resulting in low-inertia power systems. We show that the standard
state-space model based on least squares or subspace estimators fails to exist
for these models. We overcome this challenge by considering a least-squares
estimator directly on the coupled swing-equation model but not on its
transformed first-order state-space form. We specifically focus on estimating
inertia (mechanical and virtual) and damping constants, although our method is
general enough for estimating other parameters. Our theoretical analysis
highlights the role of network topology on the parameter estimates of an
individual generator. For generators with greater connectivity, estimation of
the associated parameters is more susceptible to variations in other generator
states. Furthermore, we numerically show that estimating the parameters by
ignoring their ill-conditioning aspects yields highly unreliable results.

##  I INTRODUCTION

Accurate knowledge of power system model parameters, including inertia and
damping, is essential to assess operating states, perform dynamic simulations,
and study stability margins. Recently, with increasing penetration of
inverter-based (IB) distributed energy resources (DERs) in the bulk power
system, the effective system inertia is decreased, making it challenging to
stabilize demand-supply mismatch. Further, this increase in IB-DERs
significantly increases the number of unknown system parameters to estimate.

Estimating dynamic parameters of synchronous machines and other network
devices and loads, is a classical problem [1, 2]. Numerous algorithms have
been proposed for parameter estimation, both in the presence and absence of
closed-loop controllers using local or wide-area ambient measurements,
including [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]. Within this body
of work, some approaches use _white box models_ , wherein the model structure
is completely known and deterministic (e.g., model structures given by
Newton’s laws, mass and energy conservation principles). In power systems,
Heffron-Phillips models (fourth order and beyond) have been a mainstay of
estimation algorithms. The opposite extreme are _black box models_ , or purely
data-driven stochastic models in which no prior knowledge is assumed, and an
input/output relation is derived from measurements. Examples include modal
analysis, dynamic equivalents, and Koopman methods. Although black box methods
are extremely useful for wide-area monitoring, they have limited utility for
planning, contingency, and stability analysis.

Another line of research focuses on _grey box models_ [17, 18]. These models
combine the advantages of the white and black box approaches to exploit prior
knowledge of physical relationships or model structure, where possible, and
learning unknown parameters from data. This methodology is particularly
relevant in the context of IB-DERs, which inevitably introduce many unknown
parameters [19, 20, 21, 22]. However, there are many shortcomings in the
existing literature; most papers: (i) focus on net inertia111Estimated as a
weighted average of single area inertia estimates. rather than the inertia of
each device or each area as they connect to each other; (ii) focus on
estimation in the presence of transient disturbance with little work on
ambient disturbances; and (iii) do not consider the effect of frequency and
voltage dependent loads, leading to large (inertia) estimation errors (up to
40% [23]); see [24, 25], for a recent account on various parameter estimation
in low-inertia systems.

We put forth a simple strategy for overcoming the above limitations using a
simple constrained least squares estimator to estimate parameters using
ambient measurements. Least squares type estimators are already used for
estimating inertia in power systems; however, these estimators assume that the
inertia is strictly greater than zero. This assumption implies that the
electro-mechanical dynamics are well defined. However, this assumption does
not hold for converter-based generators. For e.g., droop-control based
generators provide zero inertia [22]. Consequently, the electro-mechanical
dynamics are not well-defined or ill-conditioned, thereby giving rise to a
descriptor system (see Section III for details). We develop a framework for
parameter estimation for these systems, with special attention to inertia and
damping. Beyond the motivating example of parameter estimation in power
systems, our results apply more broadly to other engineering systems modeled
using second-order differential equations, such as structural mechanical and
acoustic systems and fluid mechanics. We summarize our contributions below:

  1. (i)

For low-inertia power systems consisting of synchronous and converter-
interfaced generators, we study a constrained least-squares estimation problem
that allow us to tackle systems with exactly zero-inertia.

  2. (ii)

We highlight the role of network connectivity on the estimation performance.
Specifically, using the closed-form formulas of the estimators, we show that
for generators with greater connectivity, estimation of the associated
parameters is more susceptible to variations in other generator states.

  3. (iii)

Our simulation results on the IEEE 39 bus system show that estimating the
parameters by ignoring their ill-conditioning aspects yields highly unreliable
results

##  II Dynamics of Low-inertia Power Systems

We introduce the frequency dynamics for a low-inertia power system, comprised
of multiple synchronous generators (SGs) and converter-interfaced distributed
energy resources (DERs). We later use these models for parameter estimation
subject to suitable physical constraints.

We model a power network of $N$ buses with an undirected graph
$\mathcal{G}:=(\mathcal{V},\mathcal{E})$, where nodes
$\mathcal{V}=\\{1,\ldots,N\\}$ and edges
$\mathcal{E}\subseteq\mathcal{V}\times\mathcal{V}$ denote buses and
transmission lines, respectively. For a $i$-th node in $\mathcal{V}$ we
associate a generator (synchronous or converter-faced) whose frequency
response around a steady state is governed by the swing equation [22, 26, 27]:

$\displaystyle\frac{2H_{i}}{\omega_{0}}\Delta\dot{\omega}_{i}(t)\\!=\\!\left[\Delta
P_{m,i}(t)\\!-\\!\Delta P_{e,i}(t)\right]-FC_{i}(t)+\tilde{\epsilon}_{i}(t),$
(1)

where $\omega_{0}=120\pi$ is the rated angular frequency,
$\Delta\omega_{i}(t)=\omega_{i}(t)-\omega_{0}$, and $2H_{i}/w_{0}$ is the
inertia constant. $\Delta P_{m,i}(t)$ is the deviation from the steady
mechanical power injection. $\Delta P_{e,i}(t)$ is the deviation from the
electrical power output, and $\tilde{\epsilon}_{i}(t)$, a zero-mean Gaussian
process with a known variance, models the ambient fluctuation in loads as well
as process noise. Further, $\Delta P_{e,i}(t)$ equals the sum of deviations of
the power flows on the lines connected to node $i$ [26, 27]:

$\displaystyle\Delta P_{e,i}(t)=\sum_{ij\in\mathcal{E}}\Delta
P_{ij}(t)=\beta_{ij}(\Delta\delta_{i}(t)-\Delta\delta_{j}(t)),$ (2)

where $\beta_{ij}\\!=\\!|V_{i}||V_{j}|b_{ij}$ with $b_{ij}\\!>\\!0$ denoting
the susceptance and $|V_{i}|$, $|V_{j}|$ are the rated voltage magnitudes. The
angular deviation $\Delta\delta_{i}(t)$ is obtained by integrating
$\Delta\omega_{i}(t)$.

The frequency controller output $FC_{i}(t)$ enforces the system frequency
stability due to a large imbalance between the mechanical and electrical
power. In SGs, primary frequency controllers (PFCs) provide the frequency
support. On the other hand, in grid-forming converters, the behavior of PFC is
emulated by fast frequency regulators. We assume that this goal is achieved by
a proportional feedback control that adjusts the power generation set-point
based on the frequency deviation: $FC_{i}(t)=K_{i}\Delta\omega_{i}(t)/w_{0}$
[22] (see Remark 1).

We assume that some of the loads depend on the system frequency. Similar to
frequency controllers, these loads provide a damping stabilizing effect on the
frequency. We model these loads as $\Delta
P_{i,\text{Load}}(t)=D_{i,\text{load}}\Delta\omega_{i}(t)/w_{0}$, where
$D_{i,\text{load}}$ is the damping coefficient. By slight abuse of notation,
we denote the total frequency support by
$FC_{i}(t)=(K_{i}+D_{i,\text{load}})\Delta\omega_{i}(t)/w_{0}$ and let
$D_{i}=K_{i}+D_{i,\text{load}}$.

We drop $\Delta$ notation in the state variables. From (1) and (2), and our
discussions on the frequency controller, we can express the dynamics for all
generators compactly as

$\displaystyle\underbrace{\begin{bmatrix}I&0\\\ 0&M\end{bmatrix}}_{\triangleq
E}\begin{bmatrix}\dot{\boldsymbol{\delta}}(t)\\\
\dot{\boldsymbol{\omega}}(t)\end{bmatrix}\\!=\\!\underbrace{\begin{bmatrix}0&I\\\
-H_{\beta}&-D\end{bmatrix}}_{\triangleq
A}\begin{bmatrix}{\boldsymbol{\delta}}(t)\\\
{\boldsymbol{\omega}}(t)\end{bmatrix}\\!+\\!\begin{bmatrix}\mathbf{0}\\\
\boldsymbol{\epsilon}(t)\end{bmatrix},$ (3)

where
${\boldsymbol{\delta}}=[\delta_{1},\ldots,\delta_{N}]^{\mathsf{T}}\in\mathbb{R}^{N}$
and
${\boldsymbol{\omega}},\dot{\boldsymbol{\delta}},\dot{\boldsymbol{\omega}},\boldsymbol{\epsilon}$
and $\mathbf{0}$ are defined similarly. The $i$-th component of the process
noise $\boldsymbol{\epsilon}(t)$ is given by
$\epsilon_{i}(t)=\tilde{\epsilon}_{i}(t)+P_{m,i}(t)$. The matrices $I$ and $0$
are $N\times N$ identity and all-zeros matrices. The Laplacian matrix
$H_{\beta}$ is defined as $[H_{\beta}]_{ij}=-\beta_{ij}$ for
$(i,j)\in\mathcal{E}$ and $[H_{g}]_{ij}=0$ otherwise; and
$[H_{g}]_{ii}=\sum_{(i,j)\in\mathcal{E}}\beta_{ij}$. Finally,
$M=\text{diag}(M_{11},\ldots,M_{ii})$ and
$D=\text{diag}(D_{11},\ldots,D_{NN})$ are diagonal inertia and damping
matrices, where $D_{ii}=D_{i}/w_{0}$ and $M_{ii}=2H_{i}/w_{0}$.

From (3) note that the Laplacian $H_{g}$ is determined by the line
susceptances, and hence, it is independent of the type of the generator
(synchronous or converter-based). Thus, each generator is characterized by
$M_{i}$ and $D_{i}$. However, we show that the estimates $\hat{M}_{i}$ and
$\hat{D}_{i}$ are influenced by $H_{g}$. The effect of $H_{g}$ is ignored in
prior works, which focus on either estimating each machine’s inertia or the
aggregated inertia.

The (classical) model in (3) is a starting point for many downstream tasks,
including control design, storage placement, oscillation localization, and
stability analysis. In these applications, the model in (3) is simplified by
left multiplying $E^{-1}$ on both sides of (3). Unfortunately, in low-inertia
power systems, this kind of simplification is not possible because the inertia
constant $M_{i}$ could be small for VSMs and exactly zero for droop-control
based generators [?]. Consequently, $E$ in (3) is not invertible. In this
case, this as a _linear descriptor_ or _differential-algebraic system_. The
latter term derives from the fact that some of the equations represented by
(3) are purely algebraic (and not differential) in that the left-hand side is
zero. These systems appear in the field of robotics, economics, and circuits;
in power systems, they also arise when generator dynamics and algebraic power-
flow algebraic equations explicitly are considered together. In our case, a
descriptor system arises due to the ill-conditioning of parameters caused by
the low inertia of IB-DERs. In the following section, we discuss why parameter
estimation is difficult in these systems and then describe a new strategy to
obtain reliable parameter estimates.

###### Remark 1

In general, the frequency controller might not be a simple proportional
control and can be of higher order; for example, in SG, turbine dynamics
contribute to $FC_{i}(t)$. However, for ease of analysis, we neglect these
dynamics. This approximation is valid for converter-interfaced generators
because the controller time constants are small; however, this approximation
might not be accurate for SGs. $\square$

##  III Structure Preserving Estimation Problem

For the continuous-time model in (3), we first obtain a discrete-time model
using Euler’s method. We then formulate a constrained least squares
optimization problem for estimating the parameters using this discrete-time
model.

We assume that we can estimate the generator states ${\boldsymbol{\delta}}$
and ${\boldsymbol{\omega}}$ using PMU measurements [?]. Let $k=0,1,\ldots$ and
define $\mathbf{z}[k]\triangleq\mathbf{z}(kT_{s})$, where $T_{s}$ is the
discretization step (hereafter, the sampling period) and
$\mathbf{z}[k]=[\boldsymbol{\delta}[k]^{\mathsf{T}}\,\boldsymbol{\omega}[k]^{\mathsf{T}}]^{\mathsf{T}}$.
The relationship among $T_{s}$, resolution of the PMU measurements, and the
time-scale of the estimation horizon is explained in great detail in [28].
Using the Euler-Mayurama discretization method, we get the discrete-time
dynamics [29, 26]:

$\displaystyle
E(\mathbf{z}[k+1]-\mathbf{z}[k])=T_{s}A\mathbf{z}[k]+\begin{bmatrix}0\\\
\mathbf{r}[k]\end{bmatrix},$ (4)

where $\mathbf{r}[k]$ is the discretized process noise (cf.
$\boldsymbol{\epsilon}(t)$ in (3)), and
$\mathbf{r}[k]\sim\mathcal{N}(0,\Sigma_{\epsilon})$, where
$\Sigma_{\epsilon}=T_{s}\text{diag}(\sigma^{2}_{1},\ldots,\sigma^{2}_{N})$.
The diagonal structure of $\Sigma_{\epsilon}$ is because the ambient
fluctuations are spatially uncorrelated across different buses.

The standard practice in the literature [30, 26, 27, 22, 19] is to re-write
(4) as

$\displaystyle\mathbf{z}[k+1]=(I+T_{s}E^{-1}A)\mathbf{z}[k]+E^{-1}\begin{bmatrix}0\\\
\mathbf{r}[k]\end{bmatrix},$ (5)

and estimate $A_{d}\triangleq(I+T_{s}E^{-1}A)$ using
$\mathbf{z}[0],\ldots,\mathbf{z}[\mathcal{T}-1]$. This naïve estimate has many
drawbacks: (i) $A_{d}$ might not be well-defined if $E$ is not invertible,
which is the case for droop-control based generators, as discussed earlier;
(ii) $E^{-1}$ adversely affects the noise vector by distorting its spatially
uncorrelated property; and (iii) decomposing the estimate of $A_{d}$ to
uniquely estimate $M$ and $D$ is impossible in general.

We overcome the limitations of the naïve estimator by considering the
following constrained least-squares optimization that does not require $E$ to
be invertible:

$\displaystyle\\{\hat{M},\hat{D}\\}$
$\displaystyle=\operatorname*{arg\,min}_{M,D\,\in\,\mathcal{D}}\sum_{k=0}^{\mathcal{T}-1}\left\lVert
E(\mathbf{z}[k+1]\\!-\\!\mathbf{z}[k])\\!-\\!T_{s}A\,\mathbf{z}[k]\right\rVert_{2}^{2}$
$\displaystyle\textrm{ s.t. }0\leq D_{ii}\leq D_{\text{max}},\text{ for all
}i,$ (6) $\displaystyle\quad\,\,M_{i}=0,\text{ for }i\in\mathcal{V}_{DC},$

where $\mathcal{D}$ is the set of non-negative diagonal matrices;
$D_{\text{max}}$ is a known term that imposes practical limits on $D$; and
$\mathcal{V}_{DC}$ are the nodes corresponding to the droop-control
generators. The equality constraint in (III) ensures that the estimate
$\hat{M}_{i}$, $\text{for }i\in\mathcal{V}_{DC}$, is zero. From (5), we note
that the expression inside the norm term in (III) is the process noise
$[\mathbf{0}^{\mathsf{T}}\,\mathbf{r}[k]^{\mathsf{T}}]^{\mathsf{T}}$. Thus,
the proposed estimator attempts to find parameters that best explain the
variations of the ambient fluctuations over the time horizon
$k=0,\ldots,\mathcal{T}-1$.

We rewrite (III) in the standard least squares form. Define the vectors
$\mathbf{m}=[M_{11},\ldots,M_{NN}]^{\mathsf{T}}$,
$\mathbf{d}=[D_{11},\ldots,D_{NN}]^{\mathsf{T}}$. Let
$\tilde{\boldsymbol{\omega}}[k]=\boldsymbol{\omega}[k+1]-\boldsymbol{\omega}[k]$
and
$\boldsymbol{\delta}_{0:\mathcal{T}-1}=[\boldsymbol{\delta}[0],\ldots,\boldsymbol{\delta}[\mathcal{T}-1]]^{\mathsf{T}}$.
Let $\text{Diag}(\tilde{\boldsymbol{\omega}}[k])$ be the diagonal matrix with
the entries of $\tilde{\boldsymbol{\omega}}[k]$ on the main diagonal, and
define the data matrix:

$\displaystyle
W_{0:\mathcal{T}-1}=\begin{bmatrix}\text{Diag}(\tilde{\boldsymbol{\omega}}[0])&T_{s}\text{Diag}{\boldsymbol{\omega}}[0]\\\
\text{Diag}(\tilde{\boldsymbol{\omega}}[1])&T_{s}\text{Diag}{\boldsymbol{\omega}}[1]\\\
\vdots&\vdots\\\
\text{Diag}(\tilde{\boldsymbol{\omega}}[\mathcal{T}-1])&T_{s}\text{Diag}{\boldsymbol{\omega}}[\mathcal{T}-1]\end{bmatrix}.$
(7)

Then the optimization in (III) can be compactly expressed as

$\displaystyle\\{\hat{\mathbf{m}},\hat{\mathbf{d}}\\}$
$\displaystyle=\operatorname*{arg\,min}_{\mathbf{m},\mathbf{d}\,\in\,\mathbb{R}^{N}}\left\lVert
W_{0:\mathcal{T}-1}\begin{bmatrix}\mathbf{m}\\\
\mathbf{d}\end{bmatrix}+T_{s}(I\otimes
H_{\beta})\boldsymbol{\delta}_{0:\mathcal{T}-1}\right\rVert_{2}^{2}$ (8)
$\displaystyle\textrm{s.t. }\mathbf{0}\leq\mathbf{d}\leq
D_{\text{max}}\mathbf{1},\text{ and }\Gamma\begin{bmatrix}\mathbf{m}\\\
\mathbf{d}\end{bmatrix}=\mathbf{0},$

where $\mathbf{1}$ is the all-ones vector; $I$ is an
$\mathcal{T}\times\mathcal{T}$ identity matrix; and $\otimes$ is the matrix
Kronecker product. The $n\times 2N$ selection matrix $\Gamma$ (with $n$
denoting the size of $\mathcal{V}_{DC}$) selects the entries of $\mathbf{m}$
associated with the droop-control generators.

Optimization problems similar to (8) are recently studied in the literature of
inertia and damping estimation [?]. These studies, however, ignore the zero-
inertia constraints and $H_{\beta}$ term; hence, they require damping
constraints to make the estimation problem mathematically well-posed. In
contrast, the problem in (8) is well-posed even when we ignore the damping
constraints, thereby making it useful for the cases where $D_{\max}$ is
unknown. Using the special case below, we study the role of topology, encoded
in the susceptance matrix $H_{b}$, on the parametric estimates of the $i$-th
generator.

_Special case (unconstrained optimization)_ : Suppose that
$W_{0:\mathcal{T}-1}$ has full column rank.222For an appropriate choice of
$N$, in general, the full column rank assumption holds because of the presence
of additive noise in the measurements. Let us ignore the constraints in (8).
Then, the problem in (8) reduces to the unconstrained least squares problem,
which admits the following solution:

$\displaystyle\begin{bmatrix}\hat{\mathbf{m}}\\\
\hat{\mathbf{d}}\end{bmatrix}$
$\displaystyle=-T_{s}W_{0:\mathcal{T}-1}^{+}(I\otimes
H_{\beta})\boldsymbol{\delta}_{0:\mathcal{T}-1},$ (9)

where $W_{0:\mathcal{T}-1}^{+}$ is the pseudo-inverse of
$W_{0:\mathcal{T}-1}$, and is given by
$W_{0:\mathcal{T}-1}^{+}=(W_{0:\mathcal{T}-1}^{\mathsf{T}}W_{0:\mathcal{T}-1})^{-1}W_{0:\mathcal{T}-1}^{\mathsf{T}}$.
By exploiting the diagonal structure of the blocks in $W_{0:\mathcal{T}-1}$ in
(7), we can express estimates at the $i$-th generator node as

$\displaystyle\begin{split}\hat{m}_{i}&=-\sum_{j=1}^{N}[H_{\beta}]_{i,j}\left(\sum_{k=0}^{\mathcal{T}-1}\left[\frac{c_{i,2}}{c_{i,3}}\tilde{\omega}_{i}[k]-\frac{c_{i,1}}{c_{i,3}}{\omega}_{i}[k]\right]\delta_{j}[k]\right)\\\
\hat{d}_{i}&=-\sum_{j=1}^{N}[H_{\beta}]_{i,j}\left(\sum_{k=0}^{\mathcal{T}-1}\left[\frac{c_{i,0}}{c_{i,3}}{\omega}_{i}[k]-\frac{c_{i,1}}{c_{i,3}}\tilde{\omega}_{i}[k]\right]\delta_{j}[k]\right),\end{split}$
(10)

where the constants $c_{i,3}=c_{i,0}c_{i,2}-c^{2}_{i,1}$;
$c_{i,0}=\sum_{k=0}^{\mathcal{T}-1}\tilde{\omega}^{2}_{i}[k]$;
$c_{i,1}=\sum_{k=0}^{\mathcal{T}-1}\tilde{\omega}_{i}{\omega}_{i}[k]$; and
$c_{i,2}=\sum_{k=0}^{\mathcal{T}-1}{\omega}^{2}_{i}[k]$. In the above
expressions, we set $T_{s}=1$ for simplicity. The constants $c_{i,0},c_{i,1}$,
and $c_{i,2}$ depend on the $i$-th generator’s frequencies. They determine the
contribution of the frequency and its rate of change
$\tilde{\omega}_{i}[k]=\omega[k+1]-\omega[k]$ on the $i$-th estimate.

The $i$-th inertia (or damping) estimate in (10) is a weighted average of the
suceptance values of the lines connected to the $i$-th node. These weights
depend both on the $i$-th node’s frequencies and the angles of all generators.
Thus, for generators with greater connectivity, estimation of the associated
parameters is more susceptible to variations in other generator states.
Consequently, these parameters cannot be estimated using local measurements.
But it makes sense to estimate the inertia of a largely isolated microgrid as
it has a few or no connections with other parts of the network. Finally, we
can only estimate the parameters of a generator in a large system when both
the local frequency and the power measurements are available. Recall from (2)
that the power deviations encode the topological information.

We comment on the statistical properties of the estimate in (9). Because
$\mathbf{w}(t)$ and $\boldsymbol{\delta}(t)$ in (3) are correlated random
processes, $W_{0:\mathcal{T}-1}$ and $\boldsymbol{\delta}_{0:\mathcal{T}-1}$
are random and correlated. Thus, characterizing the distribution of the
estimate in (9) is hard. A workaround is to interpret the optimization in (8)
as a means for obtaining the parameters of the linear model:

$\displaystyle-T_{s}(I\otimes
H_{\beta})\boldsymbol{\delta}_{0:\mathcal{T}-1}=W_{0:\mathcal{T}-1}\begin{bmatrix}\mathbf{m}\\\
\mathbf{d}\end{bmatrix}+\boldsymbol{\zeta},$ (11)

where $\boldsymbol{\zeta}$ and the filtered process noise accumulated over
time have same distributions; however, $W_{0:\mathcal{T}-1}$ and
$\boldsymbol{\delta}_{0:\mathcal{T}-1}$ are responses due to the initial
state. Hence, they are deterministic terms. With these assumptions, it follows
that

$\displaystyle\begin{bmatrix}\hat{\mathbf{m}}\\\
\hat{\mathbf{d}}\end{bmatrix}\\!\sim\\!\mathcal{N}\left(\begin{bmatrix}\mathbf{m}^{*}\\\
\mathbf{d}^{*}\end{bmatrix},{T^{2}_{s}}W_{0:\mathcal{T}-1}^{+}\Sigma_{\boldsymbol{\zeta}}(W_{0:\mathcal{T}-1}^{+})^{\mathsf{T}}\right),$
(12)

where $(\mathbf{m}^{*},\mathbf{d}^{*})$ is the unknown truth, and
$\Sigma_{\boldsymbol{\zeta}}$ is the covariance matrix of
$\boldsymbol{\zeta}$. The characterization in (12) holds even for the non-
Gaussian process noise, thanks to the asymptotic (in $\mathcal{T}$) normality
of the least squares estimator (see [17]). If $\Sigma_{\boldsymbol{\zeta}}$ is
diagonal,
$W_{0:\mathcal{T}-1}^{+}\Sigma_{\boldsymbol{\zeta}}(W_{0:\mathcal{T}-1}^{+})^{\mathsf{T}}$
is a $2\times 2$ block matrix with diagonal blocks. The off-diagonal blocks
capture correlations between the inertia and damping estimate at a given node.
Thus, the variance of the estimates are not influenced by the variations in
other generator states. Unfortunately, $\Sigma_{\boldsymbol{\zeta}}$ cannot be
diagonal because the process noise gets filtered through the dynamics in (4);
and hence, $\Sigma_{\boldsymbol{\zeta}}$ is dense, and so is the covariance
matrix in (12). Hence, the variance of each estimate depends both on the
network and the variations in other generator states.

The discrete-time variance $\sigma_{i}^{2}T_{s}$ of $\mathbf{r}[k]$ (see (4))
due to the process noise and loads is hidden in the covariance matrix
$\text{Cov}(\boldsymbol{\zeta})$. By assuming $\sigma_{i}^{2}=\sigma^{2}$, for
all $i\in N$, we can write $\text{Cov}(\boldsymbol{\zeta})=\sigma_{i}T_{s}Q$,
where the matrix $Q$ solely depends on the system dynamic matrices and the
topology. Thus, the covariance term in (12) is effectively scaled by
$T_{s}^{3}\sigma^{2}$. This result highlights the trade-off between the
sampling time and the variance of load fluctuations, allowing us to down- or
up-sample measurements to improve the estimates’ quality. For instance, for
highly fluctuating loads (higher values of $\sigma^{2}$), we can down-sample
the measurements for computational speedup with minimal loss in the estimation
performance.

We close this section by pointing out the importance of the structure
preserving estimation problem in (III) in the case where we have access to a
few generator states but not all of them. Here, we cannot rewrite (III) as the
constrained form in (8). Nonetheless, we can use expectation-maximization (EM)
type algorithms, which at high level solves the optimization in (8), but the
data matrix $W_{0,\mathcal{T}-1}$ should be replaced with Kalman estimates. We
leave this study for the future.

##  IV Simulation Results

We illustrate the performance of the structure-persevering inertia and damping
constants estimator in (8) on the IEEE 39-bus, 10-generator benchmark system.
See Ref. [26] for a single line diagram of the topology and the location of
the generator buses. The inertia constants of the generators are summarized in
Table I and all damping constants are set to $d_{i}^{*}=0.0531$ p.u. We use
Kron-reduction technique [26] to obtain the matrix $H_{\beta}$ in (3). We
obtain the initial values of $(\boldsymbol{\delta(t)},\boldsymbol{\omega(t)})$
and the line susceptance values from [31]. We set the discretization time-step
$T_{s}=1/60$ sec. We use these parameter values to generate the frequency
measurements using the discrete-time model in (4).

TABLE I: IEEE 39-bus synchronous generator inertia (p.u.) $m^{*}_{1}$ | $m^{*}_{2}$ | $m^{*}_{3}$ | $m^{*}_{4}$ | $m_{5}^{*}$
---|---|---|---|---
$0.2228$ | $0.1607$ | $0.1873$ | $0.1517$ | $0.1379$
$m^{*}_{6}$ | $m^{*}_{7}$ | $m^{*}_{8}$ | $m^{*}_{9}$ | $m_{10}^{*}$
$0.1846$ | $0.1401$ | $0.18289$ | $0.1830$ | $2.6526$

###  IV-A Case study 1: estimation performance and validation

First, we explore the case where there are no converter-based generators, and
the inertia constants of all synchronous generators are not close to zero; see
Table I. We also ignore the damping constraints for simplicity. Thus, we
consider the unconstrained optimization problem.

Our first simulations focus on the inertia and damping estimation error
behavior as a function of the estimation time $\mathcal{T}$. We define the
following error metrics:

$\displaystyle
E_{\text{int}}=\frac{1}{10}\sum_{i=1}^{10}(\hat{m}_{i}-\hat{m}_{i}^{*})^{2}\,\,;D_{\text{int}}=\frac{1}{10}\sum_{i=1}^{10}(\hat{d}_{i}-\hat{d}_{i}^{*})^{2}.$
(13)

These metrics capture estimation error (squared) of a random generator node.
We set the process noise standard deviation $\sigma=0.01$ p.u. [26]. Fig. 1
illustrates Montecarlo estimate of the mean and the standard deviation (no.
of. trails = 100) of the error metrics. We note that more measurements are
required to estimate damping accurately than inertia. This is because the
inertia estimate in (10) depends more on the difference of the frequencies at
$k$ and $k+1$. Thus, the process noise is less in $\omega[k+1]-\omega[k]$ than
compared to $\omega[k]$. On other hand, the damping estimator rely more on
$\omega[k]$; and hence, its performance is strongly influenced by the process
noise. As a result, it requires more measurements to accurately estimate the
true damping.

Our second simulations focus on the probability distribution of the estimation
error for a random generator. We chose $i=3$. Fig. 2 and Fig. 3 illustrate
empirical histograms for the estimation time horizons $\mathcal{T}=50$ and
$\mathcal{T}=200$.

###  IV-B Case study 2: comparison with the naïve esimator in [26]

Next, we examine the estimator’s performance in the presence of both
synchronous and converter-faced generators. For the latter, we chose VSMs,
whose behavior is emulated by setting the inertia constants to be close (but
not exactly) to zero. In particular, we set $m^{*}_{3}=0.0019$,
$m^{*}_{4}=0.0015$, and $m^{*}_{5}=0.0014$. We also compared the performance
of our estimator with the naïve estimator that first estimates $A_{d}$ in (5)
and then extract the inertia constants. We estimate $A_{d}$ using the maximum-
likelihood technique suggested in [26]. We report our findings in Table II.
Therein, the values in the parenthesis indicate relative estimation errors.
For all the generators, including the VSMs, our structure preserving estimator
quite accurately estimated the inertia constants.

TABLE II: IEEE 39-bus synchronous generator inertia (p.u.) True | our method | naïve estimator
---|---|---
$m^{*}_{1}$ = 0.2228 | 0.2228 (-0.005e-03) | -0.0384 (-1.1724)
$m^{*}_{2}$ = 0.1607 | 0.1607 (-0.251e-03) | -0.0014 (-1.0090)
$m^{*}_{3}$ = 0.0019 | 0.0019 (-0.042e-03) | -0.0008 (-1.4535)
$m^{*}_{4}$ = 0.0015 | 0.0015 (-0.031e-03) | -0.0002 (-0.8677)
$m^{*}_{5}$ = 0.0014 | 0.0014 (-0.873e-03) | -0.0002 (-1.1791)
$m^{*}_{6}$ = 0.1846 | 0.1845 (-0.054e-03) | -0.0915 (-1.4959)
$m^{*}_{7}$ = 0.1401 | 0.1401 (-0.019e-03) | -0.0864 (-0.3833)
$m^{*}_{8}$ = 0.1289 | 0.1289 (-0.015e-03) | -0.0144 (-0.8880)
$m^{*}_{9}$ = 0.1830 | 0.1830 (-0.023e-03) | -0.0369 (-1.2015)
$m^{*}_{10}$ = 2.6526 | 2.6526 (-0.004e-03) | 1.6507 (-0.3777)

The simulations presented in this section supported many of our theoretical
observations and outperformed methods that do not consider the ill-
conditioning aspects as in studies in case 2. These observations have
implications for design and implementation of real-time algorithms for
estimating inertia and damping in low-inertia systems.

Figure 1: Estimation error as a function of estimation time horizon. The
shaded region denotes the standard deviation (averaged over 100 trails). From
both the top and bottom panels, we clearly see that the average error in (13)
decreases by increasing $\mathcal{T}$. However, compared to inertia, we need
more measurements to estimate damping accurately. Figure 2: Empirical
probability distribution of the error deviation of inertia and damping of
generator labelled 3. For $\mathcal{T}=50$, the top panel presents the
histograms of error deviations of inertia for various noise levels. The bottom
panel presents similar plots for damping. In both the panels, the spread
increases (range of x-axis) with increase in $\sigma$. However, this is more
pronounced for the case of error deviations of damping constant. Figure 3:
Empirical probability distribution of the error deviation of inertia and
damping of generator labelled 3. For $\mathcal{T}=200$, the top panel presents
the histograms of error deviations of inertia for various noise levels. The
bottom panel presents similar plots for damping. Compared to Fig. 2, the
distribution is more concentrated around zero. This agrees with our intuition
that estimation error decreases with the increase in measurements.

##  V Concluding Remarks

A simple observation that the parameters of multiple areas or generators could
be directly estimated using a descriptor or ill-conditioned electro-mechanical
dynamics allowed us to estimate the inertia and damping of power systems with
a mix of synchronous and converter-interfaced generators. The latter includes
synchronous virtual machines and droop-control-based generators for which the
inertia constants are exactly or approximately zero, thereby rendering the
utility of the existing inertia and damping estimation methods, which almost
always assume non-negligible inertia. We overcome this limitation by studying
a constrained least-squares estimator on the descriptor-type dynamics, where
the constraints set the inertia of droop-controlled generators to zero. We
argued that the proposed estimator is well-posed and admits a unique solution,
at least for a special case. Furthermore, we discussed some limitations of the
naïve estimator in the context of inertia and damping estimation.

Our analysis highlighted the role of network connectivity on the estimators’
performance, which has not been properly studied in the literature. In
particular, using the closed-form expressions of the estimators, we showed
that for generators with greater connectivity, estimation of the associated
parameters is more susceptible to variations in other generator states.
Finally, our simulation results showed that estimating the parameters by
ignoring the ill-conditioning aspects yields highly unreliable results.

## References

  * [1] LA Kilgore.  Calculation of synchronous machine constants-reactances and time constants affecting transient characteristics.  Transactions of the American Institute of Electrical Engg., 50(4):1201–1213, 1931.
  * [2] Sherwin H Wright.  Determination of synchronous machine constants by test reactances, resistances, and time constants.  Transactions of the American Institute of Electrical Engineers, 50(4):1331–1350, 1931.
  * [3] M Burth, George C Verghese, and M Velez-Reyes.  Subset selection for improved parameter estimation in on-line identification of a synchronous generator.  IEEE Transactions on Power Systems, 14(1):218–225, 1999.
  * [4] Artem Mikhalev, Alexander Emchinov, Samuel Chevalier, Yury Maximov, and Petr Vorobev.  A bayesian framework for power system components identification.  In 2020 IEEE Power & Energy Society General Meeting (PESGM), pages 1–5. IEEE, 2020.
  * [5] Dexin Li, Haifeng Zhang, Bo Wang, Guanqun Zhuang, and Deyou Yang.  Data-driven electromechanical parameter estimation in dynamic model of a synchronous generator.  In 2021 IEEE/IAS Industrial and Commercial Power System Asia, pages 858–863. IEEE, 2021.
  * [6] Behrooz Zaker, Ramtin Khalili, Hadi Rabieyan, and Mehdi Karrari.  A new method to identify synchronous generator and turbine-governor parameters of a gas unit using a closed-loop model.  International Transactions on Electrical Energy Systems, 31(11):e13110, 2021.
  * [7] Arindam Mitra, Abheejeet Mohapatra, Saikat Chakrabarti, and Subrata Sarkar.  Online measurement based joint parameter estimation of synchronous generator and exciter.  IEEE Transactions on Energy Conversion, 36(2):820–830, 2020.
  * [8] Andrey Gorbunov, Anatoly Dymarsky, and Janusz Bialek.  Estimation of parameters of a dynamic generator model from modal PMU measurements.  IEEE Trans. on Power Systems, 35(1):53–62, 2019.
  * [9] Song Guo, Sean Norris, and Janusz Bialek.  Adaptive parameter estimation of power system dynamic model using modal information.  IEEE Trans. on Power Systems, 29(6):2854–2861, 2014.
  * [10] Junbo Zhao, Antonio Gómez-Expósito, Marcos Netto, Lamine Mili, Ali Abur, Vladimir Terzija, Innocent Kamwa, Bikash Pal, Abhinav Kumar Singh, Junjian Qi, et al.  Power system dynamic state estimation: Motivations, definitions, methodologies, and future work.  IEEE Transactions on Power Systems, 34(4):3188–3198, 2019.
  * [11] S Armina Foroutan and Anurag Srivastava.  Generator model validation and calibration using synchrophasor data.  In 2019 IEEE Industry Applications Society Annual Meeting, pages 1–6. IEEE, 2019.
  * [12] Dmitry Kosterev.  Hydro turbine-governor model validation in pacific northwest.  IEEE Trans. on Power Systems, 19(2):1144–1149, 2004.
  * [13] Jin Ma, DONG Han, W-J Sheng, R-M He, C-Y Yue, and J Zhang.  Wide area measurements-based model validation and its application.  IET generation, transmission & distribution, 2(6):906–916, 2008\.
  * [14] Yikui Liu, Lei Wu, and Jie Li.  D-PMU based applications for emerging active distribution systems: A review.  Electric Power Systems Research, 179:106063, 2020.
  * [15] Lingling Fan and Yasser Wehbe.  Extended Kalman filtering based real-time dynamic state and parameter estimation using PMU data.  Electric Power Systems Research, 103:168–177, 2013.
  * [16] Xiaozhe Wang.  Estimating dynamic load parameters from ambient pmu measurements.  In 2017 IEEE Power & Energy Society General Meeting, pages 1–5. IEEE, 2017.
  * [17] Lennart Ljung.  System identification.  In Signal analysis and prediction, pages 163–173. Springer, 1998\.
  * [18] Henrik Melgaard.  Identification of physical models.  DTU Compute, 1994.
  * [19] Jinpeng Guo, Xiaozhe Wang, and Boon-Teck Ooi.  Online model-free estimation of the dynamic system model for a power system with renewables in ambient conditions.  IEEE Access, 8:96878–96887, 2020.
  * [20] Xiaozhe Wang, Janusz W Bialek, and Konstantin Turitsyn.  Pmu-based estimation of dynamic state jacobian matrix and dynamic system state matrix in ambient conditions.  IEEE Transactions on Power Systems, 33(1):681–690, 2017.
  * [21] Muyang Liu, Junru Chen, and Federico Milano.  On-line inertia estimation for synchronous and non-synchronous devices.  IEEE Transactions on Power Systems, 36(3):2693–2701, 2020.
  * [22] Diala Nouti, Ferdinanda Ponci, and Antonello Monti.  Heterogeneous inertia estimation for power systems with high penetration of converter-interfaced generation.  Energies, 14(16):5047, 2021.
  * [23] Dimitrios Zografos and Mehrdad Ghandhari.  Estimation of power system inertia.  In 2016 IEEE Power and Energy Society General Meeting (PESGM), pages 1–5. IEEE, 2016.
  * [24] Evelyn Heylen, Fei Teng, and Goran Strbac.  Challenges and opportunities of inertia estimation and forecasting in low-inertia power systems.  Renewable and Sustainable Energy Reviews, 147:111176, 2021.
  * [25] Bendong Tan, Junbo Zhao, Marcos Netto, Venkat Krishnan, Vladimir Terzija, and Yingchen Zhang.  Power system inertia estimation: Review of methods and the impacts of converter-interfaced generations.  International Journal of Electrical Power & Energy Systems, 134:107362, 2022.
  * [26] Andrey Y Lokhov, Marc Vuffray, Dmitry Shemetov, Deepjyoti Deka, and Michael Chertkov.  Online learning of power transmission dynamics.  In 2018 Power Systems Computation Conference, pages 1–7. IEEE, 2018\.
  * [27] Andrey Y Lokhov, Deepjyoti Deka, Marc Vuffray, and Michael Chertkov.  Uncovering power transmission dynamic model from incomplete pmu observations.  In 2018 IEEE Conference on Decision and Control (CDC), pages 4008–4013. IEEE, 2018.
  * [28] Federico Milano.  Rotor speed-free estimation of the frequency of the center of inertia.  IEEE Transactions on Power Systems, 33(1):1153–1155, 2017.
  * [29] Simo Särkkä and Arno Solin.  Applied stochastic differential equations, volume 10.  Cambridge University Press, 2019.
  * [30] Ujjwol Tamrakar, Nischal Guruwacharya, Niranjan Bhujel, Felipe Wilches-Bernal, Timothy M Hansen, and Reinaldo Tonkoski.  Inertia estimation in power systems using energy storage and system identification techniques.  In 2020 International Symposium on Power Electronics, Electrical Drives, Automation and Motion (SPEEDAM), pages 577–582. IEEE, 2020.
  * [31] Ian Hiskens.  Ieee pes task force on benchmark systems for stability controls.  Technical report, 2013.


# How the Experts Do It: Assessing and Explaining Agent Behaviors in Real-Time
Strategy Games

Jonathan Dodge, Sean Penney, Claudia Hilderbrand, Andrew Anderson, Margaret
Burnett

Oregon State University  Corvallis, OR; USA  $\\{$ dodgej, penneys, minic,
anderan2, burnett<EMAIL_ADDRESS>
###### Abstract

How should an AI-based explanation system explain an agent’s complex behavior
to ordinary end users who have no background in AI? Answering this question is
an active research area, for if an AI-based explanation system could
effectively explain intelligent agents’ behavior, it could enable the end
users to understand, assess, and appropriately trust (or distrust) the agents
attempting to help them. To provide insights into this question, we turned to
human expert explainers in the real-time strategy domain – “shoutcasters” – to
understand (1) how they foraged in an evolving strategy game in real time, (2)
how they assessed the players’ behaviors, and (3) how they constructed
pertinent and timely explanations out of their insights and delivered them to
their audience. The results provided insights into shoutcasters’ foraging
strategies for gleaning information necessary to assess and explain the
players; a characterization of the types of implicit questions shoutcasters
answered; and implications for creating explanations by using the patterns and
abstraction levels these human experts revealed.

###### category:

H.1.2 User/Machine systems Human Information Processing

###### category:

H.5.2 Information interfaces and presentation (e.g., HCI) User Interfaces

###### category:

I.2.m Artificial Intelligence Miscellaneous

###### keywords:

Explainable AI; Intelligent Agents; RTS Games; StarCraft; Information Foraging

.

##  1 Introduction

Real-time strategy (RTS) games are becoming more popular artificial
intelligence (AI) research platforms. A number of factors have contributed to
this trend. First, RTS games are a challenge for AI because they involve real-
time adversarial planning within sequential, dynamic, and partially observable
environments [22]. Second, AI advancements made in the RTS domain can be
mapped to real world combat mission planning and execution such as an AI
system trained to control a fleet of drones for missions in simulated
environments [31].

Figure 1: Two shoutcasters providing commentary for a professional StarCraft
match.

People without AI training will need to _understand_ and ultimately _assess_
the decisions of such a system, based on what such intelligent systems
recommend or decide to do on their own. For example, imagine “Jake,” the proud
owner of a new self-driving car, who needs to monitor the AI system driving
his car through complex traffic like the Los Angeles freeway system at rush
hour, assessing when to trust the system [16] and when he should take control.
Ideally, an interactive explanation system could help Jake assess whether and
when the AI is making its decisions “ _for the right reasons_ ” — in real
time.

Scenarios like this are the motivation for an emerging area of research
referred to as “Explainable AI,” where an automated explanation device
presents an AI system’s decisions and actions in a form useful to the intended
audience — here, Jake. There are recent research advances in explainable AI,
as we discuss in the Related Work section, but only a few focus on explaining
_complex strategy environments_ like RTS games and fewer draw from expert
explainers. To help fill this gap, we conducted an investigation in the
setting of StarCraft II, a popular RTS game [22] available to AI researchers
[33].

We looked to “shoutcasters” (sportscasters for e-sports like RTS games) like
those in Figure 1. In StarCraft e-sports, two players compete while the
shoutcasters provide _real-time_ commentary. They are helpful to investigate
for explaining AI agents in real time to people like Jake for two reasons.
First, they face an _assessment_ task similar to explaining Jake’s car-driving
agent to him. Specifically, they must 1) discover the actions of the player,
2) make sense of them and 3) assess them, particularly if they discover good,
bad, or unorthodox behavior. They must do all this while simultaneously
constructing an explanation of their discoveries in real-time.

Second, shoutcasters are _expert explainers_. As communication professionals,
they are paid to inform an audience they cannot see or receive
feedback/questions from. Hoffman & Klein [10] researched five stages of
explanation, looking at how explanations are formed from observation of an
event, generating one or more possible explanations, judging the plausibility
of said explanations, and either resolving or extending the explanation. Their
findings help to illustrate the complexity of shoutcasters’ task, due to its
abductive nature of explaining the past and anticipating the future. In short,
shoutcasters must anticipate and _answer the questions the audience are not
able to ask_ , all while passively watching the video stream.

Because shoutcasters explain in parallel to gathering their information, we
guided part of our investigation using Information Foraging Theory (IFT) [26],
which explains how people go about their information seeking activities. It is
based on naturalistic predator-prey models, in which the _predator_
(shoutcaster) searches _patches_ (parts of the information environment) to
find _prey_ (evidence of players’ decision process) by following the _cues_
(signposts in the environment that seem to point toward prey) based on their
_scent_ (predator’s guess at how related to the prey a cue is). IFT constructs
have been used to explain and predict people’s information-seeking behavior in
several domains, such as understanding navigations through web sites or
programming and software engineering environments [6, 8, 9, 15, 20, 23, 24,
25, 30]. However, to our knowledge, it has not been used before to investigate
explaining RTS environments like StarCraft.

Using this framework, we investigated the following research questions (RQs):

  1. RQ1

_The What and the Where_ : What information do shoutcasters seek to generate
explanations, and where do they find it?

  2. RQ2

_The How_ : How do shoutcasters seek the information they seek?

  3. RQ3

_The Questions_ : What implicit questions do shoutcasters answer and how do
they form their answers?

  4. RQ4

_The Explanations_ : What relationships and objects do shoutcasters use when
building their explanations?

##  2 Background and Related Work

In the HCI community, research has begun to investigate the benefits to humans
of explaining AI. “Jake” (our test pilot) improving his mental model is
critical to his success, since Jake needs a reasonable mental model of the AI
system to assess whether it is making decisions _for the right reasons_.
Mental models, defined as “internal representations that people build based on
their experiences in the real world,” enable users to predict system behavior
[21]. Kulesza et al. [14] found those who adjusted their mental models most in
response to explanations of AI (a recommender system) were best able to
customize recommendations. Further, participants who improved their mental
models the most found debugging more worthwhile and engaging.

Building upon this finding, Kulesza et al. [13] then identified principles for
explaining (in a “white box” fashion) to users how a machine learning system
makes its predictions more transparent to the user. In user studies with a
prototype following these principles, participants’ quality of mental models
increased by up to 52%, and along with these improvements came better ability
to customize the intelligent agents. Kapoor et al. [11] also showed that
explaining AI increased user satisfaction and interacting with the
explanations enabled users to construct classifiers that were more aligned
with target preferences. Bostandjiev et al.’s work on a music recommendation
system [3] found that explanation led to a remarkable increase in user-
satisfaction with their system.

As to what people _want_ explained about AI systems, one influential work into
explainable AI has been Lim & Dey’s [18] investigation into information
demanded from context-aware intelligent systems. They categorized users’
information needs into various “intelligibility types,” and investigated which
types provided the most benefit to user understanding. Among these types were
“What” questions (What did the system do?), “Why” questions (Why did the
system do X?), and so on. In this paper, we draw upon these results to
categorize the kinds of questions that shoutcasters’ explanations answered.

Other research confirms that explanations containing _certain_ intelligibility
types make a difference in user attitude towards the system. For example,
findings by Cotter et al. [7] showed that justifying _why_ an algorithm works
(but not on _how_ it works) were helpful for increasing users’ confidence in
the system — but not for improving their trust. Other work shows that the
relative importance of the intellibility types may vary with the domain; for
example, findings by Castelli et al. [4] in the domain of smart homes showed a
strong interest in “What” questions, but few of the other intellibility types.

| Tournament | Shoutcasters | Players | Game
---|---|---|---|---
1 | 2017 IEM Katowice | ToD and PiG | Neeb vs Jjakji | 2
2 | 2017 IEM Katowice | Rotterdam and Maynarde | Harstem vs TY | 1
3 | 2017 GSL Season 1 Code S | Artosis and tasteless | Soo vs Dark | 2
4 | 2016 WESG Finals | Tenshi and Zeweig | DeMuslim vs iGXY | 1
5 | 2017 StarLeague S1 Premier | Wolf and Brendan | Innovation vs Dark | 1
6 | 2016 KeSPA Cup | Wolf and Brendan | Maru vs Patience | 1
7 | 2016 IEM Geonggi | Kaelaris and Funka | Byun vs Iasonu | 2
8 | 2016 IEM Shanghai | Rotterdam and Nathanias | ShowTime vs Iasonu | 3
9 | 2016 WCS Global Finals | iNcontroL and Rotterdam | Nerchio vs Elazer | 2
10 | 2016 DreamHack Open Leipzig | Rifkin and ZombieGrub | Snute vs ShowTime | 3
Table 1: Summary of StarCraft 2 games studied. Please consult our
supplementary materials for transcripts and links to videos.

Constructing effective explanations of AI is not straightforward, especially
when the underlying AI system is complex. Both Kulesza et al. [13] and
Guestrin et al. [27] point to a potential trade-off between _faithfulness_ and
_interpretability_ in explanation. The latter group developed an algorithm
that can explain (in a “black box” fashion) predictions of any classifier in a
faithful way, and also approximate it locally with an interpretable model. In
their work, they described the fidelity-interpretability trade-off, in which
making an explanation more faithful was likely to reduce its interpretability,
and vice versa. However, humans manage this trade-off by accounting for many
factors, such as the audience’s current situation, their background, amount of
time available, etc. One goal of the current study is to understand how expert
human explainers, like our shoutcasters, manage this trade-off.

In the domain of assessing RTS intelligent agents, Kim et al. [12] invited 20
experienced players to assess the skill levels of AI bots playing StarCraft.
They observed that human rankings were different in several ways to a ranking
computed from the bots’ competition win rate, because humans weighed certain
factors like decision-making skill more heavily. The mismatch between
empirical results and perception scores may be because AI bots that are
effective against each other proved less effective against humans.

Cheung et al. [5] studied StarCraft from a different perspective, that of non-
participant spectators. Their investigations produced a set of nine personas
that helped to illuminate _who_ these spectators are and _why_ they watch.
Since shoutcasters are one of the personas, they discussed how shoutcasters
affect the spectator experience and how they judiciously decide how and when
to reveal different types of information, both to entertain and inform the
audience.

The closest work to our own is Metoyer et al.’s [19] investigation into the
vocabulary and language structure of explaining RTS games. In their study,
novices and experts acted in pairs, with the novice watching the expert play
and providing questions, while the expert thought aloud and answered
questions. They developed qualitative coding schemes of the content and
structure of the explanations the expert players offered. Our investigation is
subtly different in that our explainers are expert _communicators_ about the
game and must _anticipate_ audience questions on their own. However, given the
pertinence of their work, we modified their code set to analyze shoutcasters’
utterances.

##  3 Methodology

Figure 2:  A screenshot from an analyzed game, modified to highlight the
patches available to our casters: _HUD_ [1, bottom] (Information about current
game state, e.g., resources held, income rate, supply, and upgrade status);
_Minimap_ [2, lower left] (Zoomed out version of the main window); _“Tab”_ [3,
top left] (Provides details on demand, currently set on “Production”);
_Workers killed_ [4, center left] (Shows that 9 Red workers have died
recently); _Popup_ [5, center] (visualizations that compare player
performance, usually shown briefly). Regions 3 and 5 will be detailed in
Figures 4 and 5.

In order to study high quality explanations and capable players, we considered
only games from professional tournaments denoted as “Premier” by
TeamLiquid111http://wiki.teamliquid.net/starcraft2/Premier_Tournaments. Using
these criteria, we selected 10 matches available with video on demand from
professional StarCraft 2 tournaments from 2016 and 2017 (Table 1).
Professional matches have multiple games, so we randomly selected one game
from each match for analysis. 16 distinct shoutcasters appeared across the 10
videos, with two casters222Here, caster pair (_caster_ or _pair_ for short)
differentiates our observed individuals from the population of shoutcasters as
a whole. commentating each time.

Shoutcasters should both inform and _entertain_ , so they fill dead air time
with jokes. Thus, we filtered the casters’ utterances by relevance. To do so,
two researchers independently coded 32% of statements in the corpus as
relevant or irrelevant to explaining the game. We achieved a 95% inter-rater
reliability (IRR), as measured by the Jaccard index. (The Jaccard index is the
size of the intersection of the codes applied by the researchers divided by
the size of the union.) Then, the researchers split up and coded the rest of
the corpus.

Research questions RQ1 and RQ2 investigated how the casters seek information
onscreen, so we used IFT constructs to discover the types of information
casters sought and how they unearthed it. For RQ1 (the patches in which they
sought information), we simply counted the casters’ navigations among patches.
Changes in the display screen identified these for us automatically. For RQ2
(_how_ they went about their information foraging), we coded the 110 instances
of caster navigation by the context where it took place, based on player
actions — Building, Fighting, Moving, Scouting — or simply caster navigation.
Two researchers independently coded 21% of the data in this manner, with IRR
of 80% (Jaccard). After achieving IRR, one researcher coded the remainder of
the data.

For RQ3 (implicit questions the shoutcasters answered), we coded the casters’
utterances by the Lim & Dey [18] questions they answered. We added a judgment
code to capture caster evaluation on the _quality_ of actions. The complete
code set will be detailed in the RQ3 Results section. Using this code set, two
researchers independently coded 34% of the 1024 explanations in the corpus,
with 80% inter-rater reliability (Jaccard). After achieving IRR, the
researchers split up the remainder of the coding.

To investigate RQ4 (explanation content), we drew content coding rules from
Metoyer et al. [19]’s analysis of explaining Wargus games and added some codes
to account for differences in gameplay and study structure. (For ease of
presentation, in this paper we use the terms “numeric quantity” and
“indefinite quantity” instead of their terms “identified discrete” and
“indefinite quantity”, respectively.) Two researchers independently coded the
corpus, one category at a time (e.g., Objects, Actions, …), achieving an
average of 78% IRR on more than 20% of the data in each category. One
researcher then finished coding the rest of the corpus. Since all data sources
are public, we have provided all data and coding rules in supplementary
materials to enable replicability and support further research.

##  4 Results

###  4.1 RQ1 Results: What information do shoutcasters seek to generate
explanations, and where do they find it?

We used two frameworks to investigate casters’ information seeking behaviors.
To situate _what_ information casters sought in a common framework for
conceptualizing intelligent agents, we turned to the Performance, Environment,
Actuators, Sensors (PEAS) model [28]. We drew from Information Foraging Theory
(IFT) to understand _where_ casters did their information seeking, beginning
with the places their desired information could be found. These places are
called information “patches” in IFT terminology.

Table 2 columns 1 and 2 show the correspondence between PEAS constructs and
patches in the game that the casters in our data actually used. Performance
measures showed assets, resources, successes, and failures, e.g., Figure 2
region 4 (showing that Blue has killed 9 of Red’s workers) and region 5
(showing that Blue has killed 19 units to Red’s 3, etc.). Table 2 shows that
casters rarely consulted performance measures, especially those that examined
_past_ game states. However, they discussed basic performance measures
available in the HUD (Figure 2 region 1), which contained _present_ state
information, e.g., resources held or upgrade status.

|  Patch Name | State | Agg. |  Usage |  Pair 1 |  Pair 2 |  Pair 3 |  Pair 4 |  Pair 5 |  Pair 6 |  Pair 7 |  Pair 8 |  Pair 9 |  Pair 10
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
_Performance_ |  Units Lost popup: Shows count and resource value of the units each player has lost. | Past | High | 6 |  |  |  |  |  | 2 | 2 | 1 | 1 |
Units Lost tab: Same as above, but as a tab. | Past | High | 5 | 1 | 1 |  |  | 1 | 1 | 1 |  |  |
Income tab: Provides resource gain rate. | Present | High | 2 |  |  |  | 1 |  | 1 |  |  |  |
Income popup: Shows each player’s resource gain rate and worker count. | Present | High | 2 |  |  |  |  | 1 | 1 |  |  |  |
Army tab: Shows supply and resource value of currently held non-worker units. | Present | High | 1 |  |  |  | 1 |  |  |  |  |  |
Income Advantage graph: Shows time series data comparing resource gain rate. | Past | High | 1 | 1 |  |  |  |  |  |  |  |  |
Units Killed popup: Essentially the opposite of the Units Lost popup | Past | High | 1 | 1 |  |  |  |  |  |  |  |  |
_Environment_ |  Units tab: Shows currently possessed units. | Present | Low | 51 | 1 | 2 | 2 | 10 | 1 | 13 | 20 | 2 |  |
Upgrades tab: Like Units tab, but for upgrades to units and buildings. | Present | Low | 5 |  |  |  | 1 | 3 |  |  | 1 |  |
Structures tab: Like Units tab, but buildings. | Present | Low | 2 | 1 |  |  | 1 |  |  |  |  |  |
_Actuator_ |  Production tab: Shows the units, structures, and upgrades that are in progress, i.e. have been started, but are yet to finish. | Present | Low | Preferred (by choice) “always-on” tab, not counted
_Sensor_ |  Minimap: Shows zoomed out map view | Present | Med | Too many to count
Vision Toggle: Shows only the vision available to _one_ of the players. | Present | Low | 36 | 5 | 8 | 1 | 2 | 1 | 5 | 1 | 7 | 5 | 1
Table 2: This table illustrates description, classification, and usage rates
of the patches and enrichment operations we observed casters using. Each patch
is classified by: 1\. The part of the PEAS model that this patch illuminates
best (column 1), 2\. whether it examines past or present game states (column
3), and 3\. degree to which the patch aggregates data in its visualization
(column 4). The remaining columns show total usage counts, as well as per
caster pair usage. Note that there are additional patches passively available
(Main Window and HUD) which do not have navigation properties.

The Environment, or where the agent is situated, corresponds to the game state
(map, units, structures, etc.), shown in the main window in Figure 2. The
Environment mattered for ambient awareness, but casters navigated to Sensor
and Actuator information most actively, so we turn to those constructs next.

Sensors helped the agent collect information about the environment and
corresponded to the local vision area provided by individual units themselves
in our domain. Figure 2 region 2 (Minimap) shows a “bird’s eye view” of the
portion of the environment observable by the Sensors. Casters used patches
containing information about Sensors very often, with Minimap and Vision
Toggles being among the most used patches in Table 2. The casters had
“superpowers” with respect to Sensors (and performance measures) — their
interface allowed _full_ observation of the environment, whereas players could
only _partially_ observe it. As the only ways for casters to peer through the
players’ sensors, the casters extensively used the Minimap and the Vision
Toggle.

Actuators were the means for the agents to interact with their environment,
such as building a unit. Figure 2 region 3 (Production Tab) shows some of the
actuators the player was using, namely that Player Blue was building 5 types
of objects, whereas Red was building 8. Casters almost always kept
visualizations of actions _in progress_ on display. RTS actions had a
_duration_ , meaning that when a player took an action, time passed before its
consequence had been realized. The Production tab’s popularity was likely due
to the fact that it is the _only stable_ view of information about actuators
and their associated actions.

In fact, prior to the game in our corpus, Pair 3 had this exchange, which
demonstrated their perception of the production tab’s importance to doing
their job,

> Pair 3a: _“What if we took someone who knows literally nothing about
> StarCraft, just teach them a few phrases and _what everything is on the
> production tab_?”
> _Pair 3b:_ “Oh, I would be out of a job.”_

####  4.1.1 Implications for an interactive explainer

Abstracting beyond the StarCraft components to the PEAS model revealed a
pattern of the casters’ behaviors with implications for future explanation
systems, which we characterize as: “keep your Sensors close, but your
Actuators closer.” This aligns with other research, showing that real-time
visualization of agent actions can improve system transparency [34].

However, these results contrast with many of today’s explanation systems,
which tend to prioritize Performance measures, but the Actuators and Sensors
seemed to form the very core of these expert explainers’ information seeking
for presentation to their audience. Our results instead suggest that an
explanation system should prioritize useful, readily accessible information
about what an agent did or can do (Actuators) and of what it can see or has
seen (Sensors).

###  4.2 RQ2 Results: The How: How do shoutcasters seek the information they
seek?

Section RQ1 discussed the What and Where (i.e., the content casters sought and
locations where they sought it.) We then considered how they decided to move
among these places.

The casters’ foraging moves seemed to follow a common foraging “loop” through
the available information patch types: an Actuators-Environment-Performance
loop with occasional forays over to Sensors (Figure 3). Specifically, the
casters tended to start at the “always-on” Actuator-related patches of current
state’s _actions in-progress_ ; then when something triggered a change in
their focus, they checked the Environment for current game state information
and occasionally Performance measures of past states. If they needed more
information along the way, they went to the Sensors to see through a player’s
eyes. We will refer to this process as the “A-E-P+S loop”.

Figure 3: The A-E-P+S loop was a common information foraging strategy some
casters used in foraging for agent behavior. It starts at the Actuators, and
returns there throughout the foraging process. If a caster interrupted the
loop, they usually did so to return to the Actuators.

To help derive what caused casters to leave Actuator patches, which seemed to
have so much importance to them, Information Foraging Theory (IFT) explains
why people (information predators) leave one patch to move to another as a
cost/benefit decision, based on the value of information in the patch a
predator is already in versus the value per cost of going to another patch
[26]. Staying in the same patch is generally the least expensive, but when
there is less value to be gained by staying versus moving to another patch,
the predator moves to the other patch. However, the predator is not
omniscient: decisions are based upon the predator’s _perception_ of the cost
and value that other patches will actually deliver. They formed these
perceptions from both their prior experience with different patch types [24]
and from the cues (signposts in their information environment) that provided
concise information about content available in other patches. For the casters,
certain types of cues tended to trigger a move.

Figure 4:  The Units Lost tab (left image) shows the number of units lost and
their total value, in terms of resources spent, for both players. In this
example from Pair 2, we see that Blue Player (top) has lost merely 2800
minerals worth of units so far in this game, while Red has lost more than
7000. The Units Killed popup (right image) allows shoutcasters to quickly
compare player performance via a “tug-of-war” visualization. In this example
from Pair 1, as we see that Blue Player (left) has killed 19 units, while Red
has killed merely 3. The main difference between these two styles of
visualization is that the tab offers more options and information depth to
“drill down” into.

Impending combat was the most common cue triggering a move from the Actuators
type (Production tab) to the Environment type (Units tab) — i.e., from A to E
in the A-E-P+S loop. In Figure 3, the cue was co-located, opposing units,
indicative of imminent combat, which led to caster navigation to a new patch
to illuminate the environment. In fact, combat cues triggered navigations to
the Units tab most frequently, accounting for 30 of the 51 navigations there
(Table 2).

Interestingly, this cue type was different from the static cues most prior IFT
research has used. In previous IFT investigations, cues tended to be static
decorations (text or occasionally images) that label a navigation device, like
a hyperlink or button that leads to another information patch. In contrast,
cues like the onset of combat are dynamic and often did not provide an
affordable direct navigation. However, cues like this were considered cues
because they “provide users with concise information about content that is not
immediately available” [26]. In the case of combat, they suggested high value
in another location, namely the Units tab.

Combat ending was a dynamic cue that triggered a move to a Performance
measure. Of the 13 navigations to a past-facing Performance measure (via tab
or popup), 10 occurred shortly after combat ended as a form of “after-action
review.” Occasionally, the shoutcasters visited other Performance patches,
such as the Income, Units Lost, and Army tabs, to demonstrate reasons why a
player had accrued an in-game lead, or the magnitude of that lead (7
navigations). However, signs of completed fighting were the main cues for
visiting a Performance patch.

Figure 5: The Production tab, showing the build actions currently in progress
for each player. Each unit/structure type is represented by a glyph (which
serves as a link to navigate to that object), provided a progress bar for
duration, and given the number of objects of that type. Thus, we can see that
Blue Player (top row) is building 5 different types of things, while Red
(bottom row) is building 4 types of things. The Structures, Upgrades, and
Units tab look fairly similar to the Production tab.

The most common detour out of the A-E-P part of the loop to a Sensor patch was
to enrich the information environment via the Vision Toggle (36 navigations).
The data did not reveal exactly what cue(s) led to this move, but the move
itself had a common theme: to assess scouting operations. The casters used the
Vision Toggle to allow themselves to see the game through the eyes of only
_one_ of the players, but their default behavior was to view the game with ALL
vision. This provided the casters with the ability to observe _both_ players’
units and structures simultaneously. Toggling the Vision Sensor in this way
enabled them to assess what information was or had been gathered by each
player via their scouting actions (29 of the 36 total Vision Toggles), since
an enemy unit would only appear to the player’s sensors if they had a friendly
unit (e.g., a scout) nearby. Toggling the vision Sensor was the second most
common patch move.

Besides the act of following cues, IFT has another foraging operation:
_enriching_ their information environment to make it more valuable or cost-
efficient [26]. The aforementioned Vision Toggle was one example of this, and
another was when casters added on information visualizations derived from the
raw data, like Performance measure popups or other basic visualizations. Two
examples of the data obtained through this enrichment is shown in Figure 4.

These Performance measures gave the shoutcasters at-a-glance information about
the ways one player was winning. For example, the most commonly used tab,
Units Lost tab (Figure 4) showed the number of units lost and their total
value, in terms of resources spent. This measure achieves “at a glance” by
aggregating _all_ the data samples together by taking a _sum_ ; derived values
like this allow the visualization to scale to large data sets [29]. However,
Table 2 indicates that the lower data aggregation patches were more heavily
used. As Figure 5 shows, the casters used the Production tab to see units
grouped by type, so _type_ information was maintained with only _positional_
data lost. This contrasts with the Minimap (medium aggregation), in which type
information is discarded but positional information maintained at a lower
_granularity_. The casters used Performance measure patches primarily to
understand present state data (HUD), but these patches were also the only way
to access _past_ state information (Table 2).

####  4.2.1 Implications for an interactive explainer

These results have several implications for automated explanation systems in
this domain. First, the A-E-P+S loop and how the casters traversed it reveals
priority and timing implications for automated explanation systems. For
example, the cues that led them to switch to different information patches
could also be cues in an automated system about the need to avail different
information at appropriate times. For example, our casters showed a strong
preference for actuator information as “steady state” visualization, but
preferred performance information upon conclusion of a subtask.

Viewing the casters’ behaviors through the dual lens of PEAS + IFT has
implications for not only the kinds of patches that an explanation system
would need to provide, but also the cost to users of not providing these
patches in a readily accessible format. For example, PEAS + IFT revealed a
costly foraging problem for the casters due to the relative inaccessibility of
some Actuator patches. In StarCraft, there is no easily accessible mechanism
by which they could navigate to an Actuator patch with fighting or scouting
actions in progress.

Instead, the only way the casters could get access to these actions was via
_painstaking_ camera placement. To accomplish this, the casters made countless
navigations to move the camera using the Minimap, traditional scrolling, or
via tabs with links to the right unit or building. But despite all these
navigation affordances, sometimes the casters were unable to place the camera
on all the actions they needed to see.

For example, at one point when Pair 4 had the camera on a fight at Xy’s base,
a second fight broke out at DeMuslim’s base, which they completely missed:

> Pair 4a: _< surprised, noticing something amiss>
> “Xy actually killed the 3rd base of DeMuslim.”
> …<the pair tries to figure what must have happened>…
> _Pair 4b:_ “Oh my god, you’re right Alex.”
> _Pair 4a:_ “Yeah, it was killed during all that action.”_

###  4.3 RQ3 Results: What implicit questions do shoutcasters answer and how
do they form their answers?

Figure 6: Frequency of Lim & Dey questions answered by casters, with one line per caster pair. Y-Axis represents percentages of the utterances which answered that category of question (X-Axis). Note how casters structured answers consistently.  Code |  Freq |  Description |  Example
---|---|---|---
What |  595 |  What the player did or anything about game state |  “The liberators are moving forward as well”
What-could- |  376 |  What the player could have done or what will happen |  “Going to be chasing those medivacs away”
happen |  |  |
How-to |  233 |  Explaining rules, directives, audience tips, high level strategies |  “He should definitely try for the counter attack right away”
*How-good/bad- |  112 |  Evaluation of player actions |  “Very good snipe there for Neeb”
was-that-action |  |  |
Why-did |  27 |  Why the player performed an action |  “…that allowed Dark to hold onto that 4th base, it allowed him to get those ultralisks out”
Why-didn’t |  6 |  Why the player did not perform an action |  “The probe already left a while ago, so we knew it wasn’t going to be a pylon rush”
Table 3: Utterance type code set, slightly modified from the schema proposed
by Lim & Dey. The asterisk denotes the code that we added, How-good/bad-was-
that-action because the casters judged actions based on their quality.

For the first two research questions, we considered how the shoutcasters
gathered and assessed information. We now shift focus to the explanations
themselves.

Much of the prior research into explaining agent behavior starts at some kind
of observable effect and then explains something about that effect or its
causes [11, 13, 17, 32]. In RTS games, most such observable effects are the
result of player actions, and recall from RQ1 that the casters spent most of
their information-gathering effort examining the players’ Actuators to
discover and understand actions.

The casters used the information they gained to craft explanations to answer
implicit questions (i.e., questions their audience “should be” wondering)
about player actions. Thus, drawing from prior work about the nature of
questions people ask about AI, we coded the 1024 casters’ explanations using
the Lim & Dey “intelligibility types” [18].

The shoutcasters were remarkably consistent (Figure 6) in the types of
implicit questions they answered. As Table 3 sums up, casters overwhelmingly
chose to answer What, with What-could-happen and How-to high on their list.
(The total is greater than 1024 because explanations answered multiple
questions and/or fit into multiple categories.)

These results surprised us. Whereas Lim & Dey [17] found that Why was the most
demanded explanation type from users, the casters rarely provided Why answers.
More specifically, in the Lim & Dey study, approximately 48 of 250
participants, (19%) demanded a Why explanation. To contrast with our study,
only 27 of the casters’ 1024 utterances (approximately 3%) were Why answers.

####  4.3.1 Discussion and implications for an interactive explainer

Why so few Whys? Should an automated explainer, like our shoutcasters, eschew
Why explanations, in favor of What?

One possibility is that the casters delivered exactly what their audience
wanted, and thus the casters’ distribution of explanation types was well
chosen. After all, the casters were experts paid to provide commentary for
prestigious tournaments, so they would know their audience well. The expertise
level of the audience may have been fairly high, because the tournament videos
were available only _on demand_ (as opposed to _broadcast_ like some
professional sports) at websites that casual audience members may not even
know about. If a well-informed audience expected the players to do exactly
what they did, their expectations would not be violated, which, according to
Lim & Dey, suggests less demand for Why [17]. This suggests that the extent to
which an automated explainer needs to emphasize Why explanations may depend on
both the _expertise_ of the intended audience, which drives their
expectations, and the agent’s _competence_ , which drives failure to meet
reasonable expectations.

However, another possibility is that the audience really did want Why
explanations, but the casters rarely provided them because of the time they
required — both theirs and the audience’s. The shoutcasters explained in _real
time_ as the players performed their actions. It takes time to understand the
present, predict the future, and link present to future; and spending time in
these ways reduces the time allowable for explaining interesting activities
happening in present. The corpus showed casters interrupting themselves and
each other as new events transpired, as they tried to keep up with the time
constraints. This also has implications to the audience’s workflow, because it
takes time for the audience to mentally process shoutcasters’ departures from
the present, particularly when interesting actions continuously occur.

Even more critical to an explanation system, Why questions also tend to
require extra effort (cognitive or computing resources), because they require
connecting two time slices:

> Pair 10: _“After seeing the first phoenix and, of course, the second one
> confirmed, Snute is going to invest in a couple spore crawlers.”_

In this example, the casters had to connect past information (scouting the
phoenix, a flying unit) with a prediction of the future (investing in spore
crawlers, an air defense structure).

Answering Why-didn’t questions was even rarer than answering Why questions
(Table 3). Like Why questions, Why-didn’t questions required casters to make a
connection between previous game state and a potential current or future game
state. For example, Pair 2: _“The probe already left a while ago, so we knew
it wasn’t going to be a pylon rush..”_ Why-didn’t answers’ rarity is
consistent with the finding that understanding a Why-didn’t explanation
requires even more mental effort than a Why explanation [18]. As for an
interactive explanation system, supporting Why questions requires solving both
a _temporal credit assignment problem_ (determining the effect of an action
taken at a particular time on the outcome) and a _structural_ one (determining
the effect of a particular system element on the outcome). See [2] for an
accessible explanation of these problems.

Table 4: Occurrence frequencies for each code, as a percent of the total
number of utterances in the corpus. From left to right:  Object (pink),
Action (orange),  Spatial (green),  Temporal (yellow), and  Quantitative
(blue) codes. The casters were consistent about kinds of the content they
rarely included, but inconsistent about the kinds of content they most
favored.

The casters found a potentially “satisficing” approximation of Why, a
combination of What and What-could-happen, the two most frequent explanation
types. Their What answers explained what the player did, what happened in the
game, and description of the game state. These were all things happening in
the present, and did not require the additional cognitive steps required to
answer Why or Why-didn’t, which may have contributed to its high frequency.
Further, the audience needed this kind of “play-by-play” information to stay
informed about the game’s progression; for example, Pair 4: _“This one hero,
marine, is starting to kill the vikings.”_ When adding on What-could-happen,
casters were pairing What with what the player will or could do, i.e., a
hypothetical outcome. For example,

> Pair 1: _“…if he gets warning of this he’ll be able to get back up behind
> his wall in.”_

Although answering the question What-could-happen required predicting the
future, it did not also require the casters to tie together information from
_past_ and future.

The other two frequent answers, How-good/bad-was-that-
action and How-to, also sometimes contained “why” information. For How-
good/bad-was-that-action, casters _judged_ an action e.g.: Pair 1: _“Nice
maneuver from Jjakji, he knows he can’t fight Neeb front on right now, he
needs to go around the edges.”_ For How-to, casters gave the audience tips and
explained high level strategies. For example, consider this rule-like
explanation, which implies the reason “why” the player used a particular army
composition: Pair 10: _“Roach ravager in general is really good…”_

The next rule-like How-to example is an even closer approximation to “why”
information. Pair 8: _“Obviously when there are 4 protoss units on the other
side of the map, you need to produce more zerglings, which means even fewer
drones for Iasonu.”_ In this case, the casters are giving a rule: given a
general game state (protoss units on their side of the map) the player should
perform an action (produce zerglings). But the example does more; it also
implies a Why answer to the question “Why isn’t Iasonu making more drones?”
Since this implied answer simply relates the present to a rule or best
practice, it was produced at much lower expense than a true Why answer that
required tying past events to the present.

Mechanisms casters used to circumvent the need for disruptive and resource-
intensive Why explanations, such as using How-to, may also be ways to
alleviate the same problems in explanation systems.

###  4.4 RQ4 Results: What relationships and objects do shoutcasters use when
building their explanations?

To inform future explanation systems’ content by expert explanations — the
patterns of nouns, verbs, and adjectives/adverbs in these professionally
crafted explanations — we drew upon a code set from prior work [19] (see the
Methodology section). Table 4 shows how much caster pairs used each of these
types of content, grouping the objects (nouns) in the first group of columns,
then actions (verbs), and then properties (adjectives and adverbs). Table 5
shows how the casters’ explanations used these concepts _together_ , i.e.,
which properties they paired with which objects and actions.

The casters’ explanation sentences tended to be noun-verb constructions, so we
began with the nouns. The most frequently described objects were  fighting
object,  production object, and  enemy, with frequencies of 53%, 40%, and 9%,
respectively, as shown in Figure 4. (This is similar to results from [19],
where production, fighting, and enemy objects were the three most popular
object subcodes.) As to the actions (“verbs”), the casters mainly discussed
fighting (40%) and  building (23%). It is not surprising that the casters
frequently discussed  fighting, since combat skills are important in StarCraft
[12], and  producing is often a prerequisite to  fighting. This may suggest
that, in RTS environments, an explanation system may be able to focus on only
the most important subset of actions and objects, without needing to track and
reason about most of the others.

The casters were quite strategic in how they put together these nouns and
verbs with properties. The casters’ used particular properties with these
nouns and verbs to paint the bigger picture of how the game was going for each
player, and how that tied to the players’ strategies. We illustrate in the
next subsections a few of the ways casters communicated about player decisions
— succinctly enough for real time.

##### “This part of the map is mine!”: Spatial properties

RTS players claim territory in battles with the  arrangement of their military
units, e.g.:

> Pair 3: _“He’s actually arcing these roaches out in such a great way so that
> he’s going to block anything that’s going to try to come back.”_

As the  arrangement column of Table 5 shows, the objects that were used most
with  arrangment were  fighting objects (12%, 72 instances) and  enemy, (10%,
26 instances). Note that  arrangement is very similar to  point/region, but at
a smaller scale;  Arrangement of  production object, such as exactly where
buildings are placed in one’s base, appeared to be less significant, co-
occurring only 5% of the time.

The degree to which an RTS player wishes to be aggressive or passive is often
evident in their choice of what  distance to keep from their opponent, and the
casters often took this into account in their explanations. One example of
this was evaluation of potential new base locations.

> Pair 5: _“…if he takes the one [base] thats closer that’s near his natural
> [base], then it’s close to Innovation so he can harass.”_

Here, the casters communicated the control of parts of the map by describing
_bases_ as a  region, and then relating two regions with a  distance. The
magnitude of that distance then informed whether the player was able to more
easily attack. Of the casters’ utterances that described  distance along with
production object, 27 out of 44 referred to the distance between bases or
moving to/from a base.

##### “When should I…”: Temporal properties

Casters’ explanations often reflected players’ priorities for allocating
limited resources. One way they did so was using  speed properties: Pair 4:
_“We see a really quick third [base] here from XY, like five minutes third.”_
Since extra bases provide additional resource gathering capacity, the audience
could infer that the player intended to follow an “economic” strategy, as
those resources could have otherwise been spent on military units or upgrades.
This contrasts with the following example, Pair 8: _“He’s going for very fast
lurker den…”_ The second example indicated the player’s intent to follow a
different strategy: unlocking stronger units (lurkers).  Speed co-occurred
with  building/producing most often (12%, 36 instances).

Table 5: Co-Occurrence Matrix. Across rows:  Object (pink, top rows) and
Action (orange, bottom rows) codes. Across columns:  Spatial (green, left),
Temporal (yellow, center), and  Quantitative (blue, right). Co-occurrence
rates were calculated by dividing the intersection of the subcodes by the
union.

##### “Do I care how many?”: Quantitative properties

We found it surprising how often the casters described quantities without
numbers. In fact, the casters often did not even include _type_ information
when they described the players’ holdings, instead focusing on  comparative
properties (Table 5). For example, Pair 1: _“There is too much supply for him
to handle. Neeb finalizes the score here after a fantastic game.”_ Here,
“supply” is so generic, we do not even know what kind of things Neeb had –
only that he had “too much” of it.

In contrast, when the casters discussed cheap military units, like “marines”
and “zerglings,” they tended to provide _type_ information, but about half of
their mentions still included no precise numbers. Perhaps it was a matter of
the high cost to get that information: cheap units are often built in large
quantities, so deriving a precise quantity is often very tedious. Further,
adding one weak unit that is cheap to build has little impact on army
strength, so getting a precise number may not have been worthwhile – i.e. the
value of knowing precise quantities is low. To illustrate, consider the
following example, which quantified the army size of both players vaguely,
using _indefinite quantity_ properties: Pair 6: _“That’s a lot of marines and
marauders and not enough stalkers.”_

In the RTS domain, workers are a very important unit. Consistent with this
importance, workers are the only unit where the casters were automatically
alerted to their death (Figure 2, region 4), and are also available at a
glance on the HUD (Figure 2 region 1). Correspondingly, the casters often gave
precise quantities of workers (a  production object). Workers (workers,
drones, scvs, and probes) had 46 co-occurrences with  numeric quantities, but
only 12 with  indefinite quantities (e.g., lot, some, few). Pair 2: _“…it
really feels like Harstem is doing everything right, and [yet] somehow ended
up losing 5 workers.”_

####  4.4.1 Implications for an interactive explainer

These results have particularly important implications for interactive
explanation systems with real-time constraints. Namely, the results suggest
that an effective way to communicate about strategies and tactics is to modify
the critical objects and actions with particular properties that suggest
strategies. This not only affords a _succinct_ way to communicate about
strategies and tactics (fewer words) but also a _lighter load_ for both the
system and the audience than attempting to build and process a rigorous
explanation of strategy.

Specifically, spatial properties can communicate beyond the actual properties
of objects to strategies themselves; for example, casters used distance to
point out plans to attack or defend. Temporal properties can be used in
explanations of strategies when choices in resource allocation determines
available strategies.

Finally, an interactive explanation system could use the quantitive property
results to help ensure alignment in the level of abstraction used by the human
and the system. For example, a player can abstract a quantity of units into a
_single group_ or think of them as _individual units_. Knowing the level of
abstraction the human players use in different situations can help an
interactive explanation system choose the level of abstraction that will meet
human expectations. Using properties in these strategic ways may enable an
interactive explanation system to meet its real-time constraints while at the
same time improving its communicativeness to the audience.

##  5 Conclusion

The results of our study suggest that explaining intelligent agents to humans
has much to gain from looking to the human experts. In our case, the expert
explainers — RTS shoutcasters — revealed implications into what, when, and how
human audiences of such systems need explanations, and how real-time
constraints can come together with explanation-building strategies. Among the
results we learned were:

  1. RQ1

Investigating the what’s and where’s of casters’ real-time information
foraging to _assess and understand_ the players showed that the most commonly
used _patches_ of the information environment were the Actuators (“A” in the
PEAS model). This suggests that, in contrast to today’s explanation systems,
which tend to present mostly Performance measures, explanations should
consider presenting more from the Actuators and Sensors.

  2. RQ2

The how’s of casters’ foraging revealed a common pattern, which we termed the
A-E-P+S loop, and the most common cues and triggers that led shoutcasters to
move through this loop. Future explanation systems may be well-served to
prioritize and recommend explanations according to this loop and its triggers.

  3. RQ3

As _model explainer_ , the casters revealed strategies for “satisficing” with
explanations that may not have precisely answered all the questions the
audience had in mind, but were feasible given the time and resource
constraints in effect when comprehending, assessing, and explaining, all in
_real time_ as play progresses. These strategies may be likewise applicable to
interactive explanation systems.

  4. RQ4

The detailed contents of the casters’ explanations revealed patterns of how
they paired properties (“adjectives and adverbs”) with different objects
(“nouns”) and actions (“verbs”). Interactive explanation systems may be able
to leverage these patterns to communicate succinctly about an agent’s tactics
and strategies.

Ultimately, both shoutcasters’ and explanation systems’ jobs are to improve
the audience members’ mental model of the agents’ behavior. As Cheung, et al.
[5] put it, “…commentators are valued for their ability to expose the depth of
the game.” Hopefully, future explanation systems will be valued for the same
reason.

##  6 Acknowledgments

Removed for anonymous review.

## References

  * [1]
  * [2] Adrian K Agogino and Kagan Tumer. 2004.  Unifying temporal and structural credit assignment problems. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 2. IEEE Computer Society, 980–987.
  * [3] Svetlin Bostandjiev, John O’Donovan, and Tobias Höllerer. 2012.  TasteWeights: A visual interactive hybrid recommender system. In Proceedings of the Sixth ACM Conference on Recommender Systems. ACM, 35–42.
  * [4] Nico Castelli, Corinna Ogonowski, Timo Jakobi, Martin Stein, Gunnar Stevens, and Volker Wulf. 2017.  What happened in my home?: An end-user development approach for smart home data visualization. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 853–866.
  * [5] Gifford Cheung and Jeff Huang. 2011.  Starcraft from the stands: Understanding the game spectator. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’11). ACM, New York, NY, USA, 763–772.  DOI:http://dx.doi.org/10.1145/1978942.1979053
  * [6] Ed H Chi, Peter Pirolli, Kim Chen, and James Pitkow. 2001.  Using information scent to model user information needs and actions and the web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 490–497.
  * [7] Kelley Cotter, Janghee Cho, and Emilee Rader. 2017.  Explaining the news feed algorithm: An analysis of the “News Feed FYI” blog. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1553–1560.
  * [8] Scott D. Fleming, Chris Scaffidi, David Piorkowski, Margaret Burnett, Rachel Bellamy, Joseph Lawrance, and Irwin Kwan. 2013.  An information foraging theory perspective on tools for debugging, refactoring, and reuse tasks.  ACM Transactions on Software Engineering and Methodology (TOSEM) 22, 2 (2013), 14.
  * [9] Wai-Tat Fu and Peter Pirolli. 2007.  SNIF-ACT: A cognitive model of user navigation on the world wide web.  Human-Computer Interaction 22, 4 (2007), 355–412.
  * [10] Robert R Hoffman and Gary Klein. 2017.  Explaining explanation, part 1: theoretical foundations.  IEEE Intelligent Systems 32, 3 (2017), 68–73.
  * [11] Ashish Kapoor, Bongshin Lee, Desney Tan, and Eric Horvitz. 2010.  Interactive optimization for steering machine classification. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1343–1352.
  * [12] Man-Je Kim, Kyung-Joong Kim, SeungJun Kim, and Anind K Dey. 2016.  Evaluation of starcraft artificial intelligence competition bots by experienced human players. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1915–1921.
  * [13] Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015\.  Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of the 20th International Conference on Intelligent User Interfaces. ACM, 126–137.
  * [14] Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012.  Tell me more?: the effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1–10.
  * [15] Sandeep Kaur Kuttal, Anita Sarma, and Gregg Rothermel. 2013.  Predator behavior in the wild web world of bugs: An information foraging theory perspective. In Visual Languages and Human-Centric Computing (VL/HCC), 2013 IEEE Symposium on. IEEE, 59–66.
  * [16] Jason Levine. 2017.  Americans are right not to trust self-driving cars.  (2017).
  * [17] Brian Y Lim and Anind K Dey. 2009.  Assessing demand for intelligibility in context-aware applications. In Proceedings of the 11th international conference on Ubiquitous computing. ACM, 195–204.
  * [18] Brian Y. Lim, Anind K. Dey, and Daniel Avrahami. 2009.  Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2119–2128.
  * [19] Ronald Metoyer, Simone Stumpf, Christoph Neumann, Jonathan Dodge, Jill Cao, and Aaron Schnabel. 2010.  Explaining how to play real-time strategy games.  Knowledge-Based Systems 23, 4 (2010), 295–301.
  * [20] Nan Niu, Anas Mahmoud, Zhangji Chen, and Gary Bradshaw. 2013.  Departures from optimality: Understanding human analyst’s information foraging in assisted requirements tracing. In Proceedings of the 2013 International Conference on Software Engineering. IEEE Press, 572–581.
  * [21] Donald A Norman. 1983.  Some observations on mental models.  Mental models 7, 112 (1983), 7–14.
  * [22] S. Ontañón, G. Synnaeve, A. Uriarte, F. Richoux, D. Churchill, and M. Preuss. 2013.  A survey of real-time strategy game AI research and competition in StarCraft.  IEEE Transactions on Computational Intelligence and AI in Games 5, 4 (Dec 2013), 293–311.  DOI:http://dx.doi.org/10.1109/TCIAIG.2013.2286295
  * [23] Alexandre Perez and Rui Abreu. 2014.  A diagnosis-based approach to software comprehension. In Proceedings of the 22nd International Conference on Program Comprehension. ACM, 37–47.
  * [24] David Piorkowski, Scott D. Fleming, Christopher Scaffidi, Margaret Burnett, Irwin Kwan, Austin Z Henley, Jamie Macbeth, Charles Hill, and Amber Horvath. 2015.  To fix or to learn? How production bias affects developers’ information foraging during debugging. In Software Maintenance and Evolution (ICSME), 2015 IEEE International Conference on. IEEE, 11–20.
  * [25] David Piorkowski, Austin Z Henley, Tahmid Nabi, Scott D Fleming, Christopher Scaffidi, and Margaret Burnett. 2016.  Foraging and navigations, fundamentally: Developers’ predictions of value and cost. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 97–108.
  * [26] Peter Pirolli. 2007.  Information foraging theory: Adaptive interaction with information.  Oxford University Press.
  * [27] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016.  Why should I trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1135–1144.
  * [28] Stuart J. Russell and Peter Norvig. 2003.  Artificial Intelligence: A modern approach (2 ed.).  Pearson Education.
  * [29] Robert Spence. 2007.  Information Visualization: Design for interaction (2Nd Edition).  Prentice-Hall, Inc., Upper Saddle River, NJ, USA.
  * [30] Sruti Srinivasa Ragavan, Sandeep Kaur Kuttal, Charles Hill, Anita Sarma, David Piorkowski, and Margaret Burnett. 2016.  Foraging among an overabundance of similar variants. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 3509–3521.
  * [31] Katia Sycara, Christian Lebiere, Yulong Pei, Donald Morrison, and Michael Lewis. 2015.  Abstraction of analytical models from cognitive models of human control of robotic swarms. In International Conference on Cognitive Modeling. University of Pittsburgh.
  * [32] Joe Tullio, Anind K Dey, Jason Chalecki, and James Fogarty. 2007.  How it works: A field study of non-technical users interacting with an intelligent system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 31–40.
  * [33] Oriol Vinyals. 2016.  DeepMind and blizzard to release starcraft II as an AI research environment.  (2016).  https://deepmind.com/blog/deepmind-and-blizzard-release-starcraft-ii-ai-research-environment/
  * [34] Robert H Wortham, Andreas Theodorou, and Joanna J Bryson. 2017.  Improving robot transparency:real-time visualisation of robot AI substantially improves understanding in naive observers, In IEEE RO-MAN 2017. IEEE RO-MAN 2017 (August 2017).  http://opus.bath.ac.uk/55793/