# Efficient model selection in switching linear dynamic systems

###### Abstract

The computation required for a switching Kalman Filter (SKF) increases
exponentially with the number of system modes. In this paper, a systematic
graph representation for a switching linear dynamic system (SLDS) is proposed
along with a minimum-sum clustering method to reduce the switching mode
cardinality offline, before collecting measurements. It is shown that if mode
detection is perfect then the induced mode clustering error can be quantified
exactly. The simulation results verify that clustering based on the proposed
framework effectively reduces model complexity given uncertain mode detection
and that the induced error can be well approximated.

Index Terms—  Switching linear dynamic systems, Switching Kalman filter,
Recursive estimation, Model mismatch, Model complexity.

##  1 Introduction

The estimation of a dynamical system state from a collection of indirect,
noisy measurements is a pervasive problem found in virtually all branches of
the physical sciences and engineering, e.g., [1, 2, 3, 4]. Diverse
applications ranging, e.g., from financial forecasting, multiple target
tracking, and neural pattern discrimination [5, 6, 7] are faithfully
represented as a switching linear dynamic system (SLDS) obtained by augmenting
hidden discrete random variables to a linear state space model [8]. Joint
estimation and detection of the hidden system mode can be accomplished with a
switching Kalman filter (SKF) [9, 10, 11].

The computational burden of a SKF increases exponentially with the number of
system modes. In this work, we reduce the number of system modes by combining
those that minimize the ultimate impact on the estimation error. Our approach
involves a systematic approach to model an SLDS as a fully connected graph
based on information determined using online/batch system identification
algorithms[12, 13, 14, 15]. Node clustering, informed by the estimation error
tradeoff, is then used to reduce the system mode cardinality.

Our approach directly relates to efficient mode selection for a Gaussian
mixture (GM) distribution and many algorithms exist for this task. For
example, GM parameters may be learned from data using expectation-maximization
(EM), and the Bayesian information criterion (BIC) and Akaike information
criterion (AIC) may be used to choose the number of modes[16]. Other
approaches range from adhoc measure-based clustering[17, 18] and covariance
union methods [19] to clustering using KL divergence-based measures[20]. This
paper addresses the more difficult problem of GM mode selection in the dynamic
(spatial and temporal) as opposed to static (spatial-only) context.

The remainder of the paper is organized as follows. The SLDS signal model and
KF/SKF formulations are reviewed in Sections 2 and 3, respectively. Section 4
derives estimation error for mismatched KFs and SKFs and Section 5 introduces
the graph structure. The effect of clustering on computational complexity is
given in Section 6. Simulations investigate the algorithm performance in
Section 7 and conclusions are given in Section 8.

##  2 Signal model

We adopt the following notational conventions:
• $\bm{x}\sim N(\bm{m},\bm{C})$: the random vector $\bm{x}$ has a Gaussian
distribution with mean $\bm{m}$ and covariance $\bm{C}$.
• $\operatorname{\mathbb{E}},\operatorname{\mathbb{C}},p$ refer to
expectation, covariance, and probability, respectively, $\bm{I}$ refers to the
identity matrix, and superscript $T$ refers to matrix transpoistion.
• $\mathds{1}(S)=1$ if the statement $S$ is true, and it is zero otherwise.

The state-space model for an SLDS is defined as

$\displaystyle\bm{x}_{n}$ $\displaystyle=\bm{A}_{n}\bm{x}_{n-1}+\bm{\nu}_{n}$
(1) $\displaystyle\bm{y}_{n}$
$\displaystyle=\bm{H}_{n}\bm{x}_{n}+\bm{\omega}_{n}$ (2)

where the subscript $n$ is the time step, $\bm{x}_{n}$ is the hidden state
variable to be estimated, and the given model parameters are the measurement
vector $\bm{y}_{n}$, the $z\times z$ evolution matrix
$\bm{A}_{n}\in\\{\bm{A}^{(S_{n})}\\}$ (where $S_{n}=1,...,r$ is the hidden
random variable that represents the system mode and $r$ is the number of
modes), the $m\times z$ (where $m$ is the number of measurements and $z$ is
the state dimension) measurement matrix $\bm{H}_{n}$, and
$\bm{Q}_{n}\in\\{\bm{Q}^{(S_{n})}\\}$ and $\bm{R}_{n}$ (where
$\bm{\nu}_{n}\sim N(0,\bm{Q}_{n})$, $\bm{\omega}_{n}\sim N(0,\bm{R}_{n})$ such
that
$\operatorname{\mathbb{E}}[\bm{\nu}_{n}\bm{\nu}_{n^{\prime}}^{T}]=\bm{Q}_{n}\,\delta(n-n^{\prime})$,
$\operatorname{\mathbb{E}}[\bm{\omega}_{n}\bm{\omega}_{n^{\prime}}^{T}]=\bm{R}_{n}\,\delta(n-n^{\prime})$,
and
$\operatorname{\mathbb{E}}[\bm{\omega}_{n}\bm{\nu}_{n^{\prime}}^{T}]=\bm{0}$).

##  3 State Estimators

The KF is the MAP and MMSE optimal estimator for a linear dynamic system with
Gaussian measurement and process noise. Let
$\bm{y}_{1}^{n}\equiv\\{\bm{y}_{1},\bm{y}_{2},...,\bm{y}_{n}\\}$,
$\bm{x}_{0}\sim N(\bm{x}_{0|0},\bm{P}_{0|0})$,
$\bm{x}_{n|n}=\operatorname{\mathbb{E}}[\bm{x}_{n}|\bm{y}_{1}^{n}]$, and
$\bm{P}_{n|n}=\operatorname{\mathbb{C}}(\bm{x}_{n}|\bm{y}_{1}^{n})$. The
$Filter$ operator is

$(\bm{x}_{n|n},\bm{P}_{n|n})=Filter(\bm{A}_{n},\bm{H}_{n},\bm{x}_{n-1|n-1},\\\
\bm{P}_{n-1|n-1},\bm{Q}_{n},\bm{R}_{n},\bm{y}_{1}^{n})$ (3)

which involves the repeated application of a time update step

$\displaystyle\bm{x}_{n|n-1}$ $\displaystyle=\bm{A}_{n}\bm{x}_{n-1|n-1}$ (4)
$\displaystyle\bm{P}_{n|n-1}$
$\displaystyle=\bm{A}_{n}\bm{P}_{n-1|n-1}\bm{A}_{n}^{T}+\bm{Q}_{n}$ (5)

and a measurement update step

$\displaystyle\bm{K}_{n}$
$\displaystyle=\bm{P}_{n|n-1}\bm{H}_{n}^{T}\bigl{(}\bm{H}_{n}\bm{P}_{n|n-1}\bm{H}_{n}^{T}+\bm{R}_{n}\bigr{)}^{-1}$
(6) $\displaystyle\bm{x}_{n|n}$
$\displaystyle=\bm{x}_{n|n-1}+\bm{K}_{n}\bigl{(}\bm{y}_{n}-\bm{H}_{n}\bm{x}_{n|n-1}\bigr{)}$
(7) $\displaystyle\bm{P}_{n|n}$
$\displaystyle=(\bm{I}-\bm{K}_{n}\bm{H}_{n})\bm{P}_{n|n-1}.$ (8)

The application of a single-mode KF to the general SLDS (1)-(2) results in
erroneous estimates. The well known SKF detects the switching mode at each
time step and its corresponding model parameters ($\bm{A}_{n},\bm{Q}_{n}$)
using a likelihood ratio test at each time step, and estimates the state
variables accordingly. Upon perfect detection of the modes, one can obtain
optimal estimates of the state variable $\bm{x}_{n}$ in terms of both MAP and
MSE metrics; however, due to the exponentially increasing computational cost
of exact SKFs, approximate SKFs based on a finite memory of past trajectories
are used in practice to provide suboptimal estimates.

In this manuscript, we assume the conditional probability of detection given
the correct mode to be known and the studied SKF algorithm through the paper
refers to the MAP estimator, which runs the KF corresponding to the mode with
the largest probability at each time step. Calculating the measures studied in
this paper for the MMSE SKF estimator is much more difficult due to complex,
non-Gaussian distributions that appear in the MMSE estimates. Since
$MSE(\bm{x}_{n|n}^{KF})\geq MSE(\bm{x}_{n|n}^{MAP})\geq
MSE(\bm{x}_{n|n}^{MMSE})$ (where KF refers to a single KF), we claim that
graph edges found using the MMSE estimator will be larger than those obtained
using the MAP estimator considered in this work. Quantifying this increase is
a challenging problem itself and requires further work.

##  4 Estimation Error Quantification

We now calculate the MSE when a mismatched model, i.e., incorrect mode, is
used instead of the ground truth SLDS model before collecting a measurement
and using a MAP estimator. Let $l_{n}$ and $q_{n}$ refer to a sequence of
modes representing the ground truth and detected trajectories, respectively,
where $l_{n}=[l_{n-1},i]$, $q_{n}=[q_{n-1},j]$, and $i,j\in\\{1,...,r\\}$. Let
$\bm{e}_{n}^{(l_{n};q_{n})}=\bm{x}_{n}^{l_{n}}-\hat{\bm{x}}_{n}^{(l_{n};q_{n})}$
be the conditional estimation error term based on these trajectories, which
can be calculated recursively given the trajectories $l_{n}$ and $q_{n}$:

$\displaystyle\bm{e}_{n}^{(l_{n};q_{n})}$
$\displaystyle=([\bm{A}^{i}-\bm{A}^{j}+\bm{K}_{n}^{j}\bm{H}_{n}\bm{A}^{j}-\bm{K}_{n}^{j}\bm{H}_{n}\bm{A}^{i}]\bm{x}_{n-1}^{l_{n-1}}$
$\displaystyle\quad+(\bm{I}-\bm{K}_{n}^{j}\bm{H}_{n})(\bm{A}^{j}\bm{e}_{n-1}^{(l_{n-1};q_{n-1})}+\bm{\nu}_{n})-\bm{K}_{n}^{j}\bm{\omega}_{n})$
(9)

where $\bm{x}_{n}^{l_{n}}$,$\hat{\bm{x}}_{n-1}^{(l_{n};q_{n})}$,
$\bm{e}_{n}^{(l_{n};q_{n})}$, and $\bm{K}_{n}^{j}$ refer to the ground truth
state, the estimated state variable, the estimation error given trajectory
$l_{n}$ occurs and trajectory $q_{n}$ is used for estimation, and the KF gain
assuming mode $j$ is used for estimation at time $n$, respectively. The error
then may be written as

$\displaystyle\bm{e}_{n}=$
$\displaystyle\sum_{l_{n}}\sum_{q_{n}}\delta_{l_{n};q_{n}}\bm{e}_{n}^{(l_{n};q_{n})}$
(10)

where $\delta_{(l_{n};q_{n})}$ equals one when $l_{n}$ occurs and $q_{n}$ is
used in estimation, and 0 otherwise. The mean of this random process is
calculated as

$\displaystyle\operatorname{\mathbb{E}}[\bm{e}_{n}]$
$\displaystyle=\sum_{l_{n}}\sum_{q_{n}}\pi_{l_{n},q_{n}}\operatorname{\mathbb{E}}[\bm{e}_{n}^{(l_{n};q_{n})}],$
(11)

where $\pi_{l_{n},q_{n}}$ is the probability that trajectory $l_{n}$ occurs
and trajectory $q_{n}$ is detected, which may also be calculated recursively.
Similarly, we compute
$\epsilon_{n}=\operatorname{\mathbb{E}}[\bm{e}_{n}^{T}\bm{e}_{n}]=\sum_{l_{n}}\sum_{q_{n}}\pi_{l_{n},q_{n}}\times\operatorname{\mathbb{E}}[\bm{e}_{n}^{(l_{n};q_{n})^{T}}{(\bm{e}_{n}^{(l_{n};q_{n})})}]$
and the covariance
$\operatorname{\mathbb{C}}(\bm{e}_{n})=\operatorname{\mathbb{E}}[\bm{e}_{n}\bm{e}_{n}^{T}]-\operatorname{\mathbb{E}}[\bm{e}_{n}]\operatorname{\mathbb{E}}[\bm{e}_{n}]^{T}$.
The statistics of the error then may be calculated by quantifying and
propagating the mismatch error at each time step recursively. Using this
formulation, one is able to calculate the MSE of applying any mismatched KF to
an SLDS; these MSEs will be used in Section 5. In this work, the detection
rate (or its lower bound) is assumed to be known at each time step. Finding a
lower bound for the detection rate using the likelihood ratio test, as well as
the details on calculating these statistics, are eliminated due to space
restrictions.

##  5 Graph representation of the switching modes

An SLDS may be fully represented by a graph consisting of its switching mode
parameters, initial values, and detection rates. At a given time step, the
SLDS graph representation is as follows. The nodes are the current linear
dynamic system switching parameters, represented by
$\bm{A}^{(i)},\bm{Q}^{(i)},i=1,2,...,r$, as well as their prior probabilities
and transition probabilities at each time step. An edge between every pair of
nodes ($i$ and $j$) is defined as the average of the quantified estimation
error when using one node’s statistics
$\mathcal{S}^{(i)}:\\{\bm{A}^{(i)},\bm{Q}^{(i)}\\}$ or
$\mathcal{S}^{(j)}:\\{\bm{A}^{(j)},\bm{Q}^{(j)}\\}$ instead of both modes
$\mathcal{S}^{(i,j)}:\big{\\{}\\{\bm{A}^{(i)},\bm{Q}^{(i)}\\},\\{\bm{A}^{(j)},\bm{Q}^{(j)}\\}\big{\\}}$
(using this metric is justified based on Theorem 1 and calculated using the
results of Section 4). The edge weight $d(i,j)$ between nodes $i$ and $j$ is
given by

$\displaystyle\epsilon_{i}[n]$
$\displaystyle=\operatorname{\mathbb{E}}\bigl{[}(\bm{x}_{n}-{\bm{x}}_{n|n})^{T}(\bm{x}_{n}-{\bm{x}}_{n|n})\mid\mathcal{S}^{(i)}\bigr{]}$
(12) $\displaystyle\epsilon_{i,j}[n]$
$\displaystyle=\operatorname{\mathbb{E}}\bigl{[}(\bm{x}_{n}-{\bm{x}}_{n|n})^{T}(\bm{x}_{n}-{\bm{x}}_{n|n})\mid\mathcal{S}^{(i,j)}\bigr{]}$
(13) $\displaystyle d{(i,j)}(T_{0})$
$\displaystyle=\sum_{n=1}^{T_{0}}\bigl{(}{\epsilon_{i}[n]-\epsilon_{i,j}[n]}\bigr{)}+\bigl{(}{\epsilon_{j}[n]-\epsilon_{i,j}[n]}\bigr{)}$
(14)

where edges can be calculated for a single time step ($T_{0}=1$; corresponding
to the result obtained in Theorem 1) or accumulated over a few time steps
(which in general must be limited to avoid the resultant combinatoric
explosion in calculating the graph elements). Note that $d{(i,j)}$ is a semi-
metric (no triangle inequality).

Once the graph is constructed using the defined nodes and edges, multiple
questions may be addressed, including: • What is the error when we approximate
the $q$-modal SLDS with a $p$-modal SLDS?
• What is the SLDS with a minimum number of modes that satisfies a condition
on the additional incurred estimation error?

###  5.1 Clustering

Once the graph representation is derived, a lower complexity model can be
determined for an SLDS by clustering the nodes (SLDS modes). When cluster
members are assigned based on the proposed metric, it is possible to quantify
the effect of model complexity reduction in the estimation error. In this
work, we assume a cluster center is randomly chosen from the members of that
cluster.

######  Theorem 1.

Let the MAP estimator of the random variable $\bm{x}$ that uses the ground
truth model including all nodes in the graph be referred to as $\bm{x}^{*}$.
Let the MAP estimator when the ground truth nodes are grouped into $m$
clusters and the center of each cluster is randomly sampled from the nodes in
that cluster based on prior probabilities be referred to as $\hat{\bm{x}}$,
and let $G^{*}=\\{g_{1},...,g_{n}\\}\text{ and
}\hat{G}=\\{g_{1}^{\prime},...,g_{m}^{\prime}\\}$ refer to the set of nodes
for the full graph and clustered graph, respectively. Then, assuming each mode
is exclusively detected by the representative in its cluster and the weight of
the true mode is always greater than the sum of the weights of the other
cluster nodes

$\displaystyle\operatorname{\mathbb{E}}[(\bm{x}-\hat{\bm{x}})^{2}]$
$\displaystyle=\operatorname{\mathbb{E}}[(\bm{x}-\bm{x}^{*})^{2}]$
$\displaystyle\quad\quad+\sum_{i=1}^{m}\sum_{(j,k)\in\\{S_{i}\\}^{2}}\mathds{1}\\{j\neq
k\\}\pi_{j}{\pi_{k}}d(k,j)$

where $\pi_{j}$ refers to the prior probability of each node,
$\\{S_{i}\\}^{2}$ is the set of pairs of nodes in the $i^{th}$ cluster, and
$d(j,k)$ refers to the edge weight between nodes $j$ and $k$.

###### Proof.

$\displaystyle\operatorname{\mathbb{E}}[(\bm{x}-\hat{\bm{x}})^{2}-(\bm{x}-{\bm{x}}^{*})^{2}]$
$\displaystyle=\operatorname{\mathbb{E}}\big{[}\operatorname{\mathbb{E}}[(\bm{x}-\hat{\bm{x}})^{2}-(\bm{x}-{\bm{x}}^{*})^{2}\mid
j:j\in G^{*}]\big{]}$
$\displaystyle=\sum_{j=1}^{n}\pi_{j}\operatorname{\mathbb{E}}[((\bm{x}-\hat{\bm{x}})^{2}-(\bm{x}-{\bm{x}}^{*})^{2}\mid
j]$
$\displaystyle=\sum_{j=1}^{n}\pi_{j}\operatorname{\mathbb{E}}[((\bm{x}-\hat{\bm{x}})^{2}-(\bm{x}-{\bm{x}}^{*})^{2}\mid
j]\mathds{1}\\{j\in\hat{G}\\}$
$\displaystyle\quad+\sum_{j=1}^{n}\pi_{j}\operatorname{\mathbb{E}}[(\bm{x}-\hat{\bm{x}})^{2}-(\bm{x}-{\bm{x}}^{*})^{2}\mid
j]\mathds{1}\\{j\notin\hat{G}\\}$
$\displaystyle=0+\sum_{i=1}^{m}\sum_{j=1}^{n}\mathds{1}\\{j\notin\hat{G}\\}\pi_{j}\operatorname{\mathbb{E}}[(\bm{x}-\hat{\bm{x}})^{2}-(\bm{x}-{\bm{x}}^{*})^{2}\mid
j]\qquad(*)$ $\displaystyle=\sum_{i=1}^{m}\sum_{j\in S_{i}}\sum_{k\in
S_{i}}\mathds{1}\\{j\notin\hat{G}\\}\pi_{j}{\pi_{k\neq
j}}\bigg{(}\operatorname{\mathbb{E}}\bigl{[}(\bm{x}-\hat{\bm{x}})^{2}\mid
j\textit{:occurs, }k\textit{:detected}\bigr{]}$
$\displaystyle\quad-\operatorname{\mathbb{E}}\bigl{[}(\bm{x}-\bm{x}^{*})^{2}\mid
j\textit{:occurs}\bigr{]}\bigg{)}$
$\displaystyle=\sum_{i=1}^{m}\sum_{(j,k)\in\\{S_{i}\\}^{2}}\mathds{1}\\{j\neq
k\\}\pi_{j}{\pi_{k}}d(k,j)$ (15)

where $S_{i}$ is the set of nodes in cluster $i$ and $\hat{\bm{x}}$ refers to
the mismatched estimation. Equation $(*)$ follows from the fact that if the
occured node exists in the clustered model, then $\bm{x}^{*}=\hat{\bm{x}}$
based on the stated assumptions. The last step of the proof follows because
each cluster is randomly replaced with one of the nodes inside that cluster
and $d(k,j)=d(j,k)$. ∎

If the stated conditions are satisfied, then the above theorem shows that the
proposed metric is natural, i.e., based on the estimation error and not some
heuristic approach. If the prior probabilities are the same, the edges are
sufficient to find the optimal choice of clusters. In a more general case
where the mode prior probabilities are different, one needs to minimize the
summation of errors in (15), which makes the clustering rule dependent on the
prior probabilities.

As stated in the Theorem 1, the proposed error quantification is exact given
the stated assumptions. In practice, these assumptions are (approximately)
satisfied when, with a high probability, 1) inter-cluster edges, i.e., those
that connect nodes between different clusters, are much larger than intra-
cluster edges, i.e., those between nodes inside a cluster, and 2) the modes
inside a cluster are well-separated such that they can be confidently detected
using the observations. Obviously, satisfying these assumptions depends on the
distance between the modes as well as the extent (covariance) of each mode.
Since deriving quantitative results on the approximation accuracy needs
extensive further discussion, a heuristic study of the approximate additional
error as well as the required conditions for it to be accurate is provided in
the simulations in Section 7.

##  6 Computation vs. error trade off

Consider a $p$-modal SLDS reduced to a $q$-modal SLDS using clustering. Let
$u$ be the memory of past trajectories used by the SKF. In this scenario, the
obtained computational saving by clustering is equal to running $u^{p}-u^{q}$
KFs. The computational cost of running an $n$-dimensional KF when the number
of measurements is $m$ is $O(n(m^{2})+n^{2}m+n^{3})$. Therefore, the
computational saving obtained by reducing the number of modes to $q<p$ is
$(u^{p}-u^{q})\times O(n(m^{2})+n^{2}m+n^{3})$.

##  7 Numerical experiment

Consider a 60-dimensional state space model simulated based on a diffusion
process. The diffusivity may switch between 4 different values
($\eta_{1}=0.01$, $\eta_{2}=1$, $\eta_{3}=0.19$, and $\eta_{4}=0.71$), the
measurement operator is $\bm{H}=\bm{I}_{60\times 60}$, and
$\bm{Q}_{i}=0.0004\bm{I}_{N\times N}$ and $\bm{R}=0.04\bm{I}_{m\times m}$ (the
average SNR at the first time step is $6.6$ dB). Clustering modes associated
with comparable diffusivities is expected to increase the MSE by only a small
margin relative to the optimal estimator’s MSE, which will be later verified
by the simulation results. We assume there is no prior information on the
occurrence of modes, which means that the mode prior probabilities are equal,
mode transitions are equiprobable, and the state variable’s initial value is a
step function. In order to construct the graph, the distance between each pair
of modes is calculated using the proposed measure for each time step. For
example, at the first time step, the calculated edge matrix $D_{i,j}=d(i,j)$
is:

$\displaystyle D$ $\displaystyle=\begin{bmatrix}0&8.86&0.2&4.28\\\
8.86&0&6.08&0.84\\\ 0.20&6.16&0&2.44\\\ 4.28&0.84&2.44&0\end{bmatrix}$

Clustering of the edge matrix based on the sum of edges inside a cluster
(minimum-sum clustering) shows that the best three mode structures are
$[(1\&3),2,4]$, $[1,3,(2\&4)]$, $[(1\&3),(2\&4)]$. For verification, a data
set is generated by the 4-modal SLDS for $10$ time steps through $1000$ Monte
Carlo simulations. The estimation error using a 4-modal SKF and complexity
reduced SKFs are calculated. The estimation error in Figure 1 shows that the
decision based on the MC simulations is consistent with the decision made
based on the graph edges. The additional estimation error by clustering the
modes for some case studies including
$[(1\&3),2,4],[(1\&3),(2\&4)],[(1\&2),(3\&4)]$ for the first time step is
calculated as $0.05,0.26,2.42$, respectively, compared to the case when using
a SKF including all 4 modes. Comparing the obtained values using the error
quantification with those obtained from the MC simulations (see Figure 1), it
is observed that the calculated error is close to the expected MC-obtained
error for the first two mode structures, namely,
$[(1\&3),2,4],[(1\&3),(2\&4)]$, due to the fact that the assumptions required
in theorem 1 are satisfied with a high probability. The value obtained for the
clusters $[(1\&2),(3\&4)]$ is not consistent with the simulated error since
the assumptions mentioned in Section 5.1 are not satisfied for this structure.
Specifically, node $1$ is more probable to be detected by mode $3$, which is
not in the same cluster as $1$.

The run time of some of filters is presented below:

| $[1,2,3,4]$ | $[1,3,(2\&4)]$ | $[(1\&3),(2\&4)]$
---|---|---|---
Run time (sec) | 0.13 | .09 | .05

The same can be applied to all time steps by calculating the mean and
covariances recursively, or an accumulation of the distances over a few time
steps can be used to make a consistent decision for all time steps.

Fig. 1: MC estimation error using SKF with the ground truth model as well as
the reduced models showing that the proposed algorithm is able to detect the
best clusters using the calculated edge matrix.

##  8 Conclusion and future work

A graph-based representation for an SLDS is proposed. The nodes and edges of
this graph are defined based on the parametric models of the SLDS. This graph
structure is used to find an efficient set of modes to use as an estimator for
the SLDS and quantify the error induced by this model reduction. The
performance of this methodology is verified in the simulations. Proposing a
partially connected graph and using this algorithm to make decisions in real-
time / batch system identification algorithms are subjects of future work.

## References

  * [1] R. A. Frazin, M. D. Butala, A. Kemball, and F. Kamalabadi,  “Time-dependent reconstruction of nonstationary objects with tomographic or interferometric measurements,”  The Astrophysical Journal, vol. 635, no. 2, pp. L197–L200, dec 2005\.
  * [2] M. D. Butala, F. Kamalabadi, R. A. Frazin, and Y. Chen,  “Dynamic tomographic imaging of the solar corona,”  IEEE Journal of Selected Topics in Signal Processing, vol. 2, no. 5, pp. 755–766, Oct 2008.
  * [3] Geir Evensen,  “Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics,”  Journal of Geophysical Research: Oceans, vol. 99, no. C5, pp. 10143–10162, 1994.
  * [4] Kevin Murphy,  “Modelling gene expression data using dynamic bayesian networks,”  2006\.
  * [5] J. H. Cochrane,  “Time series for macroeconomics and finance,”  1997\.
  * [6] T. Ardeshiri, U. Orguner, C. Lundquist, and T. B. Schön,  “On mixture reduction for multiple target tracking,”  in 2012 15th International Conference on Information Fusion, 2012, pp. 692–699.
  * [7] T Ogawa, H Sonoda, S Ishiwa, H Sawaguchi, and K Wakayama,  “[an application of autoregressive model to pattern discrimination of brain electrical activity mapping],”  No to hattatsu. Brain and development, vol. 21, pp. 340–7, 08 1989\.
  * [8] Kevin Patrick Murphy and Stuart Russell,  Dynamic Bayesian Networks: Representation, Inference and Learning,  Ph.D. thesis, University of California Berkeley, 2002.
  * [9] Kevin P. Murphy,  “Switching Kalman filters,”  Tech. Rep., University of California Berkeley, 1998.
  * [10] Zoubin Ghahramani and Geoffrey E. Hinton,  “Switching state-space models,”  Tech. Rep., King’s College Road, Toronto M5S 3H5, 1996.
  * [11] C. B. Chang and M. Athans,  “State estimation for discrete systems with switching parameters,”  IEEE Transactions on Aerospace and Electronic Systems, vol. AES-14, no. 3, pp. 418–425, 1978.
  * [12] R. Vidal, S. Soatto, Yi Ma, and S. Sastry,  “An algebraic geometric approach to the identification of a class of linear hybrid systems,”  in 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475), 2003, vol. 1, pp. 167–172 Vol.1.
  * [13] A. L. Juloski, S. Weiland, and W. P. M. H. Heemels,  “A bayesian approach to identification of hybrid systems,”  IEEE Transactions on Automatic Control, vol. 50, no. 10, pp. 1520–1533, 2005.
  * [14] E. Fox, E. B. Sudderth, M. I. Jordan, and A. S. Willsky,  “Bayesian nonparametric inference of switching dynamic linear models,”  IEEE Transactions on Signal Processing, vol. 59, no. 4, pp. 1569–1585, 2011.
  * [15] René Vidal,  “Recursive identification of switched arx systems,”  Automatica, vol. 44, pp. 2274–2287, 09 2008.
  * [16] Christopher M. Bishop,  Pattern Recognition and Machine Learning (Information Science and Statistics),  Springer-Verlag, Berlin, Heidelberg, 2006.
  * [17] D. J. Salmond,  “Mixture reduction algorithms for target tracking,”  in IEE Colloquium on State Estimation in Aerospace and Tracking Applications, 1989, pp. 7/1–7/4.
  * [18] A. Assa and K. N. Plataniotis,  “Wasserstein-distance-based gaussian mixture reduction,”  IEEE Signal Processing Letters, vol. 25, no. 10, pp. 1465–1469, 2018.
  * [19] O. Bochardt, R. Calhoun, J. K. Uhlmann, and S. J. Julier,  “Generalized information representation and compression using covariance union,”  in 2006 9th International Conference on Information Fusion, 2006, pp. 1–7.
  * [20] A. R. Runnalls,  “Kullback-leibler approach to gaussian mixture reduction,”  IEEE Transactions on Aerospace and Electronic Systems, vol. 43, no. 3, pp. 989–999, 2007.