# Cover It Up! Bipartite Graphs Uncover Identifiability in Sparse Factor
Analysis

Darjus Hosszejni111Department of Finance, Accounting, and Statistics, WU
Vienna University of Economics and Business, Austria. Email:
<EMAIL_ADDRESS>Sylvia Frühwirth-Schnatter222Department of Finance,
Accounting, and Statistics, WU Vienna University of Economics and Business,
Austria. Email<EMAIL_ADDRESS>
###### Abstract

Despite the popularity of factor models with sparse loading matrices, little
attention has been given to formally address identifiability of these models
beyond standard rotation-based identification such as the positive lower
triangular constraint. To fill this gap, we present a counting rule on the
number of nonzero factor loadings that is sufficient for achieving generic
uniqueness of the variance decomposition in the factor representation. This is
formalized in the framework of sparse matrix spaces and some classical
elements from graph and network theory. Furthermore, we provide a
computationally efficient tool for verifying the counting rule. Our
methodology is illustrated for real data in the context of post-processing
posterior draws in Bayesian sparse factor analysis.

Keywords: Computational complexity; factor analysis; shrinkage prior;
sparsity; variance identification

JEL classification: C11, C38, C63

##  1 Introduction

Ever since the pioneering work of Thurstone (1931, 1935), factor analysis has
been a popular method to model the covariance matrix $\mathbb{V}(y_{t})$ of
correlated, multivariate observations $y_{t}$ of dimension $m$,

$\begin{split}\mathbb{V}(y_{t}\mid{\beta},\Psi,\Sigma_{f})&={\beta}\Sigma_{f}{\beta}^{\top}+\Psi,\\\
\mathbb{E}(y_{t}\mid{\beta},f_{t},\Psi)&={\beta}f_{t},\\\
\mathbb{V}(y_{t}\mid{\beta},f_{t},\Psi)&=\Psi,\end{split}\qquad\begin{split}{}\\\
\mathbb{E}(f_{t})&=0,\\\ \mathbb{V}(f_{t})&=\Sigma_{f}=I_{r},\end{split}$ (1)

where $f_{t}$ is $r$-dimensional, ${\beta}$ is $m\times r$-dimensional,
$\Psi=\text{diag}(\sigma_{1}^{2},\ldots,\sigma_{m}^{2})$, and $\Sigma_{f}$ is
the identity matrix in the paper at hand. As discussed by the comprehensive
textbooks of Gorsuch (1983) and Anderson (2003), factor analysis is an
umbrella term for a large number of heuristic or statistical methods, which
handle various aspects of this underdetermined task. The researcher has to
make many decisions during the analysis, and, unfortunately, this extensive
and complex toolbox is often applied unprofessionally by practitioners,
according to recurrent reviews (Ford et al., 1986; Conway and Huffcutt, 2003;
Howard, 2016).

On the other hand, there are common ways to structure a factor analysis into
subtasks. Following the title of the textbook by Thompson (2004), it is
customary to first separate any factor analysis into two stages. During
exploratory factor analysis (EFA), decisions are made, among others, about the
number of factors $r$, whether correlated or orthogonal factors are more
appropriate, the identifying rotation of the factor loadings ${\beta}$, and
potential insignificant entries in ${\beta}$, i.e. fixed zeros $\beta_{ij}=0$.
Then, conditionally on EFA, during confirmatory factor analysis, factor
loadings ${\beta}$ and factor scores $f_{t}$ are estimated and hypothesis
tests are conducted. In this setting, we advocate the standardization of a
further step within EFA: a check on the unique identifiability of the
idiosyncratic variances
$\Psi=\text{diag}(\sigma_{1}^{2},\ldots,\sigma_{m}^{2})$.

Classical results about generic global and local identification by Shapiro
(1985) and Bekker and ten Berge (1997) solve the question of variance
identification in what we call _dense_ factor analysis (i.e., no structural
zeros in ${\beta}$) by referring to the Ledermann bound (Ledermann, 1937).
However, these results do not apply in a zero-inflated model space because the
exact zeros may fix ${\beta}$ in a locally or even globally non-variance-
identified space. A simple example is ${\beta}=(0,\alpha,0)^{\top}$ with $r=1$
and $m=3$. This factor can be eliminated and built into the idiosyncratic
variance $\sigma_{2}^{2}$ for all $\alpha\neq 0$.

Identification becomes vital when the number of factors $r$ is estimated.
Reiersøl (1950) shows that if there is a solution to a factor analysis, i.e. a
successfully estimated model with an assumed number of factors $k=r$, then
there are infinitely many solutions with more factors $k>r$. Consequently, it
is easy to overestimate the number of factors if parameters are not uniquely
identified. We demonstrate this in Section 4.

###  1.1 Insignificant Factor Loadings

In this paper, we are concerned with the sparse latent factor model introduced
by West (2003). He describes the model as a hierarchical prior distribution,
which we also use in Section 4, and applies it as a variable selection tool to
tame a model with potentially many latent factors (referred to as “large
$p$”). Our motivation also falls under variable selection. However, we
consider the model useful even outside the realm of large $p$; rather, we see
sparsity as a natural part of the usual practice of latent factor analysis.

Henceforward, assume that at the end of EFA $r$ is specified and
$\Sigma_{f}=I_{r}$ is picked and, most importantly for us, a zero-nonzero
pattern is fixed for ${\beta}$. We formalize the insignificance of some
entries in ${\beta}$ by the _sparse space_
${\Theta_{\text{sp}}=({\delta},\mathbb{R}^{\sum_{i,j}{\delta}_{ij}})}$, which
is a pair of a binary matrix ${\delta}\in\\{0,1\\}^{m\times r}$ and a real
vector space of dimension $\sum_{i,j}{\delta}_{ij}$. $\Theta_{\text{sp}}$ is
equipped with a probability measure that is absolutely continuous with respect
to the $\sum_{i,j}{\delta}_{ij}$-dimensional Lebesgue-measure. Then, a matrix
${\beta}$ is said to be _generated by_ $\Theta_{\text{sp}}$ if for all $1\leq
i\leq m$ and $1\leq j\leq r$ we have that ${\delta}_{ij}=0$ implies
$\beta_{ij}=0$. On the other hand, the set of ${\beta}$ matrices that are
generated by $\Theta_{\text{sp}}$ and have other zeros than ${\delta}$ is a
zero probability event. Finally, $\Theta_{\text{sp}}$ is said to be
_represented by_ ${\delta}$.

Typically, a factor analysis starts in an _unrestricted space_ with loading
matrix ${\beta}_{1}\in\mathbb{R}^{m\times r}$. Then, the researcher moves into
their _preferred space_ ${\beta}_{2}={\beta}_{1}U({\beta}_{1})$ via the
orthonormal rotation matrix $U({\beta}_{1})$, which, as indicated, often
depends on ${\beta}_{1}$ (e.g., the varimax rotation (Kaiser, 1958) or the
generalized lower triangular rotation (Frühwirth-Schnatter and Lopes, 2018)).
The preferred space is where signal and noise are separated, resulting in the
sparse space $\Theta_{\text{sp}}$. In the Bayesian paradigm, spike-and-slab
priors make this procedure explicit and exact via Bernoulli conditional
distributions (West, 2003; Lucas et al., 2006; Ghahramani et al., 2007;
Frühwirth-Schnatter and Lopes, 2010, 2018; Kaufmann and Schuhmacher, 2019),
while continuous spike-and-slab priors (Ročková and George, 2017) approximate
it. More generally applicable methods emulating a sparse space include
heuristic thresholding (Bhattacharya and Dunson, 2011; Legramanti et al.,
2020), thresholding in relation with sample size (Gorsuch, 1983, “salient
loading”), highlighting (Fabrigar et al., 1999), and color coding (Bolfarine
et al., 2021; Zhao et al., 2016). There are examples in the connected
literature of principal component analysis as well (McCrae and Costa, 1997;
Leckman et al., 1997).

In this paper, identifiability of $\Psi$ is examined based on the classical
rule by Anderson and Rubin (1956). Others have developed variants on the
Anderson-Rubin rule for specific cases (Dunn, 1973; Bollen, 1989), for
dedicated but correlated factors (Conti et al., 2014) and for correlated
idiosyncratic errors (Williams, 2020). Our main contribution is two-fold: we
prove that a counting rule for zero-nonzero pattern of ${\beta}$ is
generically equivalent to the Anderson-Rubin rule, and we provide a proof and
an algorithm for this counting rule to be efficiently verified. Further, we
introduce bipartite graphs to factor analysis and provide empirical evidence
for the usefulness of variance identification in reducing the estimated number
of factors.

The paper is structured as follows. The simple counting rule of Anderson and
Rubin (1956, Thm 5.5) is generalized in Section 2. The efficient verification
of the counting rule is non-trivial, and it is handled in Section 3. After a
numerical illustration in Section 4, the paper concludes in Section 5.

###  1.2 Note on Proof Techniques

In this section, we take a detour and introduce classical notions and results
from graph theory for completeness. In the following sections there are two
theorems, and in both of them the zero-nonzero pattern is represented by a
customized undirected finite bipartite graph. Then, there are three parts to
the proofs: first, the appropriateness of the representation needs to be shown
and, at the same time, the task is translated into the language of graph
theory; second, a solution to the translated task is found as a minimal vertex
cover or a maximal matching (see definitions below); third, when applicable,
efficient computation of the solution is outlined.

######  Definition 1 (Graph and Weighted Graph).

A graph $G=(V,E)$ is a pair of a set of vertices (also called nodes) $V$ and a
set of undirected edges $E\subseteq\\{\\{u,v\\}:u\in V,v\in
V\setminus\\{u\\}\\}$. A vertex-weighted graph $G=(V,E,w)$ is a graph extended
with a weight mapping $w:V\mapsto\mathbb{R}$.

A graph is a special case of a vertex-weighted graph where $w(v)=1$ for each
$v\in V$.

######  Definition 2 (Bipartite Graph).

A bipartite graph $B=(V_{1},V_{2},E)$ is a triplet of two disjoint sets of
vertices $V_{1}$ and $V_{2}$ and a set of undirected edges
$E\subseteq\\{\\{u,v\\}:u\in V_{1},v\in V_{2}\\}$.

Note that a bipartite graph $B=(V_{1},V_{2},E_{B})$ is isomorphic to the graph
$G=(V,E_{G})$ for $V=V_{1}\cup V_{2}$ and $E_{G}=E_{B}$.

######  Definition 3 (Bipartite Adjacency Matrix and Generated Bipartite
Graph).

Given a binary matrix ${\delta}$, an equivalent graph representation is the
bipartite graph $B=(V_{\text{col}},V_{\text{row}},E_{B})$, where vertices of
$V_{\text{col}}$ and $V_{\text{row}}$ correspond to the columns and,
respectively, rows of ${\delta}$, and an edge is drawn between $u\in
V_{\text{col}}$ and $v\in V_{\text{row}}$ if the corresponding matrix element
${\delta}_{v,u}$ is nonzero. Then, ${\delta}$ describes which pairs of
vertices of $B$ are adjacent (i.e., connected by an edge) and therefore
${\delta}$ is called the adjacency matrix of $B$. Turned around, $B$ is called
the generated bipartite graph of ${\delta}$.

Figure 1 shows a binary matrix ${\delta}$ and its generated bipartite graph
$B=(V_{\text{col}},V_{\text{row}},E_{B})$ with
$V_{\text{col}}=\\{u_{1},u_{2},u_{3},u_{4}\\}$ and
$V_{\text{row}}=\\{v_{1},v_{2},v_{3},v_{4}\\}$. The edge set is
$E_{B}=\\{\\{u_{1},v_{1}\\},\\{u_{1},v_{2}\\},\\{u_{1},v_{4}\\},\\{u_{2},v_{2}\\},\allowbreak\\{u_{2},\allowbreak
v_{3}\\},\\{u_{3},v_{3}\\},\\{u_{4},v_{4}\\}\\}$.

| $u_{1}$ | $u_{2}$ | $u_{3}$ | $u_{4}$
---|---|---|---|---
$v_{1}$ | 1 | 0 | 0 | 0
$v_{2}$ | 1 | 1 | 0 | 0
$v_{3}$ | 0 | 1 | 1 | 0
$v_{4}$ | 1 | 0 | 0 | 1
$\qquad\iff\qquad$$u_{1}$$u_{2}$$u_{3}$$u_{4}$$v_{1}$$v_{2}$$v_{3}$$v_{4}$
Figure 1: Example binary matrix and corresponding generated bipartite graph.

######  Definition 4 (Matching).

A matching in graph $G=(V,E)$ is a set of pairwise disconnected edges
$M\subseteq E$; i.e., no two edges of $M$ have the same endpoint. A maximum
matching in $G$ is a matching with the maximal number of edges among all
matchings in $G$. For a subset of vertices $S\subseteq V$, an $S$-saturating
matching is a matching in $G$ that covers all vertices of $S$.

Note that there may exist matchings in $G$ that cannot be extended to larger
matchings but are not maximum matchings. In Figure 1, there are many
matchings: e.g., $\emptyset$, $\\{\\{u_{1},v_{1}\\}\\}$, and
$\\{\\{u_{1},v_{4}\\},\\{u_{2},v_{3}\\}\\}$ is one that cannot be extended
further; moreover, $\\{\\{u_{1},v_{2}\\},\\{u_{2},v_{2}\\}\\}$ is not a
matching because $v_{2}$ appears twice as end point.
$\\{\\{u_{i},v_{i}\\}\\}_{i=1}^{4}$ is the only maximum matching, and it is
also $V_{1}$-saturating. Furthermore, if we remove edge $\\{u_{2},v_{2}\\}$
from $E_{B}$, then there is no $V_{1}$-saturating matching but two maximum
matchings of size three.

######  Definition 5 (Vertex Cover).

A vertex cover in graph $G=(V,E)$ is a set of vertices $C\subseteq V$ such
that every edge in $E$ has an endpoint in $C$. A minimum vertex cover in $G$
is a vertex cover with the minimal set size among all vertex covers in $G$. A
minimum vertex cover in weighted graph $G=(V,E,w)$ is a vertex cover with the
minimal total weight among all vertex covers in $G$.

Note that there may exist vertex covers in $G$ that cannot be reduced to
smaller vertex covers but are not minimum vertex covers. In Figure 1, there
are many vertex covers: e.g., $V_{\text{row}}\cup V_{\text{col}}$, and
$\\{u_{2},u_{3},v_{1},v_{2},v_{4}\\}$ is one that cannot be reduced further;
$\\{u_{1},u_{2}\\}$ is not a vertex cover because both end points of edge
$\\{u_{3},v_{3}\\}$ are missing from the set. There are several minimum vertex
covers, e.g., $V_{\text{col}}$, all of them of size four.

######  Theorem (Kőnig (1931) and Egerváry (1931)).

The size of a maximum matching is equal to the size of a minimum vertex cover
in bipartite graphs.

In Figure 1, both the maximum matching and the minimum vertex cover are of
size four. If we remove edge $\\{u_{2},v_{2}\\}$ from $E_{B}$, then both
reduce to size three.

######  Theorem (Marriage Theorem by Hall (1935)).

In any bipartite graph $B=(V_{1},V_{2},E)$, there exists a $V_{1}$-saturating
matching if and only if for every subset $W\subseteq V_{1}$, the neighborhood
of $W$ in $V_{2}$ is a larger set than $W$.

An exemplary counterfactual statement: if we remove edge $\\{u_{2},v_{2}\\}$
from Figure 1, then there is no $V_{\text{col}}$-saturating matching, and the
neighborhood of $W=\\{u_{2},u_{3}\\}$ becomes the smaller set $\\{v_{3}\\}$ at
the same time.

For a more detailed introduction to graph theoretic notions, see chapters 3.1
and 3.2 of West (2001).

##  2 Sufficient Condition for Generic Global Variance Identification

Anderson and Rubin (1956) provide a sufficient condition for variance
identification based on the following row deletion property.

######  Definition 6 (Row Deletion Property, $\text{RD}({r},{s})$).

Let ${\beta}\in\mathbb{R}^{m\times r}$. If any $s>0$ rows of ${\beta}$ are
removed, then the remaining rows can be grouped into two matrices of rank $r$.

Note that $m\geq 2r+s$ is implied by $\text{RD}({r},{s})$.

######  Theorem (Anderson and Rubin (1956, Theorem 5.1)).

$\text{RD}({r},{1})$ is a sufficient condition for identification of $\Psi$.

The theorem holds for every ${\beta}\in\mathbb{R}^{m\times r}$ and thus also
for a matrix with zero entries. This makes it relevant for sparse matrix
spaces.

The $s>1$ case appears in overfitting factor models, when specific factors are
allowed, as introduced by Tumura and Sato (1980).

######  Theorem (Tumura and Sato (1980, Theorem 1)).

Assume that $\text{RD}({r},{s})$ holds and the factor decomposition of
equation (1) exists with rank $r$. Denote the solution by ${\beta}_{r}$. If
there is another factor decomposition with ${\beta}_{r+s}$ of rank $r+s$, then
there exists a rotation $G$ such that ${\beta}_{r+s}G=[{\beta}_{r}\ S]$, where
$S$ is a collection of specific factors; i.e. $SS^{\top}$ is diagonal.

Later, Sato (1992) introduces a counting rule that is necessary for the row
deletion property.

######  Definition 7 (3-5-7-9 Counting Rule, $\text{CR}({r},{s})$).

Let ${\delta}$ be a binary matrix of size $m\times r$. For every $1\leq q\leq
r$, every submatrix containing $q$ columns of ${\delta}$ has at least $2q+s$
nonzero rows.

######  Theorem (Sato (1992, Theorem 3.4)).

A necessary condition for $\text{RD}({r},{s})$ is that $\text{CR}({r},{s})$
holds for the zero-nonzero pattern ${\delta}$ of ${\beta}G$, where $G$ is any
$r\times r$ unitary matrix.

There are three issues with applying this Theorem to verify identification of
$\Psi$ for a given sparse ${\beta}$: the exponential number of submatrices
within $\text{CR}({r},{s})$, the infinite number of rotations $G$, and that
this is a necessary condition for a sufficient condition. Frühwirth-Schnatter
and Lopes (2018) overcome the latter two in sparse matrix spaces
$\Theta_{\text{sp}}$ with probability one, although they need to assume that
${\delta}$ is a generalized lower triangular matrix. In the next section, we
show that $\text{RD}({r},{s})$ and $\text{CR}({r},{s})$ are equivalent almost
everywhere without further assumptions. We apply this result in Section 4 to
restrict model search to a space of variance identified models in a Bayesian
setting.

Henceforth, let us consider a sparse matrix space
${\Theta_{\text{sp}}=({\delta},\mathbb{R}^{\sum_{i,j}{\delta}_{ij}})}$ and a
matrix ${\beta}$ generated by $\Theta_{\text{sp}}$. Following Shapiro (1985)
and Bekker and ten Berge (1997), we consider generic variance identification
in this section; however, we adapt it to our sparse setting.

######  Definition 8 (Generic Global Variance Identification).

Model (1) is globally variance identified at $({\beta},\Psi)$ if the only
diagonal covariance matrix $\Omega$, for which $\mathbb{V}(y)-\Omega$ is also
a covariance matrix and
$\text{rank}(\mathbb{V}(y)-\Omega)\leq\text{rank}(\mathbb{V}(y)-\Psi)$, is
$\Omega=\Psi$. The model is generically variance identified within
$\Theta_{\text{sp}}$ if it is globally variance identified except for a set of
Lebesgue-measure zero in
$\mathbb{R}^{\sum_{i,j}{\delta}_{ij}}\times(\mathbb{R}^{+})^{m}$, which is the
product of the domains of ${\beta}$ and $\Psi$.

Note that the theorem of Anderson and Rubin ascertains point-wise (not
generic) global variance identification.

We start with a simple lemma.

######  Lemma 1.

Assume that ${\delta}$ satisfies $\text{CR}({r},{s})$. If any $s>0$ rows are
deleted from ${\delta}$, then the remaining matrix satisfies
$\text{CR}({r},{0})$. ∎

Note that the inverse of this statement is false. If zero rows are added to a
$\text{CR}({r},{0})$-matrix, the extended matrix still only satisfies
$\text{CR}({r},{0})$.

In the following, we assume that $m\geq 2r+s$ and ${\delta}$ satisfies
$\text{CR}({r},{s})$. By the end of this section, we show that if $s$ rows are
deleted from ${\delta}$ and accordingly from ${\beta}$, then, with probability
one in $\Theta_{\text{sp}}$, the remaining rows of ${\beta}$ can be grouped
into two matrices ${{\beta}^{\scalebox{0.5}{$\,\square$}}_{A}}$ and
${{\beta}^{\scalebox{0.5}{$\,\square$}}_{B}}$ such that both have rank $r$.
With Definition 9 and Lemma 2, we consider first a square matrix that will
later be one of the two matrices of rank $r$ in Definition 6.

######  Definition 9 (RCM Matrix).

A row-column-matching (RCM) matrix space of dimension $r$ is a sparse matrix
space
$\Theta_{\text{sp}}^{{\scalebox{0.5}{$\,\square$}}}=({\delta}^{\scalebox{0.5}{$\,\square$}},\mathbb{R}^{\sum_{i,j}{\delta}^{\scalebox{0.5}{$\,\square$}}_{ij}})$
with the following property: ${\delta}^{\scalebox{0.5}{$\,\square$}}$ is
$r\times r$-dimensional, and in the bipartite graph
$B=(V_{\text{col}},V_{\text{row}},E_{B})$ generated from
${\delta}^{\scalebox{0.5}{$\,\square$}}$ there is a
$V_{\text{col}}$-saturating matching. An RCM matrix is a square matrix
generated by an RCM matrix space.

Since the number of rows and columns is the same, a
$V_{\text{col}}$-saturating matching is also a $V_{\text{row}}$-saturating
matching in $B$. Importantly, this matching directly corresponds to a
reordered diagonal of ones in ${\delta}^{\scalebox{0.5}{$\,\square$}}$, as
depicted in Figure 2.

| $u_{1}$ | $u_{2}$ | $u_{3}$ | $u_{4}$
---|---|---|---|---
$v_{1}$ | 0 | 0 | 1 | 1
$v_{2}$ | 1 | 1 | 0 | 0
$v_{3}$ | 1 | 0 | 1 | 1
$v_{4}$ | 1 | 0 | 1 | 0
$\qquad\iff\qquad$$u_{1}$$u_{2}$$u_{3}$$u_{4}$$v_{1}$$v_{2}$$v_{3}$$v_{4}$
Figure 2: Left: example RCM space of dimension $4$, represented via its zero-
nonzero pattern ${\delta}$. Right: generated bipartite graph
$B=(V_{\text{col}},V_{\text{row}},E_{B})$ of ${\delta}$. One
$V_{\text{col}}$-saturating matching and the corresponding reordered diagonal
are shown in boldface.

######  Lemma 2.

Let the $r\times r$-dimensional matrix ${\beta}^{\scalebox{0.5}{$\,\square$}}$
be generated by a sparse matrix space
$\Theta_{\text{sp}}^{{\scalebox{0.5}{$\,\square$}}}$, and consider the
following two conditions.

  1. (a)

${\beta}^{\scalebox{0.5}{$\,\square$}}$ is non-singular.

  2. (b)

$\Theta_{\text{sp}}^{{\scalebox{0.5}{$\,\square$}}}$ is an RCM space.

Then, (a) implies (b) everywhere and (b) implies (a) almost everywhere in
$\Theta_{\text{sp}}^{{\scalebox{0.5}{$\,\square$}}}$.

###### Proof.

The proof is based on the Leibniz formula for determinants. Note that the
matching specified in Definition 9 is a set of $r$ nonzero variables in
${\beta}^{\scalebox{0.5}{$\,\square$}}$ that are located in pairwise different
rows and columns. Consequently, for an $r\times r$ square matrix, the product
of the elements of the matching constitutes one summand in the Leibniz formula
(potentially after a sign switch). If ${\beta}^{\scalebox{0.5}{$\,\square$}}$
is non-singular, then its determinant is nonzero, therefore there has to be a
nonzero summand and $\Theta_{\text{sp}}^{{\scalebox{0.5}{$\,\square$}}}$ is an
RCM space. If $\Theta_{\text{sp}}^{{\scalebox{0.5}{$\,\square$}}}$ is an RCM
space, then ${\beta}^{\scalebox{0.5}{$\,\square$}}$ has a nonzero summand with
probability one. There may be further summands, but only finitely many and
they all include further nonzero variables from
${\beta}^{\scalebox{0.5}{$\,\square$}}$, and therefore their sum is nonzero
with probability one. Hence, $\det({\beta}^{\scalebox{0.5}{$\,\square$}})\neq
0$ and ${\beta}^{\scalebox{0.5}{$\,\square$}}$ is non-singular with
probability one. ∎

The following statement is a straightforward implication of Lemma 2, which
connects the row deletion property to RCM spaces. Let us fix $s$ arbitrary
rows that are removed from ${\beta}$. We will denote the remaining rows by
${{\beta}}^{\text{rem}}$ and the induced remainder objects by
${{\delta}}^{\text{rem}}$ and ${\Theta_{\text{sp}}^{\text{rem}}}$.

######  Corollary 3.

Consider the following two conditions.

  1. (a)

${{\beta}}^{\text{rem}}$ satisfies $\text{RD}({r},{0})$.

  2. (b)

Two distinct groups of $r$ rows of ${{\delta}}^{\text{rem}}$ can be selected
to form square matrices ${\delta}^{\scalebox{0.5}{$\,\square$}}_{A}$ and
${\delta}^{\scalebox{0.5}{$\,\square$}}_{B}$ such that the sparse matrix
spaces represented by ${\delta}^{\scalebox{0.5}{$\,\square$}}_{A}$ and
${\delta}^{\scalebox{0.5}{$\,\square$}}_{B}$ are both RCM matrix spaces.

Then, (a) implies (b) everywhere and (b) implies (a) almost everywhere in
${\Theta_{\text{sp}}^{\text{rem}}}$. ∎

We demonstrate in Figure 3 the ingredients to prove that $\text{CR}({r},{s})$
is sufficient for Corollary 3(b). On the left hand side of the figure,
${\delta}$ satisfies $\text{CR}({3},{2})$. We have to examine every
${{\delta}}^{\text{rem}}$ that results from removing two rows of ${\delta}$.
Consider now removing the first and sixth rows. Then, we need to group the
remaining rows into two three-dimensional RCM matrices, and for this we
introduce the graph representation $B_{\text{suf}}$ of
${{\delta}}^{\text{rem}}$ shown on the right hand side of the figure: we take
the generated bipartite graph of ${{\delta}}^{\text{rem}}$ and duplicate the
set of column nodes with their edges. The duplicated nodes are indicated by a
superscript star. We highlighted a matching $\mathcal{M}$ of size six in this
graph and the corresponding nonzero elements in ${\delta}$. By going through
the column nodes $u_{1}$, $u_{2}$, and $u_{3}$, and their matched row nodes
$v_{5}$, $v_{2}$, and $v_{3}$, respectively, we find that the sequence
$(v_{5},v_{2},v_{3})$ specifies a submatrix of ${\delta}$ with a fully nonzero
diagonal; this is the first RCM matrix. Similarly, by going through the
duplicate column nodes $u_{1}^{*}$, $u_{2}^{*}$, and $u_{3}^{*}$, and their
matched row nodes, we find that $(v_{8},v_{7},v_{4})$ specifies a separate
submatrix of ${\delta}$ with a fully nonzero diagonal, which is the second RCM
matrix. We prove below that a matching similar to $\mathcal{M}$ always exists
for a $\text{CR}({r},{0})$ binary matrix ${{\delta}}^{\text{rem}}$. Then we
are done due to Lemma 1.

For our purposes it is enough to prove that a matching of size $2r$ exists in
$B_{\text{suf}}$ if ${{\delta}}^{\text{rem}}$ satisfies $\text{CR}({r},{0})$:
such a matching necessarily connects every column and their duplicates to
unique rows, therefore it automatically specifies two disjoint RCM submatrices
${\delta}^{\scalebox{0.5}{$\,\square$}}_{A}$ and
${\delta}^{\scalebox{0.5}{$\,\square$}}_{B}$ of ${{\beta}}^{\text{rem}}$
needed for Corollary 3(b). According to Kőnig’s theorem, it is sufficient to
prove that the smallest vertex cover is of size $2r$. Note that a trivial
vertex cover consists of all the columns and their duplicates, and there are
exactly $2r$ vertices in this vertex cover. If there was a smaller vertex
cover, then it would include $k$ rows and exclude $l$ columns (including
potential duplicates), where $k<l$. Since this would be a vertex cover, this
would mean that all edges that go from those $l$ columns are covered by the
$k$ rows. However, since ${{\delta}}^{\text{rem}}$ satisfies
$\text{CR}({r},{0})$, the $l$ columns and duplicates should neighbor at least
$l$ rows: if the net number of columns is $l/2$, they already neighbor at
least $2\cdot l/2=l$ rows; if the net number of columns is larger, then even
more rows. Therefore $k\geq l$: a contradiction. This concludes the proof. The
statement is summarized in Theorem 4.

| $u_{1}$ | $u_{2}$ | $u_{3}$
---|---|---|---
~~$v_{1}$~~ | 1 | 1 | 1
$v_{2}$ | 1 | 1 | 0
$v_{3}$ | 1 | 0 | 1
$v_{4}$ | 0 | 0 | 1
$v_{5}$ | 1 | 0 | 0
~~$v_{6}$~~ | 0 | 1 | 0
$v_{7}$ | 1 | 1 | 0
$v_{8}$ | 1 | 0 | 0
$\qquad\iff\qquad$$u_{1}$$u_{2}$$u_{3}$$u_{1}^{*}$$u_{2}^{*}$$u_{3}^{*}$$v_{2}$$v_{3}$$v_{4}$$v_{5}$$v_{7}$$v_{8}$
Figure 3: Demonstration of the proof of Theorem 4 for $r=3$ and $s=2$. The
${\delta}$ on the left hand side satisfies $\text{CR}({r},{s})$. Consequently,
after we remove any two rows, e.g., the first and the sixth in this instance,
there is a sufficiently large matching in the graph representation of
${{\delta}}^{\text{rem}}$ used within the proof.

######  Theorem 4.

If ${\delta}$ satisfies the counting rule $\text{CR}({r},{s})$, then the
extended row deletion property $\text{RD}({r},{s})$ generically holds within
$\Theta_{\text{sp}}$. ∎

Note that due to Sato’s theorem (Sato, 1992), the counting rule is still
necessary for the row deletion property.

Now we are ready to connect the 3-5-7-9 counting rule with the Anderson-Rubin
theorem. This is the foundation of our framework that is put to use in Section
4.

######  Corollary 5.

If ${\delta}$ satisfies the counting rule $\text{CR}({r},{1})$, then Model (1)
is generically globally variance identified within $\Theta_{\text{sp}}$. ∎

######  Remark.

Given a binary matrix ${\delta}$, the verification of $\text{CR}({r},{1})$ is
a finite task. However, a naïve approach may need to visit $2^{r}-1$ matrices
in order to make a decision. The combinatorial explosion quickly becomes an
issue in practice as $r$ increases. In Section 3, we establish the
applicability of Theorem 4 for large models.

######  Remark.

The counting rule is not necessary. The following sparse space is an example
where the counting rule does not hold but the model is generically globally
variance identified.

${\beta}=\begin{pmatrix}\beta_{11}&0&0\\\ \beta_{21}&\beta_{22}&0\\\
0&\beta_{32}&\beta_{33}\\\ \beta_{41}&0&\beta_{43}\\\ 0&\beta_{52}&0\\\
0&0&\beta_{63}\end{pmatrix},\quad\mathbb{V}(y)=\begin{pmatrix}v_{1}&\cdot&\cdot&\cdot&\cdot&\cdot\\\
c_{21}&v_{2}&\cdot&\cdot&\cdot&\cdot\\\ 0&c_{32}&v_{3}&\cdot&\cdot&\cdot\\\
c_{41}&c_{42}&c_{43}&v_{4}&\cdot&\cdot\\\ 0&c_{52}&c_{53}&0&v_{5}&\cdot\\\
0&0&c_{63}&c_{64}&0&v_{6}\end{pmatrix}.$

To see this, observe that all factor loadings can generically be computed (up
to sign switches in each column) from the lower triangular elements of
$\mathbb{V}(y)$, e.g., $\beta_{11}=\sqrt{c_{21}c_{41}/c_{42}}$, and then all
elements of $\Psi$ can generically be computed given the factor loadings and
the diagonal of $\mathbb{V}(y)$.

##  3 Efficient Verification of the Sufficient Condition

We extend the previous section and describe an algorithm that verifies
$\text{CR}({r},{1})$ without visiting all the non-empty submatrices of
${\delta}$. In our construction, we do not just consider the nonzero rows of
${\delta}$; rather, we focus on unions of nonzero rows and nonzero columns
such that the submatrix of ${\delta}$ that is left out of the union is a zero
matrix. We first show that $\text{CR}({r},{1})$ is equivalent to all such
unions being “not too small,” or, in other words, that the smallest such union
is not too small. In the second half of the proof, we show how this
optimization task can be solved efficiently.

As a starting point, consider two special cases: zero columns and zero rows
can be removed from the sparsity matrix ${\delta}$. First, when zero columns
are removed, then the unnecessary factors have to be discarded as well, and
$r$ has to be adjusted. Second, the addition or removal zero rows does not
influence rank conditions in tall matrices; therefore, $\text{RD}({r},{1})$
holds for ${\beta}$ if and only if it holds for ${\beta}$ without its zero
rows. In the remaining part of the derivation, we assume that every row and
column of ${\delta}$ has at least one nonzero element.

The main part of the proof consists of two steps:

  1. 1.

Reformulate $\text{CR}({r},{1})$ as a minimum weighted vertex cover (MWVC)
task, and

  2. 2.

Show that the MWVC at hand can be solved efficiently via a polynomial
algorithm.

#### Step 1.

Let $B_{\text{adj}}=(V_{\text{col}},V_{\text{row}},E_{\text{adj}})$ be the
bipartite graph that is generated from ${\delta}$. Furthermore, we equip the
vertices of $B_{\text{adj}}$ with weights: elements of $V_{\text{col}}$ and
$V_{\text{row}}$ are weighted with $2r+1$ and, respectively, with $r$. The top
row of Figure 4 shows an example. Then, the following lemma provides the basis
for the polynomial algorithm. Terminology introduced in Definitions 1 and 5 is
used.

######  Lemma 6.

${\delta}$ satisfies $\text{CR}({r},{1})$ if and only if the total weight
$M^{\star}$ of the MWVC in $B_{\text{adj}}$ is at least $r(2r+1)$.

The intuition behind the vertex cover is that the submatrix formed by the rows
and columns that are left out is a zero matrix in ${\delta}$.

###### Proof of Lemma 6.

To prove this lemma, first note that $M^{\star}\leq r(2r+1)$ always holds.
Indeed, the set $V_{\text{col}}$ has total weight $(2r+1)r+r\cdot 0$ and it is
a vertex cover. Another interesting case is the opposite: $V_{\text{row}}$ is
also always a vertex cover with weight $(2r+1)\cdot 0+rm\geq r(2r+1)$. Now we
turn to the statement.

For the first direction of the proof, assume that $\text{CR}({r},{1})$ does
not hold; i.e., there exists a submatrix ${\delta}_{q}$ made of $1\leq q\leq
r$ columns of ${\delta}$ with at most $2q$ nonzero rows. Then
$M^{\star}<r(2r+1)$. Indeed, the $2q$ rows of ${\delta}_{q}$ and the $r-q$
columns outside of ${\delta}_{q}$ constitute a vertex cover. Therefore,
$M^{\star}\leq(2r+1)(r-q)+r\cdot 2q=r(2r+1)-q<r(2r+1)$.

For the opposite direction, we assume that $\text{CR}({r},{1})$ holds, and we
would like to show that the objective function takes always at least the
aforementioned value $r(2r+1)$. Let us take any vertex cover and denote by $k$
and $l$ the number of columns and, respectively, rows that are included in the
vertex cover. For $k=r$, the total weight is at least $r(2r+1)$. Now, consider
$0\leq k\leq r-1$. There are $r-k$ columns in ${\delta}$ excluded from the
vertex cover; the submatrix constructed from these columns contains at least
$2(r-k)+1$ nonzero rows. Hence, in order to cover these nonzero rows, we must
have $l\geq 2(r-k)+1$. This means that the total weight for this setting
evaluates to $(2r+1)k+rl\geq(2r+1)k+r(2(r-k)+1)=r(2r+1)+k\geq r(2r+1)$. Since
our argument holds for all vertex covers, we have shown that $M^{\star}\geq
r(2r+1)$. ∎

######  Remark.

In its current form, the proof cannot be extended to $\text{CR}({r},{s})$ by
choosing different weights for $V_{\text{col}}$ or $V_{\text{row}}$. In
particular, if $x$ denotes the ratio of vertex weights in $V_{\text{col}}$ and
$V_{\text{row}}$ (i.e., $x=(2r+1)/r$ above), then it can be shown that
$x>2+s-1$ and $x\leq 2+s/r$ are both necessary for the proof. This interval is
non-empty only if $s\leq 1$.

#### Step 2.

For the purposes of this section, we refer to notions from network theory.

######  Definition 10 (Network).

A network is a set of nodes combined with a set of directed edges between the
nodes. There are two distinguished nodes: the source node $s$ and the sink
node $t$. Each edge has a weight, also called capacity, which is a non-
negative number.

######  Definition 11 (Cut in a Network).

A cut is a partition of the nodes into two sets $A$ and $B$, where $s\in A$
and $t\in B$. Correspondingly, cut-set is the set of edges that go from $A$ to
$B$. Equivalently, a cut-set is a set of edges whose removal disconnects the
source node of the network from the sink node of the network. For simplicity,
we refer to cut-sets as cuts as well. The value of a cut is the sum of weights
of the edges in the cut. A minimal cut is a cut whose value is minimal among
all cuts in the network.

To introduce the algorithm, we reformulate the MWVC task as a minimal network
cut problem. We first construct the network $N$.333This network construction
was inspired by Goemans (2008). Its nodes are the source node $s$, one node
for every element of $F$ and one for every element of $Y$, and the sink node
$t$. There are three groups of edges in $N$: for every node $u\in
V_{\text{col}}$, an edge goes from $s$ to $u$ with weight $2r+1$; for every
pair $u,v$ with ${\delta}_{v,u}=1$, an edge goes from $u$ to $v$ in $N$ with
infinite weight; and for every node $v\in V_{\text{row}}$, an edge goes from
$v$ to $t$ with weight $r$. Figure 4 shows an example of the construction.
Importantly, the size of $N$ is polynomial in the size parameters $m$ and $r$
of the model as $N$ has $m+r+2$ nodes and less than $m+r+mr$ edges. In the
remaining part of this section, we will show that the minimal cut of $N$ is
equal to $M^{\star}$. Then, we are finished, since the minimal cut can be
computed in $P(r,m)=O((m+r+2)^{2}(m+r+mr))$ steps using Dinic’s algorithm
(Tarjan, 1987, chapter 8), where $O$ is the Big-O notation.

In the above MWVC task, let us denote by $S\subseteq V_{\text{col}}\cup
V_{\text{row}}$ a set of vertices. If $S$ is a vertex cover, then it
corresponds to a finite-value cut $C$: for every included column $u\in S\cap
V_{\text{col}}$ (respectively, included row $v\in S\cap V_{\text{row}}$), the
corresponding edge going from $s$ to $u$ (respectively, from $v$ to $t$) is
included in $C$, and nothing else. $C$ has finite value because it only
contains finite-weighted edges. More concretely, the value of $C$ coincides
with the weight $M^{\star}$ of the MWVC. $C$ is also a cut, since for each
edge $u\to v$ going from $V_{\text{col}}$ to $V_{\text{row}}$, either $s\to u$
or $v\to t$ is included in $C$; that is the implication of $S$ being a vertex
cover. Conversely, each finite valued cut corresponds to a vertex cover by the
same construction.

Since there exists at least one vertex cover (e.g., all vertices), there
exists a minimum, and, as we have seen, its total weight is equal to the value
of a minimal cut in $N$. What we have proven is summarized in Theorem 7.

######  Theorem 7.

Property $\text{CR}({r},{1})$ can be verified algorithmically in $P(r,m)$
steps, where $P(r,m)$ is a polynomial in $r$ and $m$. ∎

This theorem and its proof give a means for efficient verification of generic
variance identification in any sparse factor analysis based on the Anderson-
Rubin row deletion theorem.

######  Remark.

As noted above, $P(r,m)=O((m+r+2)^{2}(m+r+mr))$, where $O$ is the Big-O
notation. For fixed $r$, this is $O(m^{3})$, and for fixed $m$, this is
$O(r^{3})$. Furthermore, if we look back to Theorem 4 and consider the removal
of $s$ rows as a sequential procedure, first $s-1$ rows and then another, then
we get a polynomial time (in $r$ and $m$ for fixed $s$) algorithm for
$\text{CR}({r},{s})$ the following way. It is easy to see that ${\delta}$ is
$\text{CR}({r},{s})$ if and only if after removing any $s-1$ rows the
remaining binary matrix is $\text{CR}({r},{1})$. This trivially gives rise to
an algorithm with complexity $O(m^{s-1})P(r,m)$, which may be practical for
$s=2$ or $s=3$ for small $m$.

| $u_{1}$ | $u_{2}$ | $u_{3}$
---|---|---|---
$v_{1}$ | 1 | 0 | 0
$v_{2}$ | 0 | 1 | 0
$v_{3}$ | 1 | 1 | 0
$v_{4}$ | 1 | 0 | 1
$v_{5}$ | 1 | 1 | 1
$v_{6}$ | 0 | 0 | 1
$v_{7}$ | 0 | 1 | 1
$v_{8}$ | 0 | 1 | 0
$\qquad\iff\qquad$$u_{1}$$u_{2}$$u_{3}$$v_{1}$$v_{2}$$v_{3}$$v_{4}$$v_{5}$$v_{6}$$v_{7}$$v_{8}$$w=7$$w=3$

$\Big{\Updownarrow}$$\Big{\Updownarrow}$$s$$u_{1}$$u_{2}$$u_{3}$$v_{1}$$v_{2}$$v_{3}$$v_{4}$$v_{5}$$v_{6}$$v_{7}$$v_{8}$$t$$7$$\infty$$3$

Figure 4: An example ${\delta}$ (top left) with $r=3$ and $m=8$, and a
corresponding bipartite graph $B_{\text{adj}}$ and network $N$ (bottom).
Binary variables $u_{i}$ and $v_{j}$ correspond to the columns and rows of
${\delta}$, respectively, $i=1,2,3$, $j=1,\ldots,8$. In $B_{\text{adj}}$,
vertices of $V_{\text{col}}$ and $V_{\text{row}}$ have weight $7$ and $3$,
respectively. In $N$, edges between $s$ and $V_{\text{col}}$, between
$V_{\text{col}}$ and $V_{\text{row}}$, and between $V_{\text{row}}$ and $t$
have weight $7$, $\infty$, and $3$, respectively.

##  4 Numerical Illustration

We demonstrate that missing variance identification may unnecessarily inflate
the estimated number of factors during EFA. Bayesian estimation is applied,
which allows us to emulate matrix sparsity using a hierarchical prior
distribution on ${\beta}$. With that, we compare two estimation strategies.
Under the first one, variance identification as a step is ignored.
Alternatively, the prior on ${\beta}$ is restricted to the sufficient
condition of Section 2 for variance identification. Two measures are
considered for the comparison: out-of-sample predictive likelihood and the
number of factors used during prediction.

###  4.1 Setup

Weekly returns of 17 currencies against the EUR are investigated between
January, 2003, and December, 2005. The series include the currencies of big
trading partners of the Eurozone (Australian Dollar, Canadian Dollar, British
Pound, Hong Kong Dollar, Japanese Yen, South Korean Won, New Zealand Dollar,
Russian Ruble, Turkish Lira, and US Dollar), and important local partners
(Swiss Franc, Czech Koruna, Danish Krone, Norwegian Krone, Polish Zloty,
Romanian Leu, and Swedish Krona). The chosen time period mostly avoids large
international crises and heavy-tailed return distributions, as depicted in
Figure 5, which renders the static latent factor model of Equation (1)
appropriate for its analysis. For this demonstration, however, we make the
simplifying parametric assumption that observations and factors follow a
multivariate Gaussian distribution:

$y_{t}\sim N_{m}({\beta}f_{t},\Psi),\qquad f_{t}\sim N_{r}(0,\Sigma_{f}).$ (2)

Furthermore, to facilitate variance identification through
$\text{CR}({r},{1})$, we follow the tradition of West (2003) and introduce
indicator variables ${\delta}_{ij}\in\\{0,1\\}$ for every factor loading as
parameters to estimate. This also gives us the opportunity to easily embed
Equation (2) into a potentially overfitting factor model, where the indicator
matrix ${\delta}=\\{{\delta}_{ij}\\}$, $i=1,\ldots,17$, and
$j=1,\ldots,{r_{\text{max}}}$, attains the maximal444$2{r_{\text{max}}}+1\leq
m=17$, where $m$ is the number of observation series, is essential for
variance identification via $\text{RD}({r},{1})$, and therefore also via
$\text{CR}({r},{1})$. width ${r_{\text{max}}}=8$ for this data set.

Figure 5: Data set of 17 exchange rates against EUR.

Following Conti et al. (2014), and Kaufmann and Schuhmacher (2019), Bayesian
posterior sampling is applied with a conjugate prior on ${\beta}$ and $\Psi$,
combined with column-wise shrinkage on the indicators;

$\begin{split}\beta_{ij}\mid{\delta}_{ij}=0&\equiv 0,\\\
\beta_{ij}\mid{\delta}_{ij}=1&\sim N(0,\sigma_{i}^{2}),\\\ \sigma_{i}^{2}&\sim
IG(c_{0},C_{0}),\\\ {\delta}_{ij}&\sim Ber(\tau_{j}),\\\ \tau_{j}&\sim
B(a_{0},b_{0}),\end{split}$ (3)

where $IG(c_{0},C_{0})$ denotes the inverted Gamma distribution with kernel
density $x^{-c_{0}-1}\exp(-C_{0}/x)$, $Ber(\tau_{j})$ is the Bernoulli
distribution with success probability $\tau_{j}$, and $B(a_{0},b_{0})$ is the
Beta distribution with kernel density $x^{a_{0}-1}(1-x)^{b_{0}-1}$. The choice
of $\sigma_{i}^{2}$ as the variance lets $\beta_{ij}$ capture potential
scaling differences between the observation series. The fairly vague prior
$(c_{0},C_{0})=(1,0.3)$ is adopted from Legramanti et al. (2020), where
$i=1,\ldots,m$. Following Frühwirth-Schnatter and Lopes (2018),
$(a_{0},b_{0})=(0.2,0.6)$ is chosen first, which, Ghahramani et al. (2007)
argue, is an example of a sticky prior and achieves shrinkage within each
column. Later, the uniform prior $(a_{0},b_{0})=(1,1)$ is also considered for
comparison.

###  4.2 Prediction exercise

Denote by $\theta=({\beta},{\delta},\\{\sigma^{2}_{i}\\},\\{\tau_{j}\\})$ the
collection of parameters to be estimated. After one run of Markov chain Monte
Carlo (MCMC) sampling, we compare two strategies in this section. Under the
first one, variance identification is ignored and every draw $\tilde{\theta}$
is kept. Alternatively, and this is implemented as a post-processing step
after MCMC, we only keep realizations $\tilde{\theta}$ such that
$\tilde{\delta}$ satisfies $\text{CR}({\tilde{r}},{1})$, where
$\tilde{r}\leq{r_{\text{max}}}$ is the number of nonzero columns in
$\tilde{\delta}$, and compute summaries from the filtered sample. This way,
due to the hierarchical setup of equation (3), we restrict ${\delta}\mid{y}$
such that $\tilde{\Psi}\mid({y},\tilde{\delta})$ is (generically) identified.
Given that $\tilde{\delta}$ is accepted for further use, realizations of
$(\tilde{\beta},\tilde{\Psi})\mid({y},\tilde{\delta})$ is used for prediction.
Note that the predictive density of model (2) is invariant to rotations of
$\tilde{\beta}$, therefore we do not take steps toward the identification of
$\tilde{\beta}$.

Estimation is done on 52 weekly returns, and the log-predictive-likelihood is
estimated for the next weekly return as the mean of the realized log-
predictive-likelihoods. Then, the time window is shifted by one week, and
estimation and prediction are repeated. The procedure is done 100 times, which
covers approximately two years of weekly predictions under a moving window
regime.

###  4.3 Results

The top chart of Figure 6 shows the difference in log-predictive-likelihoods
between an unidentified and an identified sparse matrix space via Corollary 3
when applying column shrinkage to ${\delta}$. Restricting the prior to
identified patterns does not significantly affect predictive performance of
the factor model. The bottom chart displays the estimated number of factors
$\tilde{r}$ for both priors and the two strategies throughout the shifting
periods. Variance identification consistently reduces $\tilde{r}$ by
$0.6$–$1.1$ for the shrinkage prior. The difference is also significant: not
just the posterior mean but the entire posterior distribution shifts downwards
in all time periods as variance identification is incorporated into EFA.

Figure 6: Top: Log-predictive-likelihood (LPL) without variance identification
minus LPL with the prior restricted to variance identified models. Dots
represent the difference in posterior means and the gray intervals are the
$\pm 2$ posterior standard deviation credible region under the unrestricted
prior. The uniform prior results in a very similar image. Bottom: Posterior
distribution of the number of factors through time for the shrinkage prior
(top; $a_{0}=0.2$ and $b_{0}=0.6$) and the uniform prior (bottom; $a_{0}=1$
and $b_{0}=1$), and with (left) and without (right) variance identification.
The size of the gray bubbles represents the posterior probability, and only
every fourth period is shown for improved legibility. The black line and dots
highlight the posterior mean for every time period.

As a simple prior sensitivity check, we consider a uniform prior
$(a_{0},b_{0})=(1,1)$ on the column-specific shrinkage parameter $\tau_{j}$,
$j=1,...,{r_{\text{max}}}$, and compare it to the aforementioned shrinkage
prior. As shown in Figure 6, a significant increase in $\tilde{r}$ results
from the reduced shrinkage in both columns. At the same time, the bottom row
provides evidence for variance identification continuing to reject too large
models while maintaining predictive performance (results not reported here for
the uniform prior for brevity).

In our experience, the share of variance identified matrices increases in the
posterior sample with more shrinkage, and this is reflected in Figure 7, which
shows the posterior proportion of variance identified ${\delta}$ matrices
under the two prior specifications. The shrinkage prior prefers either close
to empty or close to full columns a priori, separately for each column. In
contrast, the uniform prior produces close to half full columns a priori. This
strongly spills over to the posterior distribution for this data set as can be
seen from the proportions. The counting rule is more likely satisfied with
more crowded columns, which results in less than every 10th draw to be
accepted in most time periods under the uniform prior, making the estimation
procedure for $\tilde{r}$ rather inefficient. At the same time, between 20 and
40% are accepted in most time periods under the shrinkage prior.

Figure 7: Fraction of variance identified draws in the posterior sample under
the shrinkage and the uniform prior.

##  5 Conclusion

In this paper, we have provided a computationally efficient tool for generic
variance identification in sparse factor models. To that end, we have
introduced the framework of sparse matrix spaces and some classical elements
from graph and network theory, which are rarely used in the econometrics
literature. As demonstrated through a numerical illustration with real data,
this new approach can be used to reduce the estimated number of factors while
retaining predictive performance of the sparse factor model.

## References

  * Anderson [2003] Theodore W. Anderson.  _An Introduction to Multivariate Statistical Analysis_.  Wiley, Chichester, 3rd edition, 2003.
  * Anderson and Rubin [1956] Theodore W. Anderson and Herman Rubin.  Statistical inference in factor analysis.  In _Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability_ , volume V, pages 111–150, 1956.
  * Bekker and ten Berge [1997] Paul A. Bekker and Jos M. F. ten Berge.  Generic global indentification in factor analysis.  _Linear Algebra and its Applications_ , 264:255–263, 1997\.  doi: 10.1016/S0024-3795(96)00363-1.
  * Bhattacharya and Dunson [2011] Anirban Bhattacharya and David B. Dunson.  Sparse Bayesian infinite factor models.  _Biometrika_ , 98:291–306, 2011.  doi: 10.1093/biomet/asr013.
  * Bolfarine et al. [2021] Henrique Bolfarine, Carlos M. Carvalho, Hedibert F. Lopes, and Jared S. Murray.  Decoupling shrinkage and selection in Gaussian linear factor analysis.  arXiv Preprint 2006.11908, 2021.
  * Bollen [1989] Kenneth A. Bollen.  _Structural Equations with Latent Variables_.  John Wiley & Sons, Inc., Hoboken, NJ, USA, 1989.  doi: 10.1002/9781118619179.
  * Conti et al. [2014] Gabriella Conti, Sylvia Frühwirth-Schnatter, James Joseph Heckman, and Rémi Piatek.  Bayesian exploratory factor analysis.  _Journal of Econometrics_ , 183:31–57, 2014.  doi: 10.1016/j.jeconom.2014.06.008.
  * Conway and Huffcutt [2003] James M. Conway and Allen I. Huffcutt.  A review and evaluation of exploratory factor analysis practices in organizational research.  _Organizational Research Methods_ , 6(2):147–168, 2003.  doi: 10.1177/1094428103251541.
  * Dunn [1973] James E. Dunn.  A note on sufficiency condition for uniqueness of a restricted factor matrix.  _Psychometrika_ , 38:141–143, 1973.  doi: 10.1007/BF02291181.
  * Egerváry [1931] Jenő Egerváry.  Mátrixok kombinatorikus tulajdonságairól.  _Matematikai és Fizikai Lapok_ , 38:16–28, 1931.  In Hungarian.
  * Fabrigar et al. [1999] Leandre R. Fabrigar, Duane T. Wegener, Robert C. MacCallum, and Erin J. Strahan.  Evaluating the use of exploratory factor analysis in psychological research.  _Psychological Methods_ , 4(3):272–299, 1999\.  doi: 10.1037/1082-989X.4.3.272.
  * Ford et al. [1986] J. Kevin Ford, Robert C. MacCallum, and Marianne Tait.  The application of exploratory factor analysis in applied psychology: A critical review and analysis.  _Personnel Psychology_ , 39(2):291–314, 1986\.  doi: 10.1111/j.1744-6570.1986.tb00583.x.
  * Frühwirth-Schnatter and Lopes [2010] Sylvia Frühwirth-Schnatter and Hedibert Lopes.  Parsimonious Bayesian factor analysis when the number of factors is unknown.  Research report, Booth School of Business, University of Chicago, 2010\.
  * Frühwirth-Schnatter and Lopes [2018] Sylvia Frühwirth-Schnatter and Hedibert Lopes.  Sparse Bayesian factor analysis when the number of factors is unknown.  arXiv Preprint 1804.04231, 2018.
  * Ghahramani et al. [2007] Zoubin Ghahramani, Thomas L. Griffiths, and Peter Sollich.  Bayesian nonparametric latent feature models (with discussion and rejoinder).  In J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West, editors, _Bayesian Statistics 8_ , pages 1–25. Oxford University Press, Oxford, 2007.
  * Goemans [2008] Michel Xavier Goemans.  Advanced algorithms: Problem set solution 5.  MIT OpenCourseWare, Course Number 6.854J/18.415J, 2008.
  * Gorsuch [1983] Richard L. Gorsuch.  _Factor Analysis_.  Lawrence Erlbaum Associates, Hillsdale, New Jersey, 2nd edition, 1983\.  doi: 10.4324/9780203781098.
  * Hall [1935] Philip Hall.  On representatives of subsets.  _Journal of the London Mathematical Society_ , s1-10(1):26–30, 1935.  doi: 10.1112/jlms/s1-10.37.26.
  * Howard [2016] Matt C. Howard.  A review of exploratory factor analysis decisions and overview of current practices: What we are doing and how can we improve?  _International Journal of Human-Computer Interaction_ , 32(1):51–62, 2016.  doi: 10.1080/10447318.2015.1087664.
  * Kaiser [1958] Henry F. Kaiser.  The varimax criterion for analytical rotation in factor analysis.  _Psychometrika_ , 23:187–200, 1958.  doi: 10.1007/BF02289233.
  * Kaufmann and Schuhmacher [2019] Sylvia Kaufmann and Christian Schuhmacher.  Bayesian estimation of sparse dynamic factor models with order-independent and ex-post identification.  _Journal of Econometrics_ , 210:116–134, 2019.  doi: 10.1016/j.jeconom.2018.11.008.
  * Kőnig [1931] Dénes Kőnig.  Gráfok és mátrixok.  _Matematika és Fizikai Lapok_ , 38:116–119, 1931.  In Hungarian.
  * Leckman et al. [1997] James F. Leckman, Dorothy E. Grice, James Boardman, Heping Zhang, Amy Vitale, Colin Bondi, John Alsobrook, Bradley S. Peterson, Donald J. Cohen, Steven A. Rasmussen, Wayne K. Goodman, Christopher J. McDougle, and David L. Pauls.  Symptoms of obsessive-compulsive disorder.  _American Journal of Psychiatry_ , 154(7):911–917, 1997.  doi: 10.1176/ajp.154.7.911.
  * Ledermann [1937] Walter Ledermann.  On the rank of the reduced correlational matrix in multiple-factor analysis.  _Psychometrika_ , 2:85–93, 1937.  doi: 10.1007/BF02288062.
  * Legramanti et al. [2020] Sirio Legramanti, Daniele Durante, and David B. Dunson.  Bayesian cumulative shrinkage for infinite factorizations.  _Biometrika_ , 107:745–752, 2020.  doi: 10.1093/biomet/asaa008.
  * Lucas et al. [2006] Joe Lucas, Carlos Carvalho, Quanli Wang, Andrea Bild, Joe R. Nevins, and Mike West.  Sparse statistical modelling in gene expression genomics.  In K. Do, P. Müller, and M. Vannucci, editors, _Bayesian Inference for Gene Expression and Proteomics_ , pages 155–176. Cambridge University Press, Cambridge, UK, 2006.  doi: 10.1017/CBO9780511584589.009.
  * McCrae and Costa [1997] Robert R. McCrae and Paul T. Costa.  Personality trait structure as a human universal.  _American Psychologist_ , 52(5):509–516, 1997\.  doi: 10.1037/0003-066X.52.5.509.
  * Reiersøl [1950] Olav Reiersøl.  On the identifiability of parameters in Thurstone’s multiple factor analysis.  _Psychometrika_ , 15:121–149, 1950.  doi: 10.1007/BF02289197.
  * Ročková and George [2017] Veronika Ročková and Edward I. George.  Fast Bayesian factor analysis via automatic rotation to sparsity.  _Journal of the American Statistical Association_ , 111:1608–1622, 2017.  doi: 10.1080/01621459.2015.1100620.
  * Sato [1992] Manabu Sato.  A study of an identification problem and substitute use of principal component analysis in factor analysis.  _Hiroshima Mathematical Journal_ , 22:479–524, 1992.  doi: 10.32917/HMJ/1206128499.
  * Shapiro [1985] Alexander Shapiro.  Identifiability of factor analysis: Some results and open problems.  _Linear Algebra and its Applications_ , 70:1–7, 1985.
  * Tarjan [1987] Robert Endre Tarjan.  _Data Structures and Network Algorithms_.  Society for Industrial and Applied Mathematics, 1987.  doi: 10.1137/1.9781611970265.
  * Thompson [2004] Bruce Thompson.  _Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications_.  American Psychological Association, Washington, 2004.  doi: 10.1037/10694-000.
  * Thurstone [1931] Louis Leon Thurstone.  Multiple factor analysis.  _Psychological Review_ , 38(5):406–427, 1931\.  doi: 10.1037/h0069792.
  * Thurstone [1935] Louis Leon Thurstone.  _The Vectors of Mind_.  University of Chicago, Chicago, 1935.
  * Tumura and Sato [1980] Yosiro Tumura and Manabu Sato.  On the identification in factor analysis.  _TRU Mathematics_ , 16:121–131, 1980.  doi: 10.20604/00001766.
  * West [2001] Douglas B. West.  _Introduction to Graph Theory_.  Prentice Hall, Hoboken, New Jersey, 2nd edition, 2001.
  * West [2003] Mike West.  Bayesian factor regression models in the “large $p$, small $n$” paradigm.  In J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West, editors, _Bayesian Statistics 7_ , pages 733–742. Oxford University Press, Oxford, 2003.
  * Williams [2020] Benjamin Williams.  Identification of the linear factor model.  _Econometric Reviews_ , 39:92–109, 2020.  doi: 10.1080/07474938.2018.1550042.
  * Zhao et al. [2016] Shiwen Zhao, Chuan Gao, Sayan Mukherjee, and Barbara E. Engelhardt.  Bayesian group factor analysis with structured sparsity.  _Journal of Machine Learning Research_ , 17:1–47, 2016\.