††thanks: Corresponding author††thanks: Corresponding author

# Node metadata can produce predictability transitions in network inference
problems

Oscar Fajardo-Fontiveros<EMAIL_ADDRESS>Department of Chemical
Engineering, Universitat Rovira i Virgili, 43007 Tarragona, Catalonia  Marta
Sales-Pardo<EMAIL_ADDRESS>Department of Chemical Engineering,
Universitat Rovira i Virgili, 43007 Tarragona, Catalonia  Roger Guimerà
<EMAIL_ADDRESS>ICREA, 08010 Barcelona, Catalonia  Department of
Chemical Engineering, Universitat Rovira i Virgili, 43007 Tarragona, Catalonia

###### Abstract

Network inference is the process of learning the properties of complex
networks from data. Besides using information about known links in the
network, node attributes and other forms of network metadata can help to solve
network inference problems. Indeed, several approaches have been proposed to
introduce metadata into probabilistic network models and to use them to make
better inferences. However, we know little about the effect of such metadata
in the inference process. Here, we investigate this issue. We find that,
rather than affecting inference gradually, adding metadata causes abrupt
transitions in the inference process and in our ability to make accurate
predictions, from a situation in which metadata does not play any role to a
situation in which metadata completely dominates the inference process. When
network data and metadata are partly correlated, metadata optimally
contributes to the inference process at the transition between data-dominated
and metadata-dominated regimes.

###### pacs:

Many systems can be represented as networks, with nodes representing units
(for example, people in a social network, or proteins in a protein-protein
interaction network), and links representing interactions between the units
(for example, friendship relationships or physical binding interactions
between proteins). Network inference is the process of inferring the
properties of those networks from data; typical network inference problems
include the identification of groups of nodes with similar connection
patterns, or the identification of unobserved interactions, that is, link
prediction Liben-Nowell and Kleinberg (2007); Clauset  _et al._ (2008);
Guimerà and Sales-Pardo (2009); Lü  _et al._ (2015); Ghasemian  _et al._
(2020); Guimerà (2020). Network inference and, in particular, link prediction
are increasingly important in problems with applications ranging from the
prediction of interactions between drugs Guimerà and Sales-Pardo (2013);
Tarrés-Deulofeu  _et al._ (2019); Menden  _et al._ (2019) to the prediction of
human preferences and decisions Guimerà and Sales-Pardo (2011); Guimerà  _et
al._ (2012); Godoy-Lorite  _et al._ (2016a); Cobo-López  _et al._ (2018).

Typically, network inference starts from observations of some of the links in
the network, which are used to predict unobserved links or to infer other
network properties. However, other sources of information such as system
dynamics Timme (2007); Peixoto (2019) or node attributes Tallberg (2004); Yang
_et al._ (2013); Hric  _et al._ (2016); Newman and Clauset (2016); White and
Murphy (2016); Peel  _et al._ (2017); Cobo-López  _et al._ (2018); Stanley
_et al._ (2019); Contisciani  _et al._ (2020) can also be used to aid in the
inference process. Here we study how node attributes are introduced in the
inference process, and what is the effect of using such metadata.

We present our work in terms of the problem of link prediction in recommender
systems Koren  _et al._ (2009); Guimerà  _et al._ (2012); Godoy-Lorite  _et
al._ (2016a), in which the goal is to predict the association between users
and items (for example, books or movies). However, our conclusions apply to
network inference problems in general. We introduce a multipartite network
model that encompasses and generalizes previous attempts to use node metadata
in network inference problems (Fig. 1). Within this framework, the problem of
link prediction in general unipartite or bipartite networks is just a
particular case. Unlike most previous approaches, our multipartite network
model allows us to control the importance of the node metadata and thus to
investigate when and how metadata helps in the inference.

We find that, contrary to what one may expect, node metadata do not affect the
inference problem gradually. Rather, even when the weight of metadata
increases smoothly, the inference process undergoes a transition from a
situation in which metadata does not play any role, to a situation in which
metadata completely dominates the inference process. When network data and
metadata are partly correlated, metadata optimally contributes to the
inference process at the transition between data-dominated and metadata-
dominated regimes.

##  I Multipartipartite mixed-membership stochastic block models with labeled
links

Figure 1: Multipartipartite mixed-membership stochastic block model with
labeled links. (a), We cast the recommendation problem (in which one aims to
predict how users will rate certain items) into a network inference problem.
Here, users rate movies with three possible ratings (green, orange or red).
Additionally, we have excluding attributes for users (two excluding genders
and three excluding age groups, represented by different shades of the same
color) and non-excluding attributes for movies (two movie genres; the
connection to these attributes is binary, yes/no, but in general it does not
need to be). Similar to ratings, we represent these attributes as bipartite
networks. Although we frame our description of the model in terms of
recommendations or link prediction in a bipartite network, the problem of link
prediction in regular unipartite networks is just a particular case in which
user nodes and item nodes are the same. (b) Each bipartite network in the
multipartite network is modeled using a mixed-membership stochastic block
model (see text). The individual block models are coupled by the user and item
membership vectors ($\boldsymbol{\theta}$ and $\boldsymbol{\eta}$,
respectively), shown in (c) along with all other model parameters and their
dimensions (see text).

We introduce a very general network model based on stochastic block models
Holland  _et al._ (1983); Nowicki and Snijders (2001); Guimerà and Sales-Pardo
(2009) that allows us to deal with (directed or undirected) unipartite and
bipartite networks, whose links are binary or labeled, and with node
attributes of different types that can be combined as needed (Fig. 1). As we
discuss below, this model extends and generalizes previous models.

In what follows we use the terminology of recommender systems Koren  _et al._
(2009); Guimerà  _et al._ (2012); Godoy-Lorite  _et al._ (2016a) although, as
previously mentioned, the model is completely general and applicable to any
type of relational data with node attributes. Our objective is to model a
bipartite network with labeled links connecting $N$ users to $M$ items (for
example, movies or books). Links $r_{ij}$ represent ratings of users $i$ to
items $j$ and are labeled, that is, $r_{ij}$ can take values in a finite
discrete set such as $\\{{\rm like},{\rm dislike}\\}$, $\\{{\rm green},{\rm
yellow},{\rm red}\\}$, or $\\{0,1,\dots,R\\}$. To model these ratings, we
assume that: (i) there are user and item groups, and users and items belong to
mixtures of such groups; (ii) the probability that a user $i$ rates item $j$
with $r_{ij}$ depends only of the groups to which they belong.

These assumptions lead to a bipartite Guimerà and Sales-Pardo (2011); Guimerà
_et al._ (2012); Yen and Larremore (2020) mixed-membership Airoldi  _et al._
(2008) stochastic block model Godoy-Lorite  _et al._ (2016a) in which the
probability that user $i$ gives item $j$ a rating $r$ is

$\Pr[r_{ij}=r]=\sum_{\alpha\beta}\theta_{i\alpha}\eta_{j\beta}p_{\alpha\beta}(r)\;.$
(1)

Here, $\boldsymbol{\theta}_{i}$ is the normalized membership vector of user
$i$, and each element $\theta_{i\alpha}$ represents the probability that user
$i$ belongs to group $\alpha$ (with $\sum_{\alpha}\theta_{i\alpha}=1$).
Similarly, $\boldsymbol{\eta}_{j}$ is the normalized membership vector of item
$j$; $\eta_{j\beta}$ represents the probability that item $j$ belongs to group
$\beta$. Finally, $p_{\alpha\beta}(r)$ is the probability that a user in group
$\alpha$ and an item in group $\beta$ are connected with a rating $r$. The
normalization condition here is $\sum_{r}p_{\alpha\beta}(r)=1$.

We note that the association between nodes (users and items) and attributes
can also be represented as a bipartite network. Therefore we can model node-
attribute associations in a similar manner to ratings. Because we are
interested in how node attributes can help in the inference of the model for
ratings ($\boldsymbol{\theta},\boldsymbol{\eta},{\bf p})$, we consider that
membership vectors for users ($\boldsymbol{\theta}$) and items
($\boldsymbol{\eta}$) in their respective attribute networks are the same as
in the model for the ratings.

We consider both excluding and non-excluding attributes. For excluding
attributes, having one attribute excludes from having another; for example, a
user’s age group cannot be 30-39 years old and 40-49 years old simultaneously.
We model each set of excluding attributes as a single attribute node (for
example, an age node) that is connected to users or items through labeled
links (each label representing a mutually excluding age group in the example).
The probability that user $i$ has an excluding attribute $e$ (that is, the
probability that the link $e_{i\ell}$ between user $i$ and attribute node
$\ell$ is of type $e$) is

$\Pr[e_{i\ell}=e]=\sum_{\alpha}\theta_{i\alpha}q_{\alpha}(e)\;,$ (2)

where $q_{\alpha}(e)$ is the probability that a user of group $\alpha$ has an
attribute of type $e$, and $\sum_{e}q_{\alpha}(e)=1$. For items, the
expression is identical except that we use item membership vectors $\eta$
instead of user membership vectors $\theta$.

We also consider non-excluding attributes, such as item genre (for example, a
movie could be both “action” and “western”). We model each of these non-
excluding attribute types as individual attribute nodes connected to user or
item nodes by links that are typically binary (either do or do not have the
attribute) but that could in general be also labeled. Then, the probability
that item $i$ has attribute $g$ of type $a$ is also modeled using a mixed-
membership, bipartite stochastic block model

$\Pr[a_{ig}=a]=\sum_{\alpha\gamma}\theta_{i\alpha}\,\zeta_{g\gamma}\,\hat{q}_{\alpha\gamma}(a)$
(3)

where $\zeta_{g\gamma}$ is the membership vector of attribute $g$ and
$\hat{q}_{\alpha\gamma}(a)$ is the probability that a user in group $\alpha$
has an attribute of type $a$ for an attribute in attribute group $\gamma$. As
before, the expression for item non-excluding attributes is identical, just
replacing user membership vectors $\boldsymbol{\theta}$ by item membership
vectors $\boldsymbol{\eta}$.

##  II Model posterior and inference

Our objective is to model the observed ratings $R^{O}$, and to predict the
value of some unobserved ratings $R$. For this, and given Eq. (1), we need to
infer the parameters $\boldsymbol{\theta},\boldsymbol{\eta}$ and
$\boldsymbol{p}$ from $R^{O}$; the posterior distribution over these
parameters is given by

$\displaystyle P(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p}|R^{O})$
$\displaystyle\propto$ $\displaystyle
P(R^{O}|\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})\,P(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})$
(4) $\displaystyle\equiv$ $\displaystyle
L^{R}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})\,P(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})\;,$

where
$L^{R}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})=P(R^{O}|\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})$
is the likelihood of the model and
$P(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})$ is the prior over
model parameters. According to Eq. (1), the likelihood is

$L^{R}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})=\prod_{(i,j)\in
R^{O}}\left[\sum_{\alpha\beta}\theta_{i\alpha}\eta_{j\beta}p_{\alpha\beta}(r^{O}_{ij})\right]\;.$
(5)

Similarly, if we decide to jointly model the ratings and the metadata encoded
in the observed user and item attributes $A^{O}$, we also need to infer the
values of the parameters $\boldsymbol{\zeta}$, $\boldsymbol{q}$ and
$\boldsymbol{\hat{q}})$ using the posterior

$\displaystyle
P(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\zeta},\boldsymbol{p},\boldsymbol{q},\boldsymbol{\hat{q}}|R^{O},A^{O})$
$\displaystyle\propto$ $\displaystyle
L^{R}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})\times$ (6)
$\displaystyle\times$
$\displaystyle\prod_{k}L^{A_{k}}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\zeta},\boldsymbol{q},\boldsymbol{\hat{q}})\times$
$\displaystyle\times$ $\displaystyle
P(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\zeta},\boldsymbol{p},\boldsymbol{q},\boldsymbol{\hat{q}})$

where
$L^{A_{k}}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\zeta},\boldsymbol{q},\boldsymbol{\hat{q}})=P(A_{k}^{O}|\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\zeta},\boldsymbol{q},\boldsymbol{\hat{q}})$
is the likelihood of the $k$-th attribute network (for example, the age
attribute network for users, or the genre attribute network for items). For
the $k$-th excluding attribute, this likelihood reads

$L^{A_{k}}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{q})=\prod_{(i,\ell_{k})\in
A_{k}^{O}}\left[\sum_{\alpha}\theta_{i\alpha}q^{k}_{\alpha}((e_{k}^{O})_{i\ell_{k}})\right]\;,$
(7)

where $\ell_{k}$ is the $k$-th non-excluding attribute and the product is over
all nodes $i$ for which we observe attribute $\ell_{k}$.

For the $k$-th non excluding attribute we have

$L^{A_{k}}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\zeta},\boldsymbol{\hat{q}})=\prod_{(i,g)\in
A_{k}^{O}}\left[\sum_{\alpha\gamma}\theta_{i\alpha}\zeta^{k}_{g\gamma}\hat{q}^{k}_{\alpha\gamma}((a_{k}^{O})_{ig})\right]\;.$
(8)

where the product is over all observed associations between nodes $i$ and
attributes $g$ within the $k$-th class of non-excluding attributes.

Ignoring normalizing constants, and in a spirit similar to Refs. Yang  _et
al._ (2013); Contisciani  _et al._ (2020), we define a parametric log-
posterior as

$\displaystyle\pi(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\zeta},\boldsymbol{p},\boldsymbol{q},\boldsymbol{\hat{q}}|R^{O},A^{O})$
$\displaystyle=$
$\displaystyle\mathcal{L}^{R}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})+$
(9) $\displaystyle+$
$\displaystyle\sum_{k}\lambda_{k}\mathcal{L}^{A_{k}}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\zeta},\boldsymbol{q},\boldsymbol{\hat{q}})\;,$

where $\mathcal{L}^{R}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p})$
and
$\mathcal{L}^{A_{k}}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\zeta},\boldsymbol{q},\boldsymbol{\hat{q}})$
are the log-likelihoods of ratings and attributes, respectively. For
$\lambda_{k}=0$, we recover Eq. (4) with uniform priors on the parameters,
thus completely ignoring all metadata. Conversely, for $\lambda_{k}=1$, we are
jointly modeling the network of ratings and the network of attributes as in
Eq. (6), with uniform priors on the parameters. By tuning the values of
$\lambda_{k}$ we can interpolate between these situations, and extrapolate to
situations with $\lambda_{k}>1$ in which we would eventually only model the
attribute network ($\lambda_{k}\gg 1$). The terms corresponding to the
attribute models can indistinctly be interpreted as part of the likelihood of
a joint model of ratings and attributes, similar to Refs. Yang  _et al._
(2013); Hric  _et al._ (2016); Stanley  _et al._ (2019); Contisciani  _et al._
(2020), or as a non-uniform prior over membership vectors as in Refs. Tallberg
(2004); Newman and Clauset (2016); White and Murphy (2016). If interpreted as
part of a joint model, then $\lambda_{k}$ can be seen as some factors that are
needed because attribute data are somehow less (or more) reliable than rating
data, perhaps because we have reason to believe that attributes are more (or
less) subject to noise, or because each rating corresponds, in fact, to a mean
over several observations. Conversely, if interpreted as priors over the
partitions, $\lambda_{k}$ should be interpret as hyperparameters defining how
certain we are a priori about the importance of node attributes.

Either way, this parametrized posterior allows us to investigate how the
metadata encoded in the attribute networks enter the inference process for the
ratings, and under which conditions it results in better and more predictive
models for those ratings. To do this, we maximize the posterior for fixed
values of $\lambda_{k}$ using an expectation-maximization algorithm Godoy-
Lorite  _et al._ (2016a); Newman and Clauset (2016); Stanley  _et al._ (2019);
Contisciani  _et al._ (2020) (see Appendix A), which gives the most plausible
parameter values. Because the posterior landscape is in general rugged, we
perform several runs of the EM algorithm and compute the average probability
for each unobserved rating to make predictions (see Godoy-Lorite  _et al._
(2016a) and Appendix A).

##  III Relationship to previous work

The literature on using metadata for link prediction and recommender systems
is vast, and includes all sort of approaches ranging from simple heuristics to
sophisticated machine learning methods. However, our interest here is more
closely related to probabilistic approaches to network inference, even when
those approaches are not applied directly to link prediction Tallberg (2004);
Yang  _et al._ (2013); Hric  _et al._ (2016); Newman and Clauset (2016); White
and Murphy (2016); Peel  _et al._ (2017); Cobo-López  _et al._ (2018)—as shown
in Refs. Stanley  _et al._ (2019); Contisciani  _et al._ (2020), once model
parameters are inferred for, for example, community detection, they can easily
be used to predict links as well. Our focus on approaches based on
probabilistic generative models is motivated by three characteristics of such
approaches: (i) all assumptions in them are explicit; (ii) principled (as
opposed to heuristic) and sometimes even exact inference approaches are
possible; and (iii) their results are more readily interpretable. These three
characteristics make probabilistic approaches especially appropriate for our
ultimate goal of understanding how node attributes enter and help in the
inference process.

From this perspective, the multipartite mixed-membership stochastic block
model is useful because it extends and generalizes previous models. By
introducing excluding and non-excluding attributes, the model can accommodate
simultaneously attributes like those considered in Refs. Newman and Clauset
(2016); Contisciani  _et al._ (2020) (excluding) and in Refs. Yang  _et al._
(2013); Hric  _et al._ (2016) (non-excluding). It can also combine an
arbitrary number of attributes of different types, unlike approaches that can
only deal with single attributes Newman and Clauset (2016); Contisciani  _et
al._ (2020) or, more often, with a single type of attribute; and it deals
naturally with missing attribute data, unlike approaches that require all node
attributes to be known Tallberg (2004); White and Murphy (2016). Since
attributes are modeled with a stochastic block model, our approach also
automatically clusters attributes that have similar effects on the data (for
example, age groups that show similar behavior) as in Ref. Hric  _et al._
(2016). Unlike most previous approaches for attributed networks, nodes and
attributes in our model belong to mixtures of groups, which makes the model
more expressive Godoy-Lorite  _et al._ (2016a), links between nodes and to
attributes can be labeled, and the influence of the attributes can be tuned on
and off (as in Ref. Contisciani  _et al._ (2020)). As stated above, this last
feature is precisely the main focus of our work.

##  IV Synthetic data

We first use synthetic data to validate the expectation-maximization inference
approach and to investigate the role of introducing node attributes. We
generate synthetic data with a model similar to the model Fig. 1. Our
synthetic rating networks consist of 200 users and 200 items, partitioned into
$K=2$ groups of users and $L=4$ groups of items. Users have an excluding
attribute labeled “male” or “female”, and items have an excluding attribute
labeled from 0 to 3, which may represent four different genres.

In the simplest case, in which ratings and attributes are completely
correlated, all female users have membership vectors $\boldsymbol{\theta}_{\rm
f}=(0.8,0.2)$; conversely, all male users have $\boldsymbol{\theta}_{\rm
m}=(0.2,0.8)$. Similarly, an item with attribute $a$ has a membership of 0.8
to group $a$ and 0.067 to all other groups. To simulate partial correlation
$c$ or even no correlation ($c=0$) between membership vectors and attributes,
with probability $1-c$ we reassign each node attribute to a value selected
uniformly at random among all possibilities (2 for users and 4 for items).

For the experiments reported in Fig. 2, we consider all attribute links, but
only a number $|R^{O}|=400$ of observed ratings (that is, $1\%$ of all
generated ratings). Although the synthetic data are created with item genre as
an excluding attribute, we carry out the inference process assuming that genre
is a non-excluding attribute, which is what one would likely assume in real
settings where the generating model is unknown.

We infer the values of the model parameters using the expectation-maximization
equations, and use the inferred parameters to predict unobserved ratings in
the bipartite ratings network. We do this for different levels of correlation
$c$ between the ratings and the attribute networks (Fig. 2), from a situation
$c=1$ in which the attributes are perfectly correlated with user and item
membership vectors (all male users belong to one group and have identical
parameters, and all females belong to another group with different parameters;
items with each genre belong to the exact same mixture of groups) to a
situation $c=0$ in which user and item memberships and attributes are
completely uncorrelated (Fig. 2).

Figure 2: Predictive performance and effect of metadata on synthetic ratings.
We create synthetic ratings from 200 users on 200 items, with different levels
of correlation $c$ between ratings and node attributes (see text). We then use
5-fold cross-validation to calculate the performance of the expectation-
maximization equations at predicting unobserved ratings. In particular, we
take as a reference the predictive accuracy ${a}_{0}$ of the algorithm when
all attributes are ignored ($\lambda_{\rm user}=\lambda_{\rm item}=0$), and
measure relative accuracy $\alpha$ for a given pair $(\lambda_{\rm
user},\lambda_{\rm item})$ as the log-ratio $\alpha(\lambda_{\rm
user},\lambda_{\rm item})=\log\left[{a}(\lambda_{\rm user},\lambda_{\rm
item})/{a}_{0}\right]$. The value $\alpha(\lambda_{\rm user},\lambda_{\rm
item})=0$ (dashed line) thus indicates no change with respect to the reference
${a}_{0}$, and $\alpha(\lambda_{\rm user},\lambda_{\rm item})>0$
(respectively, $\alpha(\lambda_{\rm user},\lambda_{\rm item})<0$) indicates
predictions that are more (less) accurate than those obtained by ignoring node
attributes. The maximum possible relative performance (dotted line) is
obtained when each rating is assigned the exact probability that was used to
generate it. For each value of the correlation ((a)-(b), full correlation,
$c=1$; (c)-(d), $c=0.75$; (e)-(f), $c=0.50$; (g)-(h), no correlation, $c=0$)
we show the variation of $\alpha(\lambda_{\rm user},\lambda_{\rm item})$ with
$\lambda_{\rm item}$ for different values of $\lambda_{\rm user}$ (left), and
the whole dependence of $\alpha(\lambda_{\rm user},\lambda_{\rm item})$ on
both $\lambda_{\rm user}$ and $\lambda_{\rm item}$ (right).

Since we focus on sparse observations in which the number of observed ratings
is low (only $1\%$ of all ratings), model parameters cannot be inferred
accurately from the ratings alone. Therefore, when we only consider the
observed ratings $R^{O}$ and ignore all attributes $A^{O}$ by setting
$\lambda_{\rm user}=\lambda_{\rm item}=0$ in Eq. (9) ($\lambda_{\rm user}$ and
$\lambda_{\rm item}$ correspond to the user and item attribute networks,
respectively), the prediction of unobserved links is suboptimal, that is, the
inferred probabilities of unobserved links differ significantly from the
actual probabilities used to build the network.

When there is perfect correlation between node attributes and group
memberships, considering the attributes $A^{O}$ by setting $\lambda_{\rm
user}>0$ and $\lambda_{\rm item}>0$ should in principle help in the inference
process. In fact, since attributes are perfectly correlated to group
memberships, in the limit $\lambda_{\rm user}\rightarrow\infty$ and
$\lambda_{\rm item}\rightarrow\infty$ nodes will be forced into the correct
groups and predictions should be near optimal. This is what we observe in our
numerical experiments (Fig. 2a). Interestingly, as we increase the weight of
the attributes in the log-posterior from $\lambda_{\rm user}=\lambda_{\rm
item}=0$, the effect on prediction accuracy is not smooth. Rather, below
certain threshold values of $\lambda_{\rm user}$ and $\lambda_{\rm item}$,
using the attributes does not have any significant effect on prediction
accuracy. Then, at those threshold values, a transition occurs and prediction
accuracy increases abruptly until it reaches its theoretical maximum, as
expected.

When attributes and ratings are completely uncorrelated (Fig. 2d), the role of
attributes is reversed. Predictions are equally suboptimal at $\lambda_{\rm
user}=\lambda_{\rm item}=0$, but then, as $\lambda_{\rm user}$ and
$\lambda_{\rm item}$ cross certain threshold values, predictions suddenly
worsen as user and item nodes are forced into groups that are uncorrelated
with their real membership vectors and, thus, with the observed ratings.

Figure 3: Transition between data-dominated and metadata-dominated inference
regimes. For the synthetic data in Fig. 2, we plot the log-posterior
$\pi(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\zeta},\boldsymbol{p},\boldsymbol{q},\boldsymbol{\hat{q}}|R^{O},A^{O})$
as a function of the hyperparameter $\lambda=\lambda_{\rm item}=\lambda_{\rm
user}$ for three models: the model that maximizes the data likelihood $L^{R}$,
the model that maximizes the metadata likelihood $L^{A}$, and the model that
maximizes the posterior when two previous cases cross (that is, have equal
posteriors). The position of the crossing coincides with the transitions and
the maxima observed in Fig. 2.

Unlike the extreme cases of total correlation or zero correlation, when
attributes are partly correlated with the true group memberships of the nodes,
the change in performance is not monotonic as we increase the importance of
the attributes. As before, when $\lambda_{\rm user}$ and $\lambda_{\rm item}$
are small enough, we observe no difference with the situation in which the
attributes are ignored entirely. In the other extreme, when $\lambda_{\rm
user}\rightarrow\infty$ and $\lambda_{\rm item}\rightarrow\infty$ user and
item nodes are forced into groups that match partly, but not perfectly, the
true group memberships of the nodes, so the performance may increase or
decrease with respect to the situation with no attributes, depending on
whether the correlation is high (Fig. 2b) or low (Fig. 2c). However, we find
that the most predictive models in this case are those at intermediate values
of $\lambda_{\rm user}$ and $\lambda_{\rm item}$, precisely at the transition
region where both the observed ratings and the observed attributes play a role
in determining the most plausible group memberships. In this case, the
inferred node memberships do not coincide with either those that maximize
$L^{R}$ of those that maximize $L^{A_{k}}$.

To understand the transition from the rating-dominated to the attribute-
dominated regime, we study the posterior of the two extreme models
corresponding to the maximum a posterior estimates obtained by expectation-
maximization for $\lambda_{\rm user}=\lambda_{\rm item}=0$ and for
$\lambda_{\rm user}=\lambda_{\rm item}\rightarrow\infty$ (Fig. 3). These are
the most plausible models when only data (ratings) and only metadata
(attributes) are taken into consideration, respectively. Regardless of the
correlation between ratings and attributes, we find that the transition in
predictability in Fig. 2 coincides with the region where the data-dominated
and metadata-dominated posteriors cross. By considering Eq. (9) we see that
this must be the case. Indeed, for each attribute network we find three
regimes—one dominated by the $L^{R}$ term, one dominated by the $L^{A}$ term,
and one in which both terms are comparable. Unless there is perfect or almost
perfect correlation between attributes and node memberships, any improvement
in predictive power must come from considering both the observed ratings and
the observed attributes, and therefore in the transition region.

##  V Real data

Finally, we analyze two empirical data sets and study whether we observe the
same behaviors as in the synthetic data. First, we consider the 100K MovieLens
data set Harper and Konstan (2015), which contains 100,000 ratings of movies
by users. Age and gender attributes are available for users, which we model as
excluding attributes (Fig. 4). Movies have genre attributes, which we model as
non-excluding attributes. The relative weights of user and movie attributes
are given by the parameters $\lambda_{\rm users}$ and $\lambda_{\rm items}$.

Figure 4: Predictive performance and effect of metadata on the MovieLens data
set. As in Fig. 2, we take as a reference the predictive accuracy ${a}_{0}$ of
the algorithm when all attributes are ignored ($\lambda_{\rm
user}=\lambda_{\rm item}=0$), and measure relative accuracy $\alpha$ for a
given pair $(\lambda_{\rm user},\lambda_{\rm item})$ as the log-ratio
$\alpha(\lambda_{\rm user},\lambda_{\rm item})=\log\left[{a}(\lambda_{\rm
user},\lambda_{\rm item})/{a}_{0}\right]$. We consider three different
attributes for user nodes: (a)-(b), age; (c)-(d), gender; (e)-(f), age and
gender combined as a single attribute. We plot the whole range of
$\lambda_{\rm user}$ (left), and zoom into the intermediate (shaded) region of
$\lambda_{\rm user}$ in which predictions are significantly more accurate than
the reference (right).

Just as in the synthetic networks with small but finite correlation, we
observe an intermediate value of $\lambda_{\rm user}$ and $\lambda_{\rm item}$
that provides more accurate rating predictions than either considering the
observed ratings alone or considering the node attributes alone. This behavior
is similar when we consider age only, gender only, or age and gender
simultaneously. As in synthetic networks, the optimal combination of rating
data and node metadata occurs for values of $\lambda$ such that the ratings
network and the attributes networks have comparable contributions to the log-
posterior.

Second, we consider a data set on the votes of 441 members of the U.S. House
of Representatives in the 108th U.S. Congress Waugh  _et al._ (2009) (Fig. 5).
Between Jannuary 2003 and Jannuary 2005, these representatives voted on 1,217
bills, casting one of 9 different types of vote, which, following previous
analyses, we simplify to Yes, No, and Other Waugh  _et al._ (2009).

Figure 5: Predictive performance and effect of metadata on the U.S. Congress
data set. As in Fig. 2, we take as a reference the predictive accuracy
${a}_{0}$ of the algorithm when all attributes are ignored ($\lambda_{\rm
user}=0$), and measure relative accuracy $\alpha$ for a given $\lambda_{\rm
user}$ as the log-ratio $\alpha(\lambda_{\rm user})=\log\left[{a}(\lambda_{\rm
user})/{a}_{0}\right]$. We consider three different attributes for user nodes:
Party, State, and party and State simultaneously.

In this data set, “users” are the representatives and “items” are the bills.
The ratings represent the votes of the representatives on the bills. For
representatives, we have attribute data indicating their party and state,
which we model as excluding attributes. Although all votes of all members are
recorded in the data set (in total, 536,698 votes), for the purpose of our
analysis we infer the parameters of the multipartite mixed-membership
stochastic block model using 1% of the data, and predict the remaining 99%
(and repeat this using each 1% of the data as training set).

Again, the effects of introducing the attributes in the inference process are
very similar to those we encounter in synthetic data (Fig. 5). When using only
the state of the representatives, we observe a behavior that is compatible
with small but finite correlation between attribute and voting patterns, since
the optimal predictive performance is observed at intermediate values of
$\lambda_{\rm user}$. Rather, when we consider party affiliation we observe a
behavior that is compatible with almost perfect correlation between attribute
and voting behavior. Indeed, in this case the predictive performance of the
model increases monotonically with $\lambda_{\rm user}$, with an abrupt
transition at $\lambda_{\rm user}\approx 1$, just as for perfectly correlated
attributes in synthetic data. When state and party are combined into a single
excluding attribute (for example, “Democrat from Texas” is a group), we
observe a behavior compatible with strong (but imperfect) correlation between
attributes and voting behavior. In this case, predictive accuracy does not
improve monotonically with $\lambda_{\rm user}$ because, for very large
values, representatives are forced into small groups that are more prone to
fluctuations, that is, the model overfits the data thus worsening the
predictive power with respect to considering large groups associated to party
affiliation alone.

##  VI Conclusion

There is ample evidence that using node metadata can help to solve network
inference problems. As we have discussed, several approaches have been
proposed in recent years to introduce node attributes into probabilistic
network models, and to use them to make better inferences about, for example,
the group structure of networks or the existence of unobserved interactions.
In these approaches, node attributes are introduced either as part of a whole-
system model (including both the links between nodes and node attributes), or
as priors over the parameters of the model for the links (for example, as
priors for the node group memberships that, in turn, determine the probability
of existence of links). However, beyond the improvement in performance that
they may entail in a given task such as group detection or link prediction, we
know little about the effect that node attributes have in the inference
process. Here, our goal has been to clarify this issue.

Regardless of whether attributes are introduced as part of a whole model or as
a prior for model parameters, they appear in probabilistic models as
additional terms in the likelihood or the posterior. As we have shown, our
results depend on this simple observation alone—only when all terms in these
likelihoods or posteriors are comparable in magnitude, or when attributes are
perfectly correlated with ratings, can we expect attributes to improve the
inference process. In this sense, our findings here may be expected to be
universal.

From a practical point of view, our work helps to understand when certain
approaches will not work. For example, our results suggest that modeling data
and metadata jointly will only improve link predictions (or other network
inference problems) if two conditions are fulfilled simultaneously: (i) the
metadata are correlated to the data; (ii) as we have mentioned, the balance
between amount of data and metadata is such that their likelihoods ($L^{R}$
and $L^{A}$ above) are of the same order. If the first condition is not
fulfilled, using metadata will in general worsen predictions, rather than
improving them; if the second condition is not fulfilled, one may, in
practice, inadvertently ignore either the data or the metadata and thus make,
again, suboptimal predictions.

Some works have intuitively addressed this problem by introducing tuning
parameters akin to our $\lambda_{k}$ Yang  _et al._ (2013); Contisciani  _et
al._ (2020). However, the impact of those parameters has not been studied in
detail and, instead, their values are typically chosen among a very limited
set by means of cross-validation. Our work clarifies how the value of those
parameters should be chosen, and why.

From a broader perspective, our work opens the door to understanding the role
of different terms in probabilistic network models, as well as the transitions
that occur between the regimes in which one term or another dominates. This
sets the stage for more systematic approaches to building better probabilistic
models of network systems.

###### Acknowledgements.

The authors acknowledge support by the Spanish Ministerio de Economía y
Competitividad (Grants FIS2016-78904-C3-P-1 and PID2019-106811GB-C31) and by
the Government of Catalonia (Grant 2017SGR-896).

##  Appendix A Expectation-maximization equations

We aim to maximize the parametric log-posterior in Eq. (9) as a function of
the model parameters
$\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p},\boldsymbol{\zeta},\boldsymbol{q}$
and $\boldsymbol{\hat{q}}$. Because logarithms of sums are hard to deal with,
we use a variational trick that first introduces an auxiliary distribution
$p(x)$ with $\sum_{x}p(x)=1$ into a sum of terms as
$\sum_{x}x=\sum_{x}p(x)\left(x/p(x)\right)$. Then because
$\sum_{x}p(x)\left(x/p(x)\right)=\langle x/p(x)\rangle$ we can use Jensens’
inequality $\log\langle y\rangle\geq\langle\log y\rangle$ to write
$\log\left[\sum_{x}p(x)\left(x/p(x)\right)\right]\geq\sum_{x}p(x)\log\left[x/p(x)\right]$.

Because both rating and attribute terms in Eq. (9) contain logarithms of sums,
we introduce an auxiliary distribution for each of the terms as follows. For
the ratings, we have

$\displaystyle\mathcal{L}^{R}$ $\displaystyle=$ $\displaystyle\sum_{(i,j)\in
R^{O}}\log\sum_{\alpha\beta}\theta_{i\alpha}\eta_{j\beta}p_{\alpha\beta}(r^{O}_{ij})r$
(10) $\displaystyle=$ $\displaystyle\sum_{(i,j)\in
R^{O}}\log\sum_{\alpha\beta}\omega_{ij}(\alpha,\beta)\frac{\theta_{i\alpha}\eta_{j\beta}p_{\alpha\beta}(r^{O}_{ij})}{\omega_{ij}(\alpha,\beta)}$
$\displaystyle\geq$ $\displaystyle\sum_{(i,j)\in
R^{O}}\sum_{\alpha\beta}\omega_{ij}(\alpha,\beta)\log\frac{\theta_{i\alpha}\eta_{j\beta}p_{\alpha\beta}(r^{O}_{ij})}{\omega_{ij}(\alpha,\beta)}$

where $\omega_{ij}(\alpha,\beta)$ is the auxiliary distribution.

For the term corresponding to excluding node attributes we have

$\displaystyle\mathcal{L}^{A_{k}}$ $\displaystyle=$
$\displaystyle\sum_{(i,\ell_{k})\in
A_{k}^{O}}\log\sum_{\alpha}\theta_{i\alpha}q_{\alpha}^{k}(i\ell_{k})$ (11)
$\displaystyle=$ $\displaystyle\sum_{(i,\ell_{k})\in
A_{k}^{O}}\log\sum_{\alpha}\sigma_{i\ell_{k}}^{k}(\alpha)\frac{\theta_{i\alpha}q_{\alpha}^{k}(i\ell_{k})}{\sigma_{i\ell_{k}}(\alpha)}$
$\displaystyle\geq$ $\displaystyle\sum_{(i,\ell_{k})\in
A_{k}^{O}}\sum_{\alpha}\sigma_{i\ell_{k}}^{k}(\alpha)\log\frac{\theta_{i\alpha}q_{\alpha}^{k}(i\ell_{k})}{\sigma_{i\ell_{k}}^{k}(\alpha)}$

where $\sigma^{k}_{i\ell_{k}}(\alpha)$ is the auxiliary distribution, and to
simplify the notation we have defined $q^{k}_{\alpha}(i\ell_{k})\equiv
q^{k}_{\alpha}(\left(e_{k}^{O}\right)_{i\ell_{k}})$.

Finally, for the term corresponding to non-excluding node attributes we have

$\displaystyle\mathcal{L}^{A_{k}}$ $\displaystyle=$
$\displaystyle\sum_{(i,g)\in
A_{k}^{O}}\log\sum_{\alpha\gamma}\theta_{i\alpha}\zeta_{g\gamma}^{k}\hat{q}_{\alpha\gamma}(ig)$
(12) $\displaystyle=$ $\displaystyle\sum_{(i,g)\in
A_{k}^{O}}\log\sum_{\alpha\gamma}\hat{\sigma}_{ig}^{k}(\alpha,\gamma)\frac{\theta_{i\alpha}\zeta_{g\gamma}^{k}\hat{q}_{\alpha\gamma}(ig)}{\hat{\sigma}_{ig}^{k}(\alpha,\gamma)}$
$\displaystyle\geq$ $\displaystyle\sum_{(i,g)\in
A_{k}^{O}}\sum_{\alpha\gamma}\hat{\sigma}_{ig}^{k}(\alpha,\gamma)\log\frac{\theta_{i\alpha}\zeta_{g\gamma}^{k}\hat{q}_{\alpha\gamma}(ig)}{\hat{\sigma}_{ig}^{k}(\alpha,\gamma)}$

where $\hat{\sigma}_{ig}^{k}(\alpha,\gamma)$ is the auxiliary distribution,
and to simplify the notation we have defined
$\hat{q}^{k}_{\alpha}(ig)\equiv\hat{q}_{\alpha\gamma}^{k}(\left(a_{k}^{O}\right)_{ig})$.

Note that, in Eqs. (10)-(12) above, the equality is satisfied when maximizing
with respect to the auxiliary distributions. By solving these optimization
problems we obtain

$\displaystyle\omega_{ij}(\alpha,\beta)$ $\displaystyle=$
$\displaystyle\frac{\theta_{i\alpha}\eta_{j\beta}p_{\alpha\beta}(r^{O}_{ij})}{\sum_{\alpha^{\prime}\beta^{\prime}}\theta_{i\alpha^{\prime}}\eta_{j\beta^{\prime}}p_{\alpha^{\prime}\beta^{\prime}}(r^{O}_{ij})}\;,$
(13) $\displaystyle\sigma_{i\ell_{k}}^{k}(\alpha)$ $\displaystyle=$
$\displaystyle\frac{\theta_{i\alpha}q_{\alpha}^{k}(i\ell_{k})}{\sum_{\alpha^{\prime}}\theta_{i\alpha^{\prime}}q^{k}_{\alpha^{\prime}}(i\ell_{k})}\;,$
(14) $\displaystyle\hat{\sigma}_{ig}^{k}(\alpha,\gamma)$ $\displaystyle=$
$\displaystyle\frac{\theta_{i\alpha}\zeta_{g\gamma}^{k}\hat{q}_{\alpha\gamma}(ig)}{\sum_{\alpha^{\prime}\gamma^{\prime}}\theta_{i\alpha^{\prime}}\zeta_{g\gamma^{\prime}}\hat{q}_{\alpha^{\prime}\gamma^{\prime}}(ig)}\,.$
(15)

Therefore, the auxiliary distributions have the following interpretations:
$\omega_{ij}(\alpha,\beta)$ is the contribution of user group $\alpha$ and
item group $\beta$ to the probability that user $i$ gives item $j$ a rating
$r^{O}_{ij}$; $\sigma^{k}_{i\ell_{k}}(\alpha)$ is the contribution of user
group (or item group) $\alpha$ to the probability that user (item) $i$ has
attribute type $(e_{k}^{O})_{i\ell_{k}}$ in the $k$-th excluding attribute;
and, finally, $\hat{\sigma}^{k}_{ig}(\alpha,\gamma)$ is the contribution of
groups $\alpha$ and $\gamma$ to the probability that, for the $k$-th non-
excluding attribute, the association between node $i$ and attribute $g$ is of
type $(a_{k}^{O})_{ig}$.

Using Lagrange multipliers for the normalization constraints, and equating to
zero the derivatives of the log-posterior with respect to the model parameters
yields

$\theta_{i\alpha}=\frac{\sum_{j\in\partial
i}\sum_{\beta}\omega_{ij}(\alpha,\beta)+\sum_{k}\lambda_{k}\sigma^{k}_{i\ell_{k}}(\alpha)+\sum_{l}\lambda_{l}\sum_{g\in{\partial_{i}}^{k}}\sum_{\gamma}\hat{\sigma}_{ig}^{l}(\alpha,\gamma)}{d_{i}+\sum_{k}\lambda_{k}\delta_{i}^{k}+\sum_{l}\lambda_{l}\Delta_{i}^{l}}$
(16)

where $\partial_{i}^{k}$ is the set of $k$-th attributes associated with user
$i$, $d_{i}$ is the degree of user $i$ in the network of ratings, and
$\Delta_{i}^{l}=|{\partial_{i}}^{l}|$. Note that the term
$\sigma^{k}_{i\ell_{k}}(\alpha)$ is equal to zero if user $i$ does not have
attribute $\ell_{k}$, so that $\delta_{i}^{k}=1$ if user $i$ has exclusive
attribute $\ell_{k}$ and zero otherwise.

$\eta_{j\beta}=\frac{\sum_{i\in\partial
j}\sum_{\alpha}\omega_{ij}(\alpha,\beta)+\sum_{k}\lambda_{k}\sigma^{k}_{j\ell_{k}}(\beta)+\sum_{l}\lambda_{l}\sum_{i\in\partial_{j}^{k}}\sum_{\gamma}\hat{\sigma}_{ij}^{l}(\beta,\gamma)}{d_{j}+\sum_{k}\lambda_{k}\delta_{j}^{k}+\sum_{l}\lambda_{l}\Delta_{j}^{l}}$
(17)

where $\partial_{j}^{k}$ is the set of $k$-th attributes associated with item
$j$, $d_{j}$ is the degree of item $j$ in the network of ratings, and
$\Delta_{j}^{l}=|{\partial_{j}}^{l}|$. As before, the term
$\sigma^{k}_{j\ell_{k}}(\beta)$ is equal to zero if item $j$ does not have
attribute $\ell_{k}$, so that $\delta_{j}^{k}=1$ ifitem $j$ has exclusive
attribute $\ell_{k}$ and zero otherwise.

$\zeta_{g\gamma}^{k}=\frac{\sum_{i\in\partial_{g}^{k}}\sum_{\alpha}\hat{\sigma}_{ig}^{k}(\alpha,\gamma)}{\Delta_{g}^{k}}$
(18)

where where $\partial_{g}^{k}$ is the set of nodes associated with attribute
$g$, and $\Delta_{g}^{k}=|{\partial_{g}}^{k}|$. Additionally, we have

$p_{\alpha\beta}(r)=\frac{\sum_{(i,j)\in
R^{O}|r_{ij}^{0}}=r\omega_{ij}(\alpha,\beta)}{\sum_{(i,j)\in
R^{O}}\omega_{ij}(\alpha,\beta)}$ (19)

$q_{\alpha}^{k}(e)=\frac{\sum_{(i,\ell_{k})\in
A_{k}^{O}|(e^{O}_{k})_{i\ell_{k}}=e}\sigma_{i\ell_{k}}^{k}(\alpha)}{\sum_{(i,\ell_{k})}\sigma_{i\ell_{k}}^{k}(\alpha)}$
(20) $\hat{q}_{\alpha\gamma}^{k}(a)=\frac{\sum_{(i,g)\in
A_{k}^{O}|(a^{O}_{k})_{ig}}=a\hat{\sigma}_{ig}^{k}(\alpha,\gamma)}{\sum_{(i,g)\in
A_{k}^{O}}\hat{\sigma}_{ig}^{k}(\alpha,\gamma)}$ (21)

##  Appendix B Expectation-maximization algorithm

To obtain a maximum of the posterior we start by berating random initial
conditions for each model parameter
$\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p},\boldsymbol{\zeta},\boldsymbol{q},\boldsymbol{\hat{q}}$.

The we perform iteratively two steps until model parameters convergence:

  1. 1.

Expectation step: compute the auxiliary functions $\omega_{ij}(\alpha,\beta)$,
$\sigma^{k}_{i\ell_{k}}(\alpha)$, and $\hat{\sigma}^{k}_{ig}(\alpha,\gamma)$
using current values for
$\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p},\boldsymbol{\zeta},\boldsymbol{q},\boldsymbol{\hat{q}}$
using Eqs. A.4, A.5 and A.6.

  2. 2.

Maximization step: Compute the new values for the model parameters using the
values for the auxiliary functions and Eqs. A.7 - A.12.

Because the posterior landscape is very rugged, to make predictions we perform
the EM algorithm 10 times and consider all of the models to estimate the
average probability that user $i$ rates item $j$ with rating $r$ (see Godoy-
Lorite  _et al._ (2016b)) as follows:

$\langle
p(r_{ij}=r|R^{O},A^{O}_{k})\rangle\approx\frac{1}{N}\sum_{n=1}^{N}p_{n}(r_{ij}=r|R^{O},A^{O}_{k},(\dots))$
(22)

where
$(\dots)=\\{\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{p},\boldsymbol{\zeta},\boldsymbol{q},\boldsymbol{\hat{q}}\\}$,
and $p_{n}(r_{ij}=r|R^{O},A^{O}_{k},(\dots))$ is the probability that user $i$
rates item $j$ with rating $r$ in run $n$ of the EM algorithm.

## References

  * Liben-Nowell and Kleinberg (2007) D. Liben-Nowell and J. Kleinberg, “The link-prediction problem for social networks,” J. Am. Soc. Inf. Sci. Tec. 58, 1019–1031 (2007).
  * Clauset  _et al._ (2008) A. Clauset, C. Moore, and M. E. J. Newman, “Hierarchical structure and the prediction of missing links in networks.” Nature 453, 98–101 (2008).
  * Guimerà and Sales-Pardo (2009) R. Guimerà and M. Sales-Pardo, “Missing and spurious interactions and the reconstruction of complex networks.” Proc. Natl. Acad. Sci. U. S. A. 106, 22073–22078 (2009).
  * Lü  _et al._ (2015) L. Lü, L. Pan, T. Zhou, Y.-C. Zhang, and H.E. Stanley, “Toward link predictability of complex networks,” Proc. Natl. Acad. Sci. U.S.A. 112, 2325–2330 (2015).
  * Ghasemian  _et al._ (2020) A. Ghasemian, H. Hosseinmardi, A. Galstyan, E. M. Airoldi, and A. Clauset, “Stacking models for nearly optimal link prediction in complex networks,” Proc. Natl. Acad. Sci. USA 117, 23393–23400 (2020).
  * Guimerà (2020) R Guimerà, “One model to rule them all in network science?” Proc. Natl. Acad. Sci. USA 117, 25195–25197 (2020).
  * Guimerà and Sales-Pardo (2013) R. Guimerà and M. Sales-Pardo, “A network inference method for large-scale unsupervised identification of novel drug-drug interactions,” PLoS Comput. Biol. 9, e1003374 (2013).
  * Tarrés-Deulofeu  _et al._ (2019) M. Tarrés-Deulofeu, A. Godoy-Lorite, R. Guimerà, and M. Sales-Pardo, “Tensorial and bipartite block models for link prediction in layered networks and temporal networks,” Phys. Rev. E 99, 032307 (2019).
  * Menden  _et al._ (2019) Michael P. Menden, Dennis Wang, Mike J. Mason, Bence Szalai, Krishna C. Bulusu, Yuanfang Guan, Thomas Yu, Jaewoo Kang, Minji Jeon, Russ Wolfinger, Tin Nguyen, Mikhail Zaslavskiy, AstraZeneca-Sanger Drug Combination DREAM Consortium, In Sock Jang, Zara Ghazoui, Mehmet Eren Ahsen, Robert Vogel, Elias Chaibub Neto, Thea Norman, Eric K. Y. Tang, Mathew J. Garnett, Giovanni Y. Di Veroli, Stephen Fawell, Gustavo Stolovitzky, Justin Guinney, Jonathan R. Dry, and Julio Saez-Rodriguez, “Community assessment to advance computational prediction of cancer drug combinations in a pharmacogenomic screen,” Nat. Comm. 10, 2674 (2019).
  * Guimerà and Sales-Pardo (2011) R. Guimerà and M. Sales-Pardo, “Justice blocks and predictability of U.S. Supreme Court votes,” PLoS ONE 6, e27188 (2011).
  * Guimerà  _et al._ (2012) R. Guimerà, A. Llorente, E. Moro, and M. Sales-Pardo, “Predicting human preferences using the block structure of complex social networks,” PLoS ONE 7, e44620 (2012).
  * Godoy-Lorite  _et al._ (2016a) A. Godoy-Lorite, R. Guimerà, C. Moore, and M. Sales-Pardo, “Accurate and scalable social recommendation using mixed-membership stochastic block models,” Proc. Natl. Acad. Sci. U.S.A. 113, 14207 –– 14212 (2016a).
  * Cobo-López  _et al._ (2018) S. Cobo-López, A. Godoy-Lorite, J. Duch, M. Sales-Pardo, and R. Guimerà, “Optimal prediction of decisions and model selection in social dilemmas using block models,” EPJ Data Sci. 7 , 48 (2018) 7, 48 (2018).
  * Timme (2007) M. Timme, “Revealing Network Connectivity from Response Dynamics,” Phys. Rev. Lett. 98, 224101 (2007).
  * Peixoto (2019) T.P. Peixoto, “Network reconstruction and community detection from dynamics,” Phys. Rev. Lett. 123, 128301 (2019).
  * Tallberg (2004) C. Tallberg, “A Bayesian approach to modeling stochastic blockstructures with covariates,” J. Math. Sociol. 29, 1–23 (2004).
  * Yang  _et al._ (2013) J. Yang, J. McAuley, and J. Leskovec, “Community detection in networks with node attributes,” in  _2013 IEEE 13th International Conference on Data Mining_ (2013) pp. 1151–1156.
  * Hric  _et al._ (2016) D. Hric, T. P. Peixoto, and S. Fortunato, “Network structure, metadata, and the prediction of missing nodes and annotations,” Phys. Rev. X 6, 031038 (2016).
  * Newman and Clauset (2016) M. E. J. Newman and A. Clauset, “Structure and inference in annotated networks,” Nat. Comm. 7, 11863 (2016).
  * White and Murphy (2016) A. White and T. B. Murphy, “Mixed-membership of experts stochastic blockmodel,” Netw. Sci. 4, 48–80 (2016).
  * Peel  _et al._ (2017) L. Peel, D. B. Larremore, and A. Clauset, “The ground truth about metadata and community detection in networks,” Sci. Adv. 3 (2017), 10.1126/sciadv.1602548.
  * Stanley  _et al._ (2019) N. Stanley, T. Bonacci, and R. Kwitt, “Stochastic block models with multiple continuous attributes,” Appl. Netw. Sci. 4, 54 (2019).
  * Contisciani  _et al._ (2020) M. Contisciani, E. A. Power, and C. De Bacco, “Community detection with node attributes in multilayer networks,” Sci. Rep. 10, 1–16 (2020).
  * Koren  _et al._ (2009) Y. Koren, R. Bell, and C. Volinsky, “Matrix factorization techniques for recommender systems,” Computer 42, 30–37 (2009).
  * Holland  _et al._ (1983) P. W. Holland, K. B. Laskey, and S. Leinhardt, “Stochastic blockmodels: First steps,” Soc. Networks 5, 109–137 (1983).
  * Nowicki and Snijders (2001) K. Nowicki and T. A. B. Snijders, “Estimation and prediction for stochastic blockstructures,” J. Am. Stat. Assoc. 96, 1077–1087 (2001).
  * Yen and Larremore (2020) T.-C. Yen and D. B. Larremore, “Community detection in bipartite networks with stochastic block models,” Phys. Rev. E 102, 032309 (2020).
  * Airoldi  _et al._ (2008) E. M. Airoldi, D. M. Blei, S. E Fienberg, and E. P. Xing, “Mixed membership stochastic blockmodels,” J. Mach. Learn. Res. 9, 1981–2014 (2008).
  * Harper and Konstan (2015) F. M. Harper and J. A. Konstan, “The Movielens datasets: History and context,” ACM Trans. Interact. Intell. Syst. 5 (2015).
  * Waugh  _et al._ (2009) A. S. Waugh, L. Pei, J. Fowler, P. Mucha, and M. A. Porter, “Party polarization in Congress: A network science approach,” arXiv: Physics and Society (2009).
  * Godoy-Lorite  _et al._ (2016b) A. Godoy-Lorite, R. Guimerà, C. Moore, and M. Sales-Pardo, “Accurate and scalable social recommendation using mixed-membership stochastic block models,” Proc. Natl. Acad. Sci. U.S.A. 113, 14207 –– 14212 (2016b).