# On the Fusion Strategies for Federated Decision Making

###### Abstract

We consider the problem of information aggregation in federated decision
making, where a group of agents collaborate to infer the underlying state of
nature without sharing their private data with the central processor or each
other. We analyze the non-Bayesian social learning strategy in which agents
incorporate their individual observations into their opinions (i.e., soft-
decisions) with Bayes rule, and the central processor aggregates these
opinions by arithmetic or geometric averaging. Building on our previous work,
we establish that both pooling strategies result in asymptotic normality
characterization of the system, which, for instance, can be utilized in order
to give approximate expressions for the error probability. We verify the
theoretical findings with simulations and compare both strategies.

##  1 Introduction and Related Work

The centralized strategy has been the traditional norm for collaborative
decision-making, where data is collected from all sources and processed at a
central controller, fusion center or cloud. However, this approach raises
concerns about data privacy and communication costs. In this work, we study
the paradigm of federated decision-making, which enables cooperative decision-
making without data collection [1]. For example, in healthcare applications,
federated decision-making can be used to test possible hypotheses without
gathering private data from siloed institutions [2]. Moreover, in
classification tasks, agents (e.g., mobile device users) may have different
views of a physical phenomenon such as scenery as in multi-view learning [3],
and the server may want to classify the scene or infer the state of
environment without collecting the local data of users.

The setting we consider in this work, which is described in detail in Sec. 2,
is an instance of locally Bayesian (a.k.a. non-Bayesian) social learning [4,
5, 6, 7] that is based on general decentralized networks. Here, we only
consider the special case of a star network topology where all agents are
connected to a central node. In social learning, the goal is to infer the
hypothesis that best describes the observations received from the environment.
The true hypothesis is common for all agents and hence the cooperation is of
interest for all agents. To this end, at each time instant, $(i)$ distributed
agents process their private data locally and send their opinions (i.e., soft-
decisions) to the server, $(ii)$ the server pools the information received
from agents and broadcasts the aggregated belief back to the agents, $(iii)$
the agents use the broadcasted belief as a prior belief for their subsequent
opinions.

Note that this setting has close ties with the distributed detection
literature [8, 9, 10, 11], which typically involves sharing log-likelihood
ratios for each data and assumes spatial independence and/or homogeneity
across agents. In contrast, in the social learning framework, agents share
beliefs (posteriors) formed over time, rather than a statistic about
individual data. By this way, they can incorporate their individual priors as
well. Furthermore, in social learning, data is not required to be independent
between agents nor identically distributed. In fact, agents can possess
different data types (e.g., text, video), as in multi-modal machine learning —
see Fig. 2. The federated decision-making is also related to federated
learning [12, 13] as they both fall under the umbrella of federated analytics
[1], which aims to enable cooperation among multiple users while preserving
privacy of data. However, federated learning focuses on training machine
learning models with labeled data samples, whereas the current setting
involves making collaborative inferences without sharing raw observations.

Fig. 1: Data types at the edge devices can be highly heterogeneous.222Image
credit: freepik.com.

In this paper, we address the following problem: How should the server
aggregate information received from multiple distributed agents that process
the data locally and share only their local soft decision statistics? Two of
the most widely used methods for combining probability vectors are
_arithmetic_ averaging (AA), a.k.a. linear opinion pooling, and _geometric_
averaging (GA), a.k.a. logarithmic opinion pooling. The distinct properties of
these procedures are well-established in the literature — see, e.g., [14]. In
our previous work [15], we investigated the performances of AA and GA
strategies in the general social learning setting. For the special case of
federated architectures, we derived an exact expression for the asymptotic
convergence rates of beliefs. Different from prior works that only compared AA
and GA with limited assumptions, such as assuming only one step of fusion [16,
17, 18] or assuming specific/particular distributions like Gaussian or Poisson
[16, 18, 19]; our analysis considered repeated fusion of beliefs without any
assumption on the distribution of data. In this work, we extend our previous
findings and show that the asymptotic behaviors of both AA and GA can be
characterized with normal distributions and the asymptotic error probabilities
of both strategies can be established accordingly. The obtained results
provide further insight into the arithmetic and geometric fusion rules, and
can be utilized for designing federated decision-making systems.

Notation. Random variables are written in boldface letters. We use
“proportional to” symbol $\propto$ whenever LHS of an equation is a proper
normalization of RHS, and is an entry of a probability mass function. The KL-
divergence between two distributions $p$ and $q$ is denoted as
$D_{\textup{KL}}(p||q)$.

##  2 Federated Decision Making

We consider a distributed hypothesis testing task in which a group
$\mathcal{K}$ of $K$ agents, aided by a central controller or server, perform
inferences on a phenomenon of interest in their environment. More formally,
the agents seek to choose the true hypothesis $\theta^{\circ}$ from a finite
set of $H$ hypotheses $\Theta=\\{1,2,\dots,H\\}$. At each time instant $i$,
agent $k$ receives personal observation $\bm{\xi}_{k,i}$ from the environment
that is distributed according to the marginal likelihood function
$L_{k}(\cdot|\theta^{\circ})$ (since the true hypothesis is $\theta^{\circ}$).
Notice that the likelihood functions depend on agent index $k$, i.e., the data
distributions across different agents can be heterogeneous. We assume that the
environment is stationary, and that the data is independent and identically
distributed (i.i.d.) over time at each agent. However, we do not make any
assumptions on spatial dependence, i.e., there is no assumption on the
dependence among different agents. Each agent $k$ only knows its own local
likelihood model $L_{k}(\xi|\theta)$ for each $\theta\in\Theta$ that
represents how likely that an observation $\xi$ is produced by $\theta$.
Furthermore, the server does not need to possess any information about the
distribution of data on agents.

The agents do not share their private raw data with the server: all data
processing and ownership are local. Each agent $k$ only shares its belief
(i.e., opinion) $\bm{\psi}_{k,i}$ about the possible hypotheses, which is a
probability mass function over $\Theta$. In other words,
$\bm{\psi}_{k,i}(\theta)$ represents the amount of confidence agent $k$ has on
the proposition “$\theta=\theta^{\circ}$”, at time instant $i$. Once the
server receives the intermediate beliefs from distributed agents, it pools
them and broadcasts the averaged belief back to the agents. In this work, we
study the widely used linear (AA) and logarithmic (GA) pooling strategies,
although there are other variations [14]. We now motivate the setting with two
real-world applications.

######  Example 1 (Crowd Size Estimation).

Crowd counting aims to estimate the number of people in a given area using
visual or other sensory data. The possible hypotheses are the size of the
crowd, such as “the number of people in the event is between 500-1000”.
Agents, i.e., users cooperating with the server, can obtain data in the form
of photos and videos of their surroundings, or their mobile devices can
measure the number of possible connections over Bluetooth. Due to privacy
concerns, users may prefer not to share their raw data with the server. In
addition, transmitting raw data can create communication bottlenecks at the
server. Therefore, employing the social learning strategy, users can only
share their beliefs over the set of hypotheses. They can form their beliefs by
using local likelihood models, e.g., by using pre-trained neural network
models on their devices that can estimate the crowd size based on images [20].
∎

######  Example 2 (Flexible Spectrum Use).

Spatially distributed sensors or mobile devices can try to estimate the number
of targets or active wireless emitters within a particular area. For example,
in the radio spectrum, the mobile devices can measure the local occupancy and
field levels to distinguish between idle and congested spectrum, or map the
radio environment. This practice can enhance the environmental awareness, and
allocate the resources optimally in order to improve the wireless
communication. Exchanging beliefs only rather than raw data can be
advantageous in terms of privacy and communication costs. ∎

To form the beliefs, agents execute the non-Bayesian social learning strategy
[4, 5, 6, 7] under the special case of star topologies. At each time instant
$i$, agents first process their observations in a _locally_ Bayesian manner:

$\displaystyle\bm{\psi}_{k,i}(\theta)$ $\displaystyle\propto
L_{k}(\bm{\xi}_{k,i}|\theta)\bm{\mu}_{i-1}(\theta)\qquad\text{(Adapt)},$ (1)

and then share these intermediate beliefs with the server. If the server
employs AA for information fusion, the updated belief is a weighted arithmetic
average of the intermediate beliefs from different agents:

$\displaystyle\bm{\mu}_{i}(\theta)$
$\displaystyle=\sum_{k\in\mathcal{K}}{\pi_{k}}\bm{\psi}_{k,i}(\theta)\qquad\text{(AA)}.$
(2)

Here, $\pi=[\pi_{1},\dots,\pi_{K}]^{\operatorname{\mathsf{T}}}$ is a vector of
the confidence weights $\pi_{k}$ the server assigns to each agent $k$ [9, 21].
We assume that these weights are positive, sum up to 1, and constant over
time. Alternatively, if the fusion rule is GA, the updated belief is a
weighted geometric average:

$\displaystyle\bm{\mu}_{i}(\theta)$
$\displaystyle\propto\prod_{k\in\mathcal{K}}(\bm{\psi}_{k,i}(\theta))^{\pi_{k}}\quad\text{(GA)}.$
(3)

The server then sends the updated belief to the agents, and they execute the
same steps repeatedly over time by using the updated beliefs and locally
observed data.

######  Remark 1.

An important distinction between AA and GA lie in their handling of initial
beliefs and their sensitivity to individual beliefs. In AA, only one agent
having a positive initial belief on the true hypothesis is sufficient. In
contrast, in GA, all agents must have positive initial beliefs on the truth in
order not to discard it from the beginning. Furthermore, GA is more sensitive
to small beliefs compared to AA. This is because, if some agents’ beliefs are
very small, they impact the product more significantly compared to the sum. In
other words, GA grants the agents a veto power: if one agent transmits a
belief entry as 0, the combined belief also becomes 0. In general, giving this
level of autonomy to individual agents depends on the application at hand. ∎

##  3 Asymptotic Normality of AA and GA

In this section, we analyze the asymptotic behavior under both fusion
strategies. In order to avoid pathological cases, we assume that each
observation has finite information about the true hypothesis. That is to say,
for each agent $k$ and for each hypothesis $\theta\in\Theta$, we assume
$D_{\textup{KL}}(L_{k}(\cdot|\theta^{\circ})||L_{k}(\cdot|\theta))<\infty$.
This implies that likelihood functions have the same support. Moreover, in
order to uniquely distinguish the true hypothesis, we need the following
condition.

######  Assumption 1 (Global Identifiability).

For each wrong hypothesis $\theta\neq\theta^{\circ}$, there exists at least
one clear-sighted agent $k$ with
$D_{\textup{KL}}(L_{k}(\cdot|\theta^{\circ})||L_{k}(\cdot|\theta))>0$.
$\blacksquare$

Note that this assumption does not require _local_ identifiability, which is
an agent’s ability of inferring $\theta^{\circ}$ without any cooperation. As a
result, all agents can benefit from cooperation. Under Assumption 1, it is
already known that AA [4, 5, 22] and GA [6, 7] result in consistent truth
learning, that is, the beliefs on the true hypothesis converge to one almost
surely. In this work, we further show that under both AA and GA fusion rules,
the asymptotic error probabilities can be calculated with Gaussian cumulative
distribution functions (c.d.f.s). We start with the AA strategy.

######  Proposition 1 (Asymptotic normality of AA).

For each wrong hypothesis $\theta\neq\theta^{\circ}$, we define the mean and
variance

$\rho_{\textup{A}}\triangleq-\mathbb{E}\Big{[}\log\big{(}\sum_{k\in\mathcal{K}}\pi_{k}\bm{r}_{k,i}\big{)}\Big{]},\
\sigma_{\textup{A}}^{2}\triangleq\textup{Var}\Big{(}\log\big{(}\sum_{k\in\mathcal{K}}\pi_{k}\bm{r}_{k,i}\big{)}\Big{)},$
(4)

in terms of the likelihood ratio

$\bm{r}_{k,i}\triangleq\frac{L_{k}(\bm{\xi}_{k,i}|\theta)}{L_{k}(\bm{\xi}_{k,i}|\theta^{\circ})}.$
(5)

If $\sigma_{\textup{A}}^{2}<\infty$, it holds under Assumption 1 that

$\lim_{i\to\infty}\mathbb{P}\bigg{(}\frac{\log\bm{\mu}_{i}(\theta)+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\leq
t\bigg{)}=\Phi(t),$ (6)

where $\Phi(t)$ is the standard Gaussian cumulative distribution function.

###### Proof Sketch.

For the complete proof, see Appendix A. Here, we only give a sketch of the
proof. Consider an arbitrary $\theta\neq\theta^{\circ}$. For notational
simplicity, define

$\bm{x}_{i}\triangleq\log{\bm{\mu}_{i}(\theta)},\quad\bm{L}_{i}\triangleq\log\big{(}\sum_{k\in\mathcal{K}}\pi_{k}\bm{r}_{k,i}\big{)}.$
(7)

Since $\bm{L}_{i}$’s are assumed to be i.i.d. and of finite variance, the
central limit theorem [23] yields:

$\lim_{i\to\infty}\mathbb{P}\bigg{(}\dfrac{\sum_{j=1}^{i}\bm{L}_{j}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\leq
t\bigg{)}=\Phi(t).$ (8)

Therefore, if we show that the distance between $\frac{1}{\sqrt{i}}\bm{x}_{i}$
and $\frac{1}{\sqrt{i}}\sum_{j=1}^{i}\bm{L}_{j}$ goes to zero with high
probability, we are done. To that end, we utilize the result from [15] which
states

$\frac{1}{i}\bm{x}_{i}\to-\rho_{\textup{A}}\quad\text{as}\>\>i\to\infty,\>\>\text{almost
surely}.$ (9)

In particular, we use the following bootstrapping argument on (9): There
exists an $i_{0}$ after which (i.e., $\forall i\geq i_{0}$)

$\bm{\mu}_{i}(\theta)\leq\exp\\{{-i(\rho_{\textup{A}}-\epsilon)}\\},\
\forall\theta\neq\theta^{\circ}$ (10)

with high probability. This result enables a finer study of the evolution of
$\bm{x}_{i}$ as $i\to\infty$. ∎

Note that Proposition 1 can equivalently be given in terms of the log-belief
ratios:

######  Corollary 1.

If we define the log-belief ratio
$\bm{\lambda}_{i}(\theta)\triangleq\log\dfrac{\bm{\mu}_{i}(\theta^{\circ})}{\bm{\mu}_{i}(\theta)}$,
for each $\theta\neq\theta^{\circ}$, Proposition 1 implies that

$\lim_{i\to\infty}\mathbb{P}\bigg{(}\frac{\bm{\lambda}_{i}(\theta)-\rho_{A}i}{\sigma_{A}\sqrt{i}}\leq
t\bigg{)}=\Phi(t).$ (11)

###### Proof Sketch.

For the complete proof, see Appendix B. Essentially, the proof follows from
consistent truth learning, i.e., $\bm{\mu}_{i}(\theta^{\circ})~{}\to~{}1$
almost surely as $i\to\infty$. This implies that, as $i\to\infty$,
$\log{\bm{\mu}_{i}(\theta)}$ and
$\log{\bm{\mu}_{i}(\theta)}-\log{\bm{\mu}_{i}(\theta^{\circ})}$ behaves
similarly. ∎

Next, we investigate the behavior of beliefs under the GA fusion rule.

######  Proposition 2 (Asymptotic normality of GA).

For each wrong hypothesis $\theta\neq\theta^{\circ}$, we define the mean

$\\!\\!\rho_{\textup{G}}\triangleq-\mathbb{E}\Big{[}\sum_{k\in\mathcal{K}}\pi_{k}\log\bm{r}_{k,i}\Big{]}\\!\\!=\\!\\!\sum_{k\in\mathcal{K}}\\!\pi_{k}D_{\textup{KL}}(L_{k}(.|\theta^{\circ})||L_{k}(.|\theta))$
(12)

and the variance

$\sigma_{\textup{G}}^{2}\triangleq\textup{Var}\Big{(}\sum_{k\in\mathcal{K}}\pi_{k}\log\bm{r}_{k,i}\Big{)}.\vspace{-0.5em}$
(13)

If $\sigma_{\textup{G}}^{2}<\infty$, then under Assumption 1 it holds that

$\lim_{i\to\infty}\mathbb{P}\bigg{(}\frac{\bm{\lambda}_{i}(\theta)-\rho_{\textup{G}}i}{\sigma_{\textup{G}}\sqrt{i}}\leq
t\bigg{)}=\Phi(t).$ (14)

###### Proof.

The proofs from [6, 7] for the convergence rate analysis can be easily
extended to establish asymptotic normality. From (1) and (3), the log-belief
ratio $\bm{\lambda}_{i}(\theta)$ evolves according to the recursion:

$\bm{\lambda}_{i}(\theta)=\bm{\lambda}_{i-1}(\theta)-\sum_{k\in\mathcal{K}}\pi_{k}\log\bm{r}_{k,i}.$
(15)

Expanding the recursion over time, we obtain

$\bm{\lambda}_{i}(\theta)=-\sum_{j=1}^{i}\bigg{(}\sum_{k\in\mathcal{K}}\pi_{k}\log\bm{r}_{k,j}\bigg{)}+\bm{\lambda}_{0}(\theta).$
(16)

Since $(i)$ $\bm{\lambda}_{0}(\theta)<\infty$ due to initial beliefs being
positive, $(ii)$ $\bm{r}_{k,i}$’s are i.i.d., and $(iii)$
$\sigma_{\textup{G}}^{2}<\infty$, an application of the central limit theorem
[23] concludes the proof. ∎

(a) a

(b) b

Fig. 2: (a) For the first experiment with Gaussian data, the histograms of
$\widetilde{\bm{\lambda}}^{(A)}_{i}(\theta)$ and
$\widetilde{\bm{\lambda}}^{(G)}_{i}(\theta)$ are drawn as shaded bar plots
with colors blue and red respectively. The standard normal density
$\mathcal{G}(t)$ is shown as solid black line. (b) For the second experiment
with exponentially distributed data, the histograms of
$\widetilde{\bm{\lambda}}^{(A)}_{i}(\theta)$ and
$\widetilde{\bm{\lambda}}^{(G)}_{i}(\theta)$ are drawn as shaded bar plots
with colors blue and red respectively. The standard normal density
$\mathcal{G}(t)$ is shown as solid black line.

(a) a

(b) b

Fig. 3: (a) For the uncorrelated Gaussian case, histograms of
${\bm{\lambda}}^{(A)}_{i}(\theta)$ and ${\bm{\lambda}}^{(G)}_{i}(\theta)$ are
drawn as shaded bar plots with colors blue and red respectively. (b) For the
correlated Gaussian case, histograms of ${\bm{\lambda}}^{(A)}_{i}(\theta)$ and
${\bm{\lambda}}^{(G)}_{i}(\theta)$ are drawn as shaded bar plots with colors
blue and red respectively.

In Propositions 1 and 2, we have established the asymptotic normality for each
wrong hypothesis, which allows approximating the error probabilities when
agents or the central processor employs a maximum a-posteriori (MAP) rule to
estimate $\theta^{\circ}$. It is important to note that under spatial
independence, GA matches the optimal Bayes posterior, which attains minimum
error probability, when the confidence weight $\pi_{k}$ on each agent $k$ is
uniform. Additionally, the convergence rates $\rho_{\textup{G}}$ and
$\rho_{\textup{A}}$ were already established in previous works [6, 7] and [15]
respectively. Notice that $\rho_{\textup{G}}\geq\rho_{\textup{A}}$ — applying
Jensen’s inequality to the equation on the left in (4), we obtain (12).
However, this does not directly imply that GA has a better performance in
probability of error criterion since it depends on the variances as well. Our
simulations indicate, however, that GA has lower error probability in general.
We leave it to a future work to determine whether GA is superior to AA in
terms of asymptotic error probability under any data distribution or
confidence weight configuration — such as in the convergence rate case.

##  4 Numerical Results

We consider a binary hypothesis testing problem with $K=10$ distributed agents
connected to a fusion center. In the first experiment, agents receive i.i.d.
Gaussian observations with mean 0 under the true hypothesis $\theta^{\circ}$,
and with mean 1 under $\theta\neq\theta^{\circ}$. For both hypotheses, the
correlation matrix is the identity matrix $I_{K}$, i.e., data is independent
across agents. A possible application of this experiment is in flexible
spectrum use, for example in cognitive radios. The possible hypotheses are
whether the spectrum is idle or not. If there is noise only, the signal
distributions would be central (i.e., zero mean). Otherwise, if there is a
signal present, it would be non-zero mean. We choose the combination vector as

$\pi=[0.13,0.2,0.09,0.15,0.08,0.05,0.1,0.05,0.1,0.05].\vspace{-0.1em}$ (17)

If we take the time horizon $i=5000$ and repeat the experiment for 500
realizations, we obtain the histograms given in Figure 2a. Specifically, we
obtain the normalized log-belief ratios

$\widetilde{\bm{\lambda}}^{(A)}_{i}(\theta)\triangleq\frac{\bm{\lambda}^{(A)}_{i}(\theta)-\rho_{\textup{A}}i}{\sqrt{i}\sigma_{\textup{A}}},\quad\widetilde{\bm{\lambda}}^{(G)}_{i}(\theta)\triangleq\frac{\bm{\lambda}^{(G)}_{i}(\theta)-\rho_{\textup{G}}i}{\sqrt{i}\sigma_{\textup{G}}}$
(18)

and plot their histograms in density forms (denoted as $f^{(A)}(t)$ and
$f^{(G)}(t)$ respectively) to compare them with the standard normal density
$\mathcal{G}(t)\triangleq\frac{1}{\sqrt{2\pi}}\exp(-t^{2}/2)$. In addition to
the visual matching supporting our claims, we performed a Shapiro-Wilk test
for testing the normality of $\widetilde{\bm{\lambda}}^{(A)}_{i}(\theta)$ and
$\widetilde{\bm{\lambda}}^{(G)}_{i}(\theta)$. The resulting $p$-values turn
out to be 0.66 and 0.54, which strongly supports the null hypothesis of normal
distribution.

For the second experiment, we assume that agents receive i.i.d. exponential
observations with mean 1 under $\theta^{\circ}$ and with mean $0.5$ under
$\theta\neq\theta^{\circ}$. The other problem parameters are the same as the
first experiment. The resulting histograms are given in Figure 2b. The
Shapiro-Wilk test yields $p$-values 0.93 and 0.53 respectively, again,
strongly supporting the null hypothesis of normal distribution.

We also conducted a third experiment to verify that correlations between the
agents affect $\rho_{\textup{A}}$ but do not affect $\rho_{\textup{G}}$ (see
(12)). To compare correlated versus uncorrelated data across agents, we
simulate the setting from the first experiment, except that this time we set
the correlation matrix as
$\Sigma=0.95\mathbbm{1}_{K}\mathbbm{1}^{\operatorname{\mathsf{T}}}_{K}+0.5I_{K}$.
Note that this choice makes the data highly correlated. Then, we plot the
histograms of $\bm{\lambda}_{i}^{(A)}(\theta)$ and
$\bm{\lambda}_{i}^{(G)}(\theta)$ (without normalization) for both uncorrelated
and correlated cases in Figure 3a and 3b. We can observe that
$\rho_{\textup{G}}=0.5$ is unchanged by the correlations — the observation
that $f^{(G)}$’s are concentrated around $i\times\rho_{\textup{G}}=2500$
supports this fact. In contrast, $\rho_{\textup{A}}$ increases drastically
under the presence of positive correlations. Nevertheless, in both cases
considered, GA outperforms AA in terms of the error probability.

##  5 Conclusion

In this work, we considered two pooling algorithms for federated decision
making, namely AA and GA. The choice between AA and GA may depend on the
specific task at hand. We study their performances in terms of error
probability, and establish asymptotic normality which is a valuable tool that
can help avoid the need to simulate the algorithm for large time horizons.
Note that the parameters in the normal approximations can be obtained either
analytically or through Monte Carlo simulations, which is more efficient
compared to the exact simulation of the entire system. We leave the study for
asymptotic normality of general decentralized networks to a future work.

## References

  * [1] A. R. Elkordy, Y. H. Ezzeldin, S. Han, S. Sharma, C. He, S. Mehrotra, and S. Avestimehr, “Federated analytics: A survey,” _APSIPA Transactions on Signal and Information Processing_ , vol. 12, no. 1, pp. 1–33, 2023.
  * [2] H. Hallock, S. E. Marshall, P. A. C. ’t Hoen, J. F. Nygård, B. Hoorne, C. Fox, and S. Alagaratnam, “Federated networks for distributed analysis of health data,” _Frontiers in Public Health_ , vol. 9, 2021.
  * [3] V. Bordignon, S. Vlaski, V. Matta, and A. H. Sayed, “Learning from heterogeneous data based on social interactions over graphs,” _IEEE Transactions on Information Theory_ , pp. 1–25, Dec. 2022.
  * [4] A. Jadbabaie, P. Molavi, A. Sandroni, and A. Tahbaz-Salehi, “Non-Bayesian social learning,” _Games and Economic Behavior_ , vol. 76, no. 1, pp. 210–225, 2012.
  * [5] X. Zhao and A. H. Sayed, “Learning over social networks via diffusion adaptation,” in _Proc. Asilomar Conference on Signals, Systems and Computers_ , Pacific Grove, 2012, pp. 709–713.
  * [6] A. Nedić, A. Olshevsky, and C. A. Uribe, “Fast convergence rates for distributed non-Bayesian learning,” _IEEE Transactions on Automatic Control_ , vol. 62, no. 11, pp. 5538–5553, 2017.
  * [7] A. Lalitha, T. Javidi, and A. D. Sarwate, “Social learning and distributed hypothesis testing,” _IEEE Transactions on Information Theory_ , vol. 64, no. 9, pp. 6161–6179, 2018.
  * [8] J. N. Tsitsiklis, “Decentralized detection,” in _In Advances in Statistical Signal Processing_. JAI Press, 1993, pp. 297–344.
  * [9] P. K. Varshney, _Distributed Detection and Data Fusion_. Springer, 2012.
  * [10] D. Bajovic, D. Jakovetic, J. M. Moura, J. Xavier, and B. Sinopoli, “Large deviations performance of consensus+ innovations distributed detection with non-Gaussian observations,” _IEEE Transactions on Signal Processing_ , vol. 60, no. 11, pp. 5987–6002, 2012.
  * [11] Y. İnan, M. Kayaalp, A. H. Sayed, and E. Telatar, “A fundamental limit of distributed hypothesis testing under memoryless quantization,” in _Proc. IEEE International Conference on Communications_ , 2022, pp. 4824–4829.
  * [12] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-efficient learning of deep networks from decentralized data,” in _Proc. International Conference on Artificial Intelligence and Statistics_ , vol. 54, April 2017, pp. 1273–1282.
  * [13] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” _IEEE Signal Processing Magazine_ , vol. 37, no. 3, pp. 50–60, 2020.
  * [14] G. Koliander, Y. El-Laham, P. M. Djurić, and F. Hlawatsch, “Fusion of probability density functions,” _Proc. IEEE_ , vol. 110, no. 4, pp. 404–453, April 2022.
  * [15] M. Kayaalp, Y. Inan, E. Telatar, and A. H. Sayed, “On the arithmetic and geometric fusion of beliefs for distributed inference,” _arXiv:2204.13741_ , 2022.
  * [16] T. Li, H. Fan, J. García, and J. M. Corchado, “Second-order statistics analysis and comparison between arithmetic and geometric average fusion: Application to multi-sensor target tracking,” _Information Fusion_ , vol. 51, pp. 233–243, 2019.
  * [17] T. Li, Y. Xin, Y. Song, E. Song, and H. Fan, “Some statistic and information-theoretic results on arithmetic average density fusion,” _arXiv:2110.01440_ , 2021.
  * [18] L. Gao, G. Battistelli, and L. Chisci, “Multiobject fusion with minimum information loss,” _IEEE Signal Processing Letters_ , vol. 27, pp. 201–205, Jan. 2020.
  * [19] K. C. Chang, C.-Y. Chong, and S. Mori, “Analytical and computational evaluation of scalable distributed fusion algorithms,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 46, no. 4, pp. 2022–2034, 2010.
  * [20] W. Liu, M. Salzmann, and P. Fua, “Context-aware crowd counting,” in _Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 5099–5108.
  * [21] A. H. Sayed, “Adaptation, learning, and optimization over networks,” _Foundations and Trends in Machine Learning_ , vol. 7, no. 4-5, pp. 311–801, July 2014.
  * [22] A. Jadbabaie, P. Molavi, and A. Tahbaz-Salehi, “Information heterogeneity and the speed of learning in social networks,” _Columbia Business School Research Paper_ , no. 13-28, pp. 1–38, May 2013.
  * [23] A. H. Sayed, _Inference and Learning from Data_. Cambridge University Press, 2022, 3 vols.
  * [24] D. Williams, _Probability with Martingales_. Cambridge University Press, 1991.

##  Appendix A Proof of Proposition 1

Consider an arbitrary $\theta\neq\theta^{\circ}$. We denote the log-belief on
$\theta$ as $\bm{x}_{i}\triangleq\log{\bm{\mu}_{i}(\theta)}$. It was shown in
[15] that $\dfrac{1}{i}\bm{x}_{i}\to-\rho_{\textup{A}}$ almost surely as
$i\to\infty$. This implies that for any $\epsilon>0$, there exists an $i_{0}$
such that

$\mathcal{A}_{0}\triangleq\\{\bm{\mu}_{i}(\theta)\leq\exp\\{{-i(\rho_{\textup{A}}-\epsilon)}\\},\
\forall i\geq i_{0},\forall\theta\neq\theta^{\circ}\\}$ (19)

is a high-probability event. After this observation, we split the proof into
two parts:

  * (a)

$\liminf_{i\to\infty}\mathbb{P}\bigg{(}\dfrac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}<t\bigg{)}\geq\Phi(t)$

  * (b)

$\limsup_{i\to\infty}\mathbb{P}\bigg{(}\dfrac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\leq
t\bigg{)}\leq\Phi(t)$.

Then, it readily follows from (a) and (b) that

$\lim_{i\to\infty}\mathbb{P}\bigg{(}\frac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\leq
t\bigg{)}=\Phi(t).$ (20)

###  A.1 Proof of (a)

Using (1) and (2), the log-belief $\bm{x}_{i}$ can be upper bounded under the
event $\mathcal{A}_{0}$ as

$\bm{x}_{i}\leq\bm{x}_{i_{0}}+\\!\\!\\!\sum_{j=i_{0}+1}^{i}\bm{L}_{j}\\!-\\!\\!\\!\sum_{j=i_{0}+1}^{i}\log(1-H\exp\\{{-j(\rho_{\textup{A}}-\epsilon)}\\})$
(21)

where we defined $\bm{L}_{i}\triangleq\log(\sum_{k}\pi_{k}\bm{r}_{k,i})$ and
recall that $H$ is the number of hypotheses. Since

$-\sum_{j=i_{0}+1}^{\infty}\log(1-H\exp\\{{-j(\rho_{\textup{A}}-\epsilon)})\\}<\infty,$
(22)

for every $\delta>0$ there must exist an $i_{1}^{+}>i_{0}$ such that for all
$i\geq i_{1}^{+}$:

$-\frac{1}{\sqrt{i}}\sum_{j=i_{0}+1}^{i}\log(1-H\exp\\{{-j(\rho_{\textup{A}}-\epsilon)})\\})\leq\delta.$
(23)

Consequently, the event

$\mathcal{A}_{1}^{+}\triangleq\Big{\\{}\frac{1}{\sqrt{i}}\bm{x}_{i}\leq\frac{1}{\sqrt{i}}\bm{x}_{i_{0}}+\frac{1}{\sqrt{i}}\sum_{j=i_{0}+1}^{i}\bm{L}_{j}+\delta,\forall
i\geq i_{1}^{+}\Big{\\}}$ (24)

is also a high-probability event. Furthermore, since $\bm{x}_{i_{0}}$ is
almost surely finite, there is an $i_{2}^{+}>i_{0}$ such that

$\mathcal{A}_{2}^{+}\triangleq\\{\frac{1}{\sqrt{i}}\bm{x}_{i_{0}}\leq\delta,\forall
i\geq i_{2}^{+}\\}$ (25)

is a high-probability event as well. Accordingly, for any outcome in
$\mathcal{A}^{+}\triangleq\mathcal{A}_{1}^{+}\cap\mathcal{A}_{2}^{+}$, and for
$i\geq\max\\{i_{1}^{+},i_{2}^{+}\\}$, we have

$\frac{1}{\sqrt{i}}\bm{x}_{i}\leq\frac{1}{\sqrt{i}}\sum_{j=i_{0}+1}^{i}\bm{L}_{j}+2\delta,$
(26)

which directly implies

$\frac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\leq\frac{\sum_{j=i_{0}+1}^{i}\bm{L}_{j}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}+\frac{2\delta}{\sigma_{\textup{A}}}.$
(27)

Finally, with $\overline{\mathcal{A}^{+}}$ denoting the complement of
$\mathcal{A}^{+}$, we obtain that for $i\geq\max\\{i_{1}^{+},i_{2}^{+}\\}$:

$\displaystyle\mathbb{P}\bigg{(}\frac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\geq
t\bigg{)}$
$\displaystyle=\mathbb{P}\bigg{(}\frac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\geq
t\>\cap\>\mathcal{A}^{+}\bigg{)}+\mathbb{P}\bigg{(}\frac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\geq
t\>\cap\>\overline{\mathcal{A}^{+}}\bigg{)}$
$\displaystyle\leq\mathbb{P}\bigg{(}\frac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\geq
t\>\cap\>\mathcal{A}^{+}\bigg{)}+\mathbb{P}(\overline{\mathcal{A}^{+}})$
$\displaystyle\leq\mathbb{P}\bigg{(}\frac{\sum_{j=i_{0}+1}^{i}\bm{L}_{j}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\geq
t-\frac{2\delta}{\sigma_{\textup{A}}}\bigg{)}+\mathbb{P}(\overline{\mathcal{A}^{+}}),$
(28)

where (28) follows from (27). Since $\bm{L}_{i}$’s are i.i.d. random variables
with finite variance, we can use the central limit theorem (CLT) [23] and
arrive at

$\limsup_{i\to\infty}\mathbb{P}\bigg{(}\frac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\geq
t\bigg{)}\leq 1-\Phi(t)$ (29)

by using the facts that $\delta$ and $\mathbb{P}(\overline{\mathcal{A}^{+}})$
can be made arbitrarily small and $\Phi(\cdot)$ is continuous. Note that (29)
is equivalent to

$\liminf_{i\to\infty}\mathbb{P}\bigg{(}\frac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}<t\bigg{)}\geq\Phi(t).$
(30)

###  A.2 Proof of (b)

Similar to the proof of (a), we obtain a lower bound for $\bm{x}_{i}$ under
the event $\mathcal{A}_{0}$ by

$\displaystyle\\!\\!\bm{x}_{i}\geq\bm{x}_{i_{0}}+\\!\\!\\!\sum_{j=i_{0}+1}^{i}\\!\\!\log\bigg{(}\sum_{k\in\mathcal{K}}\frac{\pi_{k}\bm{r}_{k,j}}{1+\exp\\{-j(\rho_{\textup{A}}-\epsilon)\\}\sum_{\theta\neq\theta^{\circ}}\bm{r}_{k,j}(\theta)}\bigg{)}$
$\displaystyle\\!\\!\\!\geq\bm{x}_{i_{0}}\\!\\!+\\!\\!\\!\\!\sum_{j=i_{0}+1}^{i}\\!\\!\\!\log\\!\bigg{(}\\!\sum_{k\in\mathcal{K}}\frac{\pi_{k}\bm{r}_{k,j}}{1+\exp\\{-j(\rho_{\textup{A}}-\epsilon)\\}\sum_{k\in\mathcal{K}}\sum_{\theta\neq\theta^{\circ}}\bm{r}_{k,j}(\theta)}\\!\bigg{)}$
$\displaystyle\\!\\!\geq\bm{x}_{i_{0}}\\!\\!+\\!\\!\\!\\!\\!\sum_{j=i_{0}+1}^{i}\\!\\!\\!\bm{L}_{j}-\\!\\!\\!\\!\\!\sum_{j=i_{0}+1}^{i}\\!\log\bigg{(}{1+\exp\\{-j(\rho_{\textup{A}}-\epsilon)\\}\sum_{k\in\mathcal{K}}\sum_{\theta\neq\theta^{\circ}}\bm{r}_{k,j}(\theta)}\bigg{)}$
$\displaystyle\\!\\!\geq\\!\bm{x}_{i_{0}}\\!\\!+\\!\\!\\!\\!\\!\sum_{j=i_{0}+1}^{i}\\!\\!\\!\bm{L}_{j}-\\!\\!\\!\\!\\!\sum_{j=i_{0}+1}^{i}\\!\\!\\!\\!\exp\\{-j(\rho_{\textup{A}}-\epsilon)\\}\sum_{k\in\mathcal{K}}\sum_{\theta\neq\theta^{\circ}}\\!\\!\bm{r}_{k,j}(\theta).$
(31)

Comparing with (21), we aim to show that the rightmost term in (31), i.e.,

$\bm{S}_{i}\triangleq\sum_{j=i_{0}+1}^{i}\exp\\{-j(\rho_{\textup{A}}-\epsilon)\\}\sum_{k\in\mathcal{K}}\sum_{\theta\neq\theta^{\circ}}\bm{r}_{k,j}(\theta)$
(32)

tends to zero almost surely when divided by $\sqrt{i}$. Observe that
$\bm{S}_{i}$ is non-decreasing. Hence,

$\bm{S}_{\infty}\triangleq\lim_{i\to\infty}\bm{S}_{i}$ (33)

exists almost surely, i.e.,

$\limsup_{i\to\infty}\bm{S}_{i}=\liminf_{i\to\infty}\bm{S}_{i}.$ (34)

Moreover, by the monotone convergence theorem [24],

$\mathbb{E}[\bm{S}_{\infty}]=\lim_{i\to\infty}\mathbb{E}[\bm{S}_{i}]<\infty.$
(35)

Consequently, $\bm{S}_{\infty}$ must be finite almost surely. Then, by
combining the inequality

$0\leq\frac{\bm{S}_{i}}{\sqrt{i}}\leq\frac{\bm{S}_{\infty}}{\sqrt{i}}$ (36)

and the limit $\dfrac{\bm{S}_{\infty}}{\sqrt{i}}\to 0$ as $i\to\infty$ (both
of which hold almost surely), we conclude that
$\dfrac{\bm{S}_{i}}{\sqrt{i}}\to 0$ almost surely as well. Therefore, for
every $\delta>0$ there must exist an $i_{1}^{-}>i_{0}$ such that for all
$i\geq i_{1}^{-}$:

$\frac{1}{\sqrt{i}}\sum_{j=i_{0}+1}^{i}\exp\\{-j(\rho_{\textup{A}}-\epsilon)\\}\sum_{k\in\mathcal{K}}\sum_{\theta\neq\theta^{\circ}}\bm{r}_{k,j}(\theta)\leq\delta.$
(37)

Then, proceeding similarly to the part (a), we define the high-probability
sets:

$\mathcal{A}_{1}^{-}\triangleq\Big{\\{}\frac{1}{\sqrt{i}}\bm{x}_{i}\geq\frac{1}{\sqrt{i}}\bm{x}_{i_{0}}+\frac{1}{\sqrt{i}}\sum_{j=i_{0}+1}^{i}\bm{L}_{j}-\delta,\forall
i\geq i_{1}^{-}\Big{\\}}$ (38)

and

$\mathcal{A}_{2}^{-}\triangleq\\{\frac{1}{\sqrt{i}}\bm{x}_{i_{0}}\geq-\delta,\forall
i\geq i_{2}^{-}\\}.$ (39)

By defining
${\mathcal{A}^{-}\triangleq\mathcal{A}_{1}^{-}\cap\mathcal{A}_{2}^{-}}$, we
arrive at the following analog to (28) for
$i\geq\max\\{i_{1}^{-},i_{2}^{-}\\}$:

$\displaystyle\\!\\!\\!\mathbb{P}\bigg{(}\frac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\leq
t\bigg{)}\leq\mathbb{P}\bigg{(}\frac{\sum_{j=i_{0}+1}^{i}\bm{L}_{j}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\leq
t+\frac{2\delta}{\sigma_{\textup{A}}}\bigg{)}+\mathbb{P}(\overline{\mathcal{A}^{-}}),$
(40)

which implies

$\limsup_{i\to\infty}\mathbb{P}\bigg{(}\frac{\bm{x}_{i}+\rho_{\textup{A}}i}{\sigma_{\textup{A}}\sqrt{i}}\leq
t\bigg{)}\leq\Phi(t),$ (41)

and the proof is complete.

##  Appendix B Proof of Corollary 1

Notice that for any outcome in $\mathcal{A}_{0}$, defined in (19), it is true
that

$1-H\exp\\{-i(\rho_{\textup{A}}-\epsilon)\\}\leq\bm{\mu}_{i}(\theta^{\circ})\leq
1,$ (42)

which implies

$\log(1-H\exp\\{-i(\rho_{\textup{A}}-\epsilon)\\})\leq\log\bm{\mu}_{i}(\theta^{\circ})\leq
0.$ (43)

Since

$\dfrac{1}{\sqrt{i}}\log(1-H\exp\\{-i(\rho_{\textup{A}}-\epsilon)\\})\to 0,$
(44)

the result of Proposition 1 applies here as well.