# WHYPE: A Scale-Out Architecture with
Wireless Over-the-Air Majority for
Scalable In-memory Hyperdimensional Computing

Robert Guirado  Abbas Rahimi  Geethan Karunaratne  Eduard Alarcón  Universitat
Politècnica de Catalunya, Barcelona, Spain  Abu Sebastian  Sergi Abadal
Universitat Politècnica de Catalunya, Barcelona, Spain

###### Abstract

Hyperdimensional computing (HDC) is an emerging computing paradigm that
represents, manipulates, and communicates data using long random vectors known
as hypervectors. Among different hardware platforms capable of executing HDC
algorithms, in-memory computing (IMC) has shown promise as it is very
efficient in performing matrix-vector multiplications, which are common in the
HDC algebra. Although HDC architectures based on IMC already exist, how to
scale them remains a key challenge due to collective communication patterns
that these architectures required and that traditional chip-scale networks
were not designed for. To cope with this difficulty, we propose a scale-out
HDC architecture called WHYPE, which uses wireless in-package communication
technology to interconnect a large number of physically distributed IMC cores
that either encode hypervectors or perform multiple similarity searches in
parallel. In this context, the key enabler of WHYPE is the opportunistic use
of the wireless network as a medium for over-the-air computation. WHYPE
implements an optimized source coding that allows receivers to calculate the
bit-wise majority of multiple hypervectors (a useful operation in HDC) being
transmitted concurrently over the wireless channel. By doing so, we achieve a
joint broadcast distribution and computation with a performance and efficiency
unattainable with wired interconnects, which in turn enables massive
parallelization of the architecture. Through evaluations at the on-chip
network and complete architecture levels, we demonstrate that WHYPE can bundle
and distribute hypervectors faster and more efficiently than a hypothetical
wired implementation, and that it scales well to tens of receivers. We show
that the average error rate of the majority computation is low, such that it
has negligible impact on the accuracy of HDC classification tasks.

##  I Introduction

Hyperdimensional computing (HDC) is an emerging computational framework and is
based on the observation that key aspects of human memory, perception and
cognition can be explained by the mathematical properties of hyperdimensional
spaces comprising high-dimensional vectors known as hypervectors [1]. The
$d$-dimensional hypervectors are generated using (pseudo)random process such
that their components are independent and identically distributed. When the
dimensionality ($d$) is in the thousands, a large number of quasi-orthogonal
hypervectors exist. This allows HDC to combine existing hypervectors into new
hypervectors using well-defined vector operations, such that the resulting
hypervector is unique and preserves the dimensionality. To review HDC and
related computational models in detail refer to [2].

HDC has been employed in a range of applications including cognitive computing
[3, 4], robotics [5], distributed computing [6, 7, 8], communications [9, 10,
11, 12, 13, 14], and in various aspects of machine learning. See [15] for a
comprehensive review. HDC has achieved particularly notable accuracy in
machine learning applications that demand few-shot learning, where other
alternative approaches have generally struggled [16, 17, 18, 19, 20, 21].
Among other advantages, HDC is extremely robust in the presence of failures,
defects, variations, and noise, all of which are synonymous to ultra-low
energy computation. For instance, it has been shown that HDC degrades
gracefully in the presence of various faults in comparison to other
alternative classifiers: HDC tolerates intermittent errors [22], permanent
hard errors in memory [23] or in logic [24], and spatio-temporal variations
[25] in emerging memory technologies. In a similar vein, it also tolerates
noise and interference in the communication channels [11, 14]. These
demonstrate robust operations of HDC under low signal-to-noise ratio and high
variability conditions thanks to the special brain-inspired properties of HDC:
(pseudo)randomness with i.i.d. components, high-dimensionality, and
holographic representations (see [22] for more details).

What different HDC algorithms have in common is that they operate on wide
vectors. Therefore, HDC calls for architectures that handle operations on a
large number of wide vectors efficiently. One of the key operation of HDC is
similarity search. It compares an input hypervector with a typically large
number of hypervectors that are stored in an associative memory. As a
similarity metric, dot-product is often used. This provides a natural fit to
exploit in-memory computing (IMC) for HDC [25, 16]. An IMC core departs from
the von Neumann architectures which move data from a processing unit to a
memory unit and vice versa by exploiting the possibility of performing
operations (dot products, in our case) within the memory device itself [26].
This improves both time complexity and energy consumption of the architecture.

IMC systems have been proposed recently to execute HDC tasks using
hypervectors as wide as 10,000-bit [25]. As further elaborated in Section II,
IMC cores are capable of performing similarity searches through dot-products
with unprecedented energy efficiency, i.e. $\sim$100$\times$ more efficiently
than a digital accelerator [25]. This has sparked interest in HDC systems that
can handle a large search space. For example, certain applications require to
continually add new hypervectors for representing novel classes in the
incremental learning regime [17, 16], and performing similarity search on
them, that can grow over thousands hypervectors. Yet still, how to scale HDC
architectures to perform searches across a large number of classes remains
unclear due to the associated challenges.

HDC architectures can be scaled by either increasing the size of the IMC cores
to accommodate many hypervectors (scale-up) or by deploying multiple
moderately-sized IMC cores to execute the similarity searches in parallel
(scale-out). On the one hand, scaling up requires large IMC cores for the
architecture to be usable in incremental learning applications. This poses a
fundamental problem in terms of array impedances and programming complexity
for the IMC core [27]. On the other hand, scaling out implies distributing
wide hypervectors across a potentially large number of modules, which puts a
large pressure on the system interconnect. More specifically, such an
architecture generates reduction and broadcast communication patterns for
which conventional Networks-on-Chip (NoC) and Networks-in-Package (NiP) suffer
to deliver competitive performance.

To address the scalability problem of IMC-based HDC architectures, we propose
to use wireless communications within the computing package. Wireless Network-
on-Chip (WNoC) have shown promise in alleviating the bottlenecks that
traditional NoC and NiP face, especially for the collective traffic patterns
that appear when scaling out HDC architectures [28, 29, 30, 31, 32]. To that
end, WNoCs provide native broadcast capabilities, which are put to use to
implement a chip-scale network that, at the same time, is able to bundle
multiple hypervectors and distribute the resulting bundled hypervector to a
number of physically distributed similarity search engines.

In this paper, we present WHYPE, a scale-out architecture that employs
wireless over-the-air (OTA) computing to enable scalable in-memory HDC. The
architecture, summarized in Fig. 1, consists of a set of $M$ encoders that
generate query hypervectors. These hypervectors are transmitted
simultaneously, bundled over the air, and received by a set of $N$ similarity
search engines. Bundling is possible thanks to the OTA computing of the bit-
wise majority of the hypervectors, which eliminates the need for a central
point of reduction in the architecture. In turn, the concept of OTA is
feasible because we have full electromagnetic knowledge of the chip package
and we can engineer the constellations to calculate the majority over the air
with low error.

The WNoC proposed for WHYPE is uniquely suited to HDC for two main reasons. On
the one hand, it delivers seamless support for the reduction and broadcast
patterns required by scale-out HDC architectures. On the other hand, it does
so while bypassing the main limitations of wireless interconnects. The
generally low aggregate bandwidth is multiplied by $M$ in WHYPE as we allow
the concurrent transmission of $M$ hypervectors. Also, the impact of the
typically high error rates is minimized thanks to the inherent resilience of
HDC algorithms to noise.

Figure 1: Overview of WHYPE, a many-core wireless-enabled IMC platform. Orange
encoders map to our wireless transmitters, while green IMCs map to our
wireless-augmented IMCs. Bit-wise majority operation required for hypervector
bundling is performed via wireless over-the-air computation.

In summary, this paper makes the following three novel contributions. First,
we present WHYPE, a wireless-enabled architecture for scale-out
hyperdimensional computing. Second, we assess the capacity of WHYPE’s wireless
interconnect to deliver lightweight all-to-all concurrent communications at
the chip scale. Third, we evaluate the impact of imperfect wireless
communications on the accuracy of the similarity search. It is worth noting
that this paper significantly expands on the work presented in [33] by:

  * •

Presenting a complete design for both a wired baseline and the WHYPE
architecture, including details on the wireless interfaces and their
connection to the encoders and similarity search engines.

  * •

Making a comparison of the throughput, area, and power of both the baseline
and the WHYPE architecture, which both motivates the need for the proposed
architecture.

  * •

Extending the performance analysis of WHYPE to the time domain to understand
the achievable bandwidth for the OTA computation.

  * •

Evaluating the accuracy of the WHYPE’s similarity search considering a
distributed dataset.

##  II Background

To facilitate the understanding of the principles behind WHYPE, here we
provide background on the topics of HDC, IMC, and wireless communications at
the chip scale.

###  II-A Hyperdimensional Computing

In this work we focus on a variant of HDC models using 512-dimensional binary
hypervectors. Under this setting, by employing a random process, it is easy to
find a huge number of non-coincident quasi-orthogonal vectors that exhibit
normalized Hamming distance close to 0.5. We call these random hypervectors
atomic hypervectors. One can further create an encoder to operate on these
atomic hypervectors by using operations such as binding, bundling (i.e.
superposition), and permutation to obtain a composite hypervector describing
an object or event of interest. In a classification task, the composite
hypervectors, generated from various examples of the same class, can be
further bundled together to create a single prototype hypervector representing
a class. In this work, the bundling operation is implemented as a logical
element-wise majority operation. During training, the prototype hypervectors
are stored in the associative memory.

In the inference stage, the query hypervectors of unknown objects/events are
generated by the same procedure as in the training stage. The query
hypervector is compared to the prototype hypervectors in the associative
memory. Then, the chosen label is the one corresponding to the prototype
hypervector that has the highest similarity to the query vector. In HDC, the
robustness to failure is given by the spreading of information across
thousands of dimensions. See [20] for more details.

###  II-B In-memory Computing

With each new technology node, the gap between the speed and efficiency of
computation and memory continues to grow. The effects of such a disparity,
commonly known as the memory wall, have been addressed with novel concepts
such as high-bandwidth memory [34] or 2.5D and 3D monolithic integration [35],
among others. However, from an architectural point of view, these solutions
are not solving the fundamental bottleneck arising from the need to move large
quantities of data from memory and back. Instead, IMC appears as a promising
candidate to overcome these challenges [26].

IMC is a form of non von-Neumann computing paradigm that leverages the memory
unit to perform in-place computational tasks, reducing the amount of data
movement and therefore cutting down the latency and energy consumption
associated with in-package communication [26]. At the core of IMC is a
crossbar array with a memory device lying at each cross point of the array.
IMC cores in which these memory devices are resistance-based, and more
specifically those based on phase-change memory (PCM) devices, have recently
shown promising results [36]. In a resistance-based IMC core, we can program
certain values as conductances of cross point memory devices. Executing a
matrix-vector multiplication (MVM), essential to any machine learning
algorithm, is as simple as, first, tuning conductances to match the matrix
values. Second, by exploiting Ohm’s law and Kirchhoff’s law, inputting the
vector as voltages from one side and finally reading the output currents from
a perpendicular side.

IMC architectures are capable of executing various HDC operations [25]. In
this work, we are particularly interested in the similarity search in the
associative memory. As shown in Fig. 2, since the prototype hypervectors
$P_{i}$ will be programmed in an IMC core, the similarity search through the
dot product can be implemented as a MVM with the query hypervector $Q$ as
input vector. This allows performing a dot-product in $O(1)$ time complexity.

Figure 2: Similarity search example in an IMC core. Since the prototype
hypervector of the third column is the most similar one to the query vector
$Q$, it will output more current than the others and its associated label will
be chosen.

###  II-C Wireless Network-on-Chip

Manycore architectures currently rely on NoCs as their interconnect backbone.
However, the performance of NoCs can quickly degrade when serving collective
communication patterns such as reductions or broadcasts, especially when
scaled. In light of this, WNoCs have been recently proposed to complement
wired interconnects due to their natural broadcast support, low system-wide
latency, and adaptive network topology [29, 37, 32, 38]. Even though WNoC
technology is not mature, proof-of-concept designs have been implemented and
tested [39].

In WNoCs, a core or a cluster of cores are equipped with RF transceivers and
antennas [40, 41, 42]. This allows them to modulate and transmit data, which
propagates through the chip package until being picked up and demodulated by
the receivers in the transmitter’s range. By tuning all antennas to the same
channel, WNoC allows to perform low-latency broadcast. However, this comes at
the cost of a low aggregate bandwidth as it two simultaneous broadcast
transmissions will interfere with each other. Moreover, for the same energy,
wireless interconnects are generally much less reliable than wired
interconnects due to their radiative nature. Fortunately, as described in next
sections, the forgiving nature of HDC and the collective nature of its
operations minimize the impact of the WNoC disadvantages.

(a) Logical (top) and physical view (bottom).

(b) Circuital view (shaded blocks are our contribution).

Figure 3: Towards a wireless-enabled scale-out HDC architecture with over-the-
air computing. (a, top) The architecture involves $M$ encoders generating
queries $q_{1}\cdots q_{M}$, the computation of a composite query $Q$ via bit-
wise majority, and $N$ IMC cores performing similarity search over multiple
copies of $Q$. (a, bottom) In the wireless implementation, the IMC cores
receive different versions of $Q$ over space ($Q^{\prime}\cdots Q^{N}$) that
need to be decoded. (b) To enable the decoding of $Q^{\prime}\cdots Q^{N}$
with low error, $q_{1}\cdots q_{M}$ are modulated with BPSK and shifted so
that the overlapped symbols can be decoded easily at the receiver.

##  III Motivation

The main aim of this work is to propose an architecture that employs IMC cores
for HDC and that can be scaled to satisfy the insatiable appetite of the most
demanding workloads. The top chart of Fig. 3(a) shows a logical diagram of a
possible architecture template for an IMC-enabled HDC-based classifier. The
encoding system on the left, possibly divided in a number $M$ of parallel
encoders, translate the input data into query hypervectors. These hypervectors
are then bundled via a bit-wise majority operation, which virtually increases
the computation throughput proportionally to the number of bundled vectors.
The search engine on the right, possibly composed by $N$ IMC cores storing $K$
class templates each, compares the bundled query hypervectors with the
$N\times K$ prototype hypervectors. We note that both the encoders and the
search engines can be implemented with IMC cores.

To scale such an architecture, two broad decisions need to be taken from an
architectural standpoint. First, whether to scale by increasing the size of
the IMC cores (scale-up; increasing $K$) or by placing more similarity search
engines in the system (scale-out, increasing $N$). Second, whether the scaling
is done in a fully integrated way, i.e. placing all encoders and search
engines within a single chip, or using disintegrated alternatives such as the
recent chiplet paradigm. Our proposed architecture, called WHYPE, is build
upon three main observations:

Figure 4: Minimum wire width requirement of crossbar arrays as a function of
the array size, in order to maintain the same effect from IR drop and other
non-idealities on the MVM accuracy

Observation 1: In-memory cores do not scale up well. Increasing the size $K$
of the IMC arrays allows accommodating a larger number and of prototype
hypervectors (classes). However increasing the crossbar size brings with
itself a number of non-ideal properties. For instance, IR drop across the
array is increased due to the interconnect resistance of longer wires and RC
latency is increased due to the increased parasitic capacitance. To counter
these problems, one has to sacrifice one or few important metrics related to
performance, power and area. For e.g. the IR drop can be reduced by increasing
the wire width. The required minimum wire width at different array sizes to
maintain the same effect of IR drop is given in Fig. 4 and in [43]. As shown
in the figure, the wire width exponentially rises, limiting the scalability of
array sizes. Moreover, the complexity of weight programming also increases
with the array size [27]. There are already IMC prototypes supporting up to
256 prototype hypervectors [16]. This may be stretched up to 512 and 1024 but
most likely not to bigger sizes due to the rise of non-ideal properties.
Therefore, scale-out architectures that employ multiple, but relatively small
IMC cores, may be preferable in this scenario.

Observation 2: There exist different application spaces in terms of required
search throughput, and the number of inputs and classes. Following [20], a
generic scalable HDC architecture for various workloads should include a set
of encoders (shown in the left hand-side of Fig. 1) and a set of similarity
search engines (shown in the right hand-side of Fig. 1). The number of
encoders is typically determined by the demand of application, ranging from
few encoders to operate on data from different sensory modalities [44, 45], to
a larger number of encoders working with independent streaming channels [14].
Similarly, the number of required similarity search engines is determined by
the application. It is based on the number of classes that the application has
to support which ranges e.g., from a handful of classes [25, 16] to over one
thousand classes [17], or half million classes [46]. Further, the number of
_active_ classes could change over time, e.g., in a continual learning regime
at the beginning there are about 60 classes that can grow up to 1600 during
the course of incremental learning [17, 16]. In computational terms, this
means that the capacity of the encoders and similarity search engines is
highly dependent on the application. We therefore aim for a generic
architecture to be able to scale out by many inputs or classes coming in.
Assuming a fully integrated design, architects would need to carefully
dimension the encoders and similarity search engines in order to cater to the
needs of a specific application. This fundamentally limits the reuse of the
system in other domains, which may either require faster/broader or
smaller/more efficient searches and where the designed architecture would
either underperform or waste area and power. Such an observation suggests that
a disintegrated architecture, possibly chiplet-based, would be a viable route
for scaling-out IMC-based HDC architectures.

Observation 3: Scaling out HDC architectures is costly. In particular, scaling
the architecture template described in Fig. 3(a) quickly leads to a
communication bottleneck, especially in chiplet-based systems. First, even
though bundling compresses $M$ hypervectors of length $d$ into a single
composite query, that comes at the cost of a heavy reduction communication
flow. This $M$-to-1 traffic pattern is due to the need to bring the different
hypervectors to a common circuit performing the bit-wise majority operation.
The majority operation, besides the significantly large buffers required to
store the $M$ inputs and the output hypervector, takes $M$ inputs of size $d$
with a known complexity of $d\times M^{3.5}$ gates [47]. Only when the
bundling is finalized, the output can be distributed to the $N$ IMC cores.
However, broadcast patterns are generally expensive with growing $N$ [29].

In summary, bundling and data movement quickly becomes a bottleneck when
scaling, especially in disintegrated architectures. In chiplet-based systems,
significant energy ($\sim$1 pJ) and latency ($\sim$20 ns) can be expected per
hop due to connectivity and I/O pin limitations [48], whereas hop counts will
grow at least proportionally to the number of encoders $M$ and the number of
search engines $N$ due to the reduction and broadcast flows, respectively
[42]. Hence, alternative implementations of the reduction-majority-broadcast
pattern are necessary to enable the effective scale-out of HDC architectures.

##  IV WHYPE: A Wireless-Enabled Architecture for Scale-Out of
Hyperdimensional Computing

As depicted in Section III, even though disintegrated scale-out is a desirable
choice for scaling HDC systems, implementing such architectures with wired
interconnects is challenging. This is because of three main reasons:

  * •

The bundling operation generates a reduction pattern that can create a
communication bottleneck at the vicinity of the bundling circuit.

  * •

The query distribution requires broadcast communication, which is inherently
costly for wired interconnects in general and in chiplet-based systems in
particular.

  * •

The bundling operation creates an implicit barrier that forces encoding and
similarity search to be done sequentially. While all three operations can be
pipelined, the end-to-end latency increases.

Our proposed architecture, called WHYPE, addresses the three problems of wired
scale-out at once. To that end, we augment a many-core HDC system with a
wireless chip-scale network specifically designed to eliminate the need to
transfer all the hypervectors to a central point for bundling. Fig. 3(b) shows
the proposed implementation of WHYPE, which is composed by $M$ encoders
augmented with wireless transmitters alongside $N$ IMC cores augmented with
wireless receivers. The transmitters and receivers are slightly modified
versions of a simple BPSK modulator and a coherent receiver and decoder,
respectively. The frequency of operation is high, e.g. 60 GHz, to minimize the
area and power overhead of the RF circuits and the antenna. Finally, given the
monolithic nature of the system, we assume that the clocks of all transmitters
are synchronized.

The mode of operation is as follows. All the encoders broadcast,
simultaneously and through the same wireless channel, the queries $q_{1}\cdots
q_{M}$ that must be bundled to form $Q$. As a consequence of wave propagation
within the package, the receivers will obtain $N$ different versions of the
superposition of the $M$ symbols transmitted concurrently, $Q^{\prime}\cdots
Q^{N}$. Each receiver will then take its received signal and attempt to decode
a single bit representing the majority operation of the $M$ transmitted
symbols. In other words, the decoders will be carefully designed to achieve
$Q^{\prime}=Q$, $\cdots$, $Q^{N}=Q$ with high probability so that,
effectively, the bit-wise majority of the transmitted hypervectors is computed
over the air.

Figure 5: Example of decision regions of over-the-air (OTA) majority
computation for three transmitters $\\{q_{1},q_{2},q_{3}\\}$ at two distinct
receivers. Blue/green regions map to 0/1.

To further illustrate the idea behind over-the-air majority computation, Fig.
5 shows a sample constellation that could result from the superposition of
three transmissions. Each point represents one of the $2^{3}=8$ possible
combinations. Since we are not interested in the values of the three
transmitted bits, but rather in the result of the majority operation, the
decoders only need to distinguish between the combinations that lead to
$maj(\cdot)=0$ and $maj(\cdot)=1$, respectively. The key objective, then, is
to modulate the information in each transmitter so that the received
constellations form two easily separable clusters for $maj(\cdot)=0$ and
$maj(\cdot)=1$. WHYPE achieves this with a very simple variant of source
coding, i.e. through pre-assigned phase shifts. As detailed further in Section
V-A, an exhaustive search is performed offline to find the transmitters’ phase
shifts leading to easy-to-decode majority at the receivers.

At its core, WHYPE exploits three key opportunities, which allow to boost the
value and minimize the disadvantages of wireless chip-scale communications:

Key Opportunity 1: Over-the-air computing is possible because the channel is
static and known beforehand. OTA computing is certainly not new, but it has
generally been hindered by the need of accurate and up-to-date channel state
information, which is extremely hard to guarantee in conventional wireless
networks [49]. In contrast, the in-package scenario is static and allows for a
pre-characterization of the channel [50], hence allowing for an OTA
calculation of the majority operations required by the bundling of
hypervectors.

Key Opportunity 2: The inherent broadcast nature of wireless communication
allows for a single-hop distribution of bundled hypervectors. By using
omnidirectional antennas such as vertical monopoles [51], data is naturally
broadcast with a latency and efficiency hard to achieve with wired on-chip
networks [52]. This feature, together with the OTA bundling, allows to
eliminate the communication bottleneck of HDC scale-out architectures.

Key Opportunity 3: The resilience of the HDC paradigm to errors makes it
tolerant to unreliable communication. A drawback of wireless communications in
general (and of OTA computing in particular) is that it can suffer from
relatively high error rates when compared to wired communications. This
generally leads to low energy efficiency. However, as we illustrate in Fig. 6,
HDC is inherently resistant to errors and opens the door to the use of
unreliable wireless communications without compromising the energy efficiency
and scalability of the architecture.

Figure 6: Impact of the bit error rate (rate of erroneous bits in a bundled
hypervector) on the accuracy of a classification task under the conditions
described in Section V-C.

Figure 7: Overview of the evaluation methodology and layout of a sample
architecture with 3 TXs and 64 RXs. The package is enclosed in a metallic lid
and empty spaces are filled with vacuum. $h_{1}=0.1$ mm; $h_{2}=0.01$ mm;
$l_{1}=7.5$ mm; $s=3.75$ mm; $L_{1}=33$ mm; $L_{2}=30$ mm.

##  V Methodology

Fig. 7 summarizes the procedures followed to evaluate WHYPE from the
perspectives of wireless communications and HDC architectures. First, the
computing package has been modeled in CST [53] assuming a disintegrated
architecture with $M+N$ chiplets with their respective antennas and
transceivers, implementing the scheme from Fig. 3(a). The output of CST
simulations is fed to MATLAB to assess the bit error rate (BER) of the
resulting constellations. The BER is then used in a python-based HDC framework
to characterize the impact of imperfect communication on the HDC
classification accuracy.

Next, we describe the methods to obtain the source coding in Sec. V-A, to
assess the speed and reliability of OTA computing in Sec. V-B, to evaluate the
classification accuracy in Sec. V-C, and the details of the classification
benchmark in Sec. V-D.

###  V-A Source Coding Optimization

As shown in Fig. 3(b), transmitters encode the bits of their queries using a
BPSK encoder plus two specific phase shifts (for the symbols ’0’ and ’1’,
respectively). That is, all symbols will have the same amplitude, but a
different phase. Since we let the $M$ encoders to transmit simultaneously,
each of the $N$ receivers will observe a slightly different superposition of
all the transmitted symbols. This leads to a $N$ different constellations of
$2^{M}$ points each.

In this context, the objective is to select the phase shifts at each
transmitter so that all the received constellations are clustered in two
separable decision regions, each corresponding to the case where
$maj(\cdot)=0$ and $maj(\cdot)=1$, respectively. This optimization process has
two constraints. On the one hand, we have to make sure that each transmitter
only uses two phases, for its symbol ’0’ and ’1’, respectively. On the other
hand, the phases at one transmitter have an impact on the constellations of
all receivers, which implies that a joint optimization considering all RXs is
needed.

The optimization process starts by simulating the simultaneous transmissions
in CST. We consider a set of 8 phases in each transmitter (i.e. in 45 degree
steps) and evaluate the amplitude and phase obtained at each of the receivers.
Then, the results are fed to MATLAB, where the decision regions are computed
using the $K$-means clustering algorithm with $K=2$ over the constellation.
These regions are used to evaluate the average error across all receivers
using the methods described in Sec. V-B. The combination of phases leading to
the lower average error rate is selected. An illustrative example with three
transmitters and three receivers is shown in Fig. 8, with the detail of a
specific receiver in Fig. 9.

Figure 8: Sweep of all possible phase combinations (top) and the one that
minimizes the error rate of the majority computation (bottom). Blue/green
symbols map to logical 0/1.

###  V-B Over-The-Air Computing Evaluation

Frequency-Domain Simulation. As described in the previous section,
electromagnetic simulations are needed to assess the performance of the over-
the-air computation process. We model an interposer-based package with $M+N$
chiplets, each with its own antenna, with the dimensions depicted in Fig. 7.
We sweep the phases in each transmitter and obtain the amplitude and phase at
the receivers in the frequency domain. The operating frequency is 60 GHz and
the transmission power is 0 dBm per antenna, compatible with existing WNoCs
[41, 39]. Further, we assume $M=3$ transmitters and $N=64$ receivers, unless
otherwise noted. Nevertheless, the analysis can be extended to higher
frequencies, different power levels, different package configurations, or
different number of transmitters and receivers.

Figure 9: Constellation and truth table with transmitted phases/ bits for a
specific RX. Blue/green symbols map to logical 0/1.

Reliability Analysis. Once the phases are swept and the candidate
constellations are obtained, we compute the BER at each RX, for all the
different possible constellations, and choose the one that leads to the lowest
average BER across RXs. In all cases, the BER is evaluated approximating the
centroids of each cluster as received symbols, and using the analytical
expression of error rate of BPSK, which is the modulation used in WHYPE. This
yields

$BER^{BPSK}=0.5\cdot\text{erfc}\bigg{(}{\frac{0.5\cdot
d_{c}}{\sqrt{N_{0}}}}\bigg{)},$ (1)

where $erfc(\cdot)$ is the complementary error function, $d_{c}$ is the
distance among centroids and $N_{0}$ is the noise spectral density.

Transmission Speed Analysis. To assess the speed at which the information can
be modulated reliably, time-domain simulations are required. In particular, we
use CST to obtain the channel impulse response $h_{i}(t)$ with simultaneous
source excitation from the $M$ transmitters to the receiver $i\in[1,N]$. Then,
the power-delay profile (PDP) of receiver $i$, $P_{i}(\tau)$, is evaluated as
$P_{i}(t)=|h(t)|^{2}$. The delay spread $\tau^{(i)}_{rms}$ between the sources
and receiver $i$ is calculated as the second moment of the PDP as

$\tau^{(i)}_{rms}=\sqrt{\frac{\int(\tau-\bar{\tau_{i}})^{2}P_{i}(\tau)d\tau}{\int
P_{i}(\tau)d\tau}},$ (2)

where $\bar{\tau_{i}}=\frac{\int\tau P_{i}(\tau)d\tau}{\int P_{i}(\tau)d\tau}$
is the mean delay of the PDP for receiver $i$. Since all transmitters need to
be synchronized, the modulation speed needs to be the same for all
transmitters and below the coherence bandwidth $B_{c}$ of the system. Hence,
we calculate the coherence bandwidth via the worst-case delay spread among all
receivers, $\tau_{rms}$, so that

$\tau_{rms}=\max_{i}\tau^{(i)}_{rms}\Longrightarrow
B_{c}=\frac{1}{\tau_{rms}}.$ (3)

###  V-C Similarity Search Evaluation

Once the final transmitter phases leading to the lowest average BER are
chosen, an in-house python framework is used to evaluate the impact of
imperfect OTA majority computation on the accuracy of the classification task.
Every similarity search engine connected to a receiver stores 64 different
prototype hypervectors, i.e., 64 different classes, each with 512-bit that
suffices for the scenario considered in this paper. This is compatible with
current experimentally validated IMC cores [54, 36]. The area and energy
evaluations in this paper assumed a scaled version of the IMC from [54, 36].
Finally, errors coming from the OTA computations are modeled as uncorrelated
bit flips over the query hypervectors.

Bundling alternatives. While the baseline bundling consists of simply
computing the bit-wise logical majority result across the different TX bits,
we also consider permuted bundling. This bundling consists of permuting the
queries in the TXs prior to applying the majority operation on them. By
permuting the hypervectors we obtain two benefits. First, this allows the
identification of the transmitter of the detected class from the composite
query. The second direct benefit of permuting the hypervectors is that it
helps increasing the quasi-orthogonality between them. This has a direct
impact on accuracy, since the majority operation over multiple non-orthogonal
hypervectors (i.e. not permuted) would yield a bundled hypervector that is
hard to classify.

###  V-D Classification Benchmarks

We perform experiments using the Omniglot dataset [55]. The dataset provides
handwritten images of characters from 50 different alphabets. The number of
characters in each alphabet varies from 14 to 55. In total there are 1623
character classes and 20 example images from each class. The dataset is
further divided into a training and test set containing 964 and 659 character
classes respectively. The goal of this benchmark is to train an encoder on the
image and label data given in the training set and evaluate classification
accuracy on the test images. We perform two types of experiments.

Experiment 1: Few-shot Learning. First, we evaluate the few-shot learning
capability of the system. We do this by, first, meta-training an encoder on
the training set data and evaluating the few-shot classification accuracy over
a series of 1000 episodes. In each episode, we select 100 classes and 20
encoded hypervector examples (shots) from each class within the test set.
These support vectors are distributed among the IMC modules on the RXs. From
the remaining images of the same 100 classes, we select 1 query image to
encode per TX. The encoded query hypervectors are over-the-air bundled (with
or without permutation) and received at each RX to perform the similarity
search. The final classification accuracy is the average accuracy across the
1000 episodes.

Experiment 2: Continual Learning. Secondly, we focus on the continual learning
capability of the system. For this, we have a similar setting to that of
Experiment 1, with few changes. Here, instead of always choosing a fixed 100
classes per episode, we start with 64 classes and gradually incorporate new
classes over a series of sessions. In each session, 64 additional classes are
selected from the set of unselected classes so far and 5 selected support
examples from these novel classes are provided to the IMCs. During a session,
queries can be selected from all classes (both novel and old) currently under
selection and one each is assigned to the encoders at the transmitters.

##  VI Performance Evaluation

We next present the evaluation of WHYPE, first from the perspective of the OTA
majority in Section VI-A, and then from the perspective of the classification
task in Section VI-B. Additionally, we evaluate the area and power overhead of
the architecture in Section VI-C.

###  VI-A Over-the-Air Computing

Fig. 8 illustrates the exhaustive search performed in our system with $M=3$
transmitters, shown only in three receivers chosen at random. The bottom chart
of the same figure shows the resulting constellations after the optimization.
The constellation in another random receiver is shown in more detail in Fig.
9, together with the phases chosen for the three transmitters: 0o/90o,
315o/135o, and 225o/180o for the symbols ’0’/’1’ of transmitters TX1, TX2 and
TX3, respectively.

Reliability Analysis. Once the phases are set, we evaluate the error rate of
all 64 receivers using Eq. (1). Fig. 10 shows the BER of each receiver. It can
be observed how the BER values are highly dependent on the receiver position,
with values as large as $\sim$0.1 and also lower than 10-5 in a significant
amount of cases. In average, the error rate is below 0.01.

Figure 10: Error rate for each individual RX in the architecture. The dashed
line indicates the average value.  Figure 11: Error rate as a function of the
number of receivers. Figure 12: Scaling of the delay spread and coherence
bandwidth over an increasing number of receivers for the different transmitted
symbols in the scenario with three transmitters.

(a) Baseline bundling

(b) Permuted bundling

Figure 13: Similarity results comparison for different forms of bundling one,
three, five, and seven hypervectors.

To understand how the error rate could scale with the number of receivers, we
re-simulate the entire architecture with a varying number of RX cores and
computing the average BER obtained in each case. As shown in Fig. 11, the
average BER generally increases with the number of receivers for which we are
optimizing the architecture. This is expected since, when accommodating more
constellations in our optimal TX phases search, we are imposing more
conditions and hindering the joint optimization across all receivers.

Time-Domain Analysis. Still with three transmitters and a number of receivers
growing from 1 to 64, we obtain the delay spread and coherence bandwidth of
the received signal for each of the $2^{3}=8$ possible transmission
combinations. As described earlier, we take the worst-case among all receivers
in each simulation. As Fig. 12 shows, all symbol combinations (marked as
different IDs in the figure) have a similar scaling behavior in terms of delay
spread. Values lower than 0.1 ns (coherence bandwidth greater than 10 GHz) are
obtained consistently for systems with less than 10 receivers. The performance
degrades to values around 0.166 ns ($\sim$ 6 GHz) in larger architectures.
Given that BPSK has a spectral efficiency of 1 b/s/Hz, the evaluated system
would have a total throughput between $3\times 6=18$ Gb/s and $3\times 10=30$
Gb/s from the encoders to the similarity search engines.

TABLE I: Accuracy in the few-shot learning experiment. Baseline Bundling |  | Number of bundled hypervectors
---|---|---
Channel | 1 | 3 | 5 | 7 | 9 | 11
Ideal | 1 | 0.966 | 0.902 | 0.803 | 0.704 | 0.543
Wireless | 1 | 0.966 | 0.9 | 0.801 | 0.699 | 0.537
Permuted Bundling |  | Number of bundled hypervectors
Channel | 1 | 3 | 5 | 7 | 9 | 11
Ideal | 1 | 1 | 1 | 1 | 0.995 | 0.978
Wireless | 1 | 1 | 1 | 1 | 0.994 | 0.963

###  VI-B Classification Experiments

To start the assessment of the HDC-based classification tasks, we first
perform the few-shot learning experiment while increasing the BER gradually.
As Fig. 6 depicts, the class accuracy remains above 99% even when we apply bit
flips equivalent to a BER of 0.26. This means that the noise robustness
provided by the HDC properties relaxes the error link conditions, ensuring a
correct behaviour under the worst-case wireless scenarios, as we show next.

Few-shot Learning. Fig. 13(a) and Fig. 13(b) show the similarity search result
for the baseline bundling and permuted bundling cases, respectively, in the
few-shot learning experiment. The figures show how a single 512-bit query can
accommodate several queries via bundling (blue line), and that the wireless
system (green line) is able to correctly classify the same queries despite
having some bits flipped.

Table I shows the numerical results of the final class accuracy for the
executed task, comparing an ideal channel without errors with our wireless
channel with a sizable BER. The effect of the imperfect bundling is negligible
in terms of accuracy, as predicted by Fig. 6. Moreover, the permuted bundling
significantly improves the baseline bundling, confirming that the proposed
approach supports the aggregation of a dozen hypervectors over the air and the
parallelization of similarity search over tens of IMCs.

Continual Learning. Figure 14 shows the evolution of the accuracy as the
system learns new classes from the initial dictionary of 100 classes, until
the entire test set is learnt. Each new set of classes degrades the
classification accuracy because possible similarity between classes
represented in 512-bit hypervectors. From the figure, it is again clear that
(i) the bit flips associated to imperfect OTA bundling have a negligible
effect over the classification accuracy, and (ii) permutation provides a
consistent improvement over the baseline bundling.

TABLE II: Accuracy in the continual learning experiment. Baseline Bundling |  | Number of bundled hypervectors
---|---|---
Channel | 1 | 3 | 5 | 7 | 9 | 11
Ideal | 0.87 | 0.60 | 0.47 | 0.40 | 0.35 | 0.33
Wireless | 0.81 | 0.59 | 0.47 | 0.39 | 0.35 | 0.32
Permuted Bundling |  | Number of bundled hypervectors
Channel | 1 | 3 | 5 | 7 | 9 | 11
Ideal | 0.87 | 0.81 | 0.73 | 0.63 | 0.55 | 0.46
Wireless | 0.81 | 0.80 | 0.73 | 0.63 | 0.51 | 0.46

Table II shows the final accuracy for 600 classes and different bundling
configurations. The results confirm the conclusions given above in terms of
impact of wireless OTA and permuted bundling. It is worth noting that the
similarity search over the Omniglot dataset in continual learning does not
perform well when bundles exceed a few hypervectors, even with permutations.
This suggests that longer hypervectors or an application with a larger dataset
would be required to fully benefit from bundling.

Figure 14: Classification accuracy for the continuous learning case with 3
bundled hypervectors.

###  VI-C Comparison with Wired Alternative

Here, we discuss about the performance of WHYPE in terms of speed and the
overheads in terms of area and power. To that end, we calculate the area and
energy of the entire architecture except for the encoders, which may widely
vary depend on the application space, leading to various types of
implementations and hardware realizations [56, 25, 57].

To evaluate the area and energy of the similarity search engines, we model
them as IMC cores using the design from [54, 36] as baseline. In more detail,
the unit cell at each cross-point of the crossbar consists of 8 transistors
and 4 PCM devices (8T4R). The PCM crossbar is back-end-of-the-line integrated
with CMOS peripherals, namely, a Pulse-Width Modulation (PWM) circuit which
convert a 8-bit digital input vector to an array of time encoded pulses, and
an ADC that digitizes the output current of the crossbar to multi-bit values.
The area and energy consumption of these components, which are summarized in
Table III, have been scaled to consider the technology and dimensions of our
architecture.

For the sake of comparison, we evaluate the area and power of both WHYPE’s
interconnect and a hypothetical wired interposer-based alternative. In the
former case, we consider wireless transmitters and receivers compatible with
the requirements of WHYPE [58, 59, 39, 60]. In the latter case, the scheme
from Figure 3(b) is implemented using off-chip interconnects. The majority
operation is performed in a central chiplet containing input/output buffers
and a 512-wide $M$-input majority gate. The majority and similarity search
chiplets are equipped with routers and circuitry to implement serial chip-to-
chip links. We assume that the chiplets form a mesh network, similarly to
[48]. Routers are modeled using DSENT [61] with a four-stage pipeline and
minimal buffering. Off-chip links are modeled as single-lane links with the
energy calculated according to the UCIe standard [62] and the area adapted
from [63]. We assume a 32nm technology node with $V_{DD}=1$ V and $f_{clk}=1$
GHz. See Table IV for a summary.

Overhead Analysis. We evaluate the area overhead of the system and the total
energy consumed in the hypervector collection, bundling, distribution, and
similarity search process using data from Table III and Table IV. It is
observed that the off-chip links and wireless transceivers are the most power-
hungry components in the interconnect, while the majority gate circuitry and
the routers are also area consuming, mostly due to the wide datapath of 512
bits required to transport the hypervectors. The IMC cores are very efficient
in terms of area compared with the interconnect components, while their energy
efficiency is penalized by the PWM and ADC circuitry.

Figure 15 shows a breakdown of the area and energy of the HDC system assuming
$M=3$ encoders and $N=8$ similarity search engines (512 classes). As can be
seen, the interconnect is the most area consuming sub-system, whereas the
similarity search is energy consuming due to the large MVM operations
performed in the IMC tiles. WHYPE reduces the interconnect area by 3.2$\times$
with a modest effect in the energy because of the small scale of the system.
To evaluate scalability, Figure 16 shows the area and energy of the
interconnect as a function of the number of similarity search engines $N$, for
$M=3$. It is observed how, even conservatively assuming single-lane wired
links, WHYPE is superior in both area and energy consumption, with a gap that
widens as the system is scaled out. This is because WHYPE eliminates any wired
connection between chiplets, while the wired alternative needs to traverse the
entire system through a mesh topology.

TABLE III: Area and energy breakdown of IMC for a 512$\times$64 matrix-vector
multiplication in 32nm technology.

Component | Area | Energy | Source
---|---|---|---
[mm2] | [nJ]
PCM Crossbar | 0.09 | 0.27 |  Scaled from [54, 36]
PWM Peripherals | 0.20 | 8.79
CCO-based ADC | 0.01 | 6.23

PCM = Phase-Change Memory; PWM = Pulse-Width Modulation
CCO = Current Controlled Oscillator; ADC = Analog-Digital Converter

TABLE IV: Area and energy breakdown of single instances of interconnect
components in 32nm technology.

| Component | Area | Energy | Source
---|---|---|---|---
| [mm2] | [pJ/bit]
Majority Chiplet | Majority Gate§ | 0.32 | 0.17 | [47]
Buffer | 0.009 | 0.005‡ | [64]
Wired Network | Router | 0.36 | 0.03 | [61]
Link (on-chip) | 4$\cdot$10-4 | 0.06
Link (off-chip) | 0.25* | 1 | [63, 62]
Wireless Interface† | SerDes | 0.04 | 0.54 | [58]
Data Converter | 0.03 | 0.07 | [59]
Transmitter | 0.12 | 1.5 |  [39],
Receiver | 0.12 | 1.3 | [60]
Antenna | 0.08 | N/A | [65]

Unless noted, the width of the components is 512 bits.
§Five-input, one-output gate. ‡Per operation (read, write).
†Dimensioned to operate at 10 Gb/s.
*Per pin, serial link operating at 16 Gb/s.

(a) Area

(b) Energy

Figure 15: Area and energy consumption in a system with $M=3$ encoders and
$N=8$ search engines, comparing the WHYPE approach with the wired baseline. I,
M and S stand for interconnect, majority, and search engines.

(a) Area overhead

(b) Energy overhead

Figure 16: Area and energy overheads of the interconnect with $M=3$ encoders
and a variable number $N$ of search engines.

Bottleneck Analysis. We assess the performance of WHYPE through an analysis of
the latency and throughput of the steps followed since the encoders output the
hypervectors until the bundled hypervectors reach the similarity search
engines. In the wireless case, the delay analysis is simple: the latency
corresponds to the time needed to wirelessly transmit 512 bits. Assuming a 10
Gb/s transmitter, the latency of the majority operation in WHYPE is 51.2 ns
independently of the number of encoders or similarity search engines. In
contrast, a wired alternative would be bottlenecked by the majority
calculation and the communications happening before and after. Latency-wise,
the main delay in the wired case is the time required for the hypervectors to
travel through the chip-to-chip network. With the conditions here assumed, the
delay scales as 2$\sqrt{M+N}$/3 which is the average number of hops to connect
two arbitrary chiplets, with each hop taking 36 ns (4 to traverse a router and
32 to transmit 512 bits through a 16 Gb/s serial link). Quickly, the delay
becomes much higher than in the wireless case. In comparison, the IMC cores
considered here can take between 10 and 128 nanoseconds to realize the
similarity calculations, depending on the required accuracy [36]. Although
majority and similarity operations can be pipelined, a wired NiP would easily
bottleneck the system.

In terms of throughput, WHYPE does not present a bottleneck due to the
seamless all-to-all connection between encoders and similarity search engines.
At 10 Gb/s of line rate, since all receivers will obtain the bundled
hypervector at the same time, the overall throughput is 10$\times$M$\times$N
Gb/s. In the hypothetical wired alternative, the bottleneck is the bisection
bandwidth of the system. Since the majority chiplet, being in a mesh network,
has only four chip-to-chip links, the bisection bandwidth between the encoders
and the similarity search engine will be, at most, twice the capacity of the
chip-to-chip links. In our assumed scenario, the throughput would be 32 Gb/s.
Therefore, WHYPE will be faster even for relatively low values of $M$ and $N$.

Comparison with 3D Interconnects. A few works have proposed to implement HDC-
based 3D ICs [23, 24]. One of the reasons could be the reduced link length as
compared to the planar NoC or NiP alternatives. However, the wired nature of
3D ICs and the _AllGather_ nature of the communication suggest that the
interconnect will continue being a bottleneck. Moreover, 3D ICs can suffer
from heat dissipation issues which limit their scaling ability. Due to this,
existing efforts only consider up to a couple dozen classes [23].

##  VII Conclusion

In this work, we introduced an OTA on-chip computing concept capable of
overcoming the scalability bottleneck present in wired NoC architectures when
scaling out IMC-based HDC systems. By using a WNoC communication layer, a
number of encoders is able to concurrently broadcast HDC queries towards all
the IMC cores within the architecture. Then, a pre-characterization of the
propagation environment allows to map the received constellations to the
computed composite query, in each core, based on a decision region strategy.
Through a proper correspondence between the TX phases, the received
constellation and the decision region, we have shown that the opportunistic
calculation of the bit-wise majority of the transmitted HDC queries is
possible with low error. We demonstrated the concept and shown its scalability
up to 11 TXs and 64 RXs, obtaining the BER of the OTA approach and later
employing it to evaluate the impact of the WNoC errors in a HDC classification
task. Overall, we conclude that the quality of the WNoC links are solid enough
to have a negligible impact on the application accuracy, mostly thanks to the
great error robustness of HDC.

## Acknowledgment

Authors gratefully acknowledge funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement No 863337 (WiPLASH),
and Horizon Europe research and innovation programme under grant agreement No
101042080 (WINC).

## References

  * [1] P. Kanerva, “Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors,” _Cognitive Computation_ , vol. 1, 06 2009.
  * [2] D. Kleyko _et al._ , “A survey on hyperdimensional computing aka vector symbolic architectures, part I: models and data transformations,” _ACM Comput. Surv._ , may 2022.
  * [3] T. A. Plate, “Analogy Retrieval and Processing with Distributed Vector Representations,” _Expert Systems_ , vol. 17, no. 1, pp. 29–40, 2000.
  * [4] P. Kanerva, “What We Mean When We Say ”What’s the Dollar of Mexico?”: Prototypes and Mapping in Concept Space,” in _Proceedings of the AAAI Fall Symposium ’10_ , 2010, pp. 2–6.
  * [5] P. Neubert _et al._ , “An Introduction to Hyperdimensional Computing for Robotics,” _KI - Künstliche Intelligenz_ , vol. 33, no. 4, pp. 319–330, 2019.
  * [6] D. Verma _et al._ , “Towards A Distributed Federated Brain Architecture using Cognitive IoT Devices,” _Proceedings of COGNITIVE_ , 2017.
  * [7] C. Simpkin _et al._ , “A Scalable Vector Symbolic Architecture Approach for Decentralized Workflows,” in _Proceedings of COLLA_ , 2018.
  * [8] R. Tomsett _et al._ , “Demonstration of Dynamic Distributed Orchestration of Node-RED IoT Workflows Using a Vector Symbolic Architecture,” in _Proceedings of IEEE SMARTCOMP ’19_ , 2019, pp. 464–467.
  * [9] P. Jakimovski _et al._ , “Collective communication for dense sensing environments,” in _Proceedings of the IE’11_ , 2011, pp. 157–164.
  * [10] D. Kleyko _et al._ , “Dependable MAC layer architecture based on holographic data representation using hyper-dimensional binary spatter codes,” in _Multiple Access Communications_ , 2012.
  * [11] H.-S. Kim, “HDM: Hyper-Dimensional Modulation for Robust Low-Power Communications,” in _IEEE International Conference on Communications_ , 2018\.
  * [12] C. W. Hsu _et al._ , “Collision-tolerant narrowband communication using non-orthogonal modulation and multiple access,” in _Proceedings of the GLOBECOM_ , 2019.
  * [13] C.-W. Hsu _et al._ , “Non-orthogonal modulation for short packets in massive machine type communications,” in _Proceedings of the GLOBECOM ’20_ , 2020.
  * [14] M. Hersche _et al._ , “Near-channel classifier: symbiotic communication and classification in high-dimensional space,” _Brain Informatics_ , vol. 8, no. 1, p. 16, Aug 2021.
  * [15] D. Kleyko _et al._ , “A survey on hyperdimensional computing aka vector symbolic architectures, part II: applications, cognitive models, and challenges,” _ACM Comput. Surv._ , jun 2022.
  * [16] G. Karunaratne _et al._ , “In-memory realization of in-situ few-shot continual learning with a dynamically evolving explicit memory,” in _IEEE European Solid-state Devices and Circuits Conference (ESSDERC)_ , 2022\.
  * [17] M. Hersche _et al._ , “Constrained Few-shot Class-incremental Learning,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2022, pp. 1–19.
  * [18] G. Karunaratne _et al._ , “Robust high-dimensional memory-augmented neural networks,” _Nature Communications_ , vol. 12, 2021.
  * [19] A. Moin _et al._ , “A Wearable Biosensing System with In-sensor Adaptive Machine Learning for Hand Gesture Recognition,” _Nature Electronics_ , vol. 4, no. 1, pp. 54–63, 2021.
  * [20] A. Rahimi _et al._ , “Efficient Biosignal Processing Using Hyperdimensional Computing: Network Templates for Combined Learning and Classification of ExG Signals,” _Proceedings of the IEEE_ , vol. 107, no. 1, pp. 123–143, 2019.
  * [21] A. Burrello _et al._ , “One-shot learning for iEEG seizure detection using end-to-end binary operations: Local binary patterns with hyperdimensional computing,” in _Proceedings of the IEEE BioCAS_ , 2018, pp. 1–4.
  * [22] A. Rahimi _et al._ , “A Robust and Energy Efficient Classifier Using Brain-Inspired Hyperdimensional Computing,” in _Proceedings of the IEEE/ACM ISLPED_ , 2016.
  * [23] H. Li _et al._ , “Hyperdimensional Computing with 3D VRRAM In-Memory Kernels: Device-Architecture Co-Design for Energy-Efficient, Error-Resilient Language Recognition,” in _Proceedings of the IEEE IEDM_ , 2016.
  * [24] T. Wu _et al._ , “Brain-Inspired Computing Exploiting Carbon Nanotube FETs and Resistive RAM: Hyperdimensional Computing Case Study,” in _Proceedings of the IEEE ISSCC_ , 2018.
  * [25] G. Karunaratne _et al._ , “In-memory hyperdimensional computing,” _Nature Electronics_ , vol. 3, pp. 327–337, 2020.
  * [26] A. Sebastian _et al._ , “Memory devices and applications for in-memory computing,” _Nature Nanotechnology_ , vol. 15, 03 2020.
  * [27] S. Yu _et al._ , “Scaling-up resistive synaptic arrays for neuro-inspired architecture: Challenges and prospect,” in _Proceedings of the IEEE IEDM_ , 2015.
  * [28] S. Laha _et al._ , “A New Frontier in Ultralow Power Wireless Links: Network-on-Chip and Chip-to-Chip Interconnects,” _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ , vol. 34, no. 2, 2015\.
  * [29] M. Ahmed _et al._ , “An asymmetric, one-to-many traffic-aware mm-wave wireless interconnection architecture for multichip systems,” _IEEE T. on Emerging Topics in Computing_ , 2020.
  * [30] S. Jog _et al._ , “One protocol to rule them all: Wireless network-on-chip using deep reinforcement learning,” in _Proceedings of the NSDI ’21_ , 2021, pp. 973–989.
  * [31] A. Ganguly _et al._ , “Interconnects for DNA, Quantum, In-Memory and Optical Computing: Insights from a Panel Discussion,” _IEEE Micro_ , 2022\.
  * [32] S. Abadal _et al._ , “Graphene-based wireless agile interconnects for massive heterogeneous multi-chip processors,” _IEEE Wireless Communications Magazine_ , 2022.
  * [33] R. Guirado _et al._ , “Wireless on-chip communications for scalable in-memory hyperdimensional computing,” in _Proceedings of the IJCNN/WCCI 2022_ , 2022.
  * [34] J. Kim _et al._ , “HBM: Memory solution for bandwidth-hungry processors,” in _2014 IEEE Hot Chips 26 Symposium (HCS)_ , 2014.
  * [35] M. Shulaker _et al._ , “Three-dimensional integration of nanotechnologies for computing and data storage on a single chip,” _Nature_ , vol. 547, pp. 74–78, 2017.
  * [36] R. Khaddam-Aljameh _et al._ , “HERMES-core–a 1.59-TOPS/mm2 PCM on 14-nm CMOS in-memory compute core using 300-ps/LSB linearized CCO-based ADCs,” _IEEE Journal of Solid-State Circuits_ , 2022.
  * [37] S. Abadal _et al._ , “Opportunistic beamforming in wireless network-on-chip,” in _Proceedings of the IEEE ISCAS_ , 2019.
  * [38] M. F. Imani _et al._ , “Metasurface-programmable wireless network-on-chip,” _Advanced Science_ , 2022.
  * [39] X. Yu _et al._ , “Architecture and design of multichannel millimeter-wave wireless noc,” _IEEE Design Test_ , vol. 31, no. 6, pp. 19–28, 2014.
  * [40] H. M. Cheema _et al._ , “The last barrier: on-chip antennas,” _IEEE Microwave Magazine_ , vol. 14, no. 1, pp. 79–91, 2013.
  * [41] X. Timoneda _et al._ , “Engineer the channel and adapt to it: Enabling wireless intra-chip communication,” _IEEE Transactions on Communications_ , vol. 68, no. 5, pp. 3247–3258, 2020.
  * [42] R. Guirado _et al._ , “Dataflow-Architecture Co-Design for 2.5D DNN Accelerators using Wireless Network-on-Package,” in _Proceedings of the ASP-DAC ’21_ , 2021, pp. 806–812.
  * [43] S. Yu _et al._ , “Scaling-up resistive synaptic arrays for neuro-inspired architecture: Challenges and prospect,” in _2015 IEEE International Electron Devices Meeting (IEDM)_ , 2015, pp. 17.3.1–17.3.4.
  * [44] E. Chang _et al._ , “Hyperdimensional Computing-based Multimodality Emotion Recognition with Physiological Signals,” in _Proceedings of the IEEE AICAS_ , 2019.
  * [45] A. Mitrokhin _et al._ , “Symbolic Representation and Learning with Hyperdimensional Computing,” _Frontiers in Robotics and AI_ , pp. 1–11, 2020.
  * [46] A. Ganesan _et al._ , “Learning with Holographic Reduced Representations,” in _Advances in Neural Information Processing Systems_ , 2021.
  * [47] J. Choudhary _et al._ , “Generalized majority voter design method for n-modular redundant systems used in mission-and safety-critical applications,” _Computers_ , vol. 8, no. 1, p. 10, 2019.
  * [48] Y. S. Shao _et al._ , “Simba: Scaling deep-learning inference with multi-chip-module-based architecture,” in _Proceedings of the MICRO-52_. ACM, 2019, p. 14–27.
  * [49] U. Altun _et al._ , “The magic of superposition: A survey on simultaneous transmission based wireless systems,” _IEEE Access_ , vol. 10, pp. 79 760–79 794, 2022.
  * [50] D. Matolak _et al._ , “Channel modeling for wireless networks-on-chips,” _IEEE Communications Magazine_ , vol. 51, no. 6, pp. 180–186, 2013.
  * [51] V. Pano _et al._ , “TSV antennas for multi-band wireless communication,” _IEEE Journal on Emerging and Selected Topics in Circuits and Systems_ , vol. 10, no. 1, pp. 100–113, 2020.
  * [52] S. Abadal _et al._ , “Orthonoc: A broadcast-oriented dual-plane wireless network-on-chip architecture,” _IEEE Transactions on Parallel and Distributed Systems_ , vol. 29, no. 3, pp. 628–641, 2018.
  * [53] “CST Microwave Studio,” [Online]. Available: https://www.cst.com. Accessed 28-September-2021.
  * [54] R. Khaddam-Aljameh _et al._ , “HERMES Core – A 14nm CMOS and PCM-based In-Memory Compute Core using an array of 300ps/LSB Linearized CCO-based ADCs and local digital processing,” in _2021 Symposium on VLSI Technology_ , 2021\.
  * [55] B. M. Lake _et al._ , “Human-level concept learning through probabilistic program induction,” _Science_ , vol. 350, no. 6266, pp. 1332–1338, 2015\.
  * [56] V. Joshi _et al._ , “Accurate deep neural network inference using computational phase-change memory,” _Nature communications_ , vol. 11, no. 1, pp. 1–13, 2020.
  * [57] F. Montagna _et al._ , “PULP-HD: Accelerating brain-inspired high-dimensional computing on a parallel ultra-low power platform,” in _2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC)_. IEEE, 2018, pp. 1–6.
  * [58] S. Saxena _et al._ , “A 2.8 mW/Gb/s, 14 Gb/s serial link transceiver,” _IEEE Journal of Solid-State Circuits_ , vol. 52, no. 5, pp. 1399–1411, 2017\.
  * [59] B. Xu _et al._ , “A 23-mW 24-GS/s 6-bit voltage-time hybrid time-interleaved ADC in 28-nm CMOS,” _IEEE Journal of Solid-State Circuits_ , vol. 52, no. 4, pp. 1091–1100, 2017.
  * [60] I. Melamed _et al._ , “A 30 GHz 4.2 mW 105 fsec jitter sub-sampling PLL with 1° phase shift resolution in 65 nm cmos,” in _2022 IEEE 22nd Topical Meeting on Silicon Monolithic Integrated Circuits in RF Systems (SiRF)_. IEEE, 2022, pp. 45–48.
  * [61] C. Sun _et al._ , “DSENT-a tool connecting emerging photonics with electronics for opto-electronic networks-on-chip modeling,” in _2012 IEEE/ACM Sixth International Symposium on Networks-on-Chip_. IEEE, 2012, pp. 201–210.
  * [62] D. D. Sharma _et al._ , “Universal chiplet interconnect express (UCIe)®: An open industry standard for innovations with chiplets at package level,” _IEEE Transactions on Components, Packaging and Manufacturing Technology_ , 2022.
  * [63] B. Zimmer _et al._ , “A 0.11 pJ/Op, 0.32-128 TOPS, Scalable Multi-Chip-Module-based Deep Neural Network Accelerator with Ground-Reference Signaling in 16nm,” _IEEE Symposium on VLSI Circuits, Digest of Technical Papers_ , vol. 2019-June, pp. C300–C301, 2019.
  * [64] R. Balasubramonian _et al._ , “Cacti 7: New tools for interconnect exploration in innovative off-chip memories,” _ACM Transactions on Architecture and Code Optimization (TACO)_ , vol. 14, no. 2, 2017.
  * [65] F. Gutierrez _et al._ , “On-chip integrated antenna structures in CMOS for 60 GHz WPAN systems,” _IEEE Journal on Selected Areas in Communications_ , vol. 27, no. 8, pp. 1367–1378, 2009.


# Mobility Edge in the Anderson model on partially disordered random regular
graphs

O. Valba  Department of Applied Mathematics, MIEM, National Research
University Higher School of Economics, Moscow, Russia<EMAIL_ADDRESS>A.
Gorsky  Kharkevich Institute for Information Transmission Problems RAS,
Moscow, Russia  Moscow Institute for Physics and Technology, Dolgoprudny
141700, Russia

###### Abstract

In this Letter we study numerically the Anderson model on partially disordered
random regular graphs (RRG) considered as the toy model for a Hilbert space of
interacting disordered many-body system. The protected subsector of zero-
energy states in a many-body system corresponds to clean nodes in RRG
ensemble. Using adjacent gap ratio statistics and IPR we find the sharp
mobility edge in the spectrum of one-particle Anderson model above some
critical density of clean nodes. Its position in the spectrum is almost
independent on the disorder strength. The possible application of our result
for the controversial issue of mobility edge in the many-body localized (MBL)
phase is discussed.

##  1 Introduction

Recently new mechanisms of ergodicity breaking in the complicated interacting
many-body systems have been uncovered. The combination of interaction and
strong enough disorder amounts to the emergent MBL phase with full ergodicity
breaking [1, 2, 3, 4, 5]. The Anderson model on RRG serves as the toy model
for a identification of MBL phase in the physical space, see [6] for the
recent review. The many-body localization in the physical space presumably
gets mapped into the one-particle localization in a Hilbert space [7]. The
localization in the Anderson model on the Bethe tree at some critical disorder
has been established long time ago [8] and more recently the similar
localization on the RRG has been derived analytically [9] and numerically.
This issue has been analyzed in some details in [10, 12, 13, rrg4, 14, 15, 16,
17] and for flat disorder $W$ the critical value has been established in RRG
for degree $p=3$ with a high accuracy $W_{cr}=18.17$.

The mechanisms for the destruction of localization are interesting both from
the theoretical and practical viewpoints. In the context of the destruction of
the MBL phase the main mechanism considered involves a inclusion of thermal
bubbles [18] which presumably yield the many-body mobility edge (MBME) in
interacting many-body systems via the avalanche mechanism. This issue is quite
controversial and there are discussions concerning the very existence of MBME
[19, 20, 21, 22, 18]. The density of special states influences the
correlations in corresponding Hilbert space hopping model [23] and the
relation of special states with the MBME was discussed if the energy [24] or
the density [19] are considered as the control parameters. It was conjectured
that the density of thermal bubbles is related to fractal dimension of the
localized part of the spectrum [25].

Another quite generic mechanism concerns the effects of the special
topologically protected degenerated modes. For instance if we assume that
there is no diagonal disorder at all and disorder is present only in the off-
diagonal hopping terms there is the single delocalized mode at zero energy
which provides the conductivity [26, 27]. Among the real systems enjoying such
mechanism one could mention the disordered integer QHE in 2d when almost all
modes are localized and a conductivity is solely due to zero energy
topological modes [30, 31]. Similarly the topological modes influence the
localization in 1d quantum wires [28, 29]. In both cases the two-coupling
renormalization group (RG) flow governs the evolution of a system from UV to
IR. One coupling measures the strength of disorder while the second one
measures the amount of topological modes estimated in some manner. The RG flow
diagram is quite nontrivial and a delocalized regime emerges near the
boundaries between phases.

Recently the effect of zero modes has been discussed for the finite spin
systems [32, 33, 34, 35] in the context of MBL. It was argued that several
systems host such exponentially degenerated symmetry protected modes. In this
Letter we shall discuss the combined effect of interaction, disorder and zero-
modes on localization in a real space via the particular toy model in the
Hilbert space. We shall mimic the presence of the symmetry protected
degenerated zero-modes in a real space by the clean degenerated nodes in RRG
ensemble. In such toy model localization in the interacting many-body system
is described via localization in the Anderson model on partially disordered
RRG. The density of the clean zero-energy nodes $\beta$ in a Hilbert space is
the second parameter which quantified the ”topological” sector of Hilbert
space. It turns out that there is the critical value of a disorder density
$\beta_{cr}$ and at $\beta<\beta_{cr}$ the one-particle mobility edge in the
Anderson model on the partially disordered RRG can be identified
unambiguously. It will be confirmed numerically by the evaluation of the
r-value statistics and IPR for the corresponding eigenmodes.

We conjecture that a one-particle mobility edge in the Hilbert space hopping
model we found is related with the MBME in a real space hence providing the
arguments that zero-mode degenerated states play the important role in the
MBME formation in disordered system. At strong disorder the finite fraction of
the protected zero modes produces the delocalized states in MBL phase in the
center of energy band and finally forms stable MBME at large enough density of
protected states.

##  2 Model

We study non-interacting spinless fermions hopping over RRG with connectivity
$p=3$ in a potential disorder described by Hamiltonian

$H=\sum_{\left<i,j\right>}\left(c_{i}^{+}c_{j}+c_{i}c_{j}^{+}\right)+\sum_{i=1}^{\beta
N}\epsilon_{i}c_{i}^{+}c_{i},$ (1)

where the first sum runs over the nearest-neighbor sites of the RRG, the
second sum runs over $\beta N$ nodes with potential disorder. The energies
$\epsilon_{i}$ are independent random variables sampled from a uniform
distribution on $\left[-W/2,W/2\right]$. We consider gaps between adjacent
levels, $\delta_{i}=E_{i+1}-E_{i}$, where the eigenvalues of a given
realization of the Hamiltonian for a given total number of particles, $E_{i}$,
are listed in ascending order. The dimensionless quantity we have chosen to
characterize the correlations between adjacent gaps in the spectrum is the
ratio of two consecutive gaps:

$r_{i}=\frac{\min(\delta_{i},\delta_{i+1})}{\max(\delta_{i},\delta_{i+1})}.$
(2)

For uncorrelated Poisson spectrum the probability distribution of this ratio
$r$ satisfy by $P(r)=2/(1+r)^{2}$ with the mean value $\left<r\right>_{P}=2\ln
2-1\approx 0.386$. For large Gaussian orthogonal ensemble the mean value
$\left<r\right>_{GOE}=\approx 0.53$.

In turn, a direct measure of the (de)localization of the eigenfunctions is
obtained by the inverse participation ratio (IPR),
$IPR(i)=\sum_{n}|\psi_{n}^{(i)}|^{4}$, where $\psi_{n}^{(i)}$ is the $i$-th
eigenstate of the matrix and $n$ is the basis state index. The eigenstate
properties are characterized according to its limiting values as

$\lim_{N\to\infty}IPR(i)\propto\begin{cases}1/N,&\text{delocalization}\\\
const,&\text{localization}\end{cases}.$ (3)

The IPR of a chaotic eigenstate is $N$-dependent, in contrast to a localized
one.

##  3 Numerical results

We study numerically the dependence of the single-particle localization on
$\beta$ and $W$ using for diagnostics the adjacent gap ratio statistics and
IPR.

Fig. 1 shows the numerical results for RRG with $N=16000$ and the degree $p=3$
and different disorder parameter $\beta$: the left column corresponds to
$\beta=0.5$, the right column corresponds to $\beta=1.0$. We analyze the ratio
$\left<r\right>$ and IPR for different parts of the spectrum. We divide the
sorting spectrum into $k=100$ equal parts and average the ratio
$\left<r\right>$ and IPR over each window. The ordinate $\alpha=i/(N-1)$ in
Fig. 1 corresponds the normalized level position with $i=0,1,\dots,N-1,$ the
energy level, the ordinate window respectively is $\Delta\alpha=1/k$. Fig. 1
(a), (b) demonstrate the ratio $\left<r\right>$ in dependence on the disorder
value $W$ and the spectrum part $\alpha$; (c), (d) presents the results for
the values $\log(IPR)$; (e), (f) are respective dependencies of $\log(IPR)$ on
the spectrum position $\alpha$ for different values of $W$.

Figure 1: The ratio $\left<r\right>$ in dependence on the disorder value $W$
and the spectrum part $\alpha$ for $\beta=0.5$ (a) and $\beta=1$ (b);the value
$\log(IPR)$ in dependence on the disorder value $W$ and the spectrum part
$\alpha$ for $\beta=0.5$ (c) and $\beta=1$ (d); the dependencies of
$\log(IPR)$ on the spectrum position $\alpha$ for different values of $W$ and
$\beta=0.5$ (e) and $\beta=1$ (f).

The heat maps Fig. 1 (a), (c) explicitly demonstrate, that there is the
mobility edge $\lambda_{m}$ separating sharply the spectrum into two different
regimes for RRG with partial disorder in vertices. For $|\lambda|>\lambda_{m}$
we observe localisation state with the ratio $\left<r\right>$ close to
$\left<r\right>_{P}$ and independence of IPR on N, while for central spectrum
part with $|\lambda|\leq\lambda_{m}$ the ratio $\left<r\right>$ and IPP
indicate on the delocalized state. Note, that the mobility edge $\lambda_{m}$
weakly depends on the disorder $W$ and is observed even for small $W$.
Moreover, we do not observe the phase transition at large $W$ to completely
localized phase which is familiar for completely disordered RRG (see Fig. 1
(b), (d), (f) with the same plots for $\beta=1.0$).

Fig. 2 demonstrate the numerical results for fixed disorder parameter:the left
column corresponds to subcritical disorder parameter $W<W_{cr}$ while the
right column corresponds to supercritical value $W>W_{cr}$. In subcritical
regime there are the mobility edges for all values $\beta$, while in
supercritical regime we observe the existence of the mobility edge just for
$\beta<\beta_{cr}$ with $\beta_{cr}=0.8$, while for $\beta>\beta_{cr}$ the
dependencies correspond the localization state as for $\beta=1.0$.

Figure 2: The ratio $\left<r\right>$ in dependence on the disorder parameter
$\beta$ and the spectrum part $\alpha$ for $W=10.0$ (a) and $W=25.0$ (b);the
value $\log(IPR)$ in dependence on the disorder parameter $\beta$ and the
spectrum part $\alpha$ for $W=10.0$ (c) and $W=25.0$ (d); the dependencies of
$\log(IPR)$ on the spectrum position $\alpha$ for different values of $\beta$
and $W=10.0$ (e) and $W=25.0$ (f).

We analyze how eigenfunctions are distributed on network vertices. We
calculate squared $i$-th eigenvector component averaging for the vertices with
disorder, i.e, $\left<\psi^{2}(i)\right>=\frac{1}{\beta N}\sum_{n\in
V_{d}}|\psi_{n}^{(i)}|^{2}$ and for the zero potential vertices:
$\left<\psi^{2}(i)\right>=\frac{1}{(1-\beta)N}\sum_{n\in
V_{0}}|\psi_{n}^{(i)}|^{2}$,where we denote $V_{d}$ and $V_{0}$ the vertices
with disorder and with zero potential. Fig. 3 shows the numerical results for
the disorder parameter$\beta=0.5$, we plot the value
$\left<\psi^{2}\right>(\alpha)$, calculated as the average in each of $k$
windows. Let us emphasize once again that the graphs clearly demonstrate the
existence of mobility edge, regardless of the disorder level. In addition, we
remark that in the localized phase, the eigenfunctions are distributed mainly
at the vertices with disorder, while in the delocalized phase, mainly at the
clean vertices.

Figure 3: The average squared eigenfunction for disordered and zero potential
vertices in dependence on the spectrum part $\alpha$ for $W=10.0$ (a) and
$W=25.0$ (b).

##  4 On the mobility edge in the MBL phase

In this study we elaborate the combined effect of interaction,disorder and the
protected states for the many-body system in the toy model of partially
disordered RRG. Let us recall the relation between the MBL in the real space
and one-particle localization in the Hilbert space [7]. The nodes in the graph
correspond to the states in Fock space in the theory without interaction. The
number of nodes equals to the dimension of the Hilbert space and, for
instance, for L spins equals to $2^{L}$. The matrix elements of the
interaction yield the links between nodes. It was conjectured that the one-
particle localization in a Hilbert space corresponds to the MBL phase in the
physical space.

This attractive idea works well qualitatively and provides the explanations
for the several phenomena. For instance, there are strong finite-size effects
shifting the apparent transition point, exponential growth of the correlation
volume when approaching the transition point, ergodicity of delocalized phase.
On the other hand there are clear-cut arguments which explain that the
relation between the localization in physical and Hilbert space can not be
exact. For example the tree-like local structure of RRG does not fit
completely the Hilbert space of the many-body system.

Having all these reservations in mind one could ask the question if the one-
particle mobility edge in the Hilbert space we have found corresponds to a
kind of MBME or another physically meaningful phenomenon in the real space.

To answer this question recall the recent scenario concerning the physics in
the vicinity of the mobility edge in MBL. It was suggested in [18] that there
are local regions -”thermal bubbles” with energy density above the mobility
edge which destroy the localization in a whole system via a resonant
tunneling. On the other hand it was suggested in [19] that MBME is stable with
respect to the local regions with high particle density which provide the
thermal bubbles from [18] and no spreading via the tunneling was observed. The
expansion of such bubbles into the whole space was argued to be more probable
origin of delocalization.

Our study suggests that there can be another scenario of MBME formation.
Instead of thermal bubbles the zero-modes could do the job and provide the
conductivity in the mid of the spectrum. We do not expect that zero-modes are
completely protected under the disorder and interaction. More relevant
scenario involves the formation of non-vanishing density of the quasi-zero
modes in the mid of the spectrum somewhat similar to the situation in QCD
although one has to have in mind a lot of reservations concerning this
analogy. Indeed we have found numerically that the states in the delocalized
mid part of the spectrum involve not only the clean modes but also the
disorder dependent admixture of the dirty modes. Therefore they propagate
mainly along the clean nodes with some disorder dependent contribution of
regions with dirty states.

In QCD the localization properties of the Dirac operator eigenfunctions in 4d
Euclidean space-time are considered. In the confinement phase the physical
many-body spectrum of QCD involves the composite mesons and baryons only. It
is well-known that the density of quasi-zero modes $\rho(0)$ of the Dirac
operator yields the breakdown of the chiral symmetry in the confinement phase
via formation of the chiral condensate according to the Casher-Banks relation
$<\bar{\Psi}\Psi>=V^{-1}\pi\rho(0)$, where V is the 4-volume. The Dirac
operator enjoys the discrete symmetry $[H_{D}^{4d},\gamma_{5}]=0$ hence its
spectrum is symmetric under $E\rightarrow-E$. All eigenmodes of the Dirac
operator are delocalized in confined phase [36] which usually is explained via
the instanton liquid model for QCD ground state. The delocalization is
attributed to the overlap of the topological fermionic zero modes localized on
the instantons and antiinstantons. Moreover the lattice studies indicate that
a delocalization occurs not in full 4D Euclidean space but only along some
lower dimensional sub-manifolds somewhat similar to the fracton picture.

When we increase the temperature the deconfinement phase transition occurs at
some critical $T_{c}$. At $T>T_{c}$ the spectrum of QCD involves the
interacting quarks and gluons. Remarkably exactly at $T=T_{c}$ the mobility
edge in the Dirac operator spectrum gets emerged [36]. It can be related to
the formation of black hole horizon in the holographic approach [37]. In the
deconfined phase two essential phenomena happen. First, the disordered
condensate of the Polyakov lines gets formed providing the strong enough
disorder. Secondly the density of the topological defects - instantons gets
decreased due to the asymptotic freedom since it is proportional to
$\exp(-\frac{1}{g_{YM}^{2}(T)})$. Hence indeed like in our model we have three
characteristics which influence a localization: the strength of interaction,
the density of topological defects and the strength of disorder. Like in our
model when the topological defects in the QCD confinement phase dominate we
have complete delocalization in one-particle spectrum while if the disorder
becomes strong enough the mobility edge gets emerged in the deconfined phase .

##  5 Discussion

In this Letter we consider partially disordered RRG as the toy model of a
Hilbert space for some interacting disordered many-body system with the
topologically protected subsector. The nodes of RRG free from disorder
correspond to zero-energy protected states in many-body system. To some extend
our model probes the effects of disorder on zero-mode states.

It is found that at some density of clean nodes in partially disordered RRG
the sharp mobility edge emerges in the spectrum of Anderson model and exists
up to arbitrarily large diagonal flat disorder $W$. We have studied the
distribution of the eigenfunctions in RRG and have found that localized states
are distributed almost solely within the dirty nodes while the delocalized
part of the spectrum mainly involves the clean nodes with small disorder
dependent contribution of the dirty nodes.

The model certainly oversimplifies the issue nevertheless it can be considered
as the indication that a one-particle mobility edge in the hopping model in
the Hilbert space and a mobility edge in the MBL phase could be related.
Indeed if the mechanism behind the mobility edge in MBL involves density of
highly degenerate zero-modes in the physical space then the density of clean
nodes in the Hilbert space is its relevant counterpart. It would be important
to get the analytic description of the observed phenomena using approach
developed in [9, 38].

The mobility edge emerges in RRG ensemble also at another occasion when a
chemical potential for the 3-cycles is added. At some critical value of the
chemical potential the RRG with node degree $p$ gets fragmented into the
$\frac{N}{p}$ clusters which are almost complete graphs [39]. The
corresponding spectral density develops the second non-perturbative band
filled by eigenvalue instantons corresponding to clusters. The emerging
mobility edge separates the continuum filled by the delocalized modes and the
second non-perturbative band with localized modes obeying Poisson statistics
[40]. The formation of clusters is suppressed by factors $e^{-N}$ however they
do not ”evaporate” completely at large $N$ yielding the star-like hubs [41].
The similar fragmentation of the RRG takes place also if the chemical
potential for 4-cycles is added [42, 43]. In this case the emerging
bipartiteness of the fragments is new peculiar feature of the model [43].

It is natural to compare this pattern with other mechanisms of Hilbert space
fragmentation which influence the localization, in particular quantum many-
body scars (QMBS), see [47, 46, 48]. The scars are related to the symmetries
of the many-body Hamiltonian and are protected algebraically when the
interaction does not ruin the symmetry completely [44, 45, 49, 50, 51]. Note
that QMBS are not degenerated enough to fill the finite part of the Hilbert
space. It is assumed that scars are quantum counterpart of the peculiar
unstable semi-classical orbits in classically chaotic systems. Somewhat
similarly the k-cycles in the Hilbert space which induce fragmentation of RRG
presumably correspond to the k- resonanses which also are quantum counterpart
of the peculiar semi-classical orbits [52]. It would be interesting to discuss
their possible relations. It would be also interesting to combine the effects
of a diagonal disorder and chemical potentials for k-cycles in RRG
representation of the Hilbert space together. It could be the toy model for a
interplay of scars with disorder discussed in [53].

We are grateful to A. Kamenev and I. Khaymovich for the useful discussions.
The work was supported by the Russian Science Foundation grant 21-11-00215.

## References

  * 1. D. Basko, I. Aleiner, B. Altshuler, ”Metal–insulator transition in a weakly interacting many-electron system with localized single-particle states”, Annals Of Physics 321 (5) (2006) 1126–1205.
  * 2. I. Gornyi, A. Mirlin, D. Polyakov, ”Interacting electrons in disordered wires: Anderson localization and low-T transport”, Physical Review Letters 95 (20) (2005) 206603.
  * 3. R. Nandkishore, D. A. Huse, ”Many-body localization and thermalization in quantum statistical mechanics”, Annu. Rev. Condens. Matter Phys. 6 (1) (2015) 15–38.
  * 4. F. Alet, N. Laflorencie, ”Many-body localization: An introduction and selected topics”, C. R. Phys. 19 (2018) 498.
  * 5. D. A. Abanin, E. Altman, I. Bloch, M. Serbyn, ”Colloquium: Many-body localization, thermalization, and entanglement”, Reviews of Modern Physics 91 (2) (2019) 021001.
  * 6. K.S. Tikhonov, A.D. Mirlin, ”From Anderson localization on Random Regular Graphs to Many-Body localization” arXiv:2102.05930
  * 7. B. L. Altshuler, Y. Gefen, A. Kamenev, L. S. Levitov, ”Quasiparticle lifetime in a finite system: A nonperturbative approach”, Physical Review Letters 78 (14) (1997) 2803.
  * 8. R. Abou-Chacra, D. Thouless, P. Anderson, ”A selfconsistent theory of localization”, Journal of Physics C: Solid State Physics 6 (10) (1973) 1734
  * 9. A. D. Mirlin, Y. V. Fyodorov, ”Localization transition in the Anderson model on the Bethe lattice: spontaneous symmetry breaking and correlation functions”, Nucl. Phys. B 366 (3) (1991) 507–532.
  * 10. K. Tikhonov, A. Mirlin, Fractality of wave functions on a cayley tree: Difference between tree and locally treelike graph without boundary, Phys. Rev. B 94 (18) (2016) 184203.
  * 11. A. De Luca, B. Altshuler, V. Kravtsov, A. Scardicchio,” Anderson localization on the Bethe lattice: nonergodicity of extended states”, Physical Review Letters 113 (4) (2014) 046806.
  * 12. K. Tikhonov, A. Mirlin, M. Skvortsov, ”Anderson localization and ergodicity on random regular graphs”, Phys. Rev. B 94 (22) (2016) 220203.
  * 13. I. Garcıa-Mata, O. Giraud, B. Georgeot, J. Martin, R. Dubertrand, G. Lemarie, ”Scaling theory of the Anderson transition in random graphs: ergodicity and universality”, Physical Review Letters 118 (16) (2017) 166801.
  * 14. G. Biroli, M. Tarzia, ”Delocalized glassy dynamics and many-body localization”, Phys. Rev. B 96 (20) (2017) 201114\.
  * 15. V. Kravtsov, B. Altshuler, L. Ioffe, ”Non-ergodic delocalized phase in Anderson model on Bethe lattice and regular graph”, Annals of Physics 389 (2018) 148–191.
  * 16. G. Biroli, M. Tarzia, ”Delocalization and ergodicity of the Anderson model on Bethe lattices”, arXiv:1810.07545 (2018).
  * 17. S. Bera, G. De Tomasi, I. M. Khaymovich, A. Scardicchio, ”Return probability for the Anderson model on the random regular graph”, Phys. Rev. B 98 (2018) 134205.
  * 18. W. De Roeck, F. Huveneers, M. Muller, and M. Schiulaz, “Absence of many-body mobility edges,” Physical Review B 93, 1–14 (2016), arXiv:1506.01505.
T. Thiery, F. Huveneers, M. Muller, W. De Roeck, ”Many-body delocalization as
a quantum avalanche”, Phys. Rev.Lett. 121 (2018) 140601.

  * 19. P. Brighi, D. A. Abanin, and M. Serbyn ”Stability of mobility edges in disordered interacting systems” Phys. Rev. B 102, 060202(R) ,2020
  * 20. T. Kohlert, S. Scherg, X. Li, H. P. Luschen, S. Das Sarma, I. Bloch, and M. Aidelsburger, “Observation of many-body localization in a one-dimensional system with a single-particle mobility edge,” Phys. Rev. Lett. 122, 170403 (2019).
  * 21. R. Yao, and J. Zakrzewski, “Many-body localization in Bose-Hubbard model: evidence for the mobility edge ,” arXiv e-prints , arXiv:2002.00381 (2020),
  * 22. D. J. Luitz, N. Laflorencie, and F. Alet, “Many-body localization edge in the random-field heisenberg chain,” Phys. Rev. B 91, 081103 (2015).
  * 23. G. De Tomasi, I. M. Khaymovich, Fr. Pollmann, S. Warzel, ”Rare thermal bubbles at the many-body localization transition from the Fock space point of view” (2020) [arXiv:2011.03048].
  * 24. Roy, S.; Logan, D. ”Fock-space anatomy of eigenstates across the many-body localization transition” 2021PhRvB.104q4201R
  * 25. M. Tarzia ”Many-body localization transition in Hilbert space” Phys. Rev. B 102, 014208 (2020)
  * 26. F. J. Dyson, ”The dynamics of a disordered linear chain”, Phys. Rev. 92, 1331 (1953).
  * 27. G. Theodorou and M. H. Cohen, ”Extended states in a one-demensional system with off-diagonal disorder” Phys. Rev. B 13, 4597 (1976)
  * 28. T. Morimoto, A. Furusaki, C. Mudry ”Anderson localization and the topology of classifying spaces” Phys. Rev. B 91, 235111 (2015)
  * 29. A. Altland, D. Bagrets, A. Kamenev ”Topology versus Anderson localization: Nonperturbative solutions in one dimension” Phys. Rev. B 91, 085429 (2015)
  * 30. D. Khmelnitskii, ”Quantization of Hall conductivity”, JETP Lett 38 (552, (1983)
  * 31. A. Pruisken ”On localization in the theory of the quantized hall effect: A two-dimensional realization of the $\theta$-vacuum”, Nucl.Phys. B 235, 277 (1984)
  * 32. M. Schecter and T. Iadecola, ”Many-body spectral reflection symmetry and protected infinite-temperature degeneracy”,(2019) Phys. Rev. B 98, 035139
  * 33. V. Karle, M. Serbyn, and A. A. Michailidis, “Area-law entangled eigenstates from nullspaces of local hamiltonians,” Phys. Rev. Lett. 127, 060602 (2021), arXiv:2102.13633 [quant-ph].
  * 34. C. J. Turner, A. A. Michailidis, D. A. Abanin, M. Serbyn, and Z. Papic, ”Quantum scarred eigenstates in a rydberg atom chain: Entanglement, breakdown of thermalization, and stability to perturbations”, Phys. Rev. B 98, 155134 (2018).
  * 35. C.-J. Lin and O. I. Motrunich, ”Exact quantum many-body scar states in the rydberg-blockaded atom chain”, Phys. Rev. Lett. 122, 173401 (2019)
  * 36. L. Ujfalusi, M. Giordano, F. Pittler, T. G. Kovacs and I. Varga, “Anderson transition and multifractals in the spectrum of the Dirac operator of Quantum Chromodynamics at high temperature,” Phys. Rev. D 92, no. 9, 094513 (2015) arXiv:1507.02162.
M. Giordano and T. G. Kovacs, Localization of Dirac Fermions in Finite-
Temperature Gauge Theory, Universe 7, no.6, 194 (2021)
doi:10.3390/universe7060194.

  * 37. A. Gorsky and M. Litvinov ”Metal or Insulator? Dirac operator spectrum in holographic QCD” Phys. Lett. B 795, 379-385 (2019),arXiv:1812.02321
  * 38. F. L. Metz, G. Parisi, L. Leuzzi, ”Finite-size corrections to the spectrum of regular random graphs: An analytical solution”, Phys. Rev. E 90 (2014) 052109.
  * 39. V. Avetisov, M. Hovhannisyan, A. Gorsky, S. Nechaev, M. Tamm and O. Valba, ”Eigenvalue tunneling and decay of quenched random network”, Phys. Rev. E 94 (2016) 062313 [arXiv:1607.03871].
  * 40. V. Avetisov, A. Gorsky, S. Nechaev and O. Valba, ”Localization and non-ergodicity in clustered random networks”, Journal of Complex Networks 8 (2) (2020) [DOI], arXiv:1611.08531
  * 41. A. Gorsky and O. Valba, ”Finite-size effects in exponential random graphs”, Journal of Complex Networks 8 (1) (2020) [DOI].
  * 42. C. Kelly, C. Trugenberger and F. Biancalana, ”Self-Assembly of Geometric Space from Random Graphs”, Classical and Quantum Gravity 36 (12) (2019). 125012 [DOI].
  * 43. A. Gorsky, O. Valba, ”Interacting Thermofield Doubles and Critical Behavior in Random Regular Graphs” Phys. Rev. D 103, 106013 (2021)
  * 44. N. Shiraishi and T. Mori, “Systematic construction of counterexamples to the eigenstate thermalization hypothesis,” Phys. Rev. Lett. 119, 030601 (2017).
  * 45. C. J. Turner, A. A. Michailidis, D. A. Abanin, M. Serbyn, and Z. Papic, “Weak ergodicity breaking from quantum many-body scars,” Nature Physics 14, 745–749 (2018).
  * 46. M. Serbyn, D. A. Abanin, Z. Papić ”Quantum Many-Body Scars and Weak Breaking of Ergodicity” arXiv:2011.09486,Nature Physics, 17, 675–685 (2021)
  * 47. S. Moudgalya, B. A. Bernevig and N. Regnault ”Quantum Many-Body Scars and Hilbert Space Fragmentation: A Review of Exact Results” arxiv:2109.00548
  * 48. Z.Papic, ”Weak ergodicity breaking through the lens of quantum entanglement”, arXiv:2108. 03460
  * 49. K. Pakrouski, P. N. Pallegar, F. K. Popov, and I. R. Klebanov, “Many-body scars as a group invariant sector of hilbert space,” Phys. Rev. Lett. 125, 230602 (2020).
”Group theoretic approach to many-body scar states in fermionic lattice
models” K. Pakrouski, P. N. Pallegar, F. K. Popov, I. R. Klebanov
arXiv:2106.10300

  * 50. J. Ren, C. Liang, and C. Fang, “Quasi-symmetry groups and many-body scar dynamics,” arXiv:2007.10380 .
  * 51. N. O’Dea, F. Burnell, A. Chandran, and V. Khemani, “From tunnels to towers: quantum scars from Lie Algebras and q-deformed Lie Algebras,” (2020), arXiv:2007.16207
  * 52. D. Basko ”Weak chaos in the disordered nonlinear Schroedinger chain: destruction of Anderson localization by Arnold diffusion”, Annals of Physics 326 (2011) 1577 [DOI].
  * 53. I. Mondragon-Shem, M. G. Vavilov , I. Martin ”Fate of Quantum Many-Body Scars in the Presence of Disorder” PRX QUANTUM 2, 030349 (2021)