sicite_

# Experimentally realized memristive memory augmented neural network

Ruibin Mao  Department of Electrical and Electronic Engineering, The
University of Hong Kong, Hong Kong SAR, China  Bo Wen  Department of
Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong
SAR, China  Yahui Zhao  Department of Electrical and Electronic Engineering,
The University of Hong Kong, Hong Kong SAR, China  Arman Kazemi  Hewlett
Packard Labs, Hewlett Packard Enterprise, Milpitas, CA, USA  Department of
Computer Science and Engineering, University of Notre Dame, Notre Dame, IN,
USA  Ann Franchesca Laguna  Department of Computer Science and Engineering,
University of Notre Dame, Notre Dame, IN, USA  Michael Neimier  Department of
Computer Science and Engineering, University of Notre Dame, Notre Dame, IN,
USA  X. Sharon Hu  Department of Computer Science and Engineering, University
of Notre Dame, Notre Dame, IN, USA  Xia Sheng  Hewlett Packard Labs, Hewlett
Packard Enterprise, Milpitas, CA, USA  Catherine E. Graves  Hewlett Packard
Labs, Hewlett Packard Enterprise, Milpitas, CA, USA<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>John Paul Strachan  Peter
Grünberg Institut (PGI-14), Forschungszentrum Jülich GmbH, Jülich, Germany
RWTH Aachen University, Aachen, Germany<EMAIL_ADDRESS>j.strachan@fz-
juelich.de<EMAIL_ADDRESS>Can Li  Department of Electrical and
Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Abstract

Lifelong on-device learning is a key challenge for machine intelligence, and
this requires learning from few, often single, samples. Memory augmented
neural network has been proposed to achieve the goal, but the memory module
has to be stored in an off-chip memory due to its size. Therefore the
practical use has been heavily limited. Previous works on emerging memory-
based implementation have difficulties in scaling up because different modules
with various structures are difficult to integrate on the same chip and the
small sense margin of the content addressable memory for the memory module
heavily limited the degree of mismatch calculation. In this work, we implement
the entire memory augmented neural network architecture in a fully integrated
memristive crossbar platform and achieve an accuracy that closely matches
standard software on digital hardware for the Omniglot dataset. The successful
demonstration is supported by implementing new functions in crossbars in
addition to widely reported matrix multiplications. For example, the locality-
sensitive hashing operation is implemented in crossbar arrays by exploiting
the intrinsic stochasticity of memristor devices. Besides, the content-
addressable memory module is realized in crossbars, which also supports the
degree of mismatches. Simulations based on experimentally validated models
show such an implementation can be efficiently scaled up for one-shot learning
on the Mini-ImageNet dataset. The successful demonstration paves the way for
practical on-device lifelong learning and opens possibilities for novel
attention-based algorithms not possible in conventional hardware.

## Introduction

Deep neural networks (DNN) have achieved massive success in data-intensive
applications but fail to tackle tasks with a limited number of examples. On
the other hand, our biological brain can learn patterns from rare classes at a
rapid pace, which could relate to the fact that we can recall information from
an associative, or content-based addressing, memory. Inspired by our brain,
recent machine learning models, such as memory augmented neural networks
(MANN)[1], have adopted a similar concept, where explicit external memories
are applied to store and retrieve the learned knowledge. While those models
have shown the ability to generalize from rare cases, they are struggled to
scale up[2, 3]. This is because, the entire external memory module needs to be
accessed from the memory to recall the learned knowledge, which immensely
increases the memory overhead. The performance is thus bottlenecked by the
memory bandwidth and capacity issues[4, 5, 6] in hardware with the traditional
von-Neumann computing architecuture[7], especially when they are deployed in
edge devices, where the energy sources are limited.

Emerging non-volatile memories, e.g., memristors[8], have been proposed and
demonstrated to solve the bandwidth and memory capacity issues in various
computing workloads, including deep neural networks[9, 10, 11, 12, 13, 14],
signal processing[15, 16], scientific computing[17, 18], solving optimization
problems[19, 20], and more. Those solutions are based on the memristor’s
ability to directly process analog signals at the location where the
information is stored. Most existing demonstrations mentioned above, however,
mainly focus on executing matrix multiplications for accelerating deep neural
networks with crossbar structures[21, 8, 9, 10, 11, 17, 18, 12], whose
experience cannot be directly applied to the models with explicit external
memories in MANNs. Recently, several pioneering works aim to solve the problem
with memristor-based hardware. One promising solution is to exploit the
hyperdimensional computing paradigm [22]. But the high dimension vectors, as
required to maintain the quasi-orthogonal nature, lead to excessive memory
requirements. The recent prototype of this framework uses a large number of
phase-change memory devices to solve the important yet simple Omniglot
problem, the most commonly used handwritten dataset for few-shot image
classification.[23]. Although a close to software-equivalent accuracy was
achieved, the number of devices required by the dimension of the binary
feature vectors will increase significantly for more complex problems.
Ferroelectric device based ternary content addressable memory (TCAM) has been
proposed to be used as the hardware to calculate the similarity directly in
the memory[24, 25], but it is only suitable for degree of mismatch up to a few
bits. Besides, the locality sensitive hashing (LSH) function that enables the
degree of mismatch search in the TCAM was implemented in software, and the
experimental demonstration was limited to a 2$\times$2 TCAM array. More
recently, a 2T-2R TCAM associative memory was used to demonstrate few-shot
learning by calculating $L_{1}$ distance [26]. In this work a 2-bit readout
scheme is employed (requires 64 cycles per row) which incurs high energy and
latency overheads, and feature extraction is again relegated to a digital
processor. The experimental demonstration of the entire MANN concept still
remains a high risk, since the imperfections in such analog hardware, such as
device variation, fluctuation, state drift, and readout noise during the
parallel matrix multiplication operations, have challenged the feasibility of
the promising hardware.

In this work, we experimentally demonstrate one- and few-shot learning with
the entire MANN fully implemented in our integrated memristor hardware. To
achieve this goal, we implement different functionalities in crossbars that
are different from the widely reported matrix multiplication operations. One
enabler is the locality-sensitive hashing (LSH) function in crossbars by fully
exploiting the intrinsic stochasticity of memristor devices. This is different
from crossbars for matrix multiplications, where the stochasticity needs to be
minimized. Another innovation is implementing the search function and using
the crossbar as a ternary content addressable memory (TCAM). In addition to
what’s possible with conventional TCAMs, the proposed scheme can also measure
the degree of mismatch reliably, which is crucial for few-shot learning
implementation. Since the requirements for those functions are different from
conventional matrix multiplications, here we introduce several hardware-
software co-optimization methods, including the introduction of the wildcard
‘X’ bit in the crossbar-based LSH, and the careful choice of conductance range
according to the device statistics.

Finally, with a fully integrated memristor computing system, we are able to
experimentally demonstrate the few-shot learning with a complete MANN model,
including a five-layer convolutional neural network (CNN), the hashing
function, and the similarity searches. Taking into consideration all
imperfections in the emerging system, our hardware achieves 94.9% $\pm$ 3.7%
accuracy in the 5-way 1-shot task with the Omniglot dataset[27], a popular
benchmark for few-shot image classification, and 74.9% $\pm$ 2.4% accuracy in
25-way 1-shot task, which is close to the software baseline (95.2% $\pm$ 2.6%
for 5-way 1-shot and 76.0% $\pm$ 2.7% for 25-way 1-shot). Our experimentally-
validated model also shows that the proposed method is scalable with a 58.7 %
accuracy to recognize rare cases (5-way 1-shot) for the Mini-ImageNet
dataset[28], where each image is a color (RGB) image of size 84$\times$84
pixels which is nine times larger than the size (28$\times$28) of images in
the Omniglot dataset. This accuracy is only 1.3% below the software baseline.
Owing to the in-memory analog processing capability and the massive
parallelism, our hardware achieves more than 4,500$\times$ improvement in
latency and 2, 900$\times$ improvement in energy consumption on crossbar
arrays as compared to the general-purpose graphic processing unit (GPGPU)
(Nvidia Tesla P100).

## Memory augmented neural networks in crossbars

The MANN architecture commonly includes a controller, an explicit memory, and
a content-based attention mechanism between the two. A controller is usually a
traditional neural network structure such as a convolutional neural network, a
recurrent neural network (RNN), or the combination of different neural
networks. The explicit memory stores the extracted feature vectors as the key-
value pairs so that the model can identify the values based on the similarity
or distances between the keys. The access of the explicit memory is the
performance bottleneck for models that run on conventional hardware, such as
the general-purpose graphic processing unit (GPGPU). It is because the
similarity search requires accessing all the content in the memory; thus the
repeated data transfer process delays the readout process and consumes
abundant energy, especially when the memory needs to be placed in a separate
chip.

We implement the entire network in our memristor-based hardware system to
address the expensive data transfer issue in conventional digital hardware.
The system aims to perform both matrix multiplication, for neural network
controllers, and the similarity search, for the explicit memory, directly in
the memristor crossbars, where the data are stored. Memristors have
demonstrated great success in accelerating matrix multiplications in neural
networks, which is also the controller part of the MANN, owing to their
extreme density[29], the compatibility with CMOS integration[30, 9], and the
capability to process analog information directly in its memory. But, the
demonstrations for explicit memories are still limited to small-scale
experiments[24, 25] or simulated multiplication-and-add with readout
conductances[23].

Fig. 1a illustrates how we implement the MANN in the crossbars. First, a
regular crossbar-based convolutional neural network is used to extract the
real-valued feature vector, and the method implementing this step has been
widely reported previously[11, 31, 32]. After that, distances are calculated
between the extracted feature vector and those stored in a memory. Cosine
distance (CD) is one of the most widely used distance metrics in the explicit
memory of various MANN implementations, but it is not straightforward to
implement with memristor-based crossbars. On the other hand, the cosine
distance between two real-valued vectors can be well approximated by the
Hamming distance (HD) using locality sensitive hashing (LSH) codes of the two
vectors[33, 24, 34]. Accordingly, in this work, instead of being stored in a
dynamic random access memory (DRAM) for future distance calculations, the
features are hashed into binary/ternary signatures in a crossbar with randomly
distributed conductance at each crosspoint exploiting the stochasticity of
memristor devices. Those signatures are then searched against those previously
stored in another crossbar that acts as a content-addressable memory enabled
by a newly proposed coding method, from which we can also calculate the degree
of mismatch that approximates the cosine distance of the original real-valued
vector.

The idea is experimentally demonstrated in our integrated memristor chip. One
of the tiled $64\times 64$ memristor crossbars in our integrated chip that we
used to experimentally implement the network is shown in Fig. 1b. The
peripheral control circuits, including the driving, sensing, analog-to-digital
conversions, and the access transistors, are implemented with a commercial 180
nm technology integrated chip (Fig. 1c). 50 nm$\times$50 nm Ta/TaO_x/Pt
memristors are integrated with back-end-of-the-line (BEOL) processing on top
of the control peripheral circuits (1d). The fabrication details, the device
characteristics and peripheral circuit designs were reported previously
elsewhere[35, 36], and the picture of our test chip and platform is shown in
Supplementary Fig. 6.

Figure 1: Memory augmented neural networks in crossbar arrays. a, The
schematic of a crossbar-based MANN architecture. The expensive data transfer
in a von Neumann architecture can be alleviated by performing analog matrix
multiplication, locality sensitive hashing, and nearest neihgbor searching
directly in memristor crossbars, where the data is stored. b, Optical image of
a 64$\times$64 crossbar array in a fully integrated memristor chip. c, Cross-
section of the memristor chip, where complementary metal-oxide-silicon (CMOS)
circuits at the bottom, inter-connection in the middle, and metal vias on the
surface for memristor integration with back-end processes. d, Top view of four
50 nm$\times$50 nm integrated cross-point memristors. Animal figures in a are
taken from www.flaticon.com

## Locality sensitive hashing in crossbar array

We first introduce the idea of performing locality sensitive hashing (LSH)
directly in crossbars employing intrinsic stochasticity, and validate that the
experimental implementation in memristor crossbars can approximate cosine
distance. LSH [37, 38, 39] is a hashing scheme that generates the same hashing
bits with a high probability for the inputs that are close to each other. One
of the hashing functions among the LSH family is implemented by random signed
projections, i.e. , applying a set of hyperplanes to divide the Hilbert space
into multiple parts such that similar inputs project to the same partition
with a high probability (Fig. 2a). This random projection is mathematically
expressed by a dot product of the input vector $\vec{a}$ and a random normal
vector $\vec{n}$, so that ‘1’ is generated if $\vec{a}\cdot\vec{n}>0$, and ‘0’
otherwise. Accordingly, LSH bits can be calculated by Equation 1 in a matrix
form,

$\vec{h}=H\left(\vec{a}~{}\mathbf{N}\right)$ (1)

where the $\vec{h}$ is the binarized hashing vector, $\vec{a}$ the input real-
valued feature vector, $\mathbf{N}$ the random projection matrix with each
column a random vector, and $H$ the Heaviside step function.

The random projection matrix can be constructed physically by exploiting the
stochastic programming dynamics of the memristor devices or the initial
randomness after the fabrication (Fig. 2b). But it is still challenging to
generate random vectors $\vec{n}$ with a zero mean value, as required by the
LSH algorithm, because the conductance of the memristor device can only be
positive values. Our solution is to take the difference between devices in the
adjacent columns[40] in the crossbar array. The devices from the columns,
assuming no interference in between, are independent of each other. Therefore,
the distribution of the conductance difference will also be uncorrelated and
random. In this way, the random normal vector $\vec{n}$ in the original
equation can be represented by the difference of two conductance vectors, i.e.
$\vec{n}=\left(\vec{g^{+}}-\vec{g^{-}}\right)/k$. The zero mean value of the
vector $\vec{n}$ is guaranteed as long as the distribution of the memristor
conductance vectors ($\vec{g^{+}},\vec{g^{-}}$) have the same mean value. $k$
is a scaling factor, which can be set as an arbitrary value because we only
need to determine if the output is larger than zero or otherwise.

Here, we experimentally program all devices in a crossbar array to the same
target conductance state. The programming process of memristor devices is
stochastic[9] as the thinnest part of the conductive filament can be only a
few atoms wide[41]. Accordingly, the final conductance values follow a random
distribution with the mean roughly matching the target conductance. To lower
the output current and thus the energy consumption, we reset all devices close
to the highest resistance state that we can achieve ($17\text{\,}\mathrm{nS}$
at the read voltage of $0.2\text{\,}\mathrm{V}$) from arbitrary initial states
using a few pulses (see Methods for details). After programming, as expected,
most devices are programmed to a conductance state near the highest resistance
state (Fig. 2c), and the difference between devices from adjacent columns
follows a random distribution with a zero mean value (Fig. 2d, e). Hashing
bits for an input feature vector are generated efficiently by performing
multiplications with the randomly configured memristor crossbar array. After
converting the real-valued input vector into the analog voltage vector, the
dot product operations are conducted by applying the voltage vectors to the
row wires of the crossbar and reading the current from the column wires. Thus,
the hashing operation is completed by comparing the current amplitude from the
adjacent columns (Fig. 2b) in one step, regardless of the vector dimension.

Imperfections in emerging memristive devices, such as conductance relaxation
and fluctuation, limit experimental performance. This is mainly because the
device conductance fluctuations incur instability of hashing planes
implemented as adjacent column pairs in crossbar arrays. This causes hashing
bits for input vectors that are close to hyperplanes to flip between 0 and 1
over time (Supplementary Fig. 9) and therefore leads to an inaccurate
approximation of the cosine distance. It should be emphasized that this
problem only becomes apparent when performing matrix multiplication in arrays,
rather than multiplying with the readout conductances reported in other
works[23].

To mitigate bit-flipping, we propose a software-hardware co-optimized ternary
locality sensitive hashing scheme (TLSH). The scheme introduces a wildcard ‘X’
value to the hashing bits (Fig. 2a), in addition to ‘0’ and ‘1’ in the
original hashing scheme. As the name implies, the Hamming distance between the
wildcard ‘X’ and any values will always be zero. The ‘X’ is generated when the
absolute value of the output current difference is smaller than a threshold
value, i.e. $I_{\text{th}}$. The value for $I_{\text{th}}$ is chosen to be
small and close to the transient analog computing error from the crossbar,
such that any bit-flipping minimally impacts the similarity search.

We validate the proposed approach by conducting experiments in our integrated
memristor crossbars. The hash outputs for 500 64-dimensional real-valued
random vectors are computed in our memristor crossbars for binary and ternary
hashing vectors. The $I_{\text{th}}$ representing the threshold of the ‘X’
wildcard is set to $4\text{\,}\mathrm{\SIUnitSymbolMicro A}$ in the ternary
hashing implementation (TLSH). Fig. 2f shows that the cosine distance is
closely correlated with the Hamming distance between the hashed vectors with
128 hashing bits in total, regardless of whether the hashing codes are
generated by a 32-bit floating-point digital processor (“software LSH” in Fig.
2f), the analog crossbar (“hardware LSH”), or the proposed co-designed ternary
hashing codes by the crossbar (“hardware TLSH”). Note that the Hamming
distance of the ternary hashing codes is smaller than that of the binary codes
because the distance to a wild card ‘X’ is always zero. The effectiveness of
the method is evaluated quantitatively by the linear correlation coefficient,
as shown in Fig. 2g. The result shows that the proposed ternary hashing
narrows the already small gap between the digital software approach and our
analog hardware approach. The performance improvement results from
significantly reduced unstable bits, which is experimentally demonstrated by
the comparison shown in Supplementary Fig. 9. The results demonstrate that
crossbar arrays, utilizing the proposed ternary scheme, can effectively and
efficiently perform hashing operations, taking advantage of intrinsic
stochasticity and massively parallel in-memory computing.

Figure 2: Robust ternary locality sensitive hashing in analog memristive
crossbars. a, Illustration of the Locality Sensitive Hashing (LSH) and the
Ternary Locality Sensitive Hashing (TLSH) concept. b, The LSH or TLSH
implemented in memristor crossbars. Each adjacent column pair represents one
hashing plane. So, crossbars with $N+1$ columns can generate $N$ hashing bits
with this method. Greyscale colors on the memristor symbol represent random
conductance states. c, A random memristor conductance distribution in a
64$\times$129 crossbar after applying five RESET pulses to each device. The
intrinsic stochastic behavior in memristor devices results in a lognormal-like
distribution near $0\text{\,}\mathrm{\SIUnitSymbolMicro S}$. d, The
distribution of the memristor conductance difference for devices in adjacent
columns. The differential conductance distribution is random with zero-mean,
matching the requirements of our hashing scheme. e, The conductance difference
map of size 64$\times$128 (including three crossbar arrays each of size
64$\times$64). f, The correlation between cosine distance and Hamming distance
with different hashing representations shows that the Hamming distance
generated by both hardware and software can well approximate the cosine
distance. IQR, interquartile range. g, The linear correlation coefficient
between Hamming distance and cosine distance increases with the number of
total hashing bits. The hardware TLSH approach shows a higher correlation
coefficient than the hardware LSH approach due to the reduced number of
unstable bits, as detailed in Supplementary Fig. 9.

## TCAM in crossbars with ability to output degree of mismatches

Following the LSH step, the binary or ternary hashing signatures will be
searched against the hashed signatures previously stored in a memory to
calculate the similarity and thus find the $k$-closest match. As mentioned
earlier, this is an extremely time- and energy-consuming step on conventional
hardware such as GPUs. Content addressable memories (CAM) or the ternary
version (TCAM) are direct hardware approaches that can find the exact match in
the memory in one step. Still, existing static random access memory (SRAM)
based CAM/TCAM implementations limit the available memory capacity and incur
high power consumption. CAMs/TCAMs based on non-volatile memories have been
developed recently, including those based on memristor/ReRAM (e.g. 2T-2R[42,
26], 2.5T-1R[43]), floating gate transistor (e.g. 2Flash[44]), ferroelectric
transistors (e.g. 2FeFET[24, 25]), etc. Although these studies demonstrated
good energy efficiency, they are limited to at most a few bits mismatches
which have difficulties to serve as attentional memory modules for scaled-up
MANNs[24, 25, 26].

We implement the TCAM functionality directly in an analog crossbar with the
additional ability to output the degree of mismatch based on the Hamming
distance, rather than only a binary match/mismatch. In contrast to
conventional TCAM implementations which sense a mismatch by a dis-charged
match-line, our crossbar-based TCAM searches through a simple encoding and a
set of dot product operations computed in the output currents. Fig. 3a shows a
schematic on how this scheme works. First, the query signature is encoded to
use a pair of voltage inputs for 1-ternary-bit, so that one column wire is
driven to a high voltage (i.e. Vsearch), while the other is grounded. The
corresponding memristor conductances that store previous signatures are
encoded with one device set to a high conductance state (i.e. , Gon) and the
other to the lowest conductance state (i.e. , Goff $\approx 0$). In this way,
for a ‘match’ case, the high voltage will be applied to the device in the low
conductance state, and therefore, very little current is added to the row
wires. In a ‘mismatch’ case, the high voltage applied to the device in the
high conductance state will contribute Vsearch$\times$Gon to the output
current of the column wires. The wildcard ‘X’ in the ternary implementation
will be naturally encoded as two low voltages as input or two low conductance
devices so that they contribute zero or very little current and thus always
yield ‘match’. In this way, the degree of mismatch, the Hamming distance,
between the query signature and all words stored in the crossbar is computed
in a constant time-step by sensing the column currents from the crossbar (see
Fig. 3b).

We have experimentally implemented the above TCAM for measuring Hamming
distance in memristive crossbars. First, eight different binary signatures,
each has eight bits but different number of ‘1’s (from one ‘1’ to eight ‘1’s),
are encoded into conductance values as shown in Fig. 3c. The conductance
values are then programmed to a crossbar with an iterative write-and-verify
method (see ref.[35] and Methods for details), with the readout conductance
matrix after successful programming shown in Fig. 3b. We choose
$150\text{\,}\mathrm{\SIUnitSymbolMicro S}$ as the Gon for a higher on/off
conductance ratio and minimal relaxation drift (Supplementary Fig. 7). Fig. 3d
shows both the distribution of Gon and Goff after programming.

After configuring the memory to store the previously generated signatures, 100
ternary signature vectors as queries are randomly chosen and the corresponding
encoded voltages are applied to the column wires of the crossbar (Fig. 3c), to
perform the search operation. The search voltage (Vsearch) is chosen to be
$0.2\text{\,}\mathrm{V}$ in this work, so each mismatched bit will contribute
approximately $30\text{\,}\mathrm{\SIUnitSymbolMicro A}$
(=$0.2\text{\,}\mathrm{V}$$\times$$150\text{\,}\mathrm{\SIUnitSymbolMicro S}$)
to the output current. In the experiment, however, results are deteriorated by
non-ideal factors. For example, the memristor in a low conductance state still
contributes a small current (Vsearch$\times$Goff$\neq$0) in a match case,
imperfect programming of Gon results in deviations in output current for each
‘mismatch’ case, etc. Our device exhibits a large enough on/off ratio[35], but
for devices with a lower conductance on/off ratio, such as MRAM[45] , the
problem would be more significant. For such cases, we propose a 3-bit encoding
that is discussed in detail in Supplementary Fig. 10. Additionally, non-zero
wire resistances cause voltage drop along wires, lowering the output current
from what would be expected ideally. Despite these factors, the output current
in our experiments exhibits a linear dependence on the number of mismatch
bits, i.e. ternary Hamming distance (Fig. 3e). Fig. 3f shows separated
distributions where each distribution represents a distinct number of mismatch
bits ranging from 0 to 8. We have thus experimentally demonstrated a robust
capability to store patterns, search patterns, and obtain the degree of
mismatch which will enable determining the closest match that is stored in an
array by simply comparing output currents.

Figure 3: TCAM implemented in crossbar array capable of conducting Hamming
distance calculation. a, Illustration of the basic principle for using dot
product to distinguish ‘match’ and ‘mismatch’ cases. b, The schematic of
calculating Hamming distance in a crossbar. The figure shows three
3-dimensional ternary key vectors stored in a $3\times 6$ crossbar with a
differential encoding. Differential voltages representing ternary bits in
search vectors are applied to the source line and the output current from the
bit line can represent the THD between the search vector and keys stored in
the memory. c, The readout conductance map after eight binary vectors
experimentally stored in the crossbar as memory. In the experiment, we set Gon
as $150\text{\,}\mathrm{\SIUnitSymbolMicro S}$ and Vsearch as
$0.2\text{\,}\mathrm{V}$. d, Distribution of Gon and Goff. e, Ouput current
shows a linear relation with Hamming distance measuring the degree of
mismatches. IQR, interquartile range. f, Current distributions are separated
from each other through which we can obtain the number of mismatch bits (i.e.
, Hamming distance).

## One- and few-shot learning experiment fully implemented in memristor
hardware

We implemented a complete MANN demonstrating one-shot and few-shot learning in
crossbars. To evaluate and compare the performance of our method, we chose the
Omniglot dataset[27], a commonly used benchmark for few-shot learning tasks.
In this dataset, there are 1,623 different handwritten characters (classes)
from various alphabets. Each character contains 20 handwritten samples from
different people. Samples from 964 randomly chosen classes are used to train
the CNN controller and the remaining 659 are used for one-shot and few-shot
learning experiments. In an $N$-way $K$-shot learning experiment, the model
should be able to learn to classify new samples from $N$ different characters
(classes) after being shown $K$ handwritten images from each character
(support set). The accuracy is evaluated by classifying an unseen sample
(query set) after learning from the limited number of samples (only one sample
each for the 1-shot problem) from each class.

Figure 4:  End-to-end experimental inference with memristive crossbar arrays.
a, Schematic of CNN structure implemented in the memristor crossbar array. The
conductance shows the weight mapping of CNN kernels. The format of dimension
representations in the figure follows the Output channel (O), Input channel
(I), Height (H), and Width (W). The conductance maps representing the whole
CNN kernels are shown in Supplementary Fig. 14. b, Linear relationship between
the sensing current from the crossbar-based TCAM and the number of mismatch
bits during the search operations. c, Classification accuracy with cosine
similarity, software-based LSH with 128 bits, and end-to-end experimental
results on crossbar arrays. We provide 5 experimental data points for each
task. Software LSH shows experimental variation due to different
initializations of the hashing planes in each experiment. d, e, Simulations of
classification accuracy of 5-way 1-shot problem (d) and 25-way 1-shot problem
(e) as a function of device fluctuations in the memristor model for both TLSH
and LSH. Fluctuations from nearly zero to
$1\text{\,}\mathrm{\SIUnitSymbolMicro S}$ are shown. The actual experimental
fluctuation level is shown with an arrow.

In our experiment, the memristor CNN controller first extracts the feature
vector from an image. Note that the CNN weights don’t need to be updated after
the meta-training process which is done in software offline. The CNN consists
of four convolutional layers, two max-pooling layers, and one fully connected
layer (Fig. 4a). There are nearly 65,000 weights in convolutional layers
altogether that are represented by 130,000 memristors, with the conductance
difference of two memristors representing one weight value. The weights of
convolutional layers are flattened and concatenated first (see Supplementary
Fig. 13 and Methods for details) and then programmed to crossbar arrays with
an iterative write-and-verify method. Limited by the available array size, we
divide larger matrices into 64$\times$64 tiles and reprogram the same arrays
when needed to accomplish all convolutional operations in the crossbar.
Experimental conductance maps (36 matrices of conductance values) for the CNN
layers after each programming of an array are shown in Supplementary Fig. 14.
The repeated programming of memristor arrays demonstrated good reliability of
the memristor devices within crossbars. After programming the convolutional
weights into the crossbar, the fully connected layer is retrained to adapt to
the hardware defects.

Before the few-shot learning, the explicit memory stored in the crossbar-based
TCAM is initialized with all ‘0’s. During the few-shot learning, the feature
vectors computed by the memristor CNN next hashed into 128 binary or ternary
signatures and searched against the entries in the crossbar-based TCAM for the
closest match, with the methods described before. The label of the closest
match will be the classification result. If correct, the nearest neighbor
entry will be updated based on the new input query vector (see Methods).
Otherwise, the signature along with the label is written to a new location in
the TCAM using differential encoding. After learning $K$ images from the
support set, The conductance map that is stored in the crossbar-based TCAM is
shown in Supplementary Fig. 15. Note that the CNN controller stays the same
for all four few-shot learning tasks, a key feature to support lifelong
learning, which is different from the previously reported few-shot learnings
based on high-dimensional computing[23]. The memory is the only part need to
be updated during lifelong learning. Even for the memory module, the update is
not frequent (1.3 times per bit for 20-shot) throughout the learning process,
as demonstrated in Supplementary Fig. 16. Considering a common statistical
endurance (about $10^{6}$) for a relative high switching window reported
previously[46, 47], we estimate about 4000-years lifetime of a face
recognition system that sees 10 times each face in one day which is much
longer than the human life.

Accuracy is evaluated experimentally in classifying new samples after few-shot
learning with four standard tasks: 5-way 1-shot, 5-way 5-shot, 25-way 1-shot,
and 25-way 5-shot, respectively. We found that the experimental sensing
currents during few-shot learning experiment are highly linear with the number
of mismatch bits, i.e. hamming distance, as shown in Fig. 4b. This is partly
enabled by the introduced wildcard ‘X’ from our TLSH method, as discussed in
detail in Supplementary Fig. 20 and Supplementary Note 3. The classification
results shown in Fig. 4c demonstrate that for 5-way problems, our full
crossbar hardware-based MANN achieves an accuracy very close to the software
baseline implemented in digital hardware with cosine similarity as the
distance metric. For 25-way problems, we find no difference between results
from our analog hardware and that from the digital hardware implementing the
same LSH plus Hamming distance algorithm. Though there exists some accuracy
drop compared to the cosine baseline, the performance can be improved to match
the baseline accuracy by increasing the number of hashing bits from 128 to 512
(see Supplementary Fig. 11). The experimental results demonstrate that the
MANN fully implemented in crossbar arrays can achieve similar accuracy as
software for this task.

## Device imperfections analysis

The accuracy can be affected by many non-idealities in emerging memory
devices, among which the two most prominent are conductance fluctuations and
relaxation. We noted that the conductance of memristor devices fluctuates up
and down (see Supplementary Fig. 8a, b) even within a very small period (at
the scale of nanoseconds). The fluctuation leads to frequent changes in
convolutional kernels, hyperplane locations in LSH operations, and stored
signature values in TCAMs, which negatively impact the accuracy. The data
(shown in Supplementary Fig. 8c) measured from our integrated array shows that
the degree of device fluctuation increases with the conductance value. This
behavior is consistent with previous reports on single device measurement[35,
48]. In addition to the conductance fluctuations, the programmed value may
also change permanently (relaxation) over time, which is characterized in
detail in Supplementary Fig. 7c. From these results, we find that conductance
relaxation is larger when device conductance is programmed to a certain range
(from around $25\text{\,}\mathrm{\SIUnitSymbolMicro S}$ to
$75\text{\,}\mathrm{\SIUnitSymbolMicro S}$). Therefore, in our implementation,
we try to avoid this range as much as possible to achieve the software
equivalent accuracy. For example, in the LSH part, we chose lower conductance
levels to minimize both conductance fluctuation and relaxation. In the TCAM
part, we chose $0\text{\,}\mathrm{\SIUnitSymbolMicro S}$ and
$150\text{\,}\mathrm{\SIUnitSymbolMicro S}$ as the low and high conductance
levels to minimize the impact of conductance relaxation. In the CNN part, we
observe that most weight values are very small (near zero), so with the
differential encoding method (details described in Methods) we can guarantee
that most memristor conductance values are below the range with higher
relaxation.

To analyze our software-competitive accuracy results and evaluate if our
method is scalable, we built an empirical model describing experimental
conductance-dependent fluctuation behavior and deviation after programming.
With the experimental calibrated model (see Methods and Supplementary Fig. 8c,
d for more details), we can match the simulation results with the experiments
in few-shot learning on Omniglot handwritten images. The detailed comparison
is shown in Supplementary Fig. 17a). The simulation also enables us to analyze
how different device fluctuations impact classification accuracy. We conducted
simulations assuming the device conductance fluctuation spanning from nearly
no fluctuation to $1\text{\,}\mathrm{\SIUnitSymbolMicro S}$ which is about ten
times larger than our device behavior. The results in Fig. 4e, f show that
with the experimental fluctuation value, the accuracy stays almost the same as
the software equivalent value, but the accuracy will sharply drop if the
fluctuation is above three-times larger than our experimental value. The
results also show that our proposed TLSH method exhibits better performance
compared to the conventional LSH, especially for more significant device
fluctuation scenarios (Fig. 4e, f and Supplementary Fig. 18). In addition to
the higher tolerance to the device fluctuation, the comparison shown in
Supplementary Fig. 17b, c also demonstrates the TLSH’s advantages in search
energy. These simulations, with experimental calibration, elucidate the
experimentally observed defect tolerance and software-equivalent accuracy.
Though there exist other defects such as stuck-at-fault, I/V nonlinearities
for high resistance states, and device-to-device variation in active
conductance range, we found these have negligible impact on the final
performance. With this tool, we are able to analyze scaling up to more complex
real-world problems.

## Scaled-up MANN for Mini-ImageNet

The methodology of crossbar-based TLSH and TCAM can be applied in many fields
of deep learning that require the distance calculation and attention
mechanism. To show the scalability of our fully crossbar-based MANN, we
conducted simulations based on our experimentally-calibrated model for one-
shot learning using the Mini-ImageNet dataset[28]. This dataset is derived
from the ImageNet dataset with 100 classes of 600 images of size $84\times 84$
color pixels per class. The task is known to be much more difficult than that
of the Omniglot handwritten dataset. A more sophisticated ResNet-18 model is
used as the controller following the state-of-the-art structure in few-shot
learning models[49], which has more than 11 million weights, 44 times larger
than the controller to classify images from the Omniglot dataset.

A challenge for this network is the required crossbar sizes (512 in one
dimension), and thus the voltage drops along the wire would significantly
reduce the computing accuracy. This is solved by partitioning large arrays
(for hyperplanes and memories) into smaller 256$\times$256 tiled crossbars
(Fig. 5a) to accommodate the model. In the simulation, we consider the
experimental device fluctuations (see Supplementary Table 2) and use the same
threshold current ($4\text{\,}\mathrm{\SIUnitSymbolMicro A}$) for the TLSH
approach as in the smaller Omniglot problem. The result in Fig. 5b shows that
the classification accuracy for the 5-way 1-shot problem increases with the
number of hashing bits, and reaches 58.7% with 4,096 hashing bits, only 1.3%
smaller than the model implemented in digital hardware with cosine similarity
as the distance metric. We also explored the performance with different
partitioned array sizes in Supplementary Fig. 19a, b, which achieves nearly
equivalent performance with arrays smaller than or equal to 256$\times$256 and
drops slightly with the 512$\times$512 array. Encouragingly, the TLSH function
can be implemented with a larger array (512$\times$512) because of lower
conductance and smaller voltage drops along the wires. From these results, we
can see that the performance of our crossbar-based MANN can scale up
effectively to at least Mini-ImageNet problems.

## Discussion

Compared to conventional von Neumann based implementations, the key advantage
of crossbar-based MANN is lower latency and higher energy efficiency through
co-located computing and memory, energy-efficient analog operations, and
intrinsic stochasticity. To evaluate the strength of the approach, we run the
same 5-way 1-shot problem with Omniglot and Mini-ImageNet datasets on a
digital graphic processing unit (GPU) (Nvidia Tesla P100). The time required
to classify a single image increases dramatically after the size of the MANN’s
external memory capacity reaches a certain threshold (only several MB) because
of the repeated off-chip data movement (see Fig. 5c). This problem on
conventional hardware has been the major bottleneck preventing the widespread
adoption of few-shot learning. The approach of directly computing in the
memory, or crossbar, provides a plausible solution to address this bottleneck.
In the crossbar-based MANNs, the matrix multiplication in the convolutional
layer, the hashing in TLSH, and the searching operation in TCAM are all
computed with single-step current readout operations. With our current proof-
of-concept experimental system, readouts take about $100\text{\,}\mathrm{ns}$,
but can be reduced to $10\text{\,}\mathrm{ns}$. We also considered the time
latency for the peripherals ($2.5\text{\,}\mathrm{ns}$) including the full-
adders for tiled crossbar arrays[50]. With these conservative forecasts, we
compared the latency for GPU and our analog in-memory hardware. The results
shown in Fig. 5d indicate a latency improvement (4,540$\times$ and
11,280$\times$) when the memory size (number of entries) is 8,192, for both
Omniglot and Mini-ImageNet datasets. Additionally, our approach also offers
high energy efficiency compared with the conventional GPU (2,857$\times$ and
50,970$\times$). Detailed analysis about energy and latency estimation can be
found in Supplementary Note 3.

Figure 5: Experiment validated simulation results of Mini-ImageNet dataset. a,
The architecture of TLSH and TCAM for the scaled-up MANN. Both matrices for
hashing and external memory are partitioned into tiled memristor crossbar
arrays (H$\times$W) to mitigate the voltage drop problem in large crossbars
and to increase the utilization rate. b, The accuracy performance from our
experiment-validated models on Mini-ImageNet dataset. The error bar shows the
95% confidence interval among 100 repeated inference experiments. c, The
execution time of search operations per inference on a GPU drastically
increases when external memory size reaches a threshold, confirming the
operation is memory intensive. d, The comparison of the search latency and
energy consumption for 5-way 1-shot learning on both the Omniglot and Mini-
ImageNet datasets. For GPU, the models for both datasets stores the same
number of entries (8,192), but Mini-ImageNet uses a larger memory capacity due
to higher dimension (64 vs. 512) of feature vectors, leading to even better
improvement on latency and energy efficiency. The number of hashing bits used
in crossbar arrays is 128 and 4096 for Omniglot and Mini-ImageNet,
respectively.

In summary, we have, for the first time, experimentally implemented a complete
MANN architecture, from the controller to distance calculation, in an analog
in-memory platform with proven high robustness and scalability. We utilize the
analog behavior of memristor devices to perform convolution operations for
CNNs and exploit the inherent stochasticity of devices to perform hashing
functions. A novel hardware-friendly hashing function (TLSH) is developed to
provide better analog computing error tolerance and lower power consumption.
In addition, a differential encoding method for crossbar-based TCAM is applied
to adapt to the ternary Hamming distance calculations requirements. In our
experiment, all dot-product operations are performed in physical crossbars,
which exhibit experimental imperfections, such as device state fluctuations,
device nonlinearities, voltage drops due to wire resistance, and peripheral
circuits. The fully hardware-implemented MANNs delivered similar accuracy
compared to software on few-shot learning with the widely used the Omniglot
dataset. Simulation results on Mini-ImageNet show the ability of crossbar-
based MANN to execute real-world tasks, with much-improved latency and energy
consumption. We demonstrate that analog in-memory computing with memristive
crossbars efficiently supports many different tasks, including convolution,
hashing, and content-based searching. The successful demonstration of these
functions opens possibilities with other machine learning algorithms, such as
attention-based algorithms, or reaching scales that are currently prohibited
by conventional hardware (e.g, Fig. 5c). Additionally, there are many
opportunities for future software-hardware co-optimization to improve the
accuracy and efficiency results further.

###  0.1 Memristor integration

The memristors are monolithically integrated on CMOS fabricated in a
commercial foundry in a $180\text{\,}\mathrm{nm}$ technology node. The
integration starts with the removal of native oxide on the surface metal with
reactive ion etching (RIE) and a buffered oxide etch (BOE) dip. Chromium and
platinum are then sputtered and patterned with e-beam lithography as the
bottom electrode, followed by reactively sputtered $2\text{\,}\mathrm{nm}$
tantalum oxide as the switching layer and sputtered tantalum metal as the top
electrode. The device stack is finalized by sputtered platinum for passivation
and improved electrical conduction.

###  0.2 Iterative write-and-verify programming method

In this work, we use the iterative write-and-verify method to program
memristor devices to the target conductance value. First, we set a target
conductance matrix and the corresponding tolerant programming error range.
After that, successive SET and RESET pulses are applied to the target devices
followed by conductance readout with READ pulses. If the device conductance is
below the target conductance minus the tolerant error, a SET pulse is applied.
A RESET pulse is applied for conductance above the tolerant values, while the
device has been programmed within the tolerant values are skipped to pertain
the state. For the crossbar-based MANN in this work, we apply the write-and-
verify method to map the weights of the CNN controller and memories in the
TCAM structure. During the programming process, we gradually increase the
programming voltage and gate voltage as shown in Supplementary Table 1. The
pulse width for both the SET and RESET process is
$1\text{\,}\mathrm{\SIUnitSymbolMicro s}$. The tolerant range we set is
$5\text{\,}\mathrm{\SIUnitSymbolMicro S}$ above or below the target
conductance value.

###  0.3 Adjacent Column Method

We apply the Adjacent Column Method (ACM)[40] to map the conductance of
memristors in crossbar arrays to weights in hashing planes. ACM subtracts the
neighboring columns as shown in Fig. 2b to generate the hash codes. Hence, for
a crossbar array with N+1 columns, the output of differential encoding
contains N values which immensely saves the area. The mathematical
representation is as follows: Provided that we get a random conductance map
after programming which is:

$G_{map}=\begin{pmatrix}G_{1,1}&G_{1,2}&\cdots&G_{1,N}\\\
G_{2,1}&G_{2,2}&\cdots&G_{2,N}\\\ \vdots&\vdots&\ddots&\vdots\\\
G_{N,1}&G_{N,2}&\cdots&G_{N,N}\end{pmatrix}$ (2)

then the ACM method is equivalent to multiplying $G_{map}$ by a transformation
matrix:

$G_{hash}=G_{map}\times\begin{pmatrix}1&0&\cdots&0&0\\\ -1&1&\cdots&0&0\\\
0&-1&\cdots&0&0\\\ \vdots&\vdots&\ddots&\vdots&\vdots\\\ 0&0&\cdots&-1&1\\\
0&0&\cdots&0&-1\end{pmatrix}=\begin{pmatrix}G_{1,1}-G_{1,2}&G_{1,2}-G_{1,3}&\cdots&G_{1,N-1}-G_{1,N}\\\
G_{2,1}-G_{2,2}&G_{2,2}-G_{2,3}&\cdots&G_{2,N-1}-G_{2,N}\\\
\vdots&\vdots&\ddots&\vdots\\\
G_{N,1}-G_{N,2}&G_{N,2}-G_{N,3}&\cdots&G_{N,N-1}-G_{N,N}\end{pmatrix}$ (3)

###  0.4 Memristor model and simulations

We build a memristor model to simulate the conductance fluctuation, which is
the most dominant non-ideality of our crossbar-based MANN. The behavior of
conductance fluctuation is assumed to be a Gaussian nature which is as
follows:

$G=G_{0}+\sigma\cdot\mathcal{N}(0,1)$ (4)

where $G_{0}$ is the conductance after programming, $\sigma$ describes the
standard deviation of the fluctuation range. In the simulation we assume that
the device only fluctuates for different VMM processes since in the real
experiment the execution time of one VMM is very small
($10\text{\,}\mathrm{ns}$) which is negligible compared to time between
succesive input vectors ($1\text{\,}\mathrm{\SIUnitSymbolMicro s}$). After
considering the device-to-device variations and fitting the parameters (see
Supplementary Table 2) to experimental measurements (see Supplementary Fig.
8), Equation 4 becomes:

$G=G_{0}+\exp(a\cdot\ln(G_{0})+b+s\cdot\mathcal{N}(0,1))\cdot\mathcal{N}(0,1)$
(5)

with $\mathcal{N}(\mu,\sigma^{2})$ being the normal distribution with mean
$\mu$ and standard deviation $\sigma$. In the simulation, we also consider the
program error to the initial conductance $G_{0}$ which is shown as:

$G_{0}=G_{t}+\mathcal{N}(0,\tilde{G_{err}^{2}})$

where $G_{t}$ is the target conductance we want to program to and
$\tilde{G_{err}}$ is the program error which we set to
$5\text{\,}\mathrm{\SIUnitSymbolMicro S}$ in the simulation.

To get the parameters of the memristor model in terms of the effect of device
fluctuation, we _SET_ 4,096 devices to 16 distinct analog states and _READ_
each device for 1,000 times. The relation between the mean value and standard
deviation of 1,000 reads is shown in Supplementary Fig. 8a and 8b. We further
analyze the standard deviation distribution for each conductance state from
$5\text{\,}\mathrm{\SIUnitSymbolMicro S}$ to
$50\text{\,}\mathrm{\SIUnitSymbolMicro S}$, plot the distributions in
logarithmic scale, and fit them with Gaussian distribution. The results are
shown in Supplementary Fig. 8c. The mean value of $s$ for each distribution
gives us the parameter for the model. In addition, we fit a linear curve with
conductance states and standard deviation in a log-log regime of measurements
(see Supplementary Fig. 8d). The fitted parameters $a$ and $b$ are used in the
simulation.

###  0.5 Ternary locality sensitive hashing

Ternary locality-sensitive hashing introduces a wildcard ’X’ to the hashing
vector to alleviate the analog computing error from nonideal factors. We have
demonstrated that this modified hashing scheme can achieve software-equivalent
performance (LSH with the same hashing bits) on our crossbar arrays. The
threshold current $I_{\text{th}}$ applied in the experiment should be
carefully chosen according to the typical value of the computing error caused
by device fluctuation. The value we chose throughout the experiment is
$4\text{\,}\mathrm{\SIUnitSymbolMicro A}$. We also show the dependence of
classification accuracy on different threshold currents in Supplementary Fig.
12.

For the simulation results in Fig. 4d and 4e, where the device fluctuation
varies, we chose different threshold currents $I_{\text{th}}$ according to the
fluctuation levels. Specifically, for our memristor model which can be
described by Equation 4, we empirically set the threshold current to be
$5\sigma\cdot V_{in}$ where $V_{in}$ is the maximum input voltage to the row
line when performing VMM. The $V_{in}$ is chosen to be
$0.2\text{\,}\mathrm{V}$ in our experiment.

To generate random hashing planes in crossbar arrays (Fig. 2c), we _RESET_ the
devices from an arbitrary high conductance state to near
$0\text{\,}\mathrm{\SIUnitSymbolMicro S}$, where the conductance is ultimately
decided by the intrinsic stochastic switching process. Regardless of the
initial states, we use 5 _RESET_ pulses with an amplitude of
$1.5\text{\,}\mathrm{V}$ and a width of $20\text{\,}\mathrm{ns}$. The _RESET_
voltage is carefully controlled to protect memristor devices because larger
voltages may cause devices to be stuck at low conductance states.

###  0.6 CNN architecture

The convolutional neural network (CNN)s in crossbars is applied as the
controller in the MANN to extract features from incoming images.

The CNN structure for the Omniglot dataset is composed of:

  * •

2 convolutional layers, each with 32 channels of shape 3x3

  * •

A 2x2 max-pooling layer

  * •

2 convolutional layers, each with 64 channels of shape 3x3

  * •

A 2x2 max-pooling layer

  * •

A fully connected layer with 64 outputs

Each convolutional layer is followed by a rectified linear unit (ReLU)
activation layer.

The ResNet-18 for the Mini-ImageNet dataset is composed of 8 residual blocks.
Each residual block has two 3x3 convolutional layers with size [layer channel,
input channel, 3, 3] and [layer channel, layer channel, 3, 3] respectively.
Each convolutional layer is followed by a batch normalization layer and a ReLU
layer. The overall architecture for ResNet-18 is:

  * •

1 convolutional layer with 64 channels of shape 3x3

  * •

2 residual blocks with 64 channels and stride 1

  * •

2 residual blocks with 128 channels and stride 2

  * •

2 residual blocks with 256 channels and stride 2

  * •

2 residual blocks with 512 channels and stride 2

  * •

1x1 adaptive average pooling layer

We map the weights of convolitional layers in the CNN to the conductance of
memristor devices using differential encoding[31]. To elaborate, in a
differential column pair, we program the positive weight to the left column
and the absolute value of negative weights to the right column, while keeping
the other at the low conductance state. The weight-to-conductance ratio we set
in our experiment is 1:$50\text{\,}\mathrm{\SIUnitSymbolMicro S}$. The feature
maps collected from the output current in Fig. 4a are converted into voltages
and then sent to another crossbar array corresponding to the subsequent
convolutional layer. The fully connected layer is retrained after mapping
convolutional layers on crossbar arrays and it’s computed in the digital
domain.

###  0.7 Memory update rules

In an $N$-way $K$-shot learning, the memory module is updated based on the
$N\times K$ images in the support set. If the label of the new input image
label doesn’t match the label of the nearest neighbor entry, we simply find a
new location in the memory and write the input image to that location.
Conversely, if the input image label matches, we need to update the memory of
the nearest neighbor. In the GPU, we assign cosine distance as the metric to
identify the label of input images and update the real-valued vectors at the
same location[33]. However, in this work, we use ternary Hamming distance as
our metric, and we apply the following rules to update the ternary vectors in
the external memory: We introduce a scoring vector to evaluate the majority of
’1’ and ’0’ for each bit of each memory vector. An element-wise mapping
function is applied to each ternary vector stored in the memory module:

$f:\\{1,0,X\\}^{D}\rightarrow\\{1,-1,0\\}^{D}$

where $D$ is the dimension of storing vectors. For example, vector
$(1,0,1,X,0)$ is mapped to $(1,-1,1,0,-1)$. We assume the scoring vector for
each storing vector as: $s_{i}=f(a_{i}),i=1,2,3,\ldots,M$, where $s_{i}$ is
the scoring vector, $a_{i}$ is the hashing vector stored in TCAM and $M$ is
the total number of memory entries. When there is a match case happening at
memory location $k$, we first update its scoring vector as below:

$s_{k}^{*}=s_{k}\oplus f(v)$ (6)

where $s_{k}^{*}$ is the new scoring vector, $v$ is the hashing vector of the
new image, $\oplus$ is element-wise add operation. Then we update the memory
at the same location using the following rules:

$a_{i}^{*}=L(a_{i});L(x)=\left\\{\begin{array}[]{ll}1&x>0\\\ X&x=0\\\
0&x<0\end{array}\right.$ (7)

where $a_{i}^{*}$ is the updated memory and $L$ is an element-wise operator.
Therefore, bits of the memory stored in TCAM are decided by the majority of
’1’ and ’0’ of incoming vectors which match the storing vectors.

###  0.8 Omniglot training

The Omniglot data set contains 1623 different handwritten characters from 50
different alphabets. Each character was drawn online by 20 different people
using Amazon’s Mechanical Turk. In the experiment, we augment the 964
different characters in the training set to 3856 through rotation. The
character types in the test set remain unchanged at 659. There is no overlap
between the training set and the test set. We use the episode training method
during the training process. Episode training is to select N x M instances
from the training set during each training, where N represents different
classes and M represents the number of instances in each class. The purpose of
episode training is to enable the learned model to focus on the common parts,
ignoring tasks, so as to achieve the purpose of learning to learn. The
specific settings in the training process are as follows: memory size is 2048,
batch size is 16, episode width is 5, episode length is 30; The length of the
output key is 128, and the validation set is used for verification every 20
times.

###  0.9 Mini-ImageNet training

The Mini-ImageNet dataset contains 100 classes randomly chosen from the
ImageNet dataset. We randomly split the dataset into a training set, a
validation set, and a test set containing 64, 16, and 20 classes,
respectively. We take the pre-trained model in ref[49] and fine-tuned it using
cosine distance as the meta-training metric. Once the meta-training process is
done, the weights for the controller won’t be updated. We use the ResNet-18
model as the CNN controller and the output feature vector of the CNN is
512-dimensional.

## Data availability

The data supporting plots within this paper and other findings of this study
are available with reasonable requests made to the corresponding author.

## Code availability

The code used to generate the results of this study is available with
reasonable requests made to the corresponding author.

## References

## References

  * [1] Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D. & Lillicrap, T.  Meta-learning with memory-augmented neural networks.  In Balcan, M. F. & Weinberger, K. Q. (eds.) _Proceedings of The 33rd International Conference on Machine Learning_ , vol. 48 of _Proceedings of Machine Learning Research_ , 1842–1850 (PMLR, New York, New York, USA, 2016).  URL http://proceedings.mlr.press/v48/santoro16.html.
  * [2] Stevens, J. R., Ranjan, A., Das, D., Kaul, B. & Raghunathan, A.  Manna: An accelerator for memory-augmented neural networks.  In _Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture_ , 794–806 (2019).
  * [3] Rae, J. W. _et al._ Scaling memory-augmented neural networks with sparse reads and writes (2016).  1610.09027.
  * [4] Strubell, E., Ganesh, A. & McCallum, A.  Energy and policy considerations for deep learning in NLP.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , 3645–3650 (Association for Computational Linguistics, Florence, Italy, 2019).  URL https://www.aclweb.org/anthology/P19-1355.
  * [5] Li, D., Chen, X., Becchi, M. & Zong, Z.  Evaluating the energy efficiency of deep convolutional neural networks on cpus and gpus.  In _2016 IEEE international conferences on big data and cloud computing (BDCloud), social computing and networking (SocialCom), sustainable computing and communications (SustainCom)(BDCloud-SocialCom-SustainCom)_ , 477–484 (IEEE, 2016).
  * [6] Ranjan, A. _et al._ X-MANN: A Crossbar based Architecture for Memory Augmented Neural Networks 1–6 (2019).
  * [7] Von Neumann, J.  First draft of a report on the edvac.  _IEEE Annals of the History of Computing_ 15, 27–75 (1993).
  * [8] Chua, L.  Memristor-the missing circuit element.  _IEEE Transactions on circuit theory_ 18, 507–519 (1971).
  * [9] Xia, Q. & Yang, J. J.  Memristive crossbar arrays for brain-inspired computing.  _Nature materials_ 18, 309–323 (2019).
  * [10] Li, C. _et al._ Long short-term memory networks in memristor crossbar arrays.  _Nature Machine Intelligence_ 1, 49–57 (2019).
  * [11] Wang, Z. _et al._ In situ training of feed-forward and recurrent convolutional memristor networks.  _Nature Machine Intelligence_ 1, 434–442 (2019).
  * [12] Prezioso, M. _et al._ Training and operation of an integrated neuromorphic network based on metal-oxide memristors.  _Nature_ 521, 61–64 (2015).  1412.0611.
  * [13] Ambrogio, S. _et al._ Equivalent-accuracy accelerated neural-network training using analogue memory.  _Nature_ 558, 60–67 (2018).  URL http://dx.doi.org/10.1038/s41586-018-0180-5.
  * [14] Chen, W. H. _et al._ CMOS-integrated memristive non-volatile computing-in-memory for AI edge processors.  _Nature Electronics_ 2, 420–428 (2019).  URL http://dx.doi.org/10.1038/s41928-019-0288-0.
  * [15] Li, C. _et al._ Analogue signal and image processing with large memristor crossbars.  _Nature electronics_ 1, 52–59 (2018).
  * [16] Sheridan, P. M. _et al._ Sparse coding with memristor networks.  _Nature Nanotechnology_ 12, 784–789 (2017).
  * [17] Le Gallo, M. _et al._ Mixed-precision in-memory computing.  _Nature Electronics_ 1, 246–253 (2018).
  * [18] Zidan, M. A. _et al._ A general memristor-based partial differential equation solver.  _Nature Electronics_ 1, 411–420 (2018).
  * [19] Yang, K. _et al._ Transiently chaotic simulated annealing based on intrinsic nonlinearity of memristors for efficient solution of optimization problems.  _Science Advances_ 6 (2020).
  * [20] Cai, F. _et al._ Power-efficient combinatorial optimization using intrinsic noise in memristor Hopfield neural networks.  _Nature Electronics_ 3, 409–418 (2020).  URL http://dx.doi.org/10.1038/s41928-020-0436-6.
  * [21] Hu, M. _et al._ Dot-product engine for neuromorphic computing: Programming 1t1m crossbar to accelerate matrix-vector multiplication.  In _2016 53nd ACM/EDAC/IEEE Design Automation Conference (DAC)_ , 1–6 (IEEE, 2016).
  * [22] Karunaratne, G. _et al._ In-memory hyperdimensional computing.  _Nature Electronics_ 3 (2020).  URL http://dx.doi.org/10.1038/s41928-020-0410-3.  1906.01548.
  * [23] Karunaratne, G. _et al._ Robust high-dimensional memory-augmented neural networks.  _Nature communications_ 12, 1–12 (2021).
  * [24] Ni, K. _et al._ Ferroelectric ternary content-addressable memory for one-shot learning.  _Nature Electronics_ 2, 521–529 (2019).
  * [25] Laguna, A. F., Yin, X., Reis, D., Niemier, M. & Hu, X. S.  Ferroelectric FET based in-memory computing for few-shot learning.  _Proceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI_ 373–378 (2019).
  * [26] Li, H. _et al._ Sapiens: A 64-kb rram-based non-volatile associative memory for one-shot learning and inference at the edge.  _IEEE Transactions on Electron Devices_ (2021).
  * [27] Lake, B., Salakhutdinov, R., Gross, J. & Tenenbaum, J.  One shot learning of simple visual concepts.  In _Proceedings of the annual meeting of the cognitive science society_ , vol. 33 (2011).
  * [28] Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K. & Wierstra, D.  Matching networks for one shot learning.  _arXiv preprint arXiv:1606.04080_ (2016).
  * [29] Pi, S. _et al._ Memristor crossbar arrays with 6-nm half-pitch and 2-nm critical dimension.  _Nature nanotechnology_ 14, 35–39 (2019).
  * [30] Cai, F. _et al._ A fully integrated reprogrammable memristor–cmos system for efficient multiply–accumulate operations.  _Nature Electronics_ 2, 290–299 (2019).
  * [31] Yao, P. _et al._ Fully hardware-implemented memristor convolutional neural network.  _Nature_ 577, 641–646 (2020).
  * [32] Shafiee, A. _et al._ Isaac: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars.  _ACM SIGARCH Computer Architecture News_ 44, 14–26 (2016).
  * [33] Kaiser, Ł., Nachum, O., Roy, A. & Bengio, S.  Learning to remember rare events.  _arXiv preprint arXiv:1703.03129_ (2017).
  * [34] Datar, M., Immorlica, N., Indyk, P. & Mirrokni, V. S.  Locality-sensitive hashing scheme based on p-stable distributions.  In _Proceedings of the twentieth annual symposium on Computational geometry_ , 253–262 (2004).
  * [35] Sheng, X. _et al._ Low-conductance and multilevel cmos-integrated nanoscale oxide memristors.  _Advanced Electronic Materials_ 5, 1800876 (2019).
  * [36] Li, C. _et al._ CMOS-integrated nanoscale memristive crossbars for CNN and optimization acceleration.  _2020 IEEE International Memory Workshop, IMW 2020 - Proceedings_ 2, 1–4 (2020).
  * [37] Gionis, A., Indyk, P., Motwani, R. _et al._ Similarity search in high dimensions via hashing.  In _Vldb_ , vol. 99, 518–529 (1999).
  * [38] Shinde, R., Goel, A., Gupta, P. & Dutta, D.  Similarity search and locality sensitive hashing using ternary content addressable memories.  In _Proceedings of the 2010 ACM SIGMOD International Conference on Management of data_ , 375–386 (2010).
  * [39] Huang, Q., Feng, J., Zhang, Y., Fang, Q. & Ng, W.  Query-aware locality-sensitive hashing for approximate nearest neighbor search.  _Proceedings of the VLDB Endowment_ 9, 1–12 (2015).
  * [40] Kazemi, A. _et al._ A device non-ideality resilient approach for mapping neural networks to crossbar arrays.  In _2020 57th ACM/IEEE Design Automation Conference (DAC)_ , 1–6 (IEEE, 2020).
  * [41] Niu, G. _et al._ Geometric conductive filament confinement by nanotips for resistive switching of hfo 2-rram devices with high performance.  _Scientific reports_ 6, 1–9 (2016).
  * [42] Li, J., Montoye, R. K., Ishii, M. & Chang, L.  1 mb 0.41 $\mu$m2 2t-2r cell nonvolatile tcam with two-bit encoding and clocked self-referenced sensing.  _IEEE Journal of Solid-State Circuits_ 49, 896–907 (2013).
  * [43] Lin, C.-C. _et al._ 7.4 a 256b-wordlength reram-based tcam with 1ns search-time and 14$\times$ improvement in wordlength-energyefficiency-density product using 2.5 t1r cell.  In _2016 IEEE International Solid-State Circuits Conference (ISSCC)_ , 136–137 (IEEE, 2016).
  * [44] Fedorov, V. V., Abusultan, M. & Khatri, S. P.  An area-efficient ternary cam design using floating gate transistors.  In _2014 IEEE 32nd International Conference on Computer Design (ICCD)_ , 55–60 (IEEE, 2014).
  * [45] Apalkov, D., Dieny, B. & Slaughter, J.  Magnetoresistive random access memory.  _Proceedings of the IEEE_ 104, 1796–1830 (2016).
  * [46] Zhao, M. _et al._ Characterizing endurance degradation of incremental switching in analog rram for neuromorphic systems.  In _2018 IEEE International Electron Devices Meeting (IEDM)_ , 20–2 (IEEE, 2018).
  * [47] Zhao, M. _et al._ Endurance and retention degradation of intermediate levels in filamentary analog rram.  _IEEE Journal of the Electron Devices Society_ 7, 1239–1247 (2019).
  * [48] Ambrogio, S. _et al._ Statistical fluctuations in hfo x resistive-switching memory: part i-set/reset variability.  _IEEE Transactions on electron devices_ 61, 2912–2919 (2014).
  * [49] Wang, Y., Chao, W.-L., Weinberger, K. Q. & van der Maaten, L.  Simpleshot: Revisiting nearest-neighbor classification for few-shot learning.  _arXiv preprint arXiv:1911.04623_ (2019).
  * [50] Ghadiry, M., Nadi, M. & A’Ain, A. K.  Dlpa: Discrepant low pdp 8-bit adder.  _Circuits, Systems, and Signal Processing_ 32, 1–14 (2013).

C.L., C.G., J.P.S contributed to the conception of the idea. R.M. performed
the experiments and analyzed data under the supervision of C.L.. R.M., Y.Z.
and A.K. performed simulations. X.S. integrated the memristors. R.M., Y.H, and
C.L. wrote the manuscript with input from all authors.

The authors declare that they have no competing interests.

Correspondence and requests for materials should be addressed to<EMAIL_ADDRESS>
##  1 Supplementary Figures

Figure 6: CMOS-integrated crossbar test platform and experimental setup. a,
Circuit schematic of a crossbar for matrix multiplication with peripheral
circuits. It consists of a 1T1R crossbar, row muxes, column muxes,
transimpedence amplifiers (TIA), sample and hold (S&H) and ADC. b, Picture of
a wire-bonded integrated memristor chip. It constains three 64$\times$64 1T1R
crossbar arrays. c, Measurement board with the integrated memristor chip that
connects to a general purpose computer through a microcontroller (MCU).
Figure 7: Memristor conductance relaxation statistics in a crossbar array. a,
Cumulative distribution function on 16 distinct programmed conductance states
at different time period. We run the test on a 64x64 array where we divide it
into 4x4 blocks and program them to 16 conductance states with iterative
write-and-verify programming scheme. b, Relation between mean conductance at
different time periods and target conductance. c, Relation between the
standard deviation at different time periods and the target conductance.
Figure 8: Memristor conductance fluctuations in a crossbar array. a, Relation
between the standard deviation of 1000 readout conductance and the mean value
at different conductance states. We test the whole 64x64 array. b, Boxplot of
the standard deviation with respect to different conductance states. c,
Distribution of logarithm of standard deviation at different conductance
levels. The distributions show that the device-to-device variation in terms of
conductance fluctuation exhibits a lognormal distribution. d, Mean value of
the logarithm of standard deviation exhibits a linear dependence on the
conductance state at low conductance range ($<$
$50\text{\,}\mathrm{\SIUnitSymbolMicro S}$) and tends to maintain within a
certain range at higher conductance range. The coefficient of the variation
brought by the read fluctuation decreases as the conductance increases.
Figure 9: Number of unstable bits using hardware TLSH and LSH. Ideally, we
expect no unstable bits when performing hardware LSH so that it’s equal to
software LSH. However, due to many device nonidealities, there will be many
unstable bits mainly caused by the conductance fluctuation. In this work, we
apply TLSH to solve this problem. Here, we use the same set of vectors in the
main text and repeat the hashing process 100 times with the same conductance
map of size $64\times 129$. We then count the number of unstable bits 100
times for both TLSH and LSH. The result shows that by applying TLSH we can
reduce the number of unstable bits, thus increasing the robustness of our
hardware in terms of hashing process. Figure 10: 3-bit encoding method for
TCAM using crossbar array. a, TCAM storage and input encoding method of 3-bit
encoding. b, The schematic of experimental set-up for THD calculation by using
dot product. Three bits are stored in crossbar array using 3-bit encoding.
Search bits are successively applied to the row line and the output current
from column line exhibits the THD between input and memory. c, Eight binary
vectors stored in crossbar as memory after 3-bit encoding. d, 100 different
ternary vectors sent to TCAM to perform THD calculation in a parallel way. e,
Distribution of Gon and Goff. f, Ouput current exhibits a linear dependence on
the THD. IQR, interquartile range. g, Current distributions are separated from
each other through which we can obtain the number of mismatch bits, i.e. ,
THD.  Figure 11: Robustness of hardware TLSH on few-shot learning. a, b, c, d,
To demonstrate the robustness of hardware TLSH which exploits the intrinsic
stochasticity of memristors, we randomly choose 20 different conductance maps
to perform the hashing process. The figure shows the classification accuracy
with hardware TLSH and software LSH based on the experimentally generated
hashing codes. The result exhibits the robustness of our memristor crossbar
array served as hashing vector generator whose behavior is nearly the same as
software LSH. Furthermore, we can obtain the same accuracy as cosine
similarity on few-shot learning tasks if we scale the hashing bits up to 512
which is feasible in the prevailing crossbar architectures.  Figure 12: Impact
of the threshold current for hardware TLSH on the classification accuracy. a,
b, As expected, the influence of the threshold current for hardware TLSH
should be positive first and negative afterward. Because introducing the
wildcard ’X’ in LSH can mitigate the defect of device nonidealities but can
also cast away the information of original input vectors. Here, we
experimentally demonstrate the classification accuracy of 25-way problems with
different threshold currents. The result shows that for our system, the
performance reaches a peak with a threshold current around
$0.4\text{\,}\mathrm{\SIUnitSymbolMicro A}$. The zero thresholds current case
denotes hardware LSH. Each task is repeated 100 times.  Figure 13:
Convolutional layers mapped to crossbar arrays a, A $3\times 3$ convolutional
kernel is flattened first and mapped to conductance using differential pair.
b, N filters with each containing M channels are concatenated together and
mapped to crossbar arrays. Different filters perform the convolution in a
parallel way.  Figure 14: Experimental conductance map of convolutional layers
Figure 15: Memory stored in the crossbar-based TCAM after TLSH operation and
binary update process Figure 16: Update times of the memory stored in
crossbar-based TCAM of multi-shot learning. The majority of the bits are
updated only once which is when writing the newly input vector into the new
location of the memory. Unlike the conventional memory update method which
need to update every value in the real-valued vectors, our proposed binary
update method only updates very few bits in the memory which is suitable for
the life-long learning given the endurance of memristors. a,  Statistical view
of the update times of bits during standard 5-shot learning. b,  To analyze
the life-long learning properties, we perform the multi-shot learning on the
25-way task from 1-shot to 20-shot. It shows that nearly 80% of the bits
stored don’t need to be updated once written to store the new input vector.
There are only smaller than 5% of bits that need to be updated higher than 3
times throughout the 20-shot learning.  Figure 17: Simulation results on
Omniglot dataset accord with experimental results a, Classification accuracy
of derived memristor device model, along with cosine similarity, software-
based LSH with 128 bits, and end-to-end experimental results on crossbar
arrays. b, Comparison over LSH and TLSH with both simulation and experiment on
few shot learning tasks. c, Average energy consumption of each TCAM search
operation with TLSH and LSH, respectively.  Figure 18: Impact of device
variation on classification accuracy. a, b The experimental device-to-device
variation is shown in Supplementary Fig. 8c. In simulation, we increase the
variation to explore the influence of it on classification accuracy. The
results show that for both 5-way 1-shot and 25-way 1-shot problem, the
classification accuracy drops by 10% when conductance variation is about 50%
larger than the experiemntal data. The value of conductance variation is the
value of parameter $s$ in Supplementary Table 2 Figure 19: Performance of
scaled-up MANNs with different array size a, TLSH performance with different
array size. The sneak path problem has no influence on the TLSH part because
most of the devices are at the OFF state and the hyperplanes are random. b,
Classification accuracy with different array size. The performance degradation
is found with a bigger array size and a small number of hashing bits. This is
because the sneak path problem affects the performance of TCAM, resulting in
miscalculating of Hamming distances. However, the accuracy remains nearly the
same for different partitioned array sizes when the number of bits is large.

##  2 Supplementary Tables

Table 1: Iterative write-and-verify method Program parameter | V1/V | V2/V |  $\Delta$V/V
---|---|---|---
SET voltage | 1.0 | 2.5 | 0.1
RESET voltage | 0.5 | 3.5 | 0.05
SET gate voltage | 1.0 | 2.0 | 0.1
RESET gate voltage | 5.0 | 5.5 | 0.1
Table 2: Derived values of memristor model parameters Parameter | Description | Value
---|---|---
$G_{0}$ | Initial conductance after program | -
$a$ | Linear fitting parameter | 0.782
$b$ | Linear fitting parameter | -2.168
$s$ | Device-to-device variation | 0.983

##  3 Supplementary Notes

Sensing margin in crossbar-based TCAM

The sensing margin between the match and mismatch cases in conventional TCAM
measures the reliability of the hardware under extreme conditions. For TCAM
circuits based on emerging memory devices, a higher sensing margin also
provides better tolerance to device variation and thus a smaller bit error
rate.

In this work, our crossbar-based TCAM does not only distinguish between match
and mismatch cases but also returns the number of mismatches. For few-shot
learning with MANN, what is required is distinguishing between the closest
match and the next close match. Here we describe the sensing margin $\beta$
for our crossbar-based TCAM in detecting the closest match in Equation 8.

$\beta=\frac{I_{m+1}-I_{m}}{I_{m}}=\frac{I_{m+1}}{I_{m}}-1$ (8)

where $I_{m}$ is the largest possible current (or voltage after the sensing
amplifier) of the closest match, and $I_{m+1}$ is the smallest possible
current (or voltage after the sensing amplifier) of the next closest match. A
higher sensing margin $\beta$ indicates a fewer error in detecting the closest
match, which is a function of a variety of parameters, including the memristor
device ON/OFF conductance ratio $r=G_{\text{ON}}/G_{\text{OFF}}$, word length
(the width of the array) $N$, the number of mismatches of the closest match
$M$, the number of wildcard ‘X’ in query vector/memory entry $K$, etc. The
following analyses aim to reveal the trade-off relationship among the
parameters. The effect of device variation and wire resistance is not taken
into consideration for simplicity.

  * •

Case 1

We start with the simplest case, where there is no wild card ’X’ in the query
vector, and we want to detect the exact match between the query and keys with
a word length $N$. The sense margin can be written as:

$\displaystyle\beta=\frac{I_{m+1}}{I_{m}}-1=\frac{G_{on}V_{s}+(N-1)G_{off}V_{s}}{NG_{off}V_{s}}-1=\frac{r-1}{N}$
(9)

where $V_{s}$ is the search voltage, and other parameters are defined
previously.

One finds that the sensing margin increases with memristor ON/OFF ratio $r$
and decreases with the word length $N$. On the other hand, if we want the
sense margin to be at least $\beta_{0}$, the word length needs to be smaller
than $\frac{r-1}{\beta_{0}}$. For example, if the ON/OFF ratio ($r$) is 100,
and the sense margin should be at least 0.5, the maximum word length
$N_{\text{max}}$ is limited to $\frac{100-1}{0.5}=198$.

  * •

Case 2

Instead of detecting the exact match, here in this case study, we detect the
closest match (with $M$ mismatch bits). Wildcard ‘X’ is also ignored in this
case. The sense margin can be written as:

$\displaystyle\beta=\frac{I_{m+1}}{I_{m}}-1$
$\displaystyle=\frac{(M+1)G_{on}V_{s}+(N-M-1)G_{off}V_{s}}{MG_{on}V_{s}+(N-M)G_{off}V_{s}}-1$
(10) $\displaystyle=\frac{(M+1)r+N-M-1}{Mr+N-M}-1$ (11)
$\displaystyle=1/\left(M+\frac{N}{r-1}\right)$ (12)

In addition to the trade-offs revealed in Case 1, one also finds that the
sensing margin $\beta$ also decreases with the number of mismatch bits $M$.
So, fewer number of mismatch bits is preferred for a given application.

  * •

Case 3

Here, we consider the case that query hash code has $K$ wildcard bit ‘X’ from
the TLSH step, and and we want to detect the closest distance with $M$
mismatch bits between the query and keys with word length of $N$. The sense
margin can be written as:

$\displaystyle\beta=\frac{I_{m+1}}{I_{m}}-1$
$\displaystyle=\frac{(M+1)G_{on}V_{s}+(N-M-K-1)G_{off}}{MG_{on}V_{s}+(N-M-K)G_{off}V_{s}}-1$
(13) $\displaystyle=\frac{(M+1)r+N-M-K-1}{Mr+N-M-K}-1$ (14)
$\displaystyle=1/\left(M+\frac{N-K}{r-1}\right)$ (15)

It is expected because introducing the wild card equivalently reduces the word
length (ignoring the wire resistance effect). From Equation 15, one concludes
that the sense margin for the closest match increases with decreasing mismatch
bits $M$ and wordlength $N$, and rising number of wildcard $K$ and device
ON/OFF ratio $r$. In addition, when we introduce ‘X’ in hashcodes of the
query, the number of mismatch bits $M$ of the nearest neighbor is usually
reduced, further increasing the sensing margin.

From the above case study, we conclude that both decreasing the number of
mismatch bits $M$ and increasing the number of the wild card bits $K$ improve
the sense margin and robustness of the search operation. The ternary locality
sensitive hashing (TLSH) that we proposed in this work achieves both goals at
the same time while maintaining the same accuracy. The main reason behind this
is that hashcodes generated by random projections have some redundancy bits
that can be cast away by assigning them to the wildcard bit ‘X’. Here we show
the statistical results extracted from the few-shot classification to
demonstrate the idea.

Figure 20:  Sense margin analysis of the few-shot classification on the
Omniglot dataset. a, Distribution of the mismatch bits of the closest match
between query and keys in 4 different tasks. We can see that by using TLSH in
the hashing operation, we can significantly decrease the number of mismatch
bits. b, Relationship between the sense margin and the number of mismatch bits
$M$ of the closest match measured in 4 tasks. The sense margin decreases as
the $M$ increases. c,  Distribution of the sense margin for TLSH and LSH. The
probability that the sense margin is larger than 2 using TLSH method is twice
as high as that using LSH. This verifies that our TLSH method can
significantly increase the sense margin.

We first analyze the statistics of the mismatch bits numbers (or hamming
distance) of the closest match $M$ during the few-shot learning experiments.
The distribution of the hamming distance is shown in Supplementary Fig. 20a.
The result shows that by introducing ’X’ in the query hashcodes, the number of
mismatch bits $M$ of the closest match decreases significantly. The
experimental sensing margin is then extracted from the output current of the
closest match (the smallest current $I_{0}$) and the next close match (the 2nd
smallest current $I_{1}$), by the equation: $\beta=\frac{I_{1}}{I_{0}}-1$. As
expected, the experimental sensing margin is inversely proportial to the the
hamming distance, i.e. $\beta\propto\frac{1}{M}$ as is shown in Supplementary
Fig. 20b. Though the sense margin here is not the worst sense margin
characterized by Equation 15, we still see a similar trend. As a result, the
sensing margin is improved by introducing the wildcard ‘X’ from the TLSH
because of the reduced hamming distance of the closest match. The conclusion
is clearly demonstrated in the experimental sensing margin distribution shown
in Supplementary Fig. 20c. Specifically, the probability that the sense margin
is larger than two with TLSH is more than two times higher than that using
LSH. In light of the above analysis, we can conclude that the reduced mismatch
bit number from our TLSH scheme improves the sense margin of the crossbar-
based TCAM for the closest match search.

Figure 21:  Sense margin analysis of the 5-way 1-shot classification on the
Mini-ImageNet dataset. a,  Average sense margin drops as the word length
increases. Sense margin on the hashcodes generated by TLSH method is always
higher than that by LSH method. b,  Reduction of mismatch bits grows with the
word length and follows the similar trend as the number of ’X’ in the query.

Similar phenomena can also be observed in a scaled problem for few-shot
learning with Mini-ImageNet dataset. The result in Supplementary Fig. 21a
shows that the sensing margin decreases with the word length $N$, and the
sensing margin with TLSH is always higher than that with LSH, because of the
introduced wildcard bit ‘X’. Supplementary Fig. 21b also show that the
introduction of the wildcard ‘X’ also reduces the number of mismatch bits,
further improving the sensing margin.

Note that our TLSH method can also combine with other TCAM structures like
2T-2Rli20131_si, li2021sapiens_si, 2Flashfedorov2014area_si 2FeFETNi2019NE_si,
Laguna2019GLSVLSI_si which have the similar characteristic, but use latched
voltage signal as the output. As mentioned, the sense margin of our crossbar-
based TCAM can be further improved with the 3-bits encoding method, as
discussed in Supplementary Fig. 10.

Energy and Latency estimation

The latency and energy consumption estimation for end-to-end MANN on the
Omniglot dataset is based on the experimental data extracted from our
integrated crossbar system. For the latency estimation in CNN layers, we
considered channel-wise parallelism, where every channel in a convolutional
layer is working in a parallel way. Therefore, the latency of CNN layers for
every input image is depended on the number of input vectors and read time
through column wires. Each read time for vector-matrix-multiplication in the
latency estimation is $10\text{\,}\mathrm{ns}$. From the CNN structure we can
get the total time latency for CNN layers:

$T=t_{read}\times N_{input}=(784+784+196+196)\times
10=$19.6\text{\,}\mathrm{\SIUnitSymbolMicro s}$$

For the latency in TLSH and TCAM, since we need to predefine the memory size
for the few-shot learning task, i.e. , 256 entries for Omniglot, we still need
to partition the hashing matrix and memory matrix into multiple $64\times 64$
arrays, which is the size in the experiment. In this case, we still need to
consider the time latency of full adders for aggregating the results from
partitioned arrays. We use the $2.5\text{\,}\mathrm{ns}$ latency of
conventional 16T 8-bit adder reported in refghadiry2013dlpa_si. Therefore, we
get the time latency for TLSH and TCAM:
$$12.5\text{\,}\mathrm{ns}$+$12.5\text{\,}\mathrm{ns}$=$25\text{\,}\mathrm{ns}$$

The energy consumption estimation for Omniglot is done by reading the
conductance matrix out throughout the experiment. The total energy is
calculated by:

$E=\sum_{i,j}V_{ji}^{2}G_{i}t_{\text{pulse}}$

where $V_{ji}$ is the $j$th input voltage vector to $i$th conductance matrix,
$G_{i}$ is the $i$th readout conductance matrix and $t_{\text{pulse}}$ is the
pulse width of the input. The final energy consumption of the entire MANN per
image for the 5-way 1-shot problem is $1.26\text{\,}\mathrm{\SIUnitSymbolMicro
J}$ where search energy only consumes $13\text{\,}\mathrm{pJ}$ (including
TLSH). We acknowledge that the energy and latency overhead of peripheral
circuits which need to realize the entire MANN is not taken into account yet
since it’s not fully optimized and it’s difficult to estimate. The numbers
here mainly give an estimation for the implementation in crossbar arrays.

For the estimation for the Mini-ImageNet dataset, we mainly focus on the
overhead in the search operation which is the key point in this work. For
latency estimation, according to our architecture for scaled-up networks, time
won’t increase since the TLSH and TCAM are all working in a parallel way which
is $25\text{\,}\mathrm{ns}$ based on our aforementioned assumption. For the
energy consumption for Mini-ImageNet on the 5-way 1-shot task, we follow the
methods above while we use the simulated conductance matrix according to our
measurement. The final energy consumption for a single image search across the
entire memory module is $594\text{\,}\mathrm{pJ}$.

In summary, we estimated about $1.26\text{\,}\mathrm{\SIUnitSymbolMicro J}$
per Omniglot image inference (5-way 1-shot) with our full crossbar-based MANN
with $>$1,000$\times$ improvement over GPU ($1.38\text{\,}\mathrm{mJ}$ on
GPU). Different from MANN implemented on a conventional GPU backed by DRAM,
which spends most energy and time on memory transfer in the memory search,
both the energy consumption and time latency in crossbar arrays for MANN
mainly comes from the CNN controller ($19.6\text{\,}\mathrm{\SIUnitSymbolMicro
s}$ for time latency and $12.5\text{\,}\mathrm{\SIUnitSymbolMicro J}$ for
energy consumption). If we consider the search operation only, the consumed
energy for one search operation is only $2.1\text{\,}\mathrm{pJ}$ for the
Omniglot dataset (128 bits), 2857$\times$ improvement over GPU. For Mini-
ImageNet dataset, we achieve $82.4\text{\,}\mathrm{pJ}$ per search (4,096
bits), over $5.1\times 10^{4}$ improvement to GPU on the same
taskkazemi2021flash_si. The GPU latency numbers reported here are acquired
using PyTorch Profilerpaszke2019pytorch_si, and the energy consumption numbers
from the NVIDIA System Management Interface (nvidia-smi).

Those numbers can be further improved by choosing smaller input voltages and
optimizing peripheral circuits with more advanced technology nodes. Here, we
demonstrate that energy consumption can be improved by simply voltage scaling.
In the experiment, we reduce the search voltage by a factor of 10: from
$0.2\text{\,}\mathrm{V}$ to $0.02\text{\,}\mathrm{V}$. In this case, the power
consumption is lowered 100$\times$ with minor accuracy loss (from 76.8% to
77.5% for the 25-way 1-shot problem). It is noteworthy that the efficiency is
not necessarily improving with larger crossbarswan2021edge_si. We found the
size of 64$\times$64 or 128$\times$128 is a sweet spot when considering the
output current, matrix utilization rate, etc. Quantitively, we compared search
energy in the distance search operation with different approaches and listed
them in the table below, clearly showing our approach’s benefit and
scalability. The result of the search energy is averaged over 11,875 search
operations in the 25-way 1-shot task based on the experimentally measured
data. To make a fair comparison, we report the search energy per bit of our
approach and achieve the lowest energy consumption over previous work.

Table 3: Energy consumption per bit per search operation of TCAM | Ref.karunaratne2021robust_si | Ref.Ni2019NE_si | Our work
---|---|---|---
Energy consumption per bit per search/fJ | 2.5 | 0.4 | 0.17

##  4 Supplementary References

naturemag sicite