IEEEexample:BSTcontrol

# Methodologies, Workloads, and Tools for Processing-in-Memory:
Enabling the Adoption of Data-Centric Architectures

Geraldo F. Oliveira ⋄ Juan Gómez-Luna ⋄ Saugata Ghose ⊙ Onur Mutlu ⋄

⋄ETH Zürich ⊙University of Illinois Urbana-Champaign

##  1 Motivation & Problem

The increasing prevalence and growing size of data in modern applications have
led to high costs for computation in traditional processor-centric computing
systems. Moving large volumes of data between memory devices (e.g., DRAM) and
computing elements (e.g., CPUs, GPUs) across bandwidth-limited memory channels
can consume more than 60% of the total energy in modern systems [1, 2]. To
mitigate these costs, the _processing-in-memory_ (PIM) [3, 4, 5, 6, 1, 7, 8,
9] paradigm moves computation closer to where the data resides, reducing (and
in some cases eliminating) the need to move data between memory and the
processor.

There are two main approaches to PIM [4]: (1) processing-near-memory (_PnM_)
[10, 11, 2, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,
28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46,
47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65,
66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77], where PIM logic is added to
the same die as memory or to the logic layer of 3D-stacked memory [78, 79,
80]; and (2) processing-using-memory (_PuM_) [81, 82, 83, 84, 85, 86, 87, 88,
89, 90, 91, 92, 93, 94, 95, 44, 96, 97, 98, 99, 100, 101, 102, 103], which
uses the operational principles of memory cells to perform computation (for
example, by exploiting DRAM’s analog operation to perform bulk bitwise AND,
OR, and NOT logic operations [83, 99, 84, 87, 97, 96]).

Many works from academia [19, 10, 11, 90, 104, 105, 106, 107, 20, 21, 82, 81,
35, 39, 15, 18, 25, 16, 17, 12, 23, 31, 38, 37, 22, 36, 108, 109, 83, 85, 86,
110, 111, 48, 99, 2, 112] and industry [50, 51, 52, 53, 54, 34, 42, 41, 43]
have shown the benefits of PnM and PuM for a wide range of workloads from
different domains. However, fully adopting PIM in commercial systems is still
very challenging due to the lack of tools and system support for PIM
architectures across the computer architecture stack [4], which includes:
_(i)_ workload characterization methodologies and benchmark suites targeting
PIM architectures; _(ii)_ frameworks that can facilitate the implementation of
complex operations and algorithms using the underlying PIM primitives (e.g.,
simple PIM arithmetic operations [19], bulk bitwise Boolean in-DRAM operations
[83, 84, 92]); _(iii)_ compiler support and compiler optimizations targeting
PIM architectures; _(iv)_ operating system support for PIM-aware virtual
memory, memory management, data allocation and mapping; and _(v)_ efficient
data coherence and consistency mechanisms.

Our _goal_ in this work is to provide tools and system support for PnM and PuM
architectures, aiming to ease the adoption of PIM in current and future
systems. With this goal in mind, we address two limitations of prior works
related to _(i)_ identifying and characterizing workloads suitable for PnM
offloading and _(ii)_ enabling complex operations in PuM architectures. First,
we develop a methodology, called _DAMOV_ , that identifies sources of data
movement bottlenecks in applications and associates such bottlenecks with PIM
suitability. Second, we propose an end-to-end framework, called _SIMDRAM_ ,
that enables the implementation of complex in-DRAM operations transparently to
the programmer.

##  2 DAMOV: Identifying and Characterizing Data Movement Bottlenecks

DAMOV introduces the first rigorous methodology to characterize memory-related
data movement bottlenecks in modern workloads and the first benchmark suite
for data movement related studies. We develop a new methodology to correlate
application characteristics with the _primary_ sources of data movement
bottlenecks and to determine the potential benefits of three example data
movement mitigation mechanisms: (1) a deep cache hierarchy, (2) a hardware
prefetcher, and (3) a general-purpose PnM architecture.

Our methodology has three steps. In Step 1 ( 1 in Figure 1), we use a hardware
profiling tool [113] to identify memory-bound functions across applications.
In Step 2 ( 2), we use an architecture-independent profiling tool [114, 115]
to collect metrics that provide insights about the memory access behavior of
each function. In Step 3 ( 3), we collect architecture-dependent metrics and
analyze the performance and energy of each function on our three data movement
mitigation mechanisms. By combining the three steps, we systematically
classify the leading causes of data movement bottlenecks in an application or
function into different bottleneck classes.

Figure 1: Overview of our three-step workload characterization methodology.

Using this new methodology, we characterize 345 applications from a wide range
of domains. Within these applications, we find (and fully characterize) 144
functions that are memory-bound and significantly contribute to the overall
execution time. These functions are the core of our data movement benchmark
suite, called DAMOV [116]. Our analyses reveal six new insights about the
sources of data movement bottlenecks and their relation to PnM:

  1. 1.

Applications with high last-level cache misses per kilo-instruction (MPKI) and
low temporal locality are _DRAM bandwidth-bound_. These applications benefit
from the large memory bandwidth available to the PnM system.

  2. 2.

Applications with low last-level cache MPKI and low temporal locality are
_DRAM latency-bound_. These applications do _not_ benefit from L2/L3 caches.
The PnM system improves performance and energy efficiency by sending L1 misses
directly to DRAM.

  3. 3.

A second group of applications with low LLC MPKI and low temporal locality are
_bottlenecked by L1/L2 cache capacity_. These applications benefit from the
PnM system at low core counts. However, at high core counts (and thus larger
L1/L2 cache space), the caches capture most of the data locality in these
applications, decreasing the benefits the PnM system provides. We make this
observation using a _new_ metric that we develop, called _last-to-first miss-
ratio (LFMR)_ , which we define as the ratio between the number of LLC misses
and the total number of L1 cache misses. We find that this metric accurately
identifies how efficient the cache hierarchy is in reducing data movement.

  4. 4.

Applications with high temporal locality and low LLC MPKI are _bottlenecked by
L3 cache contention_ at high core counts. In such cases, the PnM system
provides a cost-effective way to alleviate cache contention over increasing
the L3 cache capacity.

  5. 5.

Applications with high temporal locality, low LLC MPKI, and low arithmetic
instruction (AI) are bottlenecked by the _L1 cache capacity_. The three
candidate data movement mitigation mechanisms achieve similar performance and
energy consumption for these applications.

  6. 6.

Applications with high temporal locality, low LLC MPKI, and high AI are
_compute-bound_. These applications benefit from a deep cache hierarchy and
hardware prefetchers, but the PnM system degrades their performance.

We publicly release our 144 representative data movement bottlenecked
functions from 74 applications as the first open-source benchmark suite for
data movement, called DAMOV benchmark suite, along with the complete source
code for our new characterization methodology and simulator [116]. For more
information on our extensive data movement bottleneck characterization and on
our DAMOV benchmark suite, along with our detailed contributions (including
four use cases of our benchmark suite), please refer to our full paper [5,
117].

##  3 SIMDRAM: Enabling Complex Operations using DRAM

A common approach for PuM architectures is to make use of bulk bitwise
computation. Many widely-used data-intensive applications (e.g., databases,
neural networks, graph analytics) heavily rely on a broad set of simple (e.g.,
AND, OR, XOR) and complex (e.g., equality check, multiplication, addition)
bitwise operations. Ambit [83, 99, 84, 87, 97, 96], an in-DRAM PuM
accelerator, proposes exploiting DRAM’s analog operation to perform bulk
bitwise majority-of-three (MAJ) computation, which can be manipulated to
perform AND, OR, and NOT logic operations. Inspired by Ambit, many prior works
have explored DRAM and emerging non-volatile memory (NVM) [118, 119, 120, 121,
122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136,
137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151,
152, 153, 154] designs that are capable of performing in-memory bitwise
operations [106, 155, 156, 100, 92, 89, 157]. However, a major shortcoming
prevents these proposals from becoming widely applicable: they support only
basic operations (e.g., Boolean operations, addition) and fall short on
flexibly supporting new and more complex operations. Our _goal_ is to design a
framework that aids the adoption of processing-using-DRAM by efficiently
implementing complex operations and providing the flexibility to support new
desired operations.

To this end, we propose SIMDRAM, the first end-to-end framework for
processing-using-DRAM. At its core, we build the SIMDRAM framework around a
DRAM substrate that enables two previously-proposed techniques: (1) vertical
data layout in DRAM to support bit-shift operations, and (2) majority-based
logic. SIMDRAM consists of three key steps, illustrated in Figure 2, to enable
a desired operation in DRAM: (1) building an efficient MAJ/NOT-based
representation of the desired operation, (2) mapping the operation input and
output operands to DRAM rows and to the required DRAM commands that produce
the desired operation, and (3) executing the operation. We briefly describe
these steps.

(a) SIMDRAM Framework: Steps 1 and 2

(b) SIMDRAM Framework: Step 3

Figure 2: Overview of the SIMDRAM framework.

The goal of the first step ( 1 in Figure 2(a)) is to use logic optimization to
minimize the number of DRAM row activations, and therefore the compute latency
required to perform a specific operation. Accordingly, for a desired
computation, the first step is to derive its _optimized_ MAJ/NOT-based
implementation from its AND/OR/NOT-based implementation.

The second step ( 2 in Figure 2(a)) translates the MAJ/NOT-based
implementation into DRAM row activations. This step includes (1) mapping the
operands to the designated rows in DRAM, and (2) defining the sequence of DRAM
row activations that are required to perform the computation. SIMDRAM chooses
the operand-to-row mapping and the sequence of DRAM row activations to
minimize the number of DRAM row activations required for a specific operation.

The third step ( 3 in Figure 2(b)) is to program the memory controller to
issue the sequence of DRAM row activations to the appropriate rows in DRAM to
perform the computation of the operation from start to end. To this end,
SIMDRAM uses a _control unit_ in the memory controller that transparently
executes the sequence of DRAM row activations for each specific operation.

System Integration. To incorporate SIMDRAM into a real system, we address two
integration challenges as part of our work: (1) managing memory with both
vertical and horizontal layouts in a system, and (2) exposing SIMDRAM
functionality to programmers and compilers. As part of the support for system
integration, we introduce two components.

First, SIMDRAM adds a _transposition unit_ in the memory controller that
transforms the data layout from the conventional horizontal layout to vertical
layout (and vice versa), allowing both layouts to coexist. Using the
transposition unit, SIMDRAM provides the ability to store only the data that
is required for in-DRAM computation in the vertical layout. SIMDRAM maintains
the horizontal layout for the rest of the data and allows the CPU to
read/write its operands from/to DRAM in a horizontal layout and at full
bandwidth. Second, SIMDRAM extends the ISA to enable the user/compiler to
communicate with the SIMDRAM control unit.

Key Results.  We demonstrate SIMDRAM’s functionality using an example set of
operations including (1)  _N_ -input logic operations (e.g., AND/OR/XOR of
more than 2 input bits); (2) relational operations (e.g., equality/inequality
check, greater than, maximum, minimum); (3) arithmetic operations (e.g.,
addition, subtraction, multiplication, division); (4) predication (e.g., if-
then-else); and (5) other complex operations such as bitcount and ReLU [158].

We compare the benefits of SIMDRAM to different state-of-the-art computing
platforms (CPU, GPU, and Ambit [83, 99, 84, 87, 97, 96]). We comprehensively
evaluate SIMDRAM’s reliability, area overhead, throughput, and energy
efficiency. We leverage the SIMDRAM framework to accelerate seven application
kernels from machine learning, databases, and image processing (VGG-13 [159],
VGG-16 [159], LeNET [160], kNN [161], TPC-H [162], BitWeaving [163],
brightness [164]). Using a single DRAM bank, SIMDRAM provides (1) 2.0$\times$
the throughput and 2.6$\times$ the energy efficiency of Ambit [83], averaged
across the 16 implemented operations; and (2) 2.5$\times$ the performance of
Ambit, averaged across the seven application kernels. Compared to a CPU and a
high-end GPU, SIMDRAM using 16 DRAM banks provides (1) 257$\times$ and
31$\times$ the energy efficiency, and 88$\times$ and 5.8$\times$ the
throughput of the CPU and GPU, respectively, averaged across the 16
operations; and (2) 21$\times$ and 2.1$\times$ the performance of the CPU and
GPU, respectively, averaged across the seven application kernels. SIMDRAM
incurs no additional area overhead on top of Ambit, and a total area overhead
of only 0.2% in a high-end CPU. We also evaluate the reliability of SIMDRAM
under different degrees of manufacturing process variation, and observe that
it guarantees correct operation as the DRAM process technology node scales
down to smaller sizes.

For more information on our SIMDRAM framework and our extensive evaluation
results (including a comparison to an alternative framework for processing-
using-cache architectures), please refer to our full paper [165, 166].

##  4 Discussion

Few prior works tackle the challenge of providing end-to-end support for PIM.
We describe these works and their limitations for in-DRAM computing.

Workload Characterization and Benchmark Suites for PIM. We highlight two prior
works, [167] and PrIM [42, 41, 43] that also focus on characterizing workloads
and providing benchmark suites for PIM architectures. In [167], the authors
provide the first work that characterizes workloads for PIM. They analyze the
benefits a PIM architecture similar to [71], where vector processing compute
units are integrated into the DDRx memory modules, provides for five
applications. Even though [167] has a similar goal to DAMOV, it understandably
does not provide insights into modern data-intensive applications and PIM
architectures as it dates from 2001. The authors of [42, 41, 43] propose PrIM,
a benchmark suite of 16 workloads from different application domains (e.g.,
dense/sparse linear algebra, databases, data analytics, graph processing,
neural networks, bioinformatics, image processing) tailored to fit the
characteristics of a _real_ PIM architecture (i.e., the UPMEM-based PIM system
[34]). PrIM is open-source and publicly available at [168]. Unlike these prior
works, DAMOV is applicable to and can be used to study other PIM architectures
than processing-in/-near DRAM, including processing-in/-near cache [93, 94,
95, 68, 169, 170, 171], processing-in/-near storage [40, 172, 173, 174, 175,
176, 177, 178, 179, 180, 181], and processing-in/-near emerging NVMs [100, 82,
91, 81, 90, 182, 183]. This is possible since DAMOV’s methodology and
benchmarks are mainly concerned with broadly characterizing data movement
bottlenecks in an application, independent of the underlying PIM architecture.

Frameworks for PIM. DualityCache [95] is an end-to-end framework for in-cache
computing, which executes a fixed set of operations in a single-instruction
multiple-thread (SIMT) manner. Employing DualityCache in DRAM is not
straightforward due to the fundamental differences between in-cache computing
and in-DRAM computing (e.g., the destructive behavior of DRAM operations and
cost-sensitivity of DRAM chips). Two prior works, Hyper-A [184] and IMP [185],
propose frameworks for in-emerging-NVM computing. Since Hyper-A and IMP target
in-emerging-NVM substrates that utilize different computing paradigms (e.g.,
associative processing [186, 187]) or rely on particular structures of the NVM
array (such as analog-to-digital/digital-to-analog converters) to perform
computation, they are _not_ applicable to an in-DRAM substrate that performs
bulk bitwise operations. Olgun et al. propose the PiDRAM [188] framework, a
flexible end-to-end and open-source FPGA-based framework that enables system
integration studies and evaluation of in-DRAM computing techniques (e.g., in-
DRAM copy and initialization [86, 98] and in-DRAM true random generation [108,
189, 190]) using real unmodified DRAM chips. PiDRAM is publicly available at
[191] and can be used to prototype our SIMDRAM framework in a real system.

##  5 Conclusion & Future Work

This paper summarizes two of our recent efforts towards providing holistic
system-level support for processing-in-memory (PIM) systems. We provide _(i)_
a methodology to identify and characterize sources of data movement
bottlenecks in a workload that can enable the programmer to assess whether a
processing-near-memory (PnM) architecture can mitigate the identified data
movement bottlenecks; _(ii)_ the first benchmark suite (i.e., _DAMOV_)
tailored for analyzing data movement bottlenecks and effects of near-data
processing; and _(iii)_ an end-to-end framework (i.e., _SIMDRAM_) that enables
efficient and programmer-transparent computation of a wide range of arbitrary
and complex operations by employing processing-using-memory (PuM) in DRAM. We
believe that DAMOV can enable (1) simple and practical identification of PIM-
suitable workloads and functions, (2) a research substrate (with our benchmark
suite and simulator) for PIM-related architecture and system studies. SIMDRAM
can facilitate the broader adoption of PuM architectures by more workloads and
programmers. We hope that our work inspires future research on system-level
solutions and tools that can aid the research, development, and implementation
of PIM architectures.

## Acknowledgments

We thank the SAFARI Research Group members for valuable feedback and the
stimulating intellectual environment they provide. We acknowledge the generous
gifts provided by our industrial partners, including ASML, Facebook, Google,
Huawei, Intel, Microsoft, and VMware. We acknowledge support from the
Semiconductor Research Corporation and the ETH Future Computing Laboratory.

This invited extended abstract is a summary version of our two prior works
DAMOV [5, 117] (published at IEEE Access 2021) and SIMDRAM [165, 166]
(published at ASPLOS 2021). Presentations that describe DAMOV can be found at
[192] (short talk video), [193] (long talk video), and [194] (tutorial on the
DAMOV framework and benchmarks). A presentation that describes SIMDRAM can be
found at [195].

## References

  * [1] O. Mutlu _et al._ , “Processing Data Where It Makes Sense: Enabling In-Memory Computation,” _MICPRO_ , 2019.
  * [2] A. Boroumand _et al._ , “Google Workloads for Consumer Devices: Mitigating Data Movement Bottlenecks,” in _ASPLOS_ , 2018.
  * [3] S. Ghose _et al._ , “Processing-in-Memory: A Workload-Driven Perspective,” _IBM JRD_ , 2019.
  * [4] O. Mutlu _et al._ , “A Modern Primer on Processing in Memory,” _Emerging Computing: From Devices to Systems - Looking Beyond Moore and Von Neumann_ , 2021.
  * [5] G. F. Oliveira _et al._ , “DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks,” _IEEE Access_ , 2021.
  * [6] S. Ghose _et al._ , “The Processing-in-Memory Paradigm: Mechanisms to Enable Adoption,” in _Beyond-CMOS Technologies for Next Generation Computer Design_ , 2019.
  * [7] O. Mutlu _et al._ , “Enabling Practical Processing in and Near Memory for Data-Intensive Computing,” in _DAC_ , 2019.
  * [8] O. Mutlu and L. Subramanian, “Research Problems and Opportunities in Memory Systems,” _SUPERFRI_ , 2014.
  * [9] O. Mutlu, “Memory Scaling: A Systems Architecture Perspective,” in _IMW_ , 2013.
  * [10] J. Ahn _et al._ , “A Scalable Processing-in-Memory Accelerator for Parallel Graph Processing,” in _ISCA_ , 2015.
  * [11] L. Nai _et al._ , “GraphPIM: Enabling Instruction-Level PIM Offloading in Graph Computing Frameworks,” in _HPCA_ , 2017.
  * [12] A. Boroumand _et al._ , “LazyPIM: An Efficient Cache Coherence Mechanism for Processing-in-Memory,” _IEEE CAL_ , 2017.
  * [13] D. Zhang _et al._ , “TOP-PIM: Throughput-Oriented Programmable Processing in Memory,” in _HPDC_ , 2014.
  * [14] M. Gao and C. Kozyrakis, “HRL: Efficient and Flexible Reconfigurable Logic for Near-Data Processing,” in _HPCA_ , 2016.
  * [15] J. S. Kim _et al._ , “GRIM-Filter: Fast Seed Location Filtering in DNA Read Mapping using Processing-in-Memory Technologies,” in _APBC_ , 2018\.
  * [16] M. Drumond _et al._ , “The Mondrian Data Engine,” in _ISCA_ , 2017.
  * [17] P. C. Santos _et al._ , “Operand Size Reconfiguration for Big Data Processing in Memory,” in _DATE_ , 2017.
  * [18] G. F. Oliveira _et al._ , “NIM: An HMC-Based Machine for Neuron Computation,” in _ARC_ , 2017.
  * [19] J. Ahn _et al._ , “PIM-Enabled Instructions: A Low-Overhead, Locality-Aware Processing-in-Memory Architecture,” in _ISCA_ , 2015.
  * [20] M. Gao _et al._ , “TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory,” in _ASPLOS_ , 2017.
  * [21] D. Kim _et al._ , “Neurocube: A Programmable Digital Neuromorphic Architecture with High-Density 3D Memory,” in _ISCA_ , 2016.
  * [22] P. Gu _et al._ , “Leveraging 3D Technologies for Hardware Security: Opportunities and Challenges,” in _GLSVLSI_ , 2016.
  * [23] A. Boroumand _et al._ , “CoNDA: Efficient Cache Coherence Support for Near-Data Accelerators,” in _ISCA_ , 2019.
  * [24] K. Hsieh _et al._ , “Transparent Offloading and Mapping (TOM) Enabling Programmer-Transparent Near-Data Processing in GPU Systems,” in _ISCA_ , 2016.
  * [25] D. S. Cali _et al._ , “GenASM: A High-Performance, Low-Power Approximate String Matching Acceleration Framework for Genome Sequence Analysis,” in _MICRO_ , 2020.
  * [26] S. H. Pugsley _et al._ , “NDC: Analyzing the Impact of 3D-Stacked Memory+Logic Devices on MapReduce Workloads,” in _ISPASS_ , 2014.
  * [27] A. Farmahini-Farahani _et al._ , “NDA: Near-DRAM Acceleration Architecture Leveraging Commodity DRAM Devices and Standard Memory Modules,” in _HPCA_ , 2015.
  * [28] G. H. Loh _et al._ , “A Processing in Memory Taxonomy and a Case for Studying Fixed-Function PIM,” in _WoNDP_ , 2013.
  * [29] A. Pattnaik _et al._ , “Scheduling Techniques for GPU Architectures with Processing-in-Memory Capabilities,” in _PACT_ , 2016.
  * [30] B. Akin _et al._ , “Data Reorganization in Memory Using 3D-Stacked DRAM,” in _ISCA_ , 2016.
  * [31] K. Hsieh _et al._ , “Accelerating Pointer Chasing in 3D-Stacked Memory: Challenges, Mechanisms, Evaluation,” in _ICCD_ , 2016.
  * [32] O. O. Babarinsa and S. Idreos, “JAFAR: Near-Data Processing for Databases,” in _SIGMOD_ , 2015.
  * [33] J. H. Lee _et al._ , “BSSync: Processing Near Memory for Machine Learning Workloads with Bounded Staleness Consistency Models,” in _PACT_ , 2015.
  * [34] F. Devaux, “The True Processing in Memory Accelerator,” in _HCS_ , 2019\.
  * [35] A. Boroumand _et al._ , “Mitigating Edge Machine Learning Inference Bottlenecks: An Empirical Study on Accelerating Google Edge Models,” arXiv:2103.00768 [cs.AR], 2021.
  * [36] A. Boroumand _et al._ , “Google Neural Network Models for Edge Devices: Analyzing and Mitigating Machine Learning Inference Bottlenecks,” in _PACT_ , 2021.
  * [37] A. Boroumand _et al._ , “Polynesia: Enabling High-Performance and Energy-Efficient Hybrid Transactional/Analytical Databases with Hardware/Software Co-Design,” in _ICDE_ , 2022.
  * [38] A. Boroumand _et al._ , “Polynesia: Enabling Effective Hybrid Transactional/Analytical Databases with Specialized Hardware/Software Co-Design,” arXiv:2103.00798 [cs.AR], 2021.
  * [39] A. Boroumand, “Practical Mechanisms for Reducing Processor-Memory Data Movement in Modern Workloads,” Ph.D. dissertation, Carnegie Mellon University, 2020.
  * [40] N. M. Ghiasi _et al._ , “GenStore: A High-Performance and Energy-Efficient In-Storage Computing System for Genome Sequence Analysis,” in _ASPLOS_ , 2022.
  * [41] J. Gómez-Luna _et al._ , “Benchmarking Memory-Centric Computing Systems: Analysis of Real Processing-in-Memory Hardware,” in _CUT_ , 2021\.
  * [42] J. Gómez-Luna _et al._ , “Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecture,” arXiv:2105.03814 [cs.AR], 2021.
  * [43] J. Gómez-Luna _et al._ , “Benchmarking a New Paradigm: Experimental Analysis and Characterization of a Real Processing-in-Memory System,” _IEEE Access_ , 2022.
  * [44] M. Besta _et al._ , “SISA: Set-Centric Instruction Set Architecture for Graph Mining on Processing-in-Memory Systems,” in _MICRO_ , 2021.
  * [45] C. Giannoula _et al._ , “SynCron: Efficient Synchronization Support for Near-Data-Processing Architectures,” in _HPCA_ , 2021.
  * [46] I. Fernandez _et al._ , “NATSA: A Near-Data Processing Accelerator for Time Series Analysis,” in _ICCD_ , 2020.
  * [47] G. Singh _et al._ , “NERO: A Near High-Bandwidth Memory Stencil Accelerator for Weather Prediction Modeling,” in _FPL_ , 2020.
  * [48] S. H. S. Rezaei _et al._ , “NoM: Network-on-Memory for Inter-Bank Data Transfer in Highly-Banked Memories,” _CAL_ , 2020.
  * [49] G. Singh _et al._ , “NAPEL: Near-Memory Computing Application Performance Prediction via Ensemble Learning,” in _DAC_ , 2019.
  * [50] S. Lee _et al._ , “A 1ynm 1.25V 8Gb, 16Gb/s/Pin GDDR6-Based Accelerator-in-Memory Supporting 1TFLOPS MAC Operation and Various Activation Functions for Deep-Learning Applications,” in _ISSCC_ , 2022.
  * [51] Y.-C. Kwon _et al._ , “25.4 A 20nm 6GB Function-In-Memory DRAM, Based on HBM2 with a 1.2 TFLOPS Programmable Computing Unit Using Bank-Level Parallelism, for Machine Learning Applications,” in _ISSCC_ , 2021.
  * [52] S. Lee _et al._ , “Hardware Architecture and Software Stack for PIM Based on Commercial DRAM Technology: Industrial Product,” in _ISCA_ , 2021.
  * [53] L. Ke _et al._ , “Near-Memory Processing in Action: Accelerating Personalized Recommendation with AxDIMM,” _IEEE Micro_ , 2021.
  * [54] D. Niu _et al._ , “184QPS/W 64Mb/$mm^{2}$ 3D Logic-to-DRAM Hybrid Bonding with Process-Near-Memory Engine for Recommendation System,” in _ISSCC_ , 2022.
  * [55] C. Giannoula _et al._ , “SparseP: Towards Efficient Sparse Matrix Vector Multiplication on Real Processing-In-Memory Systems,” in _SIGMETRICS_ , 2022\.
  * [56] H. Shin _et al._ , “McDRAM: Low Latency and Energy-Efficient Matrix Computations in DRAM,” _IEEE TCADICS_ , 2018.
  * [57] S. Cho _et al._ , “McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge,” _IEEE Access_ , 2020.
  * [58] Q. Zhu _et al._ , “Accelerating Sparse Matrix-Matrix Multiplication with 3D-Stacked Logic-in-Memory Hardware,” in _HPEC_ , 2013.
  * [59] E. Azarkhish _et al._ , “Logic-Base Interconnect Design for Near Memory Computing in the Smart Memory Cube,” _IEEE VLSI_ , 2016.
  * [60] E. Azarkhish _et al._ , “Neurostream: Scalable and Energy Efficient Deep Learning with Smart Memory Cubes,” _TPDS_ , 2017.
  * [61] Q. Guo _et al._ , “3D-Stacked Memory-Side Acceleration: Accelerator and System Design,” in _WoNDP_ , 2014.
  * [62] J. P. C. de Lima _et al._ , “Design Space Exploration for PIM Architectures in 3D-Stacked Memories,” in _CF_ , 2018.
  * [63] B. Akın _et al._ , “HAMLeT: Hardware Accelerated Memory Layout Transform within 3D-Stacked DRAM,” in _HPEC_ , 2014.
  * [64] Y. Huang _et al._ , “A Heterogeneous PIM Hardware-Software Co-Design for Energy-Efficient Graph Processing,” in _IPDPS_ , 2020.
  * [65] G. Dai _et al._ , “GraphH: A Processing-in-Memory Architecture for Large-Scale Graph Processing,” _IEEE TCAD_ , 2018.
  * [66] J. Liu _et al._ , “Processing-in-Memory for Energy-Efficient Neural Network Training: A Heterogeneous Approach,” in _MICRO_ , 2018.
  * [67] P.-A. Tsai _et al._ , “Adaptive Scheduling for Systems with Asymmetric Memory Hierarchies,” in _MICRO_ , 2018-10.
  * [68] A. Denzler _et al._ , “Casper: Accelerating Stencil Computation using Near-Cache Processing,” arXiv:2112.14216 [cs.AR], 2021.
  * [69] P. Gu _et al._ , “iPIM: Programmable In-Memory Image Processing Accelerator using Near-Bank Architecture,” in _ISCA_ , 2020.
  * [70] H. Asghari-Moghaddam _et al._ , “Chameleon: Versatile and Practical Near-DRAM Acceleration Architecture for Large Memory Systems,” in _MICRO_ , 2016.
  * [71] D. Patterson _et al._ , “A Case for Intelligent RAM,” _IEEE Micro_ , 1997.
  * [72] R. Balasubramonian _et al._ , “Near-Data Processing: Insights from a MICRO-46 Workshop,” _IEEE Micro_ , 2014.
  * [73] D. G. Elliott _et al._ , “Computational RAM: Implementing Processors in Memory,” _Design and Test of Computers_ , 1999.
  * [74] M. A. Z. Alves _et al._ , “Saving Memory Movements Through Vector Processing in the DRAM,” in _CASES_ , 2015.
  * [75] A. Farmahini-Farahani _et al._ , “DRAMA: An Architecture for Accelerated Processing Near Memory,” _Computer Architecture Letters_ , 2014.
  * [76] H. Asghari-Moghaddam _et al._ , “Near-DRAM Acceleration with Single-ISA Heterogeneous Processing in Standard Memory Modules,” _IEEE Micro_ , 2016\.
  * [77] S. L. Xi _et al._ , “Beyond the Wall: Near-Data Processing for Databases,” in _DaMoN_ , 2015.
  * [78] Hybrid Memory Cube Consortium, “Hybrid Memory Cube Specification Rev. 2.0,” http://www.hybridmemorycube.org/.
  * [79] D. U. Lee _et al._ , “25.2 A 1.2V 8Gb 8-Channel 128GB/s High-Bandwidth Memory (HBM) Stacked DRAM with Effective Microbump I/O Test Methods Using 29nm Process and TSV,” in _ISSCC_ , 2014.
  * [80] D. Lee _et al._ , “Simultaneous Multi-Layer Access: Improving 3D-Stacked Memory Bandwidth at Low Cost,” _ACM TACO_ , 2016.
  * [81] P. Chi _et al._ , “PRIME: A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory,” in _ISCA_ , 2016\.
  * [82] A. Shafiee _et al._ , “ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars,” in _ISCA_ , 2016.
  * [83] V. Seshadri _et al._ , “Ambit: In-Memory Accelerator for Bulk Bitwise Operations Using Commodity DRAM Technology,” in _MICRO_ , 2017.
  * [84] V. Seshadri and O. Mutlu, “In-DRAM Bulk Bitwise Execution Engine,” arXiv:1905.09822 [cs.AR], 2019.
  * [85] S. Li _et al._ , “DRISA: A DRAM-Based Reconfigurable In-Situ Accelerator,” in _MICRO_ , 2017.
  * [86] V. Seshadri _et al._ , “RowClone: Fast and Energy-Efficient In-DRAM Bulk Data Copy and Initialization,” in _MICRO_ , 2013.
  * [87] V. Seshadri and O. Mutlu, “The Processing Using Memory Paradigm: In-DRAM Bulk Copy, Initialization, Bitwise AND and OR,” arXiv:1610.09603 [cs.AR], 2016.
  * [88] Q. Deng _et al._ , “DrAcc: A DRAM Based Accelerator for Accurate CNN Inference,” in _DAC_ , 2018.
  * [89] X. Xin _et al._ , “ELP2IM: Efficient and Low Power Bitwise Operation Processing in DRAM,” in _HPCA_ , 2020.
  * [90] L. Song _et al._ , “GraphR: Accelerating Graph Processing Using ReRAM,” in _HPCA_ , 2018.
  * [91] L. Song _et al._ , “PipeLayer: A Pipelined ReRAM-Based Accelerator for Deep Learning,” in _HPCA_ , 2017.
  * [92] F. Gao _et al._ , “ComputeDRAM: In-Memory Compute Using Off-the-Shelf DRAMs,” in _MICRO_ , 2019.
  * [93] C. Eckert _et al._ , “Neural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networks,” in _ISCA_ , 2018.
  * [94] S. Aga _et al._ , “Compute Caches,” in _HPCA_ , 2017.
  * [95] D. Fujiki _et al._ , “Duality Cache for Data Parallel Acceleration,” in _ISCA_ , 2019.
  * [96] V. Seshadri _et al._ , “Buddy-RAM: Improving the Performance and Efficiency of Bulk Bitwise Operations Using DRAM,” arXiv:1611.09988 [cs:AR], 2016.
  * [97] V. Seshadri and O. Mutlu, “Simple Operations in Memory to Reduce Data Movement,” in _Advances in Computers, Volume 106_ , 2017.
  * [98] V. Seshadri _et al._ , “RowClone: Accelerating Data Movement and Initialization Using DRAM,” arXiv:1805.03502 [cs.AR], 2018.
  * [99] V. Seshadri _et al._ , “Fast Bulk Bitwise AND and OR in DRAM,” in _IEEE CAL_ , 2015.
  * [100] S. Li _et al._ , “Pinatubo: A Processing-in-Memory Architecture for Bulk Bitwise Operations in Emerging Non-Volatile Memories,” in _DAC_ , 2016.
  * [101] J. D. Ferreira _et al._ , “pLUTo: In-DRAM Lookup Tables to Enable Massively Parallel General-Purpose Computation,” arXiv:2104.07699 [cs.AR], 2021\.
  * [102] M. Imani _et al._ , “FloatPIM: In-Memory Acceleration of Deep Neural Network Training with High Precision,” in _ISCA_ , 2019.
  * [103] Z. He _et al._ , “Sparse BD-Net: A Multiplication-Less DNN with Sparse Binarized Depth-Wise Separable Convolution,” _JETC_ , 2020.
  * [104] M. Zhang _et al._ , “GraphP: Reducing Communication for PIM-Based Graph Processing with Efficient Data Partition,” in _HPCA_ , 2018.
  * [105] S. Angizi _et al._ , “GraphS: A Graph Processing Accelerator Leveraging SOT-MRAM,” in _DATE_ , 2019.
  * [106] S. Angizi and D. Fan, “GraphiDe: A Graph Processing Accelerator Leveraging In-DRAM-Computing,” in _GLSVLSI_ , 2019.
  * [107] Y. Zhuo _et al._ , “GraphQ: Scalable PIM-Based Graph Processing,” in _MICRO_ , 2019.
  * [108] J. S. Kim _et al._ , “D-RaNGe: Using Commodity DRAM Devices to Generate True Random Numbers With Low Latency and High Throughput,” in _HPCA_ , 2019\.
  * [109] J. S. Kim _et al._ , “The DRAM Latency PUF: Quickly Evaluating Physical Unclonable Functions by Exploiting the Latency-Reliability Tradeoff in Modern Commodity DRAM Devices,” in _HPCA_ , 2018.
  * [110] Y. Wang _et al._ , “FIGARO: Improving System Performance via Fine-Grained In-DRAM Data Relocation and Caching,” in _MICRO_ , 2020.
  * [111] K. K. Chang _et al._ , “Low-Cost Inter-Linked Subarrays (LISA): Enabling Fast Inter-Subarray Data Movement in DRAM,” in _HPCA_ , 2016.
  * [112] D. S. Cali _et al._ , “SeGraM: A Universal Hardware Accelerator for Genomic Sequence-to-Graph and Sequence-to-Sequence Mapping,” in _ISCA_ , 2022.
  * [113] Intel Corp., “Intel VTune Amplifier 2019 User Guide,” https://software.intel.com/en-us/vtune-amplifier-help.
  * [114] J. Weinberg _et al._ , “Quantifying Locality in the Memory Access Patterns of HPC Applications,” in _SC_ , 2005.
  * [115] Y. S. Shao and D. Brooks, “ISA-Independent Workload Characterization and Its Implications for Specialized Architectures,” in _ISPASS_ , 2013.
  * [116] SAFARI Research Group, “DAMOV Benchmark Suite and Simulation Framework,” https://github.com/CMU-SAFARI/DAMOV.
  * [117] G. F. Oliveira _et al._ , “DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks,” arXiv:2105.03725 [cs.AR], 2021.
  * [118] B. C. Lee _et al._ , “Architecting Phase Change Memory as a Scalable DRAM Alternative,” in _ISCA_ , 2009.
  * [119] M. K. Qureshi _et al._ , “Scalable High Performance Main Memory System Using Phase-Change Memory Technology,” in _ISCA_ , 2009.
  * [120] B. C. Lee _et al._ , “Phase-Change Technology and the Future of Main Memory,” _IEEE Micro_ , 2010.
  * [121] B. C. Lee _et al._ , “Phase Change Memory Architecture and the Quest for Scalability,” _CACM_ , 2010.
  * [122] E. Kültürsay _et al._ , “Evaluating STT-RAM as an Energy-Efficient Main Memory Alternative,” in _ISPASS_ , 2013.
  * [123] P. Zhou _et al._ , “A Durable and Energy Efficient Main Memory Using Phase Change Memory Technology,” in _ISCA_ , 2009.
  * [124] H.-S. P. Wong _et al._ , “Phase Change Memory,” _Proc. IEEE_ , 2010\.
  * [125] J. Meza _et al._ , “A Case for Small Row Buffers in Non-Volatile Main Memories,” in _ICCD_ , 2012.
  * [126] J. Meza _et al._ , “A Case for Efficient Hardware/Software Cooperative Management of Storage and Memory,” in _WEED_ , 2013.
  * [127] S. Song _et al._ , “Improving Phase Change Memory Performance with Data Content Aware Access,” in _ISMM_ , 2020.
  * [128] S. Song _et al._ , “Aging-Aware Request Scheduling for Non-Volatile Main Memory,” in _ASP-DAC_ , 2021.
  * [129] S. Song _et al._ , “Enabling and Exploiting Partition-Level Parallelism (PALP) in Phase Change Memories,” _TECS_ , 2019.
  * [130] S. Bock _et al._ , “Analyzing the Impact of Useless Write-Backs on the Endurance and Energy Consumption of PCM Main Memory,” in _ISPASS_ , 2011\.
  * [131] G. W. Burr _et al._ , “Overview of Candidate Device Technologies for Storage-Class Memory,” _IBM JRD_ , 2008.
  * [132] Y. Du _et al._ , “Bit Mapping for Balanced PCM Cell Programming,” in _ISCA_ , 2013.
  * [133] L. Jiang _et al._ , “FPB: Fine-Grained Power Budgeting to Improve Write Throughput of Multi-Level Cell Phase Change Memory,” in _MICRO_ , 2012.
  * [134] L. Jiang _et al._ , “Hardware-Assisted Cooperative Integration of Wear-Leveling and Salvaging for Phase Change Memory,” _TACO_ , 2013.
  * [135] S. Kannan _et al._ , “Energy Aware Persistence: Reducing Energy Overheads of Memory-Based Persistence in NVMs,” in _PACT_ , 2016.
  * [136] M. K. Qureshi, “Pay-As-You-Go: Low-Overhead Hard-Error Correction for Phase Change Memories,” in _MICRO_ , 2011.
  * [137] M. K. Qureshi _et al._ , “Improving Read Performance of Phase Change Memories via Write Cancellation and Write Pausing,” in _HPCA_ , 2010.
  * [138] M. K. Qureshi _et al._ , “Morphable Memory System: A Robust Architecture for Exploiting Multi-Level Phase Change Memories,” in _ISCA_ , 2010.
  * [139] A. Sebastian _et al._ , “Temporal Correlation Detection Using Computational Phase-Change Memory,” _Nature Commun._ , 2017.
  * [140] R. Wang _et al._ , “Exploit Imbalanced Cell Writes to Mitigate Write Disturbance in Dense Phase Change Memory,” in _DAC_ , 2015.
  * [141] J. Yue and Y. Zhu, “Accelerating Write by Exploiting PCM Asymmetries,” in _HPCA_ , 2013.
  * [142] M. Zhou _et al._ , “Writeback-Aware Partitioning and Replacement for Last-Level Caches in Phase Change Main Memory Systems,” _TACO_ , 2012.
  * [143] M. Zhou _et al._ , “Writeback-Aware Bandwidth Partitioning for Multi-Core Systems with PCM,” in _PACT_ , 2013.
  * [144] H. Yoon _et al._ , “Techniques for Data Mapping and Buffering to Exploit Asymmetry in Multi-Level Cell (Phase Change) Memory,” SAFARI Research Group, Tech. Rep. TR-SAFARI-2013-002, 2013.
  * [145] G. Dhiman _et al._ , “PDRAM: A Hybrid PRAM and DRAM Main Memory System,” in _DAC_ , 2009.
  * [146] K. Wang _et al._ , “Low-Power Non-Volatile Spintronic Memory: STT-RAM and Beyond,” _J. Phys. D: Appl. Phys_ , 2013.
  * [147] E. Chen _et al._ , “Advances and Future Prospects of Spin-Transfer Torque Random Access Memory,” _TMAG_ , 2010.
  * [148] Z. Diao _et al._ , “Spin-Transfer Torque Switching in Magnetic Tunnel Junctions and Spin-Transfer Torque Random Access Memory,” _Journal of Physics: Condensed Matter_ , 2007.
  * [149] M. Hosomi _et al._ , “A Novel Nonvolatile Memory with Spin Torque Transfer Magnetization Switching: Spin-RAM,” in _IEDM_ , 2005.
  * [150] A. Raychowdhury _et al._ , “Design Space and Scalability Exploration of 1T-1STT MTJ Memory Arrays in the Presence of Variability and Disturbances,” in _IEDM_ , 2009.
  * [151] H.-S. P. Wong _et al._ , “Metal–Oxide RRAM,” _Proc. IEEE_ , 2012.
  * [152] J. J. Yang _et al._ , “Memristive Devices for Computing,” _Nature Nanotechnology_ , 2013.
  * [153] D. Bondurant, “Ferroelectronic RAM Memory Family for Critical Data Storage,” _Ferroelectrics_ , 1990.
  * [154] H. Yoon _et al._ , “Efficient Data Mapping and Buffering Techniques for Multilevel Cell Phase-Change Memories,” _TACO_ , 2014.
  * [155] S. Angizi _et al._ , “IMCE: Energy-Efficient Bitwise In-Memory Convolution Engine for Deep Neural Network,” in _ASP-DAC_ , 2018.
  * [156] M. F. Ali _et al._ , “In-Memory Low-Cost Bit-Serial Addition Using Commodity DRAM Technology,” in _TCAS-I_ , 2019.
  * [157] S. Li _et al._ , “SCOPE: A Stochastic Computing Engine for DRAM-Based In-Situ Accelerator,” in _MICRO_ , 2018.
  * [158] I. Goodfellow _et al._ , _Deep Learning_. MIT Press, 2016.
  * [159] K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv:1409.1556 [cs.CV], 2014.
  * [160] Y. LeCun _et al._ , “LeNet-5, Convolutional Neural Networks,” http://yann.lecun.com/exdb/lenet, 2015.
  * [161] Y. Lee, “Handwritten Digit Recognition Using k-Nearest-Neighbor, Radial-Basis Function, and Backpropagation Neural Networks,” _Neural Computation_ , 1991\.
  * [162] Transaction Processing Performance Council, “TPC-H,” http://www.tpc.org/tpch/.
  * [163] Y. Li and J. M. Patel, “BitWeaving: Fast Scans for Main Memory Data Processing,” in _SIGMOD_ , 2013.
  * [164] R. C. Gonzalez and R. E. Woods, _Digital Image Processing_ , 2nd ed. Addison-Wesley, 2002.
  * [165] N. Hajinazar _et al._ , “SIMDRAM: A Framework for Bit-Serial SIMD Processing Using DRAM,” in _ASPLOS_ , 2021.
  * [166] N. Hajinazar _et al._ , “SIMDRAM: An End-to-End Framework for Bit-Serial SIMD Computing in DRAM,” arXiv:2105.12839 [cs.AR], 2021.
  * [167] R. C. Murphy _et al._ , “The Characterization of Data Intensive Memory Workloads on Distributed PIM Systems,” in _Intelligent Memory Systems_. Springer, 2001.
  * [168] SAFARI Research Group, “PrIM Benchmark Suite,” https://github.com/CMU-SAFARI/prim-benchmarks.
  * [169] W. A. Simon _et al._ , “BLADE: An In-Cache Computing Architecture for Edge Devices,” _IEEE Transactions on Computers_ , 2020.
  * [170] A. V. Nori _et al._ , “REDUCT: Keep it Close, Keep it Cool!: Efficient Scaling of DNN Inference on Multi-Core CPUs with Near-Cache Compute,” in _ISCA_ , 2021.
  * [171] A. Nag _et al._ , “GenCache: Leveraging In-Cache Operators for Efficient Sequence Alignment,” in _MICRO_ , 2019.
  * [172] M. Wilkening _et al._ , “RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference,” in _ASPLOS_ , 2021.
  * [173] C. Y. Cho _et al._ , “XSD: Accelerating MapReduce by Harnessing the GPU Inside an SSD,” in _WoNDP_ , 2013.
  * [174] E. Riedel _et al._ , “Active Disks for Large-Scale Data Processing,” _IEEE Computer_ , 2001.
  * [175] E. Riedel _et al._ , “Active Disk Architecture for Databases,” Carnegie-Mellon University, Tech. Rep. CMU-CS-00-145, 2000.
  * [176] E. Riedel _et al._ , “Data Mining on an OLTP System (Nearly) for Free,” in _SIGMOD_ , 2000.
  * [177] E. Riedel, “Active Disks – Remote Execution for Network-Attached Storage,” Ph.D. dissertation, Carnegie Mellon University, 1999.
  * [178] E. Riedel _et al._ , “Active Storage for Large-Scale Data Mining and Multimedia Applications,” in _VLDB_ , 1998.
  * [179] E. Riedel, “Active Disks: Remote Execution for Network-Attached Storage,” Carnegie Mellon University, Tech. Rep. CMU-CS-97-198, 1997.
  * [180] K. Keeton _et al._ , “A Case for Intelligent Disks (IDISKs),” _SIGMOD_ , 1998.
  * [181] S.-W. Jun _et al._ , “BlueDBM: An Appliance for Big Data Analytics,” in _ISCA_ , 2015.
  * [182] Y. Wang _et al._ , “REREC: In-ReRAM Acceleration with Access-Aware Mapping for Personalized Recommendation,” in _ICCAD_ , 2021.
  * [183] S. Angizi _et al._ , “Exploring DNA Alignment-in-Memory Leveraging Emerging SOT-MRAM,” in _GLSVLSI_ , 2020.
  * [184] Y. Zha and J. Li, “Hyper-AP: Enhancing Associative Processing Through A Full-Stack Optimization,” in _ISCA_ , 2020.
  * [185] D. Fujiki _et al._ , “In-Memory Data Parallel Processor,” in _ASPLOS_ , 2018.
  * [186] A. E. Slade and H. O. McMahon, “A Cryotron Catalog Memory System,” in _WJCC_ , 1956.
  * [187] A. Krikelis and C. C. Weems, “Associative Processing and Processors,” _Computer_ , 1994.
  * [188] A. Olgun _et al._ , “PiDRAM: A Holistic End-to-End FPGA-Based Framework for Processing-in-DRAM,” arXiv:2111.00082 [cs.AR], 2021.
  * [189] F. N. Bostancı _et al._ , “DR-STRaNGe: End-to-End System Design for DRAM-Based True Random Number Generators,” in _HPCA_ , 2022.
  * [190] A. Olgun _et al._ , “QUAC-TRNG: High-Throughput True Random Number Generation using Quadruple Row Activation in Commodity DRAM Chips,” in _ISCA_ , 2021.
  * [191] SAFARI Research Group, “PiDRAM – GitHub Repository,” https://github.com/CMU-SAFARI/PiDRAM, 2021.
  * [192] G. F. Oliveira, “DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks – Short Talk,” https://www.youtube.com/watch?v=HkMYuYMuZOg.
  * [193] G. F. Oliveira, “DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks – Long Talk at SAFARI Live Seminar,” https://www.youtube.com/watch?v=GWideVyo0nM.
  * [194] G. F. Oliveira, “Tutorial on the DAMOV Framework and Benchmarks,” https://www.youtube.com/watch?v=GWideVyo0nM&t=8028s.
  * [195] N. Hajinazar and G. F. Oliveira, “SIMDRAM: A Framework for Bit-Serial SIMD Processing using DRAM – Talk at ASPLOS 2021,” https://www.youtube.com/watch?v=lu3Br4-kySw.


# As Cool as a Cucumber: Towards a Corpus of Contemporary Similes in Serbian

###### Abstract

Similes are natural language expressions used to compare unlikely things,
where the comparison is not taken literally. They are often used in everyday
communication and are an important part of cultural heritage. Having an up-to-
date corpus of similes is challenging, as they are constantly coined and/or
adapted to the contemporary times. In this paper we present a methodology for
semi-automated collection of similes from the world wide web using text mining
techniques. We expanded an existing corpus of traditional similes (containing
333 similes) by collecting 446 additional expressions. We, also, explore how
crowdsourcing can be used to extract and curate new similes.

Keywords: Phrase modelling, simile extraction, language resource building,
crowdsourcing

As Cool as a Cucumber: Towards a Corpus of Contemporary Similes in Serbian

Nikola Milošević, Goran Nenadić
---
School of Computer Science, University of Manchester
Kilburn building, M13 9PL, Manchester, UK,
<EMAIL_ADDRESS><EMAIL_ADDRESS>

Abstract content

##  1\. Introduction

Similes are figures of speech which compares two objects through the use of
specific connection words [Harris and others, 2002], but where comparison is
not intended to be taken literally (e.g ”hladan k’o krastavac” – as cool as a
cucumber). They are often used as metaphors, but connection words (such as
”like” and ”as”) are used explicitly [Cooper, 1986, Niculae, 2013]. In many
languages, similes are considered as an important building block of spoken
language and are part of cultural heritage. In Serbian, for example, Vuk
Karadžić attempted to collect similes, together with other short figures of
speech in the XIX century [Karadžić, 1849]. While some of the collected
similes are still used (e.g. ”bela kao sneg” – white as snow), some have
evolved (e.g. ”gladan kao kurjak” – hungry as a wolf is now more often
appearing as ”gladan kao vuk” – hungry as a wolf) and some are rarely used and
may not even be understood with the current speakers (e.g. ”crveni se kao
oderano goveče” – red as skinned ox). Of course, a number of new similes have
emerged. During the XX century, there were attempts to collect proverbs and
other short forms from particular geographical areas [Marković, 1979, Jevtić,
1969, Vukanović, 1983, Cvetanović, 1980], but little has been done in updating
similes.

To the best of our knowledge, there is no large-scale text mining approach to
collecting similes in Serbian, but there have been attempts to collect them in
other languages. For example, Veale and Hao [Veale and Hao, 2007] created an
approach for automated extraction of explicit similes in English. They
modelled them as ”X is as P as Y”, where X and Y are nouns and P is an
adjective. Bin et al. [Li et al., 2012] used similar approach, querying Baidu
search engine for Chinese similes. Aasheim [Aasheim, 2012] differentiates two
types of similes: nominal (like a/an + noun) and adjectival (as + adjective +
as + noun). In Slavic languages, simile expressions are based on the
preposition ”like” or ”as” (kak in Russian, kao in Serbian, ako in Slovak,
etc.) and introduce a noun phrase (NP) agreeing in case with the standard of
comparison [Rappaport, 1998].

In this paper we explore how we can update the corpus of similes in Serbian
using the World Wide Web and crowd-sourcing. The World Wide Web in particular
provides people with novel ways to express themselves on a variety of media
such as Wikis, question-answer databases, blogs, forums, review-sites and
social media. As such, it is an increasingly important resource for exploring
linguistic changes and trends, especially as its form is in-between written
and spoken languages. Our semi-automated methodology utilises natural language
processing, machine learning techniques and manual curation to ensure the
validity of extracted data.

##  2\. Method

###  2.1. Method Overview

Figure 1: Overview of the proposed methodology: 1) Document retrieval using a
web crawler, 2) Comparison extraction, based on part-of-speech tagging, 3)
Classification of similes; 4) Data curation with a human curator. Users can
also propose missing similes.

The methodology consists of four steps (see Figure 1): firstly we collect the
documents for processing; in the second step we process these documents and
extract all comparisons; in the third step, we use machine learning to
distinguish similes from the other expressions that function as comparisons;
in the final step, a human curator reviews the data and corrects any mistakes.
Also, users of our crowdsourcing website can propose new similes and help to
collect the similes that were not collected by the text mining pipeline; the
curator checks these manually.

###  2.2. Crawler

In order to collect texts from which to mine similes, we have developed a
number of crawlers crafted for particular websites. The aim was to download
all potentially meaningful text from these websites, but skip contentless
parts or parts where similes are unlikely, such as menus, headings and
footers. In order to do that, each crawler extracts only text that is inside
the div HTML tag with a particular id. Crawlers follow links on each page they
visit, but not outside the domain of the website.

We have developed separate crawlers for the following four domains:

  * •

laguna.rs \- one of the biggest book publishers with abstracts, reviews and
parts of the books on their website;

  * •

rastko.rs \- a project that aims to make a digital library of books and
articles in Serbian that are considered cultural heritage;

  * •

burek.com \- a large general public forum on Serbian;

  * •

tarzanija.com \- a popular blog portal with sarcastic comments on various
issues.

For crawling and processing we used Scrapy, an open source Python framework
for extracting data from website111http://scrapy.org/.

###  2.3. Modelling and extracting candidate similes

The connection word used in similes in Serbian is “kao” - like, as”, but it
also often appears in shorter forms ”k’o” or ”ko” (misspelled). There are two
main categories of simile in Serbian:

  * •

Adjectival: Adjective + Connection word (”kao”) + Noun Phrase (e.g. ”lep kao
cvet” – beautiful as a flower)

  * •

Verbal: Verb + Connection word (”kao”) + Noun Phrase (e.g. ”radi kao konj” –
”works like a horse”)

Both of these categorise can be modelled by this expression:

$(V|A|V\ se)\ (kao|ko|k^{\prime}o)\ (NP)$ (1)

A typical noun phrase consists of a noun together with zero or more dependents
of various types, such as determiners, adjectives, adjective phrases, noun
adjuncts, prepositional phrases, participle phrases or pronouns [Crystal,
2011]. However, currently publicly available part-of-speech taggers and
parsers for Serbian can tag only individual words, but not phrases. Because of
this, it was not possible to rely on a tagger to identify noun phrases. We
therefore modelled candidate similes as a verb or adjective followed by
explicit use of connection words ”kao”, ”k’o” or ”ko”, followed by one or more
adjectives and terminated with a noun:

$(V|A|V\ se)\ (kao|ko|k^{\prime}o)\ ((A|N)*)\ (N)$ (2)

The noun phrase model that we used models the majority of commonly used noun
phrases in Serbian. However, noun phrases and similes often contain
prepositions. e.g. ”smoren kao zmaj u vatrogasnoj stanici” (bored like a
dragon at a fire station). Our model will currently pick up ”smoren kao zmaj”
(bored like a dragon), which is also a simile used quite frequently, but we
would rely on manual curation to extend it with the prepositional phrase.

In the next step of the pipeline, we use retrieved textual documents to
extract candidate comparisons. We used a part of speech (POS) model for
Croatian and Serbian [Agić et al., 2013] and plugged it into the HunPos tagger
[Halácsy et al., 2007]. We used the model 2 for matching similes.

###  2.4. Simile classifier

After a review of initial outputs, we noted a large number of false positives.
The model would find similes such as ”radi kao konj” (works like a horse), but
it would also extract expressions like ”radi kao pravnik” (works as a lawyer),
which is not a comparison, but rather specifies someone’s profession. There is
no lexical feature that would give a cue that these two phrases are in any way
different and only semantics (and context) would differentiate the similes
from other expressions. However, current publicly available tools for Serbian
do not process phrases at the semantic level and thus we decided to rely on a
classifier and a final curator’s manual review of the phrases.

From the initial data, we have created a sufficient number of true positive
and false positive examples that could be used for machine learning. This data
was used to create a machine-learning based classifier that can distinguish
true similes from the other phrases that have same lexical characteristics.
The dataset contained 300 examples of true positive and 300 examples of false
positive similes to train and test our algorithm. For the features, we used

  * •

the whole simile phrase (for illustration we will use ”radi kao konj” – ”works
like a horse”),

  * •

the stemmed phrase (”rad ka konj”),

  * •

left side of the phrase that is before the connection word (”radi” – works),

  * •

left side stemmed (”rad”),

  * •

the part of phrase that is on the right from the connection word (”konj” –
horse), and

  * •

the stemmed right part (”konj”).

For stemming we used stemmer for Serbian [Milošević, 2012], that was ported
for Python222https://github.com/nikolamilosevic86/SerbianStemmer. We
experimented with Multinominal Naive Bayes, Random Forests and Support Vector
Machines (SVM) with a polynomial kernel that uses sequential minimal
optimisation. For machine learning we used Weka toolkit [Hall et al., 2009].
The models are downloadable from the corpus
website333http://ezbirka.starisloveni.com/download.html.

###  2.5. Curation interface

We have created a web portal444http://ezbirka.starisloveni.com for viewing,
searching and editing the dataset collected from the previous steps. The
portal has the following curation features:

  * •

Viewing of all currently curated similes, sorted alphabetically

  * •

Searching for simile. Searches are performed by using the stemmer for Serbian,
so the similes that are not in exactly the same form in the database can be
retrieved. Since Serbian is highly inflectional, the aim was to store only one
canonical instance (single inflection) of each simile. For example ”beo kao
sneg” (m), ”bela kao sneg” (f) and ”belo kao sneg” (n) (”white as a snow” in
three different grammatical genders - masculine, feminine, neuter), will be
treated by our search algorithm as the expression. Search interface can be
seen in Figure 2.

Figure 2: Simile search interface

  * •

Adding a new simile to the database. Users of the website are able to manually
add similes that are rare or missing from the corpus. If the user wants to add
simile that already exists in the database, he/she will be notified. Added
simile won’t be visible on the website until curator approves it.

  * •

Curation interface. Curators can login to the website and perform curation
task, such as approving, rejecting or editing similes.

##  3\. Results

The crawlers downloaded 40,239 documents from the web (See Table 1). Using
semi-automated workflow, we extracted potential similes and manually reviewed
them. A total of 446 true similes were finally harvested. A manual review of a
data also revealed that there were around 5,000 false positives. Our
classification approach proved to be useful in filtering some of these out -
the results are presented in Table 2.

Website | Number of documents
---|---
burek.com | 22,925
laguna.rs | 9,574
tarzanija.com | 4,792
rastko.rs | 2,947
Total | 40,239
Table 1: Number of documents in dataset obtained by crawlers Algorithm | Precision | Recall | F-Measure
---|---|---|---
Random Forests | 0.783 | 0.732 | 0.756
Naive Bayes | 0.782 | 0.773 | 0.777
SVM | 0.804 | 0.791 | 0.797
Table 2: Results of machine learning based classification of similes

We merged the similes obtained from the web with the simile corpus published
by Vuk Karadžić [Karadžić, 1849], who collected 333 similes. It is interesting
that there was only a small overlap between the two simile datasets. Overlap
between these two corpora can be seen in Table 4 The similes collected by
Karadžić were generally more complex, which could be due to the termination
rule on noun in our model, so the model might be missing some complex similes.
Although, this is an obvious limitation of the system, collection of complex
similes will be part of the future work, and also will rely on crowdsourcing
interface. At the moment of writing this paper, we had 852 approved similes in
our corpus.

Corpus | Number of similes
---|---
WWW Corpus | 446
Karadžić’s Corpus | 333
Intersection between Karadžić’s |
and WWW Corpus | 9
Manually added similes | 80
Total similes | 852
Table 3: Number of similes obtained from World Wide Web, Karadžić’s corpus and initial crowdsourcing. Also, number of similes that appear in both WWW and Karadžić’s corpus is presented Original simile | Translation
---|---
beo kao sneg | white as a snow
brz kao zec | fast like a rabbit
crven kao vampir | red like vampire
crven kao krv | red like blood
miran kao ovca | calm as a sheep
lako kao pero | light as a feather
ljut kao ris | angry like a lynx
slatko kao med | sweet like honey
tvrd kao kamen | heavy like a stone
Table 4: Intersection between similes found on the WWW and in Karadžić’s
collection

##  4\. Summary

In this paper we present initial work towards building a corpus of similes in
Serbian and a methodology for a semi-automated creation of simile corpus. We
have kick-started the process by processing a set of web sites that are likely
to have a number of similes. The current corpus, with 852 similes is, to the
best of our knowledge, the largest simile corpus in Serbian. We believe that,
by mining more web pages for similes and by the use of crowdsourcing, the
corpus will grow and remain up to date. Still, the current methodology will
need to be improved through the use of semantic resources (e.g. WordNet) and
contextualisation.

The International Standard Language Resource Number (ISLRN) of the Serbian
Simile Corpus is 534-837-568-404-3. It is freely available at
http://www.ezbirka.starisloveni.com/. The code of the simile extractor can be
found on
GitHub555https://github.com/nikolamilosevic86/SerbianComparisonExtractor.

## References

  * Aasheim, 2012 Aasheim, I.  (2012).  A contrastive study of similes in English and Norwegian.  UNIVERSITY OF OSLO,A master thesis presented to the Department of Literature, Area Studies and European Languages.
  * Agić et al., 2013 Agić, Ž., Ljubešić, N., and Merkler, D.  (2013).  Lemmatization and morphosyntactic tagging of croatian and serbian.  In Proceedings of ACL.
  * Cooper, 1986 Cooper, D. E.  (1986).  Metaphor (Aristotelian Society Series).  Blackwell Pub.
  * Crystal, 2011 Crystal, D.  (2011).  Dictionary of linguistics and phonetics, volume 30.  John Wiley & Sons.
  * Cvetanović, 1980 Cvetanović, V.  (1980).  Narodna književnost Srba na Kosovu. 7, Narodne poslovice.  Jedinstvo, Priština.
  * Halácsy et al., 2007 Halácsy, P., Kornai, A., and Oravecz, C.  (2007).  Hunpos: an open source trigram tagger.  In Proceedings of the 45th annual meeting of the ACL on interactive poster and demonstration sessions, pages 209–212. Association for Computational Linguistics.
  * Hall et al., 2009 Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H.  (2009).  The weka data mining software: an update.  ACM SIGKDD explorations newsletter, 11(1):10–18.
  * Harris and others, 2002 Harris, R. et al.  (2002).  A handbook of rhetorical devices.  Virtual Salt.
  * Jevtić, 1969 Jevtić, M.  (1969).  Narodne poslovice i uzrecice iz Podrinja.  Godišnjak istorijskog muzeja u Šabcu.
  * Karadžić, 1849 Karadžić, V. S.  (1849).  Srpske narodne poslovice i druge ralične kao i one u običaj uzete riječi.
  * Li et al., 2012 Li, B., Kuang, H., Zhang, Y., Chen, J., and Tang, X.  (2012).  Using similes to extract basic sentiments across languages.  In Web Information Systems and Mining - International Conference, (WISM) 2012, Chengdu, China, October 26-28, 2012. Proceedings, pages 536–542.
  * Marković, 1979 Marković, R.  (1979).  Metaforisane i nove poslovice Šumadije.  Narodni muzej, Čacak.
  * Milošević, 2012 Milošević, N.  (2012).  Stemmer for serbian language.  arXiv preprint arXiv:1209.4471.
  * Niculae, 2013 Niculae, V.  (2013).  Comparison pattern matching and creative simile recognition.  In Proceedings of the Joint Symposium on Semantic Processing.
  * Rappaport, 1998 Rappaport, G.  (1998).  The slavic noun phrase.  Position paper for Comparative Slavic Morphosyntax]. Available: http://www. indiana. edu/~ slavconf/linguistics/download. html [1999, Jan. 4].
  * Veale and Hao, 2007 Veale, T. and Hao, Y.  (2007).  Making lexical ontologies functional and context-sensitive.  In ANNUAL MEETING-ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, volume 45, page 57.
  * Vukanović, 1983 Vukanović, T.  (1983).  Vranjanske narodne poslovice.  Nova Jugoslavija, Vranje.