to test the next $HC$ value in the sweep by restoring the observed bit flips
to their original values (Line 16) depending on the data pattern ($DP$) being
tested.

Fairly Comparing Data Across Infrastructures. Our carefully-crafted RowHammer
test routine allows us to compare our test results between the two different
testing infrastructures. This is because, as we described earlier, we 1)
reverse engineer the row address mappings of each DRAM configuration such that
we effectively test double-sided RowHammer on every single row, 2) issue
activations as fast as possible for each chip, such that the activation rates
are similar across infrastructures, and 3) disable all sources of interference
in our RowHammer tests.

###  6.4 RowHammer Characterization

In this section, we present our comprehensive characterization of RowHammer on
the 1580 DRAM chips we test.111We list our full set of chips in Appendix A of
our extended technical report [166].

####  6.4.1 RowHammer Vulnerability

We first examine which of the chips that we test are susceptible to RowHammer.
Across all of our chips, we sweep the hammer count ($HC$) between 2K and 150K
(i.e., 4k and 300k activates for our double-sided RowHammer test) and observe
whether we can induce any RowHammer bit flips at all in each chip. We find
that we can induce RowHammer bit flips in all chips except many DDR3 chips.
Table 6.2 shows the fraction of DDR3 chips in which we _can_ induce RowHammer
bit flips (i.e., _RowHammerable_ chips).

Table 6.2: Fraction of DDR3 DRAM chips vulnerable to RowHammer when $\bm{HC}\,\mathbf{<150k}$. DRAM | RowHammerable chips
---|---
type-node | Mfr. A | Mfr. B | Mfr. C
DDR3-old | 24/88 | 0/88 | 0/28
DDR3-new | 8/72 | 44/52 | 96/104

Observation 1. _Newer DRAM chips appear to be more vulnerable to RowHammer
based on the increasing fraction of RowHammerable chips from DDR3-old to
DDR3-new DRAM chips of manufacturers B and C._

We find that the fraction of manufacturer A’s chips that are RowHammerable
decreases from DDR3-old to DDR3-new chips, but we also note that the number of
RowHammer bit flips that we observe across each of manufacturer A’s chips is
very low ($<20$ on average across RowHammerable chips) compared to the number
of bit flips found in manufacturer B and C’s DDR3-new chips ($87k$ on average
across RowHammerable chips) when $HC=150K$. Since DDR3-old chips of all
manufacturers and DDR3-new chips of manufacturer A have very few to no bit
flips, we refrain from analyzing and presenting their characteristics in many
plots in Section 6.4.

####  6.4.2 Data Pattern Dependence

To study data pattern effects on observable RowHammer bit flips, we test our
chips using Algorithm 6 with $hammer\\_count\ (HC)=150k$ at $50^{\circ}C$,
sweeping the 1) $victim\\_row$ and 2) $data\\_pattern$ (as described in
Section 6.3.3).222Note that for a given data pattern ($DP$), the same data is
always written to $victim\\_row$. For example, when testing Rowstripe0, every
byte in $victim\\_row$ is always written with 0x00 and every byte in the two
physically-adjacent rows are written with 0xFF.

We first examine the set of all RowHammer bit flips that we observe when
testing with different data patterns for a given $HC$. For each data pattern,
we run our RowHammer test routine ten times. We then aggregate all unique
RowHammer bit flips per data pattern. We combine all unique RowHammer bit
flips found by all data patterns and iterations into a full set of observable
bit flips. Using the combined data, we calculate the fraction of the full set
of observable bit flips that each data pattern identifies (i.e., the data
pattern’s _coverage_). Figure 6.2 plots the coverage (y-axis) per individual
data pattern (shared x-axis) for a single representative DRAM chip from each
DRAM type-node configuration that we test. Each row of subplots shows the
coverages for chips of the same manufacturer (indicated on the right y-axis),
and the columns show the coverages for chips of the same DRAM type-node
configuration (e.g., DDR3-new).

Figure 6.2: RowHammer bit flip coverage of different data patterns (described
in Section 6.3.3) for a single representative DRAM chip of each type-node
configuration.

Observation 2. _Testing with different data patterns is essential for
comprehensively identifying RowHammer bit flips because no individual data
pattern achieves full coverage alone._

Observation 3. _The worst-case data pattern (shown in Table 6.3) is consistent
across chips of the same manufacturer and DRAM type-node configuration._333We
do not consider the true/anti cell pattern of a chip [206, 170, 87] and
agnostically program the data pattern accordingly into the DRAM array. More
RowHammer bit flips can be induced by considering the true/anti-cell pattern
of each chip and devising corresponding data patterns to exploit this
knowledge [87].

Table 6.3: Worst-case data pattern for each DRAM type-node configuration at $\mathbf{50^{\circ}C}$ split into different manufacturers. DRAM | Worst Case Data Pattern at $\mathbf{50^{\circ}C}$
---|---
type-node | Mfr. A | Mfr. B | Mfr. C
DDR3-new | N/A | Checkered0 | Checkered0
DDR4-old | RowStripe1 | RowStripe1 | RowStripe0
DDR4-new | RowStripe0 | RowStripe0 | Checkered1
LPDDR4-1x | Checkered1 | Checkered0 | N/A
LPDDR4-1y | RowStripe1 | N/A | RowStripe1

We believe that different data patterns induce the most RowHammer bit flips in
different chips because DRAM manufacturers apply a variety of proprietary
techniques for DRAM cell layouts to maximize the cell density for different
DRAM type-node configurations. For the remainder of this chapter, we
characterize each chip using _only_ its worst-case data pattern.444We use the
worst-case data pattern to 1) minimize the extensive testing time, 2) induce
many RowHammer bit flips, and 3) experiment at worst-case conditions. A
diligent attacker would also try to find the worst-case data pattern to
maximize the probability of a successful RowHammer attack.

####  6.4.3 Hammer Count ($HC$) Effects

We next study the effects of increasing the hammer count on the number of
observed RowHammer bit flips across our chips. Figure 6.3 plots the effects of
increasing the number of hammers on the RowHammer bit flip rate555We define
the RowHammer bit flip rate as the number of observed RowHammer bit flips to
the total number of bits in the tested DRAM rows. for our tested DRAM chips of
various DRAM type-node configurations across the three major DRAM
manufacturers. For all chips, we hammer each row, sweeping $HC$ between 10,000
and 150,000. For each $HC$ value, we plot the average rate of observed
RowHammer bit flips across all chips of a DRAM type-node configuration.

Figure 6.3: Hammer count ($HC$) vs. RowHammer bit flip rate across DRAM type-
node configurations.

Observation 4. _The log of the number of RowHammer bit flips has a linear
relationship with the log of $HC$._666Our observation is consistent with prior
work [257].

We observe this relationship between $HC$ and RowHammer bit flip rate because
more accesses to a single row results in more cell-to-cell interference, and
therefore more charge is lost in victim cells of nearby rows.

We examine the effects of DRAM technology node on the RowHammer bit flip rate
in Figure 6.3. We observe that the bit flip rate curve shifts _upward_ and
_leftward_ when going from DDR4-old to DDR4-new chips, indicating
respectively, 1) a higher rate of bit flips for the same $HC$ value and 2)
occurrence of bit flips at lower $HC$ values, as technology node size reduces
from DDR4-old to DDR4-new.

Observation 5. _Newer DDR4 DRAM technology nodes show a clear trend of
increasing RowHammer bit flip rates: the _same_ $HC$ value causes an increased
average RowHammer bit flip rate from DDR4-old to DDR4-new DRAM chips of all
DRAM manufacturers._

We believe that due to increased density of DRAM chips from older to newer
technology node generations, cell-to-cell interference increases and results
in DRAM chips that are more vulnerable to RowHammer bit flips.

####  6.4.4 RowHammer Spatial Effects

We next experimentally study the spatial distribution of RowHammer bit flips
across our tested chips. In order to normalize the RowHammer effects that we
observe across our tested chips, we first take each DRAM chip and use a hammer
count specific to that chip to result in a RowHammer bit flip rate of
$10^{-6}$.777We choose a RowHammer bit flip rate of $10^{-6}$ since we are
able to observe this bit flip rate in most chips that we characterize with
$HC<150k$. For each chip, we analyze the spatial distribution of bit flips
throughout the chip. Figure 6.4 plots the fraction of RowHammer bit flips that
occur in a given row offset from the $victim\\_row$ out of all observed
RowHammer bit flips. Each column of subplots shows the distributions for chips
of different manufacturers and each row of subplots shows the distribution for
a different DRAM type-node configuration. The error bars show the standard
deviation of the distribution across our tested chips. Note that the
repeatedly-accessed rows (i.e., _aggressor rows_) are at $x=1$ and $x=-1$ for
all plots except in LPDDR4-1x chips from manufacturer B, where they are at
$x=-2$ and $x=2$ (due to the internal address remapping that occurs in these
chips as we describe in Section 6.3.3). Because an access to a row essentially
refreshes the data in the row, repeatedly accessing aggressor rows during the
core loop of the RowHammer test prevents any bit flips from happening in the
aggressor rows. Therefore, there are no RowHammer bit flips in the aggressor
rows across each DRAM chip in our plots (i.e., $y=0$ for $x=[-2,-1,2,3]$ for
LPDDR4-1x chips from manufacturer B and for $x=1$ and $x=-1$ for all other
chips).

Figure 6.4: Distribution of RowHammer bit flips across row offsets from the
victim row.

We make three observations from Figure 6.4. First, we observe a general trend
across DRAM type-node configurations of a given DRAM manufacturer where newer
DRAM technology nodes have an increasing number of rows that are susceptible
to RowHammer bit flips that are _farther_ from the victim row. For example, in
LPDDR4-1y chips, we observe RowHammer bit flips in as far as 6 rows from the
victim row (i.e., $x=-6$), whereas in DDR3 and DDR4 chips, RowHammer bit flips
only occur in as far as 2 rows from the victim row (i.e., $x=-2$). We believe
that this effect could be due to 1) an increase in DRAM cell density, which
leads to cell-to-cell interference extending farther than a single row, with
RowHammer bit flips occurring in rows increasingly farther away from the
aggressor rows (e.g., 5 rows away) for higher-density chips, and 2) more
shared structures internal to the DRAM chip, which causes farther (and
multiple) rows to be affected by circuit-level interference.

Observation 6. _For a given DRAM manufacturer, chips of newer DRAM technology
nodes can exhibit RowHammer bit flips 1) in more rows and 2) farther away from
the victim row._

Second, we observe that rows containing RowHammer bit flips that are farther
from the victim row have fewer RowHammer bit flips than rows closer to the
victim row. Non-victim rows adjacent to the aggressor rows ($x=2$ and $x=-2$)
contain RowHammer bit flips, and these bit flips demonstrate the effectiveness
of a single-sided RowHammer attack as only one of their adjacent rows are
repeatedly accessed. As discussed earlier (Section 6.3.3), the single-sided
RowHammer attack is not as effective as the double-sided RowHammer attack, and
therefore we find fewer bit flips in these rows. In rows farther away from the
victim row, we attribute the diminishing number of RowHammer bit flips to the
diminishing effects of cell-to-cell interference with distance.

Observation 7. _The number of RowHammer bit flips that occur in a given row
decreases as the distance from the victim row increases._

Third, we observe that only even-numbered offsets from the victim row contain
RowHammer bit flips in all chips except LPDDR4-1x chips from Manufacturer B.
However, the rows containing RowHammer bit flips in Manufacturer B’s LPDDR4-1x
chips would be even-numbered offsets if we translate all rows to physical rows
based on our observation in Section 6.3.3 (i.e., divide each row number by 2
and round down). While we are uncertain why we observe RowHammer bit flips
only in physical even-numbered offsets from the victim row, we believe that it
may be due to the internal circuitry layout of DRAM rows.

We next study the spatial distribution of RowHammer-vulnerable DRAM cells in a
DRAM array using the same set of RowHammer bit flips. Figure 6.5 shows the
distribution of 64-bit words containing x RowHammer bit flips across our
tested DRAM chips. We find the proportion of 64-bit words containing x
RowHammer bit flips out of all 64-bit words in each chip containing any
RowHammer bit flip and plot the distribution as a bar chart with error bars
for each x value.

Figure 6.5: Distribution of the number of RowHammer bit flips per 64-bit word
for each DRAM type-node configuration.

Observation 8. _At a RowHammer bit flip rate of $10^{-6}$, a single 64-bit
value can contain up to four RowHammer bit flips._

Because ECC [151, 6, 248, 259] is typically implemented for DRAM at a 64-bit
granularity (e.g., a single-error correcting code would only protect a 64-bit
word if it contains at most one error), observation 8 indicates that even at a
relatively low bit flip rate of $10^{-6}$, a DRAM chip can only be protected
from RowHammer bit flips with a strong ECC code (e.g., 4-bit error correcting
code), which has high hardware overhead.

Observation 9. _The distribution of RowHammer bit flip density per word
changes significantly in LPDDR4 chips compared to other DRAM types._

We find DDR3 and DDR4 chips across all manufacturers to exhibit an exponential
decay curve for increasing RowHammer bit flip densities with most words
containing only one RowHammer bit flip. However, LPDDR4 chips across all
manufacturers exhibit a much smaller fraction of words containing a single
RowHammer bit flip and significantly larger fractions of words containing two
and three RowHammer bit flips compared to DDR3 and DDR4 chips. We believe this
change in the bit flip density distribution is due to the on-die ECC that
manufacturers have included in LPDDR4 chips [151, 6, 248, 259], which is a
128-bit single-error correcting code that corrects and hides _most_ single-bit
failures within a 128-bit ECC word using redundant bits (i.e., _parity-check
bits_) that are hidden from the system.

With the failure rates at which we test, many ECC words contain several bit
flips. This exceeds the ECC’s correction strength and causes the ECC logic to
behave in an undefined way. The ECC logic may 1) correct one of the bit flips,
2) do nothing, or 3) introduce an _additional_ bit flip by corrupting an
error-free data bit [304, 259]. On-die ECC makes single-bit errors rare
because 1) any true single-bit error is immediately corrected and 2) a multi-
bit error can _only_ be reduced to a single-bit error when there are no more
than two bit flips within the data bits _and_ the ECC logic’s undefined action
happens to change the bit flip count to exactly one. In contrast, there are
many more scenarios that yield two or three bit-flips within the data bits,
and a detailed experimental analysis of how on-die ECC affects DRAM failure
rates in LPDDR4 DRAM chips can be found in [259].

####  6.4.5 First RowHammer Bit Flips

We next study the vulnerability of each chip to RowHammer. One critical
component of vulnerability to the double-sided RowHammer attack [170] is
identifying the weakest cell, i.e., the DRAM cell that fails with the fewest
number of accesses to physically-adjacent rows. In order to perform this
study, we sweep $HC$ at a fine granularity and record the $HC$ that results in
the first RowHammer bit flip in the chip ($HC_{\scalebox{0.7}{first}}$).
Figure 6.6 plots the distribution of $HC_{\scalebox{0.7}{first}}$ across all
tested chips as box-and-whisker plots.888A box-and-whiskers plot emphasizes
the important metrics of a dataset’s distribution. The box is lower-bounded by
the first quartile (i.e., the median of the first half of the ordered set of
data points) and upper-bounded by the third quartile (i.e., the median of the
second half of the ordered set of data points). The median falls within the
box. The _inter-quartile range_ (IQR) is the distance between the first and
third quartiles (i.e., box size). Whiskers extend an additional $1.5\times
IQR$ on either sides of the box. We indicate outliers, or data points outside
of the range of the whiskers, with pluses. The subplots contain the
distributions of each tested DRAM type-node configuration for the different
DRAM manufacturers. The x-axis organizes the distributions by DRAM type-node
configuration in order of age (older on the left to younger on the right). We
further subdivide the subplots for chips of the same DRAM type (e.g., DDR3,
DDR4, LPDDR4) with vertical lines. Chips of the same DRAM type are colored
with the same color for easier visual comparison across DRAM manufacturers.

Figure 6.6: Number of hammers required to cause the first RowHammer bit flip
($HC_{first}$) per chip across DRAM type-node configurations.

Observation 10. _Newer chips from a given DRAM manufacturer appear to be more
vulnerable to RowHammer bit flips. This is demonstrated by the clear reduction
in $HC_{\scalebox{0.7}{first}}$ values from old to new DRAM generations (e.g.,
LPDDR4-1x to LPDDR4-1y in manufacturer A, or DDR4-old to DDR4-new in
manufacturers A and C)._

We believe this observation is due to DRAM technology process scaling wherein
both 1) DRAM cell capacitance reduces and 2) DRAM cell density increases as
technology node size reduces. Both factors together lead to more interference
between cells and likely faster charge leakage from the DRAM cell’s smaller
capacitors, leading to a higher vulnerability to RowHammer. We find two
exceptions to this trend (i.e., a general increase in
$HC_{\scalebox{0.7}{first}}$ from DDR3-old to DDR3-new chips of manufacturer A
and from DDR4-old to DDR4-new chips of manufacturer B), but we believe these
potential anomalies may be due to our inability to identify explicit
manufacturing dates and correctly categorize these particular chips.

Observation 11. _In LPDDR4-1y chips from manufacturer A, there are chips whose
weakest cells fail after only 4800 hammers._

This observation has serious implications for the future as DRAM technology
node sizes will continue to reduce and $HC_{\scalebox{0.7}{first}}$ will only
get smaller. We discuss these implications further in Section 6.5. Table 6.4
shows the lowest observed $HC_{\scalebox{0.7}{first}}$ value for any chip
within a DRAM type-node configuration (i.e., the minimum values of each
distribution in Figure 6.6).

Table 6.4: Lowest $\bm{HC_{first}}$ values ($\mathbf{\times 1000}$) across all chips of each DRAM type-node configuration. DRAM |  $\bm{HC_{first}}$ (Hammers until first bit flip) $\mathbf{\times 1000}$
---|---
type-node |  Mfr. A |  Mfr. B |  Mfr. C
DDR3-old |  69.2 |  157 |  155
DDR3-new |  85 |  22.4 |  24
DDR4-old |  17.5 |  30 |  87
DDR4-new |  10 |  25 |  40
LPDDR4-1x |  43.2 |  16.8 |  N/A
LPDDR4-1y |  4.8 |  N/A |  9.6

Effects of ECC. The use of error correcting codes (ECC) to improve the
reliability of a DRAM chip is common practice, with most system-level [29, 66,
100, 167] or on-die [6, 180, 151, 259, 183] ECC mechanisms providing single
error correction capabilities at the granularity of 64- or 128-bit words. We
examine 64-bit ECCs since, for the same correction capability (e.g., single-
error correcting), they are stronger than 128-bit ECCs. In order to determine
the efficacy with which ECC can mitigate RowHammer effects on real DRAM chips,
we carefully study three metrics across each of our chips: 1) the lowest $HC$
required to cause the first RowHammer bit flip (i.e.,
$HC_{\scalebox{0.7}{first}}$) for a given chip (shown in Figure 6.6), 2) the
lowest $HC$ required to cause at least two RowHammer bit flips (i.e.,
$HC_{\scalebox{0.7}{second}}$) within any 64-bit word, and 3) the lowest $HC$
required to cause at least three RowHammer bit flips (i.e.,
$HC_{\scalebox{0.7}{third}}$) within any 64-bit word. These quantities tell
us, for ECCs of varying strengths (e.g., single-error correction code, double-
error correction code), at which $HC$ values the ECC can 1) mitigate RowHammer
bit flips and 2) no longer reliably mitigate RowHammer bit flips for that
particular chip.

Figure 6.7 plots as a bar graph the $HC$ (left y-axis) required to find the
first 64-bit word containing one, two, and three RowHammer bit flips (x-axis)
across each DRAM type-node configuration. The error bars represent the
standard deviation of $HC$ values across all chips tested. On the same figure,
we also plot with red boxplots, the increase in $HC$ (right y-axis) between
the $HC$s required to find the first 64-bit word containing one and two
RowHammer bit flips, and two and three RowHammer bit flips. These multipliers
indicate how $HC_{\scalebox{0.7}{first}}$ would change in a chip if the chip
uses single-error correcting ECC or moves from a single-error correcting to a
double-error correcting ECC. Note that we 1) leave two plots (i.e., Mfr. A
DDR3-new and Mfr. C DDR4-old) empty since we are unable to induce enough
RowHammer bit flips to find 64-bit words containing more than one bit flip in
the chips and 2) do not include data from our LPDDR4 chips because they
already include on-die ECC [6, 180, 151, 259, 183], which obfuscates errors
potentially exposed to any other ECC mechanisms [259].

Figure 6.7: Hammer Count (left y-axis) required to find the first 64-bit word
containing one, two, and three RowHammer bit flips. Hammer Count Multiplier
(right y-axis) quantifies the $HC$ difference between every two points on the
x-axis (as a multiplication factor of the left point to the right point).

Observation 12. _A single-error correcting code can significantly improve
$HC_{\scalebox{0.7}{first}}$ by up to $2.78\times$ in DDR4-old and DDR4-new
DRAM chips, and $1.65\times$ in DDR3-new DRAM chips._

Observation 13. _Moving from a double-error correcting code to a triple-error
correcting code has diminishing returns in DDR4-old and DDR4-new DRAM chips
(as indicated by the reduction in the $HC$ multiplier) compared to when moving
from a single-error correcting code to a double-error correcting code.
However, using a triple-error correcting code in DDR3-new DRAM chips continues
to further improve the $HC_{\scalebox{0.7}{first}}$ and thus reduce the DRAM
chips’ vulnerability to RowHammer._

####  6.4.6 Single-Cell RowHammer Bit Flip Probability

We examine how the failure probability of a single RowHammer bit flip changes
as $HC$ increases. We sweep $HC$ between 25k to 150k with a step size of 5k
and hammer each DRAM row over 20 iterations. For each $HC$ value, we identify
each cell’s bit flip probability (i.e., the number of times we observe a
RowHammer bit flip in that cell out of all 20 iterations). We then observe how
each cell’s bit flip probability changes as $HC$ increases. We expect that by
exacerbating the RowHammer conditions (e.g., increasing the hammer count), the
exacerbated circuit-level interference effects should result in an increasing
RowHammer bit flip probability for each individual cell. Out of the full set
of bits that we observe _any_ RowHammer bit flips in, Table 6.5 lists the
percentage of cells that have a strictly monotonically increasing bit flip
probability as we increase $HC$.

Table 6.5: Percentage of cells with monotonically increasing RowHammer bit flip probabilities as $\bm{HC}$ increases. DRAM type-node | Cells with monotonically increasing
---|---
RowHammer bit flip probabilities (%)
Mfr. A |  Mfr. B |  Mfr. C
DDR3-new |  $97.6\pm 0.2$ |  100 |  100
DDR4-old |  $98.4\pm 0.1$ |  100 |  100
DDR4-new |  $99.6\pm 0.1$ |  100 |  100
LPDDR4-1x |  $50.3\pm 1.2$ |  $52.4\pm 1.4$ |  N/A
LPDDR4-1y |  $47.0\pm 0.8$ |  N/A |  $54.3\pm 5.7$

Observation 14. _For DDR3 and DDR4 chips, an overwhelming majority (i.e., more
than 97%) of the cells tested have monotonically increasing RowHammer bit flip
probabilities for DDR3 and DDR4 chips._

This observation indicates that exacerbating the RowHammer conditions by
increasing $HC$ increases the probability that a DRAM cell experiences a
RowHammer bit flip. However, we find that the proportion of cells with
monotonically increasing RowHammer bit flip probabilities as $HC$ increases is
around only 50% in the LPDDR4 chips that we test. We believe that this
decrease is due to the addition of on-die ECC in LPDDR4 chips, which can
obscure the probability of observing a RowHammer bit flip from the system’s
perspective in two ways. First, a RowHammer bit flip at bit X can no longer be
observable from the system’s perspective if another RowHammer bit flip at bit
Y occurs within the same ECC word as a result of increasing $HC$, and the
error correction logic corrects the RowHammer bit flip at bit X. Second, the
system may temporarily observe a bit flip at bit X at a specific $HC$ if the
set of real RowHammer bit flips within an ECC word results in a miscorrection
at bit X. Since this bit flip is a result of the ECC logic misbehaving rather
than circuit-level interference, we do not observe the expected trends for
these transient miscorrected bits.

###  6.5 Implications for Future Systems

Our characterization results have major implications for continued DRAM
technology scaling since DRAM’s increased vulnerability to RowHammer means
that systems employing future DRAM devices will likely need to handle
significantly elevated failure rates. While prior works propose a wide variety
of RowHammer failure mitigation techniques (described in Sections 6.5.1 and
6.6), these mechanisms will need to manage increasing failure rates going
forward and will likely suffer from high overhead (as we show in Section
6.5.2).

While DRAM and system designers currently implement several RowHammer
mitigation mechanisms (e.g., _pseudo Target Row Refresh_ (pTRR) [148], _Target
Row Refresh_ (TRR) [202])999Frigo et al. [87] recently demonstrated that these
mechanisms do _not_ prevent _all_ RowHammer bit flips from being exposed to
the system, and an attacker can still take over a system even with these
mechanisms in place., the designers make a number of unknown implementation
choices in these RowHammer mitigation mechanisms that are not discussed in
public documentation. Therefore, we cannot fairly evaluate how their
performance overheads scale as DRAM chips become more vulnerable to RowHammer.
Instead, we evaluate five state-of-the-art academic proposals for RowHammer
mitigation mechanisms [170, 194, 303, 372] as well as an ideal refresh-based
mitigation mechanism.

We evaluate each RowHammer mitigation mechanism in terms of two major
challenges that they will face going forward as they will need to support DRAM
chips more vulnerable to RowHammer: design scalability and system performance
overhead. We first qualitatively explain and discuss the five state-of-the-art
mitigation mechanisms and how they can potentially scale to support DRAM chips
that are more vulnerable to RowHammer. We then quantitatively evaluate their
performance overheads in simulation as $HC_{\scalebox{0.7}{first}}$ decreases.
In order to show the opportunity for reducing performance overhead in
RowHammer mitigation, we also implement and study an _ideal refresh-based
mechanism_ that prevents RowHammer by refreshing a DRAM row _only_ immediately
before it is about to experience a bit flip.

####  6.5.1 RowHammer Mitigation Mechanisms

There is a large body of work (e.g., [16, 341, 178, 41, 358, 36, 169, 351, 49,
198, 94, 104, 135]) that proposes software-based RowHammer mitigation
mechanisms. Unfortunately, many of these works have critical weaknesses (e.g.,
inability to track all DRAM activations) that make them vulnerable to
carefully-crafted RowHammer attacks, as demonstrated in some followup works
(e.g., [104]). Therefore, we focus on evaluating six mechanisms (i.e., five
state-of-the-art hardware proposals and one _ideal_ refresh-based mitigation
mechanism), which address a strong threat model that assumes an attacker can
cause row activations with precise memory location and timing information. We
briefly explain each mitigation mechanism and how its design scales for DRAM
chips with increased vulnerability to RowHammer (i.e., lower
$HC_{\scalebox{0.7}{first}}$ values).

Increased Refresh Rate [170]. The original RowHammer study [170] describes
increasing the overall DRAM refresh rate such that it is impossible to issue
enough activations within one refresh window (i.e., the time between two
consecutive refresh commands to a single DRAM row) to any single DRAM row to
induce a RowHammer bit flip. The study notes that this is an undesirable
mitigation mechanism due to its associated performance and energy overheads.
In order to reliably mitigate RowHammer bit flips with this mechanism, we
scale the refresh rate such that the refresh window (i.e., $t_{REFW}$; the
time interval between consecutive refresh commands to a single row) equals the
number of hammers until the first RowHammer bit flip (i.e.,
$HC_{\scalebox{0.7}{first}}$) multiplied by the activation latency $t_{RC}$.
Due to the large number of rows that must be refreshed within a refresh
window, this mechanism inherently does not scale to
$HC_{\scalebox{0.7}{first}}$ values below 32k.

PARA [170]. Every time a row is opened and closed, PARA (Probabilistic
Adjacent Row Activation) refreshes one or more of the row’s adjacent rows with
a low probability $p$. Due to PARA’s simple approach, it is possible to easily
tune $p$ when PARA must protect a DRAM chip with a lower
$HC_{\scalebox{0.7}{first}}$ value. In our evaluation of PARA, we scale $p$
for different values of $HC_{\scalebox{0.7}{first}}$ such that the bit error
rate (BER) does not exceed 1e-15 per hour of continuous hammering.101010We
adopt this BER from typical consumer memory reliability targets [231, 143,
260, 47, 218, 219, 45, 217, 46].

ProHIT [303]. ProHIT maintains a history of DRAM activations in a set of
tables to identify any row that may be activated $HC_{\scalebox{0.7}{first}}$
times. ProHIT manages the tables probabilistically to minimize the overhead of
tracking frequently-activated DRAM rows. ProHIT [303] uses a pair of tables
labeled "Hot" and "Cold" to track the victim rows. When a row is activated,
ProHIT checks whether each adjacent row is already in either of the tables. If
a _row_ is not in either table, it is inserted into the cold table with a
probability $p_{i}$. If the table is full, the least recently inserted entry
in the cold table is then evicted with a probability
$(1-p_{e})+p_{e}/(\\#cold\\_entries)$ and the other entries are evicted with a
probability $p_{e}/(\\#cold\\_entries)$. If the row already exists in the cold
table, the row is promoted to the highest-priority entry in the hot table with
a probability $(1-p_{t})+p_{t}/(\\#hot\\_entries)$ and to other entries with a
probability $p_{t}/(\\#hot\\_entries)$. If the row already exists in the hot
table, the entry is upgraded to a higher priority position. During each
refresh command, ProHIT simultaneously refreshes the row at the top entry of
the hot table, since this row has likely experienced the most number of
activations, and then removes the entry from the table.

For ProHIT [303] to effectively mitigate RowHammer with decreasing
$HC_{\scalebox{0.7}{first}}$ values, the size of the tables and the
probabilities for managing the tables (e.g., $p_{i}$, $p_{e}$, $p_{t}$) must
be adjusted. Even though Son et al. show a low-cost mitigation mechanism for a
specific $HC_{\scalebox{0.7}{first}}$ value (i.e., 2000), they do _not_
provide models for appropriately setting these values for arbitrary
$HC_{\scalebox{0.7}{first}}$ values and how to do so is not intuitive.
Therefore, we evaluate ProHIT only when $HC_{\scalebox{0.7}{first}}$$\,=2000$.

MRLoc [372]. MRLoc refreshes a victim row using a probability that is
dynamically adjusted based on each row’s access history. This way, according
to memory access locality, the rows that have been recorded as a victim more
recently have a higher chance of being refreshed. MRLoc uses a queue to store
victim row addresses on each activation. Depending on the time between two
insertions of a given victim row into the queue, MRLoc adjusts the probability
with which it issues a refresh to the victim row that is present in the queue.

MRLoc’s parameters (the queue size and the parameters used to calculate the
probability of refresh) are tuned for $HC_{\scalebox{0.7}{first}}$$\,=2000$.
You et al. [372] choose the values for these parameters empirically, and there
is no concrete discussion on how to adjust these parameters as
$HC_{\scalebox{0.7}{first}}$ changes. Therefore we evaluate MRLoc for only
$HC_{\scalebox{0.7}{first}}$$\,=2000$.

As such, even though we quantitatively evaluate both ProHIT [303] and MRLoc
[372] for completeness and they may seem to have good overhead results at one
data point, we are unable to demonstrate how their overheads scale as DRAM
chips become more vulnerable to RowHammer.

TWiCe [194]. TWiCe tracks the number of times a victim row’s aggressor rows
are activated using a table of counters and refreshes a victim row when its
count is above a threshold such that RowHammer bit flips cannot occur. TWiCe
uses two counters per entry: 1) a lifetime counter, which tracks the length of
time the entry has been in the table, and 2) an _activation counter_ , which
tracks the number of times an aggressor row is activated. The key idea is that
TWiCe can use these two counters to determine the rate at which a row is being
hammered and can quickly prune entries that have a low rate of being hammered.
TWiCe also minimizes its table size based on the observation that the number
of rows that can be activated enough times to induce RowHammer failures within
a refresh window is bound by the DRAM chip’s vulnerability to RowHammer.

When a row is activated, TWiCe checks whether its adjacent rows are already in
the table. If so, the activation count for each row is incremented. Otherwise,
new entries are allocated in the table for each row. Whenever a row’s
activation count surpasses a threshold $t_{RH}$ defined as
$HC_{\scalebox{0.7}{first}}$$/4$, TWiCe refreshes the row. TWiCe also defines
a pruning stage that 1) increments each lifetime counter, 2) checks each row’s
hammer rate based on both counters, and 3) prunes entries that have a lifetime
hammer rate lower than a _pruning threshold_ , which is defined as $t_{RH}$
divided by the number of refresh operations per refresh window (i.e.,
$t_{RH}/(t_{REFW}/t_{REFI})$). TWiCe performs pruning operations during
refresh commands so that the latency of a pruning operation is hidden behind
the DRAM refresh commands.

If $t_{RH}$ is lower than the number of refresh intervals in a refresh window
(i.e., $8192$), a couple of complications arise in the design. TWiCe either 1)
cannot prune its table, resulting in a very large table size since every row
that is accessed at least once will remain in the table until the end of the
refresh window or 2) requires floating point operations in order to calculate
thresholds for pruning, which would significantly increase the latency of the
pruning stage. Either way, the pruning stage latency would increase
significantly since a larger table also requires more time to check each
entry, and the latency may no longer be hidden by the refresh command.

As a consequence, TWiCe does _not_ support $t_{RH}$ values lower than the
number of refresh intervals in a refresh window ($\sim 8k$ in several DRAM
standards, e.g., DDR3, DDR4, LPDDR4). This means that in its current form, we
_cannot_ fairly evaluate TWiCe for $HC_{\scalebox{0.7}{first}}$ values below
$32k$, as $t_{RH}=\,$$HC_{\scalebox{0.7}{first}}$$/4$. However, we do evaluate
an ideal version of TWiCe (i.e., _TWiCe-ideal_) for
$HC_{\scalebox{0.7}{first}}$ values below $32k$ assuming that TWiCe-ideal
solves _both_ issues of the large table size and the high-latency pruning
stage at lower $HC_{\scalebox{0.7}{first}}$ values.

Ideal Refresh-based Mitigation Mechanism. We implement an ideal refresh-based
mitigation mechanism that tracks all activations to every row in DRAM and
issues a refresh command to a row only right before it can potentially
experience a RowHammer bit flip (i.e., when a physically-adjacent row has been
activated $HC_{\scalebox{0.7}{first}}$ times).

####  6.5.2 Evaluation of Viable Mitigation Mechanisms

We first describe our methodology for evaluating the five state-of-the-art
RowHammer mitigation mechanisms (i.e., increased refresh rate [170], PARA
[170], ProHIT [303], MRLoc [372], TWiCe [194]) and the ideal refresh-based
mitigation mechanism.

##### Evaluation Methodology

We use Ramulator [3, 174], a cycle-accurate DRAM simulator with a simple core
model and a system configuration as listed in Table 6.6, to implement and
evaluate the RowHammer mitigation mechanisms. To demonstrate how the
performance overhead of each mechanism would scale to future devices, we
implement, to the best of our ability, parameterizable methods for scaling the
mitigation mechanisms to DRAM chips with varying degrees of vulnerability to
RowHammer (as described in Section 6.5.1).

Table 6.6: System configuration for simulations. Parameter |  Configuration
---|---
Processor |  4GHz, 8-core, 4-wide issue, 128-entry instr. window
Last-level Cache |  64-Byte cache line, 8-way set-associative, 16MB
Memory Controller |  64 read/write request queue, FR-FCFS [274, 383]
Main Memory |  DDR4, 1-channel, 1-rank, 4-bank groups, 4-banks per bank group, 16k rows per bank

Workloads. We evaluate 48 8-core workload mixes drawn randomly from the full
SPEC CPU2006 benchmark suite [5] to demonstrate the effects of the RowHammer
mitigation mechanisms on systems during typical use (and _not_ when a
RowHammer attack is being mounted). The set of workloads exhibit a wide range
of memory intensities. The workloads’ MPKI values (i.e., last-level cache
misses per kilo-instruction) range from 10 to 740\. This wide range enables us
to study the effects of RowHammer mitigation on workloads with widely varying
degrees of memory intensity. We note that there could be other workloads with
which mitigation mechanisms exhibit higher performance overheads, but we did
not try to maximize the overhead experienced by workloads by biasing the
workload construction in any way. We simulate each workload until each core
executes at least 200 million instructions. For all configurations, we
initially warm up the caches by fast-forwarding 100 million instructions.

Metrics. Because state-of-the-art RowHammer mitigation mechanisms rely on
additional DRAM refresh operations to prevent RowHammer, we use two different
metrics to evaluate their impact on system performance. First, we measure
_DRAM bandwidth overhead_ , which quantifies the fraction of the total system
DRAM bandwidth consumption coming from the RowHammer mitigation mechanism.
Second, we measure overall workload performance using the _weighted speedup_
metric [79, 301], which effectively measures job throughput for multi-core
workloads [79]. We normalize the weighted speedup to its baseline value, which
we denote as 100%, and find that when using RowHammer mitigation mechanisms,
most values appear below the baseline. Therefore, for clarity, we refer to
normalized weighted speedup as _normalized system performance_ in our
evaluations.

##### Evaluation of Mitigation Mechanisms

Figure 6.8 shows the results of our evaluation of the RowHammer mitigation
mechanisms (as described in Section 6.5.1) for chips of varying degrees of
RowHammer vulnerability (i.e., $200k\geq\,$$HC_{\scalebox{0.7}{first}}$$\,\geq
64$) for our two metrics: 1) DRAM bandwidth overhead in Figure 6.8a and 2)
normalized system performance in Figure 6.8b. Each data point shows the
average value across 48 workloads with minimum and maximum values drawn as
error bars.

For each DRAM type-node configuration that we characterize, we plot the
minimum $HC_{\scalebox{0.7}{first}}$ value found across chips within the
configuration (from Table 6.4) as a vertical line to show how each RowHammer
mitigation mechanism would impact the overall system when using a DRAM chip of
a particular configuration. Above the figures (sharing the x-axis with Figure
6.8), we draw horizontal lines representing the ranges of
$HC_{\scalebox{0.7}{first}}$ values that we observe for every tested DRAM chip
per DRAM type-node configuration across manufacturers. We color the ranges
according to DRAM type-node configuration colors in the figure, and indicate
the average value with a gray point. Note that these lines directly correspond
to the box-and-whisker plot ranges in Figure 6.6.

Figure 6.8: Effect of RowHammer mitigation mechanisms on a) DRAM bandwidth
overhead (note the inverted log-scale y-axis) and b) system performance, as
DRAM chips become more vulnerable to RowHammer (from left to right).

We make _five_ key observations from this figure. First, DRAM bandwidth
overhead is highly correlated with normalized system performance, as DRAM
bandwidth consumption is the main source of system interference caused by
RowHammer mitigation mechanisms. We note that several points (i.e., ProHIT,
MRLoc, and TWiCe and Ideal evaluated at higher $HC_{\scalebox{0.7}{first}}$
values) are not visible in Figure 6.8a since we are plotting an inverted log
graph and these points are very close to zero. Second, in the latest DRAM
chips (i.e., the LPDDR4-1y chips), only PARA, ProHIT, and MRLoc are viable
options for mitigating RowHammer bit flips with reasonable average normalized
system performance: 92%, 100%, and 100%, respectively. Increased Refresh Rate
and TWiCe do not scale to such degrees of RowHammer vulnerability (i.e.,
$HC_{\scalebox{0.7}{first}}$$\,=4.8k$), as discussed in Section 6.5.1. Third,
only PARA’s design scales to low $HC_{\scalebox{0.7}{first}}$ values that we
may see in future DRAM chips, but has very low average normalized system
performance (e.g., 72% when $HC_{\scalebox{0.7}{first}}$$\,=1024$; 47% when
$HC_{\scalebox{0.7}{first}}$$\,=256$; 20% when
$HC_{\scalebox{0.7}{first}}$$\,=128$). While TWiCe-ideal has higher normalized
system performance over PARA (e.g., 98% when
$HC_{\scalebox{0.7}{first}}$$\,=1024$; 86% when
$HC_{\scalebox{0.7}{first}}$$\,=256$; 73% when
$HC_{\scalebox{0.7}{first}}$$\,=128$), there are significant practical
limitations in enabling TWiCe-ideal for such low $HC_{\scalebox{0.7}{first}}$
values (discussed in Section 6.5.1). Fourth, ProHIT and MRLoc both exhibit
high normalized system performance at their single data point (i.e., 95% and
100%, respectively when $HC_{\scalebox{0.7}{first}}$$\,=2000$), but these
works do not provide models for scaling their mechanisms to lower
$HC_{\scalebox{0.7}{first}}$ values and how to do so is not intuitive (as
described in Section 6.5.1). Fifth, the ideal refresh-based mitigation
mechanism is _significantly_ and increasingly better than any existing
mechanism as $HC_{\scalebox{0.7}{first}}$ reduces below $1024$. This indicates
that there is still significant opportunity for developing a refresh-based
RowHammer mitigation mechanism with low performance overhead that scales to
low $HC_{\scalebox{0.7}{first}}$ values. However, the ideal mechanism affects
system performance at very low $HC_{\scalebox{0.7}{first}}$ values (e.g.,
99.96% when $HC_{\scalebox{0.7}{first}}$$\,=1024$; 97.91% when
$HC_{\scalebox{0.7}{first}}$$\,=256$; 93.53% when
$HC_{\scalebox{0.7}{first}}$$\,=128$), indicating the potential need for a
better approach to solving RowHammer in future ultra-dense DRAM chips.

We conclude that while existing mitigation mechanisms may exhibit reasonably
small performance overheads for mitigating RowHammer bit flips in modern DRAM
chips, their overheads do _not_ scale well in future DRAM chips that will
likely exhibit higher vulnerability to RowHammer. Thus, we need new mechanisms
and approaches to RowHammer mitigation that will scale to DRAM chips that are
highly vulnerable to RowHammer bit flips.

####  6.5.3 RowHammer Mitigation Going Forward

DRAM manufacturers continue to adopt smaller technology nodes to improve DRAM
storage density and are forecasted to reach 1z and 1a technology nodes within
the next couple of years [326]. Unfortunately, our findings show that future
DRAM chips will likely be increasingly vulnerable to RowHammer. This means
that, to maintain market competitiveness without suffering factory yield loss,
manufacturers will need to develop effective RowHammer mitigations for coping
with increasingly vulnerable DRAM chips.

##### Future Directions in RowHammer Mitigation

RowHammer mitigation mechanisms have been proposed across the computing stack
ranging from circuit-level mechanisms built into the DRAM chip itself to
system-level mechanisms that are agnostic to the particular DRAM chip that the
system uses. Of these solutions, our evaluations in Section 6.5.1 show that,
while the ideal refresh-based RowHammer mitigation mechanism, which inserts
the minimum possible number of additional refreshes to prevent RowHammer bit
flips, scales reasonably well to very low $HC_{\scalebox{0.7}{first}}$ values
(e.g., only 6% performance loss when $HC_{\scalebox{0.7}{first}}$ is 128),
existing RowHammer mitigation mechanisms either _cannot_ scale or cause severe
system performance penalties when they scale.

To develop a scalable and low-overhead mechanism that can prevent RowHammer
bit flips in DRAM chips with a high degree of RowHammer vulnerability (i.e.,
with a low $HC_{\scalebox{0.7}{first}}$ value), we believe it is essential to
explore all possible avenues for RowHammer mitigation. Going forward, we
identify two promising research directions that can potentially lead to new
RowHammer solutions that can reach or exceed the scalability of the ideal
refresh-based mitigation mechanism: (1) DRAM-system cooperation and (2)
profile-guided mechanisms. The remainder of this section briefly discusses our
vision for each of these directions.

DRAM-System Cooperation. Considering either DRAM-based or system-level
mechanisms alone ignores the potential benefits of addressing the RowHammer
vulnerability from both perspectives together. While the root causes of
RowHammer bit flips lie within DRAM, their negative effects are observed at
the system-level. Prior work [244, 238] stresses the importance of tackling
these challenges at all levels of the stack, and we believe that a holistic
solution can achieve a high degree of protection at relatively low cost
compared to solutions contained within either domain alone.

Profile-Guided Mechanisms. The ability to accurately profile for RowHammer-
susceptible DRAM cells or memory regions can provide a powerful substrate for
building targeted RowHammer solutions that efficiently mitigate RowHammer bit
flips at low cost. Knowing (or effectively predicting) the locations of bit
flips before they occur in practice could lead to a large reduction in
RowHammer mitigation overhead, providing new information that no known
RowHammer mitigation mechanism exploits today. For example, within the scope
of known RowHammer mitigation solutions, increasing the refresh rate can be
made far cheaper by only increasing the refresh rate for known-vulnerable DRAM
rows. Similarly, ECC or DRAM access counters can be used only for known-
vulnerable cells, and even a software-based mechanism can be adapted to target
only known-vulnerable rows (e.g., by disabling them or remapping them to
reliable memory).

Unfortunately, there exists no such effective RowHammer error profiling
methodology today. Our characterization in this work essentially follows the
naïve approach of individually testing each row by attempting to induce the
worst-case testing conditions (e.g., $HC$, data pattern, ambient temperature
etc.). However, this approach is extremely time consuming due to having to
test each row individually (potentially multiple times with various testing
conditions). Even for a relatively small DRAM module of 8GB with 8KB rows,
hammering each row only once for only one refresh window of 64ms requires over
17 hours of continuous testing, which means that the naïve approach to
profiling is infeasible for a general mechanism that may be used in a
production environment or for online operation. We believe that developing a
fast and effective RowHammer profiling mechanism is a key research challenge,
and we hope that future work will use the observations made in this study and
other RowHammer characterization studies to find a solution.

###  6.6 Related Work

Although many works propose RowHammer attacks and mitigation mechanisms, only
three works [170, 257, 256] provide detailed failure-characterization studies
that examine how RowHammer failures manifest in real DRAM chips. However, none
of these studies show how the number of activations to induce RowHammer bit
flips is changing across modern DRAM types and generations, and the original
RowHammer study [170] is already six years old and limited to DDR3 DRAM chips
only. This section highlights the most closely related prior works that study
the RowHammer vulnerability of older generation chips or examine other aspects
of RowHammer.

Real Chip Studies. Three key studies (i.e., the pioneering RowHammer study
[170] and two subsequent studies [257, 256]) perform extensive experimental
RowHammer failure characterization using older DDR3 devices. However, these
studies are restricted to only DDR3 devices and do not provide a scaling study
of hammer counts across DRAM types and generations. In contrast, our work
provides the first rigorous experimental study showing how RowHammer
characteristics scale across different DRAM generations and how DRAM chips
designed with newer technology nodes are increasingly vulnerable to RowHammer.
Our work complements and furthers the analyses provided in prior studies.

Simulation Studies. Yang et al. [369] use device-level simulations to explore
the root cause of the RowHammer vulnerability. While their analysis identifies
a likely explanation for the failure mechanism responsible for RowHammer, they
do not present experimental data taken from real devices to support their
conclusions.

RowHammer Mitigation Mechanisms. Many prior works [372, 303, 16, 178, 341, 41,
170, 159, 135, 99, 194, 43, 23, 25, 28, 103, 26, 24, 13, 124, 197, 86, 118,
358, 36, 169, 351, 83, 49, 198, 350] propose RowHammer mitigation techniques.
Additionally, several patents for RowHammer prevention mechanisms have been
filed [24, 25, 28, 26, 23, 102]. However, these works do not analyze how their
solutions will scale to future DRAM generations and do not provide detailed
failure characterization data from modern DRAM devices. Similar and other
related works on RowHammer can be found in a recent retrospective [241].

###  6.7 Limitations

Due to limited resources and available testing time, we were unable to study
several aspects of RowHammer that could help to further understand it as a
phenomenon.

First, while we demonstrate a scaling study on RowHammer as DRAM process
technology node size scales, we are unsure of the exact process technology
node sizes for several chip generations. Therefore, we could only provide a
relative study on how the vulnerability to RowHammer in DRAM chips across
technology node generations changes. Given the exact sizes of the technology
nodes, we could provide an analytical model such that we could predict the
progression of RowHammer vulnerability in future chips.

Second, we did not account for chip aging in our RowHammer study. We believe
that understanding how aging affects the RowHammer vulnerability in chips is
critical to understand whether a system will become more vulnerable over time
as its DRAM chip ages in use. Furthermore, if aging can affect the RowHammer
vulnerability of a chip to the point that mitigation mechanisms can no longer
effectively mitigate RowHammer bit flips, it is important to understand when
and how often a DRAM chip should be swapped out to maintain a RowHammer-free
system.

###  6.8 Summary

We provide the first rigorous experimental RowHammer failure characterization
study that demonstrates how the RowHammer vulnerability of modern DDR3, DDR4,
and LPDDR4 DRAM chips scales across DRAM generations and technology nodes.
Using experimental data from 1580 real DRAM chips produced by the three major
DRAM manufacturers, we show that modern DRAM chips that use smaller process
technology node sizes are significantly more vulnerable to RowHammer than
older chips. Using simulation, we show that existing RowHammer mitigation
mechanisms 1) suffer from prohibitively large performance overheads at
projected future hammer counts and 2) are still far from an _ideal_ selective-
refresh-based RowHammer mitigation mechanism. Based on our study, we motivate
the need for a scalable and low-overhead solution to RowHammer and provide two
promising research directions to this end. We hope that the results of our
study will inspire and aid future work to develop efficient solutions for the
RowHammer bit flip rates we are likely to see in DRAM chips in the near
future.

##  Chapter 7 Putting It All Together

Each of these four works individually provide mechanisms that improve system
performance, security or reliability. However, these works are all orthogonal
in that a subset or all of them can be implemented simultaneously on the same
system to provide each individual benefit with little interference.
Furthermore, since each mechanism does not require any changes to DRAM, the
implementation should be constrained to the memory controller. There are many
ways to potentially combine each individual mechanism, but we provide a simple
high-level example.

###  7.1 Implementing All Proposed Techniques on the Same System

Solar-DRAM, The DRAM Latency PUF, and D-RaNGe all rely on latency failure
profiles for their specific cases on a given DRAM chip. Once the profiles are
generated via characterization, the storage overhead of the profiles is simply
the combined size of each profile. Depending on which mechanism is issuing a
DRAM access, the access latency is set according to the respective profile.

As Solar-DRAM improves system performance with its faster DRAM accesses, the
latency of regular DRAM accesses should be set according to the Solar-DRAM
profile. In the relatively infrequent event that the system requests a PUF
response or a true random value, firmware for The DRAM Latency PUF or D-RaNGe
mechanism will override regular DRAM accesses and issue DRAM accesses with
latencies adjusted according to their respective profiles.

While Solar-DRAM purely provides performance benefits, evaluating PUFs (via
DRAM Latency PUF) and generating true random values (via D-RaNGe) come at the
cost of additional DRAM accesses interleaved within regular DRAM accesses.
These additional DRAM accesses likely will increase the system performance
overhead depending on the implementations (as discussed in Sections 5.6.3 and
4.6.2). Fortunately, D-RaNGE and The DRAM Latency PUF both enable flexible
implementations and can minimize their overheads depending on the user needs
and the importance of and the need for timely PUF evaluations and true random
values.

While utilizing these mechanisms in conjunction may have compound overheads in
terms of both performance and storage, we believe that enabling these
mechanisms on a system simultaneously will provide system performance,
security, and reliability benefits that outweigh the combined overheads of the
mechanisms.

###  7.2 Cost-Benefit Analysis

While each individual mechanism provides benefits to the system, whether
performance, security, or reliability, each mechanism has associated costs in
deployment. These costs come in the form of 1) performance or energy
overheads, 2) profiling time prior to deployment (or during online operation),
or 3) minor system changes. These costs must be considered when deploying each
mechanism on various systems according to the manufacturer and user
constraints (e.g., fiscal, physical, time, and other resource limitations),
and requirements (e.g., performance, reliability, security service level
agreements). However, due to the flexibility and orthogonality of these
mechanisms, costs can be reduced by implementing any subset of these works on
a system according to the constraints and requirements.

Since our works mainly focus on demonstrating the new fundamental ideas rather
than present an optimal implementation for each of these works, we cannot
provide a quantitative cost-benefit analysis of each of our mechanisms.
However, given the benefits that each mechanism provides with the relatively
low overheads in the relatively non-optimized solutions offered in the works,
we do foresee future work in supporting near-optimal deployment strategies and
implementations for each mechanism that would significantly reduce the
associated costs and improve the ease of for implementation in real systems.

##  Chapter 8 Conclusions and Future Directions

###  8.1 Conclusions

In this dissertation, we present a number of novel observations on DRAM, via
characterization studies of real DRAM chips, that we exploit to develop
mechanisms that improve system performance and enhance system security and
reliability.

First, we introduce 1) a rigorous characterization of activation failures
across 282 _real state-of-the-art LPDDR4_ DRAM modules, 2) Solar-DRAM, whose
key idea is to exploit our observations and issue DRAM accesses with variable
latency depending on the target DRAM location’s propensity to fail with
reduced access latency, and 3) an evaluation of Solar-DRAM and its three
individual components, with comparisons to the state-of-the-art [55]. We find
that Solar-DRAM provides significant performance improvement over the state-
of-the-art DRAM latency reduction mechanism across a wide variety of
workloads, _without_ requiring any changes to DRAM chips or software.

Second, we propose D-RaNGe, a mechanism for extracting true random numbers
with high throughput from unmodified commodity DRAM devices on any system that
allows manipulation of DRAM timing parameters in the memory controller.
D-RaNGe harvests fully non-deterministic random numbers from DRAM row
activation failures, which are bit errors induced by intentionally accessing
DRAM with lower latency than required for correct row activation. Our TRNG is
based on two key observations: 1) activation failures can be induced quickly
and 2) repeatedly accessing certain DRAM cells with reduced activation latency
results in reading true random data. We validate the quality of our TRNG with
the commonly-used NIST statistical test suite for randomness. Our evaluations
show that D-RaNGe significantly outperforms the previous highest-throughput
DRAM-based TRNG by up to 211x (128x on average). We conclude that DRAM row
activation failures can be effectively exploited to improve the security of a
wide range of systems that use commodity DRAM chips via a high-throughput true
random number generator, which can enable a number of security applications
such as cryptography.

Third, we introduce and analyze the DRAM latency PUF, a new DRAM PUF suitable
for runtime authentication. The DRAM latency PUF intentionally violates
manufacturer-specified DRAM timing parameters in order to provide many highly
repeatable, unique, and unclonable PUF responses with low latency. Through
experimental evaluation using 223 state-of-the-art LPDDR4 DRAM devices, we
show that the DRAM latency PUF reliably generates PUF responses at runtime-
accessible speeds (i.e., 88.2ms on average) at all operating temperatures. We
show that the DRAM latency PUF achieves an average speedup of 152x/1426x at
70∘C/55∘C when compared with a DRAM retention PUF of the same DRAM capacity
overhead, and it achieves even greater speedups at lower temperatures. We
conclude that the DRAM latency PUF enables a fast and effective substrate for
runtime device authentication across all operating temperatures, and we hope
that the advent of runtime-accessible PUFs like the DRAM latency PUF and the
detailed experimental characterization data we provide on modern DRAM devices
will enable security architects to develop even more secure systems for future
devices.

Finally, we provide the first rigorous experimental RowHammer failure
characterization study that demonstrates how the RowHammer vulnerability of
modern DDR3, DDR4, and LPDDR4 DRAM chips scales across DRAM generations and
technology nodes. Using experimental data from 1580 real DRAM chips produced
by the three major DRAM manufacturers, we show that modern DRAM chips that use
smaller process technology node sizes are significantly more vulnerable to
RowHammer than older chips. Using simulation, we show that existing RowHammer
mitigation mechanisms 1) suffer from prohibitively large performance overheads
at projected future hammer counts and 2) are still far from an _ideal_
selective-refresh-based RowHammer mitigation mechanism. Based on our study, we
motivate the need for a scalable and low-overhead solution to RowHammer and
provide two promising research directions to this end. We hope that the
results of our study will inspire and aid future work to develop efficient
solutions for the RowHammer bit flip rates we are likely to see in DRAM chips
in the near future.

###  8.2 Future Research Directions

Our four preliminary works, Solar-DRAM, the DRAM Latency PUF, D-RaNGe, and
Revisiting RowHammer demonstrate that novel observations via DRAM
characterization can be exploited to improve latency, security, and
reliability aspects of DRAM by developing mechanisms that exploit the DRAM
characteristics when they are accessed with reduced DRAM timing parameters. To
explore other methods for improving these aspects of DRAM, our proposal for
future work is to 1) characterize various additional timing parameters and
circuit-level aspects, and 2) propose mechanisms based on our prior
observations.

####  8.2.1 Reducing DRAM Latency by Exploiting Different Timing Parameters

In Chapter 3, we have shown that we can reduce the DRAM timing parameter,
tRCD, to substantially improve DRAM access latency. However, we believe that
there are a number of other DRAM timing parameters (e.g., tRP, tWR, tRTP) that
could also have substantial impact on overall system performance when reduced
below manufacturer-specified values.

The key challenge in demonstrating a viable mechanism for reliably reducing
DRAM timing parameters lies in rigorously characterizing DRAM to demonstrate
exploitable trends for efficiently reducing DRAM timing parameters. While
Chapter 3 discusses exploitable spatial distributions for reducing tRCD (i.e.,
failures constrained to local bitlines), we expect that each timing parameter,
when reduced, will result in various exploitable spatial distributions. Using
this knowledge gained from characterization, we can then safely and reliably
reduce DRAM timing parameters for system performance improvement depending on
the region of DRAM being accessed. To further exploit findings from this type
of characterization, we believe the next step would be to determine the
behavior of interactions when reducing multiple DRAM timing parameters
simultaneously, such that we can further reduce DRAM access latencies
reliably.

We identify two further challenges that are necessary to overcome for enabling
this direction. First, it is critical to determine an _efficient_ method for
profiling DRAM chips (e.g., determining the set of reliable timing
parameters). Due to the large search space in parameters (e.g., temperature,
data pattern, timing parameter, timing parameter value), a comprehensive
characterization for every chip is far too expensive. We believe that
searching for correlations between failures of cells with different
parameters, could help to reduce the characterization phase in such a
mechanism. For example, if we find that tRCD failures are correlated with tWR
failures, then we can simply characterize one of the parameters and set the
other value accordingly. Second, it is essential to develop a memory
controller and scheme that determines when an access can be issued with
reduced timing parameters and dynamically adjusts timing parameters to
minimize DRAM access latency while maintaining data correctness during
operation.

####  8.2.2 Improving Security Primitives for DRAM Chips

In Chapters 4 and 5, we introduced two security primitives, a Physical
Unclonable Function (PUF) and a True Random Number Generator (TRNG), for
commodity DRAM chips. We believe that there is significant room to improve the
security and reliability guarantees of each of our proposed mechanisms: The
DRAM Latency PUF and D-RaNGe.

First, as discussed in Chapter 4, a PUF with a larger challenge response space
can uniquely identify a larger set of devices and improve the authentication
process to minimize the probability for misidentification. In the ideal case,
the challenge response space would grow exponentially with its input
parameters. A PUF with an exponential challenge response space is classified
as a strong PUF which has substantially more use cases compared to a weak PUF.
By rigorous characterization of varying DRAM timing parameters, we can
determine whether it is feasible to increase the challenge response space
simply by using DRAM timing parameters as an additional dimension to the input
challenges. Ideally, reducing multiple DRAM timing parameters would result in
uncorrelated failure locations such that the input space would grow
exponentially with the number of additional DRAM timing parameters used. We
believe that following an experimental methodology similar to the one
described in Chapter 4 can demonstrate whether this direction is viable.

Second, we believe it is important to demonstrate practical end-to-end
implementations of both The DRAM Latency PUF and D-RaNGe for adoption in real
devices. We identify various challenges for the DRAM Latency PUF and D-RaNGe.
For a practical implementation of The DRAM Latency PUF, it is critical to
investigate 1) methods for supporting dynamically changing temperatures at
runtime such that a PUF response’s variations due to ambient temperature do
not prohibit correct authentication, and 2) a DRAM memory controller that can
schedule PUF evaluation accesses with custom DRAM timing parameters and
minimize its overhead on running workloads. For a practical implementation of
the random number generator, D-RaNGe, we identify three directions that are
critical to investigate. First, it is critical to demonstrate a process for
quickly identifying RNG cells that work under varying conditions (e.g.,
temperature). Second, it is critical to investigate a smarter DRAM memory
controller that can intelligently schedule D-RaNGe accesses with custom DRAM
timing parameters for minimizing the overhead that would come with directly
implementing D-RaNGe on any existing memory controller today. Third, it is
important to further increase the throughput and decrease the latency. We
believe that by combining the varying existing DRAM-based TRNGs (discussed in
Chapter 4.8), we can invoke the different TRNGs (or a combination of them)
depending on the running workloads to maximize the utility of idle DRAM time.

####  8.2.3 RowHammer Mitigation Going Forward

DRAM manufacturers continue to adopt smaller technology nodes to improve DRAM
storage density and are forecasted to reach 1z and 1a technology nodes within
the next couple of years [326]. Unfortunately, our findings show that future
DRAM chips will likely be increasingly vulnerable to RowHammer. This means
that, to maintain market competitiveness without suffering factory yield loss,
manufacturers will need to develop effective RowHammer mitigation solutions
for coping with increasingly vulnerable DRAM chips.

RowHammer-mitigation solutions range from circuit-level mechanisms built into
the DRAM chip itself to system-level mechanisms that are agnostic to the
particular DRAM chip that the system uses. Of these solutions, our evaluations
in Section 6.5.1 show that, while the ideal refresh-based RowHammer-mitigation
mechanism that inserts the minimum possible number of additional refreshes to
prevent RowHammer bit flips scales well to relatively low $HC_{first}$ values
(e.g., only 6% performance degradation when $HC_{first}$ is 128), currently
existing refresh-based mechanisms fall short of the ideal mechanism’s benefits
(e.g., at least 28% performance degradation when $HC_{first}$ is 128).

To achieve the scalability of the ideal refresh-based mechanism while
maintaining an efficient implementation, we believe it is essential to explore
all possible avenues for RowHammer mitigation. Going forward, we identify two
promising research directions that can potentially lead to new RowHammer-
mitigation solutions that meet or exceed the scalability of the ideal refresh-
based mitigation mechanism: (1) DRAM-system cooperation and (2) a profile-
guided mechanism. The remainder of this section briefly discusses our vision
for each of these directions.

DRAM-System Cooperation. Considering either DRAM-based or system-level
mechanisms alone ignores the potential benefits of addressing the RowHammer
vulnerability from both perspectives together. While the root causes of
RowHammer bit flips lie within DRAM, their negative effects are observed at
the system-level. Prior work [244] stresses the importance of tackling these
challenges at all levels of the stack, and we believe that a holistic solution
can achieve a high degree of protection at relatively low cost compared to
solutions contained within either domain alone.

Profile-Guided Mechanisms. The ability to accurately profile for RowHammer-
susceptible DRAM cells or memory regions can provide a powerful substrate for
building targeted RowHammer-mitigation solutions that efficiently mitigate
RowHammer bit flips at low cost. Knowing (or effectively predicting) the
locations of bit flips before they occur in practice could lead to a large
reduction in RowHammer-mitigation overhead, providing new information that no
known RowHammer-mitigation mechanism exploits today. For example, within the
scope of known RowHammer-mitigation solutions, increasing the refresh rate can
be made far cheaper by only increasing the refresh rate for known vulnerable
DRAM rows. Similarly, ECC and DRAM access counters can be used only for cells
that are known to be vulnerable, and even software-based mechanism can be
adapted to target only known vulnerable rows (e.g., disabling them, remapping
them to reliable memory).

Unfortunately, there exists no such effective RowHammer error profiling
methodology today. Our characterization in this work essentially follows the
naïve approach of individually testing each row by attempting to induce the
worst-case testing conditions (e.g., $HC$, data pattern, ambient temperature
etc.). However, this approach is extremely time consuming due to having to
test each row individually (potentially multiple times with various testing
conditions). Even for a relatively small DRAM module of 8GB with 8KB rows,
hammering each row only once for only one refresh window of 64ms requires over
17 hours of continuous testing, which means that the naïve approach to
profiling is infeasible for a general mechanism that may be used in a
production environment or for online operation. We believe that developing a
fast and effective RowHammer profiling mechanism is a key research challenge,
and we hope that future work will use the observations made in this and other
RowHammer characterization studies to find a solution.

###  8.3 Final Concluding Remarks

In this dissertation, we demonstrated that by rigorously understanding and
exploiting DRAM device characteristics, we can significantly improve system
performance and enhance system security and reliability. We have presented
four characterization-based works that show by understanding per-chip error
characteristics using a profiling mechanism, we can develop mechanisms that
exploit the chip-dependent error profiles to improve system performance or
enhance system security and reliability: 1) Solar-DRAM, which exploits our
experimental characterization on latency variation within a chip to reduce
latency with a profile that identifies regions that can be reliably accessed
with lower latencies, 2) The DRAM Latency PUF, which exploits our observation
from characterization that latency failures are unique to a DRAM chip due to
process manufacturing variation and a profile of these error characteristics
can be used generate reliable and unique identifiers, 3) D-RaNGe, which
demonstrates via characterization that a profile of error characteristics can
identify specific cells in DRAM that can be repeatedly accessed with reduced
latency to result in random failures and can be used as an efficient true
random number generator, a feature often used in security applications, and 4)
Revisiting RowHammer, which demonstrates via characterization that the DRAM-
based RowHammer vulnerability is getting worse as technology node size scales
and existing RowHammer mitigation mechanisms either do not scale or have
prohibitively high overheads to mitigate RowHammer bit flips in future DRAM
chips. We conclude and hope that the proposed characterization-based studies
of DRAM chips and novel observations will pave the way for new research that
can develop new mechanisms to improve system performance, energy efficiency,
system security, or reliability of future memory systems.

### Other Works of the Author

Throughout the course of my Ph.D. study, I have worked on several different
topics with many fellow graduate students from Carnegie Mellon University, ETH
Zurich, and other institutions. In this chapter, I would like to acknowledge
these works.

I have worked on a number of other projects on DRAM. In collaboration with
Minesh Patel, we have developed REAPER [260], a profiling mechanism for
retention failures to maintain DRAM reliability while reducing the refresh
overhead. We also characterized and studied DRAM chips with on-die ECC [259],
demonstrating how on-die ECC affects the DRAM’s properties, and we develop
Error-correction Inference (EIN), a statistical inference methodology that
infers pre-correction error rates of DRAM with on-die ECC. In a follow-up work
called BEER [258], we propose a new methodology for determining the full DRAM
on-die ECC function (i.e., its parity-check matrix) without hardware support,
hardware intrusion, or access to error syndromes and parity information. In
collaboration with Hasan Hassan, we developed CROW [118], a flexible substrate
that we use to lower DRAM activation latency to frequently accessed rows and
reduce the overhead of DRAM refresh operations. In collaboration with Yaohua
Wang, we developed CAL [349], a mechanism that predicts the next access time
of a given row, and only partially restores charge to a row that will be
accessed soon, and a substrate which can perform cache-block level data
relocation within a DRAM bank at a distance-independent latency [348]. I have
also contributed to a retrospective survey [241] on the DRAM-based security
vulnerability, RowHammer, which was first rigorously analyzed in a paper [170]
that I contributed to before my Ph.D. study. In collaboration with Lucian
Cojocar, Stefan Saroiu, and Alec Wolman at Microsoft Research and others, we
developed a methodology for testing the RowHammer vulnerability in server
nodes and identify an instruction sequence that results in the highest rate of
DRAM activations (i.e., hammers). I have also contributed to Ambit [294], an
accelerator for bulk bitwise operations in memory and a positioning paper
describing important domains of work toward the practical construction and
widespread adoption of Processing-in-Memory (PIM) architectures. I have also
contributed to a positioning paper [95], which describes the challenges that
remain for the widespread adoption of PIM.

I have also contributed to FLIN [325], a lightweight I/O request scheduling
mechanism that provides fairness among requests from different applications,
SysScale [111], a multi-domain power management technique that improves the
energy efficiency of mobile SoCs, and FlexWatts [112], a hybrid adaptive Power
Delivery Network (PDN) for modern high-end client processors whose goal is to
maintain high energy-efficiency across the processor’s wide spectrum of power
consumption and workloads.

Another topic that I have developed an interest and worked on was
bioinformatics. I authored GRIM-Filter [161], a fast seed location filtering
algorithm for DNA read mapping and AirLift [162], a methodology for quickly
and comprehensively mapping a set of reads from one reference to another
reference. I collaborated with Damla Senol on a survey of Nanopore sequencing
technologies [287] and GenASM [286], a flexible approximate string matching
acceleration framework. In addition, I also worked with Can Firtina on Apollo
[82], a sequencing-technology-independent assembly polishing algorithm, and
with Hongyi Xin on LEAP [363], an algorithm for sequence alignment.

## References

  * [1] AMD Opteron 4300 Series Processors.  http://www.amd.com/en-us/products/server/4000/4300.
  * [2] DRAMPower Source Code.  https://github.com/tukl-msd/DRAMPower.
  * [3] Ramulator Source Code.  https://github.com/CMU-SAFARI/ramulator.
  * [4] RDMA Protocol Specification.  http://www.rdmaconsortium.org/.
  * [5] Standard Performance Evaluation Corporation.  http://www.spec.org/cpu2006.
  * [6] ECC Brings Reliability and Power Efficiency to Mobile Devices.  Technical report, Micron Technology inc., 2017.
  * [7] Junwhan Ahn, Sungpack Hong, Sungjoo Yoo, Onur Mutlu, and Kiyoung Choi.  A Scalable Processing-in-Memory Accelerator for Parallel Graph Processing.  In ISCA, 2015.
  * [8] Junwhan Ahn, Sungjoo Yoo, Onur Mutlu, and Kiyoung Choi.  PIM-Enabled Instructions: a Low-overhead, Locality-aware Processing-in-Memory Architecture.  In ISCA, 2015.
  * [9] Barbara Aichinger.  DDR memory errors caused by Row Hammer.  In 2015 IEEE High Performance Extreme Computing Conference (HPEC), pages 1–5. IEEE, 2015.
  * [10] Takehiko Amaki, Masanori Hashimoto, and Takao Onoye.  An Oscillator-based True Random Number Generator with Process and Temperature Tolerance.  In DAC, 2015.
  * [11] AMD.  AMD Opteron 4300 Series Processors.  2012\.
  * [12] AMD.  BKDG for AMD Family 16h Models 00h-0Fh Processors.  2013\.
  * [13] Apple Inc.  About the Security Content of Mac EFI Security Update 2015-001.  https://support.apple.com/en-us/HT204934, 2015.
  * [14] ARM.  ARM CoreLink DMC-520 Dynamic Memory Controller Technical Reference Manual.  2016\.
  * [15] Rachata Ausavarungnirun, Kevin Kai-Wei Chang, Lavanya Subramanian, Gabriel H Loh, and Onur Mutlu.  Staged Memory Scheduling: Achieving High Performance and Scalability in Heterogeneous Systems.  In ISCA, 2012.
  * [16] Zelalem Birhanu Aweke, Salessawi Ferede Yitbarek, Rui Qiao, Reetuparna Das, Matthew Hicks, Yossi Oren, and Todd Austin.  ANVIL: Software-Based Protection Against Next-Generation Rowhammer Attacks.  In ASPLOS, 2016.
  * [17] Aydin Aysu, Ye Wang, Patrick Schaumont, and Michael Orshansky.  A New Maskless Debiasing Method for Lightweight Physical Unclonable Functions.  In HOST, 2017.
  * [18] Oreoluwatomiwa O Babarinsa and Stratos Idreos.  JAFAR: Near-Data Processing for Databases.  In SIGMOD, 2015.
  * [19] Anys Bacha and Radu Teodorescu.  Authenticache: Harnessing Cache ECC for System Authentication.  In MICRO, 2015.
  * [20] SangGeun Bae, Yongtae Kim, Yunsoo Park, and Chulwoo Kim.  3-Gb/s High-speed True Random Number Generator using Common-mode Operating Comparator and Sampling Uncertainty of D Flip-flop.  In IEEE Journal of Solid-State Circuits, volume 52, pages 605–610.
  * [21] Seungjae Baek, Sangyeun Cho, and Rami Melhem.  Refresh Now and Then.  In IEEE Transactions on Computers, volume 63, pages 3114–3126, 2014\.
  * [22] Vittorio Bagini and Marco Bucci.  A Design of Reliable True Random Number Generator for Cryptographic Applications.  In CHES, 1999.
  * [23] Kuljit Bains, John Halbert, Christopher Mozak, Theodore Schoenborn, and Zvika Greenfield.  Row Hammer Refresh Command, 2015.  US Patent 9,117,544.
  * [24] Kuljit S Bains and John B Halbert.  Row Hammer Monitoring Based on Stored Row Hammer Threshold Value, May 12 2015.  US Patent 9,032,141.
  * [25] Kuljit S Bains and John B Halbert.  Distributed Row Hammer Tracking, March 29 2016.  US Patent 9,299,400.
  * [26] Kuljit S Bains, John B Halbert, Christopher P Mozak, Theodore Z Schoenborn, and Zvika Greenfield.  Row Hammer Refresh Command.  US Patent 9,236,110, 2016.
  * [27] Kuljit S Bains, John B Halbert, Suneeta Sah, and Zvika Greenfield.  Method, apparatus and system for providing a memory refresh, May 12 2015\.  US Patent 9,030,903.
  * [28] Kuljit S Bains, John B Halbert, Suneeta Sah, and Zvika Greenfield.  Method, Apparatus and System for Providing a Memory Refresh, May 12 2015\.  US Patent 9,030,903.
  * [29] Rajeev Balasubramonian.  Innovations in the Memory System.  Synthesis Lectures on Computer Architecture, 2019.
  * [30] Mahmood Barangi, Joseph S Chang, and Pinaki Mazumder.  Straintronics-Based True Random Number Generator for High-Speed and Energy-Limited Applications.  In IEEE Transactions on Magnetics, volume 52, pages 1–9, 2016.
  * [31] Karsten Beckmann, Harika Manem, and Nathaniel Cady.  Performance Enhancement of a Time-Delay PUF Design by Utilizing Integrated Nanoscale ReRAM Devices.  In IEEE Transactions on Emerging Topics in Computing, volume 5, pages 304–316, 2017.
  * [32] Mudit Bhargava, Cagla Cakir, and Ken Mai.  Reliability Enhancement of Bi-Stable PUFs in 65nm Bulk CMOS.  In HOST, 2012.
  * [33] Mudit Bhargava, Kaship Sheikh, and Ken Mai.  Robust True Random Number Generator using Hot-carrier Injection Balanced Metastable Sense Amplifiers.  In HOST, 2015.
  * [34] Ishwar Bhati, Mu-Tien Chang, Zeshan Chishti, Shih-Lien Lu, and Bruce Jacob.  DRAM Refresh Mechanisms, Penalties, and Trade-offs.  In IEEE Transactions on Computers, volume 65, pages 108–121, 2016\.
  * [35] Ishwar Bhati, Zeshan Chishti, Shih-Lien Lu, and Bruce Jacob.  Flexible Auto-Refresh: Enabling Scalable and Energy-Efficient DRAM Refresh Reductions.  In ISCA, 2015.
  * [36] Carsten Bock, Ferdinand Brasser, David Gens, Christopher Liebchen, and Ahmad Reza Sadeghi.  RIP-RH: Preventing Rowhammer-Based Inter-Process Attacks.  In ASIA-CCS, 2019.
  * [37] Amirali Boroumand, Saugata Ghose, Youngsok Kim, Rachata Ausavarungnirun, Eric Shiu, Rahul Thakur, Daehyun Kim, Aki Kuusela, Allan Knies, Parthasarathy Ranganathan, and Onur Mutlu.  Google Workloads for Consumer Devices: Mitigating Data Movement Bottlenecks.  In ASPLOS, 2018.
  * [38] Amirali Boroumand, Saugata Ghose, Brandon Lucia, Kevin Hsieh, Krishna Malladi, Hongzhong Zheng, and Onur Mutlu.  LazyPIM: An Efficient Cache Coherence Mechanism for Processing-in-Memory.  In IEEE Computer Architecture Letters, volume 16, pages 46–50, 2017\.
  * [39] Amirali Boroumand, Saugata Ghose, Minesh Patel, Hasan Hassan, Brandon Lucia, Rachata Ausavarungnirun, Kevin Hsieh, Nastaran Hajinazar, Krishna T Malladi, Hongzhong Zheng, et al.  CoNDA: Efficient Cache Coherence Support for Near-data Accelerators.  In ISCA, 2019.
  * [40] Roelof Cornelis Botha.  The Development of a Hardware Random Number Generator for Gamma-ray Astronomy.  PhD thesis, North-West University, 2005.
  * [41] Ferdinand Brasser, Lucas Davi, David Gens, Christopher Liebchen, and Ahmad-Reza Sadeghi.  Can’t Touch This: Practical and Generic Software-only Defenses Against RowHammer Attacks.  USENIX Security, 2017.
  * [42] Ralf Brederlow, Ramesh Prakash, Christian Paulus, and Roland Thewes.  A Low-power True Random Number Generator using Random Telegraph Noise of Single Oxide-traps.  In ISSCC, 2006.
  * [43] Lake Bu, Jaya Dofe, Qiaoyan Yu, and Michel A Kinsy.  SRASA: a Generalized Theoretical Framework for Security and Reliability Analysis in Computing Systems.  Journal of Hardware and Systems Security, 3(3):200–218, 2018.
  * [44] Marco Bucci, Lucia Germani, Raimondo Luzzi, Alessandro Trifiletti, and Mario Varanonuovo.  A High-speed Oscillator-based Truly Random Number Source for Cryptographic Applications on a Smart Card IC.  volume 52, pages 403–409, 2003.
  * [45] Yu Cai.  Flash Memory SSD Errors, Mitigation, and Recovery.  In Proceedings of the IEEE, 2017.
  * [46] Yu Cai, Saugata Ghose, Erich F Haratsch, Yixin Luo, and Onur Mutlu.  Errors in Flash-Memory-Based Solid-State Drives: Analysis, Mitigation, and Recovery.  Invited Book Chapter in Inside Solid State Drives, 2018.
  * [47] Yu Cai, Erich F Haratsch, Onur Mutlu, and Ken Mai.  Error Patterns in MLC NAND Flash Memory: Measurement, Characterization, and Analysis.  In DATE, 2012.
  * [48] Sanguhn Cha, O Seongil, Hyunsung Shin, Sangjoon Hwang, Kwangil Park, Seong Jin Jang, Joo Sun Choi, Gyo Young Jin, Young Hoon Son, Hyunyoon Cho, et al.  Defect Analysis and Cost-effective Resilience Architecture for Future DRAM Devices.  In HPCA, 2017.
  * [49] Anirban Chakraborty, Manaar Alam, and Debdeep Mukhopadhyay.  Deep Learning Based Diagnostics for Rowhammer Protection of DRAM Chips.  In ATS, 2019.
  * [50] Supriya Chakraborty, Abhilash Garg, and Manan Suri.  True Random Number Generation From Commodity NVM Chips.  IEEE Transactions on Electron Devices, 67(3):888–894, 2020.
  * [51] Jose Juan Mijares Chan, Bhanu Sharma, Jiaqing Lv, Gabriel Thomas, Ruppa Thulasiram, and Parimala Thulasiraman.  True Random Number Generator using GPUs and Histogram Equalization Techniques.  In HPCC, 2011.
  * [52] Karthik Chandrasekar, Benny Akesson, and Kees Goossens.  Improved Power Modeling of DDR SDRAMs.  In DSD, 2011.
  * [53] Karthik Chandrasekar, Sven Goossens, Christian Weis, Martijn Koedam, Benny Akesson, Norbert Wehn, and Kees Goossens.  Exploiting Expendable Process-Margins in DRAMs for Run-Time Performance Optimization.  In DATE, 2014.
  * [54] Kevin K. Chang.  Understanding and Improving Latency of DRAM-Based Memory Systems.  PhD thesis, Carnegie Mellon University, 2017.
  * [55] Kevin K Chang, Abhijith Kashyap, Hasan Hassan, Saugata Ghose, Kevin Hsieh, Donghyuk Lee, Tianshi Li, Gennady Pekhimenko, Samira Khan, and Onur Mutlu.  Understanding Latency Variation in Modern DRAM Chips: Experimental Characterization, Analysis, and Optimization.  In SIGMETRICS, 2016.
  * [56] Kevin K Chang, Donghyuk Lee, Zeshan Chishti, Alaa R Alameldeen, Chris Wilkerson, Yoongu Kim, and Onur Mutlu.  Improving DRAM Performance by Parallelizing Refreshes with Accesses.  In HPCA, 2014.
  * [57] Kevin K Chang, Prashant J Nair, Donghyuk Lee, Saugata Ghose, Moinuddin K Qureshi, and Onur Mutlu.  Low-cost Inter-linked Subarrays (LISA): Enabling Fast Inter-subarray Data Movement in DRAM.  In HPCA, 2016.
  * [58] Kevin K Chang, Abdullah Yaglikci, Saugata Ghose, Adity Agrawal, Niladrish Chatterjee, Abhijith Kashyap, Donghyuk Lee, Mike O’Connor, Hasan Hassan, and Onur Mutlu.  Understanding Reduced-Voltage Operation in Modern DRAM Devices: Experimental Characterization, Analysis, and Mechanisms.  In SIGMETRICS, 2017.
  * [59] Wenjie Che, Jim Plusquellic, and Swarup Bhunia.  A Non-Volatile Memory Based Physically Unclonable Function without Helper Data.  In ICCAD, 2014.
  * [60] Pai-Yu Chen, Runchen Fang, Rui Liu, Chaitali Chakrabarti, Yu Cao, and Shimeng Yu.  Exploiting Resistive Cross-Point Array for Compact Design of Physical Unclonable Function.  In HOST, 2015.
  * [61] Abdelkarim Cherkaoui, Viktor Fischer, Laurent Fesquet, and Alain Aubert.  A Very High Speed True Random Number Generator with Entropy Assessment.  In CHES, 2013.
  * [62] Jeonghwan Choi, Youngjae Kim, Anand Sivasubramaniam, Jelena Srebric, Qian Wang, and Joonwon Lee.  Modeling and Managing Thermal Profiles of Rack-mounted Servers with Thermostat.  In HPCA, 2007.
  * [63] Pong P Chu and Robert E Jones.  Design Techniques of FPGA Based Random Number Generator.  In MAPLD, 1999.
  * [64] Patrick J Clarke, Robert J Collins, Philip A Hiskett, Paul D Townsend, and Gerald S Buller.  Robust Gigahertz Fiber Quantum Key Distribution.  Applied Physics Letters, 2011.
  * [65] Lucian Cojocar, Jeremie Kim, Minesh Patel, Lillian Tsai, Stefan Saroiu, Alec Wolman, and Onur Mutlu.  Are We Susceptible to Rowhammer? An End-to-End Methodology for Cloud Providers.  SP, 2020.
  * [66] Lucian Cojocar, Kaveh Razavi, Cristiano Giuffrida, and Herbert Bos.  Exploiting Correcting Codes: On the Effectiveness of ECC Memory Against RowHammer Attacks.  In SP, 2019.
  * [67] Mafalda Cortez, Said Hamdioui, Vincent van der Leest, Roel Maes, and Geert-Jan Schrijen.  Adapting Voltage Ramp-up Time for Temperature Noise Reduction on Memory-based PUFs.  In HOST, 2013.
  * [68] Zehan Cui, Sally A McKee, Zhongbin Zha, Yungang Bao, and Mingyu Chen.  DTail: A Flexible Approach to DRAM Refresh Management.  In SC, 2014.
  * [69] J L Danger, Sylvain Guilley, Philippe Nguyen, and Olivier Rioul.  PUFs: Standardization and Evaluation.  In MST, 2016.
  * [70] Anup Das, Hasan Hassan, and Onur Mutlu.  VRL-DRAM: Improving DRAM Performance via Variable Refresh Latency.  DAC, 2018.
  * [71] Jayita Das, Kevin Scott, Srinath Rajaram, Drew Burgett, and Sanjukta Bhanja.  MRAM PUF: A Novel Geometry Based Magnetic PUF with Integrated CMOS.  In IEEE Transactions on Nanotechnology, volume 14, pages 436–443, 2015.
  * [72] Howard David, Chris Fallin, Eugene Gorbatov, Ulf R Hanebutte, and Onur Mutlu.  Memory Power Management via Dynamic Voltage/Frequency Scaling.  In ICAC, 2011.
  * [73] Satyajit Desai.  Process Variation Aware DRAM (Dynamic Random Access Memory) Design Using Block-based Adaptive Body Biasing Algorithm.  PhD thesis, Utah State University, 2012.
  * [74] Wei Ding, Diana Guttman, and Mahmut Kandemir.  Compiler Support for Optimizing Memory Bank-level Parallelism.  In MICRO, 2014.
  * [75] Milos Drutarovsky and Pavol Galajda.  A Robust Chaos-based True Random Number Generator Embedded in Reconfigurable Switched-Capacitor Hardware.  In Radioelektronika, 2007.
  * [76] Donald Eastlake and Paul Jones.  US Secure Hash Algorithm 1 (SHA1).  Technical report, 2001.
  * [77] Charles Eckert, Fatemeh Tehranipoor, and John A Chandy.  DRNG: DRAM-based Random Number Generation Using its Startup Value Behavior.  In MWSCAS, 2017.
  * [78] Nosayba El-Sayed, Ioan A Stefanovici, George Amvrosiadis, Andy A Hwang, and Bianca Schroeder.  Temperature Management in Data Centers: Why Some (Might) Like it Hot.  In SIGMETRICS, 2012.
  * [79] Stijn Eyerman and Lieven Eeckhout.  System-level Performance Metrics for Multiprogram Workloads.  In IEEE Micro, 2008.
  * [80] Gene Fabron.  RAM Overclocking Guide: How (and Why) to Tweak Your Memory.  https://www.tomshardware.com/reviews/ram-overclocking-guide,4693.html.
  * [81] Amin Farmahini-Farahani, Jung Ho Ahn, Katherine Morrow, and Nam Sung Kim.  NDA: Near-DRAM Acceleration Architecture Leveraging Commodity DRAM Devices and Standard Memory Modules.  In HPCA, 2015.
  * [82] Can Firtina, Jeremie S Kim, Mohammed Alser, Damla Senol Cali, A Ercument Cicek, Can Alkan, and Onur Mutlu.  Apollo: A Sequencing-Technology-Independent, Scalable, and Accurate Assembly Polishing Algorithm.  Bioinformatics, 36(12):3669–3679, 2020.
  * [83] David Edward Fisch and William C Plants.  DRAM Adjacent Row Disturb Mitigation, 2017.  US Patent 9,812,185.
  * [84] Viktor Fischer and Miloš Drutarovskỳ.  True Random Number Generator Embedded in Reconfigurable Hardware.  In CHES, 2002.
  * [85] Viktor Fischer, Miloš Drutarovskỳ, Martin Šimka, and Nathalie Bochard.  High performance true random number generator in Altera Stratix FPLDs.  In FPL, 2004.
  * [86] Troy Fridley and Omar Santos.  Mitigations Available for the DRAM Row Hammer Vulnerability.  http://blogs.cisco.com/security/mitigations-available-for-the-dram-row-hammer-vulnerability, 2015\.
  * [87] Pietro Frigo, Emanuele Vannacci, Hasan Hassan, Victor van der Veen, Onur Mutlu, Cristiano Giuffrida, Herbert Bos, and Kaveh Razavi.  TRRespass: Exploiting the Many Sides of Target Row Refresh.  In SP, 2020.
  * [88] Mingyu Gao, Grant Ayers, and Christos Kozyrakis.  Practical Near-Data Processing for In-Memory Analytics Frameworks.  In PACT, 2015.
  * [89] Mingyu Gao and Christos Kozyrakis.  HRL: Efficient and Flexible Reconfigurable Logic for Near-Data Processing.  In HPCA, 2016.
  * [90] Yansong Gao, Damith C Ranasinghe, Said F Al-Sarawi, Omid Kavehei, and Derek Abbott.  Memristive Crypto Primitive for Building Highly Secure Physical Unclonable Functions.  In Scientific Reports, volume 5, 2015.
  * [91] Blaise Gassend, Dwaine Clarke, Marten Van Dijk, and Srinivas Devadas.  Silicon Physical Random Functions.  In CCS, 2002.
  * [92] Blaise LP Gassend.  Physical Random Functions.  PhD thesis, Massachusetts Institute of Technology, 2003.
  * [93] Wei Ge, Shenxin Hu, Jiquan Huang, Bo Liu, and Min Zhu.  FPGA Implementation of a Challenge Pre-processing Structure Arbiter PUF Designed for Machine Learning Attack Resistance.  IEICE Electronics Express, 2019.
  * [94] Mohsen Ghasempour, Mikel Lujan, and Jim Garside.  ARMOR: A Run-Time Memory Hot-Row Detector, 2015.
  * [95] Saugata Ghose, Amirali Boroumand, Jeremie S Kim, Juan Gómez-Luna, and Onur Mutlu.  Processing-in-memory: A Workload-driven Perspective.  IBM J. Res. & Dev., 2019.
  * [96] Saugata Ghose, Kevin Hsieh, Amirali Boroumand, Rachata Ausavarungnirun, and Onur Mutlu.  Enabling the Adoption of Processing-in-Memory: Challenges, Mechanisms, Future Research Directions.  arXiv preprint arXiv:1802.00320, 2018.
  * [97] Saugata Ghose, Tianshi Li, Nastaran Hajinazar, Damla Senol Cali, and Onur Mutlu.  Demystifying Complex Workload-DRAM Interactions: An Experimental Study.  SIGMETRICS, 2019.
  * [98] Saugata Ghose, Giray Yaglikci, Raghav Gupta, Donghyuk Lee, Kais Kudrolli, William Liu, Hasan Hassan, Kevin Chang, Niladrish Chatterjee, Aditya Agrawal, Mike O’Connor, and Onur Mutlu.  What Your DRAM Power Models Are Not Telling You: Lessons from a Detailed Experimental Study.  In SIGMETRICS, 2018.
  * [99] Hector Gomez, Andres Amaya, and Elkim Roa.  DRAM Row-hammer Attack Reduction using Dummy Cells.  In NORCAS, 2016.
  * [100] Seong-Lyong Gong, Jungrae Kim, Sangkug Lym, Michael Sullivan, Howard David, and Mattan Erez.  Duo: Exposing on-chip Redundancy to Rank-level ECC for High Reliability.  In HPCA, 2018.
  * [101] Sudhakar Govindavajhala and Andrew W Appel.  Using Memory Errors to Attack a Virtual Machine.  In SP, 2003.
  * [102] Zvika Greenfield, Kuljit S Bains, Theodore Z Schoenborn, Christopher P Mozak, and John B Halbert.  Row Hammer Condition Monitoring.  US Patent App. 13/539,417, January 2, 2014.
  * [103] Zvika Greenfield, John B Halbert, and Kuljit S Bains.  Method, Apparatus and System for Determining a Count of Accesses to a Row of Memory.  US Patent App. 13/626,479, March 27 2014.
  * [104] Daniel Gruss, Moritz Lipp, Michael Schwarz, Daniel Genkin, Jonas Juffinger, Sioli O’Connell, Wolfgang Schoechl, and Yuval Yarom.  Another Flip in the Wall of RowHammer Defenses.  In S&P, 2018.
  * [105] Daniel Gruss, Clémentine Maurice, and Stefan Mangard.  Rowhammer.js: A Remote Software-Induced Fault Attack in Javascript.  In CoRR, 2016.
  * [106] Chongyan Gu and Maire O’Neill.  Ultra-Compact and Robust FPGA-Based PUF Identification Generator.  In ISCAS, 2015.
  * [107] Jorge Guajardo, Sandeep S Kumar, Geert-Jan Schrijen, and Pim Tuyls.  FPGA Intrinsic PUFs and Their Use for IP Protection.  In CHES, 2007.
  * [108] Jorge Guajardo, Sandeep S Kumar, Geert-Jan Schrijen, and Pim Tuyls.  Physical Unclonable Functions and Public-key Crypto for FPGA IP Protection.  In FPL, 2007.
  * [109] Zvi Gutterman, Benny Pinkas, and Tzachy Reinman.  Analysis of the Linux Random Number Generator.  In SP, 2006.
  * [110] Tamas Gyorfi, Octavian Cret, and Alin Suciu.  High Performance True Random Number Generator Based on FPGA Block RAMs.  In IPDPS, 2009.
  * [111] Jawad Haj-Yahya, Mohammed Alser, Jeremie Kim, A. Giray Yaglikci, Nandita Vijaykumar, Efraim Rotem, and Onur Mutlu.  SysScale: Exploiting Multi-domain Dynamic Voltage and Frequency Scaling for Energy Efficient Mobile Processors.  In ISCA, 2020.
  * [112] Jawad Haj-Yahya, Mohammed Alser, Lois Orosa, Jeremie Kim, Efraim Rotem, Avi Mendelson, Anupam Chattopadhyay, and Onur Mutlu.  A Power- and Workload-aware Hybrid Power Delivery Network for Energy-efficient High-end Client Processors.  In MICRO, 2020.
  * [113] Takeshi Hamamoto, Soichi Sugiura, and Shizuo Sawada.  On the Retention Time Distribution of Dynamic Random Access Memory (DRAM).  In IEEE Transactions on Electron devices, volume 45, pages 1300–1309, 1998.
  * [114] Mike Hamburg, Paul Kocher, and Mark E Marson.  Analysis of Intel’s Ivy Bridge Digital Random Number Generator.  www.cryptography.com/public/pdf/Intel_TRNG_Report_20120312.pdf, 2012.
  * [115] Richard W Hamming.  Error Detecting and Error Correcting Codes.  In Bell Labs Technical Journal, 1950.
  * [116] Ghaith Hammouri, Erdinç Öztürk, Berk Birand, and Berk Sunar.  Unclonable Lightweight Authentication Scheme.  In ICICS, 2008.
  * [117] Maryam S Hashemian, Bhanu Singh, Francis Wolff, Daniel Weyer, Steve Clay, and Christos Papachristou.  A Robust Authentication Methodology Using Physically Unclonable Functions in DRAM Arrays.  In DATE, 2015.
  * [118] Hasan Hassan, Minesh Patel, Jeremie S Kim, A Giray Yağlıkçı, Nandita Vijaykumar, Nika Mansouri Ghiasi, Saugata Ghose, and Onur Mutlu.  CROW: A Low-Cost Substrate for Improving DRAM Performance, Energy Efficiency, and Reliability.  In ISCA, 2019.
  * [119] Hasan Hassan, Gennady Pekhimenko, Nandita Vijaykumar, Vivek Seshadri, Donghyuk Lee, Oguz Ergin, and Onur Mutlu.  ChargeCache: Reducing DRAM Latency by Exploiting Row Access Locality.  In HPCA, 2016.
  * [120] Hasan Hassan, Nandita Vijaykumar, Samira Khan, Saugata Ghose, Kevin Chang, Gennady Pekhimenko, Donghyuk Lee, Oguz Ergin, and Onur Mutlu.  SoftMC: A Flexible and Practical Open-source Infrastructure for Enabling Experimental DRAM Studies.  In HPCA, 2017.
  * [121] Syed Minhaj Hassan, Sudhakar Yalamanchili, and Saibal Mukhopadhyay.  Near Data Processing: Impact and Optimization of 3D Memory System Architecture on the Uncore.  In MEMSYS, 2015.
  * [122] Hisashi Hata and Shuichi Ichikawa.  FPGA Implementation of Metastability-based True Random Number Generator.  IEICE Transactions on Information and Systems, 95(2):426–436, 2012\.
  * [123] Ryan Helinski, Dhruva Acharyya, and Jim Plusquellic.  A Physical Unclonable Function Defined Using Power Distribution System Equivalent Resistance Variations.  In DAC, 2009.
  * [124] Hewlett-Packard Enterprise.  HP Moonshot Component Pack Version 2015.05.0.  http://h17007.www1.hp.com/us/en/enterprise/servers/products/moonshot/component-pack/index.aspx, 2015\.
  * [125] Hideto Hidaka, Yoshio Matsuda, Mikio Asakura, and Kazuyasu Fujishima.  The Cache DRAM Architecture: A DRAM with an on-chip Cache Memory.  MICRO, 1990.
  * [126] Daniel E Holcomb, Wayne P Burleson, and Kevin Fu.  Initial SRAM State as a Fingerprint and Source of True Random Numbers for RFID Tags.  In RFID, 2007.
  * [127] Daniel E Holcomb, Wayne P Burleson, and Kevin Fu.  Power-Up SRAM State as an Identifying Fingerprint and Source of True Random Numbers.  In IEEE Transactions on Computers, volume 58, pages 1198–1210, 2009\.
  * [128] Jeremy Holleman, Seth Bridges, Brian P Otis, and Chris Diorio.  A 3mu W CMOS True Random Number Generator with Adaptive Floating-Gate Offset Cancellation.  IEEE Journal of Solid-State Circuits, 43(5):1324–1336, 2008.
  * [129] Yohei Hori, Takahiro Yoshida, Toshihiro Katashita, and Akashi Satoh.  Quantitative and Statistical Performance Evaluation of Arbiter Physical Unclonable Functions on FPGAs.  In ReConFig, 2010.
  * [130] Kevin Hsieh, Eiman Ebrahimi, Gwangsun Kim, Niladrish Chatterjee, Mike O’Connor, Nandita Vijaykumar, Onur Mutlu, and Stephen W Keckler.  Transparent Offloading and Mapping (TOM): Enabling Programmer-Transparent Near-Data Processing in GPU Systems.  In ISCA, 2016.
  * [131] Kevin Hsieh, Samira Khan, Nandita Vijaykumar, Kevin K Chang, Amirali Boroumand, Saugata Ghose, and Onur Mutlu.  Accelerating Pointer Chasing in 3D-Stacked Memory: Challenges, Mechanisms, Evaluation.  In ICCD, 2016.
  * [132] SK Hynix.  DDR4 SDRAM Device Operation.
  * [133] Intel.  Intel Architecture Software Developer’s Manual, 2018.
  * [134] Intel Corporation.  CannonLake Intel Firmware Support Package (FSP) Integration Guide.  https://usermanual.wiki/Pdf/CannonLakeFSPIntegrationGuide.58784693.pdf, 2017\.
  * [135] Gorka Irazoqui, Thomas Eisenbarth, and Berk Sunar.  MASCAT: Stopping Microarchitectural Attacks Before Execution.  IACR Cryptology ePrint Archive, 2016.
  * [136] Anirudh Iyengar, Kenneth Ramclam, and Swaroop Ghosh.  DWM-PUF: A Low-Overhead, Memory-Based Security Primitive.  In HOST, 2014.
  * [137] Paul Jaccard.  Étude Comparative de la Distribution Florale dans une Portion des Alpes et des Jura.  In Bull Soc Vaudoise Sci Nat, 1901.
  * [138] JEDEC.  Double Data Rate 3 (DDR3) SDRAM Specification.  2012\.
  * [139] JEDEC.  Double Data Rate 4 (DDR4) SDRAM Standard.  2012\.
  * [140] JEDEC.  Low Power Double Data Rate 3 (LPDDR3).  2012\.
  * [141] JEDEC.  Low Power Double Data Rate 4 (LPDDR4) SDRAM Specification.  2014\.
  * [142] JEDEC.  Annex L: Serial Presence Detect (SPD) for DDR4 SDRAM Modules.  2015\.
  * [143] JEDEC Solid State Technology Association.  Failure Mechanisms and Models for Semiconductor Devices.  JEDEC Publication JEP122G, 2011.
  * [144] Min Kyu Jeong, Doe Hyun Yoon, Dam Sunwoo, Mike Sullivan, Ikhwan Lee, and Mattan Erez.  Balancing DRAM Locality and Parallelism in Shared Memory CMP Systems.  In HPCA, 2012.
  * [145] Sangwoo Ji, Youngjoo Ko, Saeyoung Oh, and Jong Kim.  Pinpoint Rowhammer: Suppressing Unwanted Bit Flips on Rowhammer Attacks.  In ASIACCS, 2019.
  * [146] Benjamin Jun and Paul Kocher.  The Intel Random Number Generator.  Cryptography Research Inc. white paper, 1999.
  * [147] Matthias Jung, Carl C Rheinländer, Christian Weis, and Norbert Wehn.  Reverse Engineering of DRAMs: Row Hammer with Crosshair.  In MEMSYS, 2016.
  * [148] Marcin Kaczmarski.  Thoughts on Intel Xeon e5–2600 v2 Product Family Performance Optimisation–Component Selection Guidelines, 2014.
  * [149] Kamal Y Kamal and Radu Muresan.  Mixed-signal Physically Unclonable Function with CMOS Capacitive Cells.  IEEE Access, 7:130977–130998, 2019.
  * [150] Ingab Kang, Eojin Lee, and Jung Ho Ahn.  CAT-TWO: Counter-Based Adaptive Tree, Time Window Optimized for DRAM Row-Hammer Prevention.  IEEE Access, 8:17366–17377, 2020.
  * [151] Uksong Kang, Hak-soo Yu, Churoo Park, Hongzhong Zheng, John Halbert, Kuljit Bains, S Jang, and Joo Sun Choi.  Co-Architecting Controllers and DRAM to Enhance DRAM Process Scaling.  In The Memory Forum, 2014.
  * [152] Nima Karimian and Fatemeh Tehranipoor.  How to Generate Robust Keys from Noisy DRAMs?  In GLSVLSI, 2019.
  * [153] Stefan Katzenbeisser, Ünal Kocabaş, Vladimir Rožić, Ahmad-Reza Sadeghi, Ingrid Verbauwhede, and Christian Wachsmann.  PUFs: Myth, Fact or Busted? A Security Evaluation of Physically Unclonable Functions (PUFs) Cast in Silicon.  In CHES, 2012.
  * [154] Christoph Keller, Frank Gurkaynak, Hubert Kaeslin, and Norbert Felber.  Dynamic Memory-based Physically Unclonable Function for the Generation of Unique Identifiers and True Random Numbers.  In ISCAS, 2014.
  * [155] Samira Khan, Donghyuk Lee, Yoongu Kim, Alaa R Alameldeen, Chris Wilkerson, and Onur Mutlu.  The Efficacy of Error Mitigation Techniques for DRAM Retention Failures: A Comparative Experimental Study.  In SIGMETRICS, 2014.
  * [156] Samira Khan, Donghyuk Lee, and Onur Mutlu.  PARBOR: An Efficient System-Level Technique to Detect Data-Dependent Failures in DRAM.  In DSN, 2016.
  * [157] Samira Khan, Chris Wilkerson, Donghyuk Lee, Alaa R Alameldeen, and Onur Mutlu.  A Case for Memory Content-Based Detection and Mitigation of Data-Dependent Failures in DRAM.  In IEEE Computer Architecture Letters, volume 16, pages 88–93, 2016\.
  * [158] Samira Khan, Chris Wilkerson, Zhe Wang, Alaa R Alameldeen, Donghyuk Lee, and Onur Mutlu.  Detecting and Mitigating Data-Dependent DRAM Failures by Exploiting Current Memory Content.  In MICRO, 2017.
  * [159] Dae-Hyun Kim, Prashant J Nair, and Moinuddin K Qureshi.  Architectural Support for Mitigating Row Hammering in DRAM Memories.  IEEE Computer Architecture Letters, 14(1):9–12, 2014.
  * [160] Jeeson Kim, Taimur Ahmed, Hussein Nili, Nhan Duy Truong, Jiawei Yang, Doo Seok Jeong, Sharath Sriram, Damith C Ranasinghe, and Omid Kavehei.  Nano-Intrinsic True Random Number Generation.  arXiv preprint arXiv:1701.06020, 2017.
  * [161] Jeremie S Kim, Damla Senol Cali, Hongyi Xin, Donghyuk Lee, Saugata Ghose, Mohammed Alser, Hasan Hassan, Oguz Ergin, Can Alkan, and Onur Mutlu.  GRIM-Filter: Fast Seed Location Filtering in DNA Read Mapping Using Processing-in-memory Technologies.  BMC Genomics, 19(2):23–40, 2018.
  * [162] Jeremie S Kim, Can Firtina, Damla Senol Cali, Mohammed Alser, Nastaran Hajinazar, Can Alkan, and Onur Mutlu.  AirLift: A Fast and Comprehensive Technique for Translating Alignments between Reference Genomes.  arXiv preprint arXiv:1912.08735, 2019.
  * [163] Jeremie S Kim, Minesh Patel, Hasan Hassan, and Onur Mutlu.  Solar-DRAM: Reducing DRAM Access Latency by Exploiting the Variation in Local Bitlines.  In ICCD, 2018.
  * [164] Jeremie S. Kim, Minesh Patel, Hasan Hassan, and Onur Mutlu.  The DRAM Latency PUF: Quickly Evaluating Physical Unclonable Functions by Exploiting the Latency-Reliability Tradeoff in Modern Commodity DRAM Devices.  In HPCA, 2018.
  * [165] Jeremie S Kim, Minesh Patel, Hasan Hassan, Lois Orosa, and Onur Mutlu.  D-RaNGe: Using Commodity DRAM Devices to Generate True Random Numbers with Low Latency and High Throughput.  In HPCA, 2019.
  * [166] Jeremie S. Kim, Minesh Patel, Abdullah G. Yaglikci, Hasan Hassan, Roknoddin Azizi, Lois Orosa, and Onur Mutlu.  Revisiting RowHammer: An Experimental Analysis of Modern DRAM Devices and Mitigation Techniques.  In ISCA, 2020.
  * [167] Jungrae Kim, Michael Sullivan, and Mattan Erez.  Bamboo ECC: Strong, Safe, and Flexible Codes for Reliable Computer Memory.  In HPCA, 2015.
  * [168] Kinam Kim and Jooyoung Lee.  A New Investigation of Data Retention Time in Truly Nanoscaled DRAMs.  In IEEE Electron Device Letters, volume 30, pages 846–848, 2009\.
  * [169] Moonsoo Kim, Jungwoo Choi, Hyun Kim, and Hyuk-Jae Lee.  An Effective DRAM Address Remapping for Mitigating Rowhammer Errors.  IEEE Transactions on Computers, 68(10):1428–1441, 2019.
  * [170] Yoongu Kim, Ross Daly, Jeremie Kim, Chris Fallin, Ji Hye Lee, Donghyuk Lee, Chris Wilkerson, Konrad Lai, and Onur Mutlu.  Flipping Bits in Memory Without Accessing Them: An Experimental Study of DRAM Disturbance Errors.  In ISCA, 2014.
  * [171] Yoongu Kim, Dongsu Han, Onur Mutlu, and Mor Harchol-Balter.  ATLAS: A Scalable and High-Performance Scheduling Algorithm for Multiple Memory Controllers.  In HPCA, 2010.
  * [172] Yoongu Kim, Michael Papamichael, Onur Mutlu, and Mor Harchol-Balter.  Thread Cluster Memory Scheduling: Exploiting Differences in Memory Access Behavior.  In MICRO, 2010.
  * [173] Yoongu Kim, Vivek Seshadri, Donghyuk Lee, Jamie Liu, and Onur Mutlu.  A Case for Exploiting Subarray-level Parallelism (SALP) in DRAM.  In ISCA, 2012.
  * [174] Yoongu Kim, Weikun Yang, and Onur Mutlu.  Ramulator: A Fast and Extensible DRAM Simulator.  In IEEE Computer architecture letters, volume 15, pages 45–49, 2016\.
  * [175] DJ Kinniment and EG Chester.  Design of an On-chip Random Number Generator using Metastability.  In ESSCIRC, 2002.
  * [176] Çetin Kaya Koç.  About Cryptographic Engineering.  In Cryptographic Engineering. 2009.
  * [177] Patrick Koeberl, Ünal Kocabaş, and Ahmad-Reza Sadeghi.  Memristor PUFs: A New Generation of Memory-based Physically Unclonable Functions.  In DATE, 2013.
  * [178] Radhesh Krishnan Konoth, Marco Oliverio, Andrei Tatar, Dennis Andriesse, Herbert Bos, Cristiano Giuffrida, and Kaveh Razavi.  ZebRAM: Comprehensive and Compatible Software Protection Against Rowhammer Attacks.  In OSDI, 2018.
  * [179] Sandeep S Kumar, Jorge Guajardo, Roel Maes, Geert-Jan Schrijen, and Pim Tuyls.  The Butterfly PUF Protecting IP on Every FPGA.  In HOST, 2008.
  * [180] Nohhyup Kwak, Saeng-Hwan Kim, Kyong Ha Lee, Chang-Ki Baek, Mun Seon Jang, Yongsuk Joo, Seung-Hun Lee, Woo Young Lee, Eunryeong Lee, Donghee Han, et al.  A 4.8 Gb/s/pin 2Gb LPDDR4 SDRAM with Sub-100$\mu$A Self-Refresh Current for IoT Applications.  In ISSCC, 2017.
  * [181] Sammy HM Kwok and Edmund Y Lam.  FPGA-based High-speed True Random Number Generator for Cryptographic Applications.  In TENCON, 2006.
  * [182] SiewHwee Kwok, YenLing Ee, Guanhan Chew, Kanghong Zheng, Khoongming Khoo, and ChikHow Tan.  A Comparison of Post-processing Techniques for Biased Random Number Generators.  In WISTP, 2011.
  * [183] Hye-Jung Kwon, Eunsung Seo, Chan-Yong Lee, Young-Hun Seo, Gong-Heum Han, Hye-Ran Kim, Jong-Ho Lee, Min-Su Jang, Sung-Geun Do, Seung-Hyun Cho, et al.  An Extremely Low-Standby-Power 3.733 Gb/s/pin 2Gb LPDDR4 SDRAM for Wearable Devices.  In ISSCC, 2017.
  * [184] Andrew Kwong, Daniel Genkin, Daniel Gruss, and Yuval Yarom.  RAMBleed: Reading Bits in Memory Without Accessing Them.  In SP, 2020.
  * [185] Quintessence Labs.  Random Number Generators White Paper, 2015.
  * [186] Chang Joo Lee, Veynu Narasiman, Eiman Ebrahimi, Onur Mutlu, and Yale N Patt.  DRAM-Aware Last-Level Cache Writeback: Reducing Write-Caused Interference in Memory Systems.  HPS Technical Report, 2010.
  * [187] Chang Joo Lee, Veynu Narasiman, Onur Mutlu, and Yale N Patt.  Improving Memory Bank-level Parallelism in the Presence of Prefetching.  In MICRO, 2009.
  * [188] Donghyuk Lee.  Reducing DRAM Latency at Low Cost by Exploiting Heterogeneity.  PhD thesis, Carnegie Mellon University, 2016.
  * [189] Donghyuk Lee, Saugata Ghose, Gennady Pekhimenko, Samira Khan, and Onur Mutlu.  Simultaneous Multi-Layer Access: Improving 3D-Stacked Memory Bandwidth at Low Cost.  In ACM Transactions on Architecture and Code Optimization (TACO), volume 12, pages 1–29, 2016.
  * [190] Donghyuk Lee, Samira Khan, Lavanya Subramanian, Saugata Ghose, Rachata Ausavarungnirun, Gennady Pekhimenko, Vivek Seshadri, and Onur Mutlu.  Design-Induced Latency Variation in Modern DRAM Chips: Characterization, Analysis, and Latency Reduction Mechanisms.  In SIGMETRICS, 2017.
  * [191] Donghyuk Lee, Yoongu Kim, Gennady Pekhimenko, Samira Khan, Vivek Seshadri, Kevin Chang, and Onur Mutlu.  Adaptive-latency DRAM: Optimizing DRAM Timing for the Common-case.  In HPCA, 2015.
  * [192] Donghyuk Lee, Yoongu Kim, Vivek Seshadri, Jamie Liu, Lavanya Subramanian, and Onur Mutlu.  Tiered-Latency DRAM: A Low Latency and Low Cost DRAM Architecture.  In HPCA, 2013.
  * [193] Donghyuk Lee, Lavanya Subramanian, Rachata Ausavarungnirun, Jongmoo Choi, and Onur Mutlu.  Decoupled Direct Memory Access: Isolating CPU and IO Traffic by Leveraging a Dual-Data-Port DRAM.  In PACT, 2015.
  * [194] Eojin Lee, Ingab Kang, Sukhan Lee, G Edward Suh, and Jung Ho Ahn.  TWiCe: Preventing Row-Hammering by Exploiting Time Window Counters.  In ISCA, 2019.
  * [195] Jae W Lee, Daihyun Lim, Blaise Gassend, G Edward Suh, Marten Van Dijk, and Srinivas Devadas.  A Technique to Build a Secret Key in Integrated Circuits for Identification and Authentication Applications.  In Symposium on VLSIC, 2004.
  * [196] Myoung Jin Lee and Kun Woo Park.  A Mechanism for Dependence of Refresh Time on Data Pattern in DRAM.  In IEEE Electron Device Letters, volume 31, pages 168–170, 2010\.
  * [197] Lenovo.  Row Hammer Privilege Escalation.  https://support.lenovo.com/us/en/product_security/row_hammer, 2015\.
  * [198] Congmiao Li and Jean-Luc Gaudiot.  Detecting Malicious Attacks Exploiting Hardware Vulnerabilities Using Performance Counters.  In COMPSAC, 2019.
  * [199] Yan Li, Helmut Schneider, Florian Schnabel, Roland Thewes, and Doris Schmitt-Landsiedel.  DRAM Yield Analysis and Optimization by a Statistical Design Approach.  In CSI, 2011.
  * [200] Daihyun Lim, Jae W Lee, Blaise Gassend, G Edward Suh, Marten Van Dijk, and Srinivas Devadas.  Extracting Secret Keys from Integrated Circuits.  VLSI, 2005.
  * [201] Chung Hsiang Lin, De-Yu Shen, Yi-Jung Chen, Chia-Lin Yang, and Michael Wang.  SECRET: Selective Error Correction for Refresh Energy Reduction in DRAMs.  In ICCD, 2012.
  * [202] Jiang Lin and Matthew Garrett.  Handling Maximum Activation Count Limit and Target Row Refresh in DDR4 SDRAM, 2017.  US Patent 9,589,606.
  * [203] Jie Lin, Wei Yu, Nan Zhang, Xinyu Yang, Hanlin Zhang, and Wei Zhao.  A Survey on Internet of Things: Architecture, Enabling Technologies, Security and Privacy, and Applications.  In IoT, 2017.
  * [204] Moritz Lipp, Misiker Tadesse Aga, Michael Schwarz, Daniel Gruss, Clémentine Maurice, Lukas Raab, and Lukas Lamster.  Nethammer: Inducing RowHammer Faults Through Network Requests.  arXiv preprint arXiv:1805.04956, 2018.
  * [205] Chao Qun Liu, Yuan Cao, and Chip Hong Chang.  ACRO-PUF: A Low-power, Reliable and Aging-Resilient Current Starved Inverter-Based Ring Oscillator Physical Unclonable Function.  In TCS, volume 64, pages 3138–3149, 2017.
  * [206] Jamie Liu, Ben Jaiyen, Yoongu Kim, Chris Wilkerson, and Onur Mutlu.  An Experimental Study of Data Retention Behavior in Modern DRAM Devices: Implications for Retention Time Profiling Mechanisms.  In ISCA, 2013.
  * [207] Jamie Liu, Ben Jaiyen, Richard Veras, and Onur Mutlu.  RAIDR: Retention-Aware Intelligent DRAM Refresh.  In ISCA, 2012.
  * [208] Rui Liu, Huaqiang Wu, Yachuan Pang, He Qian, and Shimeng Yu.  Experimental Characterization of Physical Unclonable Function Based on 1 KB Resistive Random Access Memory Arrays.  In IEEE Electron Device Letters, volume 36, pages 1380–1383, 2015\.
  * [209] Song Liu, Brian Leung, Alexander Neckar, Seda Ogrenci Memik, Gokhan Memik, and Nikos Hardavellas.  Hardware/Software Techniques for DRAM Thermal Management.  In HPCA, 2011.
  * [210] Wenchao Liu, Zhenhua Zhang, Miaoxin Li, and Zhenglin Liu.  A Trustworthy Key Generation Prototype Based on DDR3 PUF for Wireless Sensor Networks.  In Sensors, volume 14, pages 11542–11556, 2014.
  * [211] Zhiyu Liu, Irina Calciu, Maurice Herlihy, and Onur Mutlu.  Concurrent Data Structures for Near-Memory Computing.  In SPAA, 2017.
  * [212] Keith Lofstrom, W Robert Daasch, and Donald Taylor.  IC Identification Circuit Using Device Mismatch.  In ISSCC, 2000.
  * [213] Shih-Lien Lu, Ying-Chen Lin, and Chia-Lin Yang.  Improving DRAM Latency with Dynamic Asymmetric Subarray.  In MICRO, 2015.
  * [214] XiaoMing Lu, LiJun Zhang, YongGang Wang, Wei Chen, DaJun Huang, Deng Li, Shuang Wang, DeYong He, ZhenQiang Yin, Yu Zhou, et al.  FPGA Based Digital Phase-coding Quantum Key Distribution System.  Science China Physics, Mechanics & Astronomy, 2015.
  * [215] Chi-Keung Luk, Robert Cohn, Robert Muth, Harish Patil, Artur Klauser, Geoff Lowney, Steven Wallace, Vijay Janapa Reddi, and Kim Hazelwood.  Pin: Building Customized Program Analysis Tools with Dynamic Instrumentation.  In PLDI, 2005.
  * [216] Haocong Luo, Taha Shahroodi, Hasan Hassan, Minesh Patel, Abdullah Giray Yaglikci, Lois Orosa, Jisung Park, and Onur Mutlu.  CLR-DRAM: A Low-Cost DRAM Architecture Enabling Dynamic Capacity-Latency Trade-Off.  ISCA, 2020.
  * [217] Yixin Luo, Saugata Ghose, Yu Cai, Erich F Haratsch, and Onur Mutlu.  Enabling Accurate and Practical Online Flash Channel Modeling for Modern MLC NAND Flash Memory.  In IEEE Journal on Selected Areas in Communications, volume 34, pages 2294–2311, 2016.
  * [218] Yixin Luo, Saugata Ghose, Yu Cai, Erich F Haratsch, and Onur Mutlu.  HeatWatch: Improving 3D NAND Flash Memory Device Reliability by Exploiting Self-recovery and Temperature Awareness.  In HPCA, 2018.
  * [219] Yixin Luo, Saugata Ghose, Yu Cai, Erich F Haratsch, and Onur Mutlu.  Improving 3D NAND Flash Memory Lifetime by Tolerating Early Retention Loss and Process Variation.  SIGMETRICS, 2018.
  * [220] Xiongfeng Ma, Xiao Yuan, Zhu Cao, Bing Qi, and Zhen Zhang.  Quantum Random Number Generation.  Quantum Inf., 2016.
  * [221] Roel Maes, Pim Tuyls, and Ingrid Verbauwhede.  Intrinsic PUFs from Flip-flops on Reconfigurable Devices.  In WISSec, 2008.
  * [222] Roel Maes and Ingrid Verbauwhede.  Physically Unclonable Functions: A Study on the State of the Art and Future Research Directions.  In Towards Hardware-Intrinsic Security. 2010.
  * [223] Abhranil Maiti, Logan McDougall, and Patrick Schaumont.  The Impact of Aging on an Fpga-Based Physical Unclonable Function.  In FPL, 2011.
  * [224] Mehrdad Majzoobi, Farinaz Koushanfar, and Srinivas Devadas.  FPGA PUF Using Programmable Delay Lines.  In WIFS, 2010.
  * [225] Mehrdad Majzoobi, Farinaz Koushanfar, and Srinivas Devadas.  FPGA-based True Random Number Generation using Circuit Metastability with Adaptive Feedback Control.  In CHES, 2011.
  * [226] Mehrdad Majzoobi, Farinaz Koushanfar, and Miodrag Potkonjak.  Testing Techniques for Hardware Security.  In ITC, 2008.
  * [227] George Marsaglia.  The Marsaglia Random Number CDROM Including the Diehard Battery of Tests of Randomness.  2008\.  http://www.stat.fsu.edu/pub/diehard/.
  * [228] Kinga Marton and Alin Suciu.  On the Interpretation of Results from the NIST Statistical Test Suite.  Science and Technology, 2015.
  * [229] Sanu K Mathew, Suresh Srinivasan, Mark A Anders, Himanshu Kaul, Steven K Hsu, Farhana Sheikh, Amit Agarwal, Sudhir Satpathy, and Ram K Krishnamurthy.  2.4 Gbps, 7 mW All-digital PVT-variation Tolerant True Random Number Generator for 45 nm CMOS High-performance Microprocessors.  In IEEE Journal of Solid-State Circuits, volume 47, pages 2807–2821, 2012.
  * [230] Justin Meza, Qiang Wu, Sanjeev Kumar, and Onur Mutlu.  Revisiting Memory Errors in Large-scale Production Data Centers: Analysis and Modeling of New Trends from the Field.  In DSN, 2015.
  * [231] Rino Micheloni, Peter Z Onufryk, Alessia Marelli, Christopher IW Norrie, and Ihab Jaser.  Apparatus and Method Based on LDPC Codes for Adjusting a Correctable Raw Bit Error Rate Limit in a Memory System, 2015.  US Patent 9,092,353.
  * [232] Amir Morad, Leonid Yavits, and Ran Ginosar.  GP-SIMD Processing-in-Memory.  In ACM Transactions on Architecture and Code Optimization (TACO), volume 11, pages 1–26, 2015.
  * [233] Yuki Mori, Kiyonori Ohyu, Kensuke Okonogi, and R-I Yamada.  The Origin of Variable Retention Time in DRAM.  In IEDM, 2005.
  * [234] Thomas Moscibroda and Onur Mutlu.  Memory Performance Attacks: Denial of Memory Service in Multi-Core Systems.  In USENIX Security, 2007.
  * [235] Sven Müelich, Sebastian Bitzer, Chirag Sudarshan, Christian Weis, Norbert Wehn, Martin Bossert, and Robert FH Fischer.  Channel Models for Physical Unclonable Functions based on DRAM Retention Measurements.  In REDUNDANCY, 2019.
  * [236] Janani Mukundan, Hillery Hunter, Kyu-hyoun Kim, Jeffrey Stuecheli, and José F Martínez.  Understanding and Mitigating Refresh Overheads in High-density DDR4 DRAM Systems.  In ISCA, 2013.
  * [237] Sai Prashanth Muralidhara, Lavanya Subramanian, Onur Mutlu, Mahmut Kandemir, and Thomas Moscibroda.  Reducing Memory Interference in Multicore Systems via Application-aware Memory Channel Partitioning.  In MICRO, 2011.
  * [238] Onur Mutlu.  Memory Scaling: A Systems Architecture Perspective.  In IMW, 2013.
  * [239] Onur Mutlu.  The RowHammer Problem and Other Issues we may Face as Memory Becomes Denser.  In DATE, 2017.
  * [240] Onur Mutlu, Saugata Ghose, Juan Gómez-Luna, and Rachata Ausavarungnirun.  Processing Data Where it Makes Sense: Enabling In-memory Computation.  MicPro, 2019.
  * [241] Onur Mutlu and Jeremie S Kim.  RowHammer: A Retrospective.  TCAD, 2019.
  * [242] Onur Mutlu and Thomas Moscibroda.  Stall-Time Fair Memory Access Scheduling for Chip Multiprocessors.  In MICRO, 2007.
  * [243] Onur Mutlu and Thomas Moscibroda.  Parallelism-aware Batch Scheduling: Enhancing both Performance and Fairness of Shared DRAM Systems.  In ISCA, 2008.
  * [244] Onur Mutlu and Lavanya Subramanian.  Research Problems and Opportunities in Memory Systems.  In SUPERFRI, 2014.
  * [245] Prashant J Nair, Dae-Hyun Kim, and Moinuddin K Qureshi.  ArchShield: Architectural Framework for Assisting DRAM Scaling by Tolerating High Error Rates.  In ISCA, 2013.
  * [246] Prashant J Nair, Vilas Sridharan, and Moinuddin K Qureshi.  XED: Exposing On-Die Error Detection Information for Strong Memory Reliability.  In ISCA, 2016.
  * [247] Liao Ning, Jiang Ding, Bai Chuang, and Zou Xuecheng.  Design and Validation of High Speed True Random Number Generators Based on Prime-length Ring Oscillators.  The Journal of China Universities of Posts and Telecommunications, 22(4):1–6, 2015.
  * [248] Tae-Young Oh, Hoeju Chung, Jun-Young Park, Ki-Won Lee, Seunghoon Oh, Su-Yeon Doo, Hyoung-Joo Kim, ChangYong Lee, Hye-Ran Kim, Jong-Ho Lee, et al.  A 3.2Gbps/pin 8Gb 1.0V LPDDR4 SDRAM with Integrated ECC Engine for sub-1V DRAM Core Operation.  In ISSCC, 2014.
  * [249] Omron.  NY-series Industrial Box PC - Hardware User’s Manual.  https://assets.omron.eu/downloads/manual/en/v6/w553_ny-series_industrial_box_pc_users_manual_en.pdf, 2019\.
  * [250] Lois Orosa, Yaohua Wang, Ivan Puddu, Mohammad Sadrosadati, Kaveh Razavi, Juan Gómez-Luna, Hasan Hassan, Nika Mansouri-Ghiasi, Arash Tavakkol, Minesh Patel, Jeremie Kim, et al.  Dataplant: Enhancing System Security with Low-cost in-DRAM Value Generation Primitives.  arXiv preprint arXiv:1902.07344, 2019.
  * [251] Erdinç Öztürk, Ghaith Hammouri, and Berk Sunar.  Physical Unclonable Function with Tristate Buffers.  In ISCAS, 2008.
  * [252] Erdinç Öztürk, Ghaith Hammouri, and Berk Sunar.  Towards Robust Low Cost Authentication for Pervasive Devices.  In PerCom, 2008.
  * [253] Vijay S Pai and Sarita Adve.  Code Transformations to Improve Memory Parallelism.  In MICRO, 1999.
  * [254] Yachuan Pang, Huaqiang Wu, Bin Gao, Ning Deng, Dong Wu, Rui Liu, Shimeng Yu, An Chen, and He Qian.  Optimization of RRAM-based Physical Unclonable Function with a Novel Differential Read-Out Method.  In IEEE Electron Device Letters, volume 38, pages 168–171, 2017\.
  * [255] Fabio Pareschi, Gianluca Setti, and Riccardo Rovatti.  A Fast Chaos-based True Random Number Generator for Cryptographic Applications.  In ESSCIRC, 2006.
  * [256] Kyungbae Park, Chulseung Lim, Donghyuk Yun, and Sanghyeon Baeg.  Experiments and Root Cause Analysis for Active-Precharge Hammering Fault in DDR3 SDRAM under 3$\times$ nm Technology.  Microelectronics Reliability, 57:39–46, 2016.
  * [257] Kyungbae Park, Donghyuk Yun, and Sanghyeon Baeg.  Statistical Distributions of Row-hammering Induced Failures in DDR3 Components.  Microelectronics Reliability, 67:143–149, 2016.
  * [258] Minesh Patel, Jeremie Kim, Taha Shahroodi, Hasan Hassan, and Onur Mutlu.  Bit-Exact ECC Recovery (BEER): Determining DRAM On-Die ECC Functions by Exploiting DRAM Data Retention Characteristics.  MICRO, 2020.
  * [259] Minesh Patel, Jeremie S Kim, Hasan Hassan, and Onur Mutlu.  Understanding and Modeling On-Die Error Correction in Modern DRAM: An Experimental Study Using Real Devices.  In DSN, 2019.
  * [260] Minesh Patel, Jeremie S Kim, and Onur Mutlu.  The Reach Profiler (REAPER): Enabling the Mitigation of DRAM Retention Failures via Profiling at Aggressive Conditions.  In ISCA, 2017.
  * [261] Ashutosh Pattnaik, Xulong Tang, Adwait Jog, Onur Kayiran, Asit K Mishra, Mahmut T Kandemir, Onur Mutlu, and Chita R Das.  Scheduling Techniques for GPU Architectures with Processing-in-Memory Capabilities.  In PACT, 2016.
  * [262] Craig S Petrie and J Alvin Connelly.  A Noise-based IC Random Number Generator for Applications in Cryptography.  In IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, volume 47, pages 615–621, 2000.
  * [263] Ponemon Institute LLC.  Study on Mobile and IoT Application Security, 2017.
  * [264] Changwoo Pyo, Sungil Pae, and Gyungho Lee.  DRAM as Source of Randomness.  In Electronics Letters, volume 45, pages 26–27, 2009.
  * [265] Rui Qiao and Mark Seaborn.  A New Approach for RowHammer Attacks.  In HOST, 2016.
  * [266] Moinuddin K Qureshi, DaeHyun Kim, Samira Khan, Prashant J Nair, and Onur Mutlu.  AVATAR: A Variable-Retention-Time (VRT) Aware Refresh for DRAM Systems.  In DSN, 2015.
  * [267] Md Tauhidur Rahman, Kan Xiao, Domenic Forte, Xuhei Zhang, Jerry Shi, and Mohammad Tehranipoor.  TI-TRNG: Technology Independent True Random Number Generator.  In DAC, 2014.
  * [268] Amir Rahmati, Matthew Hicks, Daniel E Holcomb, and Kevin Fu.  Probable Cause: The Deanonymizing Effects of Approximate DRAM.  In ISCA, 2016.
  * [269] Biswajit Ray and Aleksandar Milenković.  True Random Number Generation Using Read Noise of Flash Memory Cells.  In IEEE Transactions on Electron Devices, volume 65, pages 963–969, 2018.
  * [270] Kaveh Razavi, Ben Gras, Erik Bosman, Bart Preneel, Cristiano Giuffrida, and Herbert Bos.  Flip Feng Shui: Hammering a Needle in the Software Stack.  In USENIX Security, 2016.
  * [271] Phillip J Restle, JW Park, and Brian F Lloyd.  DRAM Variable Retention Time.  In IEDM, pages 807–810, 1992.
  * [272] Ronald Rivest.  The MD5 Message-Digest Algorithm.  In RFC, 1992.
  * [273] Scott Rixner.  Memory Controller Optimizations For Web Servers.  In MICRO, 2004.
  * [274] Scott Rixner, William J Dally, Ujval J Kapasi, Peter Mattson, and John D Owens.  Memory Access Scheduling.  In ISCA, 2000.
  * [275] Scott Rixner, John D Owens, Peter Mattson, Ujval J Kapasi, and William J Dally.  Memory Access Scheduling.  In ISCA, 2000.
  * [276] Andrea Röck.  Pseudorandom Number Generators for Cryptographic Applications.  2005\.
  * [277] Garrett S Rose, Nathan McDonald, Lok-Kwong Yan, Bryant Wysocki, and Karen Xu.  Foundations of Memristor Based PUF Architectures.  In NANOARCH, 2013.
  * [278] Ulrich Rührmair, Jan Sölter, and Frank Sehnke.  On the Foundations of Physical Unclonable Functions.  In IACR Cryptology Archive, 2009.
  * [279] Andrew Rukhin, Juan Soto, James Nechvatal, Miles Smid, and Elaine Barker.  A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications.  Technical report, Booz-Allen and Hamilton Inc Mclean Va, 2001.
  * [280] Seong-Wan Ryu, Kyungkyu Min, Jungho Shin, Heimi Kwon, Donghoon Nam, Taekyung Oh, Tae-Su Jang, Minsoo Yoo, Yongtaik Kim, and Sungjoo Hong.  Overcoming the Reliability Limitation in the Ultimately Scaled DRAM using Silicon Migration Technique by Hydrogen Annealing.  In IEEE International Electron Devices Meeting, 2017.
  * [281] Samsung.  S5P4418 Application Processor Revision 0.10.  2014\.
  * [282] André Schaller, Wenjie Xiong, Nikolaos Athanasios Anagnostopoulos, Muhammad Umair Saleem, Sebastian Gabmeyer, Stefan Katzenbeisser, and Jakub Szefer.  Intrinsic Rowhammer PUFs: Leveraging the Rowhammer Effect for Improved Security.  In HOST, 2017.