DISS. ETH NO. 28621

IMPROVING DRAM PERFORMANCE, RELIABILITY, AND SECURITY
BY RIGOROUSLY UNDERSTANDING INTRINSIC DRAM OPERATION

A thesis submitted to attain the degree of

DOCTOR OF SCIENCES of ETH ZÜRICH

(Dr. sc. ETH Zürich)

presented by

HASAN HASSAN

B.Sc. & M.Sc., TOBB University of Economics and Technology

born on 4 July 1991

accepted on the recommendation of

Prof. Dr. Onur Mutlu, examiner

Prof. Dr. Derek Chiou, co-examiner

Prof. Dr. Mattan Erez, co-examiner

Dr. Mike O’Connor, co-examiner

Prof. Dr. Moinuddin Qureshi, co-examiner

Dr. Christian Weis, co-examiner

2022

Hasan Hassan: Improving DRAM Performance, Reliability, and Security by
Rigorously Understanding Intrinsic DRAM Operation, © 2022

DOI: TBD

To my wife, Seray.

## Acknowledgments

I received tremendous help from many people throughout the six years I spent
at ETH Zürich.

Foremost, I extend my deepest gratitude to my advisor, Onur Mutlu, for his
continuous support and guidance during my PhD journey. Back when I visited his
research group at CMU as an intern, his passion in teaching and research was
what steered me towards pursuing a PhD degree. As my PhD advisor, he has
always been patient with me and provided insightful criticism and advice on my
research. Without him, I may not have even attempted to pursue a PhD degree
and could not have written this thesis.

I would like to express my sincere gratitude to my PhD committee members,
Derek Chiou, Mattan Erez, Mike O’Connor, Moinuddin Qureshi, and Christian
Weis, for their time and efforts to review my thesis and provide valuable
feedback.

I am grateful to all SAFARI group members for providing stimulating
intellectual environment and their friendship. I thank Jeremie Kim and Minesh
Patel for their company since my early days at ETH as friends and excellent
researchers. Surviving PhD would not be possible without them. I thank the
other members of SAFARI, including Giray Yağlıkçı, Can Fırtına, Juan Gómez
Luna, Ataberk Olgun, Haocong Luo, Geraldo De Oliveira, Lois Orosa, Nisa
Bostancı, Rahul Bera, Yahya Can Tuğrul, Konstantinos Kanellopoulos, Jisung
Park, Nika Mansouri Ghiasi, Mohammed Alser, Jawad Haj-Yahya, Arash Tavakkol,
Yaohua Wang, Joël Lindegger, Mohammad Sadrosadati, Roknoddin Azizibarzoki,
João Ferreira, Christian Rossi, Tracy Ewen, and many others for their
collaboration and support.

I am also thankful to wonderful collaborators I had the chance to work with
during my PhD. I thank Saugata Ghose for the great assistance he provided
during my time at CMU and after. I also thank many others that I met at CMU,
including Nandita Vijaykumar, Vivek Seshadri, Samira Khan, Donghyuk Lee,
Gennady Pekhimenko, Amirali Boroumand, Kevin Chang, Yixin Luo, Lavanya
Subramanian, Rachata Ausavarungnirun, Damla Senol, and Nastaran Hajinazar. I
thank Kaveh Razavi for fascinating discussions on various security topics. I
thank Victor van der Veen, Pietro Frigo, and Emanuele Vannacci for their
contributions. I thank my internship mentors Stephan Meier and Tyler Huberty
for their guidance during my time at Apple.

Finally, I would like to thank my family for their endless support,
encouragement, and love. I thank my parents, Meryem and Ibrahim, and my
sister, Ayşegül, for always standing by me and enabling all this. Above all I
would like to thank my wife Seray, who kept me motivated to write this thesis
with her unwavering support and love.

## Abstract

DRAM is the primary technology used for main memory in modern systems.
Unfortunately, as DRAM scales down to smaller technology nodes, it faces key
challenges in both data integrity and latency, which strongly affect overall
system reliability, security, and performance. To develop reliable, secure,
and high-performance DRAM-based main memory for future systems, it is critical
to rigorously characterize, analyze, and understand various aspects (e.g.,
reliability, retention, latency, RowHammer vulnerability) of existing DRAM
chips and their architecture. The goal of this dissertation is to 1) develop
techniques and infrastructures to enable such rigorous characterization,
analysis, and understanding, and 2) enable new mechanisms to improve DRAM
performance, reliability, and security based on the developed understanding.

To this end, in this dissertation, we 1) design, implement, and prototype a
new practical-to-use and flexible FPGA-based DRAM characterization
infrastructure (called SoftMC), 2) use the DRAM characterization
infrastructure to develop a new experimental methodology (called U-TRR) to
uncover the operation of existing proprietary in-DRAM RowHammer protection
mechanisms and craft new RowHammer access patterns to efficiently circumvent
these RowHammer protection mechanisms, 3) propose a new DRAM architecture,
called Self-Managing DRAM, for enabling autonomous and efficient in-DRAM
maintenance operations that enable not only better performance, efficiency,
and reliability but also faster and easier adoption of changes to DRAM chips,
and 4) propose a versatile DRAM substrate, called the Copy-Row (CROW)
substrate, that enables new mechanisms for improving DRAM performance, energy
consumption, and reliability.

SoftMC. To develop reliable and high-performance DRAM-based main memory in
future systems, it is critical to experimentally characterize, understand, and
analyze various aspects (e.g., reliability, latency) of existing DRAM chips.
To enable this, there is a strong need for a publicly-available DRAM testing
infrastructure that can flexibly and efficiently test DRAM chips in a manner
accessible to both software and hardware developers. To this end, we design
and prototype SoftMC: a flexible and practical FPGA-based DRAM testing
infrastructure. SoftMC implements all low-level DRAM operations (i.e., DDR
commands) available in a typical memory controller (e.g., opening a row in a
bank, reading a specific column address, performing a refresh operation,
enforcing various timing constraints between commands). Using these low-level
operations, SoftMC can test and characterize any (existing or new) DRAM
mechanism that uses the existing DDR interface. SoftMC provides its users with
a simple and intuitive high-level programming interface that completely hides
the low-level details of the FPGA. SoftMC is freely available as an open-
source tool and it has enabled many research projects since its release,
leading to new understanding and new techniques.

U-TRR. RowHammer is a critical vulnerability in modern DRAM chips that can
lead to reliability, safety, and security problems in computing systems. As
such, DRAM vendors have been implementing techniques to protect DRAM chips
against RowHammer. We challenge the claim of DRAM vendors that their DRAM
chips are completely protected against RowHammer using proprietary,
undocumented, and obscure on-die Target Row Refresh (TRR) mechanisms. To
assess the security guarantees of recent DRAM chips, we develop Uncovering TRR
(U-TRR), a new experimental methodology to analyze in-DRAM TRR
implementations. U-TRR is based on the new observation that data retention
failures in DRAM enable a side channel that leaks information on how TRR
refreshes potential victim rows. U-TRR allows us to (i) understand how logical
DRAM rows are laid out physically in silicon; (ii) study undocumented on-die
TRR mechanisms; and (iii) combine (i) and (ii) to evaluate the RowHammer
security guarantees of modern DRAM chips. We show how U-TRR allows us to craft
RowHammer access patterns that successfully circumvent the TRR mechanisms
employed in 45 DRAM modules of the three major DRAM vendors. We find that the
DRAM modules we analyze are vulnerable to RowHammer, having bit flips in up to
99.9% of all DRAM rows and that simple error-correcting codes cannot prevent
bit flips. As such, more robust techniques to protect against the RowHammer
vulnerability are necessary. We publicly release the source code of our
implementation of the U-TRR methodology.

Self-Managing DRAM. To ensure reliable and secure DRAM operation, three types
of maintenance operations are typically required: 1) DRAM refresh, 2)
RowHammer protection, and 3) memory scrubbing. The reliability and security of
DRAM chips continuously worsen as DRAM technology node scales to smaller
sizes. Consequently, new DRAM chip generations necessitate making existing
maintenance operations more aggressive (e.g., lowering the refresh period) and
introducing new types of maintenance operations (e.g., targeted refresh for
mitigating RowHammer) while keeping the overheads of maintenance operations
minimal. Unfortunately, modifying the existing DRAM maintenance operations is
difficult due to the current rigid DRAM interface that places the memory
controller completely in charge of DRAM control. Implementing new or modifying
existing maintenance operations often require difficult-to-realize changes in
the DRAM interface, the memory controller, and potentially other system
components (e.g., system software). Our goal is to 1) ease, and thus
accelerate, the process of implementing new DRAM maintenance operations and 2)
enable more efficient in-DRAM maintenance operations. To this end, we propose
Self-Managing DRAM (SMD), a new low-cost DRAM architecture that enables
implementing new in-DRAM maintenance mechanisms with no further changes in the
DRAM interface, memory controller, or other system components. We use SMD to
implement six maintenance mechanisms for three use cases: 1) DRAM refresh, 2)
RowHammer protection, and 3) memory scrubbing. Our evaluations show that SMD-
based maintenance operations have significantly lower system performance and
energy overheads compared to conventional DDR4 DRAM. A combination of SMD-
based maintenance mechanisms that perform refresh, RowHammer protection, and
memory scrubbing achieve significant speedup and lower DRAM energy across a
wide variety of system configurations. SMD’s benefits increase as DRAM chips
become denser. We publicly release all SMD source code and data.

CROW. Three major challenges to DRAM scaling (i.e., high access latencies,
high refresh overheads, and increasing reliability problems like RowHammer)
are difficult to solve efficiently by directly modifying the underlying cell
array structure. This is because commodity DRAM implements an extremely dense
DRAM cell array that is optimized for low area-per-bit. Because of its
density, even a small change in the DRAM cell array structure may incur non-
negligible area overhead. Thus, we would like to lower the DRAM access
latency, reduce the refresh overhead, and improve DRAM reliability with no
changes to the DRAM cell architecture, and with only minimal changes to the
DRAM chip. To this end, we propose Copy-Row DRAM (CROW), a flexible substrate
that enables new mechanisms for improving DRAM performance, energy efficiency,
and reliability. We use the CROW substrate to implement 1) a low-cost in-DRAM
caching mechanism that lowers DRAM activation latency to frequently-accessed
rows by 38% and 2) a mechanism that avoids the use of short-retention-time
rows to mitigate the performance and energy overhead of DRAM refresh
operations. CROW’s flexibility allows the implementation of both mechanisms at
the same time. Our evaluations show that the two CROW-based mechanisms
synergistically improve system performance by 20.0% and reduce DRAM energy by
22.3% for memory-intensive four-core workloads. We publicly release the source
code of CROW.

Holistically, via these four major contributions, this dissertation shows the
importance of rigorously characterizing the reliability, latency, and
RowHammer vulnerability of existing DRAM chips and understanding their
architecture for developing practical and low-overhead mechanisms for
efficiently improving DRAM reliability, security, and performance. We believe
and hope that, via the new infrastructure, understanding, and techniques we
develop and enable, this dissertation encourages similar experimental
understanding-driven innovation in the design of future memories.

## Zusammenfassung

DRAM ist die wichtigste Hauptspeicher-Technologie in modernen Systemen. Leider
steht DRAM bei der Verkleinerung auf kleinere Technologieknoten vor grossen
Herausforderungen in Bezug auf Datenintegrität und Latenz, die die
Zuverlässigkeit, Sicherheit und Leistung des Gesamtsystems stark
beeinträchtigen. Um zuverlässige, sichere und leistungsstarke DRAM-basierte
Hauptspeicher für künftige Systeme zu entwickeln, müssen verschiedene Aspekte
(z. B. Zuverlässigkeit, Datenerhaltung, Latenz, RowHammer-Anfälligkeit) der
vorhandenen DRAM-Chips und ihrer Architektur genau charakterisiert, analysiert
und verstanden werden. Ziel dieser Dissertation ist es, 1) Techniken und
Infrastrukturen zu entwickeln, die eine solche rigorose Charakterisierung,
Analyse und ein solches rigoroses Verständnis ermöglichen, und 2) neue
Mechanismen zur Verbesserung der DRAM-Leistung, -Zuverlässigkeit und
-Sicherheit auf der Grundlage des entwickelten Verständnisses zu ermöglichen.

Zu diesem Zweck entwickeln, implementieren und prototypisieren wir in dieser
Dissertation 1) eine neue praktisch einsetzbare und flexible FPGA-basierte
DRAM-Charakterisierungsinfrastruktur (genannt SoftMC), 2) die DRAM-
Charakterisierungsinfrastruktur zur Entwicklung einer neuen experimentellen
Methodik (genannt U-TRR) verwendet, um die Funktionsweise bestehender
proprietärer In-DRAM RowHammer-Schutzmechanismen aufzudecken und neue
RowHammer-Zugriffsmuster zu entwickeln, um diese RowHammer-Schutzmechanismen
effizient zu umgehen, 3) eine neue DRAM-Architektur, genannt Self-Managing
DRAM, vorschlagen, um autonome und effiziente In-DRAM Wartungsvorgänge zu
ermöglichen, die nicht nur eine bessere Leistung, Effizienz und
Zuverlässigkeit, sondern auch eine schnellere und einfachere Annahme von
Änderungen an DRAM-Chips ermöglichen, und 4) ein vielseitiges DRAM-Substrat,
genannt Copy-Row (CROW)-Substrat, vorschlagen, das neue Mechanismen zur
Verbesserung der DRAM-Leistung, des Energieverbrauchs und der Zuverlässigkeit
ermöglicht.

SoftMC. Für die Entwicklung zuverlässiger und leistungsfähiger DRAM-basierter
Hauptspeicher in zukünftigen Systemen ist es von entscheidender Bedeutung,
verschiedene Aspekte (z. B. Zuverlässigkeit, Latenz) der vorhandenen DRAM-
Chips experimentell zu charakterisieren, zu verstehen und zu analysieren. Um
dies zu ermöglichen, besteht ein dringender Bedarf an einer öffentlich
zugänglichen DRAM-Testinfrastruktur, mit der DRAM-Chips flexibel und effizient
getestet werden können und die sowohl für Software- als auch für Hardware-
Entwickler zugänglich ist. Zu diesem Zweck entwickeln und prototypisieren wir
SoftMC: eine flexible und praktische FPGA-basierte DRAM-Testinfrastruktur.
SoftMC implementiert alle Low-Level-DRAM-Operationen (d.h. DDR-Befehle), die
in einem typischen Speicher-Controller verfügbar sind (z.B. Öffnen einer Zeile
in einer Bank, Lesen einer bestimmten Spaltenadresse, Durchführen einer
Refresh-Operation, Durchsetzen verschiedener Timing-Beschränkungen zwischen
Befehlen). Mit diesen Low-Level-Operationen kann SoftMC jeden (existierenden
oder neuen) DRAM-Mechanismus, der die existierende DDR-Schnittstelle nutzt,
testen und charakterisieren. SoftMC bietet seinen Benutzern eine einfache und
intuitive High-Level-Programmierschnittstelle, die die Low-Level-Details des
FPGAs vollständig verbirgt. SoftMC ist als Open-Source-Tool frei verfügbar und
hat seit seiner Veröffentlichung viele Forschungsprojekte ermöglicht, die zu
neuen Erkenntnissen und neuen Techniken geführt haben.

U-TRR. RowHammer ist eine kritische Schwachstelle in modernen DRAM-Chips, die
zu Zuverlässigkeits-, Sicherheits- und Schutzproblemen in Computersystemen
führen kann. Aus diesem Grund haben DRAM-Hersteller Techniken zum Schutz von
DRAM-Chips vor RowHammer implementiert. Wir hinterfragen die Behauptung von
DRAM-Anbietern, ihre DRAM-Chips seien durch proprietäre, undokumentierte und
undurchsichtige on-die Target Row Refresh (TRR)-Mechanismen vollständig gegen
RowHammer geschützt. Um die Sicherheitsgarantien aktueller DRAM-Chips zu
bewerten, entwickeln wir Uncovering TRR (U-TRR), eine neue experimentelle
Methode zur Analyse von TRR-Implementierungen im DRAM. U-TRR basiert auf der
neuen Beobachtung, dass Fehler bei der Datenerhaltung in DRAM einen
Seitenkanal ermöglichen, der Informationen darüber preisgibt, wie TRR
potenzielle Opfer-Zeilen refresht. U-TRR ermöglicht es uns, (i) zu verstehen,
wie logische DRAM-Zeilen physisch in auf dem chip angeordnet sind; (ii)
undokumentierte on-die TRR-Mechanismen zu untersuchen; und (iii) Punkte (i)
und (ii) zu kombinieren, um die RowHammer-Sicherheitsgarantien moderner DRAM-
Chips zu bewerten. Wir zeigen, wie U-TRR es uns ermöglicht, RowHammer-
Zugriffsmuster zu erstellen, die erfolgreich die TRR-Mechanismen umgehen, die
in 45 DRAM-Modulen der drei großen DRAM-Hersteller eingesetzt werden. Wir
stellen fest, dass die von uns analysierten DRAM-Module anfällig für RowHammer
sind, da in bis zu 99,9% aller DRAM-Zeilen Bitflips auftreten, und dass
einfache Fehlerkorrekturcodes Bitflips nicht verhindern können. Daher werden
robustere Techniken zur Verhinderung der RowHammer-Schwachstelle benötigt. Wir
veröffentlichen den Quellcode unserer Implementierung der U-TRR-Methode.

Self-Managing DRAM. Um einen zuverlässigen und sicheren DRAM-Betrieb zu
gewährleisten, sind in der Regel drei Arten von Wartungsvorgängen
erforderlich: 1) DRAM-Refresh, 2) RowHammer-Schutz und 3) Memory-Scrubbing.
Die Zuverlässigkeit und Sicherheit von DRAM-Chips verschlechtert sich ständig,
da die DRAM-Technologieknoten immer kleiner werden. Folglich müssen bei neuen
DRAM-Chip-Generationen die bestehenden Wartungsvorgänge aggressiver gestaltet
werden (z. B. Verringerung des Refresh-Periode) und neue Arten von
Wartungsvorgängen eingeführt werden (z. B. gezielte Refreshes zur Milderung
von RowHammer), während der Overhead der Wartungsvorgänge minimal gehalten
wird. Leider ist die Änderung der bestehenden DRAM-Wartungsvorgänge aufgrund
der derzeitigen starren DRAM-Schnittstelle, die die DRAM-Steuerung vollständig
dem Speicher-Controller überlässt, schwierig. Die Implementierung neuer oder
die Änderung bestehender Wartungsvorgänge erfordert häufig schwer zu
realisierende Änderungen an der DRAM-Schnittstelle, dem Speicher-Controller
und anderen Systemkomponenten (z. B. Systemsoftware). Unser Ziel ist es, 1)
den Prozess der Implementierung neuer DRAM-Wartungsvorgänge zu vereinfachen
und damit zu beschleunigen und 2) effizientere In-DRAM-Wartungsvorgänge zu
ermöglichen. Zu diesem Zweck schlagen wir Self-Managing DRAM (SMD) vor, eine
neue, kostengünstige DRAM-Architektur, die die Implementierung neuer In-DRAM
Wartungsmechanismen ohne weitere Änderungen an der DRAM-Schnittstelle, dem
Speicher-Controller oder anderen Systemkomponenten ermöglicht. Wir verwenden
SMD zur Implementierung von sechs Wartungsmechanismen für drei
Anwendungsfälle: 1) DRAM-Refresh, 2) RowHammer-Schutz und 3) Memory-Scrubbing.
Unsere Auswertungen zeigen, dass SMD-basierte Wartungsvorgänge im Vergleich zu
konventionellem DDR4-DRAM eine deutlich geringere Systemleistungs- und
Energiekosten haben. Eine Kombination aus SMD-basierten Wartungsmechanismen,
die Refresh-, RowHammer-Schutz- und Memory-Scrubbing-Operationen durchführen,
führt zu einer erheblichen Beschleunigung und einem geringeren DRAM-
Energieverbrauch in einer Vielzahl von Systemkonfigurationen. Die Vorteile von
SMD nehmen zu, je dichter die DRAM-Chips werden. Wir veröffentlichen den
gesamten SMD-Quellcode und -Daten.

CROW. Die drei grössten Herausforderungen bei der Skalierung von DRAM (d. h.
hohe Zugriffslatenzen, hohe Refresh-Kostenund zunehmende
Zuverlässigkeitsprobleme wie RowHammer) lassen sich nur schwer durch direkte
Änderung der zugrunde liegenden Zell-Array-Struktur effizient lösen. Dies
liegt daran, dass handelsüblicher DRAM ein extrem dichtes DRAM-Zellen-Array
implementiert, das auf eine geringe Fläche pro Bit optimiert ist. Aufgrund
dieser Dichte kann selbst eine kleine Änderung der DRAM-Zellen-Array-Struktur
zu einem nicht zu vernachlässigenden Flächen-Overhead führen. Daher möchten
wir die DRAM-Zugriffslatenz verringern, die Refresh-Kosten- reduzieren und die
DRAM-Zuverlässigkeit verbessern, ohne die DRAM-Zellenarchitektur zu verändern
und mit nur minimalen Änderungen am DRAM-Chip. Zu diesem Zweck schlagen wir
Copy-Row DRAM (CROW) vor, ein flexibles Substrat, das neue Mechanismen zur
Verbesserung der DRAM-Leistung, Energieeffizienz und Zuverlässigkeit
ermöglicht. Wir verwenden das CROW-Substrat, um 1) einen kostengünstigen In-
DRAM Caching-Mechanismus zu implementieren, der die DRAM-Aktivierungslatenz
für häufig genutzte Zeilen um 38% verringert, und 2) einen Mechanismus, der
die Verwendung von Zeilen mit kurzer Datenerhaltungsdauer vermeidet, um den
Leistungs- und Energie-Overhead von DRAM-Refresh-perationen zu verringern. Die
Flexibilität von CROW ermöglicht die gleichzeitige Implementierung beider
Mechanismen. Unsere Auswertungen zeigen, dass die beiden CROW-basierten
Mechanismen die Systemleistung um 20,0% verbessern und den DRAM-
Energieverbrauch bei speicherintensiven Vier-Kern-Workloads um 22,3% senken.
Wir veröffentlichen den Quellcode von CROW.

Mit diesen vier Hauptbeiträgen zeigt diese Dissertation, wie wichtig es ist,
die Zuverlässigkeit, Latenz und RowHammer-Anfälligkeit bestehender DRAM-Chips
genau zu charakterisieren und ihre Architektur zu verstehen, um praktische und
kostengünstige Mechanismen zur effizienten Verbesserung der Zuverlässigkeit,
Sicherheit und Leistung von DRAM zu entwickeln. Wir glauben und hoffen, dass
diese Dissertation durch die neue Infrastruktur, das Verständnis und die
Techniken, die wir entwickeln und ermöglichen, ähnliche, auf experimentellem
Verständnis beruhende Innovationen bei der Entwicklung zukünftiger Speicher
anregt.

######  ?contentsname?

  1. Acknowledgments
  2. Abstract
  3. Zusammenfassung
  4. 1 Introduction
    1. 1.1 Problem Discussion
    2. 1.2 Thesis Statement
    3. 1.3 Our Approach
      1. 1.3.1 Infrastructure for Characterizing DRAM Chips
      2. 1.3.2 Understanding in-DRAM RowHammer Protection Mechanisms
      3. 1.3.3 Enabling Easy Adoption of Efficient in-DRAM Maintenance Operations
      4. 1.3.4 Improving DRAM Access Latency and Reducing DRAM Refresh Overhead via Simultaneous Row Activation
    4. 1.4 Contributions
    5. 1.5 Outline
  5. 2 Background
    1. 2.1 DRAM Organization
    2. 2.2 DRAM Operation
      1. 2.2.1 Activate (ACT)
      2. 2.2.2 Read (RD)
      3. 2.2.3 Write (WR)
      4. 2.2.4 Precharge (PRE)
      5. 2.2.5 Refresh (REF)
    3. 2.3 DRAM Performance Issues
      1. 2.3.1 DRAM Access Latency
      2. 2.3.2 DRAM Refresh Overhead
    4. 2.4 DRAM Reliability Issues
      1. 2.4.1 Charge Leakage and Data Retention
      2. 2.4.2 Violating Access and Refresh Timings
      3. 2.4.3 Variable Retention Time (VRT)
      4. 2.4.4 Environmental Factors
    5. 2.5 DRAM Security Issues
      1. 2.5.1 RowHammer
      2. 2.5.2 Cold Boot Attacks
  6. 3 The SoftMC Infrastructure
    1. 3.1 Motivation and Goal
    2. 3.2 Related Work
    3. 3.3 SoftMC Design
      1. 3.3.1 High-Level Design
      2. 3.3.2 DRAM Command Interface
      3. 3.3.3 SoftMC API
      4. 3.3.4 Instruction Types and Encoding
      5. 3.3.5 Hardware Architecture
      6. 3.3.6 SoftMC Prototype
    4. 3.4 Example Use Cases
      1. 3.4.1 Retention Time Distribution Study
      2. 3.4.2 Evaluating the Expected Effect of Two Recently-Proposed Mechanisms in Existing DRAM Chips
    5. 3.5 Limitations of SoftMC
    6. 3.6 Research Directions Enabled by SoftMC
    7. 3.7 Other Related Work
    8. 3.8 Summary
  7. 4 Uncovering Target Row Refresh
    1. 4.1 Motivation and Goal
    2. 4.2 Overview of U-TRR
      1. 4.2.1 Overview of _Row Scout_ (_RS_)
      2. 4.2.2 Overview of _TRR Analyzer_ (_TRR-A_)
      3. 4.2.3 Required Experimental Setup for U-TRR
    3. 4.3 Row Scout (_RS_)
      1. 4.3.1 Row Scout (_RS_) Requirements
      2. 4.3.2 Row Scout (_RS_) Operation
    4. 4.4 Analyzing TRR-induced Refresh
      1. 4.4.1 TRR Analyzer Requirements
      2. 4.4.2 TRR Analyzer Operation
      3. 4.4.3 Determining Physical Row Mapping
    5. 4.5 Reverse-Engineering TRR
      1. 4.5.1 Vendor A
      2. 4.5.2 Vendor B
      3. 4.5.3 Vendor C
    6. 4.6 Bypassing TRR Using U-TRR Observations
      1. 4.6.1 Custom RowHammer Access Patterns
      2. 4.6.2 Effect on RowHammer Bit Flip Count
      3. 4.6.3 Effect on Individual Rows
      4. 4.6.4 Bypassing System-Level ECC Using U-TRR
    7. 4.7 Related Work
    8. 4.8 Summary
  8. 5 Self-Managing DRAM
    1. 5.1 Motivation and Goal
    2. 5.2 Motivation
      1. 5.2.1 DRAM Refresh
      2. 5.2.2 RowHammer Protection
      3. 5.2.3 Memory Scrubbing
    3. 5.3 Self-Managing DRAM
      1. 5.3.1 Overview of
      2. 5.3.2 Handling an ACT_NACK Signal
      3. 5.3.3 Region Locking Mechanism
      4. 5.3.4 Impact to Request Scheduling
    4. 5.4 SMD Maintenance Mechanisms
      1. 5.4.1 Use Case 1: DRAM Refresh
      2. 5.4.2 Use Case 2: RowHammer Protection
      3. 5.4.3 Use Case 3: Memory Scrubbing
    5. 5.5 Experimental Methodology
    6. 5.6 Evaluation
      1. 5.6.1 Single-core Performance
      2. 5.6.2 Multi-core Performance
      3. 5.6.3 Energy Consumption
      4. 5.6.4 Sensitivity Studies
      5. 5.6.5 Hardware Overhead
    7. 5.7 Related Work
    8. 5.8 Summary
  9. 6 Copy-Row DRAM
    1. 6.1 Motivation and Goal
    2. 6.2 Copy-Row DRAM
      1. 6.2.1 CROW: A High-Level Overview
      2. 6.2.2 Copy Rows
      3. 6.2.3 CROW Table
    3. 6.3 Applications of CROW
      1. 6.3.1 In-DRAM Caching (CROW-cache)
      2. 6.3.2 Reducing Refresh Overhead (CROW-ref)
      3. 6.3.3 Mitigating RowHammer
    4. 6.4 Circuit Simulations
      1. 6.4.1 Simultaneous Row Activation Latency
      2. 6.4.2 MRA-Based Row Copy Latency
    5. 6.5 Hardware Overhead
      1. 6.5.1 Memory Controller Overhead
      2. 6.5.2 DRAM Die Area Overhead
    6. 6.6 Methodology
    7. 6.7 Evaluation
      1. 6.7.1 CROW-cache
      2. 6.7.2 CROW-ref
      3. 6.7.3 Combining CROW-cache and CROW-ref
    8. 6.8 Related Work
    9. 6.9 Summary
  10. 7 Conclusions and Future Directions
    1. 7.1 Future Research Directions
      1. 7.1.1 Deeper DRAM Characterization
      2. 7.1.2 Extending SoftMC to Support Other DRAM Standards and Memory Technologies
      3. 7.1.3 Improving RowHammer Attacks and Defenses
      4. 7.1.4 New DRAM Maintenance Mechanisms
      5. 7.1.5 Enabling in-DRAM Data Movement
    2. 7.2 Final Concluding Remarks
  11. A Other Works of the Author

######  ?listfigurename?

  1. 2.1 Typical DRAM system organization.
  2. 2.2 Commands, timing parameters, and cell/bitline voltages during a DRAM read operation.
  3. 2.3 Typical Single-sided (SS) and Double-sided (DS) RowHammer access patterns.
  4. 3.1 Our SoftMC infrastructure (2017).
  5. 3.2 Key SoftMC instruction types.
  6. 3.3 SoftMC hardware architecture.
  7. 3.4 Number of erroneous bytes observed in retention time tests.
  8. 3.5 Timelines that illustrate the methodology for testing the improvement of (a) tRCD and (b) tRAS on highly-charged DRAM cells.
  9. 3.6 Effect of reducing tRCD on the number of errors at various refresh intervals.
  10. 3.7 Effect of reducing tRAS on the number of errors at various refresh intervals.
  11. 4.1 Overview of U-TRR.
  12. 4.2 General approach for detecting a TRR-induced refresh using _TRR-A_.
  13. 4.3 DDR4 SO-DIMM SoftMC [1] experimental setup.
  14. 4.4 Detailed _RS_ operation.
  15. 4.5 Detailed _TRR-A_ experiment.
  16. 4.6 Distribution of bit flips per DRAM row for different aggressor hammer counts in three representative modules.
  17. 4.7 Percentage of rows that experience at least one RowHammer bit flip using our custom RowHammer access patterns.
  18. 4.8 Distribution of 8-byte data chunks (log scale) with different RowHammer bit flip counts in a single DRAM bank from each of the 45 tested DDR4 modules.
  19. 5.1 A DRAM channel with four chips.
  20. 5.2  bank organization in DRAM chip.
  21. 5.3 Handling in MC.
  22. 5.4 Fixed-Rate Refresh () Operation.
  23. 5.5 Variable Refresh () Operation.
  24. 5.6 Probabilistic RowHammer Protection () Operation.
  25. 5.7  Operation.
  26. 5.8  Operation.
  27. 5.9 Four-core weighted speedup
  28. 5.10 DRAM energy consumption
  29. 5.11 Sensitivity to Refresh Period
  30. 5.12 Sensitivity to Lock Region Size.
  31. 5.13 Comparison of different policies for handling refresh divergence across DRAM chips.
  32. 5.14 DDR4 Scrubbing vs. .
  33. 5.15 PARA vs. .
  34. 5.16 ’s sensitivity to $ACT_{max}$.
  35. 5.17 Sensitivity to the number of neighbor rows affected by RowHammer.
  36. 6.1 Regular and copy rows in a subarray in CROW.
  37. 6.2 Organization of the CROW table.
  38. 6.3 Change in various DRAM latencies with different number of simultaneously-activated rows, normalized to baseline DRAM timing parameters (absolute baseline values in parentheses).
  39. (a) tRCD ($18\text{\,}\mathrm{ns}$)
  40. (b) tRAS ($42\text{\,}\mathrm{ns}$), tWR ($18\text{\,}\mathrm{ns}$), and restoration ($24\text{\,}\mathrm{ns}$)
  41. 6.4 Normalized tRCD latency as a function of normalized tRAS latency for different number of simultaneously activated DRAM rows.
  42. 6.5 Power consumption and area overhead of MRA.
  43. 6.6 CROW-cache speedup (160 four-core workloads).
  44. 6.7 DRAM energy consumption with CROW-cache.
  45. 6.8 CROW-cache vs. TL-DRAM [2] & SALP [3].
  46. (a) Energy vs. speedup
  47. (b) Chip area vs. speedup
  48. 6.9 CROW-cache and prefetching.
  49. 6.10 CROW-ref speedup and DRAM energy.

######  ?listtablename?

  1. 5.1 Simulated system configuration.
  2. 6.1 Timing parameters for new DRAM commands.
  3. 6.2 Simulated system configuration.

##  ?chaptername? 1 Introduction

###  1.1 Problem Discussion

Dynamic Random Access Memory (DRAM) [4] has long been the dominant memory
technology used in almost all computing systems due to its low latency and low
cost per bit. DRAM vendors scale the DRAM technology by shrinking DRAM cells
to continuously reduce the cost of DRAM [5]. Unfortunately, while the density
of the DRAM chips has been increasing as a result of technology scaling, high-
density DRAM chips face three critical challenges [6, 5]: (1) high access
latencies, (2) high refresh overheads, and (3) increasing reliability
problems. These three DRAM scaling challenges have a major impact on the
performance, energy, and robustness (i.e., reliability, safety, and security)
of a system. We examine each of these critical challenges in turn.

Challenge 1: High Access Latency. The high DRAM access latency is a challenge
to improving system performance and energy efficiency. While DRAM capacity
increased significantly over the last two decades [7, 8, 9, 10, 2, 11, 6, 5,
12, 13, 14, 15], DRAM access latency decreased only slightly [2, 6, 10, 5, 12,
13, 14, 11, 15, 16, 17]. The high DRAM access latency significantly degrades
the performance of many workloads [18, 19, 20, 21, 22, 11, 2, 23, 24, 25, 26,
27, 28, 29, 30, 9, 31, 32, 33, 34, 35, 36]. The performance impact is
particularly large for applications that 1) have working sets exceeding the
cache capacity of the system, 2) suffer from high instruction and data cache
miss rates, and 3) have low memory-level parallelism. While some DRAM vendors
offer latency-optimized DRAM modules [37, 38], these modules have
significantly lower capacity and higher cost compared to commodity DRAM [2,
39, 3]. Thus, reducing the high DRAM access latency _without significantly
affecting capacity and cost_ in commodity DRAM remains an important challenge
[23, 2, 5, 6].

Challenge 2: High Refresh Overhead. The high DRAM refresh overhead is a
challenge to improving system performance and energy consumption. A DRAM cell
stores data in a capacitor that leaks charge over time. To maintain
correctness, _every_ DRAM cell requires periodic _refresh_ operations that
restore the charge level in a cell. As the DRAM cell size decreases with
process technology scaling, newer DRAM devices contain both more DRAM cells
and smaller DRAM cells than older DRAM devices [40]. As a result, the
performance and energy overheads of refresh operations scale unfavorably as
DRAM technology scales into the future [41, 42, 43]. In modern DDR5 [44]
devices, the memory controller refreshes _every_ DRAM cell every
$32\text{\,}\mathrm{ms}$ at the nominal temperature range and every
$16\text{\,}\mathrm{ms}$ at the extended temperature range. Previous studies
show that 1) refresh operations incur large performance overheads, as DRAM
cells _cannot_ be accessed when the cells are being refreshed [41, 43, 45, 46,
47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60]; and 2) a large
fraction (e.g., 50% in $64\text{\,}\mathrm{Gbit}$ DRAM chips) of the total
DRAM energy is consumed by the refresh operations [43, 41, 49, 47, 57, 56, 59,
60].

Challenge 3: Increasing Reliability Problems. The increasing vulnerability of
DRAM cells to various failure mechanisms is an important challenge to
maintaining DRAM reliability with a small system performance and energy
overhead. As the process technology node size reduces, DRAM cells get smaller
and closer to each other, and thus they become more susceptible to failures
[61, 5, 62, 63, 64, 43, 65, 66, 67, 68, 69, 6, 70, 71, 72, 73, 74, 75]. A real
and prevalent example of such a failure mechanism in modern DRAM is the
RowHammer vulnerability [73, 72, 74, 76]. RowHammer causes _disturbance
errors_ (i.e., bit flips in vulnerable DRAM cells that are not being accessed)
in DRAM rows physically adjacent to a row that is repeatedly activated many
times. In addition to being detrimental to DRAM reliability, RowHammer also
poses a threat to system security and safety. Prior works [70, 77, 78, 73, 79,
80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97]
propose various mechanisms that protect systems against RowHammer at the cost
of increased performance and energy overheads. DRAM also exhibits other
reliability issues (e.g., data retention failures, soft errors) that
necessitate expensive solutions (e.g., increasing refresh rate, memory
scrubbing), which lead to high performance and energy overheads.

Our Goal. To combat the system-level implications of the technology scaling
challenges of DRAM, it is essential to rigorously understand DRAM cell
behavior and the architecture of existing DRAM chips. The goal of this
dissertation is to 1) develop techniques and infrastructures to enable such
rigorous characterization, analysis, and understanding, and 2) enable new
mechanisms to improve DRAM performance, reliability, and security based on the
developed understanding.

To this end, in this dissertation, we first design, implement, and prototype
SoftMC, a practical-to-use and flexible FPGA-based DRAM characterization
infrastructure. SoftMC enables detailed analyses on DRAM cell characteristics
of real DRAM chips via precise control on the low-level DRAM interface.
Second, we use SoftMC to uncover the operation and weaknesses of RowHammer
protection mechanisms implemented in modern DRAM chips. We develop a
methodology (called U-TRR) that uses DRAM retention failures as a side-channel
to monitor when a RowHammer protection mechanism performs additional refresh
operation to protect DRAM rows susceptible to a RowHammer attack. Based on the
understanding we develop, we craft new RowHammer access patterns that
efficiently circumvent in-DRAM RowHammer protection mechanisms implemented in
modern DDR4 DRAM chips from three major DRAM vendors. Third, we propose Self-
Managing DRAM (SMD), a new DRAM architecture and interface to enable in-DRAM
maintenance operations to efficiently perform 1) DRAM refresh, 2) RowHammer
protection, and 3) memory scrubbing. Our proposal eases the adoption of new
DRAM maintenance mechanisms by setting the memory controller free from
managing such DRAM maintenance operations. Finally, we propose the Copy-Row
(CROW) DRAM substrate that exploits the fast and efficient data movement
within a DRAM subarray to enable new mechanisms for improving DRAM
performance, energy consumption, and reliability.

###  1.2 Thesis Statement

Our thesis statement is as follows:

_Understanding DRAM characteristics and operation through rigorous
experimentation
using real DRAM chips pave the way for developing new mechanisms that
significantly
improve DRAM performance, energy consumption, reliability, and security._

###  1.3 Our Approach

We first enable detailed characterization of modern DRAM chips by designing
and prototyping a flexible FPGA-based infrastructure, which we then use to
develop understanding and mechanisms for improving system performance,
efficiency, reliability, and security. In the remainder of this section, we
briefly introduce our DRAM characterization infrastructure, new
characterization studies, and new mechanisms.

####  1.3.1 Infrastructure for Characterizing DRAM Chips

To easily study the reliability characteristics and the operation of real DRAM
chips, we design and prototype _SoftMC (Soft Memory Controller)_ , a flexible
and easy-to-use experimental DRAM testing infrastructure. SoftMC is an FPGA-
based DRAM testing infrastructure that exposes the low-level DRAM command
interface to its users via a high-level programming interface. The high-level
programming interface completely hides the low-level details of the FPGA from
users. Users implement their DRAM experiment routines or mechanisms in a high-
level language that automatically gets translated into the low-level SoftMC
operations in the FPGA. SoftMC offers a wide range of use cases, such as
characterizing the effects of variation within a DRAM chip and across DRAM
chips, verifying the correctness of new DRAM mechanisms on actual hardware,
and experimentally discovering the reliability, retention, and timing
characteristics of an unknown or newly-designed DRAM chip (or finding the best
specifications for a known DRAM chip). SoftMC was published at HPCA 2017 [1]
and released as an open source tool [98] at the same time. Since then, many
works [29, 10, 99, 100, 71, 101, 70, 102, 103, 69, 104, 75, 105, 106, 107,
108, 109, 110, 111, 112, 113, 114] used different versions of SoftMC in
various DRAM characterization studies as well as studies that introduced new
ideas.

####  1.3.2 Understanding in-DRAM RowHammer Protection Mechanisms

We use SoftMC to study the security guarantees of RowHammer protection
mechanisms that DRAM vendors implement in their chips. Different vendors
implement different RowHammer protection mechanisms, commonly referred to as
_Target Row Refresh (TRR)_. The main idea of TRR is to detect an aggressor row
(i.e., a row that is frequently activated within a time frame) and refresh its
victim rows (i.e., neighboring rows that are physically adjacent to the
aggressor row). However, the exact operation of TRR is unknown. Some of the
major DRAM vendors advertise _RowHammer-free_ DDR4 DRAM chips [115, 116, 100].
However, none of the DRAM vendors have so far disclosed the implementation
details let alone proved the protection guarantees of their TRR mechanisms.
Thus, we need new methods for identifying whether or not DRAM chips are fully
secure against RowHammer. We develop U-TRR, a practical methodology that
uncovers the inner workings of TRR mechanisms in modern DRAM chips. The goal
of U-TRR is to enable the observation (i.e., uncovering) of all protective
refreshes generated by TRR after inducing a carefully crafted sequence of DRAM
accesses. To make this possible, we make a key observation that retention
failures that occur on a DRAM row can be used as a side channel to detect
_when_ the row is refreshed, due to either TRR or periodic refresh operations.
U-TRR uncovers important details of the TRR designs of three major DRAM
vendors. We demonstrate the usefulness of these insights for developing
effective RowHammer attacks on DRAM chips from each vendor by crafting
specialized DRAM access patterns that hammer a row enough times to cause a
RowHammer bit flip _without_ alerting the TRR protection mechanism (i.e., by
redirecting potential TRR refreshes _away from_ the victim rows). In our
evaluation, we find that all tested DRAM modules with different manufacturing
dates (from 2016 to 2020) are vulnerable to the new access patterns we can
craft via U-TRR. We open source our U-TRR methodology and make it freely
available for further research and development [117].

####  1.3.3 Enabling Easy Adoption of Efficient in-DRAM Maintenance Operations

We develop a new DRAM architecture and interface, Self-Managing DRAM, to
enable autonomous in-DRAM maintenance operations. For reliable and secure
operation of modern and future DRAM chips, _DRAM maintenance operations_ ,
such as periodic refresh, RowHammer protection, and memory scrubbing are
critical. An ideal maintenance operation should efficiently improve DRAM
reliability and security with minimal system performance and energy
consumption overheads. As DRAM technology node scales to smaller sizes, the
reliability and security of DRAM chips worsen. As a result, new DRAM chip
generations necessitate making existing maintenance operations more intensive
(e.g., increasing the refresh rate [44, 118, 80] and memory scrubbing
frequency [49]) and introducing new types of maintenance operations (e.g.,
targeted refresh [103, 100, 119] and DDR5 RFM [44] as RowHammer defenses).
Unfortunately, modifying the DRAM maintenance operations is difficult due to
the current rigid DRAM interface that places the memory controller completely
in charge of DRAM control. Implementing new or modifying existing maintenance
operations often require difficult-to-realize changes in the DRAM interface,
the memory controller, and other system components (e.g., system software).
Self-Managing DRAM 1) eases the process and thus speed of implementing new
DRAM maintenance operations and 2) enables more efficient in-DRAM maintenance
operations. Using Self-Managing DRAM, we develop maintenance operations for
refresh, RowHammer protection, and memory scrubbing that incur significantly
lower system performance and energy overhead compared to existing state-of-
the-art approaches. We believe that enabling autonomous in-DRAM maintenance
operations would encourage innovation, reduce time-to-market, and ease
adoption of novel mechanisms that improve DRAM efficiency, reliability, and
security. To foster research and development in this direction, we open source
our Self-Managing DRAM framework [120] and early-release our Self-Managing
DRAM work on arXiv.org [121].

####  1.3.4 Improving DRAM Access Latency and Reducing DRAM Refresh Overhead
via Simultaneous Row Activation

We propose Copy-Row DRAM (CROW), a flexible in-DRAM substrate that can be used
in multiple different ways to address the performance, energy efficiency, and
reliability challenges of DRAM. The key idea of CROW is to provide a fast,
low-cost mechanism to duplicate select rows in DRAM that contain data that is
most sensitive or vulnerable to latency, refresh, or reliability issues. At a
high level, CROW partitions each DRAM subarray into two regions (_regular_
rows and _copy_ rows) and enables independent control over the rows in each
region. Using CROW, we develop two novel mechanisms: 1) _CROW-cache_ , which
reduces the DRAM access latency and 2) _CROW-ref_ , which reduces the DRAM
refresh overhead. The key idea of _CROW-cache_ is to 1) duplicate data from
recently-accessed regular rows into a small cache made up of _copy rows_ , and
2) simultaneously activate a duplicated regular row with its corresponding
_copy row_ when the regular row needs to be activated again. By activating
both the regular row and its corresponding _copy row_ , _CROW-cache_ reduces
the time needed to open the row and begin performing read and/or write
requests to the row by 38%. The key idea of _CROW-ref_ is to avoid storing any
data in DRAM rows that contain weak cells, such that the entire DRAM chip can
use a longer refresh interval. _CROW-ref_ uses an efficient profiling
mechanism [122] to identify weak DRAM cells, remaps the regular rows
containing weak cells to strong _copy rows_ , and records the remapping
information. Our evaluations show that _CROW-cache_ and _CROW-ref_
significantly improve system performance and energy efficiency compared to
conventional DRAM and incur small hardware area overhead. We also describe how
CROW can be used to mitigate the RowHammer vulnerability. To aid future
research and development, we open source CROW [123].

###  1.4 Contributions

This dissertation makes the following major contributions:

  1. 1.

We introduce SoftMC, the first open-source FPGA-based experimental memory
testing infrastructure. SoftMC implements all low-level DRAM operations in a
programmable memory controller that is exposed to the user with a flexible and
easy-to-use interface, and hence enables the efficient characterization of
modern DRAM chips and evaluation of mechanisms built on top of low-level DRAM
operations. To our knowledge, SoftMC is the first publicly-available
infrastructure that exposes a high-level programming interface to ease memory
testing and characterization.

    1. (a)

We provide a prototype implementation of SoftMC with a high-level software
interface for users and a low-level FPGA-based implementation of the memory
controller. We have released the software interface and the implementation
publicly as a freely-available open-source tool [98].

    2. (b)

We demonstrate the capability, flexibility, and programming ease of SoftMC by
implementing two example use cases. Our first use case demonstrates the ease
of use of SoftMC by implementing a routine for retention time characterization
of DRAM cells. Our second use case demonstrates the effectiveness of SoftMC as
a new tool to test existing or new mechanisms on existing memory chips. Using
SoftMC, we demonstrate that the expected effect (i.e., highly-charged DRAM
rows can be accessed faster than others) of two recently-proposed mechanisms
is not observable in 24 modern DRAM chips from three major vendors.

  2. 2.

Using SoftMC, we extensively analyze 45 modern DDR4 modules from three major
DRAM vendors to understand the operation and security guarantess of their in-
DRAM RowHammer protection mechanisms called Target Row Refresh (TRR). We
develop U-TRR, a new methodology for reverse-engineering TRR mechanisms and
assessing their security properties.

    1. (a)

We use U-TRR to understand and uncover the TRR implementations of 45 DDR4
modules from the three major DRAM vendors. This evaluation shows that our new
methodology is broadly applicable to any DRAM chip.

    2. (b)

Leveraging the TRR implementation details uncovered by U-TRR, we craft
specialized RowHammer access patterns that make existing TRR protections
ineffective.

    3. (c)

Our specialized U-TRR-discovered access patterns are significantly more
effective than patterns from the state-of-the-art [100]: we show that our new
RowHammer access patterns cause 1) bit flips in all 45 DDR4 modules we
comprehensively examine, 2) bit flips in up to 99.9% of the all rows in a DRAM
bank, and 3) two and more (up to $7$) bit flips in a single 8-byte dataword,
enabling practical RowHammer attacks in systems that employ ECC.

    4. (d)

We release the source code of our U-TRR methodology to aid future research and
development [117].

  3. 3.

We develop Self-Managing DRAM (SMD), a new DRAM chip design and interface to
enable autonomous and efficient in-DRAM maintenance operations. We implement
SMD with small changes to modern DRAM chips and memory controllers. The core
components of SMD incur area overhead of only 1.63% of a
$45.5\text{\,}{\mathrm{mm}}^{2}$ DRAM chip and increase DRAM row activation
latency by only 0.4%.

    1. (a)

We use SMD to implement efficient DRAM maintenance mechanisms for three use
cases: periodic refresh, RowHammer protection, and memory scrubbing.

    2. (b)

We rigorously evaluate the performance and energy of the new maintenance
mechanisms. SMD provides large performance and energy benefits while also
improving reliability and security across a variety of systems and workloads.

    3. (c)

We open source our Self-Managing DRAM framework to aid future research and
development in this direction [120].

  4. 4.

We introduce Copy-Row DRAM (CROW), a flexible and low-cost substrate in
commodity DRAM that enables mechanisms for improving DRAM performance, energy
efficiency, and reliability by providing two sets of rows that have
independent control in each subarray. CROW does not change the extremely-dense
cell array and has _low_ cost (0.48% additional area overhead in the DRAM
chip, $11.3\text{\,}\mathrm{KiB}$ storage overhead in the memory controller,
and 1.6% DRAM storage capacity overhead).

    1. (a)

We propose three mechanisms that exploit the CROW substrate: 1) CROW-cache, an
in-DRAM cache to reduce the DRAM access latency, 2) CROW-ref, a remapping
scheme for weak DRAM rows to reduce the DRAM refresh overhead, and 3) a
mechanism for mitigating the RowHammer vulnerability. We show that CROW allows
these mechanisms to be employed at the same time.

    2. (b)

We evaluate the performance, energy savings, and overhead of CROW-cache and
CROW-ref, showing significant performance and energy benefits over a state-of-
the-art commodity DRAM chip. We also compare CROW-cache to two prior proposals
to reduce DRAM latency [3, 2] and show that CROW-cache is more area- and
energy-efficient at reducing DRAM latency.

    3. (c)

We make the source code of CROW publicly available to aid future research and
development [123].

###  1.5 Outline

This dissertation consists of 8 chapters. chapter 2 provides background on
DRAM organization, operation, and problems, which are essential for
understanding our proposals in later chapters. In chapter 3, we describe
SoftMC, our new FPGA-based DRAM characterization infrastructure. chapter 4
presents our U-TRR methodology for uncovering the operation of in-DRAM TRR
mechanisms using SoftMC. chapter 5 introduces Self-Managing DRAM (SMD), a new
DRAM architecture and interface for enabling efficient and autonomous in-DRAM
maintenance operations. chapter 6 presents Copy-Row DRAM (CROW), a substrate
that makes slight modifications in DRAM subarray architecture to enable
efficient mechanisms for improving DRAM performance, energy efficiency, and
reliability. Finally, chapter 7 summarizes the dissertation and provides
future research directions that this dissertation enables.

##  ?chaptername? 2 Background

In this chapter, we provide the background on DRAM necessary for understanding
our methods, observations, and techniques presented in later chapters. We
first describe DRAM organization and operation. Afterward, we discuss the
major performance, reliability, and security problems of DRAM that we tackle.
We refer the reader to many prior works in the area [10, 41, 39, 99, 124, 17,
73, 3, 11, 2, 125, 126, 127, 43, 65, 128, 129, 130, 131, 132, 70, 100, 15, 78,
133, 134, 135, 52, 136, 102, 137, 105, 138, 75, 49, 71, 106, 103, 58, 5, 6,
72, 74, 139, 140, 141, 69, 68, 67, 66, 142, 143, 144, 145, 146, 147, 148, 149,
150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164,
165, 166, 167, 168] for various details on DRAM background.

###  2.1 DRAM Organization

Fig. 2.1 shows the typical organization of a modern DRAM system. DRAM is
organized into hierarchical arrays consisting of billions of DRAM cells in
total, where each cell stores one bit of data. In modern systems, a CPU chip
implements a set of memory controllers, where each memory controller
interfaces with a DRAM _channel_ to perform read, write, and maintenance
operations (e.g., refresh, RowHammer protection, memory scrubbing) via a
dedicated I/O bus that is independent of other channels in the system. A DRAM
channel can host one or more _DRAM modules_ , where each module consists of
one or more _DRAM ranks._ A rank is comprised of multiple _DRAM chips_ that
operate in lock step and ranks in the same channel time-share the channel’s
I/O bus.

?figurename? 2.1: Typical DRAM system organization.

A DRAM chip consists of multiple DRAM _banks_ , which share an internal bus
that connects them to the chip’s I/O circuitry. Within a DRAM bank, DRAM cells
are organized into multiple (e.g., 128) dense two-dimensional arrays of DRAM
cells called _subarrays_ [3, 41, 128] and corresponding peripheral circuitry
for manipulating the data within the subarray. A row of cells (i.e., DRAM
_row_) within a subarray share a wire (i.e., _wordline_), which is driven by a
_row decoder_ to _open_ (i.e., select) the row of cells to be read or written.
A column of cells (i.e., DRAM _column_) within a subarray share a wire (i.e.,
_bitline_), which is used to read and write to the cells with the help of a
_row buffer_ (consisting of _sense amplifiers_). This hierarchical layout of
DRAM cells enables any data in the DRAM system to be accessed and updated
using unique channel, rank, bank, row, and column addresses.

###  2.2 DRAM Operation

The memory controller interfaces with DRAM using a series of commands sent
over the I/O bus. There are four major commands that are used to access DRAM:
ACT, WR, RD, and PRE. DRAM command scheduling [143, 169, 170, 171, 172, 173,
174] is tightly regulated by a set of _timing parameters_ , which guarantee
that enough time passes after a certain command such that DRAM provides or
retains data correctly. DRAM commands and timing parameters are defined by
DRAM standards [8, 44, 44, 175] and they constitute part of the interface
between the memory controller and the DRAM chip.

Figure 2.2, which we explain in the next several subsections, illustrates the
relationship between the commands issued to perform a DRAM read, their
governing timing parameters, and their effect on cell and bitline voltages.
The memory controller enforces relevant timing parameters as it schedules each
DRAM command. Aside from the DRAM access commands, the memory controller also
periodically issues a refresh (REF) command to prevent data loss due to
leakage of charge from the cell capacitors over time.

?figurename? 2.2: Commands, timing parameters, and cell/bitline voltages
during a DRAM read operation.

####  2.2.1 Activate (ACT)

The ACT command _activates_ (opens) a DRAM row by transferring the data
contained in the cell capacitors to the row buffer. ACT latency is governed by
the tRCD timing parameter, which ensures that enough time has passed since the
ACT is issued for the data to stabilize in the row buffer (such that it can be
read by issuing a RD command).

ACT consists of two major steps: 1) capacitor-bitline _charge sharing_ and 2)
_charge restoration_. Charge sharing begins by enabling the wordline ( 1 in
Figure 2.2), which allows the cell capacitor to share charge with the bitline,
and thus perturb the precharged bitline voltage. Once the cell and bitline
voltages equalize due to charge sharing, charge restoration starts ( 2).
During charge restoration, the sense amplifiers are enabled to first detect
the bitline voltage shift, and later restore the bitline to a full $V_{ss}$ or
$V_{dd}$ depending on the direction of the shift. Once the bitline is restored
to a _ready-to-access_ voltage level ( 3), the other DRAM commands (e.g., RD,
WR) can be issued to the bank.

####  2.2.2 Read (RD)

After a row activation, the memory controller reads data from the open row by
issuing a RD command. The RD command includes a column address, which
indicates the portion of the open row to be read. When a DRAM chip receives a
RD command, it first loads the requested portion of the open row into the
global row buffer. After the data is in the global row buffer, the DRAM chip
sends the data across the data bus to the memory controller. The RD command is
governed by the timing parameter tCL, after which the data appears on the data
bus.

####  2.2.3 Write (WR)

The WR command (not shown in Figure 2.2) modifies data in an open DRAM row.
The operation of WR is analogous to ACT in that both commands require waiting
enough time for the sense amplifiers to restore the data in the DRAM cells.
Similar to how a sense amplifier restores a cell capacitor during the second
step of ACT (i.e., charge restoration), in case of a WR, the sense amplifier
restores the capacitor with the new data value that the WR command provides.
The restoration latency for WR is governed by the tWR timing parameter. For
both ACT and WR commands, the restoration latency originates from the sense
amplifier driving a bitline to replenish the charge of the DRAM cell capacitor
[3, 42, 11, 176].

####  2.2.4 Precharge (PRE)

PRE is used to _close_ an open DRAM row and prepare the DRAM bank for
activation of another row. The memory controller can follow an ACT with PRE to
the same bank after at least the time interval specified by the tRAS timing
parameter. tRAS ensures that enough time has passed to fully restore the DRAM
cells of the activated row to a _ready-to-precharge voltage_ ( 4 in Figure
2.2). The latency of PRE is governed by the tRP timing parameter, which allows
enough time to set the bitline voltage back to the reference voltage level
(e.g., $V_{dd}/2$). After tRP ( 5 in Figure 2.2), the memory controller can
issue an ACT to open a new row in the same bank.

####  2.2.5 Refresh (REF)

A DRAM cell cannot store its data permanently, as the cell capacitor leaks
charge over time. The retention time of a DRAM cell is defined as the length
of time for which the data can still be correctly read out of the cell after
data is stored in the cell. To ensure data integrity, a DRAM cell must be
periodically refreshed. To enable such periodic refresh of all DRAM cells, the
memory controller periodically issues a _refresh_ (REF) command (e.g., every
$7.8\text{\,}\mathrm{\SIUnitSymbolMicro s}$ or
$3.9\text{\,}\mathrm{\SIUnitSymbolMicro s}$ in chips implementing the DDR4
standard [8]) to ensure that every DRAM cell is refreshed once at a fixed
_refresh window_ (i.e., typically 32 or $64\text{\,}\mathrm{ms}$ in chips
implementing the DDR4 standard [8]) [177, 7, 65, 43, 175, 8, 118, 44]. A DRAM
chip refreshes several (e.g., $16$) rows upon receiving a single REF command,
which takes tRFC (e.g., $\approx$350\text{\,}\mathrm{ns}$$) to complete.
Typically $8192$ REF commands are required to refresh the entire DDR4 DRAM
chip in a refresh window. The new DDR5 standard [44] similarly completes
refreshing the entire DRAM chip with $8192$ REF commands but requires
$32\text{\,}\mathrm{ms}$ or $16\text{\,}\mathrm{ms}$ refresh window, i.e., a
REF is issued every $3.9\text{\,}\mathrm{\SIUnitSymbolMicro s}$ when operating
at up to $85\text{\,}\mathrm{\SIUnitSymbolCelsius}$ and
$1.95\text{\,}\mathrm{\SIUnitSymbolMicro s}$ when operating between
$85\text{\,}\mathrm{\SIUnitSymbolCelsius}$-$95\text{\,}\mathrm{\SIUnitSymbolCelsius}$.

###  2.3 DRAM Performance Issues

In this section, we discuss the major performance issues of DRAM that we
tackle in this dissertation.

####  2.3.1 DRAM Access Latency

In contrast to tremendous improvement in DRAM chip density, DRAM access
latency has reduced _only_ slightly [2, 11, 17, 39, 16, 178, 10, 179, 180, 12,
181, 9, 13]. Over the last two decades, DRAM density increased by more than
two orders of magnitude, whereas tRCD and tRAS latencies decreased by _only_
0.81% and 1.33% per year between 2000 and 2015 [13].

Even though technology node scaling would normally reduce latencies in the
DRAM circuit in conjunction with enabling higher DRAM capacity, DRAM vendors
typically design their DRAM chips to trade the latency reduction for further
improvements in DRAM capacity. For example, vendors amortize the large area
cost of a sense amplifier by increasing the number of DRAM cells connected to
the same sense amplifier, which increases the bitline length [17, 2].
Increasing the bitline length leads to higher resistance and parasitic
capacitance on the connections between a DRAM cell and a sense amplifier.
Consequently, a long bitline results in higher access latency than a short
bitline.

To reduce the long DRAM latencies, many prior works propose various techniques
that exploit temporal charge variation in DRAM cells [17, 182, 183, 136],
exploit design-induced latency variation [126, 184, 159], use in-DRAM caching
[39, 124, 3, 185, 186, 133], make static [2, 37, 9, 187] and dynamic [15, 179]
capacity-latency tradeoff, and reduce timing parameters under certain
operating conditions [10, 99, 11].

####  2.3.2 DRAM Refresh Overhead

Periodic DRAM refresh operations incur high system performance (and energy
consumption) overheads. As the DRAM cell size decreases with process
technology scaling, newer DRAM devices contain more DRAM cells than older DRAM
devices [40]. As a result, while DRAM capacity increases, the performance and
energy overheads of the refresh operations scale unfavorably [41, 42, 43]. In
modern LPDDR4 [188] devices, the memory controller refreshes _every_ DRAM cell
every $32\text{\,}\mathrm{ms}$ (when temperature is above
$85\text{\,}\mathrm{\SIUnitSymbolCelsius}$, every $16\text{\,}\mathrm{ms}$).
Previous studies show that 1) refresh operations incur large performance
overheads, as DRAM cells _cannot_ be accessed when the cells are being
refreshed [41, 43, 45, 46]; and 2) up to 50% of the total DRAM energy is
consumed by the refresh operations [43, 41].

To mitigate DRAM refresh overhead, prior works propose various techniques that
exploit retention time variation [43, 57, 50, 189, 162, 190, 45, 122, 49,
191], simultaneously activate multiple DRAM rows [192, 124, 179], parallelize
refreshes with accesses [41, 113], optimize scheduling of refresh commands
[51, 53, 54, 55, 46, 58, 59], and exploit data pattern variation [66, 67, 68,
69, 193].

###  2.4 DRAM Reliability Issues

In this section, we discuss the major reliability issues of DRAM that we
tackle in this dissertation.

####  2.4.1 Charge Leakage and Data Retention

A DRAM cell stores a data value in the form of charge in its capacitor (e.g.,
a charged cell can represent 0 or 1 and vice versa). Since the capacitor
naturally loses charge over time, the capacitor charge must be actively and
periodically _refreshed_ to prevent information loss due to a data retention
failure [122, 177, 7, 65, 66, 69, 68, 43, 67, 49, 194, 11, 159, 178, 10, 99,
1, 163, 160, 162, 195, 73, 12, 126, 196, 197, 198, 199, 166, 200, 141]. As we
explain in section 2.3.2, DRAM refresh incurs significant system performance
and energy consumption overheads.

Although all DRAM cell in a DRAM chip are refreshed at a fixed refresh
interval, the retention times of different cells can significantly vary [122,
43, 49, 69, 67, 68, 66, 65, 194, 11]. Many prior works [65, 49, 69, 67, 68,
66, 194, 11, 122, 159, 178, 10, 99, 1, 163, 160, 162, 195, 73, 12, 126, 196,
197, 198, 199, 166, 200, 141] perform detailed experimental studies to analyze
data retention and reliability characteristics of real DRAM chips. Other prior
works [201, 43, 156, 202, 191, 203] propose techniques to mitigate DRAM
retention failures.

####  2.4.2 Violating Access and Refresh Timings

DRAM vendors specify timing parameters that a memory controller must satisfy
when accessing and refreshing their DRAM chip in order to ensure reliable
operation. However, deliberately violating DRAM timing parameters is useful
for several cases. First, as we explain in section 2.3.1, violating DRAM
timing parameters can improve DRAM latency and energy efficiency at the cost
of increased bit error rate. Second, prior works show that deliberately
violating DRAM timing parameters provides new functions like Physical
Unclonable Functions (PUFs)[137, 204, 108, 109, 205], True Random Number
Generators (TRNGs)[102, 139, 105, 206, 107, 207], and computation using DRAM
[101, 129, 128, 106, 208]. Third, DRAM errors caused by violating DRAM timing
parameters are useful for understanding various DRAM properties and behavior
under different operating conditions [65, 49, 69, 67, 68, 66, 194, 11, 122,
159, 178, 10, 99, 1, 163, 160, 162, 195, 73, 12, 126, 196, 197, 198, 199, 166,
200, 141] and reverse-engineering DRAM circuit design [13, 141, 135, 209, 10,
103, 163, 184, 137, 165, 11, 210, 71].

####  2.4.3 Variable Retention Time (VRT)

Variable Retention Time (VRT) [211, 49, 63, 212, 213, 214, 215, 216, 217, 218,
66, 42, 65] is a phenomenon where certain DRAM cells switch between low and
high retention time states in an unpredictable manner. When in low retention
time state, a DRAM cell becomes more likely to experience a retention error.
Therefore, VRT effects make accurately profiling the retention time of DRAM
cells difficult. With DRAM technology scaling, VRT grows into an even more
critical problem [219, 42, 220]. Despite the efforts to identify and mitigate
VRT-related DRAM errors [49, 221, 122, 66, 67, 68, 69, 11, 102], VRT still
remains as a key DRAM scaling challenge [42, 65].

####  2.4.4 Environmental Factors

Various environmental factors degrade the reliability of a DRAM chip. First,
random external events such as particle strikes [222, 223, 224, 225, 199, 197,
226] may cause transient bit flips in DRAM. Second, prior experimental studies
[65, 227, 228, 229, 230, 122, 66, 67, 68, 69, 231] show that increase in DRAM
operation temperature significantly accelerates charge leakage, leading to
lower DRAM cell retention times. Thus, high operating temperatures, especially
temperatures above the range specified by DRAM vendors (e.g.,
$>$95\text{\,}\mathrm{\SIUnitSymbolCelsius}$$ [8, 44, 118, 177, 175]), can
severely impact the reliability of a DRAM chip. Third, a DRAM chip may suffer
from increased reliability issues when set to operate at reduced supply
voltage. Chang et al. [99] analyzes the DRAM bit errors at reduces voltage
levels and observes that such bit errors can be avoided by increasing DRAM
access timing parameters. Fourth, prior studies [197, 232, 196] report that
aging is an important factor affecting DRAM reliability and lifetime. Only
after 10-18 months of being in use, DRAM chips exhibit increased error rates
[197].

###  2.5 DRAM Security Issues

In this section, we discuss RowHammer and cold boot attacks, which are major
security issues of DRAM. We tackle RowHammer in this dissertation, but our
findings and techniques can potentially be useful for tackling cold boot
attacks as well.

####  2.5.1 RowHammer

Modern DRAM chips suffer from disturbance errors that occur when a high number
of activations (within a refresh window) to one DRAM row unintentionally
affects the values of cells in nearby rows [73]. This phenomenon, popularly
called _RowHammer_ [73, 74, 72, 76], stems from electromagnetic interference
between circuit elements. RowHammer becomes exacerbated as manufacturing
process technology node size (and hence DRAM cell size) shrinks and circuit
elements are placed closer together [70, 72]. As demonstrated in prior works
[73, 70, 71, 75, 103, 134, 100, 119], the RowHammer effect is strongest
between immediately physically-adjacent rows. RowHammer bit flips are most
likely to appear in neighboring rows physically adjacent to a _hammered row_
that is activated many times (e.g., $139K$ in DDR3 [73], $10K$ in DDR4 [70],
and $4.8K$ in LPDDR4 [70])111For DDR3 chips, [73] reports the minimum number
of row activations on a _single_ aggressor row (i.e., single-sided RowHammer)
to cause a RowHammer bit flip. For DDR4 and LPDDR4 chips, [70] reports the
minimum number of row activations to _each of the two_ immediately-adjacent
aggressor rows (i.e., double-sided RowHammer).. A hammered row is also called
an _aggressor row_ and a nearby row that is affected by the hammered row is
called a _victim row_ , regardless of whether or not the victim row actually
experiences RowHammer bit flips.

To most effectively exploit the RowHammer phenomenon, attackers typically
perform $i$) single-sided RowHammer (i.e., repeatedly activate one aggressor
row that is physically adjacent to the victim row, as we show in Fig. 2.3a)
[73] or $ii$) double-sided RowHammer (i.e., repeatedly activate in an
alternating manner two aggressor rows that are both physically adjacent to the
victim row, as we show in Fig. 2.3b) [233, 234]. Prior works have shown that
double-sided RowHammer leads to more bit flips and does so more quickly than
single-sided RowHammer [73, 70, 74, 233, 234, 71, 75, 103, 134, 100, 119].

?figurename? 2.3: Typical Single-sided (SS) and Double-sided (DS) RowHammer
access patterns.

Since the discovery of RowHammer, researchers have proposed many techniques
that take advantage of the RowHammer vulnerability to compromise operating
systems [233, 83, 235, 236, 77, 237, 238, 239, 240, 241, 242, 134, 243, 244],
web browsers [245, 246, 100, 247, 248], cloud virtual machines [249, 250],
remote servers [251, 252], and deep neural networks [253, 254]. To mitigate
RowHammer, early solutions were immediate quick fixes such as increasing the
DRAM refresh rate [80, 255, 256, 257] and relying on ECC to fix RowHammer bit
flips [73, 233, 249, 235, 245], which were shown to be ineffective [77, 84,
73]. Other solutions include isolating sensitive data from DRAM rows that an
attacker can hammer [81, 82, 83], keeping track of row activation counts and
refreshing potential victim rows [84, 73, 85, 79, 86, 87, 88, 90, 91, 92, 93,
258, 94, 259, 95, 96], throttling row activations to prevent a row from being
activated more than a certain threshold [78, 73, 89], and reducing wordline
voltage [75].

To this date, DRAM vendors implement proprietary in-DRAM RowHammer solutions.
Despite the claims of the DRAM vendors that their DRAM chips are “RowHammer-
free” [115], prior works [100, 103, 119] demonstrate that certain carefully-
crafted access patterns can be used to bypass these in-DRAM protection
mechanisms. In chapter 4, we further discuss the operation and security
properties of RowHammer protection mechanisms implemented in current DRAM
devices.

####  2.5.2 Cold Boot Attacks

When a DRAM chip is powered off, the DRAM cells in the chip are expected to
lose their values as cells can no longer be refreshed. However, the majority
of DRAM cells are capable of correctly retaining their data for much longer
than the typical 64 or $32\text{\,}\mathrm{ms}$ refresh period [65, 122, 69,
68, 67, 66, 11, 231, 194, 49]. Cooling the DRAM chip further extends the
retention capability of DRAM cells because their charge leakage rates greatly
decrease as the operating temperature reduces [65, 227, 228, 229, 230, 122,
66, 67, 68, 69, 231]. Halderman et al. [231] show that a DRAM chip can
correctly retain 99.9% of its data for minutes when cooled down to
$-50\text{\,}\mathrm{\SIUnitSymbolCelsius}$ using a compressed air can. The
fact that a DRAM chip retains the majority of its data after a power loss
poses a critical security problem. An attacker can retrieve sensitive
information (e.g., encryption keys) from a DRAM chip by plugging in the chip
to a machine that the attacker owns. To mitigate DRAM cold boot attacks, prior
works propose techniques that obfuscate data written to DRAM [260, 261, 262,
263], store encryption keys only in CPU registers [264, 265], use full memory
encryption [266, 267], and erase DRAM content [106, 268].

##  ?chaptername? 3 SoftMC: A Flexible and Practical Open-Source
Infrastructure
for Enabling Experimental DRAM Studies

DRAM process technology node scaling leads to key challenges that critically
impact DRAM reliability and performance. To accurately understand the DRAM
cell behavior in terms of reliability and latency, it is critical to
characterize and analyze real DRAM chips. In this chapter, we design,
prototype, and demonstrate the basic capabilities of a flexible and easy-to-
use experimental DRAM characterization infrastructure, called SoftMC (Soft
Memory Controller). SoftMC is an open-source FPGA-based DRAM characterization
infrastructure, consisting of a programmable memory controller that can
control and test memory modules designed for the commonly-used DDR (Double
Data Rate) interface. SoftMC provides a simple and intuitive high-level
programming interface that completely hides the low-level details of the FPGA
from users. Users implement their test routines or mechanisms in a high-level
language that automatically gets translated into the low-level SoftMC memory
controller operations in the FPGA.

SoftMC offers a wide range of use cases, such as characterizing the effects of
variation within a DRAM chip and across DRAM chips, verifying the correctness
of new DRAM mechanisms on actual hardware, and experimentally discovering the
reliability, retention, and timing characteristics of an unknown or newly-
designed DRAM chip (or finding the best specifications for a known DRAM chip).
We demonstrate the potential and ease of use of SoftMC by implementing two use
cases. First, we demonstrate the ease of use of SoftMC’s high-level interface
by implementing a simple experiment to characterize the retention time
behavior of cells in modern DRAM chips. Our test results match the prior
experimental studies that characterize DRAM retention time in modern DRAM
chips [43, 65, 66, 49], providing a validation of our infrastructure. Second,
we demonstrate the flexibility and capability of SoftMC by validating two
recently-proposed DRAM latency reduction mechanisms [17, 182]. These
mechanisms exploit the idea that highly-charged DRAM cells can be accessed
with low latency. Our SoftMC-based experimental analysis of 24 real DRAM chips
from three major DRAM vendors demonstrates that the expected latency reduction
effect of these mechanisms is _not_ observable in existing DRAM chips. This
experiment demonstrates (i) the importance of experimentally characterizing
real DRAM chips to understand the behavior of DRAM cells, and designing
mechanisms that are based on this experimental understanding; and (ii) the
effectiveness of SoftMC in testing (validating or refuting) new ideas on
existing memory modules.

We also discuss several other use cases of SoftMC, including the ability to
characterize emerging non-volatile memory modules that obey the DDR standard.
We hope that SoftMC inspires other new studies, ideas, and methodologies in
memory system design. In fact, since the release of SoftMC to the public as
part of this research, many works [29, 10, 99, 100, 71, 101, 70, 102, 103, 69,
104, 75, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114] used different
versions of SoftMC in various DRAM characterization studies. SoftMC is openly
and freely available at [98].

###  3.1 Motivation and Goal

At smaller technology nodes, it is becoming increasingly difficult to store
and retain enough charge in a DRAM cell, causing various reliability and
performance issues [65, 61, 5, 62, 63, 64, 43, 66, 67, 68, 69, 6, 70, 71, 72,
73, 74, 75, 41, 49, 47, 57, 56, 59, 60]. Ensuring reliable operation of the
DRAM cells is a key challenge in future technology nodes [5, 61, 269, 270, 43,
42, 65, 62, 63, 64, 66, 67, 68, 69, 6, 70, 71, 72, 73, 74, 75].

The fundamental problem of retaining data with less charge in smaller cells
directly impacts the reliability and performance of DRAM cells. First, smaller
cells placed in close proximity make cells more susceptible to various types
of interference. This potentially disrupts DRAM operation by flipping bits in
DRAM, resulting in major reliability issues [73, 271, 62, 197, 272, 196, 5,
61, 269, 270, 43, 42, 65, 63, 64, 66, 67, 68, 69, 6, 70, 71, 72, 74, 75],
which can lead to system failure [197, 196, 77, 74, 134] or security breaches
[73, 233, 234, 236, 235, 250, 249, 83, 77, 237, 238, 239, 240, 241, 242, 134,
243, 244, 245, 246, 100, 247, 248, 251, 252, 253, 254]. Second, it takes
longer time to access a cell with less charge [17, 11, 182, 126, 10, 99, 183],
and write latency increases as the access transistor size reduces [42, 11,
183]. Thus, smaller cells directly impact DRAM latency, as DRAM access latency
is determined by the worst-case (i.e., slowest) cell in any chip [159, 11,
122, 126]. DRAM access latency has not improved with technology scaling in the
past decade [2, 16, 5, 11, 17, 39, 178, 10, 179, 180, 12, 181, 9, 13], and, in
fact, some latencies are expected to increase [42], making memory latency an
increasingly critical system performance bottleneck.

As such, there is a significant need for new mechanisms that improve the
reliability and performance of DRAM-based main memory systems. In order to
design, evaluate, and validate many such mechanisms, it is important to
accurately characterize, analyze, and understand DRAM (cell) behavior in terms
of reliability and latency. For such an understanding to be accurate, it is
critical that the characterization and analysis be based on the _experimental_
studies of real DRAM chips, since a large number of factors (e.g., various
types of cell-to-cell interference [271, 62, 73, 70, 71], inter- and intra-die
process variation [10, 12, 273, 159, 11, 126], random effects [65, 63, 274,
275, 49], operating conditions [65, 276, 99, 75, 71], internal organization
[65, 277, 68, 126], stored data patterns [67, 68, 65, 69, 66, 49, 70, 71])
concurrently impact the reliability and latency of cells. Many of these
phenomena and their interactions cannot be properly modeled (e.g., in
simulation or using analytical methods) without rigorous experimental
characterization and analysis of real DRAM chips. The need for such
experimental characterization and analysis, with the goal of building the
understanding necessary to improve the reliability and performance of future
DRAM-based main memories at various levels (both software and hardware),
motivates the need for a publicly-available DRAM testing infrastructure that
can enable system users and designers to characterize real DRAM chips.

A publicly-available DRAM testing infrastructure that can characterize real
DRAM chips enables new mechanisms to improve DRAM reliability and latency. In
this work, we argue that such a testing infrastructure should have two key
features to ensure widespread adoption among architects and designers: _(i)_
flexibility and _(ii)_ ease of use.

Flexibility. As discussed in section 2.2, a DRAM module is accessed by issuing
specific commands (e.g., ACT, PRE) in a particular sequence with a strict
delay between the commands (specified by the timing parameters, e.g., tRP,
tRAS). A DRAM testing infrastructure should implement all low-level DRAM
operations (i.e., DDR commands) with tunable timing parameters without any
restriction on the ordering of DRAM commands. Such a design enables
flexibility at two levels. First, it enables comprehensive testing of _any_
DRAM operation with the ability to customize the length of each timing
constraint. For example, we can implement a retention test with different
refresh intervals to characterize the distribution of retention time in modern
DRAM chips. Such a characterization can enable new mechanisms to reduce the
number of refresh operations in DRAM, leading to performance and power
efficiency improvements. Second, it enables testing of DRAM chips with high-
level test programs, which can consist of _any combination of DRAM operations
and timings_. Such flexibility is extremely powerful to test the impact of
existing or new DRAM mechanisms in real DRAM chips.

Ease of Use. A DRAM testing infrastructure should provide a simple and
intuitive programming interface that minimizes programming effort and time. An
interface that hides the details of the underlying implementation is
accessible to a wide range of users. With such a high-level abstraction, even
users that lack hardware design experience should be able to develop DRAM
tests.

In this work, we propose and prototype a publicly-available, open-source DRAM
testing infrastructure that can enable system users and designers to easily
characterize real DRAM chips. Our experimental DRAM testing infrastructure,
called _SoftMC (Soft Memory Controller)_ , can test DDR-based memory modules
with a flexible and easy-to-use interface. In the next section, we discuss the
shortcomings of existing tools and platforms that can be used to test DRAM
chips, and explain how SoftMC is designed to avoid these shortcomings.

###  3.2 Related Work

No prior DRAM testing infrastructure provides both _flexibility_ and _ease of
use_ properties, which are critical for enabling widespread adoption of the
infrastructure. Three different kinds of tools/infrastructure are available
today for characterizing DRAM behavior. As we will describe, each kind of tool
has some shortcomings. The goal of SoftMC is to eliminate _all_ of these
shortcomings.

Commercial Testing Infrastructures. A large number of commercial DRAM testing
platforms (e.g., [278, 279, 280, 281]) are available in the market. Such
platforms are optimized for throughput (i.e., to test as many DRAM chips as
possible in a given time period), and generally apply a _fixed test pattern_
to the units under test. Thus, since they lack support for flexibility in
defining the test routine, these infrastructures are not suitable for detailed
DRAM characterization where the goal is to investigate new issues and new
ideas. Furthermore, such testing equipment is usually quite expensive, which
makes these infrastructures an impractical option for research in academia.
Industry may also have internal DRAM development and testing tools, but, to
our knowledge, these are proprietary and are unlikely to be made openly
available.

We aim for SoftMC to be a low-cost (i.e., free) and flexible open-source
alternative to commercial testing equipment that can enable new research
directions and mechanisms. For example, prior work [282] recently proposed a
random command pattern generator to validate DRAM chips against uncommon yet
supported (according to JEDEC specifications) DDR command patterns. Using the
test patterns on commercial test equipment, this work demonstrates that
specific sequences of commands introduce failures in current DRAM chips.
SoftMC flexibly supports the ability to issue an arbitrary command sequence,
and therefore can be used as a low-cost method for validating DRAM chips
against problems that arise due to command ordering.

FPGA-Based Testing Infrastructures. Several prior works proposed FPGA-based
DRAM testing infrastructures [283, 228, 284]. Unfortunately, all of them lack
flexibility and/or a simple user interface, and none are open-source. The
FPGA-based infrastructure proposed by Huang et al. [283] provides a high-level
interface for developing DRAM tests, but the interface is limited to defining
only data patterns and march algorithms for the tests. Hou et al. [228]
propose an FPGA-based test platform whose capability is limited to analyzing
only the data retention time of the DRAM cells. Another work [284] develops a
custom memory testing board with an FPGA chip, specifically designed to test
memories at a very high data rate. However, it requires low-level knowledge to
develop FPGA programs, and even then offers only limited flexibility in
defining a test routine. On the other hand, SoftMC aims to provide _full
control over all DRAM commands_ using a high-level _software interface_ , and
it is open-source.

PARDIS [285] is a reconfigurable logic (e.g., FPGA) based programmable memory
controller meant to be implemented inside microprocessor chips. PARDIS is
capable of optimizing memory scheduling algorithms, refresh operations, etc.
at run-time based on application characteristics, and can improve system
performance and efficiency. However, it does not provide programmability for
DRAM commands and timing parameters, and therefore cannot be used for detailed
DRAM characterization.

Built-In Self Test (BIST). A BIST mechanism (e.g, [286, 287, 288, 289, 290])
is implemented inside the DRAM chip to enable fixed test patterns and
algorithms. Using such an approach, DRAM tests can be performed faster than
with other testing platforms. However, BIST has two major flexibility issues,
since the testing logic is hard-coded into the hardware: _(i)_ BIST offers
only a limited number of tests that are fixed at hardware design time. _(ii)_
A limited set of DRAM chips, which come with BIST support, can be tested. In
contrast, SoftMC allows for the implementation of a wide range of DRAM test
routines and supports any off-the-shelf DRAM chip that is compatible with the
DDR interface.

We conclude that prior work lacks either the flexibility or the ease-of-use
properties that are critical for performing detailed DRAM characterization. To
fill the gap left by current infrastructures, we introduce an open-source DRAM
testing infrastructure, SoftMC, that fulfills these two properties.

###  3.3 SoftMC Design

SoftMC is an FPGA-based open-source programmable memory controller that
provides a high-level software interface, which the users can use to initiate
any DRAM operation from a host machine. The FPGA component of SoftMC collects
the DRAM operation requests incoming from the host machine and executes them
on real DRAM chips that are attached to the FPGA board. In this section, we
explain in detail the major components of our infrastructure.

####  3.3.1 High-Level Design

Figure 3.1 shows our SoftMC infrastructure. It comprises three major
components:

  * $\bullet$

In the host machine, the _SoftMC API_ provides a high-level software interface
(in C++) for users to communicate with the SoftMC hardware. The API provides
user-level _functions_ to 1) send SoftMC _instructions_ from the host machine
to the hardware and 2) receive data, which the hardware reads from the DRAM,
back to the host machine. An _instruction_ encodes and specifies an operation
that the hardware is capable of performing (see section 3.3.4). It is used to
communicate the user-level _function_ to the SoftMC hardware such that the
hardware can execute the necessary operations to satisfy the user-level
_function_. The SoftMC API provides several functions (See section 3.3.3) that
users can call to easily generate an instruction to perform any of the
operations supported by the SoftMC infrastructure. For example, the API
contains a _genACT()_ function, which users call to generate an instruction to
activate a row in DRAM.

  * $\bullet$

The _driver_ is responsible for transferring instructions and data between the
host machine and the FPGA across a PCIe bus. To implement the driver, we use
RIFFA [291]. SoftMC execution is _not_ affected by the long transfer latency
of the PCIe interface, as our design sends _all of the instructions_ over the
PCIe bus to the FPGA _before the test routine begins,_ and buffers the
commands within the FPGA. Therefore, SoftMC guarantees that PCIe-related
delays do not affect the precise user-defined timings between each
instruction.

  * $\bullet$

Within the FPGA, the core _SoftMC hardware_ queues the instructions, and
executes them in an appropriate hardware component. For example, if the
hardware receives an instruction that indicates a DRAM command, it sends DDR-
compatible signals to the DRAM to issue the command.

?figurename? 3.1: Our SoftMC infrastructure (2017).

####  3.3.2 DRAM Command Interface

DRAM commands are transmitted from the memory controller to the DRAM module
across a memory bus. On the memory bus, each command is encoded using five
output signals (CKE, CS, RAS, CAS, and WE). Enabling/disabling these signals
corresponds to specific commands (as specified by the DDR standard). First,
the CKE signal (_clock enable_) determines whether the DRAM is in “standby
mode” (ready to be accessed) or “power-down mode”. Second, the CS (_chip
selection_) signal specifies the rank that should receive the issued command.
Third, the RAS (_row address strobe_)/CAS (_column address strobe_) signal is
used to generate commands related to DRAM row/column operations. Fourth, the
WE signal (_write enable_) in combination with RAS and CAS, generates the
specific row/column command. For example, enabling CAS and WE together
generates a WR command, while enabling only CAS indicates a RD command.

####  3.3.3 SoftMC API

SoftMC offers users complete control over the DDR-based DRAM memory modules by
providing an API (Application Programming Interface) in a high-level
programming language (C++). This API consists of several C++ functions that
generate _instructions_ for the SoftMC hardware, which resides in the FPGA.
These instructions provide support for _(i)_ all possible DDR DRAM commands;
and _(ii)_ managing the SoftMC hardware, e.g., inserting a delay between
commands, configuring the auto-refresh mechanism (see section 3.3.5). Using
the SoftMC API, any sequence of DRAM commands can be easily and flexibly
implemented in SoftMC with little effort and time from the programmer.

When users would like to issue a sequence of commands to the DRAM, they _only_
need to make SoftMC API function calls. Users can easily generate
_instructions_ by calling the API _functions_. Users can insert the generated
instructions to an _InstructionSequence_ , which is a data structure that the
SoftMC API provides to keep the instructions in order and easily send them to
the SoftMC hardware via the driver. There are three types of API functions:
_(i) command functions_ , which provide the user the ability to specify DDR-
compatible DRAM commands (e.g., ACT, PRE); _(ii)_ the _wait function_ , which
provide the user the ability to specify a time interval between two commands
(thereby allowing the user to determine the latencies of each timing
parameter); and _(iii) aux functions_ , which provide the user with control
over various auxiliary operations. Programming the memory controller using
such high-level C++ function calls enables users who are inexperienced in
hardware design to easily develop sophisticated memory controllers.

Command Functions. For every DDR command, we provide a command function that
generates an instruction, which instructs the SoftMC hardware to execute the
command. For example, _genACT(b, r)_ function generates an instruction that
informs the SoftMC controller to issue an ACT to DRAM row _r_ in bank _b_.
Program LABEL:softmc_code:softmc_write shows a short example program that
writes to a single cache line (i.e., DRAM column) in a closed row. In Program
LABEL:softmc_code:softmc_write, there are three command functions: _genACT()_
, _genWR()_ , and _genPRE()_ , in lines 2, 4, and 6, which generate SoftMC
hardware instructions that inform the SoftMC hardware to issue ACT, WR, and
PRE commands to the DRAM.

⬇

1InstructionSequence iseq;

2iseq.insert(genACT(bank, row));

3iseq.insert(genWAIT(tRCD));

4iseq.insert(genWR(bank, col, data));

5iseq.insert(genWAIT(tCL+tBL+tWR));

6iseq.insert(genPRE(bank));

7iseq.insert(genWAIT(tRP));

8iseq.insert(genEND());

9iseq.execute(fpga));

Program 1: Performing a write operation to a closed row using the SoftMC API.

Wait Function. We provide a single function, _genWAIT(t)_ , which generates an
instruction to inform the SoftMC hardware to wait _t_ DRAM cycles before
executing the next instruction. With this one function, the user can implement
any timing constraint. For example, in Program LABEL:softmc_code:softmc_write,
the controller should wait for tRCD after the ACT before issuing a WR to the
DRAM. We simply add a call to _genWAIT()_ on Line 3 to insert the tRCD delay.
Users can easily provide any value that they want to test for the time
interval (e.g., a reduced tRCD, to see if errors occur when the timing
parameter is reduced).

Aux Functions. There are a number of auxiliary functions that allow the user
to configure the SoftMC hardware. One such function, _genBUSDIR(dir)_ , allows
the user to switch (i.e., turn around) the DRAM bus direction. To reduce the
number of IO pins, the DDR interface uses a bi-directional data bus between
DRAM and the controller. The bus must be set up to move data from DRAM to the
controller before issuing a RD, and from the controller to DRAM before issuing
a WR. After _genBUSDIR()_ is called, the user should follow it with a
_genWAIT()_ function, to ensure that the bus is given enough time to complete
the turn around. The time interval for this wait corresponds to the tWTR and
tRTW timing parameters in the DDR interface.

We provide an auxiliary function to control auto-refresh behavior. The SoftMC
hardware includes logic to _automatically_ refresh the DRAM cells after a
given time interval (tREFI) has passed. Auto-refresh operations are not added
to the code by the user, and are instead handled directly within our hardware.
In order to adjust the interval at which refresh is performed, the user can
invoke the _genREF_CONFIG()_ function. To disable the auto-refresh mechanism,
the user needs to set the refresh interval to 0 by calling that function. The
user can invoke the same function to adjust the number of cycles that specify
the refresh latency (tRFC). We describe the auto-refresh functionality of the
SoftMC hardware in more detail in section 3.3.5.

Finally, we provide a function, _genEND()_ , which generates an _END_
instruction to inform the SoftMC hardware that the end of the instruction
sequence has been reached, and that the SoftMC hardware can start executing
the sequence. Note that the programmer should call _iseq.execute(fpga)_ to
send the instruction sequence to the driver, which in turn sends the sequence
to the SoftMC hardware. Program LABEL:softmc_code:softmc_write shows an
example of this in Lines 8–9.

####  3.3.4 Instruction Types and Encoding

Figure 3.2 shows the encodings for the key SoftMC instruction types.111We
refer the reader to the source-code and manual of SoftMC for the encoding of
_all_ instruction types [98]. All SoftMC instructions are 32 bits wide, where
the most-significant 4 bits indicate the type of the instruction.

InstrType |
---|---
DDR (4) |  _unused_ (3) |  CKE, CS (2), RAS, CAS, WE (6) | Bank (3) | Addr (16)
WAIT (4) | cycles (28)
BUSDIR (4) |  _unused_ (27) | dir (1)
END (4) |  _unused_ (28)
?figurename? 3.2: Key SoftMC instruction types.

DDR Instruction Type. Recall from section 3.3.2 that the DDR commands are
encoded using several fields (e.g., RAS, CAS, WE) that correspond to the
signals of the DDR interface, which sits between the memory controller and the
DRAM. The SoftMC instructions generated by our API functions encode these same
fields, as shown in the instruction encoding for the DDR type instructions in
Figure 3.2. Thus, DDR type instructions can represent _any_ DDR command from
the DDR3 specification [7]. For example, the user needs to set CKE = 1, CS =
0, RAS = 0, CAS = 1, and WE = 1 to create an ACT command to open the row in
the bank specified by _Address_ and _Bank_ , respectively. Other DDR commands
can easily be represented by setting the appropriate instruction fields
according to the DDR3 specification.

Other Instruction Types. Other instructions carry information to the FPGA
using the bottom 28 bits of their encoding. For example, the _WAIT_
instruction uses these bits to encode the wait latency, i.e., wait cycles. The
_BUSDIR_ instruction uses the least significant bit of the bottom 28 bits to
encode the new direction of the data bus. The _END_ instruction requires no
additional parameters, using only the _InstrType_ field.

####  3.3.5 Hardware Architecture

The driver sends the SoftMC instructions across the PCIe bus to the FPGA,
where the _SoftMC hardware_ resides. Figure 3.3 shows the five components that
make up the _SoftMC hardware_ : _(i)_ an _instruction receiver_ , which
accumulates the instructions sent over the PCIe bus; _(ii)_ an _instruction
dispatcher_ , which decodes each instruction into DRAM control signals and
address bits, and then sends the decoded bits to the DRAM module through the
memory bus; _(iii)_ a _read capture module_ , which receives data read from
the DRAM and sends it to the host machine over PCIe; _(iv)_ an _auto-refresh
controller_ ; and _(v)_ a _calibration controller_ , responsible for ensuring
data integrity on the memory bus. SoftMC also includes two interfaces: a _PCIe
bus controller_ and a _DDR PHY_. Next, we describe each of these components in
detail.

?figurename? 3.3: SoftMC hardware architecture.

Instruction Receiver. When the driver sends SoftMC instructions across the
PCIe bus, the instruction receiver buffers them in an _instruction queue_
until the _END_ instruction is received. When the _END_ instruction arrives,
the instruction receiver notifies the instruction dispatcher that it can start
fetching instructions from the instruction queue. Execution ends when the
instruction queue is fully drained.

Instruction Dispatcher. The instruction dispatcher fetches an instruction,
decodes it, and executes the operation that the instruction indicates, at the
corresponding component. For example, if the type of the instruction is _DDR_
, the dispatcher decodes the control signals and address bits from the
instruction, and then sends them to the DRAM using the DDR PHY. Since the
FPGAs operate at a lower frequency than the memory bus, the dispatcher
supports fetching and issuing of multiple DDR commands per cycle. As another
example, if the instruction type is _WAIT_ , it is not sent to the DRAM, but
is instead used to count down the number of cycles before the instruction
dispatcher can issue the next instruction to the DRAM. Other instruction types
(e.g., _BUSDIR, END_) are handled in a similar manner.

Read Capture Module. After a RD command is sent to the DRAM module, the read
capture module receives the data emitted by the DRAM from the DDR PHY, and
sends it to the host machine via the PCIe controller. The read capture module
is responsible for ensuring data alignment and ordering, in case the DDR PHY
operates on a different clock domain or reorders the data output.

Auto-Refresh Controller. The auto-refresh controller includes two registers,
which store the refresh interval (tREFI) and refresh latency (tRFC) values
(See section 2.2). It issues periodic refresh operations to DRAM, based on the
values stored in the registers. If the instruction dispatcher is in the middle
of executing an instruction sequence that was received from the host machine,
at the time a refresh is scheduled to start, the auto-refresh controller
_delays_ the refresh operation until the dispatcher finishes the execution of
the instruction sequence. Note that a small delay does not pose any problems,
as the DDR standard allows for a small amount of flexibility for scheduling
refresh operations [41, 7, 45]. However, if the instruction sequence is too
long, and delays the refresh operation beyond a critical interval (i.e., more
than 8X the tREFI), it is the user’s responsibility to redesign the test
routine and split up the long instruction sequence into smaller sequences.

Calibration Module. This module ensures the data integrity of the memory bus.
It periodically issues a command to perform _short ZQ calibration_ , which is
an operation specified by the DDR standard to calibrate the output driver and
on-die termination circuits [177, 292]. SoftMC provides hardware support for
_automatic_ calibration, in order to ease programmer effort.

PCIe Controller and DDR PHY. The PCIe controller acts as an interface between
the host machine and the rest of the SoftMC hardware. The DDR PHY sits between
the SoftMC hardware and the DRAM module, and is responsible for initialization
and clock synchronization between the FPGA and the DRAM module.

####  3.3.6 SoftMC Prototype

We describe the key implementation details of our first prototype of SoftMC.

FPGA Board. The current SoftMC prototype targets a Xilinx ML605 FPGA board
[293]. It consists of _(i)_ a Virtex-6 FPGA chip, which we use to implement
our SoftMC design; and _(ii)_ a SO-DIMM slot, where we plug in the real DRAM
modules to be tested.

PCIe Interface. The PCIe controller uses _(i)_ the Xilinx PCIe Endpoint IP
Core [294], which implements low-level operations related to the physical
layer of the PCIe bus; and _(ii)_ the RIFFA IP Core [291], which provides
high-level protocol functions for data communication. The communication
between the host machine and the FPGA board is established through a PCIe 2.0
link.

DDR PHY. SoftMC uses the Xilinx DDR PHY IP Core [295] to communicate with the
DRAM module. This PHY implements the low-level operations to transfer data
over the memory channel to/from the DRAM module. The PHY transmits two DDR
commands each memory controller cycle. As a result, our SoftMC implementation
fetches and dispatches two instructions from the instruction queue every
cycle.

Performance. To evaluate the performance of our prototype, we time the
execution of a test routine that performs operations on a particular region of
the DRAM and then reads data back from this region, repeating the test on
different DRAM regions until the entire DRAM module has been tested. We model
the expected overall execution time of the test as:

$Expected\leavevmode\nobreak\ Execution\leavevmode\nobreak\
Time=(S+E+R)*\dfrac{Capacity}{Region\leavevmode\nobreak\ Size}$ (3.1)

where _S (Send)_ , _E (Execute)_ , and _R (Receive)_ are the host-to-FPGA PCIe
latency, execution time of the command sequence in the FPGA, and FPGA-to-host
PCIe latency, respectively, that is required for each DRAM region. For our
prototype, we measure _S_ and _R_ to both be
$22\text{\,}\mathrm{\SIUnitSymbolMicro s}$ on average (with
$\pm$$2\text{\,}\mathrm{\SIUnitSymbolMicro s}$ variation). _E_ varies based on
the length of the command sequence. For the sequence given in Program
LABEL:softmc_code:write_row, which we use in our first use case (section
3.4.1), we calculate _E_ as $16\text{\,}\mathrm{\SIUnitSymbolMicro s}$ using
typical DDR3 timing parameters. Since that command sequence tests only a
single 8KB row, the _Region Size_ is 8KB. For an experiment testing a full 4GB
memory module completely, we find that our prototype runs Program
LABEL:softmc_code:write_row on the entire module in approximately 31.5
seconds. An at-speed DRAM controller, which, at best, has _S_ and _R_ equal to
zero, would run the same test in approximately 11.5 seconds, i.e., only 2.7x
faster. As _E_ increases, the performance of our SoftMC prototype gets closer
to the performance of an at-speed DRAM controller. We conclude that our SoftMC
prototype is fast enough to run test routines that cover an entire memory
module.

###  3.4 Example Use Cases

Using our SoftMC prototype, we perform two case studies on randomly-selected
real DRAM chips from three manufacturers. First, we discuss how a simple
retention test can be implemented using SoftMC, and present the experimental
results of that test (section 3.4.1). Second, we demonstrate how SoftMC can be
leveraged to test the expected effect of two recently-proposed mechanisms that
aim to reduce DRAM access latency (section 3.4.2). Both use cases demonstrate
the flexibility and ease of use of SoftMC.

####  3.4.1 Retention Time Distribution Study

This test aims to characterize data retention time in different DRAM modules.
The retention time of a cell can be determined by testing the cell with
different refresh intervals. The cell fails at a refresh interval that is
greater than its retention time. In this test, we gradually increase the
refresh interval from the default $64\text{\,}\mathrm{ms}$ and count the
number of bytes that have an incorrect value at each refresh interval.

#####  3.4.1.1 Test Methodology

We perform a simple test to measure the retention time of the cells in a DRAM
chip. Our test consists of three steps: _(i)_ We write a reference data
pattern (e.g. all zeros, or all ones) to an entire row. _(ii)_ We wait for the
specified refresh interval, so that the row is idle for that time and all
cells gradually leak charge. _(iii)_ We read data back from the same row and
compare it against the reference pattern that we wrote in the first step. Any
mismatch indicates that the cell could not hold its data for that duration,
which resulted in a bit flip. We count the number of bytes that have bit flips
for each test.

We repeat this procedure for all rows in the DRAM module. The read and write
operations in the test are issued with the standard timing parameters, to make
sure that the only timing delay that affects the cells is the refresh
interval.

#####  3.4.1.2 Implementation with SoftMC.

Writing Data to DRAM. In Program LABEL:softmc_code:write_row, we present the
implementation of the first part of our retention time test, where we write
data to a row, using the SoftMC API. First, to activate the row, we insert the
instruction generated by the _genACT()_ function to an instance of the
_InstructionSequence_ (Lines 1-2). This function is followed by a _genWAIT()_
function (Line 3) that ensures that the activation completes with the standard
timing parameter tRCD. Second, we issue write instructions to write the data
pattern in each column of the row. This is implemented in a loop, where, in
each iteration, we call _genWR()_ (Line 5), followed by a call to _genWAIT()_
function (Line 6) that ensures proper delay between two WR operations. After
writing to all columns of the row, we insert another delay (Line 8) to account
for the _write recovery_ time (section 2.2). Third, once we have written to
all columns, we close the row by precharging it. This is done by the
_genPRE()_ function (Line 9), followed by a _genWAIT()_ function with standard
tRP timing. Finally, we call the _genEND()_ function to indicate the end of
the instruction sequence, and send the test program to the FPGA by calling the
_execute()_ function.

Employing a Specific Refresh Interval. Using SoftMC, we can implement the
target refresh interval in two ways. We can use the auto-refresh support
provided by the SoftMC hardware, by setting the tREFI parameter to our target
value, and letting the FPGA take care of the refresh operations.
Alternatively, we can disable auto-refresh, and manually control the refresh
operations from the software. In this case, the user is responsible for
issuing refresh operations at the right time. In this retention test, we
disable auto-refresh and use a software clock to determine when we should read
back data from the row (i.e., refresh the row).

⬇

1InstructionSequence iseq;

2iseq.insert(genACT(bank, row));

3iseq.insert(genWAIT(tRCD));

4for(int col = 0; col < COLUMNS; col++){

5 iseq.insert(genWR(bank, col, data));

6 iseq.insert(genWAIT(tBL));

7}

8iseq.insert(genWAIT(tCL+tWR));

9iseq.insert(genPRE(bank));

10iseq.insert(genWAIT(tRP));

11iseq.insert(genEND());

12iseq.execute(fpga));

Program 2: Writing data to a row using the SoftMC API.

Reading Data from DRAM. Reading data back from the DRAM requires steps similar
to DRAM writes (presented in Program LABEL:softmc_code:write_row). The only
difference is that, instead of issuing a WR command, we need to issue a RD
command and enforce read-related timing parameters. In the SoftMC API, this is
done by calling the _genRD()_ function in place of the _genWR()_ function, and
specifying the appropriate read-related timing parameters. After the read
operation is done, the FPGA sends back the data read from the DRAM module, and
the user can access that data using the _fpga_recv()_ function provided by the
driver.

Based on the intuitive code implementation of the retention test, we conclude
that it requires minimal effort to write test programs using the SoftMC API.
Our full test is provided in our open-source release [98].

#####  3.4.1.3 Results

We perform the retention time test at room temperature, using 24 chips from
three major manufacturers. We vary the refresh interval from
$64\text{\,}\mathrm{ms}$ to $8192\text{\,}\mathrm{ms}$ exponentially. Figure
3.4 shows the results for the test, where the x-axis shows the refresh
interval in milliseconds, and the y-axis shows the number of erroneous bytes
found in each interval. We make two major observations.

_(i)_ We do not observe any retention failures until we test with a refresh
interval of $1\text{\,}\mathrm{s}$. This shows that there is a large safety
margin for the refresh interval in modern DRAM chips, which is conservatively
set to $64\text{\,}\mathrm{ms}$ by the DDR standard.222DRAM manufacturers
perform retention tests that are similar to ours (but with proprietary in-
house infrastructures that are not disclosed). Their results are similar to
ours [65, 11, 66, 10], showing significant margin for the refresh interval.
This margin is added to ensure reliable DRAM operation for the worst-case
operating conditions (i.e., worst case temperature) and for worst-case cells,
as has been shown by prior works [11, 65, 66, 10].

?figurename? 3.4: Number of erroneous bytes observed in retention time tests.

_(ii)_ We observe that the number of failures increases drastically with the
increase in refresh interval.

Prior experimental studies on retention time of DRAM cells have reported
similar observations as ours [66, 65, 11, 228, 227]. We conclude that SoftMC
can easily reproduce prior known DRAM experimental results, validating the
correctness of our testing infrastructure and showing its flexibility and ease
of use.

####  3.4.2 Evaluating the Expected Effect of Two Recently-Proposed Mechanisms
in Existing DRAM Chips

Two recently-proposed mechanisms, ChargeCache [17] and NUAT [182], provide
low-latency access to highly-charged DRAM cells. They both are based on the
key idea that a highly-charged cell can be accessed faster than a cell with
less charge [11]. ChargeCache observes that cells belonging to _recently-
accessed_ DRAM rows are in a highly-charged state and that such rows are
likely to be accessed again in the near future. ChargeCache exploits the
highly-charged state of these recently-accessed rows to lower the latency for
later accesses to them. NUAT observes that _recently-refreshed_ cells are in
highly-charged state, and thus it lowers the latency for accesses to recently-
refreshed rows. Prior to issuing an ACT command to DRAM, both ChargeCache and
NUAT determine whether the target row is in a highly-charged state. If so, the
memory controller uses reduced tRCD and tRAS timing parameters to perform the
access.

In this section, we evaluate whether or not the expected latency reduction
effect of these two works is observable in existing DRAM modules, using
SoftMC. We first describe our methodology for evaluating the improvement in
the tRCD and tRAS parameters. We then show the results we obtain using SoftMC,
and discuss our observations.

#####  3.4.2.1 Evaluating DRAM Latency with SoftMC.

Methodology. In our experiments, we use 24 DDR3 chips (i.e., three SO-DIMMs)
from three major vendors. To stress DRAM reliability and maximize the amount
of cell charge leakage, we raise the test temperature to 80∘C (significantly
higher than the common-case operating range of 35-55∘C [11]) by enclosing our
FPGA infrastructure in a temperature-controlled heat chamber (see Figure 3.1).
For all experiments, the temperature within the heat chamber was maintained
within 0.5∘C of the target 80∘C temperature.

To study the impact of charge variation in cells on access latency in existing
DRAM chips, which is dominated by the tRCD and tRAS timing parameters [2, 11,
10] (see section 2.2), we perform experiments to test the headroom for
reducing these parameters. In our experiments, we vary one of the two timing
parameters, and test whether the original data can be read back correctly with
the reduced timing. If the data that is read out contains errors, this
indicates that the timing parameter cannot be reduced to the tested value
without inducing errors in the data. We perform the tests using a variety of
data patterns (e.g., 0x00, 0xFF, 0xAA, 0x55) because 1) different DRAM cells
store information (i.e., 0 or 1) in different states (i.e., charged or empty)
[65] and 2) we would like to stress DRAM reliability by increasing the
interference between adjacent bitlines [68, 65, 67]. We also perform tests
using different refresh intervals, to study whether the variation in charge
leakage increases significantly if the time between refreshes increases.

tRCD Test. We measure how highly-charged cells affect the tRCD timing
parameter, by using a custom tRCD value to read data from a row to which we
previously wrote a reference data pattern. We adjust the time between writing
a reference data pattern and performing the read, to vary the amount of charge
stored within the cells of a row. In Figure 3.5(a), we show the command
sequence that we use to test whether recently-refreshed DRAM cells can be
accessed with a lower tRCD, compared to cells that are close to the end of the
refresh interval. We perform the write and read operations to each DRAM row
one column at a time, to ensure that each read incurs the tRCD latency. First
( 1 in Figure 3.5(a)), we perform a reference write to the DRAM column under
test by issuing ACT, WR, and PRE successively with the _default_ DRAM timing
parameters. Next ( 2), we wait for the duration of a time interval (T1), which
is the refresh interval in practice, to vary the charge contained in the
cells. When we wait longer, we expect the target cells to have less charge at
the end of the interval. We cover a wide range of wait intervals, evaluating
values between 1 and $512\text{\,}\mathrm{ms}$. Finally ( 3), we read the data
from the column that we previously wrote to and compare it with the reference
pattern. We perform the read with the custom tRCD value for that specific
test. We evaluate tRCD values ranging from 3 to 6 (default) cycles. Since a
tRCD of 3 cycles produced errors in every run, we did not perform any
experiments with a lower tRCD.

We process multiple rows in an interleaved manner (i.e., we write to multiple
rows, wait, and then verify their data one after another) in order to further
stress the reliability of DRAM [11]. We repeat this process for all DRAM rows
to evaluate the entire memory module.

(a)

(b)

?figurename? 3.5: Timelines that illustrate the methodology for testing the
improvement of (a) tRCD and (b) tRAS on highly-charged DRAM cells.

tRAS Test. We measure the effect of accessing highly-charged rows on the tRAS
timing parameter by issuing the ACT and PRE commands, with a custom tRAS
value, to a row. We check if that row still contains the same data that it
held before the ACT-PRE command pair was issued. Figure 3.5(b) illustrates the
methodology for testing the effect of the refresh interval on tRAS. First (
1), we write the reference data pattern to the selected DRAM row with the
default timing parameters. Different from the tRCD test, we write to _every
column_ in the open row (before switching to another row) to save cycles by
eliminating a significant amount of ACT and PRE commands, thereby reducing the
testing time. Next ( 2), we wait for the duration of time interval T2, during
which the DRAM cells lose a certain amount of charge. To refresh the cells (
3), we issue an ACT-PRE command pair associated with a custom tRAS value. When
the ACT-PRE pair is issued, the charge in the cells of the target DRAM row may
not be fully restored if the wait time is too long or the tRAS value is too
short, potentially leading to loss of data. Next ( 4), we wait again for a
period of time T3 to allow the cells to leak a portion of their charge.
Finally ( 5), we read the row using the default timing parameters and test
whether it still retains the correct data. Similar to the tRCD test, to stress
the reliability of DRAM, we simultaneously perform the tRAS test on multiple
DRAM rows.

We would expect, from this experiment, that the data is likely to maintain its
integrity when evaluating reduced tRAS with shorter wait times (T2), as the
higher charge retained in a cell with a short wait time can enable a reliable
reduction on tRAS. In contrast, we would expect failures to be more likely
when using a reduced tRAS with a longer wait time, because the cells would
have a low amount of charge that is not enough to reliably reduce tRAS.

#####  3.4.2.2 Results

We analyze the results of the tRCD and tRAS tests, for 24 real DRAM chips from
different vendors, using the test programs detailed in section 3.4.2.1. We
evaluate tRCD values ranging from 3 to 6 cycles, and tRAS values ranging from
2 to 14 cycles, where the maximum number for each is the default timing
parameter value. For both tests, we evaluate refresh intervals between 8 and
$512\text{\,}\mathrm{ms}$ and measure the number of observed errors during
each experiment.

(a)

(b)

(c)

?figurename? 3.6: Effect of reducing tRCD on the number of errors at various
refresh intervals.

(a)

(b)

(c)

?figurename? 3.7: Effect of reducing tRAS on the number of errors at various
refresh intervals.

Figures 3.6 and 3.7 depict the results for the tRCD test and the tRAS test,
respectively, for three DRAM modules (each from a different DRAM vendor). We
make three major observations:

_(i)_ _Within the duration of the standard refresh interval (
$64\text{\,}\mathrm{ms}$), DRAM cells do not leak a sufficient amount of
charge to have a negative impact on DRAM access latency.333Other studies have
shown methods to take advantage of the fact that latencies can be reduced
without incurring errors [11, 10]._ For refresh intervals less than or equal
to $64\text{\,}\mathrm{ms}$, we observe little to no variation in the number
of errors induced. Within this refresh interval range, depending on the tRCD
or tRAS value, the errors generated are either zero or a constant number. We
make the same observation in both the tRCD and tRAS tests for all three DRAM
modules.

For all the modules tested, using different data patterns and stressing DRAM
operation with temperatures significantly higher than the common-case
operating conditions, we can significantly reduce tRCD and tRAS parameters,
without observing any errors. We observe errors only when tRCD and tRAS
parameters are too small to correctly perform the DRAM access, regardless of
the charge amount of the accessed cells.

_(ii)_ _The large safety margin employed by the manufacturers protects DRAM
against errors even when accessing DRAM cells with low latency._ We observe no
change in the number of induced errors for tRCD values less than the default
of 6 cycles (down to 4 cycles in modules A and B, and 5 cycles in module C).
We observe a similar trend in the tRAS test: tRAS can be reduced from the
default value of 14 cycles to 5 cycles without increasing the number of
induced errors for _any_ refresh interval.

We conclude that even at temperatures much higher than typical operating
conditions, there exists a large safety margin for access latency in existing
DRAM chips. This demonstrates that DRAM cells are much _stronger_ than their
timing specifications indicate.444Similar observations were made by prior work
[11, 159, 10]. In other words, the timing margin in most DRAM cells is very
large, given the existing timing parameters.

_(iii)_ _The expected effect of ChargeCache and NUAT, that highly-charged
cells can be accessed with lower latency, is slightly observable only when
very long refresh intervals are used._ For each of the tests, we observe a
significant increase in the number of errors at refresh intervals that are
much higher than the typical refresh interval of $64\text{\,}\mathrm{ms}$,
demonstrating the variation in charge held by each of the DRAM cells. Based on
the assumptions made by ChargeCache and NUAT, we expect that when lower values
of tRCD and tRAS are employed, the error rate should increase more rapidly.
However, we find that for all but the minimum values of tRCD and tRAS (and for
tRCD = 4 for module C), the tRCD and tRAS latencies have almost no impact on
the error rate.

We believe that the reason we cannot observe the expected latency reduction
effect of ChargeCache and NUAT on existing DRAM modules is due to the internal
behavior of existing DRAM chips that does not allow latencies to be reduced
beyond a certain point: we cannot _externally control_ when the sense
amplifier gets enabled, since this is dictated with a fixed latency
internally, regardless of the charge amount in the cell. The sense amplifiers
are enabled only after charge sharing, which starts by enabling the wordline
and lasts until sufficient amount of charge flows from the activated cell into
the bitline (See section 2.2), is expected to complete. Within existing DRAM
chips, the expected charge sharing latency (i.e., the time when the sense
amplifiers get enabled) is not represented by a timing parameter managed by
the memory controller. Instead, the latency is controlled internally within
the DRAM using a fixed value [296, 144]. ChargeCache and NUAT require that
charge sharing completes in less time, and the sense amplifiers get enabled
faster for a highly-charged cell. However, since existing DRAMs provide no way
to control the time it takes to enable the sense amplifiers, we cannot harness
the potential latency reduction possible for highly-charged cells [296].
Reducing tRCD affects the time spent _only after charge sharing_ , at which
point the bitline voltages exhibit similar behavior regardless of the amount
of charge initially stored within the cell. Consequently, we are unable to
observe the expected latency reduction effect of ChargeCache and NUAT by
simply reducing tRCD, even though we believe that the mechanisms are sound and
can reduce latency (assuming the behavior of DRAM chips is modified). If the
DDR interface exposes a method of controlling the time it takes to enable the
sense amplifiers in the future, SoftMC can be easily modified to use the
method and fully evaluate the latency reduction effect of ChargeCache and
NUAT.

Summary. Overall, we make two major conclusions from the implementation and
experimental results of our DRAM latency experiments. First, SoftMC provides a
simple and easy-to-use interface to quickly implement tests that characterize
modern DRAM chips. Second, SoftMC is an effective tool to validate or refute
the expected effect of existing or new mechanisms on existing DRAM chips.

###  3.5 Limitations of SoftMC

In the previous section, we presented and discussed some examples
demonstrating the benefits of SoftMC. These use cases illustrate the
flexibility, ease of use, and capability of SoftMC in characterizing DRAM
operations and evaluating new DRAM mechanisms (using the DDR interface).
Although our SoftMC prototype already caters for a wide range of use cases, it
also has some limitations that arise mainly from its current hardware
implementation.

Inability to Evaluate System Performance. Studies where the applications run
on the host machine and access memory within the module under test can
demonstrate the impact of memory reliability and latency on system performance
using real applications. However, such studies are difficult to perform in any
FPGA-based DRAM testing infrastructure that connects the host and the FPGA
over the PCIe bus (including SoftMC). This is due to the the long latency
associated with the PCIe bus (in microseconds [297, 298, 299]; in contrast to
DRAM access latency that is within $1580\text{\,}\mathrm{ns}$ [2, 9]). Note
that this long PCIe bus latency does _not_ affect our tests (as explained in
section 3.3.1), but it would affect the performance of a system that would use
SoftMC as the main memory controller. One way to enable performance studies
with SoftMC is to add support for trace-based execution, where the traces
(i.e., memory access requests) are collected by executing workloads on real
systems or simulating them. The host PC can transform the traces into SoftMC
instructions and transmit them to SoftMC hardware, where we can buffer the
instructions and emulate microarchitectural behavior of a memory controller
developed using SoftMC while avoiding the PCIe bus latency. This functionality
requires coordination between the system/simulator and SoftMC, and we expect
to add such support by enabling coordination between our DRAM simulator,
Ramulator [138, 300], and a future release of SoftMC.

Coarse-Grained Timing Resolution. As FPGAs are significantly slower than the
memory bus, the minimum time interval between two consecutive commands that
the FPGA can send to the memory bus is limited by the FPGA frequency. In our
current prototype, the DDR interface of the ML605 operates at 400MHz, allowing
SoftMC to issue two consecutive commands with a minimum interval of
$2.5\text{\,}\mathrm{ns}$. Therefore, the timing parameters in SoftMC can be
changed only at the granularity of $2.5\text{\,}\mathrm{ns}$. However, it is
possible to support finer-granularity resolution with faster FPGA boards
[301].

Limitation on the Number of Instructions Stored in the FPGA. The number of
instructions in one test program (that is executed atomically in SoftMC) is
limited by the size of the _Instruction Queue_ in SoftMC. To keep FPGA
resource usage low, the size of the _Instruction Queue_ in the current SoftMC
prototype is limited to 8192 instructions. In the future, instead of
increasing the size of the _Instruction Queue_ , which would increase resource
utilization in the FPGA, we plan to extend the SoftMC hardware with control
flow instructions to support arbitrary-length tests. Adding control flow
instructions would enable loops that can iterate over large structures (e.g.,
DRAM rows, columns) in hardware with a small number of SoftMC instructions,
avoiding the need for the host machine to issue a large number of instructions
in a loop-unrolled manner. Such control flow support would further improve the
ease of use of SoftMC.

###  3.6 Research Directions Enabled by SoftMC

We believe SoftMC can enable many new studies of the behavior of DRAM and
other memories. In fact, since the release of SoftMC to the public as part of
this research, many works [29, 10, 99, 100, 71, 101, 70, 102, 103, 69, 104,
75, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114] used different versions
of SoftMC in various DRAM characterization studies. In this section, we
briefly describe several other example studies that SoftMC can further enable.

More Characterization of DRAM. The SoftMC DRAM testing infrastructure can test
any DRAM mechanism consisting of low-level DDR commands. Therefore, it enables
a wide range of characterization and analysis studies of real DRAM modules
that would otherwise not have been possible without such an infrastructure. We
discuss three such example research directions.

First, as DRAM scales down to smaller technology nodes, it faces key
challenges in both reliability and latency [5, 61, 269, 270, 43, 42, 65, 67,
68, 66, 69, 74, 70, 71]. Unfortunately, there is no comprehensive experimental
study that characterizes and analyzes the trends in DRAM cell operations and
behavior with technology scaling across various DRAM generations. The SoftMC
infrastructure can help us answer various questions to this end: How are the
cell characteristics, reliability, and latency changing with different
generations of technology nodes? Do all DRAM operations and cells get affected
by scaling at the same rate? Which DRAM operations are getting worse?

Second, aging-related failures in DRAM can potentially affect the reliability
and availability of systems in the field [196, 197]. However, the causes,
characteristics, and impact of _aging_ have remained largely unstudied. Using
SoftMC, it is possible to devise controlled experiments to analyze and
characterize aging. The SoftMC infrastructure can help us answer questions