# Real-time Digital RF Emulation – II: A Near Memory Custom Accelerator

M. Mukherjee*, X. Mao*, N. Mizanur Rahman*, C. DeLude*, J. Driscoll*, S.
Sharma, P. Behnam, U. Kamal,
J. Woo, D. Kim, S. Khan, J. Tong, J. Seo, P. Sinha, M. Swaminathan, T.
Krishna, S. Pande, J. Romberg,
and S. Mukhopadhyay (*Equal Contribution Authors) This paper was produced by
the IEEE Publication Technology Group. They are in Piscataway, NJ.Manuscript
received Oct, 2023.The authors are with Georgia Institute of Technology,
Atlanta, GA, USA. Corresponding author<EMAIL_ADDRESS>work was supported by DARPA & NIWC Pacific (N66001-20-C-4001). Any opinions,
findings and conclusions are those of the author(s) and do not necessarily
reflect the views of DARPA or NIWC Pacific.

###### Abstract

A near memory hardware accelerator, based on a novel direct path computational
model, for real-time emulation of radio frequency systems is demonstrated. Our
evaluation of hardware performance uses both application-specific integrated
circuits (ASIC) and field programmable gate arrays (FPGA) methodologies: 1).
The ASIC testchipimplementation, using TSMC 28nm CMOS, leverages distributed
autonomous control to extract concurrency in compute as well as low latency.
It achieves a $518$ MHz per channel bandwidth in a prototype $4$-node system.
The maximum emulation range supported in this paradigm is $9.5$ km with $0.24$
$\mu$s of per-sample emulation latency. 2). The FPGA-based implementation,
evaluated on a Xilinx ZCU104 board, demonstrates a $9$-node test case (two
Transmitters, one Receiver, and $6$ passive reflectors) with an emulation
range of $1.13$ km to $27.3$ km at $215$ MHz bandwidth.

###### Index Terms:

hardware accelerators, near-memory, radio frequency emulator, real-time.

##  I Introduction

Virtual radio frequency (RF) emulators have the potential to greatly reduce
the  cost of testing wide-bandwidth RF systems used in radar, electronic
warfare, and advanced driver assistance  by offering an alternative to field
testing[1, 2]. Since software simulation of physical models is orders of
magnitude slower than real-time, FPGA-based emulators were developed in [1, 2,
3, 4] as a means of accelerating computations. However, these existing FPGA
testbeds cannot simultaneously achieve  the high computational throughput and
low computational latency  required to viably test many systems. Coupled with
scalability, the three main points of focus in next-generation emulator design
are throughput, scalability, and latency; necessitating the development of
custom accelerators. We begin with a brief overview of these motivating points
and where they arise in RF emulation.

In terms of throughput, since real-time RF emulation is performed sample-by-
sample, instead of in a batch, the maximum frequency of the hardware
determines the maximum sample rate of (digitized) RF signals that can be
processed, which in turn determines the maximum emulated RF bandwidth. The
existing FPGA-based frameworks are limited to $100$ MHz of bandwidth.

Basic RF emulation involves accurate modeling of signal propagation delay,
path loss, and  radar cross section (RCS)111In part I of this series we refer
to this as the scattering profile, but these are equivalent terms. scattering
between objects in a scene at a target RF signal bandwidth. The existing
tapped delay (TDL) based computational model, as described in [1, 2, 3, 4],
requires $O(M^{3})$ computation, which leads to the $16$ PMAC/s compute
throughput for $M=200$ interacting objects at $2$ GHz of channel bandwidth.
Hence, the development of scalable models is extremely important to limit the
computational requirements.

To illustrate the need for low latency, consider the three-object system in
Figure 1, where “Obj#” denotes “object number #”. Assuming free space
propagation the physical delay of an Electromagnetic (EM) signal from Obj1 to
Obj3 ($100$ m spacing) is $0.33$ $\mu$s, whereas, the propagation delay from
Obj1 to Obj2 ($100$ km spacing) is $333.3$ $\mu$s. Hence, a custom accelerator
must ensure deterministic processing delay (latency) controlled to always
match (time-varying) physical propagation delays between each source and
object in the emulated scene. The intrinsic accelerator compute latency must
also be minimized to  reduce the emulator’s smallest possible interaction
distance.

In part I of this two-part series, we developed the direct path computational
model (DPCM) for RF emulation. This model, which leverages a series of careful
factorizations and innovative modeling choices, was shown to be drastically
more computationally and memory efficient than its more traditional
counterparts. Coupled with an easily distributed computational structure, the
model was seen as an attractive option for RF emulator implementation that
merited further investigation.

In this paper, part II of the two-part series, we present the first, to the
best of our knowledge, a near-memory accelerator for real-time emulation of
interaction among wideband RF signals. It explicitly leverages the DPCM model
developed in part I, reducing the per-sample computational requirement to
$O(M^{2})$. The compute model is realized using a high-throughput near-memory
architecture that leverages concurrent processing to maximize emulated RF
bandwidth (chip frequency) as well as minimize the emulated shortest possible
distance between objects.

Figure 1: Overview of RF emulation system

We design throughput-optimized engines for computing physical models including
path loss, RCS, Doppler, and fractional delay correction (FDC)222A full
overview of these phenomena e.g. where they arise and how they are modeled, is
provided in Part I.. A memory-based Single-Input-Multiple-Output FIFO (SIMO-
FIFO) employs a novel hybrid control mechanism to simultaneously emulate
signal propagation delay to multiple objects. The SIMO-FIFO is sub-banked to
maximize throughput as well as to ensure the largest physical distance that
can be emulated. Distributed local delay distribution controllers, one
connected to each sub-bank, and a deadlock-free multi-casting network-on-chip
simultaneously distributes multiple samples per cycle. A global delay
distribution controller configures these distributed controllers to emulate
various (and time-varying) physical distances.

The proposed compute model is validated in software, and the associated
architecture is verified via cycle-level simulation for complex scenarios. A
$16$-object dynamic RF testcase is simulated on the simulator to validate the
architecture. Additionally, a test-chip is fabricated and measured in 28nm
CMOS to demonstrate the emulation of a $4$-node system. The test-chip
demonstrates range estimation in two different RF scenarios and achieves up to
$518$ MHz of RF bandwidth with an emulation range of $0.67$ km - $9.5$ km.
Finally, to further explore advanced and diverse experimental scenarios beyond
the capabilities demonstrated by the ASIC test-chip, we implemented a $9$-node
FPGA-based platform. This strategic choice was driven by the desire to extend
our research to include a wider array of complex RF signal interaction
scenarios. The FPGA platform proves to be exceptionally capable, achieving a
bandwidth of up to $215$ MHz and enabling the emulation of RF signal
interactions over ranges from $1.13$ km to $27.3$ km.

The rest of the paper is organized as follows: Section II discusses the
related work existing in literature. Section III introduces the RF system,
describes the compute model, and discusses the results used to validate it.
Section IV provides details on the accelerator architecture (Data Path and
Control Path). Section V discusses the verification of the proposed
architecture through a C++ cycle-level simulator running various scenarios.
Section VI presents the physical design challenges associated with the
implementation of the prototype data/control path and the measurement results
of experiments performed on the test-chip. Section VII provides a detailed
exposition of the implementation by using an FPGA board, alongside the
measured emulated outcomes and the hardware performance metrics of the design.
The paper is concluded in Section VIII.

##  II Related Work

In [1, 3, 5, 6] the development of hardware based RF emulators is discussed.
However, they universally implement an FPGA-based design with the existing TDL
model, restricting the maximum allowable RF bandwidth and incurring a higher
computational load. Additionally, [5] uses separate storage for each real-time
sample to be fetched. In [6], the possibility of using a single storage buffer
is discussed but it results in a large logic-based implementation in FPGA to
handle memory contention. In [7], a potential architecture for sparse FIR
filtering in RF with memory-based implementation is explored. The authors
discuss a distributed control, co-located with memory, for sample distribution
for sparse FIR filter in high throughput systems with constrained latency.
However, the work is restricted to architecture-level discussions based on the
standard TDL model for RF channels.

Our prior work in [8, 9] introduces a near-memory-based accelerator
architecture for use in RF emulators. Compared to [5], our work uses single
storage (for single point object representations) for all outputs for an RF
node. In this paper, we expand upon our previous work by adding details in
both the compute model and hardware architecture. First, we offer a more
detailed discussion on the hardware architecture, focusing on the autonomous
distributed control that forms the backbone of the design. Further experiments
are also run on the C++ simulator with a 16-object dynamic testcase being
presented in this work.  Following that, we demonstrate an ASIC-based
implementation as a prototype RF emulator for a 4-node system in 28 nm CMOS.
We discuss the testchip capabilities and present measurement-based results
from testchip experiments to demonstrate the high-performance architecture for
emulation of RF interactions. Finally, we use the ZCU104 FPGA board to build a
9-node system, thereby demonstrating the scalability of our design through
further verification of the hardware emulation results. A comparison of our
design’s performance to state-of-art design is also provided.

##  III Compute Model

The architecture follows the direct path compute model (DPCM) developed in
part I of this series. As a brief overview, in this paradigm, each object
(containing the accelerator logic engine) acts as both a transmitter and
receiver. It constantly sends and receives signals (and applies physical
models on the received signals) along the direct path between objects to
represent the channels between them. An external control software translates
the physical environment to scenario configuration packets (SCPs) and re-
configures the accelerator (RF node) every 1 ms (through a scenario
programming interface) in response to dynamic changes in emulated scenarios
(without stalling real-time operation). For node-to-node communication, high-
speed interconnects need to be used for real-time data transfer.

(a)

(b)

Figure 2: DPCM model (a) Computations in DPCM (b) The unrolled communication
graph of a simple 3 node DPCM with emitters on the left and receivers on the
right. $s_{i\to j}$ denotes a signal sent from node $i$ to node $j$.

A full description of the DPCM model can be found in part I of this series,
but we use this section to briefly highlight some of the more major modeling
aspects. Though generally congruent with part I, we have made some minor
notational changes in this abbreviated description of the model. In
particular, we use $M$ to denote the number of nodes and $\rho(\cdot)$ to
denote the path loss function, which is now equivalently a function of
temporal displacement. Additionally, we use a slightly more verbose labeling
of angles and antenna gains. This is simply to help make the variables in the
implementation more distinct since there are more of them to keep track of.

A key feature of the DPCM is its ability to efficiently factor computations.
Consider the bistatic scattering response (or RCS)
$\sigma(\theta_{in},\theta_{out})$, where $\theta_{in}$ is the incoming angle
and $\theta_{out}$ is the outgoing angle. Here we have used the convention
that $\theta=(\varphi,\vartheta)$ is a spherical angle with $\varphi$ denoting
azimuth and $\vartheta$ denoting elevation. We assume that scattering response
admits a separable representation
$\sigma(\theta_{in},\theta_{out})=\alpha(\theta_{in})\beta(\theta_{out})$,
which ultimately forms the basis for the factorization.

Leveraging this separable model we can reduce computations in an $M\times M$
system as depicted in Figure 2(a). Each node receives $M-1$ inputs
$\\{s_{m}(t)\\}_{m=2}^{M}$ from adjacent nodes at input angles
$\\{\theta_{m}\\}_{m=2}^{M}$, where we have arbitrarily reserved $m=1$ to be
the node’s transmitted signal. These are then weighted by their respective
$\alpha(\theta_{m})$ and summed to form an intermediate signal

$\displaystyle v\left(t\right)=\
\sum_{m=2}^{M}{\alpha\left(\theta_{m}\right)s_{m}(t)}.$ (1)

This intermediate signal is then buffered in the node and used to generate
$M-1$ output signals, where the $i$th output signal is given by

$\displaystyle s_{i}^{\prime}(t)$ $\displaystyle=G_{T}(\theta_{i})\cdot
s_{1}(t-\tau_{i})+\beta(\theta_{i})\cdot v(t-\tau_{i})$ $\displaystyle
s_{i}(t)$ $\displaystyle=\rho(\tau_{i})\cdot e^{-j2\pi f_{i}t}\cdot
s_{i}^{\prime}(t).$ (2)

Here $G_{T}(\cdot)$ is the transmit antenna gain333In this paper we drop the
steering angle dependence of the transmit and receive gain functions for
brevity, since steering is not explicitly examined in the experiments.,
$f_{i}$ is the Doppler frequency, and $\tau_{i}$ is the direct path
propagation delay between nodes. During emulation, these parameters, and the
function values that depend on them, are updated every SCP. These output
signals are then communicated to all adjacent nodes in the network. A simple
three-node example of this communication process is presented in Figure 2(b).

In addition to the intermediate and output signals given by (1) and (III)
respectively, a received signal is generated via a separate path. Again we
take the $M-1$ input signals and form a weighted combination

$\displaystyle r(t)=\ \sum_{m=2}^{M}{G_{R}\left(\theta_{m}\right)s_{m}(t)}.$
(3)

where $G_{R}(\cdot)$ is the receive antenna gain. (3) represents the signal
seen by the receiver of an object if it possesses one444For the single
scatterer case examined in this paper, we assume the receiver is located at
the phase center of the object..

Figure 3: Accelerator architecture based on Direct Path Compute Model (a)
Architecture of passive node (b) Architecture of transmit Node (c) Doppler
Module

##  IV Accelerator Architecture

In order to faithfully implement the DPCM in hardware, the architecture of a
node must emulate several physical phenomena. These phenomena and an
accompanying high-level summary of our approach to implementing their
respective models are given in Table I. The architecture (Figure 3) consists
of the data path (containing all compute modules) and control path (FIFO for
data buffering according to direct path delay).

TABLE I: Phenomena modeled by each node and the approach taken to implement the model within our architecture. Phenomenon |  Approach
---|---
Direct path delay |  On-chip memory to buffer and source samples.
Doppler |  Narrowband Doppler with an update rate of 2 MHz. Resource sharing is used to reduce hardware costs.
Fractional delay filter |  4-tap filter running at 25% oversampling ($\sim 2.5-2.6$ GHz) applied to each object to consider fractional delay between objects’ relative phase centers (true broadband processing & oversampling reduces filter distortion with more accurate delay approximation).
Antenna gains |  Multiplication in source and/or receiver signals.
RCS & path loss |  Seperable RCS coefficients. Multipliers are used to apply RCS and path loss in architecture.

###  IV-A Data Path

Figure 3(a) shows the architecture of the hardware accelerator for a passive
reflector node. The RCS of an object is emulated using a point scatterer with
respective input and output dependent gains
$\\{\alpha(\theta_{in}),\beta(\theta_{out})\\}$[10]. Distance dependent direct
path loss $\rho(\tau_{i})$, angle-dependent receive $G_{R}(\theta_{in})$
antenna gains, and 4-tap FIR filter-based FDCs are used in emulation, all of
which are implemented using high throughput multiply and accumulate (MAC)
units. $\beta(\theta_{out})$ and $\rho(\tau_{i})$ are lumped into a single
coefficient $=\beta(\theta_{out})\times\rho(\tau_{i})$) so that only a single
multiply unit is needed.

At the final output response generated by an object, independent narrowband
Doppler correction is also used on each output to account for relative motions
of different objects (Figuure 3(c)). This is implemented using a high
throughput complex MAC unit. The complex Doppler correction coefficient is
updated at a reduced frequency of $2$ MHz. This allows a single Doppler
coefficient generation block to be shared across all outputs to minimize area
and power overhead, taking advantage of the fact that Doppler coefficients
need not be updated every cycle. Given an input Doppler frequency, a Finite
State Machine (FSM) sequentially generates the complex exponential for Doppler
correction for each output (in a round-robin fashion) using a ROM-based lookup
table (for optimal area/power) to generate the complex exponential. Each newly
generated coefficient is buffered until all of them have been generated.
Thereafter, all output Doppler coefficients are simultaneously updated once
every $256$ cycles, to allow a single cycle update of Doppler coefficients.
The latency of generating a single Doppler coefficient is $32$ cycles and
therefore this FSM unit can generate Doppler coefficients for up to $4$
separate outputs to maintain a Doppler update rate of $2$ MHz. For $>4$
outputs, either a second Doppler FSM unit will be required (to maintain the
same Doppler update rate) or the Doppler update rate must be reduced if a
single Doppler FSM unit is used.

The receiver associated with each node sums up the signals received at each
node after weighing them with the corresponding angle dependent receive gains
$G_{R}(\theta_{in})$.

Figure 4: Detailed architecture of sample distribution (a)Sample storage and
distribution by LDDC (b)Memory collision (c) Memory collision handling

###  IV-B Control Path

The main function of the control path is to buffer the scatterer response
$v(t)$ in SIMO-FIFO (Figure 3). For each output object at a path delay
$\tau_{i}$, SIMO-FIFO distributes the appropriately delayed $v(t-\tau_{i})$
signals. The architecture uses distributed, autonomous control to extract high
throughput by maintaining a single input (write) pointer and multiple parallel
output (read) pointers (one for each object). The control path contains a
Global Delay Distribution Controller (GDDC) and multiple Local Delay
Distribution Controllers (LDDC).

The path delays for objects are realized by controlling the difference between
(single) write and (multiple) read pointers. Given a physical distance $d$ to
be emulated for an RF channel with bandwidth $f$, the SCP determines “physical
delay” (in terms of samples) as $d\times f/c$, where $c$ is the speed of
light. The design has a known deterministic processing latency in the signal
emulation path. The global delay distribution controller (GDDC) computes
‘buffer delay’ i.e., the number of cycles a sample must be held in SIMO-FIFO,
as $\text{buffer delay }=d\times f/c-\text{compute latency}$ to match the
real-world propagation delay. The SIMO-FIFO is implemented as distributed SRAM
modules, driving a multiplexer-based data distribution network-on-chip (DDNoC)
to send RF samples to the compute engines. For every scenario, the GDDC parses
the SCP and configures distributed Local DDC (LDDC) present with each SRAM.
The Configuration Transfer NoC is used to transfer these configurations from
the GDDC to the corresponding LDDC. The LDDCs are distributed FSMs,
maintaining their own read (output) pointers in real-time (Figure 4(a) shows P
LDDCs generating M outputs). Considering the write pointer location as the
reference point (location of the RF node executing the computation), the
output pointer reads data from a location offset by this buffer delay. The
write pointer is incremented every cycle to store incoming streaming RF
samples in successive memory locations and the read pointers are incremented
simultaneously, keeping the delay difference constant (according to the direct
path delay of the objects in the system with respect to the computing node).
The sample to be distributed is first “buffered” in the SIMO-FIFO according to
the computed “buffer delay.” Samples are “virtually shifted” in the SRAM-based
SIMO-FIFO through the use of dynamic pointers which update their location
every cycle while the actual data remains “physically” fixed. This approach
allows potential savings in area/power through the use of SRAMs for sample
storage. The starting sample fetch location for a scenario is decoded based on
the buffer delay and the read (output) pointer is sequentially incremented in
subsequent cycles during a scenario. The incoming real-time samples are
written to the sub-banked SRAMs through throughput optimized on-chip H-tree
like network and the samples read by the active LDDCs are distributed to the
subsequent compute modules with a data distribution NoC (DDNoC) with P inputs
and M outputs. Once the last sample in an SRAM sub-bank has been accessed,
control is transferred within a cycle to the next logically neighboring LDDC
to start data access from its associated SRAM.

Figure 5: Illustration of data prefetch

###  IV-C Memory Collision

Memory read-read conflict (Memory collision) occurs when ranges of multiple
objects are closer than the number of samples stored in a single SRAM. Prior
work [6] discusses the possibility of using a single memory to implement tap
delays for multiple objects (for RF emulation with a TDL model) but incur
large logic usage in FPGA implementation to mitigate memory contention when
the sample to be read for two tap delays are within the same memory sub-bank.
The memory space representation of a system of 4 objects with memory collision
is shown in Figure 4(b), across two successive scenarios (Scen. N and Scen.
N+1). In a cycle, SRAMs allow reading from only one row. Considering 1 data
per row, for an SRAM storing S samples, this sets a lower bound on the range
difference between objects ($|\tau_{i}-\tau_{j}|>S$). $Memory\ collison$
occurs when this condition is violated and multiple samples need to be read
from an SRAM in a cycle.

We use architectural techniques to support memory collision handling by adding
real-time registers (RTR), prefetch buffers (PB), and processing engine
controllers (PEC) to our control path. For an $M+1$ object system, there are
$M$ RTRs and $M$ PBs in one node. The GDDC contains a period look-ahead and an
upcoming $memory\ collision$ in Scen. N+1 is known at the beginning of Scen.
N. “Colliding” objects are grouped together (starting from the nearest object)
within GDDC. Only the object with the smallest path delay in the group
(nearest object in the group), referred to as the “group header,” is parsed by
the GDDC to configure the corresponding LDDC for data fetch. However, this
data is multicast from the single LDDC to the RTRs (Figure 4(c)) corresponding
to each object in the group. The DDNoC supports arbitrary uni- and multi-
casting of multiple (equal to number of output objects) samples. Real-time
samples for subsequent compute modules are tapped from the RTRs (supporting
both read/write simultaneously). The group-header data ($d_{1},d_{2}$) are
read from the most recently updated locations in their RTRs. Read locations
for non-group header objects $\tau_{i}$ are offset from the most recent
location by $\tau_{i}-\tau_{j}$. Each PEC for an object is associated with an
RTR and PB. Similar to LDDCs, they are autonomous FSMs with their own read and
write pointers. Data multicast starts from $d_{2}$ in Scen. N+1 (Figure 4(c)),
whereas the 1st correct data for object $i$ in Scen. N+1 is $d_{n}$. Data
($d_{n}-d_{2}$) for object $i$ needs to be supplied separately. Since object
locations are static during a scenario and GDDC contains a scenario look-
ahead, data $d_{n}-d_{2}$ is $prefetched$ in Period N (Figure 5) and stored in
PBs present with each RTR. Based on the path delays of objects in a group, the
data to be prefetched for Scenario N+1 may or may not be available in the
SIMO-FIFO at the start of Scenario N. Figure 5 illustrates the two possible
prefetch conditions. When the path delay of the objects in a group $>$
scenario length, the data to be prefetched is already present in the sample
storage SRAMs at the beginning of Scenario N and the GDDC configures the
corresponding LDDC to fetch this prefetch data when it is idle. However, if
the path delay of the objects in a group $<$ scenario length, the data to be
prefetched will be streaming in during Scenario N. The GDDC tracks this and
directly sends this data to the corresponding PBs along with storing it in the
SIMO-FIFO.

Figure 6: Emulator Details (a) Scenario Programming (b) C++ cycle level
simulator platform Figure 7: Architecture verification with C++ simulator
(a)Impact of fractional delay connection (b)Large scale 16-object dynamic
scenario on the C++ simulator

###  IV-D Scenario Programming

The scenario programming takes a real-life radar scenario and computes model
parameters needed by the accelerator (Figure 6(a)). It contains two parts: the
frame generator and the control software. The frame generator extracts the
scenario information to create messages and frames. The messages provide
information, like the number of objects, about the scenario to set up the
control software for the frames. The frames describe each object’s properties
like position, velocity, acceleration, and quaternion orientation and they are
updated every millisecond. Both the messages and frames are sent to the
control software through a shared memory.

On the other end, the control software is a multi-threaded process that solves
respective equations to solve for parameters (Antenna gains, Doppler
frequency, propagation delays, etc.). It runs with 8-32 threads based on the
number of objects in the scenario with three threads to read frames and
messages from the shared memory and one main thread to control the execution
of the software. The rest of the threads are used to calculate the parameters
by distributing the calculations between the threads so a thread can calculate
the parameters for eight objects. Since scenario updates are applied every 1
ms, each frame’s equations get solved within $1$ ms by vectorizing the
equations’ implementation using Intel intrinsic, which are C-style functions
to access specific ISA instructions without writing assembly code. For
example, one thread can do the same add to all eight objects with one ISA
instruction. The output parameters corresponding to each frame (SCPs) are
transferred to the accelerator through a Scenario Programming Interface (SPI).

Within the accelerator, gains are double buffered and loaded with an
internally generated scenario update (SU) pulse, aligned to the negative edge
of global CLK to prevent stalling of RF emulation for configuring new
scenarios. The SCPs are double-buffered inside the GDDC and parsed a scenario
ahead to send configuration information to LDDCs.

##  V Verification of Proposed Architecture

Figure 8: RF Application Subsystem on test-chip for demonstration  TABLE II: Bit precision exploration of FDC coefficients with full-precision data. Bit-precision of coefficients (Floating point) |  Computed sample (Accuracy) |  Range resolution (Accuracy)
---|---|---
16-bit |  0.2075 (100%) |  0.0461 (100%)
14-bit |  0.2076 |  0.0462
12-bit |  0.2073 |  0.0462
11-bit |  0.2091 |  0.0435
10-bit |  0.2093 |  0.0445

###  V-A Cycle Level Simulator

We develop a sample and cycle-level simulator in C++ to evaluate the
performance of the proposed architecture in emulating RF interactions (Figure
6(b)). For a specific RF scenario, the channel generator generates “Scenario
File” containing the compute model parameters such as path gains, RCS, direct
path delay, etc. These files are used as input by the SPI of the C++ simulator
to send scenario-specific control information (SCPs) to the necessary modules.
Matched filtering of the output with the input chirp signal is used to
determine the range of the object. The I/Q streams from MATLAB computation and
C++ platform-based emulation are compared to validate the architecture.

###  V-B Testcases on cycle level simulator

####  V-B1 Testcase 1

This testcase demonstrates the importance of fractional delay filters proposed
in our architecture. At $2.5$ GHz operating frequency, the emulated range
resolution is $\approx 0.12m$ per sample. Since we update the direct path
delay coefficients after each frame, the emulation can not capture (without
introducing fractional delay) change in a range less than $0.12$ m per frame
($1$ ms in our case) which can be a problem, especially for a very slow-moving
object. Therefore, we consider a simple testcase with 2 objects where Object 1
is located at $596.012$ m distance away from the Tx (at origin) moving at a
velocity of $0.1$ m/s. The scenario update occurs after every $20$-pulse
repetition interval (PRI). It is evident that with fractional delay filters,
the estimated range resolution improves significantly (Figure 7(a)).

####  V-B2 Testcase 2

In this simulation we present a large $16$-object dynamic scenario with memory
collision to demonstrate the handling of complex RF scenarios with the
proposed architecture (Figure 7(b)). Object 1 is located at the origin and the
range estimation of Object 2 to Object 16 is carried out over a series of
scenarios. Objects 2 to Object 16 are located at intervals of $50$ m starting
from a distance of $2.2$ km from Object 1 (Scenario 1). This is a testcase
with memory collision since the distance between objects is $50$ m which
translates to a $\sim 400$ sample delay (The number of samples stored in each
SRAM was set at 1024). Object 3 and Object 15 have relative motion compared to
Object 1. Object 3 starts from the range of $2.25$ km and moves away from
Object 1 whereas Object 15 starts from $2.85$ km away from Object 1 and
gradually moves towards it. The results show the estimated ranges across
scenarios and it clearly shows that the ranges of all objects have been
detected in the presence of memory collision and that objects 3 and 15 are
moving. Additionally, the amplitude of the detected peaks get smaller as the
range of the detected object increases, indicating the modeling of physical
phenomena by the simulated architecture.

##  VI Test-chip Implementation

Figure 9: Testchip implementation details of Receiver  Figure 10: Prototype
summary (a)Layout of a single passive node with the layout of LDDC+SRAM
(b)Experimental Setup (c)Die-shot and testchip summary

A prototype emulator is fabricated in $28$ nm CMOS with Synthesis and
Place/Route tools. We choose a real-time RF system with 4 interacting nodes as
an application to demonstrate the distributed control (Figure 8). The system
contains a Transmitter, 2 reflectors (Objects 1, 2) and a Receiver. The Tx
generates high throughput digital samples which are reflected at/between
passive objects and collected at the Rx.

###  VI-A Implementation Details of Data Path

Coefficients $\alpha(\theta_{in})$ and $\beta(\theta_{out})$ are implemented
using 16-bit floating point multipliers (both data and coefficients are
16-bits). Each complex Doppler coefficient is also implemented using 16-bit
precision MAC units to preserve accuracy as the impact of these MAC units on
overall area and power is minimal due to their small number per node
($2\times$ the number of outputs for complex coefficients). However, due to
the large number of total MAC units required for FDC ($8\times$ the number of
inputs/outputs), these 4-tap FIR filter-based FDCs are implemented as a mixed
precision MAC unit with reduced precision (5-bit exponent + 4-bit mantissa +
1-bit sign = 10-bits) FIR coefficients and 16-bit data for reduced area/power
with minimal impact on accuracy. This mixed precision MAC unit provides a
$20\%$ and $25\%$ reduction in power and area respectively compared to a
16-bit coefficient MAC unit while incurring a $0.87\%$ reduction in accuracy
of computed sample and $3.47\%$ reduction in range resolution (Table II). For
a transmitting node, the first part of the data path (until the generation of
intermediate signal $v(t)$) is replaced by a local RF source. The digital RF
generator (DRFG) is emulated using a configurable FSM, generating digitized
32-bit I/Q (16-bit I + 16-bit Q) RF data (Figure 3(b)). The digital output I/Q
stream is a periodic signal with programmable time period, duty cycle, and I/Q
values that provides flexibility in generating a large variety of RF input
signals. The angle-dependent transmit gain $G_{T}(\theta_{out})$ is accounted
for with the help of 16-bit MAC units in the transmit path.

The receiver (Figure 9) performs weighted accumulation of data from the
Transmit node and the 2 passive nodes. The receiver gains are applied to each
input using real-valued 16-bit coefficient MAC units before being combined in
an adder tree. For the prototype test-chip implementation, a high-bandwidth
I/O interface for real-time readout of the receiver output at operating
frequency was not implemented. Instead, the final samples generated at the
output of the adder tree, are sequentially stored in a local SRAM for the
duration of the emulation. Once the emulation is completed, the stored
receiver samples are read out from the SRAM through a low bandwidth on-chip
serial to parallel Interface.

###  VI-B Implementation Details of Control Path

We use a small-scale design of the proposed near-memory distributed control.
Synthesis and automatic place & route tools are used to implement the
distributed control with SRAM sub-banking to simultaneously optimize area and
performance (while avoiding register-based implementation). This prototype
emulator is implemented with the storage of 16 kilo-samples per node (16 SRAMs
per node, 1024 samples in each foundry-provided SRAM sub-bank) for
demonstration. Each sample is 32-bit wide. Memory collision is handled by the
incorporation of RTR/PB in the distributed control. The RTR/PB each needs to
store 1024 samples to provide collision support across a sample span of 1024
(full support). Similar to sample storage in baseline distributed control, the
RTR is implemented with SRAMs, but with dual ports (DP) to facilitate the
Read/Write in the same cycle. DP-SRAMs are significantly slower and larger (1
kB memory is $2\times$ larger, $23\%$ slower) than single port SRAMs. We
choose DP-SRAMs with 256 samples (1KB) each to simultaneously optimize
area/frequency in this test-chip due to the constraints on the available chip
area. This restricts the memory collision protection range to 256 samples (1
kB DP-SRAM) instead of the full 1024. Parallel loading of multiple prefetch
data is not possible in a single cycle in DP-SRAM-based RTR from PB at the end
of a Scenario. So, we implement PBs with DP-SRAM too and the roles of RTR and
PB are interchanged in every scenario (virtual parallel loading of prefetch
data) by PEC logic. Each PE controller is associated with a single 1KB DP-SRAM
for RTR and another single 1KB DP-SRAM for PB. The PEC FSM is implemented to
increment both input/output pointers every cycle. The addition of DP-
SRAMs/PECs causes a significant overhead in the area/power of the control
path.

From Figure 3(b), the SIMO-FIFO-based sample distribution has the same
requirement in a transmitting vs a passive node, with the key difference being
that in the Transmit node, the sample from the DRFG is buffered instead of the
intermediate signal $v(t)$. Additionally, with the choice of prototype RF
subsystem, the Tx sends samples to 3 other nodes while the two passive
reflectors send samples to 2 other nodes. We use the same SIMO-FIFO in all 3
nodes, with passive nodes configured to generate only 2 output samples (1 RTR
+ PB + PEC will be idle).

The LDDC+SRAM is designed together and used as hard modules in the design of
control path. The rest of the control path is created together along with hard
modules for the output Fractional Delay correction and Doppler modules to
allow the tool maximum flexibility in area/performance optimization.
Neighboring LDDC+SRAMs are placed close together (in a ring-like arrangement
where the first SRAM and last SRAM sub-bank being logically adjacent are
placed next to each other) to facilitate single cycle local transfer of
configurations once each LDDC has read the last sample from its associated
SRAM.

Figure 10(a) shows the layout of a single passive node with the placement of
the 16 LDDC + SRAM, DP-SRAMs, FDC and Doppler highlighted. The layout of an
individual LDDC + SRAM combination is also highlighted in Figure 10(a). The
area occupied by a single LDDC + SRAM is $0.014$ $mm^{2}$ with the 4 kB SRAM
occupying $56\%$ of the total area. Roughly, the area occupied by the GDDC and
DP-SRAMs is around $0.2$ $mm^{2}$ (area of dual port SRAMs = $0.04$ $mm^{2}$
). The total area occupied by a single passive node is about $1.098$ $mm^{2}$.

Figure 11: Results of prototype implementation (a)Area and Simulated power
breakup (b)Results of Experiment 1 on test-chip Figure 12: Results of
experiment 2 on test-chip

###  VI-C Measurement Results

Figure 10(b) shows the experiment setup. A Printed Circuit Board (PCB) is used
to apply the stimuli to the packaged testchip. The SCPs are generated in a CPU
and loaded into the chip through the SPI using a $0.5$ kbit/s bit-serial
interface. This serial interface is used for both loading of scenario
configurations and reading output data from the chip. The receiver SRAM-based
storage circuit on the receiver stores sub-samples of the real-time outputs
on-chip. The experiment is repeated multiple times to gather all subsamples
required to fully reconstruct the output (offline). This reconstructed output
is further post-processed (matched filtered with the input signal) to detect
the peaks indicating ranges of objects relative to Rx.

Figure 10(c) shows the die-shot and test-chip summary. The design achieves up
to $518$ MHz of per-channel bandwidth versus $100$ MHz in Colosseum, the
state-of-the-art RF emulator [1, 3] and $180$ MHz in [5]. At $518$ MHz, the
maximum emulation distance is $9.5$ km (16K samples) with 64 kB SRAM/node and
compute latency of a path is $0.24$ $\mu$s. Since the sample storage is
implemented with SRAMs, it can only support either read or write in a cycle.
The minimum emulation range(sample delay) between two objects is 0.67 km @518
MHz bandwidth (1024 samples in each SRAM plus compute latency for the path).

Figure 11(a) shows the area and simulated power breakup of a single node in
the test-chip. The control path dominates the total area but requires less
power than compute modules. The addition of memory collision support increases
area and simulated power by $28\%$ and $11.8\%$ respectively.

For Experiment 1 (Figure 11(b)), we consider a direct path between the Tx and
the Rx with passive objects disabled. Range estimation is performed by matched
filtering of samples received at the Rx with the transmitted signal. The
measured results show $<$ 1 % error in range estimation. The appearance of 1st
non-zero pulse denotes the range of the Tx from the Rx. We vary the frequency
of operation and take measurement results at $260$ MHz and $518$ MHz. Wider
detection pulses are observed at higher frequency potentially due to increased
supply noise leading to bit errors at the output.

In the 4-node dynamic experiment (Experiment 2, Figure 12), we determine
ranges of multiple objects interacting in the RF system in the presence of
memory collision. Object 1 and the Rx are in memory collision for sample
distribution in the Tx node. The range of each object is estimated based on
the appearance of new pulses in the matched filtering output. The test-chip is
able to separately detect two nodes in memory collision as shown in Figure 12.
The measurement results also demonstrate the impact of reflection between
passive objects (new pulses appearing at 11000 sample delay, Tx–Obj1–Obj2–Rx
path delay). Further, this experiment also demonstrates a scenario update, by
updating the RCS of passive object 2 in the second scenario (scenario length
is much smaller than 1ms in this prototype implementation). The measured
emulated ranges show $0.1-1.1\%$ error. For these experiments, the
coefficients for the fractional delay filters were set to 0 for the positive
and negative time lags and to non-zero for the 0-time lag. Additionally, the
Doppler frequency was set to 0.

##  VII FPGA Implementation

The above discussion has showcased the capabilities of our DPCM emulator
through ASIC-based tests. In this section, we develop this methodology further
to explore more complex scenarios involving a greater number of objects.
Utilizing the Xilinx ZCU104 FPGA board, we can enhance our DPCM emulator to
simulate the aforementioned larger-scale scenarios, bolstering our claim that
the model has promising scalability. In this new paradigm, we emulate a
dynamic test environment in a 9-node configuration, which includes 2
Transmitters, 6 Passive interacting reflectors (Object 1 to 6), and 1 Receiver
as shown in Figure 13.

###  VII-A Implementation Details of Data Path

Our FPGA-based emulator employs the same data path implementation as the chip-
test design, as depicted in Figure 3. Each TX consists of a 32-bit Digital
Signal Generator (16-bit real and 16-bit imaginary), configurable to generate
periodic, digitized RF I/Q samples with a maximum periodicity of 2048 samples.
The duty cycle can also be programmed from 3.125% to 100% and each non-zero
sample can be independently programmed to allow a wide variety of RF patterns
to be generated. Then, signals emitted by the TXs are conveyed to the 6
passive objects, where they undergo filtering, attenuation, and delay to
generate the reflected signal for each object. The receiver then aggregates
these reflected signals from all passive objects, enhances them with a
programmable gain, and subsequently, the consolidated output is extracted from
the FPGA. The memory used for the receiver is Ultra_RAM [11] memory provided
by the ZCU104 board, and was chosen for data read/write transactions in the
receiver. This type of memory supports cascading with an optimized pipeline to
enhance timing and reduce latency. However, given the Ultra_RAM’s inability to
support a read-only memory (ROM) configuration, ROM is implemented via a Block
RAM acting as the lookup table for the Doppler generator.

Figure 13: The design diagram of the 9-node test case with the signal
generator on FPGA. Figure 14: The Experiment Setup on Xilinx ZCU104 Board.

###  VII-B Implementation Details of Control Path

In the FPGA-based design, we employ two distinct variants of Ultra_RAM
memories: Single-port Ultra_RAM (SURAM) and Dual-port Ultra_RAM (DURAM), both
configured with a depth of 512 samples and a data width of 32 bits. since the
memory depth was updated to 512 samples, this necessitated adjustments to the
memory collision support, which now offers full support at the updated sample
depth. To facilitate simultaneous read/write operations within the same cycle,
the RTR leverages DURAM. However, given the finite availability of Ultra_RAM
resources in the ZCU104 board, we have substituted some DURAMs with Dual-port
Block RAM (BRAM) to maintain constant performance. Additionally, to meet the
high-speed performance requirements and adhere to timing constraints, we have
integrated 4 pipeline stages into the critical path of the emulation system’s
design.

Figure 15: (a) The experiment and setup of the 9-node test case for the static
scenario. (b) The experiment and setup of the 9-node test case for the dynamic
scenario. Figure 16: (a) The experiment result and setup of the multiple TX
Test. (b) The error estimation between expected emulation result and measured
emulation result: (I) The 9-node test case. (II) The multiple_tx test case.
Figure 17: (a) The experiment result and setup of the Minimum Latency Test
with 1024 memory bank. (b) The experiment result and setup of the Minimum
Latency Test with 512 memory bank.

###  VII-C FPGA-based RF Emulator Performace Demonstration

The measurement setup of the FPGA-based emulator is illustrated as shown in
Figure 14. A MATLAB-based channel generator is used which, given specific
scenario parameters such as object location, distance, RCS, velocity, etc. can
generate a configuration packet that is first uploaded from the PC to the FPGA
via the IP address of the ZCU104 board, and then is programmed onto the FPGA
by reading it through the Advanced Extensible Interface-General Purpose
Input/Output (AXI-GPIO) interface. This configuration packet consists of RCS
tables, Delay matrices, FIR filter coefficients, and Doppler frequency tables
for each respective object to emulate various RF scenarios. The measured
output is collected from Xilinx Hardware Management via another AXI-GPIO
interface. Finally, we compute the propagation delay of the measured output
with respect to the input signal using the MATLAB-based Matched Filter. The
performance of the system is evaluated by the accuracy (MSE) of emulation
between the expected distance values ($\hat{Y_{i}}$) and the measured emulated
distance results ($Y_{i}$) from:

$\text{MSE}=\frac{1}{n}\sum_{i=1}^{n}(Y_{i}-\hat{Y_{i}})^{2}$ (4)

Several test cases are demonstrated, and details of those are articulated
below and all test-cases were run at 215MHz of per-channel bandwidth.

####  VII-C1 Emulation of Range Estimation

To clearly demonstrate the emulator performance, We first construct a
simplified version of the 9-node system to verify the accuracy of range
emulation. We activate a single TX and maintain all nodes in a stationary
state, simulating only a single reflection per signal. We proceed with the
assumption that the passive objects do not reflect signals amongst themselves,
effectively disregarding any reflections occurring between objects OBJ1
through OBJ6. Each of the four components is positioned 3 km to 22.7 km from
the transmitter, as shown in Figure 15(a). The RX, located 3 km away from each
object, accumulates the reflected signals that are sent out of the FPGA. We
only enable the RCS to [1, 0, 0, 0, 0, 0] to make sure no reflection signal is
getting from other objects. Under this condition, the estimated range of these
components exhibits a minor deviation of 0.25% (Eq.4) when compared to their
actual range.

The second test, shown in Figure 15(b), has the exact initial locations as the
first test, except we enable “inter-object interaction” (IOI). That is, the
signal reflected from one object is again reflected by other objects,
resulting in a feedback loop (The value of the FIR coefficient remains the
same, but the RCS changes from [1, 0, 0, 0, 0, 0] to [1, 1, 1, 1, 1, 1]).
Additionally, Object 1 moves toward the other 5 objects with the mutual
Doppler effect taken into account. Figure 15(b) shows the measured result of
the IOI scenario, where the signal interaction becomes visible after the
location is shown in a red dashed line(“Signal Interaction” Occurs). Compared
with static cases, under more complex scenarios, each range estimation from
the emulation result has a <2.22% error, as indicated by part in Figure
16(a.I), compared to the actual range with result illustrated.

The Third test is the emulator with multiple TXs, which is displayed in Figure
16(a). Two TXs are applied, and those transfer the RF data to OBJ1 and OBJ2
respectively. Six objects are reflecting the signal within the emulation
system, ranging from 6.66 km to 22.65 km. The same as the 9-node test case,
the RX keeps at 3 km away from each object. For this test, the final emulation
result after the matched filter is less than 2.51% compared to the actual
range (part in Figure 16(b.II)).

Figure 18: The experiment result and setup of the Fraction Delay Correction.
Figure 19: (a) Power consumption for each component of the 6-node test case.
(b) Power consumption for each component of the 9-node test case.  Figure 20:
Overall Power Consumption on ZCU104 Board: (a) 6-node test case with
Ultra_RAM. (b) 6-node test case with BRAM only. (c) 9-node test case.

####  VII-C2 Minimum Latency Test

When two objects are too close to each other, the emulator cannot detect the
actual range between them because there is a minimum amount of time required
by the system to process the collected data (minimum latency of the hardware).
Therefore, the test of minimum latency is equivalent to examining the minimum
sample range that allows the emulator system to work normally, which is
determined by the size of the sub-banks (where simultaneous read and write
operations are not possible) and the hardware latency. For this test, we
enable only two nodes of the system, TX and Object 1. The TX is located at the
center of the origin, and Object 1 is initially located 7.994 km away from the
TX. Object 1 begins moving towards the TX in the horizontal direction while TX
remains stationary. As shown in Figure 17(a), when Object 1 moves to a
location within 1.61 km of TX, the signals completely disappear. The “no
signal” status is maintained while Object 1 is within this distance, but as
Object 1 moves further than 1.61 km away from TX, the emulator system returns
to normal operation. Thus, a minimum range of 1.61 km is required for the
system to operate normally. This value is approximately 2.4 times larger than
that observed in chip-test case one. The primary reason for this difference,
as detailed in Section IV, is related to the formula for buffer delay, which
is defined as $\text{buffer delay}=d\times f/c$. Consequently, the actual
distance can be calculated as $\text{buffer delay}\times c/f$. The discrepancy
between the two test results stems from variations in the frequency ($f$), or
bandwidth, used in each test, which ultimately influences the capabilities of
the system.

We repeat the aforementioned experiment using a smaller memory depth of 512
samples. From Figure 17(b), the emulator only operates abnormally when Object
1 is located within 0.94 km. It is evident that employing an emulator with
lower memory depth effectively reduces the window of read-write collision,
resulting in smaller system latency, enhancing (e.g. reducing) the minimum
emulation range.

TABLE III: The power, MAX IBW, and utilization of design cells based on 6-node test case and 9-node test case. Component Name |  6-node test case |  9-node test case
---|---|---
LUT | 181541 | 395450
LUTRAM | 9399 | 23081
FF | 243201 | 524938
BRAM | 91.5 | 259
URAM | 95 | 96
BUFG | 24 | 24
MAX IBW | 215 MHz | 215 MHz
Range Estimation | 1.89 km-27.3 km | 1.13 km to 27.3 km
Power On Chip | 37.93 mW | 135.7 mW
Power/100 MHz | 5.58 W | 10.9 W
TABLE IV: The power, MAX IBW and utilization of design cells based on the 6-node test case. Component Name |  UltraRAM |  BRAM only
---|---|---
LUT | 181541 | 180657
LUTRAM | 9399 | 9437
FF | 243201 | 249804
BRAM | 91.5 | 215.5
URAM | 95 | 0
BUFG | 24 | 29
MAX IBW | 215 MHz | 120 MHz
Power On Chip | 37.93 mW | 99.5 mW
Power/100 MHz | 5.58 W | 5.73 W
TABLE V: Comparison of our design with prior work. Attributes |  This Work |  D. Barcklow[1] |  Ashish. C [12] |  I. Val [13] |  kevin. C[5]
---|---|---|---|---|---
Year | Current | 2019 | 2018 | 2014 | 2009
Computation Model | DPCM | TDL | TDL | TDL | TDL
Channel Tap | 4 taps | 4 taps | 5 taps | - | 3 taps
Max Bandwidth | 215 MHz | 80 MHz | 80 MHz | 100 MHz | 90 MHz

####  VII-C3 Fraction Delay Correction

In order to highlight the impact of the FDC, we implement a test scenario
involving a single object. This object is positioned 5km away from the TX,
with the RX set at an equivalent distance from the object (total distance of
10 km from the TX to RX). We measure the emulation result in two distinct
conditions: first, with FDC enabled, and second, with FDC disabled. According
to Figure 18, the measured distance without FDC (indicated by the Blue Dash
Line) after applying the matched filter is 10.04 km.In contrast, when FDC is
enabled in the same scenario, the emulation is able to measure a range of
10.0283 km (Red Solid Line). showing the improved range emulation achieved
with FDC.

###  VII-D FPGA Hardware Performance

To further demonstrate the hardware capabilities of our FPGA-based RF
emulator, we have designed an additional test case. Specifically, we
demonstrate the hardware performance in two scenarios: 1) a 6-node test case
featuring a memory depth of 1024, including 1 TX, 4 passive interacting
reflectors, and 1 RX; and 2) our illustrated 9-node test case, which extends
our evaluation to cover a broader range of interactions and complexities.

####  VII-D1 Emulation Time Comparison

The total time required to run simulations for the 6-node test case, involving
four scenarios and each with an 8176-sample test case, is only 0.3612 ms. This
is significantly faster compared to the C++ based (10.2521s) or MATLAB (5.2
minutes) based simulator of the physical model. The proposed FPGA system,
operating at 215 MHz, significantly accelerates the processing speed and is
28,380$\times$ faster than the C++ Emulator and 82,570$\times$ faster than
MATLAB. When the same conditions are applied to the 9-node test case design,
the total simulation time is just 0.375ms, while the MATLAB-based simulator
needs to take 5.2 minutes (8.32x$10^{5}$ times longer). As expected, the
advantage of emulation is more prominent for test cases with higher complexity
such as more objects.

####  VII-D2 Power Analysis

Figure 19(a) and 19(b) show the total power consumption from Vivado Simulation
for both designs on ZCU104 board, which are 7.362 W for the 6-node test case
(6.631 W dynamic power consumption and 0.731 W static power consumption) and
16.375W for the 9-node test case design (14.973 W from dynamic power
consumption and 1.402 W static power consumption). In Figure 20, we present a
graphical representation of the actual algorithm power consumption recorded
from the ZCU104 board for each test case, with a 12 V voltage applied to the
power supply. This measurement is conducted using the PMbus library from
Python Productivity for Zynq (PYNQ) to monitor the overall power consumption
of the ZCU104 board before and after the commencement of the test.
Consequently, the algorithm’s power consumption from the design is derived as
the difference between these two power readings. Figure 20(a) illustrates the
power consumption of the 6-node test case when employing fully the Ultra_RAM
for the design. The graph depicts power measurements taken both before and
after system initialization, with values recorded as 12.33 W and 13.37 W,
respectively. As a result, the algorithm’s power consumption for this
particular case is calculated as the difference between these values, equating
to 37.93 mW. Similarly, Figure 20(b) provides the simulation result of the
algorithm’s power consumption for the same design, albeit with a transition
from Ultra_RAM to BRAM memory, which is roughly 2.6 times larger in amount.
Under this scenario, the algorithm’s power consumption registers at 99.5 mW.
Figure 20(c) offers the power analysis for a 9-node test case (135.7 mW).
Despite the memory size expanding by a factor of 1.8 when transitioning from 6
nodes to 9 nodes, the actual power consumption increases by approximately 3.6
times due to the substitution of certain memory elements from Ultra_RAM to
BRAM.

####  VII-D3 Design Component and Bandwidth Analysis

Table III presents the component utilization and total power consumption for
both systems. The inclusion of two additional nodes results in a noticeable
increase in the memory consumption of the 9-node test case design,
approximately three times larger compared to the 6-node test case design. At
the same time, the minimum emulation range of the 9-node test case design is
broader compared with the 6-node one. Both test cases can achieve a maximum
emulated IBW above 215 MHz, which is determined by the max frequency of
operation of FPGA. We model the same design by using only BRAM as the memory
component without Ultra_RAM for the 6-node test case. Compared with the
Ultra_RAM version, the IBW is limited to 120MHz. The BRAM-based design also
has a higher total power consumption and utilization of design components,
which are displayed in Table IV.

####  VII-D4 Comparison with Prior Works

Table V provides a comparative analysis of our design with other state-of-the-
art solutions from the past decade. In contrast to existing designs utilizing
TDL computation mode, our work is grounded entirely in the DPCM computation
model. Notably, we have the capacity to optimize our RF emulation system’s
maximum bandwidth by employing a 4-tap FIR filter, achieving a remarkable 215
MHz. This figure represents a significant 2.15$\times$ increase over the
capabilities of the current single FPGA-based designs.

##  VIII Conclusion

In part I of this series we developed a new “direct path” computational model
for real-time digital RF emulation. We showed, through careful mathematical
formulation and simulation, that this model can suitably emulate all channel
characteristics necessary for RF system testing. Furthermore, by leveraging
mild assumptions on the physical characteristics of scattering profiles and
antenna structures our model was shown to yield tremendous computational
benefits. This, coupled with a naturally distributed framework, motivated us
to explore hardware implementations of the model.

Part II of this series focused on the development of working implementations
of the direct path computational model. We approached this from two design
perspectives, the first being an ASIC that leverages near-memory computations
and autonomous distributed control. This allowed us to achieve high bandwidth
and low-latency performance that is not viable in off-the-shelf component-
based systems. The second design used an FPGA to implement a larger scale
(e.g. a greater number of objects) system. Though this style of implementation
operates at a lower bandwidth than the ASIC version, it demonstrates that the
model can be viably scaled to incorporate more objects as needed.

The complementary results of these two papers have, together, established a
new and interesting option for consideration in the future development of RF
emulators. Through our collaborative effort, we have coupled innovative
modeling techniques with cutting-edge high-performance computing paradigms.
The result is a series of implementations that operate at a level of
performance not achievable by more traditional designs and a model that can be
efficiently scaled to meet testing requirements.

## References

  * [1] D. Barcklow, L. Bloch, S. Sweeney, B. Ahr, W. La Cholter, S. Berhanu, H. Bisyir, J. Cook, and D. Coleman, “Radio frequency emulation system for the defense advanced research projects agency spectrum collaboration challenge,” _Johns Hopkins APL Techn. Digest_ , vol. 35, no. 1, pp. 69–78, 2019.
  * [2] J. Matai, P. Meng, L. Wu, B. Weals, and R. Kastner, “Designing a hardware in the loop wireless digital channel emulator for software defined radio,” in _IEEE Int. Conf. on Field-Program. Technol_ , December 2012.
  * [3] L. Bonati, P. Johari, M. Polese, S. D’Oro, S. Mohanti, M. Tehrani-Moayyed, D. Villa, S. Shrivastava, C. Tassie, K. Yoder, A. Bagga, P. Patel, V. Petkov, M. Seltser, F. Restuccia, A. Gosain, K. Chowdhury, S. Basagni, and T. Melodia, “Colosseum: Large-Scale Wireless Experimentation Through Hardware-in-the-Loop Network Emulation,” in _IEEE Int. Symp. on Dyn. Spectr. Access Netw._ , December 2021.
  * [4] M. Wickert and J. Papenfuss, “Implementation of a real-time frequency-selective RF channel simulator using a hybrid DSP-FPGA architecture,” vol. 49, no. 8, 2001, pp. 1390–1397.
  * [5] K. Borries, G. Judd, D. Stancil, and P. Steenkiste, “FPGA-Based Channel Simulator for a Wireless Network Emulator,” in _IEEE Veh. Technol. Conf._ , September 2009.
  * [6] S. Buscemi and R. Sass, “Design of a scalable digital Wireless Channel Emulator for networking radios,” in _IEEE Mil. Commun. Conf._ , November 2011.
  * [7] J. Seo, M. Mukherjee, N. M. Rahman, J. Tong, C. DeLude, T. Krishna, J. Romberg, and S. Mukhopadhyay, “A Configurable Architecture for Efficient Sparse FIR Computation in Real-time Radio Frequency Systems,” in _IEEE Int. Microw. Symp._ , June 2022.
  * [8] M. Mukherjee, N. M. Rahman, C. DeLude, J. Driscoll, U. Kamal, J. Woo, J. Seo, S. Sharma, X. Mao, P. Behnam, S. Khan, D. Kim, J. Tong, P. Sinha, S. Pande, T. Krishna, J. Romberg, M. Swaminathan, and S. Mukhopadhyay, “A High Performance Computing Architecture for Real-Time Digital Emulation of RF Interactions,” in _IEEE Radar Conf._ , May 2023.
  * [9] X. Mao, M. Mukherjee, N. M. Rahman, U. Kamal, S. Sharma, P. Behnam, J. Tong, J. Driscoll, T. Krishna, J. Romberg, and S. Mukhopadhyay, “FPGA-Based High-Performance Real-Time Emulation of Radar System Using Direct Path Compute Model,” in _IEEE Int. Microw. Symp._ , June 2023.
  * [10] E. Huang, C. DeLude, J. Romberg, S. Mukhopadhyay, and M. Swaminathan, “Anisotropic Scatterer Models for Representing RCS of Complex Objects,” in _IEEE Radar Conf._ , May 2021.
  * [11] Xilinx, “Ultraram: Breakthrough Embedded Memory Integration on Ultrascale+ Devices,” 2016.
  * [12] A. Chaudhari and M. Braun, “A Scalable FPGA Architecture for Flexible, Large-Scale, Real-Time RF Channel Emulation,” in _IEEE Int. Symp. on Reconf. Commun. Sys.-on-Chip_ , July 2018.
  * [13] I. Val, F. Casado, P. M. Rodriguez, and A. Arriola, “FPGA-based wideband channel emulator for evaluation of Wireless Sensor Networks in industrial environments,” in _IEEE Emerg. Technol. and Factory Autom._ , September 2014.


* Mireshghallah et al. (2022) Fatemehsadat Mireshghallah, Kartik Goyal, and Taylor Berg-Kirkpatrick. 2022.  Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Association for Computational Linguistics, Dublin, Ireland, 401–415.  https://doi.org/10.18653/v1/2022.acl-long.31
  * Mitra et al. (2015) Tanushree Mitra, C.J. Hutto, and Eric Gilbert. 2015\.  Comparing Person- and Process-Centric Strategies for Obtaining Quality Data on Amazon Mechanical Turk. In _Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems_ (Seoul, Republic of Korea) _(CHI ’15)_. Association for Computing Machinery, New York, NY, USA, 1345–1354.  https://doi.org/10.1145/2702123.2702553
  * Nan et al. (2021) Linyong Nan, Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, Yangxiaokang Liu, Nadia Irwanto, Jessica Pan, Faiaz Rahman, Ahmad Zaidi, Mutethia Mutuma, Yasin Tarabar, Ankit Gupta, Tao Yu, Yi Chern Tan, Xi Victoria Lin, Caiming Xiong, Richard Socher, and Nazneen Fatema Rajani. 2021.  DART: Open-Domain Structured Data Record to Text Generation. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics, Online, 432–447.  https://doi.org/10.18653/v1/2021.naacl-main.37
  * Niu and Bansal (2018) Tong Niu and Mohit Bansal. 2018.  Polite Dialogue Generation Without Parallel Data.  _Transactions of the Association for Computational Linguistics_ 6 (2018), 373–389.  https://transacl.org/ojs/index.php/tacl/article/view/1424
  * Novikova et al. (2018) Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. 2018.  RankME: Reliable Human Ratings for Natural Language Generation. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_. Association for Computational Linguistics, New Orleans, Louisiana, 72–78.  https://doi.org/10.18653/v1/N18-2012
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002\.  Bleu: a Method for Automatic Evaluation of Machine Translation. In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, 311–318.  https://doi.org/10.3115/1073083.1073135
  * Pascual et al. (2021) Damian Pascual, Beni Egressy, Clara Meister, Ryan Cotterell, and Roger Wattenhofer. 2021.  A Plug-and-Play Method for Controlled Text Generation. In _Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021_ , Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, 3973–3997.  https://aclanthology.org/2021.findings-emnlp.334
  * Peng et al. (2020) Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng, and Jianfeng Gao. 2020.  Few-shot Natural Language Generation for Task-Oriented Dialog. In _Findings of the Association for Computational Linguistics: EMNLP 2020_. Association for Computational Linguistics, Online, 172–182.  https://doi.org/10.18653/v1/2020.findings-emnlp.17
  * Peng et al. (2018) Nanyun Peng, Marjan Ghazvininejad, Jonathan May, and Kevin Knight. 2018.  Towards Controllable Story Generation. In _Proceedings of the First Workshop on Storytelling_. Association for Computational Linguistics, New Orleans, Louisiana, 43–49.  https://doi.org/10.18653/v1/W18-1505
  * Peters et al. (2018) Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018.  Deep Contextualized Word Representations. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_. Association for Computational Linguistics, New Orleans, Louisiana, 2227–2237.  https://doi.org/10.18653/v1/N18-1202
  * Pillutla et al. (2021) Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Yejin Choi, and Zaïd Harchaoui. 2021.  MAUVE: Human-Machine Divergence Curves for Evaluating Open-Ended Text Generation.  _CoRR_ abs/2102.01454 (2021).  arXiv:2102.01454 https://arxiv.org/abs/2102.01454
  * Prabhumoye et al. (2020) Shrimai Prabhumoye, Alan W Black, and Ruslan Salakhutdinov. 2020\.  Exploring Controllable Text Generation Techniques. In _Proceedings of the 28th International Conference on Computational Linguistics_. International Committee on Computational Linguistics, Barcelona, Spain (Online), 1–14.  https://doi.org/10.18653/v1/2020.coling-main.1
  * Pryzant et al. (2020) Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2020.  Automatically neutralizing subjective bias in text. In _Proceedings of the AAAI conference on artificial intelligence_ , Vol. 34. 480–489.
  * Puduppully et al. (2019) Ratish Puduppully, Li Dong, and Mirella Lapata. 2019\.  Data-to-text generation with content selection and planning. In _Proceedings of the AAAI conference on artificial intelligence_ , Vol. 33. 6908–6915.
  * Qian et al. (2022) Jing Qian, Li Dong, Yelong Shen, Furu Wei, and Weizhu Chen. 2022.  Controllable Natural Language Generation with Contrastive Prefixes. In _Findings of the Association for Computational Linguistics: ACL 2022_. Association for Computational Linguistics, Dublin, Ireland, 2912–2924.  https://doi.org/10.18653/v1/2022.findings-acl.229
  * Qian et al. (2019) Yusu Qian, Urwa Muaz, Ben Zhang, and Jae Won Hyun. 2019\.  Reducing gender bias in word-level language models with a gender-equalizing loss function.  _arXiv preprint arXiv:1905.12801_ (2019).
  * Qin et al. (2019) Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, and Jianfeng Gao. 2019\.  Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, 5427–5436.  https://www.aclweb.org/anthology/P19-1539/
  * Qin et al. (2022) Lianhui Qin, Sean Welleck, Daniel Khashabi, and Yejin Choi. 2022.  COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics.  https://doi.org/10.48550/ARXIV.2202.11705
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019\.  Language models are unsupervised multitask learners.  _OpenAI blog_ 1, 8 (2019), 9.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.  _Journal of Machine Learning Research_ 21, 140 (2020), 1–67.  http://jmlr.org/papers/v21/20-074.html
  * Rashkin et al. (2021) Hannah Rashkin, David Reitter, Gaurav Singh Tomar, and Dipanjan Das. 2021.  Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. Association for Computational Linguistics, Online, 704–718.  https://doi.org/10.18653/v1/2021.acl-long.58
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. Association for Computational Linguistics, Hong Kong, China, 3982–3992.  https://doi.org/10.18653/v1/D19-1410
  * Ribeiro et al. (2021a) Leonardo F. R. Ribeiro, Martin Schmitt, Hinrich Schütze, and Iryna Gurevych. 2021a.  Investigating Pretrained Language Models for Graph-to-Text Generation. In _Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI_. Association for Computational Linguistics, Online, 211–227.  https://doi.org/10.18653/v1/2021.nlp4convai-1.20
  * Ribeiro et al. (2021b) Leonardo F. R. Ribeiro, Yue Zhang, and Iryna Gurevych. 2021b.  Structural Adapters in Pretrained Language Models for AMR-to-Text Generation. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021_ , Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, 4269–4282.  https://aclanthology.org/2021.emnlp-main.351
  * Ruan and Ling (2021) Yu-Ping Ruan and Zhenhua Ling. 2021.  Emotion-Regularized Conditional Variational Autoencoder for Emotional Response Generation.  _IEEE Transactions on Affective Computing_ (2021).  https://arxiv.org/abs/2104.08857
  * Saha et al. (2022) Punyajoy Saha, Kanishk Singh, Adarsh Kumar, Binny Mathew, and Animesh Mukherjee. 2022.  CounterGeDi: A Controllable Approach to Generate Polite, Detoxified and Emotional Counterspeech. In _Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22_ , Lud De Raedt (Ed.). International Joint Conferences on Artificial Intelligence Organization, 5157–5163.  https://doi.org/10.24963/ijcai.2022/716 AI for Good.
  * Samanta et al. (2020) Bidisha Samanta, Mohit Agarwal, and Niloy Ganguly. 2020\.  Fine-grained Sentiment Controlled Text Generation.  _arXiv preprint arXiv:2006.09891_ (2020).  https://arxiv.org/abs/2006.09891
  * Scialom et al. (2020) Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, and Jacopo Staiano. 2020.  Discriminative Adversarial Search for Abstractive Summarization. In _Proceedings of the 37th International Conference on Machine Learning_ _(ICML’20)_. JMLR.org, Article 793, 10 pages.  https://arxiv.org/abs/2002.10375
  * Sellam et al. (2020) Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020.  BLEURT: Learning Robust Metrics for Text Generation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Online, 7881–7892.  https://doi.org/10.18653/v1/2020.acl-main.704
  * Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016\.  Controlling Politeness in Neural Machine Translation via Side Constraints. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics, San Diego, California, 35–40.  https://doi.org/10.18653/v1/N16-1005
  * Shao et al. (2021) Yizhan Shao, Tong Shao, Minghao Wang, Peng Wang, and Jie Gao. 2021.  A Sentiment and Style Controllable Approach for Chinese Poetry Generation. In _CIKM ’21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021_ , Gianluca Demartini, Guido Zuccon, J. Shane Culpepper, Zi Huang, and Hanghang Tong (Eds.). ACM, 4784–4788.  https://doi.org/10.1145/3459637.3481964
  * Sheng et al. (2020) Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2020.  Towards controllable biases in language generation.  _arXiv preprint arXiv:2005.00268_ (2020).
  * Sheng et al. (2021) Zhonghao Sheng, Kaitao Song, Xu Tan, Yi Ren, Wei Ye, Shikun Zhang, and Tao Qin. 2021.  SongMASS: Automatic Song Writing with Pre-training and Alignment Constraint. In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021_. AAAI Press, 13798–13805.  https://ojs.aaai.org/index.php/AAAI/article/view/17626
  * Shi et al. (2020) Wenxian Shi, Hao Zhou, Ning Miao, and Lei Li. 2020\.  Dispersed Exponential Family Mixture VAEs for Interpretable Text Generation. In _International Conference on Machine Learning_. PMLR, 8840–8851.  http://proceedings.mlr.press/v119/shi20f/shi20f.pdf
  * Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020\.  AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Online, 4222–4235.  https://doi.org/10.18653/v1/2020.emnlp-main.346
  * Sohn et al. (2015) Kihyuk Sohn, Xinchen Yan, and Honglak Lee. 2015.  Learning Structured Output Representation Using Deep Conditional Generative Models. In _Proceedings of the 28th International Conference on Neural Information Processing Systems \- Volume 2_ (Montreal, Canada) _(NIPS’15)_. MIT Press, Cambridge, MA, USA, 3483–3491.  https://dl.acm.org/doi/10.5555/2969442.2969628
  * Song et al. (2021) Haoyu Song, Yan Wang, Kaiyan Zhang, Wei-Nan Zhang, and Ting Liu. 2021.  BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021_ , Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 167–177.  https://doi.org/10.18653/v1/2021.acl-long.14
  * Song et al. (2020b) Haoyu Song, Yan Wang, Weinan Zhang, Xiaojiang Liu, and Ting Liu. 2020b.  Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 5821–5831.  https://doi.org/10.18653/v1/2020.acl-main.516
  * Song et al. (2020a) Linfeng Song, Ante Wang, Jinsong Su, Yue Zhang, Kun Xu, Yubin Ge, and Dong Yu. 2020a.  Structural Information Preserving for Graph-to-Text Generation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 7987–7998.  https://doi.org/10.18653/v1/2020.acl-main.712
  * Song et al. (2019) Zhenqiao Song, Xiaoqing Zheng, Lu Liu, Mu Xu, and Xuanjing Huang. 2019.  Generating Responses with a Specific Emotion in Dialog. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, 3685–3695.  https://doi.org/10.18653/v1/p19-1359
  * Stiennon et al. (2020) Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2020.  Learning to Summarize from Human Feedback. In _Proceedings of the 34th International Conference on Neural Information Processing Systems_ (Vancouver, BC, Canada) _(NIPS’20)_. Curran Associates Inc., Red Hook, NY, USA, Article 253, 14 pages.  https://proceedings.neurips.cc/paper/2020/file/1f89885d556929e98d3ef9b86448f951-Paper.pdf
  * Su et al. (2021) Yixuan Su, David Vandyke, Sihui Wang, Yimai Fang, and Nigel Collier. 2021.  Plan-then-Generate: Controlled Data-to-Text Generation via Planning. In _Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021_ , Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, 895–909.  https://aclanthology.org/2021.findings-emnlp.76
  * Tambwekar et al. (2019) Pradyumna Tambwekar, Murtaza Dhuliawala, Lara J. Martin, Animesh Mehta, Brent Harrison, and Mark O. Riedl. 2019.  Controllable Neural Story Plot Generation via Reward Shaping. In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19_. International Joint Conferences on Artificial Intelligence Organization, 5982–5988.  https://doi.org/10.24963/ijcai.2019/829
  * Tang et al. (2019) Hongyin Tang, Miao Li, and Beihong Jin. 2019.  A Topic Augmented Text Generation Model: Joint Learning of Semantics and Structural Features. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. Association for Computational Linguistics, Hong Kong, China, 5090–5099.  https://doi.org/10.18653/v1/D19-1513
  * Tu et al. (2020) Lifu Tu, Richard Yuanzhe Pang, Sam Wiseman, and Kevin Gimpel. 2020\.  ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 2819–2826.  https://doi.org/10.18653/v1/2020.acl-main.251
  * van der Lee et al. (2021) Chris van der Lee, Albert Gatt, Emiel van Miltenburg, and Emiel Krahmer. 2021.  Human evaluation of automatically generated text: Current trends and best practice guidelines.  _Computer Speech & Language_ 67 (2021), 101151.  https://doi.org/10.1016/j.csl.2020.101151
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need. In _Advances in neural information processing systems_. 5998–6008.  https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
  * Vechtomova et al. (2018) Olga Vechtomova, Hareesh Bahuleyan, Amirpasha Ghabussi, and Vineet John. 2018.  Generating lyrics with variational autoencoder and multi-modal artist embeddings.  _CoRR_ abs/1812.08318 (2018).  arXiv:1812.08318 http://arxiv.org/abs/1812.08318
  * Wang et al. (2021a) Dingmin Wang, Ziyao Chen, Wanwei He, Li Zhong, Yunzhe Tao, and Min Yang. 2021a.  A Template-guided Hybrid Pointer Network for Knowledge-based Task-oriented Dialogue Systems. In _Proceedings of the 1st Workshop on Document-grounded Dialogue and Conversational Question Answering (DialDoc 2021)_. Association for Computational Linguistics, Online, 18–28.  https://doi.org/10.18653/v1/2021.dialdoc-1.3
  * Wang and Wan (2018) Ke Wang and Xiaojun Wan. 2018.  SentiGAN: Generating Sentimental Texts via Mixture Adversarial Networks. In _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18_. International Joint Conferences on Artificial Intelligence Organization, 4446–4452.  https://doi.org/10.24963/ijcai.2018/618
  * Wang et al. (2020) Ruize Wang, Zhongyu Wei, Ying Cheng, Piji Li, Haijun Shan, Ji Zhang, Qi Zhang, and Xuanjing Huang. 2020\.  Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communication. In _Proceedings of the 28th International Conference on Computational Linguistics_. International Committee on Computational Linguistics, Barcelona, Spain (Online), 2250–2260.  https://doi.org/10.18653/v1/2020.coling-main.204
  * Wang et al. (2019) Wenlin Wang, Zhe Gan, Hongteng Xu, Ruiyi Zhang, Guoyin Wang, Dinghan Shen, Changyou Chen, and Lawrence Carin. 2019\.  Topic-Guided Variational Auto-Encoder for Text Generation. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_. Association for Computational Linguistics, Minneapolis, Minnesota, 166–177.  https://doi.org/10.18653/v1/N19-1015
  * Wang et al. (2021b) Yufei Wang, Ian Wood, Stephen Wan, Mark Dras, and Mark Johnson. 2021b.  Mention Flags (MF): Constraining Transformer-based Text Generators. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. Association for Computational Linguistics, Online, 103–113.  https://doi.org/10.18653/v1/2021.acl-long.9
  * Wei et al. (2019) Wei Wei, Jiayi Liu, Xianling Mao, Guibing Guo, Feida Zhu, Pan Zhou, and Yuchong Hu. 2019.  Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction. In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019_ , Wenwu Zhu, Dacheng Tao, Xueqi Cheng, Peng Cui, Elke A. Rundensteiner, David Carmel, Qi He, and Jeffrey Xu Yu (Eds.). ACM, 1401–1410.  https://doi.org/10.1145/3357384.3357937
  * Welleck et al. (2019) Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2019\.  Neural Text Generation With Unlikelihood Training. In _International Conference on Learning Representations_.  https://arxiv.org/abs/1908.04319
  * Wolf et al. (2019) Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019\.  Transfer transfo: A transfer learning approach for neural network based conversational agents.  _arXiv preprint arXiv:1901.08149_ (2019).  https://arxiv.org/abs/1901.08149
  * Wu et al. (2021) Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Koncel-Kedziorski, Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf, and Bill Dolan. 2021.  A Controllable Model of Grounded Response Generation. In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021_. AAAI Press, 14085–14093.  https://ojs.aaai.org/index.php/AAAI/article/view/17658
  * Xu et al. (2019) Peng Xu, Yanshuai Cao, and Jackie Chi Kit Cheung. 2019\.  Unsupervised Controllable Text Generation with Global Variation Discovery and Disentanglement.  _CoRR_ abs/1905.11975 (2019).  arXiv:1905.11975 http://arxiv.org/abs/1905.11975
  * Xu et al. (2020) Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, and Bryan Catanzaro. 2020.  MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Online, 2831–2845.  https://doi.org/10.18653/v1/2020.emnlp-main.226
  * Yang and Klein (2021) Kevin Yang and Dan Klein. 2021.  FUDGE: Controlled Text Generation With Future Discriminators. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics, Online, 3511–3535.  https://doi.org/10.18653/v1/2021.naacl-main.276
  * Yang et al. (2022) Kexin Yang, Dayiheng Liu, Wenqiang Lei, Baosong Yang, Mingfeng Xue, Boxing Chen, and Jun Xie. 2022.  Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation.  https://doi.org/10.48550/ARXIV.2204.13362
  * Yang et al. (2019b) Pengcheng Yang, Lei Li, Fuli Luo, Tianyu Liu, and Xu Sun. 2019b.  Enhancing Topic-to-Essay Generation with External Commonsense Knowledge. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Florence, Italy, 2002–2012.  https://doi.org/10.18653/v1/P19-1193
  * Yang et al. (2018) Xiaopeng Yang, Xiaowen Lin, Shunda Suo, and Ming Li. 2018\.  Generating Thematic Chinese Poetry Using Conditional Variational Autoencoders with Hybrid Decoders. In _Proceedings of the 27th International Joint Conference on Artificial Intelligence_ (Stockholm, Sweden) _(IJCAI’18)_. AAAI Press, 4539–4545.  https://arxiv.org/abs/1711.07632
  * Yang et al. (2019a) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019a.  XLNet: Generalized Autoregressive Pretraining for Language Understanding. In _Advances in Neural Information Processing Systems_ , H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32. Curran Associates, Inc.  https://proceedings.neurips.cc/paper/2019/file/dc6a7e655d7e5840e66733e9ee67cc69-Paper.pdf
  * Zeldes et al. (2020) Y. Zeldes, D. Padnos, O. Sharir, and B. Peleg. 2020\.  Technical Report: Auxiliary Tuning and its Application to Conditional Text Generation.  (2020).  https://arxiv.org/abs/2006.16823
  * Zeng and Nie (2021) Yan Zeng and Jian-Yun Nie. 2021.  A Simple and Efficient Multi-Task Learning Approach for Conditioned Dialogue Generation. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021_ , Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, 4927–4939.  https://doi.org/10.18653/v1/2021.naacl-main.392
  * Zhang and song (2022) Hanqing Zhang and Dawei song. 2022.  DisCup: Discriminator Cooperative Unlikelihood Prompt Tuning for Controllable Text Generation. In _The 2022 Conference on Empirical Methods in Natural Language Processing_. Abu Dhabi.  https://arxiv.org/abs/2210.09551
  * Zhang et al. (2017) Rui Zhang, Zhenyu Wang, and Dongcheng Mai. 2017.  Building emotional conversation systems using multi-task Seq2Seq learning. In _National CCF Conference on Natural Language Processing and Chinese Computing_. Springer, 612–621.  https://link.springer.com/chapter/10.1007/978-3-319-73618-1_51
  * Zhang et al. (2019b) Rui Zhang, Zhenyu Wang, Kai Yin, and Zhenhua Huang. 2019b.  Emotional text generation based on cross-domain sentiment transfer.  _IEEE Access_ 7 (2019), 100081–100089.  https://ieeexplore.ieee.org/document/8772090
  * Zhang et al. (2018a) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018a.  Personalizing Dialogue Agents: I have a dog, do you have pets too?. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Association for Computational Linguistics, Melbourne, Australia, 2204–2213.  https://doi.org/10.18653/v1/P18-1205
  * Zhang et al. (2018b) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018b.  Personalizing Dialogue Agents: I have a dog, do you have pets too?. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Association for Computational Linguistics, Melbourne, Australia, 2204–2213.  https://doi.org/10.18653/v1/P18-1205
  * Zhang et al. (2020b) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020b.  BERTScore: Evaluating Text Generation with BERT. In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.  https://openreview.net/forum?id=SkeHuCVFDr
  * Zhang et al. (2020c) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020c.  DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations_. Association for Computational Linguistics, Online, 270–278.  https://doi.org/10.18653/v1/2020.acl-demos.30
  * Zhang et al. (2020d) Yizhe Zhang, Guoyin Wang, Chunyuan Li, Zhe Gan, Chris Brockett, and Bill Dolan. 2020d.  POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Online, 8649–8670.  https://doi.org/10.18653/v1/2020.emnlp-main.698
  * Zhang et al. (2019a) Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019a.  ERNIE: Enhanced Language Representation with Informative Entities. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Florence, Italy, 1441–1451.  https://doi.org/10.18653/v1/P19-1139
  * Zhang et al. (2020a) Zhengyan Zhang, Xu Han, Hao Zhou, Pei Ke, Yuxian Gu, Deming Ye, Yujia Qin, YuSheng Su, Haozhe Ji, Jian Guan, Fanchao Qi, Xiaozhi Wang, Yanan Zheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen, Daixuan Li, Zhenbo Sun, Zhiyuan Liu, Minlie Huang, Wentao Han, Jie Tang, Juanzi Li, Xiaoyan Zhu, and Maosong Sun. 2020a.  CPM: A Large-scale Generative Chinese Pre-trained Language Model.  _CoRR_ abs/2012.00413 (2020).  arXiv:2012.00413 https://arxiv.org/abs/2012.00413
  * Zhao et al. (2020) Chao Zhao, Marilyn A. Walker, and Snigdha Chaturvedi. 2020\.  Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 2481–2491.  https://doi.org/10.18653/v1/2020.acl-main.224
  * Zhao et al. (2017) Junbo Jake Zhao, Michaël Mathieu, and Yann LeCun. 2017\.  Energy-based Generative Adversarial Networks. In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.  https://openreview.net/forum?id=ryh9pmcee
  * Zheng et al. (2020) Yinhe Zheng, Rongsheng Zhang, Minlie Huang, and Xiaoxi Mao. 2020\.  A Pre-Training Based Personalized Dialogue Generation Model with Persona-Sparse Data. In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020_. AAAI Press, 9693–9700.  https://aaai.org/ojs/index.php/AAAI/article/view/6518
  * Zhong et al. (2020) Peixiang Zhong, Chen Zhang, Hao Wang, Yong Liu, and Chunyan Miao. 2020.  Towards Persona-Based Empathetic Conversational Models. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Online, 6556–6566.  https://doi.org/10.18653/v1/2020.emnlp-main.531
  * Zhou et al. (2018) Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018.  Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory. In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018_ , Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 730–739.  https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16455
  * Zhou and Xu (2020) Wangchunshu Zhou and Ke Xu. 2020.  Learning to compare for better training and evaluation of open domain natural language generation models. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 9717–9724.  https://arxiv.org/abs/2002.05058
  * Zhu et al. (2018) Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018.  Texygen: A Benchmarking Platform for Text Generation Models. In _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_ (Ann Arbor, MI, USA) _(SIGIR ’18)_. Association for Computing Machinery, New York, NY, USA, 1097–1100.  https://doi.org/10.1145/3209978.3210080
  * Ziegler et al. (2019) Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul F. Christiano, and Geoffrey Irving. 2019.  Fine-Tuning Language Models from Human Preferences.  _CoRR_ abs/1909.08593 (2019).  arXiv:1909.08593 http://arxiv.org/abs/1909.08593
  * Zou et al. (2021) Xu Zou, Da Yin, Qingyang Zhong, Hongxia Yang, Zhilin Yang, and Jie Tang. 2021\.  Controllable Generation from Pre-trained Language Models via Inverse Prompting.  _CoRR_ abs/2103.10685 (2021).  arXiv:2103.10685 https://arxiv.org/abs/2103.10685