# Automatic Datapath Optimization using E-Graphs

Samuel Coward  Numerical Hardware Group
Intel Corporation
Email<EMAIL_ADDRESS>George A. Constantinides  Electrical and
Electronic Engineering
Imperial College London
Email<EMAIL_ADDRESS>Theo Drane  Numerical Hardware Group
Intel Corporation
Email<EMAIL_ADDRESS>
###### Abstract

Manual optimization of Register Transfer Level (RTL) datapath is commonplace
in industry but holds back development as it can be very time consuming. We
utilize the fact that a complex transformation of one RTL into another
equivalent RTL can be broken down into a sequence of smaller, localized
transformations. By representing RTL as a graph and deploying modern graph
rewriting techniques we can automate the circuit design space exploration,
allowing us to discover functionally equivalent but optimized architectures.
We demonstrate that modern rewriting frameworks can adequately capture a wide
variety of complex optimizations performed by human designers on bit-vector
manipulating code, including significant error-prone subtleties regarding the
validity of transformations under complex interactions of bitwidths. The
proposed automated optimization approach is able to reproduce the results of
typical industrial manual optimization, resulting in a reduction in circuit
area by up to 71%. Not only does our tool discover optimized RTL, but also
correctly identifies that the optimal architecture to implement a given
arithmetic expression can depend on the width of the operands, thus producing
a library of optimized designs rather than the single design point typically
generated by manual optimization. In addition, we demonstrate that prior
academic work on maximally exploiting carry-save representation and on
multiple constant multiplication are both generalized and extended, falling
out as special cases of this paper.

###### Index Terms:

hardware optimization, design automation, datapath design

##  I Introduction

In industry and academia, Register Transfer Level (RTL) development is limited
to minimal design space exploration due to the complexity of design space and
is slowed down by long debug timelines. RTL optimization of datapath designs
is still often a manual task, as synthesis tools are unable to achieve the
results of a skilled engineer [1].

A key observation is that manual RTL optimization is typically performed by
applying a number of known ‘useful’ transformations to a design. These
transformations, and their domain of validity, are accumulated through years
of engineer design experience. In combination, these transformations may
result in substantial changes to the underlying RTL. Apart from some simple
transformations implemented automatically in modern ASIC design tools [2], the
process of determining a sequence of transformations to apply to an RTL design
is currently based on designer intuition [3], largely due to the non-convex
nature of the design space: it is often necessary to apply an early
transformation that results in a worse-quality circuit before then applying a
later one leading to an overall improvement. It is this process we seek to
automate.

Such automation facilitates the creation of bitwidth dependent architectures,
where different parameterisations may result in different architectures: the
best design approach for narrow bitwidths may not be the best design approach
for wider bitwidths. A range of RTLs automatically generated from a single
parameterisable input retains the ease-of-use benefits of parameterisable RTL
without sacrificing quality.

Existing commercial synthesis tools are capable of merging together
consecutive additions in a circuit design to make best use of carry-save
representations [1]. However when arithmetic is interspersed with logic, the
tools frequently miss potential optimization opportunities [4].

We aim to leverage existing commercial synthesis tools by transforming RTL to
a form the existing tools can maximally optimize. We focus on combinational
RTL, although the techniques described are equally applicable to pipelined
designs via retiming. Given a design in the form of an RTL implementation $R$,
we aim to find an RTL implementation $R^{\prime}$ that minimises
$\textrm{cost}(R^{\prime})$ for some cost function, such that the two RTLs are
functionally equivalent, $R\simeq R^{\prime}$. We define the equivalence
relation $\simeq$ as $R\simeq R^{\prime}$ if and only if for all possible
inputs, all outputs of $R$ and $R^{\prime}$ are equal.

We represent such RTL as a data-flow graph, where operators and operands are
represented by nodes with edges, labelled by bitwidth, connecting operands and
operators. This graphical representation allows us to formulate the problem as
a graph optimization problem, where we are allowed to manipulate the graph
with equivalence-preserving transformations. This formulation allows us to
take advantage of recent advances in e(quivalence)-graph and equality
saturation [5] technology, discussed in Section II, alongside previous
motivating work in automated RTL optimization and design. Application of
e-graphs to the RTL optimization problem is presented in Section III. We
demonstrate results in Section IV and validate our cost metric in Section V.

The paper contains the following novel contributions:

  * •

application of e-graphs and equality saturation to automate datapath RTL
optimization,

  * •

a precisely defined set of rewrites that facilitate efficient design space
exploration together with their domains of applicability in designs utilizing
multiple bitwidths,

  * •

an automated method to optimize architectures as a function of bitwidth
parameters,

  * •

quantification of a ‘noise floor’ in datapath logic synthesis using ‘fuzzing’
techniques from software testing.

##  II Background

###  II-A Datapath Optimization

A useful example of transformation-based datapath improvement comes from
Verma, Brisk and Ienne [4], which automatically applies data-flow
transformations to maximally exploit carry-save representation. Their primary
objective is to cluster additions together in the data-flow graph, a useful
target as full carry-propagate addition is generally expensive and can often
be avoided. This can be done by deploying compressor trees, circuits taking
three or more input words which get reduced to two output words: a carry and a
save. Using a carry-propagate adder to sum the carry and the save returns the
sum of all the inputs. This can be beneficial as, for example, combining two
consecutive carry-propagate adders into a compressor tree and one carry-
propagate adder saves the cost of one carry-propagate adder at the expense of
a compressor tree. We generalize this work using our methodology, which is
able to replicate the results obtained in [4] as a special case.

Another well-studied transformation beyond the reach of standard commercial
synthesis tools is the multiple constant multiplication (MCM) problem [6, 7].
The MCM problem asks, given a set of integer coefficients
$\\{a_{1},...,a_{n}\\}$ and variable $x$, what is the optimal architecture to
compute the set $\\{a_{1}\times x,...,a_{n}\times x\\}$? Competing solutions
use a fixed number representation of the constants [7], often canonical signed
digit (CSD) representation [8], and/or deploy an adder graph algorithm [6]. A
transformation based approach also captures both of these methods.

In addition to these special cases, there is a wide variety of transformations
that can be captured through standard arithmetic rewrites, e.g. associativity,
distributivity, etc. Often these rewrites interact with each other, in the
sense that applying one type of transformation opens or closes the door to
applying a different class of transformation.

###  II-B E-graphs and Equality Saturation

Equivalence graphs, commonly called e-graphs, provide a dense representation
of equivalence classes (e-classes) of expressions [9]. Often found in theorem
provers, this data structure enables a graph optimization technique called
equality saturation [5, 10, 11]. The e-graph represents expressions, where the
nodes, known as e-nodes, represent functions (including variables and
constants, as 0-arity functions) and are partitioned into a set of e-classes.
The intuition is that e-classes can be used to compactly represent equivalent
expressions, whose evaluation always leads to the same result. Edges represent
function inputs and are from e-nodes to e-classes; see Figure 1, where dashed
lines represent e-class boundaries, solid ellipses are e-nodes and arrows are
edges. We define $\mathcal{C}$ to be the set of e-classes, $\mathcal{N}$ the
set of e-nodes and $E\subseteq\mathcal{N}\times\mathcal{C}$ the set of edges.
We also introduce $\mathcal{N}_{c}$ to denote the set of e-nodes in a given
e-class $c$.

Rewrites define equivalences over expressions, for example $x+x\rightarrow
2\times x$ says that $x+x$ is equivalent to $2\times x$. Such rewrites are
applied constructively to the e-graph, meaning that the left hand side of the
rewrite remains in the data structure. Constructive rewrite application avoids
the concern of which order to apply rewrites in. As rewrites are applied, the
e-graph grows monotonically, representing more and more equivalent
expressions, and hence naturally capturing the interaction between different
rewrite rules.

(a) Initial e-graph contains
$(2\times x)>>1$

(b) Apply $x\times 2\rightarrow x<<1$

(c) Apply $(x<<s)>>s\rightarrow x$

Figure 1: E-graph rewriting for standard integer arithmetic. Dashed boxes
represent e-classes of equivalent expressions. Green nodes represent newly
added nodes. Red dashed boxes highlight which e-class has been modified.

Equality saturation provides us with a stopping condition. At the point where
further rewrites add no additional information, we say that the e-graph has
saturated. From an e-graph representing potentially infinitely many equivalent
expressions we may choose the “best” expression [5].

egg is a recent Rust e-graph library, which is intended to be a general
purpose and reusable implementation [5]. It adds powerful performance
optimizations over existing, usually bespoke, e-graph implementations along
with some useful additional features. It has been used to automatically
improve the numerical stability of floating point expressions [12], map
programs onto hardware accelerators [13] and optimize linear algebra [14]. To
build a functioning e-graph optimization tool, egg must be supplied with a
language definition – that is a set of operator names together with their
arity, and a rewrite set – that is a set of equivalences over the given
language definition.

##  III Methodology

This section demonstrates how e-graphs can be applied to the RTL optimization
problem. We use a natural graphical representation of RTL, using data-flow
graphs allowing us to fit the optimization problem into the e-graph framework.
In this framework the e-graph contains classes of equivalent bitvector
manipulating expressions and the rewrites transform such expressions to
alternative equivalent expressions. The tool parses input Verilog using Yosys
[15] and converts it into nested S-expressions in Common Lisp [16].

term::=(operator [term] [term]…[term])

The syntax is defined by the language described in Section III-A. These
expressions are converted into e-graphs by egg. From the e-graph we extract a
nested S-expression from which RTL is automatically generated, writing one
operation per line. Figure 2 provides a flow diagram of the tool.

Input Verilog E-graphRewrite Extract (ILP) Optimised Verilog egg Figure 2: Flow diagram describing the operation of the tool. TABLE I: Operators defined in our egg implementation of RTL optimization. We include the architecture chosen for theoretical cost assignment. Operator | Symbol | Arity | Architecture
---|---|---|---
Left/Right Shift | $<</>>$ | 2 | Mux Tree
Addition/Subtraction | +/- | 2 | Prefix Adder (PA) [17]
Negation | - | 1 | PA
Multiplication | $\times$ | 2 | Booth Radix-4 [18]
Multiplexer | $\cdot?\cdot:\cdot$ | 3 | Mux gates
Not/Inversion | $\sim$ | 1 | One-input gates
Concatenate | {,} | n | Wiring
Comparison | $>/<$ | 2 | PA
1-4 Sum | SUM | n | CSA and PA
Muxed Mult Array | MUXAR | 3 | Array Reduction and PA
Fused Multiply-Add | FMA | 3 | Booth Radix-4

###  III-A Language

We consider the problem from the abstraction level of a Verilog parser and
operate on finite length bitvectors. We target bitvector arithmetic and
bitwise Boolean operations as they form the basis of low-level datapath
optimization. Including the bitwidth of these vectors is crucial to correctly
model the circuit’s behaviour. Bitwidths are also essential to correctly
evaluate the cost of a given operation, clearly an 8-bit addition is less
expensive than a 32-bit addition. The defined operations represented by nodes
in the e-graph are described in Table I, and are defined for all inputs.

The first set in the table is a subset of the operators defined in Verilog
[19]. These operators are fundamental in most arithmetic circuit designs and
manipulating designs using them can have significant effects on power,
performance and area. Bitwidth information is included in the e-graph as edge
labels between the nodes. In addition to these basic operators, we introduce a
second set of operators in Table I (below the dashed line), which encode the
merging and sharing capabilities of modern synthesis tools. In [4], the
benefits of merging adjacent additions to maximise the use of carry-save are
exploited. We are able to capture the same effect in an automated manner
through introduction of a SUM node in our language definition, which combines
an arbitrary number of additions into a single node. This node encodes the
compressor tree and carry-propagate adder described in Section I. Figure 3
shows how consecutive additions can be rewritten as a SUM node. Other such
merging operators are the fused multiply-add (FMA) and the less familiar muxed
multiplication array (MUXAR), which blends two disjoint multiplication arrays
into one. These are discussed further in Section III-B.

(a) Consecutive additions

(b) Merged additions encoded as a SUM

Figure 3: The edge labels contain the operand’s index and the operand’s
bitwidth in square brackets.

###  III-B Rewrites

TABLE II: The set of bitwidth dependent rewrites supplied to egg. Left subscript notation, $\prescript{}{p}{x}$, denotes a bitvector $x$ with length $p$ bits. The $*$ operation represents both $\\{+,\times\\}$. The square brackets represent Verilog bit slicing, where $a[x:y]$ means we take bits $x$ down to $y$ of $a$. The rules are conditionally applied according to column 4, which is sufficient but not always necessary. Class | Name | Left-hand Side $\rightarrow$ Right-hand Side | Sufficient Condition
---|---|---|---
Bitvector Arithmetic Identities | Commutativity | $\prescript{}{r}{(}\prescript{}{p}{a}*\prescript{}{q}{b})\rightarrow\prescript{}{r}{(}\prescript{}{q}{b}*\prescript{}{p}{a})$ | True
|  | $(q\geq t\lor r+s\leq q)$
Mult Associativity | $\prescript{}{t}{(}\prescript{}{u}{(}\prescript{}{p}{a}\times\prescript{}{r}{b})\times\prescript{}{s}{c})\rightarrow\prescript{}{t}{(}\prescript{}{p}{a}\times\prescript{}{q}{(}\prescript{}{r}{b}\times\prescript{}{s}{c}))$ | $\land(u\geq t\lor p+r\leq u)$
Add Associativity | $\prescript{}{t}{(}\prescript{}{u}{(}\prescript{}{p}{a}+\prescript{}{r}{b})+\prescript{}{s}{c})\rightarrow\prescript{}{t}{(}\prescript{}{p}{a}+\prescript{}{q}{(}\prescript{}{r}{b}+\prescript{}{s}{c}))$ | $(q\geq t\lor\max(r,s)<q)$
$\land(u\geq t\lor\max(p,r)<u)$
Distribute Mult over Add | $\prescript{}{r}{(}\prescript{}{p}{a}\times\prescript{}{q}{(}\prescript{}{s}{b}+\prescript{}{t}{c}))\rightarrow\prescript{}{r}{(}\prescript{}{u}{(}\prescript{}{p}{a}\times\prescript{}{s}{b})+\prescript{}{v}{(}\prescript{}{p}{a}\times\prescript{}{t}{c}))$ | $\min(q,u,v)\geq r$
Sum Same | $\prescript{}{q}{(}\prescript{}{p}{a}+\prescript{}{p}{a})\rightarrow\prescript{}{q}{(}\prescript{}{2}{2}\times\prescript{}{p}{a})$ | True
Mult Sum Same | $\prescript{}{r}{(}\prescript{}{s}{(}\prescript{}{p}{a}\times\prescript{}{q}{b})+\prescript{}{q}{b})\rightarrow\prescript{}{r}{(}\prescript{}{t}{(}\prescript{}{p}{a}+\prescript{}{1}{1})\times\prescript{}{q}{b})$ | $t>p\land s\geq p+q$
Add Zero | $\prescript{}{p}{(}\prescript{}{p}{a}+\prescript{}{q}{b})\rightarrow\prescript{}{p}{(}a)$ | $b\equiv 0\mod 2^{p}$
Sub to Neg | $\prescript{}{r}{(}\prescript{}{p}{a}-\prescript{}{q}{b})\rightarrow\prescript{}{r}{(}\prescript{}{p}{a}+\prescript{}{q}{(}-\prescript{}{q}{b}))$ | True
Mult by One | $\prescript{}{p}{(}\prescript{}{p}{a}\times\prescript{}{q}{b})\rightarrow\prescript{}{p}{(}a)$ | $b\equiv 1\mod 2^{p}$
Mult by Two | $\prescript{}{r}{(}\prescript{}{p}{a}\times\prescript{}{2}{2})\rightarrow\prescript{}{r}{(}\prescript{}{p}{a}<<\prescript{}{1}{1})$ | True
Bitvector Logic Identities | Merge Left Shift | $\prescript{}{r}{(}\prescript{}{u}{(}\prescript{}{p}{a}<<\prescript{}{q}{b})<<\prescript{}{s}{c})\rightarrow\prescript{}{r}{(}\prescript{}{p}{a}<<\prescript{}{t}{(}\prescript{}{q}{b}+\prescript{}{s}{c}))$ | $t>\max(q,s)\land u\geq r$
Merge Right Shift | $\prescript{}{r}{(}\prescript{}{u}{(}\prescript{}{p}{a}>>\prescript{}{q}{b})>>\prescript{}{s}{c})\rightarrow\prescript{}{r}{(}\prescript{}{p}{a}>>\prescript{}{t}{(}\prescript{}{q}{b}+\prescript{}{s}{c}))$ | $t>\max(q,s)\land u\geq p$
Redundant Sel | $\prescript{}{p}{(}\prescript{}{1}{b}?\prescript{}{p}{a}:\prescript{}{p}{a})\rightarrow\prescript{}{p}{a}$ | True
Neg Not | $\prescript{}{r}{(}-\prescript{}{p}{a})\rightarrow\prescript{}{r}{(}\prescript{}{p}{(}\sim(\prescript{}{p}{a}))+\prescript{}{1}{1})$ | $r\leq p$
Not over Con | $\prescript{}{r}{(}\sim(\prescript{}{q+s}{\\{}\prescript{}{q}{a},\prescript{}{s}{b}\\}))\rightarrow\prescript{}{r}{\\{}\prescript{}{q}{(}\sim(\prescript{}{q}{a})),\prescript{}{s}{(}\sim(\prescript{}{s}{b}))\\}$ | $q+s\geq r$
Constant Expansion |  | $\prescript{}{r}{(}\prescript{}{q}{c}\times\prescript{}{p}{x})\rightarrow$ |
Mult Constant | $\prescript{}{r}{(}\prescript{}{r}{(}\prescript{}{q}{(}\prescript{}{2}{2}\times\prescript{}{q-1}{c}[q-1:1])\times\prescript{}{p}{x})+\prescript{}{p}{(}\prescript{}{1}{c}[0]\times\prescript{}{p}{x}))$ | $c$ constant
One to Two Mult | $\prescript{}{p}{(}\prescript{}{1}{1}\times\prescript{}{p}{x})\rightarrow\prescript{}{p}{(}\prescript{}{q}{(}\prescript{}{2}{2}\times\prescript{}{p}{x})-\prescript{}{p}{x})$ | $q>p$
Arithmetic Logic Exchange | Left Shift Add | $\prescript{}{r}{(}\prescript{}{s}{(}\prescript{}{p}{a}+\prescript{}{q}{b})<<\prescript{}{t}{c})\rightarrow\prescript{}{r}{(}\prescript{}{u}{(}\prescript{}{p}{a}<<\prescript{}{t}{c})+\prescript{}{u}{(}\prescript{}{q}{b}<<\prescript{}{t}{c}))$ | $(s\geq r\lor\max(p,q)<s)\land u\geq r$
|  | $q\geq t\land s\geq p+2^{u}-1$
Add Right Shift | $\prescript{}{r}{(}\prescript{}{p}{a}+\prescript{}{q}{(}\prescript{}{t}{b}>>\prescript{}{u}{c}))\rightarrow\prescript{}{r}{(}\prescript{}{v}{(}\prescript{}{s}{(}\prescript{}{p}{a}<<\prescript{}{u}{c})+\prescript{}{t}{b})>>\prescript{}{u}{c})$ | $\land v>\max(s,t)$
Left Shift Mult | $\prescript{}{r}{(}\prescript{}{t}{(}\prescript{}{p}{a}\times\prescript{}{q}{b})<<\prescript{}{u}{c})\rightarrow\prescript{}{r}{(}\prescript{}{v}{(}\prescript{}{p}{a}<<\prescript{}{u}{c})\times\prescript{}{q}{b})$ | $t\geq r\land v\geq r$
Sel Add | $\prescript{}{r}{(}\prescript{}{1}{e}?\prescript{}{r}{(}\prescript{}{p}{a}+\prescript{}{q}{b}):\prescript{}{r}{(}\prescript{}{p}{c}+\prescript{}{q}{d}))\rightarrow$ | True
$\prescript{}{r}{(}\prescript{}{p}{(}\prescript{}{1}{e}?\prescript{}{p}{a}:\prescript{}{p}{c})+\prescript{}{q}{(}\prescript{}{1}{e}?\prescript{}{q}{b}:\prescript{}{q}{d}))$
Sel Add Zero | $\prescript{}{p}{(}\prescript{}{1}{e}?\prescript{}{p}{(}\prescript{}{p}{a}+\prescript{}{q}{b}):\prescript{}{p}{c})\rightarrow\prescript{}{p}{(}\prescript{}{p}{(}\prescript{}{1}{e}?\prescript{}{p}{a}:\prescript{}{p}{c})+\prescript{}{q}{(}\prescript{}{1}{e}?\prescript{}{q}{b}:\prescript{}{q}{0}))$ | True
Move Sel Zero | $\prescript{}{r}{(}\prescript{}{p}{(}\prescript{}{1}{b}?\prescript{}{p}{0}:\prescript{}{p}{a})\times\prescript{}{q}{c})\rightarrow\prescript{}{r}{(}\prescript{}{p}{a}\times\prescript{}{q}{(}\prescript{}{1}{b}?\prescript{}{q}{0}:\prescript{}{q}{c}))$ | True
| Concat to Add | $\prescript{}{r}{\\{}\prescript{}{p}{a},\prescript{}{q}{b}\\}\rightarrow\prescript{}{r}{(}\prescript{}{s}{(}\prescript{}{p}{a}<<\prescript{}{u}{q})+\prescript{}{q}{b})$ | $s\geq p+2^{u}-1\land u\geq\lceil\log_{2}(q+1)\rceil$
Merging Ops |  | $\prescript{}{q_{1}}{(}\prescript{}{p_{1}}{a}1+\prescript{}{q_{2}}{(}\prescript{}{p_{2}}{a}2+\prescript{}{q_{3}}{(}\prescript{}{p_{3}}{a}3+...+\prescript{}{p_{n}}{a}n)...))\rightarrow$ | $q_{i}>\max(p_{i},q_{i+1}),i=1,...,n-2$
Merge Additions | $\prescript{}{q_{1}}{(}\texttt{SUM}(\prescript{}{p_{1}}{a}1,\prescript{}{p_{2}}{a}2,...,\prescript{}{p_{n}}{a}n))$ | $\land q_{n-1}>\max(p_{n-1},p_{n})$
| $\prescript{}{t}{(}\prescript{}{s}{(}\prescript{}{q}{a}\times\prescript{}{r}{b})+\prescript{}{s}{(}\prescript{}{q}{c}\times\prescript{}{r}{(}\sim(\prescript{}{r}{b}))))$ |
Merge Mult Array | $\rightarrow\prescript{}{t}{(}\texttt{MUXAR}(\prescript{}{r}{b},\prescript{}{q}{a},\prescript{}{q}{c}))$ | $s\geq q+r\land t>s$
| FMA Merge | $\prescript{}{t}{(}\prescript{}{s}{(}\prescript{}{p}{a}\times\prescript{}{q}{b})+\prescript{}{r}{c})\rightarrow\prescript{}{t}{(}\texttt{FMA}(\prescript{}{p}{a},\prescript{}{q}{b},\prescript{}{r}{c}))$ | $s\geq p+q\land t>\max(s,r)$

We have identified and formalized rules that capture many of the manual
transformations that Intel’s Numerical Hardware Group regularly deploy
manually. The set of rewrites described in Table II define equivalences over
expressions. Each of the rewrites incorporates bitwidth information, where
bitwidths are associated with operands. We introduce a left subscript
notation, $\prescript{}{p}{x}$, to denote a bitvector $x$ with length $p$
bits. Left subscript takes the lowest operator precedence.

egg supports conditional rewrites. We make use of this facility to allow us to
decide whether a rule may be applied based on its bitwidth information. The
sufficient (but not always necessary) conditions under which each rule applies
are described in Table II. As an example, consider the “Distribute Mult Over
Add” rewrite, where the sufficient condition is a function of the bitwidth
parameters, $\min(q,u,v)\geq r$. When this condition is satisfied the rewrite
can be safely applied as the equivalence holds. For some rewrites, a parameter
appears on the right hand side but not on the left, for example $q$ in the
associativity rewrites. This corresponds to a condition constraining but not
fully determining the values that these undefined parameters can take, in
which case we select the minimum feasible bitwidth. The conditions have been
validated by creating a parameterizable SMT query for each rewrite using the
theory of fixed size bitvectors [20], which have been checked for all
combinations of bitwidths in {1,…,10}, however a formal equivalence check
between the generated and original RTL ensures that we do not need to trust
the correctness of our rewrites or of the egg library.

Arithmetic Logic Exchange rewrites are inspired by a set taken from [4], which
focus on the interplay of logic and addition. To extend the prior work we have
added rewrites that are able to move other arithmetic operators and we have
generalized the rules so they are able to be applied in a multiple bitwidth
setting.

The “Merging Ops” rewrites allow us to correctly evaluate the cost of specific
sequences of operations, that logic synthesis tools are able to effectively
optimize. We have seen that the SUM node merges consecutive additions,
highlighting that the area usage for consecutive additions is not additive.
This captures what logic synthesis tools will do to an expression such as
$a+b+c$, converting it to a compressor tree and deploying a single carry-
propagate adder. Further merging optimization capabilities of Synopsys Design
Compiler are documented [1]. The “Merge Mult Array” rewrite does not make use
of carry-save format but identifies that two disjoint multiplication arrays
can be merged. Letting $a[i]$ represent bit $i$ of $a$ and
$u=\lceil\log_{2}(r)\rceil$, MUXAR is shorthand for the right hand side of the
rewrite, where the SUM represents array reduction:

$\displaystyle\prescript{}{t}{(}\prescript{}{s}{(}\prescript{}{q}{a}$
$\displaystyle\times\prescript{}{r}{b})+\prescript{}{s}{(}\prescript{}{q}{c}\times\prescript{}{r}{(}\sim(\prescript{}{r}{b}))))\rightarrow$
$\displaystyle\prescript{}{t}{(}\texttt{SUM}($
$\displaystyle\prescript{}{s}{(}\prescript{}{q}{(}\prescript{}{1}{b}[0]?\prescript{}{q}{a}:\prescript{}{q}{c})<<\prescript{}{u}{0}),$
$\displaystyle\prescript{}{s}{(}\prescript{}{q}{(}\prescript{}{1}{b}[1]?\prescript{}{q}{a}:\prescript{}{q}{c})<<\prescript{}{u}{1}),...,$
$\displaystyle\prescript{}{s}{(}\prescript{}{q}{(}\prescript{}{1}{b}[r-1]?\prescript{}{q}{a}:\prescript{}{q}{c})<<\prescript{}{u}{(}r-1)))).$

Logic synthesis is capable of exploiting this optimization if it identifies an
expression of the form $(a\times b)+(c\times\sim b)$, but we must indicate the
merging opportunity to our tool.

We added the “Constant Expansion” rules to re-express multiplication of a
variable by a constant. These rules allow us to recreate results from the
literature described in Section II-A on the MCM problem. In addition to the
explicit rewrite rules, constant folding is implemented as an e-class analysis
in egg [5].

###  III-C Extraction

Having applied rewrites to the e-graph until saturation or timeout limits are
reached, we now must extract the optimal design from potentially infinitely
many choices. The extraction process must select a set of e-classes to
implement and for each e-class, which e-node within that class to implement,
subject to the constraint that the e-class children of each selected e-node
must also be selected. Choosing an optimal design requires some metric that
allows us to discriminate between competing implementations. Industrial
circuit design is typically judged on area, latency and power consumption. In
this contribution we will only use an area metric, therefore our definition of
optimal will be the smallest circuit implementation.

We have developed a theoretical area estimate cost function in terms of the
number of two-input gates required for the operator. It assigns a cost per
operator that is a function of the input and output bitwidths. Table I lists
the architectures on which we base the cost of the more complex operators. We
introduce different costs for when at least one of the operands is a constant.
The cost of a complete design is then the sum of the operator costs, and the
objective is to minimise this cost.

The major benefit of using a theoretical cost metric as opposed to using
metrics derived from logic synthesis or HLS tools is the computation speed
which, when combined with equality saturation, enables effective design space
exploration. Cost metric validation is addressed in Section V.

By explicitly introducing rules for operator merging and sharing, we are able
to define a cost for each node based only on its type and argument bitwidths,
capturing downstream synthesis optimizations for SUM, MUXAR and FMA.

When extracting an RTL implementation from an e-graph, one is immediately
faced with the question of common subexpressions. Common subexpressions are
naturally extracted as part of the e-graph construction process [5] and
ideally we would want to utilize this information in the resulting hardware,
for example extracting $(x+1)\times(x+1)$ as $\texttt{let }y=x+1\text{ in
}y\times y$. However, this makes the extraction problem an inherently global
problem over the e-classes, in the sense that the optimal e-node
implementation for a given e-class may depend on the selected e-node
implementation of the other e-classes in the graph. Previous solutions have
solved this by posing optimal extraction as an integer linear programming
(ILP) problem [13, 14], and we follow the same approach in this work.

Using the notation defined in Section II-B, for each e-node $n\in\mathcal{N}$
we associate a cost, $\text{cost}(n)$, given by the theoretical cost function,
and a binary variable $x_{n}\in\\{0,1\\}$, which indicates whether $n$ is
implemented in the final extracted RTL. Our objective is to minimize the total
implementation cost, as described by (1). (2) then guarantees that for every
node $n$, we implement a node from each of its child e-classes. Lastly we
introduce $\mathcal{S}$, the set of e-classes representing the desired
expressions to implement in RTL. (3) then ensures that all these outputs are
produced by the final RTL.

$\displaystyle\text{minimize: }\sum_{n\in\mathcal{N}}\text{cost}(n)x_{n}\text{
subject to:}$ (1) $\displaystyle\forall(n,c)\in
E.\;x_{n}\leq\sum_{n^{\prime}\in\mathcal{N}_{c}}x_{n^{\prime}}$ (2)
$\displaystyle\forall c\in{\mathcal{S}}.\;\sum_{n\in\mathcal{N}_{c}}x_{n}=1.$
(3)

E-graphs may contain cycles, e.g. $x+0\to x$ induces a cycle. By introducing a
topological sorting variable, $t_{c}$ for each e-class $c$, and associated
constraints (4) where $N$ is the number of e-classes and $\mathcal{C}(n)$ is
the e-class containing node $n$, we ensure that the output expression is
acyclic.

$\forall(n,k)\in E\quad t_{\mathcal{C}(n)}-Nx_{n}-t_{k}\geq 1-N$ (4)

If we select a node $n\in\mathcal{N}_{c}$ with child $k$, i.e. $x_{n}=1$, this
constraint simplifies to $t_{c}\geq t_{k}+1$ to get a topologically sorted
result, whereas in the case $x_{n}=0$, the constraint is vacuously satisfied.
We use the open source GLPK solver to calculate solutions to this ILP [21].

We deploy the ILP extraction method in cases where sharing common
subexpressions offers some improvement, otherwise for improved performance we
can resort to the standard egg extraction method [5].

##  IV Results

The RTL test cases are automatically optimized using our egg-based
implementation and optimized RTL is extracted. Original and optimized RTLs are
synthesized using Synopsys Design Compiler for a TSMC 7nm cell library. We
proved the formal equivalence of the original and optimized RTLs using
Synopsys HECTOR technology, a formal equivalence checking tool that runs in
minutes on these testcases.

The original and optimized RTLs were synthesised at the minimum delay of the
slowest of the two, and then similarly at the minimum area of the larger of
the two designs. These represent comparisons towards the endpoints of a
standard area-delay curve for a design. Table III summarises the results. In
Figure 4, we present the area-delay profiles for the competing architectures
of a Smoothing Kernel, which highlights the points of comparison used in Table
III.

We consider two sets of examples. First we demonstrate how the tool exploits
complex datapath blocks to optimize designs. Then we consider bitwidth-
dependent architectures, where the optimal design may vary as a function of
bitwidth.

TABLE III: Logic synthesis results using Synopsys Design Compiler. We compared the automatically optimized results at the minimum delay which both designs could meet and at the smallest area that both designs could meet. Benchmark | Min Achievable Delay | Min Achievable Area
---|---|---
| Delay (ns) | Orig Area ($\mu m^{2}$) | Opt Area ($\mu m^{2}$) | Area ($\mu m^{2}$) | Orig Delay (ns) | Opt Delay (ns)
Smoothing Kernel | 0.289 | 550 | 158 (-71%) | 150 | 1.25 | 0.29 (-77%)
FIR Filter Kernel | 0.611 | 1710 | 679 (-60%) | 570 | 1.38 | 2.48 (+80%)
ADPCM Decoder [22] | 0.102 | 103 | 102 (- 1%) | 32 | 0.72 | 0.45 (-38%)
Shifted FMA | 0.181 | 310 | 210 (-32%) | 81 | 0.97 | 0.57 (-41%)
MCM Solution | 0.132 | 90 | 104 (+15%) | 40 | 0.72 | 0.56 (-22%)
TABLE IV: Benchmark complexity and resulting e-graphs size after rewriting. $\dagger$ means that we removed the associativity of addition rewrite to limit e-graph growth. ILP column indicates whether ILP or standard egg extraction was used. Benchmark | Ops | E-graph Nodes | ILP | Runtime (sec)
---|---|---|---|---
Smoothing Kernel | 17 | 27,000 | No | 140
FIR Filter Kernel $\dagger$ | 23 | 550 | Yes | 100
ADPCM Decoder | 9 | 6,700 | No | 19
Shifted FMA | 3 | 22 | No | 0.04
MCM Solution | 3 | 4,900 | Yes | 31

###  IV-A Datapath Optimizations using Complex Blocks

The first benchmark is a kernel from a media module, which is an industrially
relevant example supplied by Intel. It was manually optimized by a single
engineer over the course of one week. Our tool automatically matches the
results obtained via manual optimization, making use of the “Merge Mult Array”
rewrite. Figure 4 shows the area-delay curves for the original and optimized
architectures. The optimized design performs strictly better, reducing the
minimum achievable delay by 13% with a 28% area reduction.

Figure 4: Area-delay profile of the Smoothing Kernel. The horizontal/vertical
grey lines represents the minimum area/delay comparison points.

Next we have a set of benchmarks taken directly from [4], which are intended
to show how this prior work is generalized by our optimization tool. The first
example is a finite-impulse response (FIR) filter with 8 taps. The second
example, a computational kernel of the ADPCM decoder [22], is an approximation
to a 16$\times$4 multiplier. These two examples are optimized by deploying the
“Arithmetic Logic Exchange” class of rewrites described in Section III-B, to
cluster additions together. The FIR filter example is particularly interesting
as it also introduces a multiple constant multiplication (MCM) problem [7].
Since the ILP formulation of extraction correctly accounts for the cost of
common subexpressions, the minimal area solution it extracts maximally shares
common subexpressions, to generate $\\{2\times x,3\times x,...,7\times x\\}$.
For example, $2\times x$ and $3\times x$ are constructed as follows

$x_{2}=x<<1,\hskip 5.0ptx_{3}=x_{2}+x.$ (5)

Looking more generally at the MCM problem, using the “Constant Expansion”
rewrites, we are able to match the operator count of a solution from [6]. An
example from this paper generates adder graphs to compute $\\{3\times
x,7\times x,21\times x\\}$. The optimal design from our tool maximally shares
common subexpressions to generate the data-flow graph shown in Figure 5, which
uses 3 addition/subtraction operations to generate the results. Since logic
synthesis likely implements constant multiplication using a CSD representation
[8], sharing common subexpressions generates a $15\%$ larger circuit than the
basic architecture for small delay targets as the design does not match the
performance of CSD.

Figure 5: Data-flow graph representing an optimal adder tree to compute the
set $3\times x,7\times x$ and $21\times x$. The blue nodes represent additions
which generate the result in the node label.

Finally we consider a shifted FMA, which discovers the opportunity to merge a
multiplication and addition that were originally separated by a left shift
$(a\times b)<<S+c$. Using the “Left Shift Mult” and “FMA Merge” rewrites this
can be implemented as an FMA block. The approach proposed by Verma, Brisk and
Ienne misses this opportunity since it will not move the shift over the
multiplication [4].

###  IV-B Bitwidth Dependent Optimal Architecture

The last set of results considers parameterizable RTL, as in general, RTL
engineers do not generate alternative architectures for different bitwidths.
As a result, designers may be sacrificing performance. To demonstrate this we
again consider the FIR filter, but the 4 tap variant, where the original
architecture is shown in Figure 6, which we will refer to as Architecture 0.
We optimized Architecture 0 using our tool for increasing bitwidths and
observed that the selected architecture varied between Architectures 0, 1 and
2 (Figure 6). The difference between Architectures 1 and 2 is that the
addition involving $Z4$ in Architecture 1 has been pushed back over the right
shift, incurring the cost of an extra left shift, but saving a full carry-
propagate adder.

The architectural choices are determined by the theoretical two-input gate
cost metric, discussed in Section III-C. We synthesized each of the three
architectures for bitwidths $4,8,...64$, updating the delay target each time
to the minimum delay that all three architectures could meet at that bitwidth.
The theoretically optimal architecture for each bitwidth can be seen in Figure
6. For 56% of the bitwidths, the architecture selected by the theoretical cost
metric gave the lowest area result from logic synthesis. The ability to always
choose the minimum area design according to logic synthesis using the
theoretical metric is limited, since logic synthesis always incorporates delay
considerations into its results and there is noise in the results. We
demonstrate this ‘noise floor’ in Section V.

(a) Architecture 0 {4}(b) Architecture 1 {8,…,28,36,…,48}(c) Architecture 2
{32,52,…,64} Figure 6: Simplified FIR filter data-flow graphs representing
optimal architectures for different choices of the input bitwidth parameter
$p$ and shift bitwidth parameter $q$. The sets in curly braces are bitwidths
for which that architecture is optimal. In these graphs $Zi=Xi\times Ai$ and
$2S$ and $3S$ are computed according to (5).

###  IV-C Performance

For these test cases only the “Shifted FMA” e-graph saturated as we limited
the exploration to 10 iterations of e-graph rewriting [5]. For test cases
using ILP extraction we limited the solver to a 100 second time budget, which
means that the FIR Filter solution is classified as feasible but is possibly
sub-optimal. There is a tradeoff between growing the e-graph and solving the
ILP efficiently. Only the MCM solution included the “Constant Expansion”
rewrite class since these rewrites lead to exponential growth of the e-graph.

##  V Cost Metric Validation

This section evaluates the theoretical cost metric in terms of its ability to
steer optimal architecture choices. To do so we compare the theoretical cost
to logic synthesis area costs. The results from Table III demonstrate that for
all test cases, the theoretically chosen architecture improved at least the
performance or the area of the design.

When determining the accuracy of a cost estimate, it is necessary to consider
inherent variability of the logic synthesis process. Small non-functional
tweaks, e.g. changing a variable name in RTL code, can have impact on the
synthesis results. This forms a ‘noise floor’ against which any theoretical
cost model can be validated. To evaluate this noise floor, we used a technique
known as fuzzing [23], which involves automatically generating random
mutations to a program. We fuzz the RTL allowing two types of semantics-
preserving mutations: variable renaming and swapping the order of
always/assign blocks [19] in the code, modifications which one would not
expect to have a meaningful impact on synthesis results.

We provide results for the Smoothing Kernel and 4 tap FIR Filter, synthesizing
30 fuzzed designs in each case at relevant delay targets. Variability of the
results is shown in Figure 7.

Figure 7: A violin plot depicting the logic synthesis area results for 30
fuzzed designs of the Smoothing Kernel and the FIR Filter at a 0.5ns delay
target. For each violin, the area results are normalised by the mean. Figure
8: Histogram plot of logic synthesis area results for 30 fuzzed designs for
each of the three FIR Filter architectures for 12 bits (Fig 6).

Figure 8 highlights how this noise can affect the architectural choices made
in Section IV-B. For 12 bit inputs the synthesis results for fuzzed
Architectures 0 and 1 overlap, with Architecture 1 generating lower area
results on average whilst Architecture 0 obtains the minimum area result. This
noise is not captured by the theoretical cost metric, as these fuzzed designs
are theoretically identical. Applying this to other bitwidth inputs there are
cases where there is clearly an optimal choice.

##  VI Conclusion

This paper has demonstrated the application of e-graphs and equality
saturation to the RTL datapath optimization problem. Applying a precisely
defined set of bitwidth dependent equivalence preserving transformations, in
the form of rewrites, we efficiently explore the design space and extract
optimized RTL using egg. We also quantify the noise floor in logic synthesis
results to understand the limits of a theoretical cost metric.

The results show that this automated rewriting technique can match the results
of a skilled hardware engineer within a short timescale. The tool was able to
achieve up to 71% area improvement and up to 77% delay improvement. We also
demonstrate that automatic RTL optimization can generate different
architectures for different bitwidth designs, since the tradeoff points are
bitwidth dependent.

Future work will address the limitations of an area-only cost metric by
incorporating delay and power allowing us to generate Pareto optimal solution
curves [24]. We will expand the domain to tackle floating point operations and
incorporate greater support for automated design verification. We will also
address the scalability limits of applying equality saturation by
incorporating intelligent design space search procedures.

## Acknowledgment

The authors would like to thank Yann Herklotz for his assistance in setting up
the fuzzing experiments.

## References

  * [1] Synopsys, “Coding Guidelines for Datapath Synthesis,” Synopsys, Mountain View, Tech. Rep., 12 2019.
  * [2] ——, “Design Compiler User Guide S-2021.06-SP2,” Synopsys, Mountain View, Tech. Rep., 6 2021.
  * [3] A. K. Verma, P. Brisk, and P. Ienne, “Challenges in automatic optimization of arithmetic circuits,” in _Proceedings - IEEE Symposium on Computer Arithmetic_ , 2009, pp. 213–218.
  * [4] ——, “Data-flow transformations to maximize the use of carry-save representation in arithmetic circuits,” _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ , vol. 27, no. 10, pp. 1761–1774, 2008.
  * [5] M. Willsey, C. Nandi, Y. R. Wang, O. Flatt, Z. Tatlock, and P. Panchekha, “Egg: Fast and extensible equality saturation,” _Proceedings of the ACM on Programming Languages_ , vol. 5, no. POPL, 2021.
  * [6] O. Gustafsson, “A difference based adder graph heuristic for multiple constant multiplication problems,” in _IEEE International Symposium on Circuits and Systems_ , 2007, pp. 1097–1100.
  * [7] R. I. Hartley, “Subexpression Sharing in Filters Using Canonic Signed Digit Multipliers,” _IEEE Transactions on Circuits and Systems_ , vol. 11, 1996\.
  * [8] M. D. Ercegovac and T. Lang, _Digital arithmetic_. Elsevier, 2004.
  * [9] C. G. Nelson, “Techniques for program verification,” Ph.D. dissertation, Stanford University, 1980.
  * [10] R. Joshi, G. Nelson, and K. Randall, “Denali: A goal-directed superoptimizer,” in _Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI)_ , 2002.
  * [11] R. Tate, M. Stepp, Z. Tatlock, and S. Lerner, “Equality saturation: A new approach to optimization,” in _ACM SIGPLAN Notices_ , vol. 44, no. 1, 2009\.
  * [12] P. Panchekha, A. Sanchez-Stern, J. R. Wilcox, and Z. Tatlock, “Automatically improving accuracy for floating point expressions,” _ACM SIGPLAN Notices_ , vol. 50, no. 6, pp. 1–11, 2015.
  * [13] G. H. Smith, A. Liu, S. Lyubomirsky, S. Davidson, J. McMahan, M. Taylor, L. Ceze, and Z. Tatlock, “Pure tensor program rewriting via access patterns (representation pearl),” in _Proceedings of the 5th ACM SIGPLAN International Symposium on Machine Programming_ , 2021.
  * [14] Y. R. Wang, S. Hutchison, J. Leang, B. Howe, and D. Suciu, “SPORES: Sum-product optimization via relational equality saturation for large scale linear algebra,” _Proceedings of the VLDB Endowment_ , vol. 13, no. 11, 2020\.
  * [15] C. Wolf and J. Glaser, “Yosys-A Free Verilog Synthesis Suite,” in _Proceedings of Austrochip_ , 2013.
  * [16] G. Steele, _Common LISP: the language_. Elsevier, 1990.
  * [17] A. Beaumont-Smith and C.-C. Lim, “Parallel prefix adder design,” in _Proceedings - IEEE Symposium on Computer Arithmetic_ , 2001, pp. 218–225.
  * [18] I. Koren, _Computer arithmetic algorithms_. AK Peters/CRC Press, 2018.
  * [19] D. Thomas and P. Moorby, _The Verilog® hardware description language_. Springer Science & Business Media, 2008.
  * [20] C. Barrett, P. Fontaine, and C. Tinelli, “The Satisfiability Modulo Theories Library (SMT-LIB),” www.SMT-LIB.org, 2016.
  * [21] A. Makhorin, “GLPK (GNU linear programming kit),” 2008. [Online]. Available: http://www.gnu.org/s/glpk/glpk.html
  * [22] C. Lee, M. Potkonjak, and W. H. Mangione-Smith, “MediaBench: A tool for evaluating and synthesizing multimedia and communications systems,” in _Proceedings of the Annual International Symposium on Microarchitecture_ , 1997.
  * [23] B. P. Miller, L. Fredriksen, and B. So, “An empirical study of the reliability of UNIX utilities,” _Communications of the ACM_ , vol. 33, no. 12, pp. 32–44, 1990.
  * [24] E. Ustun, I. San, J. Yin, C. Yu, and Z. Zhang, “IMpress: Large Integer Multiplication Expression Rewriting for FPGA HLS,” in _2022 IEEE 30th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)_ , 2022, pp. 1–10.


Further author information: (Send correspondence to A.A.A.)
A.A.A.: E-mail<EMAIL_ADDRESS>Telephone: 1 505 123 1234
B.B.A.: E-mail<EMAIL_ADDRESS>Telephone: +33 (0)1 98 76 54 32 *Address
correspondence to Antonia J.F.Hubbard<EMAIL_ADDRESS>
# First operation of Transition-Edge Sensors in space with the Micro-X
sounding rocket

Joseph S. Adams  NASA Goddard Space Flight Center, 8800 Greenbelt Rd,
Greenbelt, Maryland, USA  Robert Baker  NASA Goddard Space Flight Center, 8800
Greenbelt Rd, Greenbelt, Maryland, USA  Simon R. Bandler  NASA Goddard Space
Flight Center, 8800 Greenbelt Rd, Greenbelt, Maryland, USA  Noëmie Bastidon
Northwestern University, 2145 Sheridan Road, Evanston, Illinois, USA  Current
affiliation: The Australian National University, Canberra ACT 0200, Australia
Meredith E. Danowski  Northwestern University, 2145 Sheridan Road, Evanston,
Illinois, USA  Current affiliation: Ball Aerospace, 1600 Commerce Street,
Boulder, Colorado, USA  William B. Doriese  National Institute for Standards
and Technology, 325 Broadway, Boulder, Colorado, USA  Megan E. Eckart
Lawrence Livermore National Laboratory, 7000 East Avenue, Livermore,
California, USA  Enectalí Figueroa-Feliciano  Northwestern University, 2145
Sheridan Road, Evanston, Illinois, USA  Joshua Fuhrman  Northwestern
University, 2145 Sheridan Road, Evanston, Illinois, USA  David C. Goldfinger
Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge,
Massachusetts, USA  Current affiliation: Harvard and Smithsonian Center for
Astrophysics, 60 Garden Street, Cambridge, Massachusetts, USA  Sarah N.T.
Heine  Massachusetts Institute of Technology, 77 Massachusetts Avenue,
Cambridge, Massachusetts, USA  Gene C. Hilton  National Institute for
Standards and Technology, 325 Broadway, Boulder, Colorado, USA  Antonia J.F.
Hubbard  Lawrence Livermore National Laboratory, 7000 East Avenue, Livermore,
California, USA  Daniel Jardin  Northwestern University, 2145 Sheridan Road,
Evanston, Illinois, USA  Richard L. Kelley  NASA Goddard Space Flight Center,
8800 Greenbelt Rd, Greenbelt, Maryland, USA  Caroline A. Kilbourne  NASA
Goddard Space Flight Center, 8800 Greenbelt Rd, Greenbelt, Maryland, USA
Renée E. Manzagol-Harwood  Northwestern University, 2145 Sheridan Road,
Evanston, Illinois, USA  Dan McCammon  University of Wisconsin, 1150
University Avenue, Madison, Wisconsin, USA  Takashi Okajima  NASA Goddard
Space Flight Center, 8800 Greenbelt Rd, Greenbelt, Maryland, USA  Frederick S.
Porter  NASA Goddard Space Flight Center, 8800 Greenbelt Rd, Greenbelt,
Maryland, USA  Carl D. Reintsema  National Institute for Standards and
Technology, 325 Broadway, Boulder, Colorado, USA  Peter Serlemitsos  NASA
Goddard Space Flight Center, 8800 Greenbelt Rd, Greenbelt, Maryland, USA
Stephen J. Smith  NASA Goddard Space Flight Center, 8800 Greenbelt Rd,
Greenbelt, Maryland, USA  Patrick Wikus  Bruker Biospin, Industriestrasse 26,
8117 Fällanden, Switzerland

###### Abstract

With its first flight in 2018, Micro-X became the first program to fly
Transition-Edge Sensors and their SQUID readouts in space. The science goal
was a high-resolution, spatially resolved X-ray spectrum of the Cassiopeia A
Supernova Remnant. While a rocket pointing error led to no time on target, the
data was used to demonstrate the flight performance of the instrument. The
detectors observed X-rays from the on-board calibration source, but a
susceptibility to external magnetic fields limited their livetime. Accounting
for this, no change was observed in detector response between ground operation
and flight operation. This paper provides an overview of the first flight
performance and focuses on the upgrades made in preparation for reflight. The
largest changes have been upgrading the SQUIDs to mitigate magnetic
susceptibility, synchronizing the clocks on the digital electronics to
minimize beat frequencies, and replacing the mounts between the cryostat and
the rocket skin to improve mechanical integrity. As the first flight
performance was consistent with performance on the ground, reaching the
instrument goals in the laboratory is considered a strong predictor of future
flight performance.

###### keywords:

microcalorimeter; transition-edge sensor; sounding rocket; x-ray spectroscopy

##  1 Introduction

Micro-X is a sounding rocket mission designed to obtain high-resolution
spectra of extended X-rays sources. It is also intended to further the
technology readiness of Transition-Edge Sensor (TES) microcalorimeters and
Superconducting QUantum Interference Devices (SQUIDs) for space operation. The
instrument must fly on a sounding rocket because X-rays in the signal bandpass
are attenuated by the atmosphere. Sounding rockets are suborbital missions,
and each flight provides approximately five minutes of observation above an
altitude of 160 km. Sounding rockets provide a good testbed for new
technologies, as with Micro-X, because the instrument can be recovered after
flight, providing the opportunity to modify the instrument for any future
flights. Specialized engineering is required to successfully operate sensitive
detectors, like microcalorimeters, in the vibrationally intense and time-
constrained conditions of a sounding rocket flight [1, 2].

This paper reviews the design (§2) and performance (§3) of the instrument in
the first flight, with particular emphasis on the impact of external magnetic
fields (§3.1) and instrument configuration (§3.2) on performance. The
instrument performance in flight and post-flight testing drove the
modifications that have been implemented in preparation for the reflight (§4).
A comprehensive description of the instrument and first flight performance are
provided in [1].

##  2 First flight instrument

The Micro-X instrument, shown in Figure 1, combines an X-ray mirror and a TES
microcalorimeter array to achieve the instrument specifications provided in
Table 1. Astronomical X-rays are focused by the X-ray mirror onto the detector
array, which sits inside the cryostat. The detectors require a pumped liquid
helium Adiabatic Demagnetization Refrigerator (ADR) to maintain a stable 75 mK
$\pm$ 10 $\upmu$K operating temperature. The on-board electronics maintain the
instrument operating conditions by executing flight operations and running the
control loops for the ADR and SQUIDs.

Figure 1: X-rays enter the Micro-X instrument from the aft side (right in figure). They are focused by the X-ray mirror onto the detector array. The array sits inside the cryostat, with temperature control maintained by the pumped liquid helium ADR. On-board electronics control the state of the system and read out data. Parameter | Specification
---|---
Array size | 7.2$\times$7.2 mm2
Effective area (1 keV) | 300 cm2
Field of view | 11.8’
Angular resolution (HPD) | 2.4 arcminutes
Energy bandpass | 0.3 - 2.5 keV
Quantum efficiency (0.2 - 4.0 keV) |  $\sim$100%
Spectral resolution (0.2 - 3.0 keV) | 4.5 – 11 eV
Table 1: The Micro-X instrument specifications are optimized for high-
resolution observations of extended X-ray sources.

The detector array is comprised of 128 TES microcalorimeter pixels [3, 4].
Microcalorimeters measure X-ray energy by tracking the temperature of an X-ray
absorber. The absorber heats up when hit by an X-ray, then it cools down as
heat dissipates through a weak thermal link into a cold bath. The rise and
decay times of the thermal pulse are engineered when designing the pixel, and
the pulse height (i.e., temperature rise) is proportional to the X-ray energy.

TESs are thin, superconducting films that are biased into the transition
region between their superconducting and normal states. In this regime, a
small change in the temperature of the absorber leads to a large change in
resistance of the TES, providing a high-resolution temperature measurement.
Each Micro-X pixel uses a 140$\times$140 $\upmu$m2 Mo/Au TES to track the
temperature of a 590$\times$590 $\upmu$m2 Bi/Au absorber.

The TES array is read out with SQUIDs, which are low-noise, magnetically
sensitive current amplifiers. A three-stage DC SQUID Time-Division
Multiplexing (TDM) readout scheme amplifies the TES signals to a level
compatible with the room temperature electronics [5, 6, 7]. The TES array is
divided into two independent, 64-pixel TDM readout chains, each with four
“columns” of 16 “rows.” [7] The biases to the first-stage SQUIDs (SQ1) are
cycled sequentially to turn on one row at a time. All 16 SQ1s in a column are
connected in series to a summing coil, which couples the signal from the
biased SQ1 to the second-stage SQUID (SQ2). The signal from the SQ2 is
amplified by the third-stage Series Array (SA), which amplifies the signal by
a factor of 100 before it reaches the room-temperature electronics. The room-
temperature electronics bias each component, process the digital feedback
loops for the SQUIDs, and record the science data.

Figure 2: The FEA holds the TES array and the first two SQUID stages. The GRTs
are used by the ADR to keep the FEA at a stable 75 mK.

The TES array and the first two SQUID stages (SQ1 and SQ2) are kinematically
mounted to the Front End Assembly (FEA), shown in Figure 2. The TES array sits
on a chip in the center of the FEA. The SQ1 and SQ2 are lithographically
fabricated onto a set of multiplexer (MUX) chips. The MUX chips are glued to
an interface (IF) chip, which holds bandwidth-limiting inductors and shunt
resistors. There are two sets of MUX and IF chips, one for each science chain.
The FEA base is a magnesium plate that is thermally connected to the ADR salt
pill to maintain a stable operating temperature. Two Germanium Resistance
Thermometers (GRTs) are used for temperature control and monitoring.

The FEA must be shielded from infrared radiation, vibration, and magnetic
fields. Six infrared blocking filters sit between the FEA and the cryostat
aperture. They minimize the transmission of blackbody radiation from the
warmer stages of the instrument while allowing 78% of X-rays through [1].
Three stages of vibration isolation keep heat from coupling into the detector
thermal stage during flight. These stages include: (1) wire rope isolators
that mount the cryostat to the rocket (44 Hz resonant frequency); (2) G10
plastic tubes that support the liquid helium tank inside the cryostat (96 Hz
resonant frequency); (3) Kevlar suspension that hangs the detectors from the
liquid helium tank (325 Hz resonant frequency) [8]. The increasingly high
frequencies of the inner stages dampen the outer stages for an effective
isolation scheme.

TESs and SQUIDs are sensitive to magnetic fields, and three stages of
shielding minimize the effective field at the detector array, due to both the
Earth’s magnetic field and the ADR magnet. An external field coupling into the
SQUIDs can move their response away from their stable operating point, and
even outside the range of their flux-locked loops. The SQ2 stage is projected
to have a larger magnetic susceptibility than the SQ1 stage due to the
difference in their effective areas (468.5 $\upmu$m2 and 5.0 $\upmu$m2,
respectively) [9]. The first stage of magnetic shielding is a bucking coil
between the ADR magnet and the FEA. The second stage is a superconducting Nb
can that surrounds the FEA to reject external magnetic fields through the
Meissner-Ochsenfeld effect [10]. In addition to the openings for X-ray entry
and the thermally conductive rod to the ADR, the shield has two holes for
harnessing to the FEA. The final stage of shielding is a field coil inside the
Nb shield to counteract any trapped static field. It was not used in the first
flight. The SA are not mounted on the FEA; they are mounted at the 2 K stage,
and a separate Nb box surrounds them. In addition to these shielding layers, a
removable Metglas blanket is wrapped around the cryostat each time it is
cooled down from room temperature. This is done to minimize the amount of the
Earth’s magnetic field that freezes into the Nb shield when it goes
superconducting.

To monitor detector response in flight, a radioactive calibration source
inside the cryostat provides a continuous source of X-rays just outside the
science band. It is mounted on the inside of the Nb can lid, roughly 1 cm from
the detector array. The calibration source uses a ring of 55Fe to illuminate a
KCl scintillator ring, producing Cl and K lines. The source rate in flight was
measured to be $0.7$ counts/pixel/s, corresponding to a K-K$\upalpha$ rate of
0.23 $\pm$ 0.06 counts/pixel/s.

The rocket payload includes both the science instrument and several standard
payload systems that are supplied by the NASA Sounding Rocket Operations
Contract (NSROC) [11]. These systems include the celestial Attitude Control
System (ACS) for pointing, the S-19 boost guidance system for trajectory
control, the Ogive Recovery System Assembly (ORSA) to control the descent, and
three telemetry systems for data transmission.

##  3 Flight performance

Micro-X launched from the White Sands Missile Range in New Mexico at midnight
on July 22, 2018. The science target was the Cassiopeia A Supernova Remnant.
In flight, an ACS failure led the payload to spin and not spend any time on
target. Despite this, the flight data was used to analyze the performance of
the instrument. All flight operations were successfully executed, and the full
data stream was recovered from on-board flash memory. The instrument achieved
a stable 75 mK environment during the science observation, demonstrating the
successful performance of both the ADR and the vibration isolation system.
Despite the lack of a bright astrophysical target, the flight performance of
the detectors was analyzed with X-rays from the calibration source (§3.1–3.2).
Details on the flight performance can be found in [1].

The rocket launched with the instrument in a protected configuration. The ADR
was holding at 300 mK to maximize the available cooling power, and the
instrument was closed off to space. The shutter door to the optics was closed,
as was the aperture valve on the cryostat that exposes the detector array to
the optics. Figure 3 shows that no discernible heat coupled into the detector
stage (top) during powered flight (bottom). Once the payload was through
powered flight, timer commands were sent to the instrument electronics to
prepare for the science observation. The ADR began tight temperature control
at 75 mK, and the apertures to the optics and the cryostat opened to expose
the detector array to space. This was the start of the science observation.
The observation lasted for 320 s, after which point the apertures closed in
preparation for atmospheric re-entry. An apparent heat deposition during re-
entry, likely from a glue failure in the wire rope isolators (§4.2), was
successfully mitigated by the ADR, and tight temperature control was then
maintained until impact. At 4.6 km above the ground, the parachute deployed
for the remainder of the descent.

Figure 3: The thermal performance of the instrument was successful. The
detector stage was launched at 300 mK, and no discernible heat coupled into
the stage during powered flight, which saw 13.4$g$ thrust acceleration on the
exterior of the rocket. Stable temperature regulation of 75 mK$\pm$5 $\upmu$K
was achieved during the science observation.

###  3.1 SQUID magnetic susceptibility

The magnetic susceptibility of the SQUIDs to the Earth’s field was illustrated
using data from powered flight, when the TESs were normal. During this period,
the rocket was spinning in the Earth’s magnetic field, as measured by an on-
board three-axis magnetometer. The SQUIDs response during this stage tracked
the magnetometer reading, demonstrating a magnetic susceptibility, likely at
the SQ2 stage. The SQUIDs were the most susceptible to the Earth’s field in
the direction of the holes in the superconducting Nb shield for FEA
harnessing, shown in Figure 4.

Figure 4: A superconducting Nb magnetic shield surrounds the FEA. The SQUIDs
exhibited the largest magnetic susceptibility in the direction of the holes in
the shielding for the harnessing.

The external magnetic field moved some SQUIDs far enough from their operating
points that they unlocked, meaning that they effectively stopped reading out.
This unlocking is qualitatively consistent with models simulating the response
of the 3-stage SQUID readout to magnetic fields [1]. Under a normal flight
trajectory, the payload would have been effectively stationary during the
science observation. Due to the ACS failure, the payload continued to tumble
throughout the science observation, so the SQUIDs continued to experience a
changing field. As a result of this unlocking, the science observation had an
exposure equivalent to 118 seconds of the full array.

This magnetic susceptibility was confirmed with post-flight Helmholtz coil
testing. The coils were placed along three orthogonal axes around the exterior
of the rocket skin, with a magnetometer recording the applied field. They
simulated spinning in the Earth’s field by sweeping the magnetic field by one
Earth field in either direction. The SAs were not discernibly susceptible to
the external field, indicating successful magnetic shielding at the SAs.
Testing of the entire 3-stage SQUID readout demonstrated that a large fraction
of the SQUIDS reliably unlocked when exposed to simulated spinning. In
addition to the field strength, a strong dependence on the speed of the
magnetic field change (dB/dt) was observed. A gradual progression up to one
Earth field moved the average composite SQUID response by 0.1$\Phi_{0}$, where
$\Phi_{0}$ represents one period in the oscillatory SQUID response. By
contrast, a discrete jump from no field to three Earth fields shifted the
average composite SQUID response by 0.7$\Phi_{0}$. When the Helmholtz coils
returned to zero field, the SQUID response did not return to its original
position, but showed a significantly hysteretic response. This complicates
comparisons to flight data. The susceptibility in the direction of the holes
in the Nb shield was not probed directly; the closest measurement was clocked
22.5∘ from that direction. This was the most susceptible of the measured
directions, consistent with the flight observation that the most susceptible
direction was that of the Nb shield holes.

###  3.2 System noise

Integrating the instrument impacted detector performance, as described in
detail in [1]. The fundamental detector resolution, in ideal laboratory
conditions, was measured to be 4.5 eV. When running with one flight science
chain, the average resolution of across the array was 6.5 eV, with 104 of the
116 active pixels achieving resolutions better than 10 eV. The instrument was
then integrated, which means that the electronics were mounted above the
cryostat and both science chains were turned on. In this configuration, the
average resolution across the array increased to 11 eV, with 41 pixels
achieving resolutions better than 10 eV. Figure 5 shows the difference in the
two configurations for one pulse (YA07), which saw a moderate increase in
system noise. A second pixel (YA08) picked up higher levels of system noise,
with a corresponding degradation in resolution. A significant portion of the
degradation is hypothesized to be from non-stationary noise due to beating
between the clocks of the two science chains. Each science chain received its
own clock from an MV encoder, and these clocks were not synchronized for the
first flight.

The flight performance is consistent with the ground performance in the
integrated configuration, excluding the magnetic field effects described in
§3.1. No change to the noise spectrum or pulse shape was observed between the
two datasets, as shown for a single pixel in Figure 5. Additionally, no change
in the noise spectrum or pulse shape was observed in flight between the array
being open to space and the apertures to space being closed. This implies that
performance improvements implemented on the ground are likely to predict
flight performance.

Figure 5: Pre-flight testing showed changes in detector performance,
presumably driven from beating between the two science chain clocks, once the
instrument was integrated. YA07 shows a moderate change in noise between the
configurations, while YA08 shows significant noise. Post-flight testing
confirmed that synchronizing the science chain clocks mitigates the observed
beating. The pulses have been offset for clarity.

Post-flight noise testing was consistent with the hypothesis that the science
chain clocks were largely responsible for the observed beating. Laboratory
testing used encoder simulators rather than the flight MV encoders,
introducing uncertainty in the predicted flight response based on this
testing. The beating observed in flight was qualitatively reproduced with the
encoder simulators, and it was only observed when both simulators were on. It
was mitigated by feeding a single clock into both simulators, effectively
synchronizing them. The laboratory setup used for this synchronization test
introduced additional noise to the system during the test, so no measurement
was made of the impact of this synchronization on the total noise floor. The
MV encoders have been synchronized for the reflight, as described in §4.2. A
definitive statement on the impact of this synchronization on detector
performance will be made once the payload is fully integrated for the
reflight.

##  4 Second flight system

Modifications to the instrument were motivated by performance in flight and in
post-flight testing. To address the magnetic susceptibility of the SQUIDs, a
newer generation of MUX chips was installed on the FEA, and the material was
changed for both the FEA kinematic mounts and the spacer in the
superconducting Nb magnetic shield (§4.1). To address system noise, the clocks
on both science chains were synchronized, and to avoid thermal failure, the
wire rope isolators were replaced with a higher temperature version of the
same model (§4.2). Each of these modifications has been implemented.

###  4.1 Magnetic Susceptibility Changes

The MUX chips have been upgraded from NIST MUX06a to NIST MUX18b, which have a
demonstrated lower magnetic susceptibilty [12]. The new readout scheme is
shown in Figure 6. It uses flux-activated SQUIDs in the SQ1 stage, rather than
the current-activated SQ1s used in the MUX06a sheme. The TES array is the same
as the first flight. The impact of the new SQUIDs on the noise floor is under
investigation.

In addition to the MUX chip replacement, the kinematic mounts that hold the
TES and SQUID chips were replaced. The first flight kinematic mounts were made
of tungsten carbide, which had become magnetized. The new mounts are made of
sapphire. There is no indication that the old mounts impacted performance, but
the modification was made to prevent any possible impact.

The spacer on the inside of the superconducting Nb magnetic shield was
replaced to improve the efficacy of the shield. The first flight spacer was
made of stainless steel. It was replaced with Pb, which is superconducting at
the instrument’s operating temperature. The Nb is designed cover the entire
spacer, but this replacement will provide a more robust magnetic shield if
there are any gaps in the Nb layer.

Figure 6: The MUX18b readout scheme uses flux-activated switches in the SQ1
stage and has demonstrated lower magnetic susceptibility than the MUX06a flown
in the first Micro-X flight. Image from [12].

These three changes were implemented at the same time, and the combination of
these changes significantly reduced the SQUID susceptibility to external
magnetic fields. The Helmholtz coil testing that unlocked the first flight
SQUIDs in the simulated Earth’s magnetic field did not unlock the new SQUIDs.
Introducing an external field equivalent to one Earth field moved the SQUID
response by 0.016$\Phi_{0}$. The SQUIDs maintained a stable lock throughout
testing. The SQUID response is independent of the rate of the change of the
magnetic field, and no hysteresis is observed. The tests were performed with
the TESs both on and off; no difference was observed between configurations.
If the flight performance continues to follow the ground performance, these
SQUIDs are not projected to unlock in flight.

###  4.2 Additional Changes

The MV encoder clocks were synchronized to eliminate the beating noise that
degraded detector performance in the integrated configuration (§3.2). The
telemetry engineers at the Wallops Flight Facility synchronized the encoders,
and the new system passed handshake testing with the instrument electronics.
Micro-X will be the first program to fly synchronized MV encoders. The impact
of the synchronization on detector performance will be measured once the
payload is integrated for the next flight.

The wire rope isolators that mount the cryostat to the rocket skin experienced
the only mechanical failure of the flight. The glue on the isolators is
presumed to have failed due to its thermal contact with the rocket skin. This
failure was reproduced in thermal testing in the laboratory after flight. The
force required to compress the isolators by 1 mm was measured at 10∘C
intervals from 20–170∘C, and the response was observed to degrade above 60∘C.
In flight, the skin reached 60∘C during powered flight, and it stayed above
this temperature through impact. The isolators were therefore replaced with a
higher-temperature-rated version of the same model to maintain their
effectiveness as part of the vibration isolation scheme. The new isolators
passed thermal testing up to 170∘C. This is acceptable for flight, as the
rocket skin reached a maximum temperature of 110∘C in the first flight.

Figure 7: The new wire rope isolators maintain a constant response up to 170∘C
in laboratory testing. This is a significant improvement over the first flight
isolators’ response, which degraded above 60∘C. The rocket skin reached 110∘C
in flight.

##  5 Conclusions

The maiden flight of Micro-X saw the first operation of TESs and SQUIDs in
space. Despite the ACS failure, the flight provided a measurement of the
instrument flight performance. The flight operations and a number of
instrument systems, notably the cryogenic system, had nominal performance.
These systems are anticipated to fly again without any changes. The instrument
flight performance was impacted by the magnetic susceptibility of the SQUIDs,
increased system noise when the instrument was integrated, and the mechanical
failure of the wire rope isolators. Each of these issues has been addressed by
modifications in preparation for the next flight.

The SQUIDs’ magnetic susceptibility was addressed with a combination of new
SQUIDs and material selection. The MUX chips were upgraded to NIST MUX18b. The
kinematic mounts were converted from tungsten carbide to sapphire, and the
spacer in the superconducting Nb magnetic shield was converted from stainless
steel to lead. These changes were made at the same time, and the combination
of them decreased the magnetic susceptibility to the point that the SQUIDs are
not projected to unlock in future flights. The system noise was addressed by
synchronized the encoder clocks that supply the clocks for the science chains.
The wire rope isolator failure was addressed by replacing them with a higher-
temperature-rated version of the same model. As the flight performance was
consistent with the ground performance in the first flight, the ground
performance following these upgrades is expected to predict future flight
performance. Work is underway to characterize the noise performance of the
system with the new SQUIDs, and the system will then move towards flight
integration.

## References

  * [1] Adams, J. S., Baker, R., Bandler, S. R., Bastidon, N., Castro, D., Danowski, M. E., Doriese, W. B., Eckart, M. E., Figueroa-Feliciano, E., Fuhrman, J., Goldfinger, D. C., Heine, S. N. T., Hilton, G. C., Hubbard, A. J. F., Jardin, D., Kelley, R. L., Kilbourne, C. A., Manzagol-Harwood, R. E., McCammon, D., Okajima, T., Porter, F. S., Reintsema, C. D., Serlemitsos, P., Smith, S. J., and Wikus, P., “First Flight Performance of the Micro-X Microcalorimeter X-Ray Sounding Rocket,” in preparation (2021).
  * [2] McCammon, D., Almy, R., Apodaca, E., Bergmann Tiest, W., Cui, W., Deiker, S., Galeazzi, M., Juda, M., Lesser, A., Mihara, T., Morgenthaler, J. P., Sanders, W. T., Zhang, J., Figueroa-Feliciano, E., Kelley, R. L., Moseley, S. H., Mushotzky, R. F., Porter, F. S., Stahle, C. K., and Szymkowiak, A. E., “A high spectral resolution observation of the soft x-ray diffuse background with thermal detectors,” The Astrophysical Journal 576, 188–203 (2002).
  * [3] Eckart, M. E., Adams, J. S., Bandler, S. R., Brekosky, R. P., Brown, A., Chervenak, J. A., Ewin, A. J., Finkbeiner, F. M., Kelley, R. L., Kilbourne, C. A., Porter, F. S., Sadleir, J. E., Smith, S. J., Figueroa-Feliciano, E., and Wikus, P., “Large-Absorber TES X-ray Microcalorimeters and the Micro-X Detector Array,” in [The Thirteenth International Workshop on Low Temperature Detectors—LTD13 ], Young, B., Cabrera, B., and Miller, A. J., eds., American Institute of Physics Conference Series 1185, 699–702 (Dec. 2009).
  * [4] Eckart, M. E., Adams, J. S., Bandler, S. R., Busch, S. E., Chervenak, J. A., Ewin, A. J., Finkbeiner, F. M., Kelley, R. L., Kilbourne, C. A., Porter, F. S., Porst, J. ., Sadleir, J. E., Smith, S. J., Wassell, E. J., and Figueroa-Feliciano, E., “Development of tes microcalorimeter arrays for the micro-x sounding rocket experiment,” IEEE Transactions on Applied Superconductivity 23(3), 2101705–2101705 (2013).
  * [5] Chervenak, J. A., Grossman, E. N., Irwin, K. D., Martinis, J. M., Reintsema, C. D., Allen, C. A., Bergman, D. I., Moseley, S. H., and Shafer, R., “Performance of multiplexed SQUID readout for Cryogenic Sensor Arrays,” Nuclear Instruments and Methods in Physics Research A 444, 107–110 (Apr. 2000).
  * [6] de Korte, P. A. J., Beyer, J., Deiker, S., Hilton, G. C., Irwin, K. D., Macintosh, M., Nam, S. W., Reintsema, C. D., Vale, L. R., and Huber, M. E., “Time-Division Superconducting Quantum Interference Device Multiplexer for Transition-Edge Sensors,” Review of Scientific Instruments 74, 3807–3815 (Aug. 2003).
  * [7] Reintsema, C. D., Adams, J. S., Baker, R., Bandler, S. R., Doriese, W. R., Figueroa-Feliciano, E., Hilton, G. C., Irwin, K. D., Kelly, R. L., Kilbourne, C. A., Krinsky, J. W., Porter, F. S., and Wikus, P., “Electronics for a Next-Generation SQUID-Based Time-Domain Multiplexing System,” in [The Thirteenth International Workshop on Low Temperature Detectors—LTD13 ], Young, B., Cabrera, B., and Miller, A. J., eds., American Institute of Physics Conference Series 1185, 237–240 (Dec. 2009).
  * [8] Danowski, M. E., Heine, S. N. T., Figueroa-Feliciano, E., Goldfinger, D., Wikus, P., McCammon, D., and Oakley, P., “Vibration Isolation Design for the Micro-X Rocket Payload,” Journal of Low Temperature Physics , 1–7 (2016).
  * [9] Stiehl, G. M., Cho, H. M., Hilton, G. C., Irwin, K. D., Mates, J. A. B., Reintsema, C. D., and Zink, B. L., “Time-Division SQUID Multiplexers With Reduced Sensitivity to External Magnetic Fields,” IEEE Transactions on Applied Superconductivity 21, 298–301 (June 2011).
  * [10] Meissner, W. and Ochsenfeld, R., “Ein neuer effekt bei eintritt der supraleitfähigkeit,” Naturwissenschaften 21(44), 787–788 (1933).
  * [11] SRPO, [NASA Sounding Rockets User Handbook ], NASA Goddard Space Flight Center, Wallops Island, VA (2015).
  * [12] Reintsema, C. D., Bennett, D. A., Denison, E. V., Durkin, M., Doriese, W. B., Fowler, J. W., Gard, J. D., Grigorian, A., Hilton, G. C., Hubmayr, J., O’Neil, G. C., Mates, J. A. B., Morgan, K. M., Schmidt, D. R., Stevens, R. W., Swetz, D. S., Vale, L. R., Ullom, J. N., Irwin, K. D., Chaudhuri, S., Titus, C. J., and Dawson, C. S., “High-throughput, dc-parametric evaluation of flux-activated-switch-based tdm and cdm squid multiplexers,” IEEE Transactions on Applied Superconductivity 29(5), 1–6 (2019).

###### Acknowledgements.

We gratefully acknowledge the technical support of Ernie Buchanan, John
Bussan, Travis Coffroad, Sam Gabelt, Rob Hamersma, Kurt Jaehnig, Frank Lantz,
Ken Simms, Tomomi Watanabe, George Winkert, and the WFF team. Micro-X operates
under NASA Grant 80NSSC18K1445. Part of this work was performed under the
auspices of the U.S. Department of Energy by Lawrence Livermore National
Laboratory under Contract DE-AC52-07NA27344.