# A universal duplication-free quantum neural network

Xiaokai Hou  Institute of Fundamental and Frontier Sciences, University of
Electronic Science and Technology of China, Chengdu, Sichuan, 610051, China
Guanyu Zhou<EMAIL_ADDRESS>Institute of Fundamental and Frontier
Sciences, University of Electronic Science and Technology of China, Chengdu,
Sichuan, 610051, China  Qingyu Li  Institute of Fundamental and Frontier
Sciences, University of Electronic Science and Technology of China, Chengdu,
Sichuan, 610051, China  Shan Jin  Institute of Fundamental and Frontier
Sciences, University of Electronic Science and Technology of China, Chengdu,
Sichuan, 610051, China  Xiaoting Wang<EMAIL_ADDRESS>Institute of
Fundamental and Frontier Sciences, University of Electronic Science and
Technology of China, Chengdu, Sichuan, 610051, China

###### Abstract

Universality of neural networks describes the ability to approximate arbitrary
function, and is a key ingredient to keep the method effective. The
established models for universal quantum neural networks(QNN), however,
require the preparation of multiple copies of the same quantum state to
generate the nonlinearity, with the copy number increasing significantly for
highly oscillating functions, resulting in a huge demand for a large-scale
quantum processor. To address this problem, we propose a new QNN model that
harbors universality without the need of multiple state-duplications, and is
more likely to get implemented on near-term devices. To demonstrate the
effectiveness, we compare our proposal with two popular QNN models in solving
typical supervised learning problems. We find that our model requires
significantly fewer qubits and it outperforms the other two in terms of
accuracy and relative error.

##  1 Introduction

As an important subfield in machine learning(ML), neural networks(NNs),
especially deep NNs, have generated a series of impactful results in many
application scenarios [1, 2, 3, 4]. One of the most striking features of NNs
is their ability to learn the hidden patterns of a given data set and to make
reliable predictions based on these patterns [5]. Such feature originates from
the capability to approximate any continuous function, and is known as the
universality. Most NNs proposed in literature are proved to be universal and
such results are called universal approximation theorems [6, 7]. Due to the
power of quantum computation, the idea of quantum machine learning(QML) is
proposed to implement ML on quantum circuits, in order to achieve
computational advantage compared to the classical counterparts. Such advantage
has been shown for many QML algorithms [8], including quantum support vector
machine [9], $k$-means clustering [10], quantum principle component analysis
[11], quantum data-fitting algorithm [12] and quantum Boltzmann machine [13],
but not for QNNs. It is shown that certain QNNs have a distinctive prediction
advantage on certain designed data sets [14], but less is known for the
general case. In fact, research on the advantage of QNNs is still in progress,
partly due to the reason that even the complexity of classical NN algorithms
has not been addressed without controversy.

Besides the quantum advantage issue, universality is also crucial to keep QNNs
effective. The universality of classical NNs is determined by the nonlinearity
of neurons. When it comes to the QNNs, how to generate nonlinearity is one of
the biggest impediments to achieve the universal QNN. To address the problem,
different QNN models have been proposed such as the continuous variable
quantum neural network [15], the quantum neuron [16, 17], the circuit-centric
quantum classifiers algorithm [18] and the quantum circuit learning algorithm
[19], but not all of them have been rigorously proved to be universal. In some
of these proposals, nonlinearity relies on using multiple copies of the
quantum data, resulting in a rapid increase of the size of the quantum
register. In order to solve this problem, in this work, we propose a
duplication-free QNN structure which also guarantees the universality of the
neural network.

In this work, we aim to construct a universal QNN model without the need of
multiple duplications of quantum data. We design the duplication-free quantum
neural network (DQNN) whose nonlinearity is generated by the classical sigmoid
function. We further compare the DQNN with two well-known QNN models, the
circuit-centric quantum classifiers (CCQ) algorithm [18] and the quantum
circuit learning (QCL) algorithm [19] in terms of the circuit complexity and
the performance on the supervised learning tasks. The results show that the
DQNN with fewer qubits outperforms the other two in terms of accuracy and
relative error. Besides that, the DQNN has the ability to find the complexity
pattern hidden in the real-world data sets and the quantum phase recognition
(QPR) task.

##  2 DQNN and its structure

The universality of DQNN refers to the ability to learn a target function,
$f$, hidden in a given data set $D=\\{(\bm{x}_{i},y_{i})\\}^{m}_{i=1}$ where
$\bm{x}_{i}\in G\subset\mathbb{R}^{d}$ and $y_{i}$ is determined by the target
function with the data noise, $y_{i}=f(\bm{x}_{i})+\epsilon_{i}$. The goal of
the DQNN is to appropriate the function $f$ using $D$. To achieve this goal,
the DQNN uses the structure as shown in Fig. 1. It consists of three parts, a
quantum processor (QP), a classical processor (CP) and a classical optimizer
(CO). QP part is a parameterized quantum circuit and its output is the
expectation values of some measurement observables. CP part contains some
parameterized sigmoid function and a linear transformation. CO minimizes a
loss function by using the gradient of the parameters in QP and CP.

Before implementing the loop of the three parts, we need to encode the
classical data into quantum system using the amplitude encoding method [20].
We firstly find a continuous injection, $F$, mapping $\bm{x}_{i}$ to the
$2^{n}$-dim quantum state Hilbert space $\mathbb{C}_{2}^{\otimes n}$ with
$d<2^{n}$. If $0\notin G$, we can transform the input $\bm{x}$ into
$|\bm{\bar{x}}\rangle\in\mathbb{S}_{0}\subset\mathbb{C}_{2}^{\otimes n}$ as

$F:\bm{x}\in
G\rightarrow|\bm{\bar{x}}\rangle\equiv\frac{1}{\gamma}(x_{1},...,x_{d},\tilde{x},0,...,0)^{T}$
(1)

where $\tilde{x}=\frac{|\bm{x}|}{1+|\bm{x}|}$ and
$\gamma=(|\bm{x}|^{2}+\tilde{x}^{2})^{\frac{1}{2}}$; if $0\in G$, we can
perform a shift transformation, $\bm{x}\rightarrow\bm{x}+\alpha$, such that
$0\notin G$ which is a ring domain
$\\{\bm{x}\in\mathbb{R}^{d}|0<\kappa_{1}\leq|\bm{x}|\leq\kappa_{2}\\}$. It is
worthwhile to mention that $0<\kappa_{1}\leq|\bm{x}|\leq\kappa_{2}$ implies
$\frac{\kappa_{1}}{1+\kappa_{1}}\leq\tilde{x}\leq\frac{\kappa_{2}}{1+\kappa_{2}}$
and

$(1+(1+\kappa_{2})^{2})^{-\frac{1}{2}}<\bar{x}_{d+1}<(1+(1+\kappa_{1})^{2})^{-\frac{1}{2}}$
(2)

with $\bar{x}_{d+1}=\frac{\tilde{x}}{\gamma}$. After obtaining the new data
set $\\{|\bm{\bar{x}}_{i}\rangle,y_{i}\\}$, we implement the loop of QP, CP
and CO to approximate $f$.

Figure 1: (a) The framework of the DQNN. The measurement results of the
quantum processor are the inputs of the classical processor. The parameters
are updated by a classical optimizer. (b) The circuit ansatz used in the
numerical simulation. (c)The circuit structure of DQNN. The directed line
represents the classical information, and the undirected line represents the
quantum information.

The specific structure of QP and CP are shown in Fig. 1. In this paper, the QP
part uses a specific circuit ansatz which is presented in [18]. As shown in
Fig. 1, the circuit ansatz represents the $U(\theta)$ in Fig. 1. It contains
$n$ parameterized single-qubit gates, $G(\theta_{1},\theta_{2},\theta_{3})$,
and $n$ parameterized two-qubit control gates,
$CG(\theta_{1},\theta_{2},\theta_{3}):=|0\rangle\langle
0|\otimes\mathbb{I}+|1\rangle\langle 1|\otimes
G(\theta_{1},\theta_{2},\theta_{3})$ where $G$ is written as

$G(\theta_{1},\theta_{2},\theta_{3})=\left(\begin{matrix}e^{i\theta_{2}}\cos(\theta_{1})&e^{i\theta_{3}}\sin(\theta_{1})\\\
-e^{-i\theta_{3}}\sin(\theta_{1})&e^{-i\theta_{2}}\cos(\theta_{1})\end{matrix}\right).$
(3)

QP outputs the measurement results, $\langle
B_{i}\rangle=\mathrm{Tr}(U(\bm{\theta})|\bar{\bm{x}}\rangle\langle\bar{\bm{x}}|{U(\bm{\theta})}^{\dagger}B_{i})$
with $N$ observables. The number, $N$, depends on the problem itself, and
$\\{B_{i}\\}_{i=1}^{N}$ is a subset of the Pauli basis
$\\{P_{i}\\}_{i=1}^{4^{n}}$.

The CP part applies the parameterized sigmoid function $\sigma^{(i)}$ to each
of $\langle B_{i}\rangle$ where the sigmoid function is defined as

$\sigma^{(i)}(\langle B_{i}\rangle)\equiv\frac{1}{1+\exp\\{-(a^{(i)}(\langle
B_{i}\rangle-c^{(i)}))\\}}$ (4)

with $a^{(i)}>2$ and $c^{(i)}\in[0,1]$. Then CP feeds $\\{\sigma^{(i)}(\langle
B_{i}\rangle)\\}_{i=1}^{N}$ into a classical linear node and results in

$Q(\bar{\bm{x}}):=\sum_{j=1}^{N}\alpha_{j}\sigma(a_{j}(\langle
B_{j}(\bar{\bm{x}},\bm{\theta})\rangle-c_{j})),$ (5)

where $\bm{\alpha}=\\{\alpha_{j}\\}$ are trainable.

The final part of the DQNN, CO, minimizes a loss function
$L(\bm{\theta},\bm{\alpha},\bm{a},\bm{c}):=\sum_{k=1}^{m}||Q(\bar{\bm{x}}_{k})-y_{k}||$
by using some gradient-based methods such as SGD [21], ADAM [22] and BFGS
[23], and obtains the optimal parameters,
$(\bm{\theta}^{*},\bm{\alpha}^{*},\bm{a}^{*},\bm{c}^{*})=\arg\min
L(\bm{\theta},\bm{\alpha},\bm{a},\bm{c})$. The gradient of each parameter is
analytically given below:

$\displaystyle\frac{Q(\bar{\bm{x}})}{\partial\theta_{j}}$
$\displaystyle=\frac{1}{2}\sum_{i}\alpha_{i}a_{i}\sigma^{(i)}(1-\sigma^{(i)})(\langle
B_{i}\rangle_{j}^{+}-\langle B_{i}\rangle_{j}^{-}),$
$\displaystyle\frac{Q(\bar{\bm{x}})}{\partial a_{j}}$
$\displaystyle=\alpha_{j}\sigma^{(j)}(1-\sigma^{(j)})(\langle B_{j}\rangle-
c_{j}),$ $\displaystyle\frac{Q(\bar{\bm{x}})}{\partial c_{j}}$
$\displaystyle=-\alpha_{j}\sigma^{(j)}(1-\sigma^{(j)})a_{j},$
$\displaystyle\frac{Q(\bar{\bm{x}})}{\partial\alpha_{j}}$
$\displaystyle=\sigma^{(j)}.$

where $\sigma^{(i)}$ represents $\sigma(a_{i}(\langle
B_{i}(\bar{\bm{x}},\bm{\theta})\rangle-c_{i}))$ and $\langle
B_{i}\rangle_{j}^{+}$ and$\langle B_{i}\rangle_{j}^{-}$ denotes the
expectation value $\langle B_{i}\rangle$ inserting $\pm\frac{\pi}{2}$ into the
$j$-th quantum parameter $\theta_{j}$ according to the parameter-shift rule
[19].

The basic idea to design such a structure is whenever the QNN is designed by
using the variational quantum circuit, the classical part containing in it can
generate and enhance the nonlinearity of the hybrid system and further
decrease the number of required qubits to achieve the quantum neural network.
In this way, the universality can be proved and the complexity can be reduced.

##  3 Universality of DQNN

The universality of DQNN is guaranteed by the following theorem:

######  Theorem 1.

Given $f(\bar{\bm{x}})\in L^{2}(\mathbb{S}_{0})$, for arbitrary small
$\epsilon$, we can select appropriate $N\in\mathbb{N}$, the unitary
$U(\bm{\theta})$, observables $B_{i}$, and parameters
$\alpha_{i}\in\mathbb{R}$, $a_{i}\in\mathbb{R}_{+}$ and
$c_{i}\in[0,1](i=1,...,N)$ such that

$\int_{\mathbb{S}_{0}}\left|\sum_{i=1}^{N}\alpha_{i}\sigma(a_{i}\langle
B_{i}(\bar{\bm{x}},\bm{\theta})\rangle-
c_{i})-f(\bar{\bm{x}})\right|^{2}d\mu(\bar{\bm{x}})<\epsilon.$ (6)

###### Proof.

Denoting the quantum circuit of the DQNN in Fig. 1 as $U(\bm{\theta})$, which
maps the quantum data $|\bar{\bm{x}}\rangle$ into
$|\bar{\bm{x}}_{f}\rangle=U(\bm{\theta})|\bar{\bm{x}}\rangle$, the output $y$
of DQNN is derived through measuring a set of observables
$\\{B_{i}\\}_{i=1}^{N}$ on the final state $|\bar{\bm{x}}_{f}\rangle$. Based
on measurement statistics, $y=\sum_{i=1}^{N}\alpha_{i}\sigma(a_{i}(\langle
B_{i}(\bar{\bm{x}},\bm{\theta})\rangle-c_{i}))$ is found according to:

$\displaystyle\langle
B_{i}(\bar{\bm{x}},\bm{\theta})\rangle=\langle\bar{\bm{x}}_{f}|B_{i}|\bar{\bm{x}}_{f}\rangle=\sum_{j}\lambda_{i,j}|\langle\bar{\bm{x}}|\bm{\xi}_{i,j}\rangle|^{2}$

where $\lambda_{i,j}$ and $|\bm{b}_{i,j}\rangle$ are the eigenvalues and the
eigenvectors of $B_{i}$, and $|\bm{\xi}_{i,j}\rangle\equiv
U^{{\dagger}}(\bm{\theta})|\bm{b}_{i,j}\rangle$. In the following, we make two
assumptions: the number of observables $N$ is sufficiently large, and the
circuit ansatz, represented as $U(\bm{\theta})$, should have enough
expressibility to approximate any arbitrary unitary evolution. The former is
guaranteed if the local Pauli operators on each qubit can be measured, and the
latter is guaranteed if the set of physically-implementable gates form a
universal gate set. Without loss of generality, we will restrict ourselves to
the case of $\lambda_{i,1}=1$ and $\lambda_{i,j}=0$ ($j\neq 1$) where $\langle
B_{i}\rangle=|\langle\bar{\bm{x}}|\bm{\xi}_{i}\rangle|^{2}$. The output of the
DQNN is the function

$q(\bar{\bm{x}})_{\bm{\alpha,a,\xi_{1},\cdots,\xi_{N},c}}=\sum_{i=1}^{N}\alpha_{i}\sigma(a_{i}(|\langle\bar{\bm{x}}|\bm{\xi}_{i}\rangle|^{2}-c_{i})).$

Denoted by $Q(\mathbb{S}_{0})$, the function space is composed of the finite
linear combination of the sigmoid-type functions:

$\displaystyle
Q(\mathbb{S}_{0})=\Big{\\{}q(\bar{\bm{x}})_{\bm{\alpha,a,\xi_{1},\cdots,\xi_{N},c}}\
:\ N\in\mathbb{N},\bm{\alpha}\in\mathbb{R}^{N},$ (7)
$\displaystyle\quad\bm{a}\in\mathbb{R}_{+}^{N},\bm{c}\in[0,1]^{N},\\{\bm{\xi}_{i}\\}_{i=1}^{N}\subset\mathbb{S}_{\mathbb{C}^{N}}\Big{\\}}.$

To prove the universality, it suffices to verify that $Q(\mathbb{S}_{0})$ is
dense in $L^{2}(\mathbb{S}_{0})$. We assume that the closure
$\overline{Q(\mathbb{S}_{0})}\neq L^{2}(\mathbb{S}_{0})$. The contradiction is
shown in supplemental material. ∎

One can see that the two above assumptions are crucial for the validity of the
proof. In order to satisfy them, the circuit can become very long and is
required to be repeated many times to derive the measurement outcomes for all
$B_{i}$. In theory, the classical NN requires infinite neurons in one layer to
be universal [6]. However, in practice, the classical case only uses a finite
number of neurons and has obtained excellent results in various aspects.
Analogously, for a given data set, a chosen DQNN structure, a finite number of
observables in DQNN are often sufficient to approximate the target function
well, and this will be demonstrated in the following examples. In addition,
the length of the DQNN circuit depends on the given data set, and it may
become very long for a special data set. In fact, it is an important unsolved
question on how exactly the complexity of the DQNN circuit depends on the data
set.

##  4 Circuit complexity for DQNN

The advantage of DQNN is introducing the classical sigmoid function to
generate the nonlinearity and significantly reduces the circuit complexity
compared with two duplication-based QNNs, quantum circuit learning(QCL)
algorithm [19] and circuit-centric quantum classifiers(CCQ) algorithm [18].
The circuit complexity to implement the QNNs is given as $C:=O(n_{g}n_{b})$
where $n_{g},n_{b}$ respectively denote the number of quantum gates and the
number of measurement observables. Without loss of generality, we assume the
number of gates is polynomial to the number of required qubits.

To show the difference of complexity among QNN, QCL and CCQ, we concentrate on
a polynomial function approximation problem whose goal is approximating an
$M$-order polynomial function of $\bm{x}\in\mathbb{R}^{d}$. The CCQ stores
$\bm{x}$ into the amplitudes of data qubits as
$|\bm{x}\rangle=\frac{1}{||\bm{x}||}\sum_{i=1}^{d}x_{i}|i\rangle$ and needs
$O(M)$ copies of $|\bm{x}\rangle$. After applying $n_{g}=O(poly(M\lceil\log
d\rceil))$ gates, $O(1)$ POVM operators is used to measure the system. Its
complexity is $C_{\text{CCQ}}=O(poly(M\lceil\log d\rceil))$. In the meanwhile,
the QCL encodes $\bm{x}$ as $\rho^{\otimes
M}(\bm{x})=\frac{1}{2^{d}}\otimes_{i=1}^{d}\left(\otimes_{k=1}^{M}[\mathcal{I}+x_{i}\sigma_{x}^{(k)}+\sqrt{1-x_{i}^{2}}\sigma_{z}^{(k)}]\right)$
using $M$ copies of the data qubits $\rho(\bm{x})$. QCL uses $O(1)$ Pauli
operators to measure the circuit. Therefore, its complexity is
$C_{\text{QCL}}=O(poly(Md))$. Because the classical sigmoid function makes
DQNN need no duplication and the encoding method is the same as CCQ, the DQNN
only needs $n_{g}=O(poly(\lceil\log d\rceil))$ gates. Moreover, the number of
observables is independent to the number of required qubits but a
hyperparameter determined by the problem. The complexity of DQNN is
$C_{\text{DQNN}}=O(poly(\lceil\log d\rceil))$. The comparison is summarized in
Table. 1. It can be seen that DQNN efficiently reduces the number of required
qubits compared with CCQ and QCL.

Algorithm | # Duplication | # Data qubits
---|---|---
QCL | $M$ | $d$
CCQ | $M$ | $\lceil\log d\rceil$
DQNN | $1$ | $\lceil\log d\rceil$
Table 1: The number of required qubits among three proposals to approximate an
$M$-order polynomial function of $\bm{x}\in\mathbb{R}^{d}$.

Regression Task
---
Algorithm | Qubits Number | Layers Number | Copies Number | $c$ | Mean Relative Error
DQNN | 2 | 1 | 1 | 120 | 4.29%
QCL | 2 | 2 | 1 | 24 | 65.27%
QCL | 4 | 3 | 2 | 144 | 50.58%
QCL | 6 | 6 | 3 | 648 | 87.51%
Classification Task
Algorithm | Qubits Number | Layers Number | Copies Number | $c$ | Accuracy
DQNN | 2 | 1 | 1 | 240 | 97.63%
CCQ | 2 | 3 | 1 | 78 | 56%
CCQ | 4 | 3 | 2 | 300 | 81.25%
CCQ | 6 | 6 | 3 | 1674 | 82.63%

Table 2: The running result and specific setup of DQNN, QCL and QCL. $c$ is a
specific value of the circuit complexity which is calculated by
$c=n_{g}*n_{b}$.

Task | n | $N_{train}$ | $N_{test}$ | $\epsilon_{Train}$ | $\epsilon_{Test}$
---|---|---|---|---|---
MNIST2 | $8$ | $12665$ | $2115$ | $0.0047$ | $0.0009$
MNIST3 | $8$ | $18623$ | $3147$ | $0.0172$ | $0.0114$
Wine | $4$ | $143$ | $35$ | $0.0000$ | $0.0286$
Breast Cancer | $4$ | $560$ | $39$ | $0.0143$ | $0.0432$

Table 3: Implement the DQNN on the real-world data sets. We use ADAM algorithm
to optimize the parameters. $n$ indicates the number of qubits, $N_{Train}$
and $N_{Test}$ respectively represent the number of the training set and the
test set. $\epsilon_{Train}$ and $\epsilon_{Test}$ represent the errors on
each data set.

##  5 Applications

We design two data sets, regression and classification data sets, to show the
advantage of the DQNN compared with QCL and CCQ. The regression data set (Fig.
2) contains $400$ data samples which are randomly generated by
$f=(0.715625-1.0125x_{1}^{2}+x_{1}^{4})(0.715625-1.0125x_{2}^{2}+x_{2}^{4})$
with $x_{1},x_{2}\in[-0.8,0.8]$. The classification data set (Fig. 2) has
$800$ data samples where the boundaries are generated by
$x_{1}^{2}+x_{2}^{2}=0.16$ and $x_{1}^{2}+x_{2}^{2}=0.81$ with
$x_{1},x_{2}\in[-1,1]$. In the donut-like area, the data samples are labeled
$y^{(i)}=[1,0]^{T}$ and others are $y^{(i)}=[0,1]^{T}$. We use different
numbers of duplications and layers in QCL and CCQ. The DQNN uses $5$ and $10$
randomly generated observables respectively in the regression and
classification task. The simulation results (Table. 3) show that the
performances of DQNN are better than other proposals which complexity is
around $5$ times larger both in regression and classification. With the
increasing of the number of copies and layers, CCQ in the classification task
shows underfitting which leads to a lower accuracy.

Figure 2: $(a)$ The regression data set with a polynomial function. $(b)$ The
classification data set with two circular decision boundaries.

Additionally, in order to compare with the classical counterpart, we implement
the classical neural network (NN) based on above two tasks to compare the
performance with QNN. The results show that with similar number of parameters,
the DQNN has the same power as its classical analogue. In the regression task,
the training process (Fig. 3) shows that both proposals have the similar rate
of convergence and the mean relative error on NN is $4.19\%$ which is similar
with DQNN. As for the classification task, the training process with the
accuracy shown in. Fig. 3 demonstrates that the neural network with the
similar number of parameters achieves $74.5\%$ (NNv2). Meanwhile, the
classical neural network which contains three times more parameters than DQNN
can achieves the similar accuracy as DQNN.

To verify the power of DQNN on the real-world data sets, we further implement
some classification tasks. Firstly, on the handwritten digits data set, MNIST
[24], we choose $0$ and $1$ for a binary classification and $0$, $1$, $2$ for
a multi-target classification on MNIST. Each picture is reshaped into
$16\times 16$ and encoded into $8$ qubits. Additionally, on the Wine and
Breast Cancer data sets [25], we randomly divide the data sets into five
equivalent part and pick one of them as the test set. The result (Table. 3)
shows the DQNN has the ability to find the complexity pattern hidden in the
real-world data sets.

Besides the classical tasks above, DQNN provides the ability to investigate
the intrinsic property of quantum mechanics such as the quantum phase
recognition (QPR). Specifically, we apply the DQNN to the
$\mathbb{Z}_{2}\times\mathbb{Z}_{2}$ symmetry-protected topological (SPT)
phase discrimination task [26]. The ground states of a parameterized
spin$-1/2$ chain Hamiltonian,

$\displaystyle H=$
$\displaystyle-J\sum_{i=1}^{N-2}\sigma_{z}^{(i)}\sigma_{x}^{(i+1)}\sigma_{z}^{(i+2)}-h_{1}\sum_{i=1}^{N}\sigma_{x}^{(i)}$
$\displaystyle-h_{2}\sum_{i=1}^{N-1}\sigma_{x}^{(i)}\sigma_{x}^{(i+1)}$

where $h_{1}$, $h_{2}$ and $J$ are parameters, corresponds to the different
topological phases. The phase diagram of the Hamiltonian is given in Fig. 3.
The train data takes $400$ equally spaced points from $h_{1}\in[0,1.6]$ and
$h_{2}\in[-1.6,1.6]$. And the test data contains $4096$ equally spaced points.
The ground state corresponding to each point is labeled $[1,0]^{T}$, if it
belongs to the SPT phase. Otherwise it is labeled $[0,1]^{T}$. We numerically
implement the DQNN with $15$ qubits, $420$ parameters and $10$ observables.
The accuracy on the test data achieves $99.10\%$. It shows the DQNN could find
the relation between the ground states of the Hamiltonian and their
corresponding phase.

Figure 3: $(a)$ The MSE loss with training episodes of QNN and classical
neural network. $(b)$ The classification accuracy with training episodes of
QNN and classical neural network. $(c)$ The phase diagram of the spin$-1/2$
chain. The phase boundary is generated by the 2-degree polynomial regression
based on some boundary points.

##  6 Conclusion

In this article, we present the universal duplication-free quantum neural
network whose nonlinearity is generated by the classical sigmoid function. The
simulation results show that the DQNN significantly reduces the number of
required qubits to complete the supervised learning tasks compared with
previous work and has the ability to recognize the SPT phase of a spin-$1/2$
chain Hamiltonian. However, how to design an appropriate circuit ansatz for a
certain problem still remains an open question. Besides the scenarios
discussed in this work, we expect the duplication-free quantum neural network
has broad applications in other area, including natural language processing,
computer version and reinforcement learning.

##  7 Acknowledgement

The authors gratefully acknowledge the grant from National Key R&D Program of
China, Grant No. 2018YFA0306703. We also thank Chu Guo, Bujiao Wu, Yusen Wu,
Shaojun Wu, Yuhan Huang, Dingding Wen and Yi Tian for helpful discussions.

## References

  * Kalchbrenner et al. [2014] Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom.  A convolutional neural network for modelling sentences.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics_ , volume 1, pages 655–665. Association for Computational Linguistics, 2014.  doi: https://doi.org/10.3115/v1/P14-1062.
  * Mikolov et al. [2011] T. Mikolov, S. Kombrink, L. Burget, J. Černocký, and S. Khudanpur.  Extensions of recurrent neural network language model.  In _2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 5528–5531, 2011.  doi: 10.1109/ICASSP.2011.5947611.
  * Bojarski et al. [2016] Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al.  End to end learning for self-driving cars.  _arXiv preprint arXiv:1604.07316_ , 2016.  URL https://arxiv.org/abs/1604.07316.
  * Bishop [2006] Christopher M Bishop.  _Pattern recognition and machine learning_.  springer, 2006.  doi: https://doi.org/10.1007/978-1-4615-7566-5.
  * Ian et al. [2016] Goodfellow Ian, Bengio Yoshua, and Courville Aaron.  _Deep learning_ , volume 1.  MIT Press, 2016.  URL https://books.google.co.in/books?id=Np9SDQAAQBAJ.
  * Cybenko [1989] G. Cybenko.  Approximation by superpositions of a sigmoidal function.  _Mathematics of Control, Signals, and Systems_ , 2(4):303–314, 1989.  doi: https://doi.org/10.1007/BF02551274.
  * Hornik [1991] Kurt Hornik.  Approximation capabilities of multilayer feedforward networks.  _Neural networks_ , 4(2):251–257, 1991.  doi: https://doi.org/10.1016/0893-6080(91)90009-T.
  * Biamonte et al. [2017] Jacob Biamonte, Peter Wittek, Nicola Pancotti, Patrick Rebentrost, Nathan Wiebe, and Seth Lloyd.  Quantum machine learning.  _Nature_ , 549(7671):195–202, 2017.  doi: https://doi.org/10.1038/nature23474.
  * Rebentrost et al. [2014] Patrick Rebentrost, Masoud Mohseni, and Seth Lloyd.  Quantum support vector machine for big data classification.  _Physical review letters_ , 113(13):130503, 2014\.  doi: 10.1103/PhysRevLett.113.130503.
  * Lloyd et al. [2013] Seth Lloyd, Masoud Mohseni, and Patrick Rebentrost.  Quantum algorithms for supervised and unsupervised machine learning.  _arXiv preprint arXiv:1307.0411_ , 2013.  URL https://arxiv.org/abs/1307.0411.
  * Lloyd et al. [2014] Seth Lloyd, Masoud Mohseni, and Patrick Rebentrost.  Quantum principal component analysis.  _Nature Physics_ , 10(9):631–633, 2014.  doi: https://doi.org/10.1038/nphys3029.
  * Wiebe et al. [2012] Nathan Wiebe, Daniel Braun, and Seth Lloyd.  Quantum algorithm for data fitting.  _Physical review letters_ , 109(5):050505, 2012\.  doi: 10.1103/PhysRevLett.109.050505.
  * Amin et al. [2018] Mohammad H. Amin, Evgeny Andriyash, Jason Rolfe, Bohdan Kulchytskyy, and Roger Melko.  Quantum boltzmann machine.  _Phys. Rev. X_ , 8:021050, May 2018.  doi: 10.1103/PhysRevX.8.021050.
  * Huang et al. [2021] Hsin-Yuan Huang, Michael Broughton, Masoud Mohseni, Ryan Babbush, Sergio Boixo, Hartmut Neven, and Jarrod R. McClean.  Power of data in quantum machine learning.  _Nature Communications_ , 12(1), 2021.  ISSN 2041-1723.  doi: https://doi.org/10.1038/s41467-021-22539-9.
  * Killoran et al. [2019] Nathan Killoran, Thomas R Bromley, Juan Miguel Arrazola, Maria Schuld, Nicolás Quesada, and Seth Lloyd.  Continuous-variable quantum neural networks.  _Physical Review Research_ , 1(3):033063, 2019\.  doi: 10.1103/PhysRevResearch.1.033063.
  * Ventura and Martinez [1998] Dan Ventura and Tony Martinez.  An artificial neuron with quantum mechanical properties.  In _Artificial Neural Nets and Genetic Algorithms_ , pages 482–485. Springer, 1998.  doi: https://doi.org/10.1038/s41534-019-0140-4.
  * Cao et al. [2017] Yudong Cao, Gian Giacomo Guerreschi, and Alán Aspuru-Guzik.  Quantum neuron: an elementary building block for machine learning on quantum computers.  _arXiv preprint arXiv:1711.11240_ , 2017.  URL https://arxiv.org/abs/1711.11240.
  * Schuld et al. [2020] Maria Schuld, Alex Bocharov, Krysta M Svore, and Nathan Wiebe.  Circuit-centric quantum classifiers.  _Physical Review A_ , 101(3):032308, 2020.  doi: 10.1103/PhysRevA.101.032308.
  * Mitarai et al. [2018] Kosuke Mitarai, Makoto Negoro, Masahiro Kitagawa, and Keisuke Fujii.  Quantum circuit learning.  _Physical Review A_ , 98(3):032309, 2018.  doi: https://doi.org/10.1103/PhysRevA.98.032309.
  * Plesch and Brukner [2011] Martin Plesch and Časlav Brukner.  Quantum-state preparation with universal gate decompositions.  _Phys. Rev. A_ , 83:032302, Mar 2011.  doi: 10.1103/PhysRevA.83.032302.
  * Bottou [2010] Léon Bottou.  Large-scale machine learning with stochastic gradient descent.  In Yves Lechevallier and Gilbert Saporta, editors, _Proceedings of COMPSTAT’2010_ , pages 177–186. Physica-Verlag HD, 2010.  doi: 10.1007/978-3-7908-2604-3_16.
  * Kingma and Ba [2015] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , 2015.  doi: 10.1080/10556788.2019.1649672.
  * Buckley and LeNir [1985] Albert Buckley and A LeNir.  Algorithm 630: Bbvscg–a variable-storage algorithm for function minimization.  _ACM Transactions on Mathematical Software (TOMS)_ , 11(2):103–119, 1985.  doi: 10.1145/363219.363231.
  * LeCun and Cortes [2010] Yann LeCun and Corinna Cortes.  Mnist handwritten digit database.  2010\.  URL http://yann.lecun.com/exdb/mnist/.
  * Dua and Graff [2017] Dheeru Dua and Casey Graff.  UCI machine learning repository, 2017.  URL http://archive.ics.uci.edu/ml.
  * Cong et al. [2019] Iris Cong, Soonwon Choi, and Mikhail D Lukin.  Quantum convolutional neural networks.  _Nature Physics_ , 15(12):1273–1278, 2019.  doi: 10.1038/s41567-019-0648-8.
  * Hahn [1927] Hans Hahn.  Über lineare gleichungssysteme in linearen räumen.  _Journal für die reine und angewandte Mathematik_ , 1927(157):214–229, 1927.  doi: https://doi.org/10.1515/crll.1927.157.214.
  * Fréchet [1907] Maurice Fréchet.  Sur les ensembles de fonctions et les opérations linéaires.  _CR Acad. Sci. Paris_ , 144:1414–1416, 1907.  doi: https://doi.org/10.1090/S0002-9947-1904-1500687-0.

##  Appendix A The proof of Theorem 1

It suffices to show that $Q(\mathbb{S}_{0})$ is dense in
$L^{2}(\mathbb{S}_{0})$. We assume that the closure
$\overline{Q(\mathbb{S}_{0})}\neq L^{2}(\mathbb{S}_{0})$ and show the
contradiction. By the Hahn-Banach theorem [27], there exists a bounded linear
functional $\mathcal{L}$ of $L^{2}(\mathbb{S}_{0})$ such that
$\mathcal{L}(Q(\mathbb{S}_{0}))=0$ and $\mathcal{L}\neq 0$. By the Riesz
representation theorem [28], there exists a function $g(\bar{\bm{x}})\in
L^{2}(\mathbb{S}_{0})$ such that

$\mathcal{L}(f)=\int_{\mathbb{S}_{0}}f(\bar{\bm{x}})g(\bar{\bm{x}})d\mu(\bar{\bm{x}})\quad\text{for
all}f\in L^{2}(\mathbb{S}_{0})$ (8)

where $\mathcal{L}\neq 0$ implies that $g(\bar{\bm{x}})\neq 0$. Since
$\mathcal{L}(Q(\mathbb{S}_{0}))=0$, we have

$\int_{\mathbb{S}_{0}}\sigma(a(|\langle\bar{\bm{x}}|\bm{\xi}\rangle|^{2}-c))g(\bar{\bm{x}})d\mu(\bar{\bm{x}})=0$
(9)

In particular, there exists an open subset $E\subset\mathbb{S}_{0}$ with the
measure $\mu(E)>0$ such that $g(\bar{\bm{x}})\neq 0$ in $E$. Without loss of
generality, we assume $g(\bar{\bm{x}})\geq k>0$ in $E$. Since $E$ is open,
there exists a small ball
$B(\bm{\xi}^{*},\delta)=\\{\bar{\bm{x}}:|\bar{\bm{x}}-\bm{\xi}^{*}|<\delta\\}\subset
E$.

For $\bm{\xi}^{*}$, $\bar{\bm{x}}\in\mathbb{S}_{\mathbb{R}^{N}}$, we have

$|\bar{\bm{x}}-\bm{\xi}^{*}|^{2}=2-2\langle\bar{\bm{x}}|\bm{\xi}^{*}\rangle.$
(10)

Therefore $|\langle\bar{\bm{x}}|\bm{\xi}^{*}\rangle|^{2}>c$ is equivalent to

$|\bar{\bm{x}}-\bm{\xi}^{*}|^{2}<2(1-\sqrt{c})\quad\text{or}\quad|\bar{\bm{x}}-\bm{\xi}^{*}|^{2}>2(1+\sqrt{c}).$
(11)

We claim that $|\bar{\bm{x}}-\bm{\xi}^{*}|^{2}>2(1+\sqrt{c})$ is impossible to
hold if $c$ is closed to $1$. Or else, by using

$|\bar{\bm{x}}-\bm{\xi}^{*}|^{2}+|\bar{\bm{x}}+\bm{\xi}^{*}|^{2}=4$ (12)

we find that $|\bar{\bm{x}}+\bm{\xi}^{*}|^{2}<2(1-\sqrt{c})$. From Eqn. (2)
and $\bm{\xi}^{*}$, $\bar{\bm{x}}\in\mathbb{S}_{0}$, we see that
$|\bar{\bm{x}}+\bm{\xi}^{*}|^{2}\geq(\xi^{*}_{d+1}$ \+ $\bar{x}_{d+1})^{2}\geq
4(1+(1+\kappa_{2})^{2})^{-1}$. Hence for
$c>(1-2(1+(1+\kappa_{2})^{2})^{-1})^{2}$, the latter case of Eqn. (11) makes
no sense, and $|\langle\bar{\bm{x}}|\bm{\xi}^{*}\rangle|^{2}>c$ is only
equivalent to

$|\bar{\bm{x}}-\bm{\xi}^{*}|^{2}<2(1-\sqrt{c})\quad\forall\bm{\xi}^{*},\bar{\bm{x}}\in\mathbb{S}_{0}.$
(13)

Therefore, passing to the limit $a\rightarrow\infty$ we obtain

$\sigma(a(|\langle\bar{\bm{x}}|\bm{\xi}^{*}\rangle|^{2}-c))\rightarrow\left\\{\begin{aligned}
&1\quad\forall\bar{\bm{x}}\in B(\bm{\xi}^{*},\delta_{1})\\\
&0\quad\forall\bar{\bm{x}}\notin
B(\bm{\xi}^{*},\delta_{1})\end{aligned}\right.$ (14)

with $\delta_{1}=(2(1-\sqrt{c}))^{\frac{1}{2}}$. By taking $c$ sufficiently
close to 1 such that $\delta_{1}\leq\delta$, and using Lebesgue dominate
convergence theorem, from Eqn.(B13) we obtain

$\displaystyle 0$
$\displaystyle=\int_{\mathbb{S}_{0}}\sigma(a(|\langle\bar{\bm{x}}|\bm{\xi}\rangle|^{2}-c))g(\bar{\bm{x}})d\mu(\bar{\bm{x}})$
(15) $\displaystyle\geq
k\int_{B(\bm{\xi}^{*},\delta_{1})}\sigma(a(|\langle\bar{\bm{x}}|\bm{\xi}\rangle|^{2}-c))d\mu(\bar{\bm{x}})$
(16) $\displaystyle\rightarrow
k\int_{B(\bm{\xi}^{*},\delta_{1})}1d\mu(\bar{\bm{x}})=k\mu(B(\bm{\xi}^{*},\delta_{1}))>0$
(17)

which comes out a contradiction. Hence, we conclude that $Q(\mathbb{S}_{0})$
is dense in $L^{2}(\mathbb{S}_{0})$. Thus for any $f\in L^{2}(\mathbb{S}_{0})$
and $\epsilon>0$, we can find a $q(\bar{\bm{x}})\in Q(\mathbb{S}_{0})$ such
that

$||f-q(\bar{\bm{x}})||^{2}_{L^{2}(\mathbb{S}_{0})}=\int_{\mathbb{S}_{0}}|f(\bar{\bm{x}})-q(\bar{\bm{x}})|^{2}d\mu(\bar{\bm{x}})\leq\epsilon$
(18)

which proves the theorem.


# PowerGAN: A Machine Learning Approach for Power Side-Channel Attack on
Compute-in-Memory Accelerators

Ziyu Wang, , Yuting Wu, Yongmo Park, Sangmin Yoo, Xinxin Wang,
Jason K. Eshraghian, , and Wei D. Lu Z. Wang, Y. Wu, Y. Park, S. Yoo, X. Wang,
J. K. Eshraghian and W. D. Lu are with the Department of Electrical
Engineering and Computer Science, the University of Michigan, Ann Arbor, MI,
48109, USA.
J. K. Eshraghian is now with Department of Electrical and Computer
Engineering, University of California, Santa Cruz, CA 95064, USA.
Corresponding Author: Wei D. Lu<EMAIL_ADDRESS>
###### Abstract

Analog compute-in-memory (CIM) systems are promising for deep neural network
(DNN) inference acceleration due to their energy efficiency and high
throughput. However, as the use of DNNs expands, protecting user input privacy
has become increasingly important. In this paper, we identify a potential
security vulnerability wherein an adversary can reconstruct the user’s private
input data from a power side-channel attack, under proper data acquisition and
pre-processing, even without knowledge of the DNN model. We further
demonstrate a machine learning-based attack approach using a generative
adversarial network (GAN) to enhance the data reconstruction. Our results show
that the attack methodology is effective in reconstructing user inputs from
analog CIM accelerator power leakage, even at large noise levels and after
countermeasures are applied. Specifically, we demonstrate the efficacy of our
approach on an example of U-Net inference chip for brain tumor detection, and
show the original magnetic resonance imaging (MRI) medical images can be
successfully reconstructed even at a noise-level of 20% standard deviation of
the maximum power signal value. Our study highlights a potential security
vulnerability in analog CIM accelerators and raises awareness of using GAN to
breach user privacy in such systems.

###### Index Terms:

side-channel attack, compute-in-memory, machine learning, generative
adversarial network, memristor, deep learning security

##  1 Introduction

Machine learning, especially deep neural networks (DNNs), are being used in a
broad range of applications, including language processing, computer vision,
speech recognition and financial analysis [1][2][3][4]. However, the state-of-
the-art DNN models require significant amounts of data movement between memory
and processor during both training and inference, leading to severe memory
bottleneck effects [5] [6] [7]. Compute-in-memory (CIM) architectures
accelerate inference by mapping the pre-trained weights on chip and processing
locally in memory, and can thus effectively address the memory bottleneck and
result in high energy efficiency and throughput [8] [9] [10]. A typical CIM
implementation employs an array of programmable memristive elements, e.g.
resistive random-access memory (RRAM), to store the weight values as
conductance values and perform vector-matrix multiplication (VMM) through
Ohm’s law and Kirchhoff’s current law in place and in parallel [11] [12].

Although DNN models are implemented on a wide range of platforms, from
datacenter GPUs to edge devices, the importance of data security cannot be
overstated. Attacks on DNN data used in critical applications such as medical
diagnosis, autonomous driving, and financial transactions can compromise the
user privacy as well as proprietary algorithm information [13]. The main areas
of concern in DNN security include model extraction, adversarial attacks, and
privacy breaches [13] [14] [15]. Model extraction involves extracting the DNN
architecture and reconstruct weights to reproduce its functionality, which can
be used to breach the intellectual property of DNN design [14]. Adversarial
attacks involve manipulating the input data deliberately to make the
modification invisible to human but causing misclassification or other
unexpected behavior to the DNN model [15]. Privacy breaches involve
reconstructing sensitive private input or training data by an attacker [13]
[16].

Figure 1: (a) Schematic of the U-Net model architecture for segmentation. It
is composed of a down sampling-encoder on the left and an up sampling-decoder
on the right forming a U-shaped structure. The skip connections between the
encoder and decoder layers are formed by concatenating the feature maps from
the encoder layers to the corresponding decoder layers. (b) An example brain
MRI image with a tumor. (c) The segmentation result from U-Net. The ground
truth and prediction are displayed in green and red, respectively. (d) A CIM
chip and the system board. An adversary can potentially perform power side
channel attacks through power trace measurements on the chip’s power lines.
(e) A representative power trace of a CIM accelerator at inference runtime.
(f-g) Examples of a user’s private input sent to the CIM inference accelerator
(f) and reconstructed image from power-side channel attacks (g).

While the abovementioned security attacks on DNNs have been extensively
evaluated on systems such as GPUs, CPUs and FPGAs [14] [16] [17] [18] [19]
[20], security and vulnerability analysis of analog CIM accelerators is
largely lacking. Recently, Wang et al. [21] proposed a side-channel attack
methodology to extract the DNN architecture from analog RRAM-based CIM
systems. Read et al. [22] reverse engineered the weights and biases of DNN
models mapped on analog CIM systems. Cherupally et al. [23] studied
adversarial attacks on analog RRAM-based CIM systems. As a potential solution
for prevalent low-power edge-based computing, understanding security
vulnerabilities of CIM systems including data privacy becomes paramount.

In this study, we propose a machine learning-based approach to reconstruct
users’ private input data from power side-channel profiling of CIM systems,
without prior knowledge of the DNN model mapped on chip. By scrutinizing the
leaked power data features, we identified the security issue of data
dependency between power side-channel leakage and private input data. We
demonstrated a flexible and automated attack approach based on a generative
adversarial network (GAN). The GAN-assisted side-channel attack was shown to
be effective even with large noise levels and countermeasure implementations.
We evaluated the proposed approach on a human brain magnetic resonance imaging
(MRI) dataset [24], which includes detailed features of sulci and gyri in
cerebral cortex that can be difficult to reconstruct. In contrast to previous
studies on analog CIM system vulnerability, our work highlights a potential
significant privacy concern at user’s end.

##  2 Background

In recent years, RRAM-based CIM systems have been widely studied for DNN
inference applications due to their ability to perform in-situ single-step VMM
through bit-line current summation [9] [11] [12]. One area that has greatly
benefited from this technology is computer vision, specifically convolution
neural network (CNN), which require intensive VMM operations. Moreover, the
RRAM-based CIM systems can incorporate transposed convolution for up-sampling
in encoder-decoder networks [25] [26]. This has been shown to improve
efficiency and support various DNN models, as evidenced by several studies
[27] [28] [29] [30].

Medical diagnosis, which is a subset of computer vision, involves medical
image reconstruction, segmentation, and super-resolution [31], and CIM schemes
are promising platforms for medical image processing [32]. The U-Net
architecture, one of the most widely used CNN architectures, has been
extensively used for image segmentation tasks [33] [34]. U-Net, shown in
Figure 1(a), has a “U”-shaped architecture composed of a down-sampling encoder
and an up-sampling decoder, which perform feature extraction and
reconstruction through convolutional and transposed convolutional layers,
respectively. The skip connections in U-Net connect down-sampling and up-
sampling paths by concatenating the feature maps to preserve spatial
information and improve accuracy. U-Net has been proven effective for multiple
medical image segmentation tasks, such as identifying tumors or lesions in MRI
scans, even with limited training data. Figure 1(b) shows an example of an
original MRI image used as input to the U-Net studied here, whose outputs are
masks of the tumor region. Figure 1(c) shows the output results of the
network, where the green line represents the ground truth and the red line
represents the network prediction results.

###  2.1 Side-Channel Attacks

CIM systems use pre-trained models for inference acceleration on chip. Any
potential security threat can be exploited by a malicious adversary who could
launch a side-channel attack on the system through the side-channel leakage.
Side-channel attacks aim at extracting private data from a hardware system by
measuring and analyzing physical parameters during execution [35] [36]. Such
parameters include supply power, execution time, and electromagnetic emission.
Attackers can reverse engineer the sensitive data or architectural information
by deliberately measuring and analyzing the side-channel dissipation of the
chip. Side-channel data acquisition can be either invasive or non-invasive,
depending on whether the chip needs to be decapsulated [37] [38]. Both can be
considered to measure the data dependent signals leaked from RRAM tiles. As
RRAM devices are fabricated in the back-end-of-line, invasively etching away
the passivation layers of CIM macros can provide immediate access to the top-
level metal lines of each tile for potential probing. The adversary can then
probe the power lines directly and extract the power data using an
oscilloscope, as shown in Figure 1(d). Non-invasive techniques include the use
of electromagnetic probes to measure the electromagnetic emission of RRAM
tiles, taking advantage of the spatial locality of model mapping [39] [40]
[41]. The side-channel attack is performed on measured leaked traces, with one
such example shown in Figure 1(e), where the power trace of a CIM module at
inference runtime is shown. In this work, the goal is to reconstruct the
user’s private input data from power traces measurements. Figure 1(f) and (g)
show an example of the original private input and the reconstruction result
using the proposed approach, respectively.

Accurately measuring side-channel leakage signals requires sophisticated data
acquisition scheme design and high-precision measurement equipment. To study
the vulnerability of the chip design and secure it iteratively, it is more
practical to simulate the attack scenarios using real device measurement data,
followed by redesigning the chip with security considerations before massive
production, especially for the emerging CIM systems. In our study, we simulate
an CIM system designed with TSMC 28 nm technology, with details described in
[21].

A general assumption for side-channel attacks is that the attacker already
knows the hardware architecture and implementation of the CIM chip, which in
this case includes the RRAM array size and a mixed-signal interface design
that includes analog-to-digital converters (ADCs). However, the attacker has
no knowledge of the neural network mapped on chip. Although the inference
tasks of the RRAM-based CIM system can be pipelined to improve the throughput,
the attackers have full control over the input and output sequence, allowing
them to halt the next input until the completion of the current one. This
enables the attacker to gain more accurate power measurements without blurring
valuable power signals. The finest grain of the chip that an attacker can
access is a tile, and they have no access to individual RRAM cells.

##  3 CIM System for DNN Inference Acceleration

Figure 2: Schematic diagrams of (a) DNN mapping on a tiled CIM accelerator.
(b) A CIM tile with bit-serial inputs and shared ADC. (c) Readout measurements
of the devices’ state after the devices are programed to 16 conductance
levels. (d) Schematic of an 8-bit charge redistribution SAR ADC with timing
diagram is inset at the top. (e) Switching energy of the SAR ADC with respect
to the ADC output code. (f) Weight mapping scheme when 4 ADCs are shared by
128 columns in 1 tile. (g) 8-bit bit-serial input activation mapping at
inference runtime. (h) Timing diagram of CIM accelerator at inference runtime,
with bit-serial inputs applied to the array (blue) followed by analog-to-
digital conversion (purple). The execution sequence of analog array and ADC is
marked with corresponding colors in the power trace below.

###  3.1 RRAM Array and Device

In analog RRAM-based CIM system, pretrained DNN weight matrices are mapped on
RRAM macros in a tiled architecture, as shown in Figure 2(a). VMMs are
performed in each tile, and the outputs are accumulated from tiles belonging
to the same layer, followed by neuron function conversion before being passed
to the next layer through on-chip routers. Detailed schematic of an analog
RRAM-based CIM tile is shown in Figure 2(b). The pretrained weights are mapped
to device conductance values according to the device conductance range. RRAM
devices can support multi-bit mapping as they offer multiple conductance
levels due to the gradual modulation of conductive paths [42] [43] [44]. Our
experiments are based on devices with 16 conductance levels, i.e. 4-bit
precision, as shown in Figure 2(c). Bit-serial input is used in this study.
The outputs of VMMs are returned as bit-line currents, subsequently sampled by
the peripheral readout circuitry and converted to binary digital values by ADC
for further downstream processing and communication.

###  3.2 Mixed-Signal Interface

Figure 2(d) depicts the schematic of an 8-bit charge redistribution successive
approximation registers (SAR) ADC utilized in the mixed-signal interface. A
SAR ADC is composed of three parts: a comparator, a SAR logic controller, and
a capacitive digital-to-analog converter (DAC). Of these, the DAC consumes the
majority of the overall power dissipation [45]. Compared with other
approaches, the SAR ADC has advantages in process scaling, power efficiency,
high precision and conversion speed [46]. It converts the analog inputs by
switching each capacitor in the DAC from $V_{in}$ to $V_{ref}$ one by one to
perform a binary successive approximation, as shown in the timing diagram
inset in Figure 2(d). The conversion takes eight consecutive steps, and the
energy consumption of the n-th step can be calculated from the change in
$V_{x}$, and the capacitance connected to $V_{ref}$ using Equation 1,

$E=\begin{cases}-C_{1}V_{ref}(\Delta V_{x}-V_{ref}),&n=1\\\ -V_{ref}(\Delta
V_{x}\sum_{i=1}^{n-1}C_{i}D_{i}+C_{n}(\Delta V_{x}-V_{ref})),&n\neq
1\end{cases}$ (1)

where $D_{i}$ is the $i$ th MSB output code. When $D_{i}=1$, the $i$-th switch
connects to $V_{ref}$, otherwise, it connects to ground.

The energy during the eight DAC switching steps dominates the total analog-
digital conversion energy consumption. Figure 2(e) shows the normalized total
DAC switching energy, and it exhibits a clear data pattern dependency [47]
[48].

###  3.3 Mapping and Inference Execution

For inference tasks, pretrained weights are mapped across RRAM tiles, and
remain fixed during operations. To balance accuracy and inference efficiency,
the weights and input activations are typically quantized to 8 bits, which has
been shown to cause minor accuracy losses, especially when coupled with
quantization-aware training techniques [29]. Since a single RRAM cell
considered here only offers 4-bit storage, multiple cells are used to store
one weight value. Figure 2(f) illustrates the mapping scheme of 8-bit weights
from a convolution layer to 4-bit devices in Figure 2(c). Each convolution
kernel is flattened to a vector, and the positive and negative values are
separated into two columns. The weights are then quantized to 8-bit precision
and split into two 4-bit numbers based on their significance (i.e., 4 MSBs and
4 LSBs). In this case, the total number of crossbar columns for mapping is
four times the output channel size, with two for polarity and two for
significance. The number of crossbar rows is the flattened kernel size,
$K^{2}C_{in}$, where K is the kernel dimension and $C_{in}$ in the input
feature depth.

The output of the convolutional layer is computed by sliding the convolution
window through the input feature maps, as illustrated in Figure 2(g). The
inputs are flattened to vectors to match the kernels, and outputs from all
kernels are computed simultaneously through CIM tiles. Each 8-bit input is
mapped to bit-serial represented voltage pulses, and it takes eight steps to
apply them on the crossbar array. The timing diagram of CIM tiles during
computation is shown in Figure 2(h). An input bit is applied to the array, and
the resulting output current from bit-lines is sampled and held by the
peripheral circuit. For area and power efficiency considerations, chip
designers often implement an ADC-sharing scheme instead of assigning each
column with an ADC. The analog outputs from the shared columns are
consecutively converted by the ADC through the multiplexer (MUX). In our
experiment, the number of ADCs per array is set to 4. The power trace shown in
Figure 2(h) is simulated for the abovementioned U-Net and hardware
configurations through the simulation flow described in [21]. The blue regions
represent the analog computation cycle, and the purple regions represent ADC
conversion cycle. Due to the time required to read a stable output from the
analog computation, the analog execution can be readily identified from the
ADC execution power traces even with high measurement noise.

Figure 3: (a) Overview of power side-channel attack flow for private input
reconstruction. (b) Block diagram of CUDA SIMT execution for fast simulation
of the crossbar array analog computing power and ADC switching energy. (c)
Data preprocessing steps after power trace acquisition: First, the power trace
in each sliding window is broken into single bits in bit-serial input format,
where each bit execution contains an analog computing and an analog-to-digital
conversion phase. Second, the power consumption of each sliding window is
weighted based on the bit significancy. Third, the weighted sum results are
recorded in the two power feature matrices, corresponding to array analog
computing and ADC conversion, respectively. (d) Array power feature matrix,
and (e) ADC power feature matrix obtained in an ideal simulation without
adding noise, normalized into 8-bit unsigned integer range. They represent the
extracted input information leakages. (f) Power feature matrices with
different levels of noise injection. The noise level corresponds to the ratio
of the standard deviation to the maximum value in the power feature matrix.

##  4 Side-Channel Leakage in CIM Systems

###  4.1 Attack Flow and Power Modeling

Figure 3(a) illustrates the attack flow aimed at reconstructing private input
data from the power side-channel leakage. The attack targets the first layer
of the DNN model since it is the closest layer to the input port and directly
executes the input data. The methodologies proposed in [21] can be used to
extract the property of the first layer, i.e., structure of the convolutional
layer and the associated CIM tiles used to execute the layer. As DNN models
are trained on specific tasks, an adversary can collect power traces using
similar input data and learn the dependency between them. In this case, the
adversary can feed other MRI images and collect their own power traces to
train an attack model before attacking unknow inputs. Once power traces are
allocated, power feature extraction and data preprocessing are required to
find the correlation between input and leakage. We refer to the data after
preprocessing as power feature matrices, which can be utilized to train a
machine learning model for input data reconstruction. In our study, we
employed a GAN for reconstruction since it shows good noise tolerance and can
overcome noise-injection countermeasures.

The power trace simulator proposed in [21] provides valuable insights into
dynamic power and timing data. However, processing every single data point in
a large dataset is impractical due to the time-consuming nature of simulating
power dissipation at sub-nanosecond-level precision, as well as the generation
of tremendous data files. To address this, we developed a fast power feature
simulator based on NVIDIA Compute Unified Device Architecture (CUDA),
leveraging modern GPU’s single instruction multiple threads (SIMT) to process
every bit input in the bit-serial fashion, as shown in Figure 3(b). The CUDA
kernel functions simulate power dissipation and total energy of RRAM arrays
and ADCs at each execution, respectively. Both kernels take bit-serial inputs
and conductance weight matrices as inputs. The kernel function for array power
simulation computes the power using applied input voltage and device
conductance values. The kernel function for ADC is more complex. First, we
compute the switching energy of each output code (Figure 2(e)) and store them
in a lookup table (LUT). Then, the kernel function computes the analog current
of each bit-line, followed by scaling it into the proper range to index the
ADC energy from the LUT. Due to ADC sharing, the ADC energy in one execution
is the sum of four ADCs.

To convert all outputs in an array with 128 columns and 4 ADCs requires 32
executions. However, the ADC traces are similar to each other, making it
impractical to distinguish every execution with measurement noise, as shown in
the noisy trace Figure 3(c). Therefore, we treat the energy of all 32 ADC
executions as a whole for further data preprocessing.

###  4.2 Data Preprocessing and Leakage

When an input image is processed by the model stored on the CIM chip, the
convolution window in the first convolution layer slides through the entire
input image. Here we define two power feature matrices, corresponding to the
analog array computation and ADC conversion energy when performing the
convolution operation, respectively. The power feature matrices will have the
same size as the output feature map of the first convolution layer. An entry
in a power feature matrix corresponds to the collected power information
during the convolution operation at the corresponding position in the input
feature map. Within each convolution window, the input data are reshaped and
scaled into 8 bits before being applied to the crossbar array. Processing an
input bit in CIM can be divided in two steps: analog array computation and
analog-to-digital conversion, as shown in Figure 3(c). Because the eight input
bits are of varying significancy, the input-dependent power leakage at each
bit computation should not be treated equally. To recover the input-dependent
power data, we used a weighted sum approach for the array power and ADC energy
according to the bit significance, as shown in Equation 2 and 3.

$P_{array}=\sum_{i=0}^{7}{P_{array}[i]\times 2^{i}}$ (2)

$E_{ADC}=\sum_{i=0}^{7}{E_{ADC}[i]\times 2^{i}}$ (3)

Once the weighted sum results of each convolution window have been computed,
they are then populated into the two power feature matrices. Figure 3(d) and
(e) show examples of the obtained array power feature matrix and the ADC power
feature matrix, respectively, without considering noise in the power data
measurement. The power side-channel leakage exhibits a strong dependency on
the input data and reveals a security risk.

Both the array power dissipation and ADC energy consumption are functions of
the inputs and weights, which explains the strong dependency between the power
side-channel leakage and input data. During inference, the weights are held
constant leading to a linear relationship between the power feature matrices
and the original input, as shown in Equation 4,

$y_{i,j}=\mathcal{F}\left(\left[\begin{matrix}x_{i-r,\ j-r}&\cdots&x_{i-r,\
j+r}\\\ \vdots&\ddots&\vdots\\\ x_{i+r,\ j-r}&\cdots&x_{i+r,\ j+r}\\\
\end{matrix}\right]\right)$ (4)

where i, j are indices of the entry from power feature matrices, r is the
radius of the convolution window, x is the the input data and y is the
corresponding entry in the power feature matrices. The function $\mathcal{F}$
is used to convert the input data into power feature data after weighted sum.
Regardless of whether $\mathcal{F}$ represents array or ADC, it is always a
one-to-one projection. As a result, the input information is preserved in the
power feature matrices and leading to a severe security issue.

Additionally, as shown in Figure 3(d) and (e), the array power feature matrix
contains more detailed information than the ADC power feature matrix. This is
because each entry in the ADC power feature matrix represents the total energy
of 32 executions, with each execution involving four ADCs operating together.
Compared with the array computation power feature matrix which entries
indicating single execution, the ADC power feature matrix produces coarser
granularity. Furthermore, from the weight mapping scheme in Figure 2(f), the
energy consumption of the four ADCs are associated with a weight value in four
representations of MSB+, MSB-, LSB+ and LSB-. Since the LSBs and MSBs have
different impacts on the weight value, the ADC energy output is not a direct
linear transformation of the input data, where the array power feature matrix
corresponds to a linear transformation of the input data (directly
proportional to the product of the input and the weight matrix).

###  4.3 Noise and Countermeasures

The power side-channel attack approach mentioned above is capable of reverse
engineering private inputs. However, for a real-world applications non-ideal
effects such as noise during data acquisition must be considered. Noise can
originate from multiple sources of the chip, including thermal noise and
human-made noise [49] [50] [51]. Thermal noise is a physical phenomenon that
cannot be eliminated. Thermal noise can be found on RRAM devices. Although
ideal capacitors have no thermal noise, when they are coupled with other
components in the circuit, there will be a combination of kTC noise. Other
noise from power lines or measurement equipment may have higher noise power
and will lower the signal-to-noise ratio when the adversary measures the power
side-channel leakage. The power feature data, accounting for the presence of
noise, are obscured by a noise term N, which can be mathematically described
by Equation 5.

$y_{i,j}=\mathcal{F}\left(\left[\begin{matrix}x_{i-r,\ j-r}&\cdots&x_{i-r,\
j+r}\\\ \vdots&\ddots&\vdots\\\ x_{i+r,\ j-r}&\cdots&x_{i+r,\ j+r}\\\
\end{matrix}\right]\right)+\mathcal{N}\left(\cdots\right)$ (5)

Noise can also be used to protect the system. One common countermeasure to
mitigate power analysis side-channel attacks is noise injection. Noise
injection works by adding a random noise signal to the original signal to mask
the correlation between the leaked information and the secret information [52]
[53]. The noise signal is designed to be random and uncorrelated with the
original signals, so statistical analysis methods for denoising may not be
valid anymore.

Figure 4: (a) Schematic of the conditional GAN model architecture. (b)
Schematic of the pix2pix GAN architecture for reconstructing image from noisy
power traces. The power feature matrices are concatenated as the input to the
generator, which is based on a U-Net architecture. The generated image, along
with the original image and the power feature matrices, are fed to the
PatchGAN discriminator for differentiation. (c) PatchGAN architecture, which
outputs a 2D score matrix where each element corresponds to a local patch of
the input image. (d) Evolution of the image reconstruction from power feature
matrices over 150 training epochs, with a noise level of 20%.

In Figure 3(f), we simulate different noise levels from 5% to 20% at side-
channel data acquisition. The percentage levels here are defined as the ratio
of the noise’s standard deviation to the maximum measured signal in the power
feature. As the noise level increases, details of the cerebrum region in the
power feature matrices are lost. It is noteworthy that when the noise level
reaches 20%, the cerebrum region was effectively masked from the power feature
matrices, and only noise is circled by the skull. Hence, noise injection is a
powerful countermeasure to mitigate the side-channel leakage in the CIM
system. To deal with noise injection, at the adversary end, a more effective
attack approach is required.

##  5 Machine Learning Assisted Side-Channel Attack

###  5.1 Generative Adversarial Network for Side-Channel Attac

Adversaries often attempt to design elaborate denoising schemes to stripe the
noise signal and expose the valuable original signal [16]. However, using
conventional denoising techniques requires considerable effort in denoising
design at both the hardware and algorithm levels. The adversary needs to
specify the noise frequency and apply a low-pass filter to cut off high-
frequency noise during measurement. Then, the adversary needs to identify the
working spectrum of the CIM system, and recover the distortion induced by the
power measurement circuit. To establish the relationship between the restored
power curve and the original data, the adversary is required to conduct
circuit analysis of the power data at each execution frame, which can be
inefficient when dealing with large volumes of input. Therefore, a flexible
attacking approach with noise tolerance is essential for efficient side-
channel attacks and for prompting to designing more secure and reliable CIM
systems.

Machine learning approaches have recently been explored for side-channel
attacks since they can be highly automated and scalable, allowing attackers to
extract sensitive information with minimal human intervention from large
volumes of data [54] [55] [56]. Machine learning-based side-channel attacks
involve a training phase and an attack phase. The training phase can be
controlled by the adversary by building a leakage model from the trace
collected earlier using known input data, which alleviates the requirement of
collecting sufficient traces from limited resources. The adversary can design
a system-specific model to prompt a side-channel attack and recover the target
data from newly measured trace during the attack phase.

GANs are a type of DNN architecture consisting of two neural networks for
generating synthetic data: a generator and a discriminator [57]. In
conventional GANs, the generator takes random noise as input and tries to
generate data to mimic the training data, while the discriminator takes both
the real data and generated data and tries to distinguish between them. During
training, the two networks are trained together in a min-max game, where the
generator tries to produce data that can fool the discriminator, and the
discriminator tries to evaluate the authenticity of the generated data. Unlike
conventional GANs, which take random noise as input, conditional GANs (cGANs)
take extra information as input, such as image features or text description
[58] [59] [60] [61], as shown in Figure 4(a). The target of cGANs is to
generate more controlled outputs belonging to a certain category or containing
certain desired features. cGANs have many potential applications in various
fields such as image-to-image translation, text-to-image synthesis, and style
transfer. In a side-channel attack, the adversary’s goal is to train a neural
network to reconstruct the input information from side-channel leakage, making
cGANs a good fit for this task.

Noise is required during training of both conventional GANs and cGANs because
it provides the generator with a source of randomness, allowing it to produce
diverse and realistic data and improve the model performance. Consequently,
GANs are excellent candidates for coping with noise (and leveraging the noise)
during side-channel data measurement, enabling the elimination of complex
conventional denoising schemes for data acquisition. GAN-based DNNs can make
side-channel attacks more flexible and enhance attack success rate.

###  5.2 Experiment Setup

We note the reconstruction of the original image from power feature matrices
is analogous to image-to-image translation. In this study, we adopted the
pip2pix cGAN [60] architecture to reconstruct the input images from side-
channel leakage. Pix2pix is a specialized version of cGANs designed to map an
input image from one domain to an output image in another domain.

The pix2pix architecture used in our experiment is shown in Figure 4(b). The
generator is a standard U-Net architecture, similar to the one discussed in
Figure 1(a) for medical image segmentation. The array power feature matrix and
ADC power feature matrix, shown above and below in the left of Figure 4(a),
are concatenated in the channel direction before being fed into the neural
network. The discriminator is a patch-based CNN called PatchGAN. Two
discriminator networks are used in the discrimination phase, by analyzing
pairs of images, a real MRI image and a fake image generated by the generator,
along with the stacked power feature matrices as input, and outputs a patch-
level prediction. The architecture of the PatchGAN discriminator is shown in
Figure 4(c). For a input image, the PatchGAN takes a patch of the image as
input and outputs a matrix of values, where each value in the matrix indicates
the probability that the corresponding patch in the input image is real. For
the $256\times 256$ images, patch size of $70\times 70$ can achieve realistic
reconstruction results. In this case, the prediction map size is $4\times 4$.
By computing the average of the values in the prediction map, the PatchGAN can
measure the overall realism of the generated images.

PatchGAN has several advantages in reconstructing the lost details in the
power feature matrices. Firstly, it encourages the generator to produce more
detailed and high-frequency information in the output, as it has to fool the
discriminator at the smaller patch level (vs the larger image level).
Secondly, the discriminator can provide more fine-grained feedback to the
generator since it evaluates images at the patch level. Thirdly, by using
smaller patches rather than the entire image, PatchGAN can capture local image
features.

The pix2pix cGAN model is trained on the brain MRI dataset with power feature
matrices with size of $256\times 256$. A total of 3143 MIR image data are used
in 200 training epochs with batch size of 1. The data are split into train,
validation, and test sets randomly.

Figure 5: Image reconstruction of four representative brain MRI images. The
left column shows the original MRI images, and the right columns show
reconstruction results at different injected noise levels.

###  5.3 Results and Discussion

Figure 4(d) shows the image reconstruction results as a function of training
epochs. The test image used for reconstruction was injected with a 20% noise
level in the power feature matrices. The model is capable of generating a
high-level brain structure, including a highlighted tumor region, in just 5
epochs. However, some tiling artifacts can be observed in the reconstructed
images when the epoch number is below 20, and the detailed information of
brain lobes has not been fully reconstructed. The model is fine-tuned in the
subsequent epochs to achieve more precise reconstructions of the MRI images.
Human brains are characterized by a folded cerebral cortex, which exhibits
diverse details across different MRI images, which makes it hard to
reconstruct all detailed information. As the epoch number increases, the model
was able to reconstruct more details of sulci and gyri in the cerebral cortex,
along with more precise locations of the tumor region.

Figure 5 shows reconstruction results for brain MRI images corresponding to
different horizontal sections, tumor types and brain lobes structures, with
varying levels of noise. The original images are shown on the left, and the
reconstructed counterparts with different noise levels are shown on the right.
All the reconstructed images demonstrate accurately restored large-scale
structures, including tumor type, tumor region, and brain lobes structures.
Furthermore, local details such as sulci and gyri in the cerebral cortex are
also well preserved in the reconstruction results, and closely resemble the
original MRI images. It should be noted that as the noise level increases, the
quality of the reconstructed images slighted decreases, leading to slightly
blurred images and missing or adding sulci and gyri. However, most of
significant information from the original images remains intact, even at high
noise levels of up to 20%. Consequently, the GAN-assisted side-channel attack
is effective in defeating this level of noise injection countermeasures in CIM
systems.

###  5.4 Conclusion

In this study, we analyzed power side-channel attacks on CIM accelerators, and
show carefully designed side-channel attacks can reverse engineer private
input data from the user without any prior knowledge of the DNN model used on
the chip, thus revealing a potential significant security vulnerability. We
propose an automated input reconstruction scheme based on pix2pix cGAN for
input image reconstruction from power side-channel leakage, and demonstrate
the effectiveness of the proposed attack method on a brain MRI dataset. Our
experiments show the power side-channel attack can tolerate a high noise level
in power data acquisition, and defeat conventional noise-injection
countermeasures. This work highlights a critical vulnerability in CIM systems
and underscores the need for greater attention to security considerations in
CIM architecture design.

## Acknowledgments

This work was support in part by SRC and DARPA through the Applications
Driving Architectures (ADA) Research Center, and by the National Science
Foundation through Award CCF-1900675.

## References

  * [1] OpenAI, “Gpt-4 technical report,” _arXiv_ , 2023.
  * [2] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [3] E. Variani, X. Lei, E. McDermott, I. L. Moreno, and J. Gonzalez-Dominguez, “Deep neural networks for small footprint text-dependent speaker verification,” in _2014 IEEE international conference on acoustics, speech and signal processing (ICASSP)_. IEEE, 2014, pp. 4052–4056.
  * [4] A. M. Ozbayoglu, M. U. Gudelek, and O. B. Sezer, “Deep learning for financial applications: A survey,” _Applied Soft Computing_ , vol. 93, p. 106384, 2020\.
  * [5] Y. Wu, X. Wang, and W. D. Lu, “Dynamic resistive switching devices for neuromorphic computing,” _Semiconductor Science and Technology_ , vol. 37, no. 2, p. 024003, 2021.
  * [6] J.-M. Hung, C.-J. Jhang, P.-C. Wu, Y.-C. Chiu, and M.-F. Chang, “Challenges and trends of nonvolatile in-memory-computation circuits for ai edge devices,” _IEEE Open Journal of the Solid-State Circuits Society_ , vol. 1, pp. 171–183, 2021.
  * [7] A. Sebastian, M. Le Gallo, R. Khaddam-Aljameh, and E. Eleftheriou, “Memory devices and applications for in-memory computing,” _Nature nanotechnology_ , vol. 15, no. 7, pp. 529–544, 2020.
  * [8] Y. Chen, Y. Xie, L. Song, F. Chen, and T. Tang, “A survey of accelerator architectures for deep neural networks,” _Engineering_ , vol. 6, no. 3, pp. 264–274, 2020.
  * [9] X. Wang, Y. Wu, and W. D. Lu, “Rram-enabled ai accelerator architecture,” in _2021 IEEE International Electron Devices Meeting (IEDM)_. IEEE, 2021, pp. 12–2.
  * [10] P. Chi, S. Li, C. Xu, T. Zhang, J. Zhao, Y. Liu, Y. Wang, and Y. Xie, “Prime: A novel processing-in-memory architecture for neural network computation in reram-based main memory,” _ACM SIGARCH Computer Architecture News_ , vol. 44, no. 3, pp. 27–39, 2016.
  * [11] M. Prezioso, F. Merrikh-Bayat, B. D. Hoskins, G. C. Adam, K. K. Likharev, and D. B. Strukov, “Training and operation of an integrated neuromorphic network based on metal-oxide memristors,” _Nature_ , vol. 521, no. 7550, pp. 61–64, 2015.
  * [12] M. A. Zidan, J. P. Strachan, and W. D. Lu, “The future of electronics based on memristive systems,” _Nature electronics_ , vol. 1, no. 1, pp. 22–29, 2018\.
  * [13] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in _2017 IEEE symposium on security and privacy (SP)_. IEEE, 2017, pp. 3–18.
  * [14] X. Hu, L. Liang, S. Li, L. Deng, P. Zuo, Y. Ji, X. Xie, Y. Ding, C. Liu, T. Sherwood _et al._ , “Deepsniffer: A dnn model extraction framework based on learning architectural hints,” in _Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems_ , 2020, pp. 385–399.
  * [15] S. Gu and L. Rigazio, “Towards deep neural network architectures robust to adversarial examples,” _arXiv preprint arXiv:1412.5068_ , 2014.
  * [16] L. Wei, B. Luo, Y. Li, Y. Liu, and Q. Xu, “I know what you see: Power side-channel attack on convolutional neural network accelerators,” in _Proceedings of the 34th Annual Computer Security Applications Conference_ , 2018, pp. 393–406.
  * [17] Y. Xiang, Z. Chen, Z. Chen, Z. Fang, H. Hao, J. Chen, Y. Liu, Z. Wu, Q. Xuan, and X. Yang, “Open dnn box by power side-channel attack,” _IEEE Transactions on Circuits and Systems II: Express Briefs_ , vol. 67, no. 11, pp. 2717–2721, 2020.
  * [18] W. Hua, Z. Zhang, and G. E. Suh, “Reverse engineering convolutional neural networks through side-channel information leaks,” in _Proceedings of the 55th Annual Design Automation Conference_ , 2018, pp. 1–6.
  * [19] L. Batina, S. Bhasin, D. Jap, and S. Picek, “Csi nn: Reverse engineering of neural network architectures through electromagnetic side channel,” 2019.
  * [20] Y. Zhang, R. Yasaei, H. Chen, Z. Li, and M. A. Al Faruque, “Stealing neural network structure through remote fpga side-channel analysis,” _IEEE Transactions on Information Forensics and Security_ , vol. 16, pp. 4377–4388, 2021\.
  * [21] Z. Wang, F.-h. Meng, Y. Park, J. K. Eshraghian, and W. D. Lu, “Side-channel attack analysis on in-memory computing architectures,” _IEEE Transactions on Emerging Topics in Computing_ , 2023.
  * [22] J. Read, W. Li, and S. Yu, “A method for reverse engineering neural network parameters from compute-in-memory accelerators,” in _2022 IEEE Computer Society Annual Symposium on VLSI (ISVLSI)_. IEEE, 2022, pp. 302–307.
  * [23] S. K. Cherupally, J. Meng, A. S. Rakin, S. Yin, I. Yeo, S. Yu, D. Fan, and J.-S. Seo, “Improving the accuracy and robustness of rram-based in-memory computing against rram hardware noise and adversarial attacks,” _Semiconductor Science and Technology_ , vol. 37, no. 3, p. 034001, 2022.
  * [24] M. Buda. Brain mri segmentation dataset. [Online]. Available: https://www.kaggle.com/datasets/mateuszbuda/lgg-mri-segmentation
  * [25] H. Mao, M. Song, T. Li, Y. Dai, and J. Shu, “Lergan: A zero-free, low data movement and pim-based gan architecture,” in _2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)_. IEEE, 2018, pp. 669–681.
  * [26] F. Chen, L. Song, and Y. Chen, “Regan: A pipelined reram-based accelerator for generative adversarial networks,” in _2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC)_. IEEE, 2018, pp. 178–183.
  * [27] P. Yao, H. Wu, B. Gao, J. Tang, Q. Zhang, W. Zhang, J. J. Yang, and H. Qian, “Fully hardware-implemented memristor convolutional neural network,” _Nature_ , vol. 577, no. 7792, pp. 641–646, 2020.
  * [28] Z. Wang, C. Li, P. Lin, M. Rao, Y. Nie, W. Song, Q. Qiu, Y. Li, P. Yan, J. P. Strachan _et al._ , “In situ training of feed-forward and recurrent convolutional memristor networks,” _Nature Machine Intelligence_ , vol. 1, no. 9, pp. 434–442, 2019.
  * [29] Y. Wu, Q. Wang, Z. Wang, X. Wang, B. Ayyagari, S. Krishnan, M. Chudzik, and W. D. Lu, “Bulk-switching memristor-based compute-in-memory module for deep neural network training,” _arXiv preprint arXiv:2305.14547_ , 2023.
  * [30] Z. Li, Z. Wang, L. Xu, Q. Dong, B. Liu, C.-I. Su, W.-T. Chu, G. Tsou, Y.-D. Chih, T.-Y. J. Chang _et al._ , “Rram-dnn: An rram and model-compression empowered all-weights-on-chip dnn accelerator,” _IEEE Journal of Solid-State Circuits_ , vol. 56, no. 4, pp. 1105–1115, 2020.
  * [31] D. Shen, G. Wu, and H.-I. Suk, “Deep learning in medical image analysis,” _Annual review of biomedical engineering_ , vol. 19, pp. 221–248, 2017.
  * [32] H. Zhao, Z. Liu, J. Tang, B. Gao, Q. Qin, J. Li, Y. Zhou, P. Yao, Y. Xi, Y. Lin _et al._ , “Energy-efficient high-fidelity image reconstruction with memristor arrays for medical diagnosis,” _Nature Communications_ , vol. 14, no. 1, p. 2276, 2023.
  * [33] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18_. Springer, 2015, pp. 234–241.
  * [34] M. Buda, A. Saha, and M. A. Mazurowski, “Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithm,” _Computers in biology and medicine_ , vol. 109, pp. 218–225, 2019.
  * [35] P. C. Kocher, “Timing attacks on implementations of diffie-hellman, rsa, dss, and other systems,” in _Advances in Cryptology—CRYPTO’96: 16th Annual International Cryptology Conference Santa Barbara, California, USA August 18–22, 1996 Proceedings 16_. Springer, 1996, pp. 104–113.
  * [36] P. Kocher, J. Jaffe, and B. Jun, “Differential power analysis,” in _Advances in Cryptology—CRYPTO’99: 19th Annual International Cryptology Conference Santa Barbara, California, USA, August 15–19, 1999 Proceedings 19_. Springer, 1999, pp. 388–397.
  * [37] M. Hutle and M. Kammerstetter, “Resilience against physical attacks,” in _Smart Grid Security_. Elsevier, 2015, pp. 79–112.
  * [38] J. Fan, X. Guo, E. De Mulder, P. Schaumont, B. Preneel, and I. Verbauwhede, “State-of-the-art of secure ecc implementations: a survey on known side-channel attacks and countermeasures,” in _2010 IEEE international symposium on hardware-oriented security and trust (HOST)_. IEEE, 2010, pp. 76–87.
  * [39] H. Funato and T. Suga, “Magnetic near-field probe for ghz band and spatial resolution improvement technique,” in _2006 17th International Zurich Symposium on Electromagnetic Compatibility_. IEEE, 2006, pp. 284–287.
  * [40] Y.-T. Chou and H.-C. Lu, “Space difference magnetic near-field probe with spatial resolution improvement,” _IEEE transactions on microwave theory and techniques_ , vol. 61, no. 12, pp. 4233–4244, 2013.
  * [41] Z. Peng, X. Li, and J. Mao, “A pair of parallel differential magnetic-field probes with high spatial resolution and wide frequency bandwidth,” in _2019 IEEE MTT-S International Wireless Symposium (IWS)_. IEEE, 2019, pp. 1–3.
  * [42] Y. Wu, F. Cai, L. Thomas, T. Liu, A. Nourbakhsh, J. Hebding, E. Smith, R. Quon, R. Smith, A. Kumar _et al._ , “Demonstration of a multi-level $\mu$a-range bulk switching reram and its application for keyword spotting,” in _2022 International Electron Devices Meeting (IEDM)_. IEEE, 2022, pp. 18–4.
  * [43] C. Du, W. Ma, T. Chang, P. Sheridan, and W. D. Lu, “Biorealistic implementation of synaptic functions with oxide memristors through internal ionic dynamics,” _Advanced Functional Materials_ , vol. 25, no. 27, pp. 4290–4299, 2015.
  * [44] Y. Sun, C. Song, J. Yin, L. Qiao, R. Wang, Z. Wang, X. Chen, S. Yin, M. Saleem, H. Wu _et al._ , “Modulating metallic conductive filaments via bilayer oxides in resistive switching memory,” _Applied Physics Letters_ , vol. 114, no. 19, p. 193502, 2019.
  * [45] M. Saberi, R. Lotfi, K. Mafinezhad, and W. A. Serdijn, “Analysis of power consumption and linearity in capacitive digital-to-analog converters used in successive approximation adcs,” _IEEE Transactions on Circuits and Systems I: Regular Papers_ , vol. 58, no. 8, pp. 1736–1748, 2011.
  * [46] P. Harpe, “Successive approximation analog-to-digital converters: Improving power efficiency and conversion speed,” _IEEE Solid-State Circuits Magazine_ , vol. 8, no. 4, pp. 64–73, 2016.
  * [47] B. P. Ginsburg and A. P. Chandrakasan, “An energy-efficient charge recycling approach for a SAR converter with capacitive DAC,” in _2005 IEEE international symposium on circuits and systems_. IEEE, 2005, pp. 184–187.
  * [48] V. Hariprasath, J. Guerber, S.-H. Lee, and U.-K. Moon, “Merged capacitor switching based SAR ADC with highest switching energy-efficiency,” _Electronics letters_ , vol. 46, no. 9, pp. 620–621, 2010.
  * [49] R. Rohrer, L. Nagel, R. Meyer, and L. Weber, “Computationally efficient electronic-circuit noise calculations,” _IEEE Journal of Solid-State Circuits_ , vol. 6, no. 4, pp. 204–213, 1971.
  * [50] R. Schreier, J. Silva, J. Steensgaard, and G. C. Temes, “Design-oriented estimation of thermal noise in switched-capacitor circuits,” _IEEE Transactions on Circuits and Systems I: Regular Papers_ , vol. 52, no. 11, pp. 2358–2368, 2005.
  * [51] A. Sheikholeslami, “Power supply noise [circuit intuitions],” _IEEE Solid-State Circuits Magazine_ , vol. 12, no. 3, pp. 15–17, 2020.
  * [52] R. Spreitzer, V. Moonsamy, T. Korak, and S. Mangard, “Systematic classification of side-channel attacks: A case study for mobile devices,” _IEEE Communications Surveys & Tutorials_, vol. 20, no. 1, pp. 465–488, 2017.
  * [53] D. Das, S. Maity, S. B. Nasir, S. Ghosh, A. Raychowdhury, and S. Sen, “High efficiency power side-channel attack immunity using noise injection in attenuated signature domain,” in _2017 IEEE International Symposium on Hardware Oriented Security and Trust (HOST)_. IEEE, 2017, pp. 62–67.
  * [54] S. Picek, G. Perin, L. Mariot, L. Wu, and L. Batina, “Sok: Deep learning-based physical side-channel analysis,” _ACM Computing Surveys_ , vol. 55, no. 11, pp. 1–35, 2023.
  * [55] T. Kubota, K. Yoshida, M. Shiozaki, and T. Fujino, “Deep learning side-channel attack against hardware implementations of aes,” _Microprocessors and Microsystems_ , vol. 87, p. 103383, 2021.
  * [56] B. Hettwer, S. Gehrer, and T. Güneysu, “Applications of machine learning techniques in side-channel attacks: a survey,” _Journal of Cryptographic Engineering_ , vol. 10, pp. 135–162, 2020.
  * [57] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” _Communications of the ACM_ , vol. 63, no. 11, pp. 139–144, 2020.
  * [58] M. Mirza and S. Osindero, “Conditional generative adversarial nets,” _arXiv preprint arXiv:1411.1784_ , 2014.
  * [59] S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee, “Generative adversarial text to image synthesis,” in _International conference on machine learning_. PMLR, 2016, pp. 1060–1069.
  * [60] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 1125–1134.
  * [61] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2223–2232.