# A duplication-free quantum neural network for universal approximation

Xiaokai Hou  Institute of Fundamental and Frontier Sciences, University of
Electronic Science and Technology of China, Chengdu, 610051, China  Guanyu
Zhou<EMAIL_ADDRESS>Institute of Fundamental and Frontier Sciences,
University of Electronic Science and Technology of China, Chengdu, 610051,
China  Qingyu Li  Institute of Fundamental and Frontier Sciences, University
of Electronic Science and Technology of China, Chengdu, 610051, China  Shan
Jin  Institute of Fundamental and Frontier Sciences, University of Electronic
Science and Technology of China, Chengdu, 610051, China  Xiaoting Wang
<EMAIL_ADDRESS>Institute of Fundamental and Frontier Sciences,
University of Electronic Science and Technology of China, Chengdu, 610051,
China

###### Abstract

The universality of a quantum neural network refers to its ability to
approximate arbitrary functions and is a theoretical guarantee for its
effectiveness. A non-universal neural network could fail in completing the
machine learning task. One proposal for universality is to encode the quantum
data into identical copies of a tensor product, but this will substantially
increase the system size and the circuit complexity. To address this problem,
we propose a simple design of a duplication-free quantum neural network whose
universality can be rigorously proved. Compared with other established
proposals, our model requires significantly fewer qubits and a shallower
circuit, substantially lowering the resource overhead for implementation. It
is also more robust against noise and easier to implement on a near-term
device. Simulations show that our model can solve a broad range of classical
and quantum learning problems, demonstrating its broad application potential.

##  I Introduction

Machine learning (ML) is a powerful data-analyzing tool that has generated a
series of impactful results in image recognition [1, 2, 3], natural language
processing [4, 5, 6], self-driving [7, 8, 9], etc. Meanwhile, quantum machine
learning has become an emerging interdisciplinary subject combining machine
learning with quantum computing. It studies two fundamental questions [10],
one on applications of classical ML to quantum problems [11, 12, 13, 14, 15,
16], and the other on implementations of ML algorithms on a quantum processor.
Recent works point out that implementing ML algorithms on a quantum computer
to process quantum data requires exponentially fewer experiments than dealing
with the same task on a classical computer [17]. In addition, it has been
shown that many ML algorithms can be implemented on a quantum machine with
certain levels of advantage, ranging from data fitting [18], support vector
machine [19], nearest neighbors [20], principle component analysis [21], and
linear regression [22], to PageRank algorithm [23], reinforcement learning
[24, 25, 26], anomaly detection [27] and Boltzmann machine [28]. For other ML
topics, especially the quantum neural networks (QNNs), the existence of
quantum advantage is largely unknown [29], although an interesting result on
the prediction advantage for QNNs has been recently achieved [30].

A neural network (NN) is a parameterized composite mapping comprised of
activation functions and is very powerful in data fitting. A QNN is an NN
implemented on a quantum computer [31] and there are various proposals to
design it, such as the variational QNN [32, 33]. Besides quantum advantage,
designing an effective and efficient QNN is also important. Not every QNN
design effectively solves the given ML problem, and the concept of _universal
approximation_ is introduced to explain why some neural networks fail to work
[34, 35]. Universal approximation, or the _universality_ of an NN, refers to
its ability to approximate arbitrary functions. For classical NNs,
universality is easy to achieve; for QNNs, a smart design is required to make
them universal and practical. In fact, the universality of NNs is closely
related to the nonlinear activation functions, and the key point of designing
a universal QNN is to generate the desired nonlinearity for function fitting.
One method is to construct the quantum neurons, which are the building blocks
of some QNNs [36, 37, 38, 39, 40, 41]; the other method is to construct a
variational QNN from a parameterized quantum circuit, and the nonlinearity can
be achieved by duplicating the quantum data into a tensor product of multiple
copies [42, 43]. Examples of the latter method include the quantum circuit
learning (QCL) algorithm [32] and the circuit-centric quantum classifiers
(CCQ) algorithm [33]. So far, the universality of the QCL algorithm has been
proved using the Weierstrass theorem, while the universality of the CCQ
algorithm remains open. For both QCL and CCQ, approximating a highly-nonlinear
function would require a tensor product of many data-encoding subsystems,
resulting in a large overall system size with a considerable circuit
complexity, conflicting with the principle of NISQ computing where a
relatively small quantum system with a shallow circuit is preferred.

To address this problem, we propose a duplication-free quantum neural network
(DQNN) based on a variational quantum circuit and rigorously prove its
universality. Our model utilizes the classical sigmoid function to generate
nonlinearity without duplicating the quantum data into a tensor product of
multiple copies. Compared with the CCQ or QCL algorithms, our DQNN
significantly reduces the system size and gate complexity, and hence the
overall noise effect. Therefore, it is more likely to be implemented on near-
term devices. Numerical simulations show that our DQNN outperforms the other
two variational QNN algorithms with better performance on typical regression
and classification problems and is more robust against noise. In addition,
through solving a broad range of classical and quantum learning problems, our
model has well demonstrated its wide application potential.

##  II Two types of designs for universal approximation

Before we discuss the DQNN design, let’s briefly review two common types of
designs for the universal approximation of a classical NN. It turns out that
universality is closely related to different kinds of convergence. Type I
approximation relies on pointwise convergence; a typical example is the
polynomial approximation, using the Weierstrass theorem and polynomials to
approximate arbitrary functions; Type II approximation depends on the
convergence in Lebesgue integration, e.g., $L^{2}$-norm convergence. For
polynomial approximation, there are several limitations. First, the
Weierstrass theorem is only valid on a compact set, while many NN problems are
defined on unbounded set, on which polynomials could fail to approximate a
Lebesgue integrable function. For example, the function $f(x)=e^{-x}$ defined
on $[0,+\infty)$ can not be well approximated by polynomials since the latter
diverges to $\pm\infty$ while $e^{-x}$ vanishes as $x\rightarrow\infty$.
Second, polynomial approximation has a numerical problem called Runge’s
phenomenon [44] when the polynomial order becomes very large, which is why
using high-order polynomials for function fitting is not favored in practical
use. Due to these limitations, polynomials are ineffective in designing
universal NNs. It has been proved that the necessary and sufficient condition
for a universal feedforward NN is that its activation functions are not
polynomial [35]. In comparison, Type II approximation utilizes basis functions
such as the hat function in finite-element method [45] or the sigmoid/ReLU
function composite with linear function in neural networks [46]. Usually, the
optimal approximation is the solution to a minimization problem, and the error
is evaluated in a suitable function norm. In particular, universal NNs based
on $L^{2}$-norm approximation are widely applied and found to be very
effective. Such examples include the ones with activation functions, such as
the sigmoid or the ReLU functions. Hence, this work aims to discuss how to
construct a universal and effective QNN based on $L^{2}$ approximation, using
the trick analogous to the sigmoid function instead of the polynomials. We
expect that such QNN is effective in a broad range of applications compared to
the QNN based on polynomial approximation.

##  III DQNN and its universality

Mathematically, a variational QNN is a mapping
$q_{{\bm{\theta}}}:{\bm{x}}\in\mathbb{R}^{d}\to
q_{{\bm{\theta}}}(\bm{x})\in\mathbb{R}$ built on a parameterized quantum
circuit $U({\bm{\theta}})$. It involves a hybrid quantum-classical algorithm
that utilizes a quantum processor to generate the output
$q_{{\bm{\theta}}}({\bm{x}})$ for each input ${\bm{x}}$, and a classical
processor to optimize ${\bm{\theta}}$ to solve learning problems.
Specifically, given an unknown function $f:{\bm{x}}\to f(\bm{x})$, and a data
set $D=\\{(\bm{x}^{(i)},y^{(i)})\\}^{m}_{i=1}$, where
$y^{(i)}=f(\bm{x}^{(i)})$ is the label of ${\bm{x}}^{(i)}$, the aim of QNN is
to find an optimal ${\bm{\theta}}^{*}$ such that $q_{{\bm{\theta}}^{*}}$ best
approximates the target function $f$. Our DQNN model consists of three parts,
a quantum processor, a classical processor and a classical optimizer, as
illustrated in Fig. 1(a). First, each data point
${\bm{x}}=[x_{1},x_{2},\cdots,x_{d}]^{T}\in\mathbb{R}^{d}$ is encoded into a
state of an $n$-qubit register ($n=\lceil\log d\rceil$):

$|\bar{{\bm{x}}}\rangle=\frac{1}{\gamma}[x_{1},x_{2},\cdots,x_{d},\tilde{x},0,\cdots
0]^{T}\in\mathbb{C}_{2}^{\otimes n},$ (1)

using the amplitude encoding method [47], where
$\tilde{x}\equiv\frac{\|\bm{x}\|}{1+\|\bm{x}\|}$ is a padding term chosen by
the user and $\gamma$ is a normalization factor (Appendix A). After initial
state preparation, we apply to $|\bar{{\bm{x}}}\rangle$ a series of
variational quantum circuits
$\\{U^{(j)}(\bm{\theta}^{(j)})\\}_{j=1}^{n_{\text{cir}}}$ according to
$|\bar{\bm{x}}_{f}^{(j)}\rangle=U^{(j)}(\bm{\theta}^{(j)})|\bar{\bm{x}}\rangle$,
followed by measurements of a set of observables
$\\{B_{i}\\}_{i=1}^{n_{\text{obs}}}$, to obtain the outcomes $\langle
B_{i}\rangle_{{\bm{\theta}}^{(j)},{\bm{\bar{}}{x}}}=\mathrm{Tr}(|\bar{\bm{x}}_{f}^{(j)}\rangle\langle\bar{\bm{x}}_{f}^{(j)}|B_{i})$.
The choices of $\\{B_{i}\\}$ are not unique and they can be chosen randomly
from the generalized Pauli basis $\\{P_{l}\\}_{l=1}^{4^{n}}$ where
$P_{l}=A_{1}^{(l)}\otimes A_{2}^{(l)}\otimes\cdots\otimes A_{n}^{(l)}$ with
$A_{k}^{(l)}\in\\{I,X,Y,Z\\}$, and $\\{X,Y,Z\\}$ are the Pauli matrices.
$U^{(j)}({\bm{\theta}}^{(j)})$ comprises $L$-layers of quantum circuits. Each
layer consists of $n$ parameterized $R$-rotations (with one on each qubit),
and $n$ parameterized controlled-$R$ gates, as shown in Fig. 1, with

$R=R(\theta_{1},\theta_{2},\theta_{3})=\left(\begin{matrix}e^{i\theta_{2}}\cos\theta_{1}&e^{i\theta_{3}}\sin\theta_{1}\\\
-e^{-i\theta_{3}}\sin\theta_{1}&e^{-i\theta_{2}}\cos\theta_{1}\end{matrix}\right).$

Notice that the order of the $2n$ gates in each layer is not unique and can be
chosen randomly [33]. Next, based on $\langle
B_{i}\rangle_{{\bm{\theta}}^{(j)},{\bm{\bar{}}{x}}}$ and the sigmoid function
$\sigma(x)=1/(1+e^{-x})$, the classical processor computes and obtains the
output:

$q_{{\bm{\theta}},{\bm{a}},{\bm{c}},{\bm{\alpha}}}(\bar{\bm{x}})\equiv\sum_{j=1}^{n_{\text{cir}}}\sum_{i=1}^{n_{\text{obs}}}\alpha_{i}^{(j)}\sigma(a_{i}^{(j)}(\langle
B_{i}\rangle_{{\bm{\theta}}^{(j)},\bar{\bm{x}}}-c_{i}^{(j)}))$ (2)

with $a_{i}^{(j)}>0$, $c_{i}^{(j)}\in[0,1]$ where $\bm{\theta}$ and
$({\bm{a}},{\bm{c}},{\bm{\alpha}})$ are parameters to be trained. The entire
process from $|\bar{{\bm{x}}}\rangle$ to
$q_{\bm{\theta},\bm{a},\bm{c},\bm{\alpha}}(\bar{\bm{x}})$ is summarized in
Fig. 1. Finally, one can find the optimal values of
$({\bm{\theta}},{\bm{a}},{\bm{c}},{\bm{\alpha}})$ to solve the given learning
problem, through gradient-based optimization using algorithms such as SGD
[48], ADAM [49] or BFGS [50]. The benefit of such construction is that DQNN
can be proved to be universal based on $L^{2}$ approximation, which is
summarized in the following theorem:

######  Theorem 1.

Let $\bar{G}$ be a subset of the complex sphere $\mathbb{S}$ in
$\mathbb{C}_{2}^{\otimes n}$, and
$f(\bar{{\bm{x}}}):\bar{G}\rightarrow\mathbb{R}$ be an arbitrary square-
integrable function on $\bar{G}$. We define the following parameterized
functions:

$\displaystyle
q_{\bm{z}_{1},\cdots,\bm{z}_{n_{s}},\bm{a},\bm{c},\bm{\alpha}}(\bar{\bm{x}})\equiv\sum_{j=1}^{n_{s}}\alpha_{j}\sigma(a_{j}(|\langle\bar{\bm{x}}|\bm{z}_{j}\rangle|^{2}-c_{j}))$
(3)

and denote
$Q(\bar{G})\equiv\\{q_{\bm{z}_{1},\cdots,\bm{z}_{n_{s}},\bm{a},\bm{c},\bm{\alpha}}\\}$
as the set of all such functions, where $n_{s}\in\mathbb{N}$,
$\\{\bm{z}_{j}\\}_{j=1}^{n_{s}}\subset\mathbb{S}$,
$\bm{a}\in\mathbb{R}_{+}^{n_{s}}$, $\bm{c}\in[0,1]^{\otimes n_{s}}$, and
$\bm{\alpha}\in\mathbb{R}^{n_{s}}$. Then $Q(\bar{G})$ is dense in
$L^{2}(\bar{G})$ in the following sense: for any $\epsilon>0$,

$\int_{\bar{G}}|q_{\bm{z}_{1},\cdots,\bm{z}_{n_{s}},\bm{a},\bm{c},\bm{\alpha}}(\bar{\bm{x}})-f(\bar{\bm{x}})|^{2}d\mu<\epsilon.$
(4)

The detailed proof of Theorem 1 can be found in Appendix B. It turns out that
any $q_{\bm{z}_{1},\cdots,\bm{z}_{n_{s}},\bm{a},\bm{c},\bm{\alpha}}\in
Q(\bar{G})$ in Eq. (3) can be generated as the output of the DQNN in Eq. (2).
Specifically, given the parameters
$(\bm{z}_{1},\cdots,\bm{z}_{n_{s}},\bm{a},\bm{c},\bm{\alpha})$, we can design
a DQNN with a single observable $B\equiv|b\rangle\langle b|$,
$n_{\text{obs}}=1$, and a series of quantum circuits
$\\{U^{(j)}(\bm{\theta}^{(j)})\\}_{j=1}^{n_{s}}$ such that
$U^{(j)}(\bm{\theta}^{(j)})|b\rangle=|z_{j}\rangle$. Then the output of the
DQNN in Eq. (2) is reduced to:

$\displaystyle
q_{{\bm{\theta}},{\bm{a}},{\bm{c}},{\bm{\alpha}}}(\bar{\bm{x}})$
$\displaystyle=\sum_{j=1}^{n_{s}}\alpha_{j}\sigma(a_{j}(\langle
B\rangle_{{\bm{\theta}}^{(j)},\bar{\bm{x}}}-c_{j}))$
$\displaystyle=\sum_{j=1}^{n_{s}}\alpha_{j}\sigma(a_{j}(|\langle\bar{\bm{x}}|\bm{z}_{j}\rangle|^{2}-c_{j}))$
$\displaystyle=q_{\bm{z}_{1},\cdots,\bm{z}_{n_{s}},\bm{a},\bm{c},\bm{\alpha}}(\bar{\bm{x}}).$
(5)

(a) large

Figure 1: (a) The framework of the DQNN consists of three parts, a quantum
processor, a classical processor and a classical optimizer. Measurement
results of a quantum processor are the inputs of a classical processor. The
parameters in the quantum and classical processors are updated by a classical
optimizer. (b) One layer variational quantum circuit of
$U^{(j)}(\bm{\theta}^{(j)})$. (c) The transformation from
$|\bar{{\bm{x}}}\rangle$ to
$q_{\bm{\theta},\bm{a},\bm{c},\bm{\alpha}}(\bar{\bm{x}})$. Yellow balls
represent the quantum qubits; green circles represent the classical sigmoid
function.

Thus we have proved the following corollary:

######  Corollary 1.

The DQNN designed in Fig. 1 (c) with the output in Eq. (2) is universal.

Notice that choosing multiple
$\\{U^{(j)}(\bm{\theta}^{(j)})\\}_{j=1}^{n_{\text{cir}}}$ and a single
observable $B$ is only a sufficient condition to prove the universality in
Corollary 1. In practice, we often design a DQNN with a single
$U(\bm{\theta})$ ($n_{\text{cir}}=1$) and a set of $\\{B_{i}\\}$ as the first
trial to solve the given problem. If it is not good enough, we then construct
a new DQNN with a larger $n_{\text{cir}}$. It turns out a DQNN with
$n_{\text{cir}}=1$ is often sufficient for most learning problems we have
studied. In addition, in the proof of Corollary 1, we have assumed that the
variational circuits $\\{U^{(j)}(\bm{\theta}^{(j)})\\}$ with
$U^{(j)}(\bm{\theta}^{(j)})|b\rangle=|z_{j}\rangle$ can be generated by our
designed multi-layer variational circuits as shown in Fig. 1 (b). More layers
in $U^{(j)}(\bm{\theta}^{(j)})$ will make it more powerful to approximate
arbitrary unitary gate, but will also make the optimization process
computationally more expensive. Hence, in practice, we often construct a DQNN
starting from a single $U(\bm{\theta})$ consisting of one or two layers. In
the following, we will study the performance of DQNN with different values of
$n_{\text{cir}}$, and for simplicity, we will denote the corresponding network
as DQNN${}_{n_{\text{cir}}}$.

##  IV Performance comparison among QNN models

In Section II, we have mentioned that universality serves as a necessary
condition to build an effective QNN, and among the two types of universal
design, an NN with $L^{2}$ convergence using sigmoid basis functions works
more effectively in a broader range of applications. As shown above, the
nonlinearity and the universality of DQNN originates essentially from the
$L^{2}$ convergence of sigmoids functions, which is distinctive from other
polynomial-approximation-based QNN models, including the CCQ [33] and the QCL
[32], whose nonlinearity is mainly generated by duplicating the same data-
encoding state into a tensor product of multiple subsystems. In this way, DQNN
has significantly reduced the number of qubits required to construct a
universal QNN. In the following, we will demonstrate the advantages of DQNN by
comparing its performance with those of the two well-established QNN models,
the CCQ and the QCL.

For an $n_{\text{tot}}$-qubit QNN model, we define its complexity as $C\equiv
n_{\text{gate}}n_{\text{m}}$, where $n_{\text{gate}}$ denotes the number of
quantum gates and $n_{\text{m}}$ denotes the number of quantum measurements.
Notice that given a measurement precision, $n_{\text{m}}$ is proportional to
the number of measurement observables $n_{\text{obs}}$. Hence, with a further
assumption that the QNN variational circuit can be efficiently generated,
i.e., $n_{\text{gate}}=O(\text{poly}\big{(}n_{\text{tot}})$, the overall
complexity of the QNN then becomes
$C=O(\text{poly}\big{(}n_{\text{tot}})n_{\text{obs}}\big{)}$. We hope to
figure out the numbers of qubits and observables required to approximate an
$M$-order polynomial function $f(\bm{x})=\sum_{i=0}^{M}c_{i}\bm{x}^{i}$ with
$||\bm{x}||=1$ using CCQ, QCL and DQNN respectively. We denote by
$n_{\text{data}}$ the number of qubits in the data-encoding register, and by
$n_{\text{copy}}$ the number of copies of the data-encoding register used in a
QNN model. To approximate such $f$, CCQ requires $n_{\text{copy}}=\frac{M}{2}$
duplicates of the data-encoding register
$|\bar{\bm{x}}\rangle=\frac{1}{\gamma}\sum_{i=1}^{d+1}x_{i}|i\rangle$ such
that the amplitude of the input state,
$|\psi_{{\bm{x}}}\rangle=|\bar{\bm{x}}\rangle^{\otimes\frac{M}{2}}$, is an
$\frac{M}{2}$-order polynomial of $\bm{x}$. After applying a circuit
$U(\bm{\theta})$, we measure the expectation values of a given observable $B$
according to $\langle
B\rangle_{\bm{\theta},\bm{x}}=\langle\psi_{\bm{x}}|U(\bm{\theta})^{\dagger}BU(\bm{\theta})|\psi_{{\bm{x}}}\rangle$.
The output of the CCQ is an $M$-order polynomial of $\bm{x}$. Then the goal of
CCQ is to find the best value $\bm{\theta}=\bm{\theta}^{*}$ through
optimization such that $\bm{x}\rightarrow\langle
B\rangle_{\bm{\theta}^{*},\bm{x}}$ is the optimal function to approximate
$\bm{x}\rightarrow f(\bm{x})$. The number of qubits in the data-encoding
register is $n_{\text{data}}=\lceil\log d\rceil$ and the number of observables
is $n_{\text{obs}}=1$. Hence the total complexity of CCQ is equal to
$C_{\text{CCQ}}=O(\text{poly}(M\lceil\log d\rceil))$. For QCL, a
$d$-dimensional classical data $\bm{x}$ is encoded into a $d$-qubit data-
encoding register as
$\rho(\boldsymbol{x})=\frac{1}{2^{d}}\otimes_{i=1}^{d}(\text{I}+x_{i}X+\sqrt{1-x_{i}^{2}}Z])$
using several single-qubit rotation gates. In order to generate an $M$-order
polynomial, $\rho(\boldsymbol{x})$ is duplicated into $M$ identical copies in
a tensor product as
$\rho_{\text{in}}(\boldsymbol{x})=\otimes_{j=1}^{M}\rho(\boldsymbol{x})$.
Applying a circuit $U(\bm{\theta})$ to $\rho_{\text{in}}$, we measure the
expectation value of a given observable $B$. The outcome $\langle
B\rangle_{\bm{\theta},\bm{x}}$ is an $M$-order polynomial of $\bm{x}$. Through
optimization, we can find the best value $\bm{\theta}^{*}=\bm{\theta}$ such
that $\bm{x}\rightarrow\langle B\rangle_{\bm{\theta}^{*},\bm{x}}$ will well
approximate $\bm{x}\rightarrow f(\bm{x})$. The total number of qubits in QCL
is $n_{\text{tot}}=Md$ and the number of observables is $n_{\text{obs}}=1$.
Hence the total complexity of QCL is $C_{\text{QCL}}=O(\text{poly}(Md))$.

Different from the above two duplication-based QNN models, DQNN only uses
$\lceil\log d\rceil$ qubits to encode $\bm{x}$ as
$|\bar{\bm{x}}\rangle=\frac{1}{||\bm{x}||}\sum_{i=1}^{d+1}x_{i}|i\rangle$. We
sequentially apply a set of variational quantum circuits
$\\{U^{(j)}(\bm{\theta}^{(j)})\\}_{j=1}^{n_{\text{cir}}}$ to
$|\bar{\bm{x}}\rangle$ and measure the expectation $\langle
B_{i}\rangle_{\bm{\theta}^{(j)},\bar{\bm{x}}}=\langle\bar{\bm{x}}|U^{(j)\dagger}(\bm{\theta}^{(j)})B_{i}U^{(j)}(\bm{\theta}^{(j)})|\bar{\bm{x}}\rangle$
for a set of chosen observables $\\{B_{i}\\}_{i=1}^{n_{\text{obs}}}$. Next we
apply the parameterized classical sigmoid function $\sigma$ and derive the
final output $q_{\bm{\theta},\bm{a},\bm{c},\bm{\alpha}}$. Through the
optimization, we can find the optimal values
$(\bm{\theta}^{*},\bm{a}^{*},\bm{c}^{*},\bm{\alpha}^{*})$ to approximate the
function $f$. The number of measurement observables $n_{\text{obs}}$ is
determined by $f$. Hence the total complexity of DQNN is
$C_{\text{DQNN}}=O(\text{poly}(\lceil\log d\rceil)n_{\text{obs}})$. The
comparison is summarized in Table 1. It can be seen that for large $d$ and
large $M$, our DQNN can significantly reduce the total complexity to generate
nonlinearity. More details can be found in Appendix C.

Algorithm | $n_{\text{copy}}$ | $n_{\text{data}}$ | Complexity
---|---|---|---
QCL | $M$ | $d$ | $O(\text{poly}(Md))$
CCQ | $\frac{M}{2}$ | $\lceil\log d\rceil$ | $O(\text{poly}(M\lceil\log d\rceil))$
DQNN | $1$ | $\lceil\log d\rceil$ | $O(\text{poly}(\lceil\log d\rceil)n_{\text{obs}})$
Table 1: The number of required qubits and the circuit complexity among three
proposals to approximate an $M$-order polynomial function $f$ of $\bm{x}$.

Figure 2: (a) The regression data set generated by a polynomial function. (b)
The classification data set with the ring-shaped boundaries. The points in the
yellow part are labeled $0$; the others are labeled $1$. (c) The approximation
process of the DQNN1 with one variational quantum circuit and $4$ observables
in solving the regression problem based on the dataset in (a). In the task,
one more sigmoid function (the rightmost node) is added into the DQNN1 to
transform the output of DQNN1 into a range $(0,1)$.

To demonstrate that our DQNN has advantage in solving classical ML problems
and in reducing circuit complexity, we apply DQNN, QCL, and CCQ to two typical
supervised learning problems, a regression problem and a classification
problem, and compare their performance. The first problem is learning from a
data set $\\{{\bm{x}}^{(i)},y^{(i)}\\}$ to approximate a highly nonlinear
polynomial
$f({\bm{x}})=(0.7156-1.0125x_{1}^{2}+x_{1}^{4})(0.7156-1.0125x_{2}^{2}+x_{2}^{4})$
with $x_{1},x_{2}\in[-0.8,0.8]^{2}$ (Fig. 2). For input ${\bm{x}}^{(i)}$, the
output of the QNN is denoted as $q^{(i)}$. One can define the relative error
$\epsilon\equiv\frac{1}{N}\sum_{i}|\frac{q^{(i)}-y^{(i)}}{y^{(i)}}|$ to
measure the performance of the QNN in solving regression problems. A tiny
relative error implies that the QNN approximates the target function $f$ well.
First, we run the hybrid optimization process for each model based on the data
set. To show the effectiveness of the three QNN models, we choose different
values of the number of layers for each model so that their circuit complexity
$C$ is similar in size. We use the same definition of $n_{\text{copy}}$ as in
the above section. Then, we compare the optimal relative errors achieved for
each model. We choose two types of DQNN to solve the regression problem. One
has a single-layer variational quantum circuit and multiple observables. We
name it DQNN1. The other has $4$ single-layer variational quantum circuits and
one observable. We name it DQNN4. We find that, with no duplicate,
$n_{\text{copy}}=1$, the relative error achieved by DQNN is substantially
lower than those achieved by QCL and CCQ, as shown in Table 2(a). If we add
one more duplicate to both the QCL and the CCQ models, their optimal relative
error will decrease, but their complexity will increase and surpass that of
the DQNN. In addition, we implement such comparison for a few other examples,
and simulation results suggest that DQNN can prominently reduce the circuit
complexity to approximate highly nonlinear functions compared with QCL and
CCQ. The approximation process of the DQNN1 with $4$ observables is shown in
Fig. 2, and more details of the comparison can be found in Appendix D.

We apply the three models to a binary classification problem on a ring-shaped
data set in the second task. The boundaries of the two sub-datasets are
determined by six curves $x_{1}^{2}+x_{2}^{2}=0.16$,
$x_{1}^{2}+x_{2}^{2}=0.81$, and $x_{1}=x_{2}=\pm 1$ (Fig. 2). One can use
classification accuracy to quantify the performance of QNN model in solving
classification problems, which is defined as
$\epsilon\equiv\frac{1}{N}\sum_{i}1_{\delta(q^{(i)}-y^{(i)})}$ where $q^{(i)}$
is the output of QNN, $y^{(i)}$ represents the label of ${\bm{x}}^{(i)}$ and
$\delta$ is the Dirac delta function [51]. Analogous to the first task, we
choose the DQNN with one variational quantum circuit and multiple observables
(DQNN1) and the DQNN with $4$ variational quantum circuits and one observable
(DQNN4) for the classification task. We choose appropriate values for
$n_{\text{copy}}$ and $n_{\text{lay}}$, so that the circuit complexity $C$ of
the three models is similar in size. After optimizing the three QNN models, we
find that accuracy achieved by the DQNN is substantially higher than those
achieved by QCL and CCQ. In addition, increasing the number of duplicates does
help QCL and CCQ to improve their optimal accuracy, but even a $5$-copy model
(with 12 qubits) for QCL or CCQ cannot perform equally well as a DQNN with
only two qubits (Table 2(b)). Next, we implement such a comparison for a few
other examples. Simulation results suggest that DQNN can prominently reduce
the circuit complexity compared to QCL and CCQ, but maintain a good
performance in solving classification problems.

Model | $n_{\text{data}}$ | Copy number ($n_{\text{copy}}$) | # Qubits ($n_{\text{tol}}$) | # Layers ($n_{\text{lay}}$) | # Observables ($n_{\text{obs}}$) | $C$ | Relative error
---|---|---|---|---|---|---|---
DQNN1 | 2 | 1 | 2 | 1 | 4 | 48 | 6.79%
DQNN4 | 2 | 1 | 2 | 1 | 1 | 48 | 5.46%
QCL | 2 | 1 | 2 | 4 | 1 | 48 | 8.21%
CCQ | 2 | 1 | 2 | 4 | 1 | 51 | 12.92%
QCL | 2 | 2 | 4 | 5 | 1 | 120 | 7.35%
CCQ | 2 | 2 | 4 | 4 | 1 | 99 | 4.74%
(a) Regression problem to approximate
$f({\bm{x}})=(0.7156-1.0125x_{1}^{2}+x_{1}^{4})(0.7156-1.0125x_{2}^{2}+x_{2}^{4})$

Model | $n_{\text{data}}$ | Copy number ($n_{\text{copy}}$) | # Qubits ($n_{\text{tol}}$) | # Layers ($n_{\text{lay}}$) | # Observables ($n_{\text{obs}}$) | $C$ | Accuracy
---|---|---|---|---|---|---|---
DQNN1 | 2 | 1 | 2 | 1 | 20 | 240 | 91.40%
DQNN4 | 2 | 1 | 2 | 1 | 1 | 48 | 97.63%
QCL | 2 | 2 | 4 | 5 | 2 | 240 | 74.18%
CCQ | 2 | 2 | 4 | 10 | 1 | 243 | 75.20%
QCL | 2 | 6 | 12 | 10 | 2 | 1440 | 76.93%
CCQ | 2 | 6 | 12 | 10 | 1 | 723 | 80.63%
(b) A classification problem defined on a ring-shaped dataset

Table 2: A comparison of the performance among DQNN, QCL and CCQ in two
classical ML problems. $n_{\text{data}}$ indicates the number of qubits to
store the classical data. The total number of qubits in each QNN model is
equal to $n_{\text{tot}}=n_{\text{data}}n_{\text{copy}}$. We use $C\equiv
n_{\text{gate}}n_{\text{obs}}$ to represent the circuit complexity where
$n_{\text{gate}}$ is the number of quantum gates in the variational quantum
circuit and $n_{\text{obs}}$ is the number of observables. For QCL and CCQ,
$n_{\text{copy}}$, $n_{\text{lay}}$ are free to choose; for DQNN,
$n_{\text{lay}}$ and $n_{\text{obs}}$ are free to choose. We choose two types
of DQNN in the simulations: DQNN1 contains one circuit and multiple
observables, while DQNN4 contains four circuits and one observable. All of the
results are obtained by averaging $10$ independent simulations.

Since the DQNN can reduce the circuit complexity, we expect that it can
achieve a better performance in the presence of noise, and hence is easier to
be implemented on NISQ devices. We will consider two types of noise in this
work: one is the coherent noise caused by the imprecision of the classical
control on the parameter values in the QNN circuits; the other is the
decoherence generated by interactions between the quantum register and its
environment. We add a coherent noise $\epsilon_{j}$ to the $j$-th parameter,
$\theta_{j}$, in the variational circuit according to
$\tilde{\theta}_{j}=\theta_{j}+\epsilon_{j}$ when we train the three QNN
models, and test their performance under the noisy environment. The coherent
noise $\epsilon_{j}$ is generated by sampling from a Gaussian distribution,
$N(0,\Delta^{2})$ where $\Delta$ indicates the noise intensity. It can be seen
from Fig. 3 and 3 that for the regression task, the DQNN achieves lower
relative error than QCL and CCQ when we increase the coherent noise.
Meanwhile, for the classification task (as shown in Fig. 3), DQNN is more
robust than the other two when the coherent noise increases. In addition to
the coherent noise, we further apply the decoherence, $\epsilon$, to the two-
qubit control gates, $U_{CR}$, contained in the quantum circuits according to
$\epsilon(U_{CR}\rho U_{CR}^{\dagger})=(1-p)U_{CR}\rho U_{CR}^{\dagger}+p\sum
P_{i}U_{CR}\rho U_{CR}^{\dagger}P_{i}^{\dagger}$ where $\\{P_{i}\\}$ indicates
the complete Pauli basis and $p$ represents the probability of decoherence
occurrence. From Fig. 3, we can find that, in the regression task, the DQNN
has a significantly lower relative error than QCL and CCQ as the probability
of decoherence increases. These numerical results demonstrate that the DQNN
can decrease the influence of noise accumulation in the training process and
is more likely to be implemented on near-term devices.

Figure 3: Comparison of the robustness for three QNN models in solving the
regression and classification problems. In the regression task, we use DQNN1,
$2$-qubit QCL and $2$-qubit CCQ for the comparison. In the classification
task, we use DQNN1, $4$-qubit QCL and $4$-qubit CCQ for the comparison.
(a)-(b) Relative errors achieved by QCL, CCQ and DQNN as the coherent noise
increases in the regression task. (c) The comparison among three QNNs in the
classification task as the coherent noise increases. (d) The comparison among
three QNNs in the regression task as the probability of decoherence occurrence
increases.

Besides the comparison among three QNN models over two supervised learning
problems, we further demonstrate the performance of the different QNNs in
solving the typical real-world machine learning problem based on the
handwritten digits data set, MNIST [52]. In this task, a QNN is first trained
with $60,000$ figures, each of which has a number ranging from $0$ to $9$, and
then is tested with $10,000$ new figures to generate the classification
accuracy. A figure in the MNIST data set is reshaped to be a $256$-dimensional
vector and encoded into an $8$-qubit quantum state by using the amplitude
encoding method. Since the QCL needs $256$ qubits to encode the classical
data, it is intractable to implement the QCL in solving this task and we only
compare the performance of the DQNN and the CCQ. For our DQNN, we use $20$
$2$-layer variational circuits and one observable $B=Z^{(1)}\otimes
Z^{(2)}\otimes\cdots\otimes Z^{(8)}$. The classification accuracy over
$10,000$ new figures can achieve $94.36\%$. For the CCQ, we use one
variational quantum circuit and a set of POVM operators
$\\{B_{j}\\}_{j=1}^{10}$ to generate the final output. We adopt different
layers for the variational quantum circuit, however, the performance of the
CCQ is still significantly lower than the DQNN. For instance, the CCQ with a
$20$-layer variational quantum circuit only achieves $83.52\%$ accuracy. The
result demonstrates that the QNN without a strict universality guarantee will
fail in solving machine learning problems.

##  V Comparison with classical neural networks

To demonstrate that our DQNN can perform at least as effectively as the
classical NN, we apply the DQNN to solve three typical real-world
classification problems and further compare its performance with the classical
NN in the minimum parameters required the number of iterations to achieve a
similar classification accuracy. Since our DQNN is essentially with a single
hidden layer, we will choose a single-hidden-layer classical neural network
for the comparison. On the handwritten digits data set, MNIST [52], we
implement a binary classification (MNIST2) to distinguish the digits $0$ and
$1$, and a $3$-target classification (MNIST3) to recognize the digits $0$,
$1$, $2$. Each sample is reshaped into a $256$-dimensional vector and taken as
the input of DQNN and classical NN. Meanwhile, we implement two more
classification tasks based on the Wine data set [53] and the Breast Cancer
data set [53]. We use a $4$-qubit state to store the $13$-dimensional
classical data for the Wine problem, and a $4$-qubit state to store the
$10-$dimensional data for the Breast Cancer problem. As shown in Table 3, the
DQNN performs as well as the classic neural networks in discovering complex
patterns hidden in real-world data sets. Notice that DQNN uses fewer
parameters to achieve similar accuracy than the classical counterpart but is
at the extra cost of required iterations in some cases.

Model | Task | $N_{\text{Train}}$ | $N_{\text{Test}}$ | # Parameters | # Iterations | $\text{Acc}_{\text{Train}}$ | $\text{Acc}_{\text{Test}}$
---|---|---|---|---|---|---|---
DQNN | MNIST2 | $12665$ | $2115$ | $54$ | $150$ | $98.66\%$ | $99.05\%$
Classical NN | MNIST2 | $12665$ | $2115$ | $259$ | $26$ | $98.54\%$ | $99.07\%$
DQNN | MNIST3 | $18623$ | $3147$ | $495$ | $254$ | $98.61\%$ | $99.03\%$
Classical NN | MNIST3 | $18623$ | $3147$ | $523$ | $367$ | $98.48\%$ | $99.02\%$
DQNN | Wine | $143$ | $35$ | $27$ | $172$ | $96.29\%$ | $100\%$
Classical NN | Wine | $143$ | $35$ | $83$ | $230$ | $97.04\%$ | $100\%$
DQNN | Breast Cancer | $560$ | $39$ | $34$ | $282$ | $94.46\%$ | $95.74\%$
Classical NN | Breast Cancer | $560$ | $39$ | $55$ | $105$ | $93.57\%$ | $95.89\%$
Table 3: A comparison of the minimum parameters required and the number of
iterations to achieve a similar classification accuracy between DQNN and
classical neural network on three typical real-world data sets. We use the
ADAM algorithm to optimize the parameters. $N_{\text{Train}}$ and
$N_{\text{Test}}$ respectively denote the number of the training set and the
test set. $\text{Acc}_{\text{Train}}$ and $\text{Acc}_{\text{Test}}$ represent
the classification accuracy on each data set.

##  VI Quantum phase recognition

Besides the classical ML problems, our DQNN model can solve ML problems on
quantum data sets. In the following, we concentrate on a class of quantum
phase recognition problems, the discrimination of the
$\mathbb{Z}_{2}\times\mathbb{Z}_{2}$ symmetry-protected topological (SPT)
phase on a $S=1$ Haldane chain [54]. The ground state of the Hamiltonian,

$\displaystyle H=$
$\displaystyle-J\sum_{i=1}^{N-2}Z^{(i)}X^{(i+1)}Z^{(i+2)}-h_{1}\sum_{i=1}^{N}X^{(i)}$
$\displaystyle-h_{2}\sum_{i=1}^{N-1}X^{(i)}X^{(i+1)},$

has three topological phases, the SPT phase, the paramagnetic phase and the
antiferromagnetic phase, where $h_{1}$ and $h_{2}$ are parameters. Our goal is
to identify whether a given state sampled from the phase diagram in Fig. 4
belongs to the SPT phase. To complete such a task, we take $20\times 20$
equally spaced points from $h_{1}\in[0,1.6]$ and $h_{2}\in[-1.6,1.6]$ as the
training set and $64\times 64$ equally spaced points as the test set. If the
ground state belongs to the SPT phase, it is labeled $[1,0]^{T}$; otherwise,
it is labeled $[0,1]^{T}$. We numerically implement the DQNN with a single
$15$-qubit variational quantum circuit with $420$ parameters and $10$
observables. After training, the accuracy of the DQNN on the test data
achieves $99.10\%$. It indicates that our DQNN model has solved the quantum
phase recognition problem with good performance. However, the training set to
train the DQNN is larger than the training set to train the quantum
convolutional neural network [55] for phase recognition task. The reason for
the difference is that the DQNN used in the comparison is only a prototype,
while the quantum convolutional neural network in [55] has been optimized for
this task, with a stronger ability to analyze the local information of quantum
states. Hence, as a future work, we hope to improve the DQNN design to
optimize its performance for the given ML problem.

Figure 4:  The phase diagram of the $S=1$ Haldane chain. The phase boundary is
generated by the 2-degree polynomial regression based on some boundary points.

##  VII Conclusion

There have been many successes in achieving larger scale and higher fidelity
of the NISQ devices. These improvements provide the quantum neural network
with the opportunity to deal with larger data sets and even outperform the
classical counterpart. This paper presents a duplication-free quantum neural
network model, DQNN, to solve machine learning problems. Meanwhile, we provide
it with the universal guarantee to approximate any arbitrary continuous
function using several variational quantum circuits, multiple measurement
observables and the classical parameterized sigmoid function. We can enhance
the expressibility of DQNN by increasing the number of quantum circuits and
the number of observables without requiring auxiliary qubits. This property
for the number of qubits makes DQNN more likely to be implemented on
intermediate-scale quantum computers.

Unlike other variational QNN proposals, DQNN utilizes the classical computer
in the hybrid system to generate nonlinearity. Without duplicates, DQNN
significantly reduces the number of required qubits and decreases circuit
complexity compared with two well-known QNN models. Our simulation results
demonstrate that, compared with QCL and CCQ, DQNN uses fewer qubits and fewer
quantum gates to outperform the other two QNN models and hence weaken the
influence of the circuit noise. Furthermore, DQNN requires fewer parameters
than the classical counterpart but is at an extra cost of the iterations in
some cases. DQNN can also be used to solve quantum problems. These results
indicate that the DQNN is an efficient QNN model which can find the patterns
hidden in the classical and quantum data sets.

Like the classical naive neural network, DQNN can be considered a subroutine
to construct more complex quantum deep learning models. In addition, DQNN can
also be improved by optimizing its structure for the given ML problem. For
example, DQNN can be combined with the concept of recurrent neural network to
efficiently solve the natural language processing tasks, or integrated into a
quantum reinforcement learning framework. Moreover, due to its simple design,
DQNN is relatively easy to be implemented on NISQ devices for QNN
demonstration.

## Acknowledgements

The authors gratefully acknowledge the grant from the National Key R&D Program
of China, Grant No. 2018YFA0306703. We also thank Chu Guo, Bujiao Wu, Yusen
Wu, Shaojun Wu, Yuhan Huang, Donghong Han, Yingli Yang and Yi Tian for helpful
discussions.

Figure 5: Via the amplitude encoding process
$F:\bm{x}\mapsto|\bar{\bm{x}}\rangle$, approximating
$f(\bm{x}):G\rightarrow\mathbb{R}$ is equivalent to the the problem of
approximating $f(F^{-1}|\bar{\bm{x}}\rangle)$, which can be achieved
efficiently by DQNN.

##  Appendix A Amplitude encoding method

Suppose $\\{{\bm{x}}^{(i)}\\}$ of the data set
$D=\\{{\bm{x}}^{(i)},y^{(i)}\\}$ is contained in a subset $G$ of
$\mathbb{R}^{d}$. Without loss of generality, we can choose an appropriate
coordinate frame through translation such that $\bm{0}\notin G$ and every
$\boldsymbol{x}\in G$ satisfies
$0<\kappa_{1}\leq|\boldsymbol{x}|\leq\kappa_{2}$ with constants $\kappa_{1}$
and $\kappa_{2}$. In order to apply variational QNN, we encode the classical
data into the states of a quantum register. In this work, we define the
encoding mapping $F:G\rightarrow\bar{G}$ such that
$|\bar{\bm{x}}\rangle=F(\boldsymbol{x})$ is an $n$-qubit state in the Hilbert
space $\mathbb{C}_{2}^{\otimes n}$ with $d\leq 2^{n}$:

$F:\bm{x}=(x_{1},...,x_{d})^{T}\mapsto|{\bar{\bm{x}}}\rangle\equiv\gamma^{-1}(x_{1},...,x_{d},\tilde{x},0,...,0)^{T}$
(6)

where $\tilde{x}=\frac{\|\boldsymbol{x}\|}{1+\|\boldsymbol{x}\|}$,
$\gamma=(\|\boldsymbol{x}\|^{2}+\tilde{x}^{2})^{\frac{1}{2}}$. It is easy to
validate that
$\frac{\kappa_{1}}{1+\kappa_{1}}\leq\tilde{x}\leq\frac{\kappa_{2}}{1+\kappa_{2}}$
and

$(1+(1+\kappa_{2})^{2})^{-\frac{1}{2}}<\bar{x}_{d+1}<(1+(1+\kappa_{1})^{2})^{-\frac{1}{2}}$
(7)

with $\bar{x}_{d+1}=\gamma^{-1}\tilde{x}$. Such encoding method is called
amplitude encoding, since the classical information $\bm{x}$ is encoded into
the amplitudes of $|\bar{\bm{x}}\rangle$, as illustrated in Fig. 5. Therefore,
approximating $f(\bm{x}):G\rightarrow\mathbb{R}$ is equivalent to the
approximation of $f(F^{-1}|\bar{\bm{x}}\rangle):\bar{G}\rightarrow\mathbb{R}$.

##  Appendix B Proof of Theorem 1

It suffices to show
$Q(\bar{G})=\\{q_{\bm{z}_{1},\cdots,\bm{z}_{n_{s}},\bm{a},\bm{c},\bm{\alpha}}\\}$
is dense in $L^{2}(\bar{G})$. We will prove it by contradiction. Assume that
the closure $\overline{Q(\bar{G})}\neq L^{2}(\bar{G})$. By the Hahn-Banach
theorem [56], there exists a bounded linear functional $l$ on $L^{2}(\bar{G})$
such that $l(Q(\bar{G}))=0$ and $l\neq 0$. By the Riesz representation theorem
[57], there exists a function $g(\bar{\bm{x}})\in L^{2}(\bar{G})$ such that
the linear functional $l$ can be represented as

$l(f)=\int_{\bar{G}}f(\bar{\boldsymbol{x}})g(\bar{\boldsymbol{x}})d\mu\qquad\text{for
all }f\in L^{2}(\bar{G}).$ (8)

Note that $l\neq 0$ implies $g\neq 0$ on $\bar{G}$. Therefore, the following
two sets
$S_{1}=\\{\bar{\boldsymbol{x}}\in\bar{G}:g(\bar{\boldsymbol{x}})>0\\}$ and
$S_{2}=\\{\bar{\boldsymbol{x}}\in\bar{G}:g(\bar{\boldsymbol{x}})<0\\}$ are
measurable and at least one of them has a positive measure. Without loss of
generality, we assume that $g(\bar{\boldsymbol{x}})>0$ almost everywhere in an
open subset $E\subset\bar{G}$ with $\mu(E)>0$. Substituting
$f(\bm{x})=\sigma(a(|\langle\bar{\bm{x}}|\bm{z}\rangle|^{2}-c))\in Q(\bar{G})$
into above equation, and in view of $l(Q(\bar{G}))=0$, we have:

$\int_{\bar{G}}\sigma(a(|\langle\bar{\boldsymbol{x}}|\boldsymbol{z}\rangle|^{2}-c))g(\bar{\boldsymbol{x}})d\mu=0.$
(9)

As the open set $E$ contains a small ball
$B(\boldsymbol{z}^{*},\delta)=\\{\bar{\boldsymbol{x}}\in\bar{G}:|\bar{\boldsymbol{x}}-\boldsymbol{z}^{*}|<\delta\\}$,
we will show that, by choosing large enough $a>0$ and $0<c<1$, the function
$\sigma(a(|\langle\bar{\boldsymbol{x}}|\boldsymbol{z}\rangle|^{2}-c))$ tends
to $1$ in a smaller ball $B(\boldsymbol{z}^{*},\delta_{1})\subset
B(\boldsymbol{z}^{*},\delta)$ $(\delta_{1}<\delta)$ and vanishes quickly
outside of $B(\boldsymbol{z}^{*},\delta_{1})$, which leads to a contradiction
to Eq. (9).

Since $\bar{G}$ is a real subset of the unit sphere,

$|\bar{\boldsymbol{x}}-\boldsymbol{z}^{*}|^{2}=2-2\langle\bar{\boldsymbol{x}}|\boldsymbol{z}^{*}\rangle\qquad\text{
for all }\bar{\boldsymbol{x}}\in\bar{G},$ (10)

which implies that
$|\langle\bar{\boldsymbol{x}}|\boldsymbol{z}^{*}\rangle|^{2}>c$ is equivalent
to one of the following two cases:

$\text{(i)}\
|\bar{\boldsymbol{x}}-\boldsymbol{z}^{*}|^{2}<2(1-\sqrt{c})\quad\text{or}\quad\text{(ii)}\
|\bar{\boldsymbol{x}}-\boldsymbol{z}^{*}|^{2}>2(1+\sqrt{c}).$ (11)

For $c\to 1$, case (i) means $\bar{\boldsymbol{x}}$ is sufficiently close to
$\boldsymbol{z}^{*}$, and (ii) means $\bar{\boldsymbol{x}}$ is sufficiently
close to $-\boldsymbol{z}^{*}$, but the latter case is impossible.
Specifically, for $c$ sufficiently close to $1$, and a given value
$\kappa_{2}>0$, we can choose $c$ such that
$c\geq(1-2(1+(1+\kappa_{2})^{2})^{-1})^{2}$, and together with Eq. (7), we
have:

$|\bar{\boldsymbol{x}}+\boldsymbol{z}^{*}|^{2}\geq(z^{*}_{d+1}+\bar{x}_{d+1})^{2}\geq
4(1+(1+\kappa_{2})^{2})^{-1}\geq 2(1-\sqrt{c})$ (12)

If (ii) were true, then by
$|\bar{\boldsymbol{x}}-\boldsymbol{z}^{*}|^{2}+|\bar{\boldsymbol{x}}+\boldsymbol{z}^{*}|^{2}=4$,
we would find $|\bar{\boldsymbol{x}}+\boldsymbol{z}^{*}|^{2}<2(1-\sqrt{c})$,
contradictory to Eq. (12). Hence case (ii) is impossible. Therefore, when
$c\rightarrow 1$,
$|\langle\bar{\boldsymbol{x}}|\boldsymbol{z}^{*}\rangle|^{2}>c$ is only
equivalent to case (i). Passing to the limit $a\rightarrow+\infty$, we get

$\sigma(a(|\langle\bar{\boldsymbol{x}}|\boldsymbol{z}^{*}\rangle|^{2}-c))\rightarrow\left\\{\begin{aligned}
&1\quad\text{ for $\bar{{\bm{x}}}$ with
}|\bar{\boldsymbol{x}}-\boldsymbol{z}^{*}|<\delta_{1},\\\ &0\quad\text{ for
$\bar{{\bm{x}}}$ with
}|\bar{\boldsymbol{x}}-\boldsymbol{z}^{*}|>\delta_{1},\end{aligned}\right.$
(13)

with $\delta_{1}=(2(1-\sqrt{c}))^{\frac{1}{2}}$. Taking $c$ sufficiently close
to $1$, we have $\delta_{1}\leq\delta$, and from Eq. (9) we obtain

$\displaystyle 0$
$\displaystyle=\int_{\bar{G}}\sigma(a(|\langle\bar{\boldsymbol{x}}|\boldsymbol{z}\rangle|^{2}-c))g(\bar{\boldsymbol{x}})d\mu$
(14)
$\displaystyle\rightarrow\int_{B(\boldsymbol{z}^{*},\delta_{1})}1g(\bar{\boldsymbol{x}})d\mu>0\quad(\text{as
}a\rightarrow+\infty),$ (15)

which is a contradiction. Hence, we conclude that $Q(\bar{G})$ is dense in
$L^{2}(\bar{G})$. Thus for any $f\in L^{2}(\bar{G})$ and $\epsilon>0$, there
exists a $q(\bar{\boldsymbol{x}})\in Q(\bar{G})$ such that

$\|f-q\|^{2}_{L^{2}(\bar{G})}=\int_{\bar{G}}|f(\bar{\boldsymbol{x}})-q(\bar{\boldsymbol{x}})|^{2}d\mu\leq\epsilon,$
(16)

which proves Theorem 1.

######  Remark 1.

If $\sigma$ is a ReLU-type function, then the conclusion still holds. In fact,
instead of Eq.(13), we have
$\sigma(a(|\langle\bar{\boldsymbol{x}}|\boldsymbol{z}\rangle|^{2}-c))>0$ for
all $\bar{\boldsymbol{x}}\in B(\boldsymbol{z}^{*},\delta_{1})$ and
$\sigma(a(|\langle\bar{\boldsymbol{x}}|\boldsymbol{z}\rangle|^{2}-c))=0$
otherwise, which leads to the contradiction:

$\displaystyle 0$
$\displaystyle=\int_{\bar{G}}\sigma(a(|\langle\bar{\boldsymbol{x}}|\boldsymbol{z}\rangle|^{2}-c))g(\bar{\boldsymbol{x}})d\mu$
$\displaystyle=\int_{B(\boldsymbol{z}^{*},\delta_{1})}\sigma(a(|\langle\bar{\boldsymbol{x}}|\boldsymbol{z}\rangle|^{2}-c))g(\bar{\boldsymbol{x}})d\mu>0.$

##  Appendix C Introduction to other variational QNN models

In order to compare the performance of DQNN with other models, in this
section, we will present a brief introduction to two established variational
QNN models, circuit-centric quantum classifier (CCQ) [33] and quantum circuit
learning (QCL) [32], and will apply these models to solve the following
quantum learning problem: constructing a QNN to approximate an $M$-order
polynomial $f$: ${\bm{x}}\to f({\bm{x}})$, where $M$ is an even number.

First, we consider the CCQ model, where the classical data
$\bm{x}=[x_{1},x_{2},\cdots,x_{d}]^{T}\in\mathbb{R}^{d}$ with $||\bm{x}||=1$
is mapped into a $n_{\text{data}}$-qubit register with
$n_{\text{data}}\equiv\lceil\log d\rceil$, using the following encoding
scheme:

$|\bar{{\bm{x}}}\rangle=\frac{1}{\gamma}[x_{1},x_{2},\cdots,x_{d},\tilde{x},0,\cdots
0]^{T}\in\mathbb{C}_{2}^{\otimes n_{\text{data}}},$ (17)

where $\tilde{x}$ is a padding term chosen by the user, and $\gamma$ is the
normalization factor. Notice that $|\bar{{\bm{x}}}\rangle$ alone is not
sufficient as an input to generate the desired nonlinearity, and an input
state as a tensor product of copies of $|\bar{{\bm{x}}}\rangle$ is required to
generate the high-order terms in $f({\bm{x}})$. Specifically, to approximate
the polynomial $f$, the input state $|\psi_{{\bm{x}}}\rangle$ of CCQ is chosen
as:

$\displaystyle|\psi_{{\bm{x}}}\rangle$
$\displaystyle=|\bar{{\bm{x}}}\rangle^{\otimes
n_{\text{copy}}}=|\bar{{\bm{x}}}\rangle^{\otimes\frac{M}{2}}$ (18)
$\displaystyle=\frac{1}{\gamma^{\frac{M}{2}}}[x_{1}^{\frac{M}{2}},x_{1}^{\frac{M}{2}-1}x_{2},\cdots,\tilde{x}^{\frac{M}{2}},0,\cdots,0]^{T}\in\mathbb{C}_{2}^{\otimes
n_{\text{tot}}},$ (19)

where $n_{\text{tot}}\equiv
n_{\text{copy}}n_{\text{data}}=\frac{1}{2}M\lceil\log d\rceil$. Here we have
chosen the number of copies of $|\bar{{\bm{x}}}\rangle$ in
$|\psi_{{\bm{x}}}\rangle$ to be $n_{\text{copy}}=\frac{M}{2}$.

Denoting the variational quantum circuit in CCQ by $U({\bm{\theta}})$, the
output state $|\psi_{{\bm{\theta}},{\bm{x}}}\rangle$ becomes:

$|\psi_{{\bm{\theta}},{\bm{x}}}\rangle=U({\bm{\theta}})|\psi_{{\bm{x}}}\rangle=\frac{1}{\gamma^{\frac{M}{2}}}\left[\begin{matrix}u_{1,1}({\bm{\theta}})&u_{1,2}({\bm{\theta}})&u_{1,3}({\bm{\theta}})&\cdots&u_{1,2^{n_{\text{tot}}}}({\bm{\theta}})\\\
u_{2,1}({\bm{\theta}})&u_{2,2}({\bm{\theta}})&u_{2,3}({\bm{\theta}})&\cdots&u_{2,2^{n_{\text{tot}}}}({\bm{\theta}})\\\
u_{3,1}({\bm{\theta}})&u_{3,2}({\bm{\theta}})&u_{3,3}({\bm{\theta}})&\cdots&u_{3,2^{n_{\text{tot}}}}({\bm{\theta}})\\\
\vdots&\vdots&\vdots&\ddots&\vdots\\\
u_{2^{n_{\text{tot}}},1}({\bm{\theta}})&u_{2^{n_{\text{tot}}},2}({\bm{\theta}})&u_{2^{n_{\text{tot}}},3}({\bm{\theta}})&\cdots&u_{2^{n_{\text{tot}}},2^{n_{\text{tot}}}}({\bm{\theta}})\end{matrix}\right]\left[\begin{matrix}x_{1}^{\frac{M}{2}}\\\
x_{1}^{\frac{M}{2}-1}x_{2}\\\ x_{1}^{\frac{M}{2}-1}x_{3}\\\ \vdots\\\
0\end{matrix}\right]=\left[\begin{matrix}p_{1}({\bm{\theta}},{\bm{x}})\\\
p_{2}({\bm{\theta}},{\bm{x}})\\\ p_{3}({\bm{\theta}},{\bm{x}})\\\ \vdots\\\
p_{2^{n_{\text{tot}}}}({\bm{\theta}},{\bm{x}})\end{matrix}\right]$ (20)

where each amplitude $p_{i}({\bm{\theta}},{\bm{x}})$ of
$|\psi_{{\bm{\theta}},{\bm{x}}}\rangle$ is an $\frac{M}{2}$-order polynomial
of ${\bm{x}}$. Hence, for a chosen observable $B$, the measurement outcome
$\langle
B\rangle_{{\bm{\theta}},{\bm{x}}}\equiv\langle\psi_{{\bm{\theta}},{\bm{x}}}|B|\psi_{{\bm{\theta}},{\bm{x}}}\rangle$
is an $M$-order polynomial of ${\bm{x}}$. Then the goal of CCQ is to choose an
appropriate $B$ and to find the best value ${\bm{\theta}}={\bm{\theta}}^{*}$
through quantum-classical hybrid optimization such that ${\bm{x}}\to\langle
B\rangle_{{\bm{\theta}}^{*},{\bm{x}}}$ is the optimal function to approximate
${\bm{x}}\to f({\bm{x}})$. The quantum circuit for CCQ is presented in Fig. 6.

Figure 6: The quantum circuit for CCQ to approximate the $M$-order polynomial
function ${\bm{x}}\to f({\bm{x}})$.

Next, we consider the QCL model, where the data point
$\bm{x}=[x_{1},x_{2},\cdots,x_{d}]^{T}$ is encoded into a $d$-qubit data-
encoding pure-state density operator:

$\displaystyle\rho({\bm{x}})$
$\displaystyle=|\psi_{{{\bm{x}}}}\rangle\langle\psi_{{{\bm{x}}}}|$
$\displaystyle=\otimes_{i=1}^{d}\big{(}R_{i}^{Y}(\arcsin
x_{i})|0\rangle_{i}\langle 0|_{i}R_{i}^{Y{\dagger}}(\arcsin x_{i})\big{)}$
$\displaystyle=\frac{1}{2^{d}}\otimes_{i=1}^{d}\big{(}I+x_{i}X+\sqrt{1-x_{i}^{2}}Z\big{)}$
(21)

where $R_{i}^{Y}({\bm{\theta}})$ represents a Pauli-$Y$ rotation on the $i$-th
qubit. Analogous to the CCQ case, $\rho({\bm{x}})$ along is not sufficient as
an input to generate the desired nonlinearity. In order to generate an
$M$-order polynomial, $\rho({\bm{x}})$ is duplicated into $M$ identical copies
in a tensor product. Specifically, the input state of QCL is chosen as:

$\displaystyle\rho_{\text{in}}({\bm{x}})$
$\displaystyle=\otimes_{j=1}^{M}\rho({\bm{x}})$
$\displaystyle=\frac{1}{2^{Md}}\otimes_{j=1}^{M}\big{(}\otimes_{i=1}^{d}[I+x_{i}X+\sqrt{1-x_{i}^{2}}Z\big{)}.$
(22)

Applying a parameterized quantum circuit $U({\bm{\theta}})$ to
$\rho_{\text{in}}({\bm{x}})$, followed by a measurement of a chosen observable
$B$, we obtain the measurement outcome as the output of QCL:

$\langle
B\rangle_{{\bm{\theta}},{\bm{x}}}\equiv\operatorname{Tr}\big{(}U({\bm{\theta}})\rho_{\text{in}}({\bm{x}})U^{\dagger}({\bm{\theta}})B\big{)}$
(23)

Define $\\{A_{l}\\}\equiv\\{I,X,Z\\}^{\otimes Md}$ as a set of generalized
Pauli operators. Due to Eq. (22), $\rho_{\text{in}}({\bm{x}})$ can be expanded
in terms of $\\{A_{l}\\}$:

$\rho_{\text{in}}({\bm{x}})=\frac{1}{2^{Md}}\sum_{l=1}^{3^{Md}}p_{l}(x_{1},\cdots,x_{d},\sqrt{1-x_{1}^{2}},\cdots,\sqrt{1-x_{d}^{2}})A_{l},$
(24)

where
$(x_{1},\cdots,x_{d},x_{1}^{\prime},x_{2}^{\prime},\cdots,x_{d}^{\prime})\in\mathbb{R}^{2d}\to
p_{l}(x_{1},x_{2},\cdots,x_{d},x_{1}^{\prime},x_{2}^{\prime},\cdots,x_{d}^{\prime})$,
$l=1,2,\cdots,3^{Md}$, are polynomials with respect to all $2d$ variables.
Assuming $B$ is unitarily diagonalized as

$B=W^{\dagger}\left[\begin{matrix}\lambda_{1}&\cdots&0\\\
\vdots&\ddots&\vdots\\\
0&\cdots&\lambda_{2^{n_{\text{tot}}}}\end{matrix}\right]W,$ (25)

we substitute Eq. (24) and Eq. (25) into Eq. (23) and obtain:

$\displaystyle\langle B\rangle_{{\bm{\theta}},{\bm{x}}}$
$\displaystyle=\frac{1}{2^{Md}}\sum_{l=1}^{3^{Md}}b_{l}({\bm{\theta}},\lambda)p_{l}$
$\displaystyle b_{l}({\bm{\theta}},\lambda)$
$\displaystyle\equiv\sum_{j=1}^{2^{Md}}\lambda_{j}\text{Tr}\big{(}A_{l}(WU({\bm{\theta}}))^{\dagger}|e_{j}\rangle\langle
e_{j}|WU({\bm{\theta}})\big{)},$

where $\\{|e_{j}\rangle\\}$ is the set of computational basis. Thus, the task
of QCL is to choose an appropriate observable $B$ and to find the best value
${\bm{\theta}}={\bm{\theta}}^{*}$ through quantum-classical hybrid
optimization such that ${\bm{x}}\to\langle
B\rangle_{{\bm{\theta}}^{*},{\bm{x}}}$ will well approximate ${\bm{x}}\to
f({\bm{x}})$. The quantum circuit for QCL is given in Fig. 7.

Figure 7: The quantum circuit for QCL to approximate the $M$-order polynomial
function ${\bm{x}}\to f({\bm{x}})$.

In summary, to prepare the desired initial state to solve the same polynomial
approximation problem, it requires $\frac{1}{2}M\lceil\log d\rceil$ qubits for
CCQ, and $Md$ qubits for QCL. In comparison, it only requires $\lceil\log
d\rceil$ qubits for DQNN to prepare the input state as
$|\bar{{\bm{x}}}\rangle$ in Eq. (17). This is a significant reduction for
large $M$ and large $d$. Specifically, for DQNN, after we prepare the input
state $|\bar{{\bm{x}}}\rangle$, we sequentially apply a series of variational
quantum circuits, $\\{U^{(j)}({\bm{\theta}}^{(j)})\\}_{j=1}^{n_{\text{cir}}}$,
and quantum measurement to derive the measurement outcome $\langle
B_{i}\rangle_{{\bm{\theta}}^{(j)},{\bm{x}}}=\langle\bar{{\bm{x}}}|U^{(j)\dagger}({\bm{\theta}}^{(j)})B_{i}U^{(j)}({\bm{\theta}}^{(j)})|\bar{{\bm{x}}}\rangle$
for a set of chosen observables $\\{B_{i}\\}_{i=1}^{n_{\text{obs}}}$. Next, we
apply a parameterized classical sigmoid function $\sigma$ to each $\langle
B_{i}\rangle_{{\bm{\theta}}^{(j)},\bar{{\bm{x}}}}$ to construct the final
output $q_{{\bm{\theta}},{\bm{\alpha}},{\bm{a}},{\bm{c}}}(\bar{{\bm{x}}})$.
Finally, we implement the hybrid optimization to find the optimal values of
$({\bm{\theta}},{\bm{\alpha}},{\bm{a}},{\bm{c}})$ to approximate the target
polynomial, where ${\bm{\alpha}}$, ${\bm{a}}$, ${\bm{c}}$ are parameters used
to construct the final output
$q_{{\bm{\theta}},{\bm{\alpha}},{\bm{a}},{\bm{c}}}$, with details in the
following.

##  Appendix D Solving regression problems using DQNN, CCQ and QCL

In the following, we aim to solve a regression task of approximating a highly
nonlinear function:

$\displaystyle f:(x_{1},x_{2})\in[-0.8,0.8]^{2}\to$
$\displaystyle(0.716-1.0125x_{1}^{2}+x_{1}^{4})$
$\displaystyle\times(0.716-1.0125x_{2}^{2}+x_{2}^{4})\in\mathbb{R}$ (26)

using DQNN, QCL and CCQ. The data set
$D\equiv\\{{\bm{x}}^{(i)},y^{(i)}\\}=\\{(x_{1}^{(i)},x_{2}^{(i)},y^{(i)})\\}_{i=1}^{400}$
is of size $400$, with $(x_{1}^{(i)},x_{2}^{(i)})$ randomly chosen from the
square $[-0.8,0.8]^{2}$, and each label $y^{(i)}$ is calculated according to
$y^{(i)}=f(x_{1}^{(i)},x_{2}^{(i)})$. The goal of a QNN is to generate a
nonlinear function ${\bm{x}}\to q_{\bm{\theta}}({\bm{x}})$ to approximate $f$
by optimizing over the parameters ${\bm{\theta}}$ of the QNN.

(a) DQNN1 based on one single-layer quantum circuit and $4$ observables.

(b) DQNN4 based on $4$ single-layer quantum circuits and one observable.

Figure 8: The implementation of DQNN in the regression task of Eq. (26).

First, we use the DQNN model to solve this problem. We choose two cases for
DQNN in the task. Case $1$ (DQNN1) is based on a $2$-qubit single-layer
variational quantum circuit, $U^{(1)}(\bm{\theta}^{(1)})$, and four
measurement observables, $\\{B_{i}\\}_{i=1}^{4}$, randomly chosen the
generalized Pauli basis on two qubits, as shown in Fig. 8(a). Case $2$ (DQNN4)
is based on four $2$-qubit single-layer variational quantum circuits,
$\\{U^{(j)}(\bm{\theta}^{(j)})\\}_{j=1}^{4}$, and one measurement observable
$B_{1}=Z^{(1)}\otimes Z^{(2)}$, as shown in Fig. 8(b). Through state
reparation, each data point ${\bm{x}}$ is encoded into the input state:

$|\bar{{\bm{x}}}\rangle=\frac{1}{\gamma}[x_{1},x_{2},\tilde{x},0]^{T}$ (27)

where $\tilde{x}=\frac{|\boldsymbol{x}|}{1+|\boldsymbol{x}|}$ and
$\gamma=(|\boldsymbol{x}|^{2}+\tilde{x}^{2})^{\frac{1}{2}}$. Then we
sequentially apply the variational circuit $U^{(j)}({\bm{\theta}}^{(j)})$ and
quantum measurement to obtain the measurement outcome $\langle
B_{i}\rangle_{{\bm{\theta}}^{(j)},\bar{{\bm{x}}}}$:

$\displaystyle\langle
B_{i}\rangle_{{\bm{\theta}}^{(j)},\bar{\bm{x}}}\equiv\mathrm{Tr}(U^{(j)}({\bm{\theta}}^{(j)})|\bar{\bm{x}}\rangle\langle\bar{\bm{x}}|{U^{(j)}({\bm{\theta}}^{(j)})}^{\dagger}B_{i}).$
(28)

Denoting the sigmoid function as $\sigma(x)$, with

$\sigma(x)\equiv\frac{1}{1+e^{-x}},$ (29)

for DQNN1, we obtain

$\displaystyle
u(\bar{\bm{x}})\equiv\sum_{i=1}^{4}\alpha_{i}^{(1)}\sigma(a^{(1)}_{i}(\langle
B_{i}\rangle_{{\bm{\theta}}^{(1)},\bm{\bar{x}}}-c^{(1)}_{i})),\,i=1,2,3,4.$
(30)

and finally derive

$q_{{\bm{\theta}},{\bm{a}},{\bm{c}},{\bm{\alpha}}}(\bar{\bm{x}})\equiv\sigma(a_{5}(u(\bar{\bm{x}})-c_{5}))$
(31)

where the parameter ${\bm{\theta}}$ comes from the quantum circuit
$U({\bm{\theta}})$, while the parameters $({\bm{a}},{\bm{c}},{\bm{\alpha}})$
come from the classical processing after $U({\bm{\theta}})$. For DQNN4, we
obtain

$\displaystyle
u(\bar{\bm{x}})\equiv\sum_{j=1}^{4}\alpha_{1}^{(j)}\sigma(a^{(j)}_{1}(\langle
B_{1}\rangle_{{\bm{\theta}}^{(j)},\bm{\bar{x}}}-c^{(j)}_{1})),\,j=1,2,3,4$
(32)

and derive

$q_{{\bm{\theta}},{\bm{a}},{\bm{c}},{\bm{\alpha}}}(\bar{\bm{x}})\equiv\sigma(a_{5}(u(\bar{\bm{x}})-c_{5})).$
(33)

Finally, through optimization, we can find the optimal values of parameters
$({\bm{\theta}},{\bm{a}},{\bm{c}},{\bm{\alpha}})$ that correspond to the
optimal approximation to the target function $f$.

Second, we consider the CCQ model. For any data point $(\bm{x},y)\in D$,
$\bm{x}$ can be encoded into a 2-qubit state
$|\bar{{\bm{x}}}\rangle=\frac{1}{\gamma}[x_{1},x_{2},\tilde{x},0]^{T}$ as in
Eq. (17). For the input state $|\psi_{{\bm{x}}}\rangle$ in Eq. (19), the
number of copies $n_{\text{copy}}$ can be chosen to be $n_{\text{copy}}\geq
1$. Here we consider two cases, $n_{\text{copy}}=1$ and $n_{\text{copy}}=2$.
For $n_{\text{copy}}=1$, we design $U({\bm{\theta}})$ in CCQ to be a $2$-qubit
$4$-layer variational quantum circuit; for $n_{\text{copy}}=2$,
$U({\bm{\theta}})$ in CCQ is chosen to be a $4$-qubit $4$-layer circuit. The
circuit depth of $U({\bm{\theta}})$ is $13$ in the former case, and $21$ in
the latter case. Correspondingly, the input state $|\psi_{{\bm{x}}}\rangle$ is
either a 2-qubit state or a 4-qubits state, as shown in Fig. 9. After input
state preparation in each case, we apply $U({\bm{\theta}})$ and a measurement
to $|\psi_{{\bm{x}}}\rangle$, and derive the measurement outcome

$\langle
B\rangle_{{\bm{\theta}},{\bm{x}}}\equiv\mathrm{Tr}(U({\bm{\theta}})|\psi_{{\bm{x}}}\rangle\langle\psi_{{\bm{x}}}|{U({\bm{\theta}})}^{\dagger}B)$
(34)

where the observable is chosen as $B\equiv Z^{(1)}$, the local Pauli-Z gate on
the first qubit. Then we derive the following output

$q_{{\bm{\theta}}}(\bm{x})=\frac{\langle
B\rangle_{{\bm{\theta}},{\bm{x}}}+1}{2}.$ (35)

Through optimization, we can find the optimal value ${\bm{\theta}}$ that
correspond to the best approximated $q_{{\bm{\theta}}}$ to the target function
$f$.

Figure 9: The circuit of CCQ in the regression task of Eq. (26), with
$U({\bm{\theta}})$ chosen as a $2$-qubit $4$-layer circuit (case a), and as a
$4$-qubit $4$-layer circuit (case b)

Figure 10: The circuit of QCL in the regression task of Eq. (26), with
$U({\bm{\theta}})$ chosen as a $2$-qubit $4$-layer circuit (case a), and as a
$4$-qubit $5$-layer circuit (case b).

Finally, we apply QCL to solve the same problem. We also consider two cases
where the classical data point ${\bm{x}}$ is encoded into a 2-qubit state
($n_{\text{copy}}=1$) and a 4-qubit state ($n_{\text{copy}}=2$) respectively,
as in Eq. (21):

$\rho({\bm{x}})=\frac{1}{4}\otimes_{j=1}^{n_{\text{copy}}}\otimes_{i=1}^{2}\big{(}\text{I}+x_{i}X+\sqrt{1-x_{i}^{2}}Z\big{)}.$
(36)

The entire process from the input state to the final output of QCL is
illustrated in Fig. 10. Notice that the variational quantum circuit
$U({\bm{\theta}})$ is slightly different from the original suggestion in [32],
as we find that our modified circuit has a better performance. After input
state preparation in each case, we apply $U({\bm{\theta}})$ and a measurement
to $\rho({\bm{x}})$, and derive the measurement outcome

$\langle
B\rangle_{{\bm{\theta}},{\bm{x}}}\equiv\mathrm{Tr}(U({\bm{\theta}})\rho(\bm{x}){U({\bm{\theta}})}^{\dagger}B).$
(37)

where the observable is chosen as $B\equiv Z^{(1)}$. Then we obtain the final
output of QCL:

$q_{{\bm{\theta}},a}(\bm{x})=a\langle B\rangle_{{\bm{\theta}},{\bm{x}}}$ (38)

where $a$ is a trainable parameter. Then, through optimization, we find the
optimal values of the parameters $({\bm{\theta}},a)$ that correspond to the
best approximated $q_{{\bm{\theta}},a}$ to the target function $f$.

## References

  * Zheng  _et al._ [2017] H. Zheng, J. Fu, T. Mei, and J. Luo, Learning multi-attention convolutional neural network for fine-grained image recognition, in  _IEEE International Conference on Computer Vision (ICCV)_ , edited by K. Ikeuchi, G. Medioni, and M. Pelillo (Venice, Italy, 2017) p. 5219.
  * Shi  _et al._ [2017] B. Shi, X. Bai, and C. Yao, An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition, IEEE Trans. Pattern Anal. Mach. Intell. 39, 2298 (2017).
  * Albawi  _et al._ [2017] S. Albawi, T. A. Mohammed, and S. Al-Zawi, Understanding of a convolutional neural network, in  _2017 International Conference on Engineering and Technology (ICET)_ , edited by O. Bayat, S. A. Aljawarneh, and H. F. Carlak (Akdeniz University, Antalya, Turkey, 2017) p. 1.
  * Goldberg [2016] Y. Goldberg, A primer on neural network models for natural language processing, J. Artif. Intell. Res. 57, 345 (2016).
  * Goldberg [2017] Y. Goldberg,  _Neural Network Methods for Natural Language Processing_ , Vol. 37 (Morgan & Claypool, 2017).
  * Collobert and Weston [2008] R. Collobert and J. Weston, A unified architecture for natural language processing: Deep neural networks with multitask learning, in  _Proceedings of the 25th International Conference on Machine Learning_ , edited by W. Cohen, A. McCallum, and S. Roweis (Helsinki, Finland, 2008) p. 160.
  * Nugraha  _et al._ [2017] B. T. Nugraha, S. F. Su, and Fahmizal, Towards self-driving car using convolutional neural network and road lane detector, in  _International conference on automation, cognitive science, optics, micro electro-mechanical system, and information technology_ , edited by T. Arjon, H. Taufik, and K. D. Esti (Jakarta, Indonesia, 2017) p. 65.
  * Do  _et al._ [2018] T. D. Do, M. T. Duong, Q. V. Dang, and M. H. Le, Real-time self-driving car navigation using deep neural network, in  _International Conference on Green Technology and Sustainable Development_ , edited by D. T. Trung (Ho Chi Minh City, Vietnam, 2018) p. 7.
  * Bojarski  _et al._ [2017] M. Bojarski, P. Yeres, A. Choromanska, K. Choromanski, B. Firner, L. Jackel, and U. Muller, Explaining how a deep neural network trained with end-to-end learning steers a car (2017), arXiv e-prints, ArXiv:1704.07911.
  * Biamonte  _et al._ [2017] J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, and S. Lloyd, Quantum machine learning, Nature 549, 195 (2017).
  * Carleo and Troyer [2017] G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602 (2017).
  * Deng  _et al._ [2017] D. L. Deng, X. P. Li, and S. Das Sarma, Quantum entanglement in neural network states, Phys. Rev. X 7, 021021 (2017).
  * Garrido Torres  _et al._ [2021] J. A. Garrido Torres, V. Gharakhanyan, N. Artrith, T. H. Eegholm, and A. Urban, Augmenting zero-kelvin quantum mechanics with machine learning for the prediction of chemical reactions at high temperatures, Nat. Commun. 12, 1 (2021).
  * Schütt  _et al._ [2019] K. Schütt, M. Gastegger, A. Tkatchenko, K.-R. Müller, and R. J. Maurer, Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions, Nat. Commun. 10, 1 (2019).
  * Gao and Duan [2017] X. Gao and L. M. Duan, Efficient representation of quantum many-body states with deep neural networks, Nat. Commun. 8, 1 (2017).
  * Paruzzo  _et al._ [2018] F. M. Paruzzo, A. Hofstetter, F. Musil, S. De, M. Ceriotti, and L. Emsley, Chemical shifts in molecular solids by machine learning, Nat. Commun. 9, 1 (2018).
  * Huang  _et al._ [2022] H.-Y. Huang, M. Broughton, J. Cotler, S. Chen, J. Li, M. Mohseni, H. Neven, R. Babbush, R. Kueng, J. Preskill, and J. R. McClean, Quantum advantage in learning from experiments, Science 376, 1182 (2022).
  * Wiebe  _et al._ [2012] N. Wiebe, D. Braun, and S. Lloyd, Quantum algorithm for data fitting, Phys. Rev. Lett. 109, 050505 (2012).
  * Rebentrost  _et al._ [2014] P. Rebentrost, M. Mohseni, and S. Lloyd, Quantum support vector machine for big data classification, Phys. Rev. Lett. 113, 130503 (2014).
  * Wiebe  _et al._ [2015] N. Wiebe, A. Kapoor, and K. M. Svore, Quantum algorithms for nearest-neighbor methods for supervised and unsupervised learning, Quantum Info. Comput. 15, 316 (2015).
  * Lloyd  _et al._ [2014] S. Lloyd, M. Mohseni, and P. Rebentrost, Quantum principal component analysis, Nat. Phys. 10, 631 (2014).
  * Schuld  _et al._ [2016] M. Schuld, I. Sinayskiy, and F. Petruccione, Prediction by linear regression on a quantum computer, Phys. Rev. A 94, 022342 (2016).
  * Paparo and Martin-Delgado [2012] G. D. Paparo and M. Martin-Delgado, Google in a quantum network, Sci. Rep. 2, 1 (2012).
  * Dong  _et al._ [2008] D. Y. Dong, C. L. Chen, H. X. Li, and T.-J. Tarn, Quantum reinforcement learning, IEEE Transactions on Systems, Man, and Cybernetics, Part B 38, 1207 (2008).
  * Saggio  _et al._ [2021] V. Saggio, B. E. Asenbeck, A. Hamann, T. Strömberg, P. Schiansky, V. Dunjko, N. Friis, N. C. Harris, M. Hochberg, D. Englund, S. Wölk, H. Briegel, and P. Walther, Experimental quantum speed-up in reinforcement learning agents, Nature 591, 229 (2021).
  * Paparo  _et al._ [2014] G. D. Paparo, V. Dunjko, A. Makmal, M. A. Martin-Delgado, and H. J. Briegel, Quantum speedup for active learning agents, Phys. Rev. X 4, 031002 (2014).
  * Liu and Rebentrost [2018] N. Liu and P. Rebentrost, Quantum machine learning for quantum anomaly detection, Phys. Rev. A 97, 042315 (2018).
  * Amin  _et al._ [2018] M. H. Amin, E. Andriyash, J. Rolfe, B. Kulchytskyy, and R. Melko, Quantum boltzmann machine, Phys. Rev. X 8, 021050 (2018).
  * Aaronson [2015] S. Aaronson, Read the fine print, Nat. Phys. 11, 291 (2015).
  * Huang  _et al._ [2021] H. Y. Huang, M. Broughton, M. Mohseni, R. Babbush, S. Boixo, H. Neven, and J. R. McClean, Power of data in quantum machine learning, Nat. Commun. 12 (2021).
  * Schuld  _et al._ [2014] M. Schuld, I. Sinayskiy, and F. Petruccione, The quest for a quantum neural network, Quantum Inf. Process 13, 2567 (2014).
  * Mitarai  _et al._ [2018] K. Mitarai, M. Negoro, M. Kitagawa, and K. Fujii, Quantum circuit learning, Phys. Lett. A 98, 032309 (2018).
  * Schuld  _et al._ [2020] M. Schuld, A. Bocharov, K. M. Svore, and N. Wiebe, Circuit-centric quantum classifiers, Phys. Lett. A 101, 032308 (2020).
  * Hornik  _et al._ [1989] K. Hornik, M. Stinchcombe, and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2, 359 (1989).
  * Leshno  _et al._ [1993] M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken, Multilayer feedforward networks with a nonpolynomial activation function can approximate any function, Neural Networks 6, 861 (1993).
  * Cao  _et al._ [2017] Y. D. Cao, G. G. Guerreschi, and A. Aspuru-Guzik, Quantum neuron: an elementary building block for machine learning on quantum computers (2017), arXiv e-prints, ArXiv:1711.11240.
  * Yan  _et al._ [2020] S. Yan, H. Qi, and W. Cui, Nonlinear quantum neuron: A fundamental building block for quantum neural networks, Phys. Lett. A 102, 052421 (2020).
  * Tacchino  _et al._ [2019] F. Tacchino, C. Macchiavello, D. Gerace, and D. Bajoni, An artificial neuron implemented on an actual quantum processor, npj Quantum Inf. 5, 1 (2019).
  * Kristensen  _et al._ [2021] L. B. Kristensen, M. Degroote, P. Wittek, A. Aspuru-Guzik, and N. T. Zinner, An artificial spiking quantum neuron, npj Quantum Inf. 7, 1 (2021).
  * Torrontegui and García-Ripoll [2019] E. Torrontegui and J. J. García-Ripoll, Unitary quantum perceptron as efficient universal approximator, Europhys. Lett. 125, 30004 (2019).
  * Beer  _et al._ [2020] K. Beer, D. Bondarenko, T. Farrelly, T. J. Osborne, R. Salzmann, D. Scheiermann, and R. Wolf, Training deep quantum neural networks, Nat. Commun. 11, 1 (2020).
  * Schuld  _et al._ [2015] M. Schuld, I. Sinayskiy, and F. Petruccione, Simulating a perceptron on a quantum computer, Phys. Lett. A 379, 660 (2015).
  * Wan  _et al._ [2017] K. H. Wan, O. Dahlsten, H. Kristjánsson, R. Gardner, and M. Kim, Quantum generalisation of feedforward neural networks, npj Quantum Inf. 3, 1 (2017).
  * Runge [1901] C. Runge, über empirische funktionen und die interpolation zwischen äquidistanten ordinaten, Zeitschrift für Mathematik und Physik 46, 224 (1901).
  * Brenner and Scott [2007] S. C. Brenner and L. R. Scott,  _The Mathematical Theory of Finite Element Methods_ (Springer, 2007).
  * Goodfellow  _et al._ [2016] I. Goodfellow, Y. Bengio, and A. Courville,  _Deep Learning_ (MIT Press, 2016).
  * Plesch and Brukner [2011] M. Plesch and Č. Brukner, Quantum-state preparation with universal gate decompositions, Phys. Rev. A 83, 032302 (2011).
  * Bottou [2010] L. Bottou, Large-scale machine learning with stochastic gradient descent, Comput. Stat. , 177 (2010).
  * Kingma and Ba [2015] D. Kingma and J. Ba, Adam: A method for stochastic optimization, in  _International Conference on Learning Representations_ , edited by Y. Bengio and Y. Lecun (San Diego, CA, 2015) p. 13.
  * Buckley and LeNir [1985] A. Buckley and A. LeNir, Algorithm 630: Bbvscg–a variable-storage algorithm for function minimization, ACM Trans. Math. Softw. 11, 103 (1985).
  * Bishop [2006] C. M. Bishop,  _Pattern Recognition and Machine Learning_ (Springer Press, Berlin, 2006).
  * LeCun  _et al._ [2010] Y. LeCun, C. Cortes, and C. Burges, Mnist handwritten digit database (2010), figshare http://yann.lecun.com/exdb/mnist (2010).
  * Dua and Graff [2017] D. Dua and C. Graff, UCI machine learning repository (2017), http://archive.ics.uci.edu/ml (2017).
  * Haldane [1983] F. D. M. Haldane, Nonlinear field theory of large-spin heisenberg antiferromagnets: semiclassically quantized solitons of the one-dimensional easy-axis néel state, Phys. Rev. Lett. 50, 1153 (1983).
  * Cong  _et al._ [2019] I. Cong, S. Choi, and M. D. Lukin, Quantum convolutional neural networks, Nat. Phys. 15, 1273 (2019).
  * Hahn [1927] H. Hahn, Über lineare gleichungssysteme in linearen räumen., J. für die Reine und Angew. Math. 1927, 214 (1927).
  * Fréchet [1907] M. Fréchet, Sur les ensembles de fonctions et les opérations linéaires, CR Acad. Sci. Paris 144, 1414 (1907).


# Development of wavelength-shifting PEN foils for next generation experiments

M. Kuźniak11footnotetext: Corresponding author. S. Pawłowski  A. Abramowicz
A. F. V. Cortez  M. Kumosiński  T. Łęcki  G. Nieradka

###### Abstract

Polyethylene naphthalate (PEN) foils have been demonstrated as a wavelength
shifter suitable for operation in liquid argon. At the same time, wavelength
shifting efficiency of technical grades of PEN, commercially available on the
market, is lower than that of tetraphenyl butadiene (TPB). This paper reports
on an R&D program focused on exploring the intrinsic limitations of PEN and
optimizing it for the highest achievable wavelength shifting efficiency.

##  1 Introduction

Liquid argon detectors for dark matter searches, long-baseline neutrino
experiments and neutrinoless double-beta decay searches, among other
applications, rely on detection of vacuum ultraviolet (VUV) argon
scintillation light at 128 nm. Wavelength shifters are commonly used to
convert it to visible blue light, where the photon detection efficiency of
available photosensors is significantly higher [1].

Polyethylene naphthalate (PEN), available as large format polymeric film, has
been proposed [2] as a scalable and stable alternative to traditionally
employed tetraphenyl butadiene (TPB). TPB coatings are deposited via vacuum
evaporation and hence challenging to scale up to 100-10000 m2 of surface area,
anticipated in the next generation detectors.

Commercially available, technical, PEN foil grades, however, which are neither
made nor optimized for this application, provide the wavelength shifting
efficiency (WLSE) of approx. 34-70% of TPB [3, 4, 5]. Maximizing WLSE is
essential for dark matter detectors, as it lowers their energy threshold. Due
to the steeply falling WIMP-induced nuclear recoil spectrum this translates
into a non-linear increase in sensitivity. At the same time, the literature
suggests that better PEN performance can be achieved [2]. Potential factors
negatively affecting the WLSE of commercial PEN films include:

  * •

over-coatings or additives (e.g. fillers, colorants, plasticizers,
stabilizers, catalysts), which improve the mechanical properties of the foil
in industrial applications, while degrading its optical yield,

  * •

unknown history of samples (storage, integrated UV exposure),

  * •

crystalline structure and orientation (e.g. amorphous, bi-axially stretched),

  * •

thermal history (sequence and details of cooling/warming cycles experienced by
the sample since the last melted state, e.g. during production, processing and
storage).

Recent progress on R&D aimed at identifying the key factors driving the WLSE
of PEN and, informed by that, synthesising a grade optimized for application
in liquid argon detectors, is reported here.

##  2 Analysis of commercial grades

One of the first steps was an attempt to identify the dominant factors
affecting WLSE by examining grades which were the best (Teonex Q51) and worst
(Teonex Q83) performing in terms of WLSE, based on the earlier measurements.
Differential scanning calorimetry (DSC) is a thermoanalytical technique in
which the difference in the amount of heat required to increase the
temperature of a sample and reference is measured as a function of
temperature. This allows, in particular, to map temperatures of transitions
between amorphous and crystalline phases. Both samples went through a cycle in
the $-100$-300∘C range, see Fig. LABEL:fig:dsc: 1st heating, cooldown, and 2nd
heating. The first melting erases the thermal history of the sample, allowing
to compare the intrinsic differences between materials in the subsequent
phases of the cycle.

Sample | 1st heating | Cooldown | 2nd heating
---|---|---|---
Teonex Q51 |  $T_{g}$=51, $T_{m}$=218, 250-266 |  $T_{c}$=191, $T_{g}$=123, $-48$ |  $T_{g}$=125, $T_{c}$=201, $T_{m}$=265
Teonex Q83 |  $T_{m}$=249-266 |  $T_{g}$=124, $-56$ |  $T_{g}$=126, $T_{c}$=214, $T_{m}$=265
Table 1: Summary of temperatures (in ∘C) at which DSC revealed phase
transitions in the Teonex samples: cold crystallization ($T_{c}$), glass
transition ($T_{g}$) and melting ($T_{m}$).

Phase transitions observed with DSC are listed in Tab. 1. Nearly identical
melting temperatures during both heating phases, despite somewhat broader
maximum in Teonex Q83, indicate that the same polymer and of similar molecular
mass is present in both samples. Significant differences in glass transition
(during cooldown) and cold crystallization temperatures (cooldown and 2nd
heating) typically indicate either differences in the level of cross-linking
(consequence of different molecular masses and purity grades of the base
chemicals used for production, including different co-monomers) or difference
in the type, or the very presence of, additives.

A nuclear magnetic resonance (NMR) scan was initially intended to cross-check
the DSC results and provide new insights on the nature of impurities or
additives. At the end it was not performed due to difficulties in sample
preparation required for NMR, i.e. dissolution or fine pulverization of PEN.

##  3 UV aging

Effects of UV aging on fluorescence intensity of PEN have already been
reported in the literature [6]. Aging test of the Teonex Q51 grade was
performed following the PN-EN 60794-1-22 standard, in which the samples were
exposed to a Feutron SOL500 lamp, with 650 W/m2 in the 300 to 780 nm spectral
range and at elevated temperature inside of a climatic chamber.

Figure 4: Reference PEN (Teonex Q51) sample, not exposed to UV and kept in the
dark (top), next to the sample aged for 1 day (bottom left), and next to a
sample after 14 days of exposure to the solar simulator (middle). (Right)
Fluorescence emission intensity as a function of excitation wavelength for
samples after the respective exposures.

Significant yellowing and a drop in emission intensity is observed already
after 1 day of exposure in the chamber (equivalent to about 2 months of
exposure to sunlight), see Fig. 4. The fluorescence intensity has been
characterised using a Shimadzu UV-3600 spectrophotometer equipped with a 15 cm
diameter integrating sphere accessory (LISR-3100), as in Refs. [2, 3].

##  4 Custom synthesis

Given the indication from DSC on differences in cross-linking levels or
additives between grades, an attempt has been made to synthesize an additive-
free batch of PEN, following the process outlined in the original patent [7]
and using a small glass reactor. Due to limited amount of the product, 100
$\upmu$m-thick foil samples for fluorescence intensity characterization were
made by pressing the material at 275∘C (typically at least several kg of
material are required by foil drawing/stretching machines).

Initial batches suffered from thermal degradation and yellowing of the product
caused by challenges in maintaining uniformly high and hot-spot free
temperature distribution throughout the mixture, as well as impurities in the
base chemicals. Additional purification step and better temperature control
resulted in a significantly improved white product (batch #2), see Fig. 5.

Figure 5: Custom synthesized additive-free PEN. (Top Left) Batch #1, obtained
early in the development process, with signs of yellowing of the product.
(Bottom Left) Optimized batch #2 after fine-tuning the parameters of the
process and purification of the base chemicals. (Right) Fluorescence intensity
of custom made PEN (batches #1 and #2) compared to a commercial grade Teonex
Q51 and a TPB reference, as a function of excitation wavelength, and at room
temperature.

The fluorescence intensity of batch #2, in the raw form of chunks (as
extracted from the reactor), matches that of commercial Teonex Q51 foils.
Further intensity increase due to reduced self-absorption is expected after
producing a thin foil out of batch #2 material, potentially marginally
surpassing the commercial grade PEN, to be confirmed as the next step.

##  5 Summary and outlook

In the completed characterization of commercial Teonex Q51 and Q83 grades, the
DSC results indicated differences in the level of cross-linking and/or
additives, while UV aging tests of Teonex Q51 showed yellowing and a
significant drop in intensity after climatic chamber exposure equivalent to 2
months of exposure to sunlight.

Then, a custom additive-free batch of PEN was successfully synthesised and
matched the performance of commercial PEN foils. As the room temperature WLSE
reached is still approx. 3 times lower than for TPB, examination of further
steps of the synthesis is planned, including a different choice of catalyst,
as well as a separate optimization of the foil pressing/stretching process.

The partial results described in this report contribute to a larger broadly-
scoped R&D program focused on WLS. Its remaining components are described
elsewhere: a new table-top cryogenic facility for time-resolved WLS
characterization [8], and long-term stability and performance testing in a
liquid argon environment [9].

## Acknowledgments

This work has been supported from the EU’s Horizon 2020 research and
innovation programme under grant agreement No 952480 (DarkWave), and from the
International Research Agenda Programme AstroCeNT (MAB/2018/7) funded by the
Foundation for Polish Science from the European Regional Development Fund
(ERDF). We thank Mateusz Głowacki (Orange Polska S.A.) for aging the samples
in the climatic chamber. We are grateful to Prof. Magdalena Skompska for
access to the spectrophotometer, which was purchased by CNBCh (University of
Warsaw) from the project co-financed by EU from the ERDF.

## References

  * [1] M. Kuźniak and A.M. Szelc, _Wavelength shifters for applications in liquid argon detectors_ , _Instruments_ 5 (2021) 4.
  * [2] M. Kuźniak, B. Broerman, T. Pollmann and G.R. Araujo, _Polyethylene naphthalate film as a wavelength shifter in liquid argon detectors_ , _The European Physical Journal C_ 79 (2019) 291.
  * [3] M.G. Boulay, V. Camillo, N. Canci, S. Choudhary, L. Consiglio, A. Flammini et al., _Direct comparison of PEN and TPB wavelength shifters in a liquid argon detector_ , _The European Physical Journal C_ 81 (2021) 1099.
  * [4] Y. Abraham, J. Asaadi, V. Basque, W. Castiglioni, R. Dorrill, M. Febbraro et al., _Wavelength-shifting performance of polyethylene naphthalate films in a liquid argon environment_ , _Journal of Instrumentation_ 16 (2021) P07017.
  * [5] G.R. Araujo, L. Baudis, N. McFadden, P. Krause, S. Schönert and V.H.S. Wu, _R &D of wavelength-shifting reflectors and characterization of the quantum efficiency of tetraphenyl butadiene and polyethylene naphthalate in liquid argon_, _The European Physical Journal C_ 82 (2022) 442.
  * [6] D. Mary, G. Teyssedre and C. Laurent, _UV-induced degradation of poly(ethylene naphthalate) films from the standpoint of electrical and luminescence properties_ , in _2001 Annual Report Conference on Electrical Insulation and Dielectric Phenomena (Cat. No.01CH37225)_ , pp. 165–168, 2001, https://doi.org/10.1109/CEIDP.2001.963512.
  * [7] H. Shimizu, F. Murakawa, Z. Shidara, H. Nakamura and Y. Shirakawa, _Resin for scintillators_ , patent US 2014/0166890 A1, June 2014.
  * [8] S. Choudhary, A.F.V. Cortez, M. Kuźniak, G. Nieradka, T. Sworobowicz, Ł. Świderski et al., _Cryogenic setup for the characterization of wavelength-shifting materials for noble element radiation detectors_ , 2024.  [arXiv:2401.05004].
  * [9] M. Goldbrunner et al., _Development & characterization of a PEN-based wavelength shifting reflector_, 2023.  Poster presented at LIDINE 2023.