# Transformer-QEC: Quantum Error Correction Code Decoding
with Transferable Transformers

Hanrui Wang1, Pengyu Liu2, Kevin Shao1, Dantong Li3,
Jiaqi Gu4, David Z. Pan5, Yongshan Ding3, Song Han1
1Massachusetts Institute of Technology, 2Carnegie Mellon University, 3Yale
University, 4Arizona State University, 5University of Texas at Austin

###### Abstract.

Quantum computing has the potential to solve problems that are intractable for
classical systems, yet the high error rates in contemporary quantum devices
often exceed tolerable limits for useful algorithm execution. Quantum Error
Correction (QEC) mitigates this by employing redundancy, distributing quantum
information across multiple data qubits and utilizing syndrome qubits to
monitor their states for errors. The syndromes are subsequently interpreted by
a decoding algorithm to identify and correct errors in the data qubits. This
task is complex due to the multiplicity of error sources affecting both data
and syndrome qubits as well as syndrome extraction operations. Additionally,
identical syndromes can emanate from different error sources, necessitating a
decoding algorithm that evaluates syndromes collectively. Although machine
learning (ML) decoders such as multi-layer perceptrons (MLPs) and
convolutional neural networks (CNNs) have been proposed, they often focus on
local syndrome regions and require retraining when adjusting for different
code distances. To address these issues, we introduce a transformer-based QEC
decoder, termed Transformer-QEC, which employs self-attention to achieve a
global receptive field across all input syndromes. It incorporates a mixed
loss training approach, combining both local physical error and global parity
label losses. Moreover, the transformer architecture’s inherent adaptability
to variable-length inputs allows for efficient transfer learning, enabling the
decoder to adapt to varying code distances without retraining.

Evaluation on six code distances and ten different error configurations
demonstrates that our model consistently outperforms non-ML decoders, such as
Union Find (UF) and Minimum Weight Perfect Matching (MWPM), and other ML
decoders, thereby achieving best logical error rates. Moreover, the transfer
learning can save over 10$\times$ of training cost.

##  1\. Introduction

Figure 1. Transformer for error correction decoding overview. The Transformer
takes syndrome inputs and processes them through both the transformer model
encoder and decoder. The output of this process consists of error predictions.
One notable advantage is its ability to be seamlessly applied to different
code distances due to the transformer model’s flexible input and output size.

The field of Quantum Computing (QC) has attracted significant attention in
research circles as a novel computational paradigm, poised to tackle
challenges previously beyond the reach of conventional computing with
remarkable efficiency. QC offers promising prospects across various industries
and academic fields, notably impacting areas such as cryptography (Shor,
1999), database searching (Grover, 1996), combinatorial optimization (Farhi et
al., 2014; Liang et al., 2023), molecular dynamics simulations (Peruzzo et
al., 2014), and advancements in machine learning (Lloyd et al., 2013; Liang et
al., 2021; Wang et al., 2022a, b, c, d, 2023b; Zheng et al., 2023; Cheng et
al., 2022; Liang et al., 2022), among others.

Despite significant progress in quantum computing hardware (IBM, [n. d.]),
current devices exhibit high error rates, ranging from $10^{-3}$ to $10^{-2}$,
which are orders of magnitude above the thresholds required for practical
applications (below $10^{-10}$) (Lee et al., 2021). Addressing these error
rates is crucial for advancing the field. Quantum Error Correction (QEC)
serves as a key technique for error mitigation by incorporating
redundancy—distributing the information of a single logical qubit across
multiple physical qubits. With adequate redundancy in QEC codes, the logical
error rate can decrease exponentially, provided the physical error rate $p$
remains below a specific threshold. Thus, judiciously managed redundancy via
QEC can enable quantum systems to achieve the low error rates necessary for
practical computation. In QEC, logical qubits encode across multiple data
qubits, with error detection aided by parity qubits, as shown in Figure 1.
Periodic syndrome extraction localizes errors into detectable syndromes over
several cycles. A classical decoder then interprets these syndromes to correct
errors, influenced by physical error rates and decoder performance. Decoders
often require a broad syndrome field due to potential syndrome overlap from
different errors.

Figure 2. The surface code contains data qubits and two kinds of syndrome
qubits. X-type syndrome qubits in green checks Z errors while Z-type syndrome
qubits in yellow check X errors. When error occurs on data qubits, the nearby
syndromes may be flipped depending on the parity of data qubits. When multiple
error occurs, the syndrome patterns will be more difficult to decode.

The rotated surface code (Horsman et al., 2012) is a promising candidate for
realizing fault tolerance. Well-established algorithms for surface code
include Minimum Weight Perfect Matching (MWPM) and Union Find (UF). Recently,
machine learning (ML), especially neural network (NN) based decoders have
gained attention due to a few desirable characteristics. First, they generally
run in constant time, which is necessary to prevent a backlog of syndrome
outcomes. Second, unlike MWPM, they are capable of learning both correlations
between physical errors (such as the correlation between X and Z error in
depolarizing errors) as well as learning hidden and potentially changing
underlying physical error distributions.

However, ML based decoders also bring significant challenges. First, several
models, such as those grounded in convolutional neural networks (CNN), are
limited by a small receptive field. This constraint may hinder their ability
to accurately pinpoint long error chains. Second, many models, like the multi-
layer perceptron (MLP), has a fixed size for both input and output. As a
result, changes in code distance would mandate retraining of an entirely new
model, leading to considerable overhead.

Therefore, to solve these challenges, we propose Transformer-QEC, a
transferable transformer model designed for accurate and efficient decoding of
surface code, as illustrated in Fig. 1. For the sake of simplicity, the figure
only depicts two dimensions, but in reality, the input syndromes include an
additional temporal dimension – $round$ as in Fig. 2. Our proposed model
employs a transformer structure, incorporating both an encoder and a decoder
to process the syndromes. Binary features on each syndrome qubit are projected
to token embeddings and augmented with a 3D sinusoidal positional encoding,
informing the model about the location of each qubit. The embeddings of the 3D
inputs are then flattened to 1D input sequence and processed by the
transformer encoder layer. Thanks to the global interaction capability brought
by attention layer, all input syndromes can be considered holistically which
boosts accuracy. The decoder then uses the positional encoding of the data
qubits to predict the X or Z errors on each of them. Moreover, we propose a
mixed loss that combines the loss from the local physical error of each qubit
with the loss from global parity prediction.

To optimize computational efficiency in the context of varying code distances
for quantum error correction, we advocate for a transfer learning paradigm.
Taking advantage of the transformer model’s capacity for arbitrary input
lengths, we repurpose pretrained models for different code distances by merely
modifying the input sequence. For example, a model trained initially for a
code distance $d=5$ can be efficiently fine-tuned for alternate distances
($d=7$ or $d=9$), thereby achieving enhanced performance metrics. Our approach
yields a tenfold reduction in computational costs as compared to training from
scratch, as verified through our experiments.

Figure 3. Multiple rounds of surface code measurement. The progression of time
is depicted by moving upwards from the array at the base, with each horizontal
plane representing a step in the measurement process. In reality, errors will
also occur in the syndrome extraction circuit and syndrome qubits,
necessitating the need to repeat multiple rounds for decoding. On the right
side, the measurement error on the syndrome qubit will also flip the syndrome.
Figure 4. Overall workflow of Transformer-QEC. The syndromes are firstly
processed by the transformer model to predict the errors. Since the errors may
not fully clear all syndromes, we will pass the cleared syndromes to a global
decoder to predict a global parity. The final global parity is the XOR of the
global parity from transformer predicted physical error and that predicted by
global decoder. Figure 5. Syndrome extraction circuit. Top: Z-type syndrome
qubits. Bottom: X-type syndrome qubits.

We extensively evaluate Transformer-QEC across six code distances, 3, 5, 7, 9,
11 and 13 and compare it with MWPM, UF and MLP baselines under 10 different
error rates. Our results demonstrate that Transformer-QEC consistently
surpasses these baselines, achieving the lowest logical error rates. In
summary, Transformer-QEC makes four key contributions:

  * •

A novel transformer-based model for surface code decoding which uses the
syndrome with positional encoding as inputs and predict errors.

  * •

A mixed loss approach combined loss from local physical error prediction and
global parity prediction improves the model’s trainability and performance.

  * •

Transfer learning across different code distances. For the first time, we
propose to transfer the knowledge learn on one distance to another, thus
reducing costs.

  * •

Extensive evaluations on different physical error rates and distances
demonstrates that our model consistently outperforms baselines such as MWPM,
UF and MLP.

##  2\. Background and Related Works

Quantum error correction. Quantum Error Correction (QEC) enhances logical
qubit fidelity by expanding into multiple physical qubits. The method involves
syndrome extraction and error decoding steps. Using auxiliary syndrome qubits,
errors are detected and classified into discrete Pauli categories (Nielsen and
Chuang, 2002). If the physical error rate is below a certain threshold, QEC
reduces the logical error rate, requiring more physical qubits. Quantum
techniques often have added complexity compared to classical ones, with
specific schemes viable for NISQ devices. This work focuses on the rotated
surface code.

Surface code. The surface code is a leading QEC scheme that employs a two-
dimensional lattice of interlaced data and syndrome qubits to encode a logical
qubit. Notable for its high error threshold and need for only nearest-neighbor
connectivity, it offers practicality for real-world quantum systems. The code
distance $D$ dictates the lattice size and correlates with error resilience.
Adjacent parity qubits detect errors on data qubits via a stabilizer circuit,
capable of identifying X, Z, or Y errors, as depicted in Fig. 5. The code can
correct error chains up to a length of $\lfloor\frac{D-1}{2}\rfloor$. A
simplified variant, the ’rotated’ surface code (Fig. 2), reduces qubit and
gate overhead and is often favored in practice. Characterized as a
$[[D^{2},1,D]]$ stabilizer code, it stands as a viable option for near-term
fault-tolerant quantum computation. Its design allows transversal single-qubit
operations and enables two-qubit CNOT gates via lattice surgery (Clare et al.,
2012), thereby enhancing its feasibility for physical implementations.
Decoders analyze syndromes from ancilla measurements to correct data qubit
errors. They independently address X-type and Z-type errors, implicitly fixing
Y-type errors. For large-scale FTQCs, decoders must be accurate, fast, and
scalable (Das et al., 2022). Accuracy indicates reliable error detection,
latency mandates cycle-limited operation, and scalability ensures efficient
resource use. Increased accuracy may prolong operation time.

Decoders interpret syndromes, outcomes of ancilla measurements as shown in
Fig.3, to identify and correct errors in data qubits. X-type and Z-type errors
are treated independently, which inherently rectifies Y-type errors. Effective
decoders for large-scale fault-tolerant quantum computers (FTQCs) must satisfy
three primary criteria: accuracy, latency, and scalability(Das et al., 2022).
Accuracy denotes the decoder’s reliability in error identification. Latency
requires the decoder to function within a single syndrome extraction cycle.
Scalability entails efficient resource utilization, crucial for hardware-
constrained environments. Notably, increased accuracy often comes at the cost
of extended operational time.

ML based decoder. The landscape of quantum computing has been significantly
enriched by ML-based decoders, notably through Neural Network (NN) and
Reinforcement Learning (RL) paradigms. In the realm of NNs, Boltzmann machines
initiated ML-based decoding in toric codes (Torlai and Melko, 2017), later
broadened by Multi-Layer Perceptrons (Chamberland and Ronagh, 2018) and Long
Short-Term Memory networks for surface codes (Baireuther et al., 2018). On the
RL front, advancements include transforming decoding into an RL environment
(Ryan et al., 2020) and optimizing RL decoders for specific error types
(Andreasson et al., 2019). Efforts for scalable decoding have ranged from low-
depth Convolutional Neural Networks (Breuckmann and Ni, 2018) to multilevel
architectures (Wagner et al., 2020). Guided by these contributions, our
research focuses on employing Transformer-based NNs for local decoding, aiming
to develop novel architectures and enhance scalability through a two-level
local-global design. There exist numerous hardware accelerators (Wang et al.,
2020, 2021; Ji* et al., 2023; Lin et al., 2021; Wang* et al., 2020) that can
be used to improve the efficiency and speed of NN-based decoders.

Figure 6. Transformer model architecture. The input of the syndromes will be
encoded by a ($D+1$) cubic grid. The input will go through the transformer
encoder with self attention and FFN layers. Then the transformer decoder will
produce the physical error predictions by processing the positional encoding
of data qubits with size $D$ cubic.

Non-ML based decoder. Various decoding algorithms for QEC offer distinct
advantages. Minimum Weight Perfect Matching (MWPM) uses Edmonds’ method for
optimal error syndrome pairing in topological codes (Fowler et al., 2012a; Wu
and Zhong, 2023). The Union Find (UF) decoder features linear-time complexity
for toric and surface codes (Delfosse and Zémor, 2017). Lookup Table (LUT)
decoders rely on predetermined error patterns (Tomita and Svore, 2014), while
Tensor Network (TN) decoders utilize tensor structures for high error
thresholds in topological codes (Chubb, 2021). MWPM decoders (Edmonds, 1965;
Dennis et al., 2002a; Fowler et al., 2012a; Holmes et al., 2020) are
considered to have a good balance between the decoding accuracy and speed. It
has almost linear time complexity (Fowler et al., 2012b; Higgott and Gidney,
2023; Wu and Zhong, 2023), is practical for hardware implementation for real-
time decoding (Vittal et al., 2023) and is more accurate than LUT and Union
Find decoders. (Wang et al., 2023a) proposes to tackle the noise drift by
updating the MWPM decoding graph weights.

##  3\. Methodology

In this section, we delineate the error correction pipeline of our
Transformer-QEC framework, followed by detailed discussions on the
incorporated transformer model and transfer learning.

Overall workflow. The iterative syndrome extraction procedure yields syndrome
outcomes at discrete rounds, serving as the basis for the decoder’s error
predictions. Purely ML-based decoders, while effective, may yield predictions
misaligned with the original syndromes. Therefore, an auxiliary, non-ML
decoder is employed for syndrome clearance, as depicted in Figure 4. Post-ML
decoding, the predicted errors are utilized to reconcile syndromes and
ascertain their global parity. Any residual errors are addressed by a global
decoder like MWPM, ensuring complete syndrome clearance and yielding an
additional global parity. The decoding process culminates in the XOR of the
two global parities obtained.

Transformer model. Given that the computational complexity of global decoders
such as MWPM is often proportional to the number of non-zero syndromes, the
efficiency of a preceding ML-based decoder in clearing syndromes can be highly
advantageous. To this end, we introduce a novel Transformer-based decoder,
schematically represented in Figure 6. To encode input syndromes, we employ a
cubic grid framework. For a surface code characterized by a distance $D$, a
$D+1$ square grid is utilized to ensure that each syndrome qubit resides at a
grid intersection. Typically, the number of iterative rounds for syndrome
extraction is set equal to the code distance. In our architecture, we
incorporate an additional layer dedicated to the final syndrome measurement,
thereby extending the round dimension to $D+1$. Consequently, the feature
space manifests as a $D+1$ cubic grid. Each cell within this grid encapsulates
a six-dimensional feature vector. The initial two dimensions specifically
indicate the positions of the X-check and Z-check syndrome qubits, as further
illustrated in Fig. 6.

Location encoding for the X and Z check syndrome qubits:

$\begin{bmatrix}0&1&0&1&0&0\\\ 0&0&1&0&1&0\\\ 0&1&0&1&0&0\\\ 0&0&1&0&1&0\\\
0&1&0&1&0&0\\\ 0&0&1&0&1&0\\\ \end{bmatrix}\begin{bmatrix}0&0&0&0&0&0\\\
0&1&0&1&0&1\\\ 1&0&1&0&1&0\\\ 0&1&0&1&0&1\\\ 1&0&1&0&1&0\\\ 0&1&0&1&0&0\\\
\end{bmatrix}$

Subsequent to the positional dimensions, the next two channels in the six-
dimensional feature vector encode the syndromes, varying across iterative
rounds. For ultimate error correction, data qubits are measured in a
designated basis. To enable network generalization across different syndrome
measurement rounds, we define temporal lattice boundaries. Specifically, the
fifth channel is initialized to 1 for the inaugural round and 0 for later
rounds. Analogously, the sixth channel is set to 1 solely for the terminal
round and 0 otherwise.

The feature vectors undergo dimensionality elevation via a learnable embedding
layer. Subsequently, 3D sinusoidal positional encodings are integrated to
convey qubit locations. This enhanced representation is then reshaped from 3D
to 1D before forwarding to the Transformer encoder. The encoder comprises
multiple layers, each featuring one multi-head self-attention (MHSA) module
and one feed-forward network (FFN). The MHSA facilitates long-range contextual
awareness by allowing each syndrome to attend to any other within the 3D grid.
The FFN consists of two fully-connected layers, serving to further elevate the
feature dimensionality, apply an activation function, and project back.

To predict physical errors, we employ Transformer decoder layers, using the
positional encodings of data qubit locations as inputs. Each decoder layer
comprises a self-attention module and a cross-attention module interfacing
with the encoder. Queries for cross-attention originate from the decoder
inputs, while keys and values are sourced from the encoder, thereby enabling
the decoder to incorporate preceding syndrome information. Subsequently, a FFN
layer and a prediction layer output the logits. Owing to the higher cost of
false positives relative to false negatives, a confidence threshold is used
for error prediction. Specifically, post-Sigmoid confidence $>0.95$ triggers a
positive prediction.

Mixed loss: We introduce a composite loss function during training,
integrating contributions from two distinct aspects. The first component
arises from the prediction of local physical errors, while the second pertains
to global parity prediction. The latter is computed via global average pooling
of encoder output embeddings, subsequently processed through a prediction
layer, as depicted in Figure 6 (top right). This global parity serves as
auxiliary information, enhancing the model’s generalization across diverse
syndrome patterns.

Transfer learning: Each code type encompasses a family of codes with varying
distances, which necessitates different logical error rates based on the
quantum algorithm deployed. Traditional approaches typically require
retraining for each new code distance, incurring substantial computational and
temporal overhead. To mitigate this, Transformer-QEC employs a transfer
learning strategy, capitalizing on the inherent similarities between codes of
different distances. For instance, handling syndromes in a distance-5 code
bears resemblance to managing a sub-block in a distance-7 code. Utilizing a
pre-trained model, we fine-tune it on the new distance’s dataset. This
flexibility is enabled by the Transformer architecture’s inherent ability to
accommodate sequences of arbitrary lengths. The sole aspect warranting careful
adjustment is the positional encoding for the new distance.

##  4\. Evaluation

Table 1. Comparison of logical error rates under different code distance and
physical error rates.

| Logical Error Rate $\downarrow$
---|---
Distance | Phys. Err. Rate | UF | MWPM | MLP | Transformer-QEC
3 | 0.0500 | 0.16745 | 0.14063 | 0.14794 | 0.13005
0.0100 | 0.01039 | 0.00800 | 0.00903 | 0.00784
5 | 0.0500 | 0.24120 | 0.17279 | 0.20888 | 0.17232
0.0100 | 0.00406 | 0.00268 | 0.00443 | 0.00254
7 | 0.0500 | 0.29813 | 0.20178 | 0.28454 | 0.20590
0.0100 | 0.00113 | 0.00064 | 0.00197 | 0.00059
9 | 0.0500 | 0.35250 | 0.23161 | 0.32770 | 0.23144
0.0100 | 0.00028 | 0.00002 | 0.00017 | 0.00001

###  4.1. Evaluation Methodology

Benchmarks. We have selected the rotated surface code with distances of 3, 5,
7, 9. The round is set to be the same as the distance. The phenomenological
error model (Dennis et al., 2002b) we use encompasses errors on syndrome
measurement and data qubits. Each syndrome qubit experiences a measurement
error with a probability $p$. The errors on data qubits are depolarizing
errors, which occur with a probability $p$, causing Pauli X, Y, or Z errors
with equal probability. As assumed in previous work (Das et al., 2021), the
error probabilities of these two types are considered to be equal. We choose
values of $p$ from the set 0.05, 0.04, 0.03, 0.025, 0.02, 0.015, 0.01, 0.0075,
0.005, 0.0025. The Google Stim package is used to construct the circuit and
perform stabilizer simulations.

Baselines. Our three baselines include the Union Find decoder, the MWPM
decoder as implemented in (Wu and Zhong, 2023), and a MLP architecture.
Following (Overwater et al., 2022), our MLP architecture has two hidden
layers, with the dimensions of these layers set empirically. As the MLP
requires fixed-size inputs and generates fixed-size outputs, it does not
facilitate transfer learning like the Transformer does.

Training settings. Our main model is a Transformer with 6 layers, an embedding
dimension of 256, 8 heads, and a FFN hidden dimension of 512. This model
contains 7.9 million parameters. We also have a smaller model with 6 layers,
an embedding dimension of 64, 2 heads, and an FFN hidden dimension of 128,
which includes 0.5 million parameters. For training, we collect a dataset of
1,000,000 samples with a 1% error rate. We use a learning rate of 0.001 with
linear warmup and cosine decay, a weight decay with lambda 0.0001, and we
train for 100 epochs. We utilize the Adam optimizer with a weighted binary
cross-entropy loss for local physical errors, and a normal binary cross-
entropy loss for global parity errors. For the MLP model, we use a physical
error rate of 1% for $d=3,5$ and 2.5% for $d=7,9$. Like the initial
Transformer model, we train for 100 epochs with the Adam optimizer. Training
is conducted on a single NVIDIA A6000 GPU.

Figure 7. Accuracy comparison between the Transformer-QEC an MLP baseline.
Class 0 accuracy is the accuracy of correctly identify a no error data qubit
as no error (True Negative). Class 1 accuracy is the accuracy of correctly
identify an error when the data qubit has error (True Positive).

Transfer learning settings. The distance 5 model, trained from scratch, is
used as the source model for transfer learning. For all other distances, we
use a constant learning rate of 0.0005 and train for 10 epochs. All other
settings remain identical to the settings of training from scratch.

###  4.2. Experiment Results

Figure 8. Threshold of transformer based decoder. The threshold indicated the
largest acceptable physical error rate for which using QEC can reduce error
rate. Transformer obtains about 0.038 threshold.

Main results. Table 1 presents our primary results for varying code distances
and physical error rates. Notice that all the models are transferred from the
distance 5 model. In general, Transformer-QEC achieves a lower logical error
rate for all benchmarks. The improvements over Union Find and MLP decoders are
considerably more significant than the MWPM. This is likely because the global
decoder in Transformer-QEC’s framework also employs an MWPM. The MLP model can
surpass the Union Find but is generally not as good as MWPM and Transformer-
QEC, even though the MLP models are trained individually for each code
distance. This result highlights the effectiveness of our proposed transfer
learning techniques. Furthermore, in Figure 7, we show the physical error
prediction accuracy for baseline MLP and our Transformer-QEC. The class 0
accuracy means the ratio of predicted 0 when the ground truth is 0 (true
negative). The class 1 accuracy means the ratio of predicted 1 when the ground
truth is 1 (true positive). We can see that the accuracy for class 0 is in
general much higher then class 1 because of the imbalance of training dataset.
Moreover, our Transformer-QEC can achieve 43% higher accuracy on the class 1
which means the Transformer-QEC model can identify errors with much higher
reliability. That is beneficial when we desire the low level decoder to clear
as many as syndromes as possible and speedup the end-to-end process.

Evaluation of the threshold. Figure 8 shows the threshold evaluation of
Transformer-QEC, with the X-axis as physical and Y-axis as logical error
rates. The curves of different distances intersect at a point where the
physical error rate is 0.0038 and the logical error rate is around 0.09. When
$p$ is smaller than the threshold, larger code distances reduce the logical
error rate. However, when $p$ is larger than the threshold, larger distances
do not help. Instead, we observe larger logical error rates. This trend can be
attributed to the increased error introduced by larger system sizes, which
eclipses the benefits of greater redundancy with more qubits.

Effectiveness of mixed loss. To evaluate the mixed loss function, we perform
an ablation study on the distance 5 code, as shown in Table 2. Each column
shows the comparison of the logical error rate under a specific physical error
rate. The performance with both local and global loss can achieve better or
equivalent performance for nine out of ten cases. This demonstrates that the
global parity loss provides valuable guidance during the model’s training
process.

Ablation on model size. We evaluate two models with different sizes but the
same training setting in Table 3 under code distance 5 and varying physical
error rates. It is evident that the larger model, with approximately 8 million
parameters, outperforms the smaller model with 500 thousand parameters. The
larger model is not overfitted to the training set and performs poorly on
testing. This outcome is mainly due to the large size of the training set.

Table 2. Comparison of logical error rate with global loss.

Error Rate | 0.0500 | 0.0400 | 0.0300 | 0.0250 | 0.0200
---|---|---|---|---|---
Local loss | 0.17276 | 0.10659 | 0.05207 | 0.03384 | 0.01751
\+ Global loss | 0.17232 | 0.10659 | 0.05196 | 0.03384 | 0.01744
Error Rate | 0.0150 | 0.0100 | 0.0075 | 0.0050 | 0.0025
Local loss | 0.00808 | 0.00259 | 0.00097 | 0.00039 | 0.00007
\+ Global loss | 0.00802 | 0.00254 | 0.00103 | 0.00035 | 0.00005

Table 3. Comparison of logicla error rates under different model size.

Error Rate | 0.0200 | 0.0150 | 0.0100 | 0.0075 | 0.0050
---|---|---|---|---|---
503K Params | 0.01812 | 0.00860 | 0.00290 | 0.00127 | 0.00045
7,911K Params | 0.01744 | 0.00802 | 0.00254 | 0.00103 | 0.00035

##  5\. Conclusion

In conclusion, our study presents a robust QEC decoder for rotated surface
codes using machine learning and transformer models. It outperforms existing
benchmarks across code distances and facilitates quick transfer learning. Key
factors for this success include global decoding and large Transformer models.
This work advances ML-based Transformer decoders, enhancing accuracy and speed
in error correction of the surface codes and beyond.

## References

  * (1)
  * Andreasson et al. (2019) P. Andreasson et al. 2019\.  Quantum error correction for the toric code using deep reinforcement learning.  _Quantum_ 3 (Sept. 2019), 183.
  * Baireuther et al. (2018) P. Baireuther et al. 2018\.  Machine-learning-assisted correction of correlated qubit errors in a topological code.  _Quantum_ 2 (Jan. 2018), 48.
  * Breuckmann and Ni (2018) Nikolas P. Breuckmann and Xiaotong Ni. 2018.  Scalable Neural Network Decoders for Higher Dimensional Quantum Codes.  _Quantum_ 2 (May 2018), 68.
  * Chamberland and Ronagh (2018) Christopher Chamberland and Pooya Ronagh. 2018.  Deep neural decoders for near term fault-tolerant experiments.  _Quantum Science and Technology_ 3, 4 (jul 2018).
  * Cheng et al. (2022) Jinglei Cheng, Hanrui Wang, Zhiding Liang, Yiyu Shi, Song Han, and Xuehai Qian. 2022.  TopGen: Topology-Aware Bottom-Up Generator for Variational Quantum Circuits.  _arXiv preprint arXiv:2210.08190_ (2022).
  * Chubb (2021) Christopher T. Chubb. 2021.  General tensor network decoding of 2D Pauli codes.  arXiv:2101.04125 [quant-ph]
  * Clare et al. (2012) H. Clare et al. 2012\.  Surface code quantum computing by lattice surgery.  _New Journal of Physics_ 14, 12 (dec 2012), 123011.
  * Das et al. (2022) P. Das et al. 2022\.  AFS: Accurate, Fast, and Scalable Error-Decoding for Fault-Tolerant Quantum Computers. In _HPCA_. IEEE, 259–273.
  * Das et al. (2021) Poulami Das, Aditya Locharla, and Cody Jones. 2021.  LILLIPUT: A Lightweight Low-Latency Lookup-Table Based Decoder for Near-term Quantum Error Correction.  arXiv:2108.06569 [quant-ph]
  * Delfosse and Zémor (2017) N. Delfosse and G Zémor. 2017.  Linear-Time Maximum Likelihood Decoding of Surface Codes Over the Quantum Erasure Channel.  _Quantum Information & Computation_ (2017).
  * Dennis et al. (2002a) Eric Dennis, Alexei Kitaev, Andrew Landahl, and John Preskill. 2002a.  Topological quantum memory.  _J. Math. Phys._ 43, 9 (2002), 4452–4505.
  * Dennis et al. (2002b) Eric Dennis, Alexei Kitaev, Andrew Landahl, and John Preskill. 2002b.  Topological quantum memory.  _J. Math. Phys._ 43, 9 (2002), 4452–4505.
  * Edmonds (1965) Jack Edmonds. 1965.  Maximum matching and a polyhedron with 0, 1-vertices.  _Journal of research of the National Bureau of Standards B_ 69, 125-130 (1965), 55–56.
  * Farhi et al. (2014) Edward Farhi, Jeffrey Goldstone, and Sam Gutmann. 2014.  A quantum approximate optimization algorithm.  _arXiv preprint arXiv:1411.4028_ (2014).
  * Fowler et al. (2012a) A. Fowler et al. 2012a.  Surface codes: Towards practical large-scale quantum computation.  _Physical Review A_ 86, 3 (2012), 032324.
  * Fowler et al. (2012b) Austin G Fowler, Adam C Whiteside, and Lloyd CL Hollenberg. 2012b.  Towards practical classical processing for the surface code: timing analysis.  _Physical Review A_ 86, 4 (2012), 042313.
  * Grover (1996) Lov K Grover. 1996.  A fast quantum mechanical algorithm for database search. In _STOC_. 212–219.
  * Higgott and Gidney (2023) Oscar Higgott and Craig Gidney. 2023.  Sparse Blossom: correcting a million errors per core second with minimum-weight matching.  _arXiv preprint arXiv:2303.15933_ (2023).
  * Holmes et al. (2020) Adam Holmes, Mohammad Reza Jokar, Ghasem Pasandi, Yongshan Ding, Massoud Pedram, and Frederic T Chong. 2020.  NISQ+: Boosting quantum computing power by approximating quantum error correction. In _2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA)_. IEEE, 556–569.
  * Horsman et al. (2012) C. Horsman et al. 2012\.  Surface code quantum computing by lattice surgery.  _New Journal of Physics_ 14, 12 (2012), 123011.
  * IBM ([n. d.]) IBM. [n. d.].  IBM Unveils Breakthrough 127-Qubit Quantum Processor.
  * Ji* et al. (2023) Zexi Ji*, Hanrui Wang*, Miaorong Wang, Win-San Khwa, Meng-Fan Chang, Song Han, and Anantha P. Chandrakasan. 2023.  SpAtten-Chip: A Fully-Integrated Energy-Scalable Transformer Accelerator Supporting Adaptive Model Configuration and Token Pruning for Language Understanding on Edge Devices. In _2023 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED)_. 1–6.  https://doi.org/10.1109/ISLPED58423.2023.10244459
  * Lee et al. (2021) J. Lee et al. 2021\.  Even more efficient quantum computations of chemistry through tensor hypercontraction.  _PRX Quantum_ 2, 3 (2021), 030305.
  * Liang et al. (2021) Z. Liang et al. 2021\.  Can noise on qubits be learned in quantum neural network? a case study on quantumflow. In _ICCAD_. IEEE, 1–7.
  * Liang et al. (2022) Zhiding Liang, Jinglei Cheng, Hang Ren, Hanrui Wang, Fei Hua, Yongshan Ding, Fred Chong, Song Han, Yiyu Shi, and Xuehai Qian. 2022.  Pan: Pulse ansatz on nisq machines.  _arXiv preprint arXiv:2208.01215_ (2022).
  * Liang et al. (2023) Zhiding Liang, Zhixin Song, Jinglei Cheng, Zichang He, Ji Liu, Hanrui Wang, Ruiyang Qin, Yiru Wang, Song Han, Xuehai Qian, and Yiyu Shi. 2023.  Hybrid Gate-Pulse Model for Variational Quantum Algorithms. In _2023 60th ACM/IEEE Design Automation Conference (DAC)_. 1–6.  https://doi.org/10.1109/DAC56929.2023.10247923
  * Lin et al. (2021) Yujun Lin, Zhekai Zhang, Haotian Tang, Hanrui Wang, and Song Han. 2021.  PointAcc: Efficient Point Cloud Accelerator. In _MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture_ (Virtual Event, Greece) _(MICRO ’21)_. Association for Computing Machinery, New York, NY, USA, 449–461.  https://doi.org/10.1145/3466752.3480084
  * Lloyd et al. (2013) S. Lloyd et al. 2013\.  Quantum algorithms for supervised and unsupervised machine learning.  _arXiv preprint arXiv:1307.0411_ (2013).
  * Nielsen and Chuang (2002) Michael A Nielsen and Isaac Chuang. 2002.  Quantum computation and quantum information.
  * Overwater et al. (2022) Ramon W. J. Overwater, Masoud Babaie, and Fabio Sebastiano. 2022.  Neural-Network Decoders for Quantum Error Correction Using Surface Codes: A Space Exploration of the Hardware Cost-Performance Tradeoffs.  _IEEE TQC_ (2022).
  * Peruzzo et al. (2014) A. Peruzzo et al. 2014\.  A variational eigenvalue solver on a photonic quantum processor.  _Nature communications_ 5, 1 (2014), 1–7.
  * Ryan et al. (2020) S. Ryan et al. 2020\.  Reinforcement learning decoders for fault-tolerant quantum computation.  _Machine Learning: Science and Technology_ 2, 2 (dec 2020), 025005.
  * Shor (1999) Peter W Shor. 1999.  Polynomial-time algorithms for prime factorization and discrete logarithms on a quantum computer.  _SIAM review_ 41, 2 (1999), 303–332.
  * Tomita and Svore (2014) Yu Tomita and Krysta M. Svore. 2014.  Low-distance surface codes under realistic quantum noise.  _Phys. Rev. A_ 90 (Dec 2014), 062320.  Issue 6.
  * Torlai and Melko (2017) Giacomo Torlai and Roger G. Melko. 2017.  Neural Decoder for Topological Codes.  _Phys. Rev. Lett._ 119 (Jul 2017), 030501.  Issue 3. https://doi.org/10.1103/PhysRevLett.119.030501
  * Vittal et al. (2023) Suhas Vittal, Poulami Das, and Moinuddin Qureshi. 2023.  Astrea: Accurate Quantum Error-Decoding via Practical Minimum-Weight Perfect-Matching. In _Proceedings of the 50th Annual International Symposium on Computer Architecture_ (Orlando, FL, USA) _(ISCA ’23)_. Association for Computing Machinery, New York, NY, USA, Article 2, 16 pages.  https://doi.org/10.1145/3579371.3589037
  * Wagner et al. (2020) Thomas Wagner, Hermann Kampermann, and Dagmar Bruß. 2020.  Symmetries for a high-level neural decoder on the toric code.  _Phys. Rev. A_ 102 (Oct 2020), 042411.  Issue 4. https://doi.org/10.1103/PhysRevA.102.042411
  * Wang et al. (2020) Hanrui Wang et al. 2020\.  _Efficient algorithms and hardware for natural language processing_.  Ph. D. Dissertation. Massachusetts Institute of Technology.
  * Wang et al. (2022a) Hanrui Wang, Yongshan Ding, Jiaqi Gu, Yujun Lin, David Z Pan, Frederic T Chong, and Song Han. 2022a.  QuantumNAS: Noise-adaptive search for robust quantum circuits. In _2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA)_. IEEE, 692–708.
  * Wang et al. (2022b) Hanrui Wang, Jiaqi Gu, Yongshan Ding, Zirui Li, Frederic T Chong, David Z Pan, and Song Han. 2022b.  QuantumNAT: quantum noise-aware training with noise injection, quantization and normalization. In _Proceedings of the 59th ACM/IEEE Design Automation Conference_. 1–6.
  * Wang et al. (2022c) Hanrui Wang, Zirui Li, Jiaqi Gu, Yongshan Ding, David Z Pan, and Song Han. 2022c.  QOC: quantum on-chip training with parameter shift and gradient pruning. In _Proceedings of the 59th ACM/IEEE Design Automation Conference_. 655–660.
  * Wang et al. (2022d) Hanrui Wang, Pengyu Liu, Jinglei Cheng, Zhiding Liang, Jiaqi Gu, Zirui Li, Yongshan Ding, Weiwen Jiang, Yiyu Shi, Xuehai Qian, et al. 2022d.  Quest: Graph transformer for quantum circuit reliability estimation. In _IEEE/ACM International Conference on Computer-Aided Design (ICCAD)_.
  * Wang et al. (2023a) Hanrui Wang, Pengyu Liu, Yilian Liu, Jiaqi Gu, Jonathan Baker, Frederic T. Chong, and Song Han. 2023a.  DGR: Tackling Drifted and Correlated Noise in Quantum Error Correction via Decoding Graph Re-weighting. In _arXiv preprint_.
  * Wang et al. (2023b) Hanrui Wang, Yilian Liu, Pengyu Liu, Jiaqi Gu, Zirui Li, Zhiding Liang, Jinglei Cheng, Yongshan Ding, Xuehai Qian, Yiyu Shi, David Z. Pan, Frederic T. Chong, and Song Han. 2023b.  RobustState: Boosting Fidelity of Quantum State Preparation via Noise-Aware Variational Training.  _arXiv preprint_ (2023).
  * Wang et al. (2021) Hanrui Wang, Zhekai Zhang, and Song Han. 2021.  Spatten: Efficient sparse attention architecture with cascade token and head pruning. In _2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA)_. IEEE, 97–110.
  * Wang* et al. (2020) Hanrui Wang*, Zhekai Zhang*, Song Han, and William J Dally. 2020.  Sparch: Efficient architecture for sparse matrix multiplication. In _2020 IEEE International Symposium on High Performance Computer Architecture (HPCA)_. IEEE, 261–274.
  * Wu and Zhong (2023) Yue Wu and Lin Zhong. 2023.  Fusion Blossom: Fast MWPM Decoders for QEC.  _arXiv preprint arXiv:2305.08307_ (2023).
  * Zheng et al. (2023) Han Zheng, Gokul Subramanian Ravi, Hanrui Wang, Kanav Setia, Frederic T Chong, and Junyu Liu. 2023.  SnCQA: A hardware-efficient equivariant quantum convolutional circuit architecture. In _2023 IEEE International Conference on Quantum Computing and Engineering (QCE)_.


# Optimizing Quantum Federated Learning Based on Federated Quantum Natural
Gradient Descent

###### Abstract

Quantum federated learning (QFL) is a quantum extension of the classical
federated learning model across multiple local quantum devices. An efficient
optimization algorithm is always expected to minimize the communication
overhead among different quantum participants. In this work, we propose an
efficient optimization algorithm, namely federated quantum natural gradient
descent (FQNGD), and further, apply it to a QFL framework that is composed of
a variational quantum circuit (VQC)-based quantum neural networks (QNN).
Compared with stochastic gradient descent methods like Adam and Adagrad, the
FQNGD algorithm admits much fewer training iterations for the QFL to get
converged. Moreover, it can significantly reduce the total communication
overhead among local quantum devices. Our experiments on a handwritten digit
classification dataset justify the effectiveness of the FQNGD for the QFL
framework in terms of a faster convergence rate on the training set and higher
accuracy on the test set.

Index Terms—  Quantum neural network, variational quantum circuit, quantum
federated learning, federated quantum natural gradient descent

##  1 Introduction

Deep learning (DL) technologies have been successfully applied in many machine
learning tasks such as speech recognition (ASR) [1], natural language
processing (NLP) [2], and computer vision [3]. The bedrock of DL applications
highly relies on the hardware breakthrough of the graphic processing unit
(GPU) and the availability of a large amount of training data [4, 5]. However,
the advantages of large-size DL models, such as GPT-3 [6] and BERT [7], are
faithfully attributed to the significantly powerful computing capabilities
that are only privileged to big companies equipped with numerous costly and
industrial-level GPUs. With the rapid development of noisy intermediate-scale
quantum (NISQ) devices [8, 9, 10], the quantum computing hardware is expected
to speed up the classical DL algorithms by creating novel quantum machine
learning (QML) approaches like quantum neural networks (QNN) [11, 12, 13, 14]
and quantum kernel learning (QKL) [14]. The VQC-based QNN seeks to
parameterize a distribution through some set of adjustable model parameters,
and the QKL methods utilize quantum computers to define a feature map that
projects classical data into the quantum Hilbert space. Both QML methods have
advantages and disadvantages in dealing with different machine learning tasks
and it could not be simply claimed which one is the most suitable choice.
However, two obstacles prevent the NISQ devices from applying to QML in
practice. The first challenge is that the classical DL models cannot be
deployed on NISQ devices without model conversion to quantum tensor formats
[15, 16]. For the second challenge, the NISQ devices admit a few physical
qubits such that insufficient qubits could be spared for the quantum error
correction [17, 9, 10]. More significantly, the representation power of QML is
quite limited to the small number of currently available qubits [18] and the
increase of qubits may lead to the problem of Barren Plateaus [19].

To deal with the first challenge, in this work, we introduce a variational
quantum algorithm, namely a variational quantum circuit (VQC), to enable QNN
to be simulated on the currently available NISQ devices. The VQC-based QNNs
have attained even exponential advantages over the DL counterparts on
exclusively many tasks like ASR [20, 21], NLP [22], and reinforcement learning
[23]. As for the second challenge, distributed QML systems, which consist of
local quantum machines, can be set up to enhance the quantum computing power.
One particular distributed QML architecture is called quantum federated
learning (QFL), which aims to build a decentralized computing model derived
from a classical FL [24].

Fig. 1: An illustration of quantum federated learning. The global VQC
parameter $\bar{\boldsymbol{\theta}}$ is first transmitted to local VQCs
$\boldsymbol{\theta}_{k}$. Then, the updated gradients
$\nabla\mathcal{L}(\boldsymbol{\theta}_{k})$ based on the participants’ local
data are sent back to the centralized server and then they are aggregated to
update the parameters of the global VQC.

Konečnỳ _et al._ [25] first proposed the FL strategies to improve the
communication efficiency of a distributed computing system, and McMahan _et
al._ [26] set up the FL systems with the concerns in the use of big data and a
large-scale cloud-based DL [27]. The FL framework depends on the advances in
hardware progress, making tiny DL systems practically powerful. For example,
an ASR system on the cloud can transmit a global acoustic model to a user’s
cell phone and then send the updated information back to the cloud without
collecting the user’s private data on the centralized computing server. As
shown in Figure 1, the QFL system is similar to a classical FL system and
differs from distributed learning in several ways as follows: (a) the datasets
in the framework of QFL are not necessarily balanced; (b) the data in QFL are
not assumed to be generated from an independent and identical (i.i.d.)
distribution.

Chen _et al._ [28] demonstrates the QFL architecture that is built upon the
classical FL paradigm, where the central node holds a global VQC and receives
the trained VQC parameters from participants’ local quantum devices.
Therefore, the QFL model, which inherits the advantages of the FL framework,
can unite tiny local quantum devices to generate a powerful global one. This
methodology helps to build a privacy-preserving QML system and leverages
quantum computing to further boost the computing power of the classical FL. As
shown in Figure 1, our proposed QFL and FL differ in the models utilized in
federated learning systems, where QFL employs VQC models instead of their
classical DL counterparts for FL. More specifically, the QFL comprises a
global VQC model deployed on the cloud, and there are $M$ local VQC models
assigned to users’ devices. The training process of QFL involves three key
procedures: (1) the parameters of global VQC model $\bar{\boldsymbol{\theta}}$
are transmitted to $K$ local participants’ devices; (2) each local VQC first
adaptively trains its own model based on the local users’ data, and then
separately sends the model gradients
$\nabla\mathcal{L}(\boldsymbol{\theta}_{k})$ back to the centralized platform;
(3) the uploaded gradients from local participants are averagely aggregated to
create a global gradient to update further the global model parameters
$\bar{\boldsymbol{\theta}}$.

Despite the advantages of QFL in practice, an inherent bottleneck of QFL is
the communication overhead among different VQC models, which bounds up with
the performance of QFL. To reduce the cost of communication overhead, we
expect a more efficient training algorithm to speed up the convergence rate
such that fewer counts of global model updates can be attained. Based on the
above analysis, in this work, we put forth a federated quantum learning
algorithm, namely federated quantum natural gradient descent (FQNGD), for the
training of QFL. The FQNGD algorithm, developed from the quantum natural
gradient descent (QNGD) algorithm, admits a more efficient training process
for a single VQC [29]. In particular, Stokes _et al._ [29] first claimed that
the Fubini-Study metric tensor could be employed for the QNGD. Besides,
compared with the work [28], the gradients of VQC are uploaded to a global
model rather than the VQC parameters of local devices such that the updated
gradients can be collected without being accessed to the VQC parameters as
shown in [28].

##  2 Variational Quantum Circuit

An illustration of VQC is shown in Figure 2, where the VQC model consists of
three components: (a) tensor product encoding (TPE); (b) parametric quantum
circuit (PQC); (c) measurement. The TPE initializes the input quantum states
$\lvert x_{1}\rangle$, $\lvert x_{2}\rangle$, …, $\lvert x_{U}\rangle$ from
the classical inputs $x_{1}$, $x_{2}$, …, $x_{U}$, and the PQC operator
transforms the quantum states $\lvert x_{1}\rangle$, $\lvert x_{2}\rangle$, …,
$\lvert x_{U}\rangle$ into the output quantum states $\lvert z_{1}\rangle$,
$\lvert z_{2}\rangle$, …, $\lvert z_{U}\rangle$. The outputs correspond to the
expected observations
$\langle{z}_{1}\rangle,\langle{z}_{2}\rangle,...,\langle{z}_{U}\rangle$ arised
from the measurement of the Pauli-Z operators. We present the three components
in detail next.

Fig. 2: The VQC is composed of three components: (a) TPE; (b) PQC; (c)
Measurement. The TPE utilizes a series of $R_{Y}(\frac{\pi}{2}x_{i})$ to
transform classical inputs into quantum states. The PQC consists of CNOT gates
and single-qubit rotation gates $R_{X}$, $R_{Y}$, $R_{Z}$ with trainable
parameters $\boldsymbol{\alpha}$, $\boldsymbol{\beta}$, and
$\boldsymbol{\gamma}$. The CNOT gates are non-parametric and impose the
property of quantum entanglement among qubits, and $R_{X}$, $R_{Y}$ and
$R_{Z}$ are parametric gates and can be adjustable during the training stage.
The PQC model in the green dash square is repeatably copied to build a deep
model. The measurement converts the quantum states
$|z_{1}\rangle,|z_{2}\rangle,...,|z_{U}\rangle$ into the corresponding
expectation values
$\langle{z}_{1}\rangle,\langle{z}_{2}\rangle,...,\langle{z}_{U}\rangle$. The
outputs
$\langle{z}_{1}\rangle,\langle{z}_{2}\rangle,...,\langle{z}_{U}\rangle$ is
connected to a loss function and the gradient descent algorithms can be used
to update the VQC model parameters. Besides, both CNOT gates and $R_{X}$,
$R_{Y}$ and $R_{Z}$ correspond to unitary matrices as shown below the VQC
framework.

The TPE model was first proposed in [30]. It aims to convert a classical
vector x into a quantum state $\lvert\textbf{x}\rangle$ by setting up a one-
to-one mapping as Eq. (1).

$\begin{split}|\textbf{x}\rangle&=\left(\otimes_{i=1}^{U}R_{Y}(\frac{\pi}{2}x_{i})\right)|0\rangle^{\otimes
U}\\\ &=\begin{bmatrix}\cos(\frac{\pi}{2}x_{1})\\\
\sin(\frac{\pi}{2}x_{1})\end{bmatrix}\otimes\begin{bmatrix}\cos(\frac{\pi}{2}x_{2})\\\
\sin(\frac{\pi}{2}x_{2})\end{bmatrix}\otimes\cdot\cdot\cdot\otimes\begin{bmatrix}\cos(\frac{\pi}{2}x_{U})\\\
\sin(\frac{\pi}{2}x_{U})\end{bmatrix},\end{split}$ (1)

where $R_{Y}(\cdot)$ refers to a single-qubit quantum gate rotated across
$Y$-axis and each $x_{i}$ is constrained to the domain of $[0,1]$, which
results in a reversely one-to-one conversion between x and
$|\textbf{x}\rangle$.

Moreover, the PQC is equipped with the CNOT gates for quantum entanglement and
learnable quantum gates, i.e., $R_{X}(\alpha_{i})$, $R_{Y}(\beta_{i})$, and
$R_{Z}(\gamma_{i})$, where the qubit angles $\alpha_{i}$, $\beta_{i}$, and
$\gamma_{i}$ are tuned in the training process. The PQC framework in the green
dash square is repeatedly copied to set up a deep model, and the number of the
PQC frameworks is called the depth of the VQC. The operation of the
measurement outputs the classical expected observations $|z_{1}\rangle$,
$|z_{2}\rangle$, …, $|z_{U}\rangle$ from the quantum output states. The
expected outcomes are used to calculate the loss value and the gradient
descents [31], which are used to update the VQC model parameters by applying
the back-propagation algorithm [32] based on the stochastic gradient descent
(SGD) optimizer.

##  3 Quantum Natural Gradient Descent

As shown in Eq. (2), at step $t$, the standard gradient descent minimizes a
loss function $\mathcal{L}(\boldsymbol{\theta})$ with respect to the
parameters $\boldsymbol{\theta}$ in a Euclidean space.

$\boldsymbol{\theta}_{t+1}=\boldsymbol{\theta}_{t}-\eta\nabla\mathcal{L}(\boldsymbol{\theta}_{t}),$
(2)

where $\eta$ is the learning rate.

The standard gradient descent algorithm conducts each optimization step in a
Euclidean geometry on the parameter space. However, since the form of
parameterization is not unique, different compositions of parameterizations
are likely to distort the distance geometry within the optimization landscape.
A better alternative method is to perform the gradient descent in the
distribution space, namely natural gradient descent [33], which is dimension-
free and invariant for different parameterization forms. Each optimization
step of the natural gradient descent chooses the optimum step size for the
update of parameter $\boldsymbol{\theta}_{t}$, regardless of the choice of
parameterization. Mathematically, the standard gradient descent is modified as
Eq. 3.

$\boldsymbol{\theta}_{t+1}=\boldsymbol{\theta}_{t}-\eta
F^{-1}\nabla\mathcal{L}(\boldsymbol{\theta}_{t}),$ (3)

$F$ denotes the Fisher information matrix, which acts as a metric tensor that
transforms the steepest gradient descent in the Euclidean parameter space to
the steepest descent in the distribution space.

Since the standard Euclidean geometry is sub-optimal for the optimization of
quantum variational algorithms, a quantum analog has the following form as Eq.
(4).

$\boldsymbol{\theta}_{t+1}=\boldsymbol{\theta}_{t}-\eta
g^{+}(\boldsymbol{\theta}_{t})\nabla\mathcal{L}(\boldsymbol{\theta}_{t}),$ (4)

where $g^{+}(\boldsymbol{\theta}_{t})$ refers to the pseudo-inverse and is
associated with the specific architecture of the quantum circuit. The
coefficient $g^{+}(\boldsymbol{\theta}_{t})$ can be calculated using the
Fubini-Study metric tensor, which it then reduces to the Fisher information
matrix in the classical limit [34].

##  4 Quantum Natural Gradient Descent for VQC

Before employing the QFNGD for a quantum federated learning system, we
concentrate on the use of QNGD for a single VQC. For simplicity, we leverage a
block-diagonal approximation to the Fubini-Study metric tensor for composing
QNGD into the VQC training on the NISQ quantum hardware.

We set an initial quantum state $\lvert\psi_{0}\rangle$ and a PQC with $L$
layers. For $l\in[L]$, we separately denote $\textbf{W}_{l}$ and
$\textbf{V}_{l}(\boldsymbol{\theta}_{l})$ as the unitary matrices associated
with non-parameterized quantum gates and parameterized quantum ones,
respectively.

Fig. 3: An illustration of unitary matrices associated with the non-parametric
and parametric gates. $\forall l\in[L]$, the matrices $\textbf{W}_{l}$
correspond to the non-parametric gates, the matrices
$\textbf{V}_{l}(\boldsymbol{\theta}_{l})$ are associated with the parametric
ones, and $\lvert\psi_{0}\rangle$ refers to the initial quantum state that is
derived from the operation of the TPE.

Let’s consider a variational quantum circuit as Eq. (5).

$U(\boldsymbol{\boldsymbol{\theta}})\lvert\psi_{0}\rangle=\textbf{V}_{L}(\boldsymbol{\theta}_{L})\textbf{W}_{L}\cdot\cdot\cdot\textbf{V}_{l}(\boldsymbol{\theta}_{l})\textbf{W}_{l}\cdot\cdot\cdot\textbf{V}_{1}(\boldsymbol{\theta}_{1})\textbf{W}_{1}\lvert\psi_{0}\rangle$
(5)

Furthermore, any unitary quantum parametric gates can be rewritten as
$\textbf{V}_{l}(\boldsymbol{\theta}_{l})=\exp(i\boldsymbol{\theta}_{l}H_{l})$,
where $H_{l}$ refers to the Hermitian generator of the gate $\textbf{V}_{L}$.
The approximation to the Fubini-Study metric tensor admits that for each
parametric layer $l$ in the variational quantum circuit, the $n_{l}\times
n_{l}$ block-diagonal submatrix of the Fubini-Study metric tensor
$g_{l,i,j}^{+}$ is calculated by Eq. (6).

$g_{l,i,j}^{+}=\langle\psi_{l}\lvert
H_{l}(i)H_{l}(j)\lvert\psi_{l}\rangle-\langle\psi_{l}\lvert
H_{l}(i)\lvert\psi_{l}\rangle\langle\psi_{l}|H_{l}(j)|\psi_{l}\rangle,$ (6)

where

$|\psi_{l}\rangle=\textbf{V}_{l}(\boldsymbol{\theta}_{l})\textbf{W}_{l}\cdot\cdot\cdot\textbf{V}_{1}(\boldsymbol{\theta}_{1})\textbf{W}_{1}|\psi_{0}\rangle.$
(7)

In Eq. (7)$,|\psi_{l}\rangle$ denotes the quantum state before the application
of the parameterized layer $l$. Figure 4 illustrates a simplified version of a
VQC, where $\textbf{W}_{1}$ and $\textbf{W}_{2}$ are related to non-parametric
gates, and $\textbf{V}_{1}(\theta_{0},\theta_{1})$ and
$\textbf{V}_{2}(\theta_{2},\theta_{3})$ correspond to the parametric gates
with adjustable parameters, respectively. Since there are two layers, each of
which owns two free parameters, the block-diagonal approximation is composed
of two $2\times 2$ matrices, $g^{+}_{1}$ and $g^{+}_{2}$, which can be
separately expressed as Eq. (8) and (9).

$g^{+}_{1}=\begin{bmatrix}\langle z_{0}^{2}\rangle-\langle
z_{0}\rangle^{2}&&\langle z_{0}z_{1}\rangle-\langle z_{0}\rangle\langle
z_{1}\rangle\\\ \langle z_{0}z_{1}\rangle-\langle z_{0}\rangle\langle
z_{1}\rangle&&\langle z_{1}^{2}\rangle-\langle z_{1}\rangle^{2}\end{bmatrix},$
(8)

and

$g^{+}_{2}=\begin{bmatrix}\langle y_{1}^{2}\rangle-\langle
y_{1}\rangle^{2}&&\langle y_{1}x_{2}\rangle-\langle y_{1}\rangle\langle
x_{2}\rangle\\\ \langle y_{1}x_{2}\rangle-\langle y_{1}\rangle\langle
x_{2}\rangle&&\langle x_{2}^{2}\rangle-\langle x_{2}\rangle^{2}\end{bmatrix}.$
(9)

The elements of $g_{1}^{+}$ and $g_{2}^{+}$ compose
$g^{+}(\boldsymbol{\theta})$ as Eq. (10).

$g^{+}(\boldsymbol{\theta})=\begin{bmatrix}g_{1}^{+}&0\\\
0&g_{2}^{+}\end{bmatrix}.$ (10)

Then, we employ Eq. (4) to update the VQC parameter $\boldsymbol{\theta}$.

Fig. 4: A demonstration of the VQC approximation method based on the Fubini-
Study metric tensor: (a) A block-diagonal approximation to VQC based on the
Fubini-Study metric tensor; (b) a measurement of $z_{0},z_{1}$ for
$|\psi_{0}\rangle$; (c) measurement of $y_{1},x_{2}$ for $|\psi_{1}\rangle$.

##  5 Federated Quantum Natural Gradient Descent

Algorithm 1 Iterative Approximation

1\. Given the dataset $S=S_{1}\cup S_{2}\cup\cdot\cdot\cdot\cup S_{K}$.
2\. Initialize global parameter $\bar{\boldsymbol{\theta}}$ and broadcast it
to participants $\boldsymbol{\theta}_{0}^{(k)}$.
3\. Assign each participant with the subset $S_{k}$.
4\. For each global model update at epoch $t=[T]$ do
5\. For each participant $k\in[K]$ in parallel do
6\. Attain
$g^{+}(\boldsymbol{\theta}^{(k)}_{t};S_{k})\nabla\mathcal{L}(\boldsymbol{\theta}_{t}^{(k)})$
for the $k^{th}$ VQC.
7\. Send gradient
$g^{+}(\boldsymbol{\theta}^{(k)}_{t})\nabla\mathcal{L}(\boldsymbol{\theta}_{t}^{(k)};S_{k})$
to the coordinator.
8\. End for
9\. The coordinator aggregates the received gradients.
10\. The coordinator updates the global model by Eq. (13).
11\. Broadcast the updated global $\bar{\boldsymbol{\theta}}_{t+1}$ to all
participants.
12\. End for

A QFL system can be built by setting up VQC models in an FL manner, given the
dataset $S$ composed of subsets $S_{1},S_{2},...,S_{K}$, the objective of QFL
can be formulated as:

$\min\limits_{\boldsymbol{\theta}}\sum\limits_{k=1}^{K}w_{k}g^{+}_{k}(\boldsymbol{\theta})\mathcal{L}(\boldsymbol{\theta};S_{k}),$
(11)

where $w_{k}$ refers to the coefficient assigned to the $k$-th gradient
participant, and each $w_{k}$ can be estimated as:

$w_{k}=\frac{|S_{k}|}{|S|}=\frac{|S_{k}|}{\sum_{k=1}^{K}|S_{k}|}.$ (12)

The QNGD algorithm is applied for each VQC and the uploaded gradients of all
VQCs are aggregated to update the model parameters of the global VQC. The
FQNGD can be mathematically summarized as:

$\bar{\boldsymbol{\theta}}_{t+1}=\bar{\boldsymbol{\theta}}_{t}-\eta\sum\limits_{k=1}^{K}\frac{|S_{k}|}{|S|}g_{k}^{+}(\boldsymbol{\theta}^{(k)}_{t})\nabla\mathcal{L}(\boldsymbol{\theta}^{(k)}_{t};S_{k}),$
(13)

where $\bar{\boldsymbol{\theta}}_{t}$ and $\boldsymbol{\theta}_{t}^{(k)}$
separately correspond to the model parameters of the global VQC and the $k$-th
VQC model at epoch $t$, and $N_{k}$ represents the amount of training data
stored in the participant $k$, and the sum of $K$ participants’ data is
equivalent to $N$.

Compared with the SGD counterparts used for QFL, the FQNGD algorithm admits
adaptive learning rates for the gradients such that the convergence rate could
be accelerated according to the VQC model status.

###  5.0.1 Empirical Results

To demonstrate the FQNGD algorithm for QFL, we perform the binary and ternary
classification tasks on the standard MNIST dataset [35], with digits
$\\{2,5\\}$ for the binary task and $\\{1,3,7\\}$ for the ternary one. There
are $11379$ training data and $1924$ test data for the binary classification,
and $19138$ training data and $3173$ test data are assigned for the ternary
classification. As for the setup of QFL in our experiments, the QFL system
consists of $6$ identically local VQC participants, each of which owns the
same amount of training data. The test data are stored in the global part and
are used to evaluate the classification performance.

Fig. 5: Simulation results of binary and ternary classifications on the training set of the MNIST database. (a) The learning curves of various optimization methods for the binary classification; (b) the learning curves of various optimization methods for the ternary classification. Table 1: The simulation results of a binary classification. Methods | Vanilla SGD | Adagrad | Adam | FQNGD
---|---|---|---|---
Accuracy | 98.48 | 98.81 | 98.87 | 99.32
Table 2: The simulation results of a ternary classification. Methods | Vanilla SGD | Adagrad | Adam | FQNGD
---|---|---|---|---
Accuracy | 97.86 | 98.63 | 98.71 | 99.12

We compare our proposed FQNGD algorithm with other three optimizers: the naive
SGD optimizer, the Adagrad optimizer [36], and the Adam optimizer [37]. The
Adagrad optimizer is a gradient descent optimizer with a past-gradient-
dependent learning rate in each dimension. The Adam optimizer refers to the
gradient descent method with an adaptive learning rate as well as adaptive
first and second moments.

As shown in Figure 5, our simulation results suggest that our proposed FQNGD
method is capable of achieving the fastest convergence rate among the
optimization approaches. It means that the FQNGD method can reduce the
communication overhead cost and maintain the baseline performance of binary
and ternary classifications on the MNIST dataset. Moreover, we evaluate the
QFL performance in terms of classification accuracy. The FQNGD method
outperforms the other counterparts with the highest accuracy values. In
particular, the FQNGD is designed for the VQC model and can attain better
empirical results than the Adam and Adagrad methods with adaptive learning
rates over epochs.

##  6 Conclusion and Future Work

This work focuses on the design of the FQNGD algorithm for the QFL system in
which multiple local VQC models are applied. The FQNGD is derived from
training a single VQC based on QNGD, which relies on the block-diagonal
approximation of the Fubini-Study metric tensor to the VQC architecture. We
put forth the FQNGD method to train the QFL system. Compared with other SGD
methods such as Adagrad and Adam optimizers, our experiments of the
classification tasks on the MNIST dataset demonstrate that the FQNGD method
attains better empirical results than other SGD methods, while the FQNGD
exhibits a faster convergence rate than the others, which implies that our
FQNGD method suggests that it is capable of reducing the communication cost
and can maintain the baseline empirical results.

Although this work focuses on the optimization methods for the QFL system, the
decentralized deployment of a high-performance QFL system for adapting to the
large-scale dataset is left for our future investigation. In particular, it is
essential to consider how to defend against malicious attacks from adversaries
and also boost the robustness and integrity of the shared information among
local participants. Besides, the deployment of other quantum neural networks
like quantum convolutional neural networks (QCNN) [38] are worth further
attempts to compose a QFL system.

## References

  * [1] Xuedong Huang, James Baker, and Raj Reddy,  “A Historical Perspective of Speech Recognition,”  Communications of the ACM, vol. 57, no. 1, pp. 94–103, 2014.
  * [2] Julia Hirschberg and Christopher D Manning,  “Advances in Natural Language Processing,”  Science, vol. 349, no. 6245, pp. 261–266, 2015.
  * [3] Athanasios Voulodimos, Nikolaos Doulamis, Anastasios Doulamis, and Eftychios Protopapadakis,  “Deep Learning for Computer Vision: A Brief Review,”  Computational Intelligence and Neuroscience, vol. 2018, 2018.
  * [4] Jun Qi, Jun Du, Sabato Marco Siniscalchi, Xiaoli Ma, and Chin-Hui Lee,  “Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network-based Vector-to-Vector Regression,”  IEEE Transactions on Signal Processing, vol. 68, pp. 3411–3422, 2020.
  * [5] Jun Qi, Jun Du, Sabato Marco Siniscalchi, and Chin-Hui Lee,  “A Theory on Deep Neural Network Based Vector-to-Vector Regression with An Illustration of Its Expressive Power in Speech Enhancement,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 12, pp. 1932–1943, 2019.
  * [6] Tom Brown and et al.,  “Language Models Are Few-Shot Learners,”  in Proc. Advances in Neural Information Processing Systems, 2020, vol. 33, pp. 1877–1901.
  * [7] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova,  “Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding,”  arXiv preprint arXiv:1810.04805, 2018.
  * [8] John Preskill,  “Quantum Computing in the NISQ Era and Beyond,”  Quantum, vol. 2, pp. 79, August 2018.
  * [9] Laird Egan and et al.,  “Fault-Tolerant Control of An Error-Corrected Qubit,”  Nature, vol. 598, no. 7880, pp. 281–286, 2021.
  * [10] Qihao Guo, Yuan-Yuan Zhao, Markus Grassl, Xinfang Nie, Guo-Yong Xiang, Tao Xin, Zhang-Qi Yin, and Bei Zeng,  “Testing A Quantum Error-Correcting Code on Various Platforms,”  Science Bulletin, vol. 66, no. 1, pp. 29–35, 2021.
  * [11] Marco Cerezo and et al.,  “Variational Quantum Algorithms,”  Nature Reviews Physics, vol. 3, no. 9, pp. 625–644, 2021.
  * [12] Hsin-Yuan Huang, Michael Broughton, Masoud Mohseni, Ryan Babbush, Sergio Boixo, Hartmut Neven, and Jarrod R. McClean,  “Power of Data in Quantum Machine Learning,”  Nature Communications, vol. 12, no. 1, pp. 1–9, 2021.
  * [13] Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Shan You, and Dacheng Tao,  “Learnability of Quantum Neural Networks,”  PRX Quantum, vol. 2, no. 4, pp. 040337, 2021.
  * [14] Hsin-Yuan Huang and et al.,  “Quantum Advantage in Learning from Experiments,”  Science, vol. 376, no. 6598, pp. 1182–1186, 2022.
  * [15] Jun Qi, Chao-Han Huck Yang, and Pin-Yu Chen,  “QTN-VQC: An End-to-End Learning Framework for Quantum Neural Networks,”  in NeurIPS 2021 Workshop on Quantum Tensor Networks in Machine Learning, 2022.
  * [16] Samuel Yen-Chi Chen, Chih-Min Huang, Chia-Wei Hsing, and Ying-Jer Kao,  “An End-to-End Trainable Hybrid Classical-Quantum Classifier,”  Machine Learning: Science and Technology, vol. 2, no. 4, pp. 045021, 2021.
  * [17] Philip Ball,  “Real-Time Error Correction for Quantum Computing,”  Physics, vol. 14, pp. 184, 2021.
  * [18] Jun Qi, Chao-Han Huck Yang, Pin-Yu Chen, and Min-Hsiu Hsieh,  “Theoretical error performance analysis for variational quantum circuit based functional regression,”  npj Quantum Information, vol. 9, no. 1, pp. 4, 2023.
  * [19] Jarrod R McClean, Sergio Boixo, Vadim N Smelyanskiy, Ryan Babbush, and Hartmut Neven,  “Barren plateaus in quantum neural network training landscapes,”  Nature communications, vol. 9, no. 1, pp. 4812, 2018.
  * [20] Chao-Han Huck Yang, Jun Qi, Samuel Yen-Chi Chen, Pin-Yu Chen, Sabato Marco Siniscalchi, Xiaoli Ma, and Chin-Hui Lee,  “Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition,”  in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, 2021, pp. 6523–6527.
  * [21] Jun Qi and Javier Tejedor,  “Classical-to-Quantum Transfer Learning for Spoken Command Recognition Based on Quantum Neural Networks,”  in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, 2021.
  * [22] Chao-Han Huck Yang, Jun Qi, Samuel Yen-Chi Chen, Yu Tsao, and Pin-Yu Chen,  “When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing,”  in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, 2022.
  * [23] Samuel Yen-Chi Chen, Chao-Han Huck Yang, Jun Qi, Pin-Yu Chen, Xiaoli Ma, and Hsi-Sheng Goan,  “Variational Quantum Circuits for Deep Reinforcement Learning,”  IEEE Access, vol. 8, pp. 141007–141024, 2020.
  * [24] Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith,  “Federated learning: Challenges, methods, and future directions,”  IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020\.
  * [25] Jakub Konečnỳ, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon,  “Federated Learning: Strategies for Improving Communication Efficiency,”  arXiv preprint arXiv:1610.05492, 2016.
  * [26] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas,  “Communication-Efficient Learning of Deep Networks from Decentralized Data,”  in Proc. Artificial Intelligence and Statistics, 2017, pp. 1273–1282.
  * [27] Reza Shokri and Vitaly Shmatikov,  “Privacy-Preserving Deep Learning,”  in Proc. ACM SIGSAC Conference on Computer and Communications Security, 2015, pp. 1310–1321.
  * [28] Samuel Yen-Chi Chen and Shinjae Yoo,  “Federated Quantum Machine Learning,”  Entropy, vol. 23, no. 4, pp. 460, 2021.
  * [29] James Stokes, Josh Izaac, Nathan Killoran, and Giuseppe Carleo,  “Quantum Natural Gradient,”  Quantum, vol. 4, pp. 269, 2020.
  * [30] Edwin Stoudenmire and David J Schwab,  “Supervised Learning with Tensor Networks,”  in Proc. Advances in Neural Information Processing Systems, 2016, vol. 29.
  * [31] Sebastian Ruder,  “An Overview of Gradient Descent Optimization Algorithms,”  arXiv preprint arXiv:1609.04747, 2016.
  * [32] Paul J Werbos,  “Backpropagation Through Time: What It Does and How to Do It?,”  Proceedings of the IEEE, vol. 78, no. 10, pp. 1550–1560, 1990.
  * [33] Shun-Ichi Amari,  “Natural Gradient Works Efficiently in Learning,”  Neural Computation, vol. 10, no. 2, pp. 251–276, 1998.
  * [34] Sam McArdle, Tyson Jones, Suguru Endo, Ying Li, Simon C Benjamin, and Xiao Yuan,  “Variational Ansatz-Based Quantum Simulation of Imaginary Time Evolution,”  NPJ Quantum Information, vol. 5, no. 1, pp. 1–6, 2019.
  * [35] Li Deng,  “The MNIST Database of Handwritten Digit Images for Machine Learning Research,”  IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141–142, 2012\.
  * [36] Agnes Lydia and Sagayaraj Francis,  “Adagrad—An Optimizer for Stochastic Gradient Descent,”  Int. J. Inf. Comput. Sci, vol. 6, no. 5, pp. 566–568, 2019.
  * [37] Diederik P Kingma and Jimmy Ba,  “Adam: A Method for Stochastic Optimization,”  in Proc. International Conference on Representation Learning, 2015\.
  * [38] Iris Cong, Soonwon Choi, and Mikhail D Lukin,  “Quantum Convolutional Neural Networks,”  Nature Physics, vol. 15, no. 12, pp. 1273–1278, 2019.