# 3D WaveUNet: 3D Wavelet Integrated Encoder-Decoder Network for Neuron
Segmentation

Qiufu Li, Linlin Shen* This work is supported by the National Natural Science
Foundation of China (Grant No. 62006156, 91959108, and U1713214) and the
Science and Technology Project of Guangdong Province (Grant No.
2018A050501014). Corresponding author: Linlin Shen.Q. Li and L. Shen are with
the Computer Vision Institute, College of Computer Science and Software
Engineering, Shen zhen University, Shenzhen 518060, China, AI Research Center
for Medical Image Analysis and Diagnosis, Shenzhen University, Shenzhen
518060, China, and Guangdong Key Laboratory of Intelligent Information
Processing, Shenzhen University, Shenzhen 518060, China (e-mail:
<EMAIL_ADDRESS>llshen@szu.edu.cn).

###### Abstract

3D neuron segmentation is a key step for the neuron digital reconstruction,
which is essential for exploring brain circuits and understanding brain
functions. However, the fine line-shaped nerve fibers of neuron could spread
in a large region, which brings great computational cost to the segmentation
in 3D neuronal images. Meanwhile, the strong noises and disconnected nerve
fibers in the image bring great challenges to the task. In this paper, we
propose a 3D wavelet and deep learning based 3D neuron segmentation method.
The neuronal image is first partitioned into neuronal cubes to simplify the
segmentation task. Then, we design 3D WaveUNet, the first 3D wavelet
integrated encoder-decoder network, to segment the nerve fibers in the cubes;
the wavelets could assist the deep networks in suppressing data noise and
connecting the broken fibers. We also produce a _Neu_ ronal _Cu_ be _Da_ taset
(NeuCuDa) using the biggest available annotated neuronal image dataset,
BigNeuron, to train 3D WaveUNet. Finally, the nerve fibers segmented in cubes
are assembled to generate the complete neuron, which is digitally
reconstructed using an available automatic tracing algorithm. The experimental
results show that our neuron segmentation method could completely extract the
target neuron in noisy neuronal images. The integrated 3D wavelets can
efficiently improve the performance of 3D neuron segmentation and
reconstruction. The code and pre-trained models for this work will be
available at https://github.com/LiQiufu/3D-WaveUNet.

###### Index Terms:

Neuron segmentation, noise, line-shape object, wavelet, deep network.

##  I Introduction

Neuron reconstruction aims to establish a digital model of the neuron
morphology structure by tracing nerve fibers in neuronal image, which is of
great significance to explore the brain microstructure and understand brain
functions. 3D microscopic optical imaging technology, such as Micro-Optical
Sectioning Tomography (MOST) [1], has established a foundation for the brain
neuron reconstruction. Tens of automatic or semi-automatic tracing algorithms
[2, 3, 4] have been developed to reconstruct the neurons in optical images.
However, the low signal-to-noise ratio and disconnected nerve fibers are
usually big challenges to these algorithms, as Fig. 1 shows.

A natural way to improve the performance of existing neuron tracing algorithms
is to extract the complete neuron from the noisy image first and then
automatically reconstruct it. Deep learning, which achieves excellent results
in many fields, has also been applied into 3D neuron segmentation [5, 6, 7, 8,
9]. As nerve fibers can spread in a very large brain region, a large neuronal
image has to be processed for segmenting the complete neuron, which leads to
great computational cost for the deep networks based 3D neuron segmentation.
In addition, the recent researches [10, 11, 12] show that the sampling
operations used in the deep networks result in aliasing effects, which lead to
noise propagation and break the basic object structures. Due to the amount of
noises available in the neuronal image and the neuron’s fine line-shaped
structure, 3D neuron segmentation is more sensitive to these aliasing effects.
Therefore, the current 3D deep networks based segmentation methods might not
work well for the 3D neurons with fine line-shaped structure in the noisy
neuronal images.

Figure 1: (a) A noisy neuronal image with size of $291\times 3298\times 1881$
(z-y-x), containing a complete neuron. (b) The manual reconstruction of the
neuron. (c) An automatic reconstruction which incorrectly takes background
noises as nerve fibers. (d) Another automatic reconstruction which overlooks
the weak nerve fibers.

In this paper, to completely segment the neuron and efficiently improve its
reconstruction, we present a 3D wavelet and deep learning based 3D neuron
segmentation method. Firstly, to simplify the task, we partition the neuronal
image into small cubes. Then, we design a 3D wavelet integrated encoder-
decoder network (3D WaveUNet) to segment the nerve fibers in the cubes. As far
as we know, 3D WaveUNet is the first 3D wavelet integrated deep network. In 3D
WaveUNet, 3D Discrete Wavelet Transform (3D DWT) and 3D Inverse DWT (3D IDWT)
are applied to down-sample and up-sample 3D neuronal data. While 3D DWT help
the encoders in maintaining the fine structure of neurons and suppressing the
noise propagation, 3D IDWT could recover the neuron details in the decoders.
In addition, based on the biggest available annotated neuronal image dataset,
BigNeuron [13], we produce a _Neu_ ronal _Cu_ be _Da_ taset (NeuCuDa) to train
and evaluate the 3D WaveUNet. Finally, we assemble the nerve fibers segmented
in cubes according to the cube locations to get the complete segmented neuron,
and reconstruct it using the automatic tracing algorithm, All-Path Pruning 2.0
(APP2) [2]. In summary:

  1. 1.

We propose a 3D wavelet and deep learning based 3D neuron segmentation method,
to segment neurons in large size 3D neuronal images.

  2. 2.

We rewrite 3D DWT/IDWT as general network layers, and design 3D WaveUNet by
applying 3D DWT and IDWT as the sampling operations in 3D encoder-decoder
network, which is the first 3D wavelet integrated deep network.

  3. 3.

We produce a neuronal cube dataset, NeuCuDa, to train and evaluate the 3D
WaveUNet. Experimental results show that 3D WaveUNet efficiently improve the
performance of 3D neuron segmentation and reconstruction in the noisy images.

##  II Related works

###  II-A Neuron reconstruction

Neuron reconstruction aims to trace the nerve fibers in the optical neuronal
images and build a digital model of the neuron’s morphology structure.

The recent brain imaging technologies, including Micro-Optical Sectioning
Tomography (MOST) [1], fluorescence MOST (fMOST) [14], CLARITY [15, 16],
Magnified Analysis of the Proteome (MAP) [17], and Stabilization under Harsh
conditions via Intramolecular Epoxide Linkages to prevent Degradation (SHIELD)
[18], etc., establish the foundation for neuron reconstruction of animal
brain.

Various 3D data visualization software, such as Vaa3D [19], NeuroBlocks [20],
ManSegTool [21], TeraVR [22], etc., have been developed to show the complex
arborization fibers in neuronal images and assist the experienced technicians
in tracing neurons. BigNeuron project [13], initiated by Peng et al., releases
hundreds of optical neuronal images and their high precision reconstructions
traced by experienced experts. More than $1\times 10^{5}$ digitally
reconstructed neurons are released on NeuroMorpho.Org [23], while the
corresponding neuronal images are not available.

It is impossible to manually reconstruct the millions of neurons in the whole
brain image, so various automatic neuron tracing algorithms have been
designed. The BigNeuron project [13] collects the tens of these algorithms,
such as SnakeTracing [24], All-Path-Pruning (APP) [25], APP2 [2], NeuroGPSTree
[4], Rivulet [26], Rivulet2 [3], etc., and integrates them into the Vaa3D
software [19]. Generally, these algorithms mathematically model neuron
morphology and trace nerve fibers based on graph theory. However, they would
either classify background noise as nerve fibers or miss lots of nerve fibers,
as Fig. 1 shows. In other words, while the tracing algorithms are sensitive to
image quality, the existing neuronal imaging technologies are usually
associated with low signal-to-noise ratio.

###  II-B Neuron segmentation

Figure 2: The pipeline of our 3D neuron segmentation method. _A_ : Partition.
_B_ : Segmentation. _C_ : Assembling. _D_ : Reconstruction.

Segmenting the neuron before reconstruction is an effective approach to
improve the performance of neuron tracing algorithms. Inspired by the superior
performance in computer vision, pattern recognition, and natural language
processing, etc., deep learning has also been introduced into the 3D neuron
segmentation [5, 6, 27, 7, 8, 9]. The first 3D residual deep network for
neuron segmentation is proposed in [5]. The authors improve their method by
applying multi-scale kernels and 3D U-Net [27]. Because they take as input the
original neuronal image, these deep networks require high computational cost.
In [6], the authors design 3D U-Net Plus to separate the target neuron from
its surrounding nerve fibers. As 3D U-Net Plus is trained and evaluated on the
neuronal images with size of $32\times 128\times 128$, the small nerve fibers
may disappear during the image down-sampling. The above deep network based
neuron segmentation methods are not suitable for neurons with long-distance
fibers. A deep network based neuron segmentation method with low computational
cost [7] is built on ray-shooting model and dual channel bidirectional Long
Short-Term Memory (LSTM). The authors extract nerve fibers from the 2D
neuronal image slice-by-slice, which does not utilize the relevance between
the nerve data in the neighbouring 2D neuronal slices. To utilize the
unlabeled 3D neuronal data, self-supervised and weakly supervised 3D neurons
segmentation methods are studied [8, 9]. In all of these works, the BigNeuron
[13] neuronal images are taken to test their methods.

###  II-C The aliasing effects in deep networks

The recent studies [10, 11, 12] reveal the aliasing effects in the deep
learning, which are introduced by the commonly used sampling operations in the
deep networks. The aliasing effects result in noise accumulation and break the
basic object structure in deep networks, which are very harmful to the 3D
neuron segmentation in noisy images.

To suppress the aliasing among the low-frequency and high-frequency
components, Richard [11] designs Anti-aliased CNNs by applying low-pass
filters before the down-sampling in common Convolutional Neural Networks
(CNNs). Zou et al [12] propose an adaptive content-aware low-pass filtering
layer to adapt the varying frequencies in different locations and feature
channels. These works only consider the low-frequency component of input data,
which cannot extract and utilize the data details represented by the high-
frequency components. In [10], the authors integrate discrete wavelet
transforms into deep networks to separate the components of data in different
frequency intervals. They illustrate the usefulness of wavelet transforms in
assisting deep networks in extracting robust features, keeping basic object
structure, and recovering data details. However, the authors only illustrate
the application of 2D wavelet integrated deep networks for image
classification, and do not analyze or design 3D wavelet integrated deep
networks. In this paper, we will rewrite 3D DWT/IDWT as general network
layers, and design 3D wavelet integrated encoder-decoder networks, to improve
the 3D neuron segmentation and reconstruction in the noisy images.

##  III The method pipeline

The pipeline of our 3D neuron segmentation method is illustrated in Fig. 2 and
summarized as below:

  1. 1)

_Partition_. The neurons with long-distance nerve fibers could spread in a
large brain region, which result in high computational cost for the 3D neuron
segmentation in the images with large size. Therefore, the neuronal image is
partitioned into small cubes to simplify the segmentation task. We set the
z-y-x-size (depth-height-width) of the cubes as $32\times 128\times 128$
111The typical voxel resolution is $1\mu\text{m}\times 0.35\mu\text{m}\times
0.35\mu\text{m}$ for brain imaging technologies, such as MOST., which is a
compromise between the connectivity of nerve fibers in the neuronal cubes and
the computational complexity of the following segmentation.

  2. 2)

_Segmentation_. Using the well-trained 3D WaveUNet, we segment the nerve
fibers in the cubes. The 3D WaveUNet is an encoder-decoder network integrated
with 3D wavelet, which are trained on the neuronal cube dataset, NeuCuDa. The
network architecture and NeuCuDa are described in Sec. IV and Sec. 7,
respectively.

  3. 3)

_Assembling_. According to the cube locations in the neuronal image, we
assemble the segmented nerve fibers to complete the 3D neuron segmentation.

  4. 4)

_Reconstruction_. Based on segmented neuronal image, we reconstruct the neuron
using APP2 [2] algorithm. Considering its low computational complexity, we
choose APP2 as the benchmark algorithm in this paper.

##  IV 3D WaveUNet

The aliasing effects could significantly affect the performance of deep
network based 3D neuron segmentation, due to the fine line-shaped structure of
nerve fibers and interference of noises. To suppress the aliasing effects in
the 3D deep networks, we design 3D wavelet integrated encoder-decoder network
(3D WaveUNet), the first 3D wavelet integrated deep network, for 3D neuron
segmentation.

###  IV-A 3D DWT/IDWT layers

Wavelets [28] are powerful analysis tools used in the fields of data denoising
[29, 30], image compression [31, 32, 33], etc., and 2D wavelets have been
introduced into deep learning [34, 35, 36, 37, 38, 39, 40, 10]. We here
introduce how to integrate 3D wavelets into deep networks.

We rewrite 3D Discrete Wavelet Transform (DWT) and Inverse DWT (IDWT) as
general network layers. For a given 3D neuronal data
$\textbf{\emph{X}}\in\mathbb{R}^{d\times m\times n}$, and eight filters of a
3D discrete wavelet, i.e., one low-pass filter $\text{f}_{lll}$ and seven
high-pass filters
$\text{f}_{llh},\text{f}_{lhl},\text{f}_{lhh},\text{f}_{hll},\text{f}_{hlh},\text{f}_{hhl},\text{f}_{hhh}$,
3D DWT decomposes the data into one low-frequency component
$\textbf{\emph{X}}_{lll}$ and seven high-frequency components
$\textbf{\emph{X}}_{llh},\textbf{\emph{X}}_{lhl},\textbf{\emph{X}}_{lhh},\textbf{\emph{X}}_{hll},\textbf{\emph{X}}_{hlh},\textbf{\emph{X}}_{hhl},\textbf{\emph{X}}_{hhh}$,
where

$\displaystyle\textbf{\emph{X}}_{c_{0}c_{1}c_{2}}=(\downarrow
2)(\text{f}_{c_{0}c_{1}c_{2}}\ast\textbf{\emph{X}}),\quad
c_{0},c_{1},c_{2}\in\\{l,h\\}$ (1)

and $\ast,(\downarrow 2)$ denote the 3D convolution and naive down-sampling,
respectively. For a tensor
$\textbf{\emph{X}}=\\{X_{i,j,k}\\}\in\mathbb{R}^{d\times m\times n}$,

$(\downarrow 2)\textbf{\emph{X}}=\\{((\downarrow
2)\textbf{\emph{X}})_{i,j,k}\\}\in\mathbb{R}^{\lfloor\frac{d}{2}\rfloor\times\lfloor\frac{m}{2}\rfloor\times\lfloor\frac{n}{2}\rfloor},$
(2)

and

$((\downarrow 2)\textbf{\emph{X}})_{i,j,k}=X_{2i,2j,2k}.$ (3)

In theory, the size of every component is $1/2$ size of X in every dimension,
i.e.,

$\displaystyle\textbf{\emph{X}}_{c_{0}c_{1}c_{2}}\in\mathbb{R}^{\lfloor\frac{d}{2}\rfloor\times\lfloor\frac{m}{2}\rfloor\times\lfloor\frac{n}{2}\rfloor},\quad
c_{0},c_{1},c_{2}\in\\{l,h\\}.$ (4)

Therefore, $d,m,n$ are usually even numbers.

Figure 3: The comparison of 3D max-pooling/max-unpooling and 3D DWT/IDWT. Max-
pooling would magnify the noise in the neuronal data, while noise in the low-
frequency component $\emph{{X}}_{lll}$ is suppressed by 3D DWT, as the three
enlarged regions show. When max-unpooling recovers the cube resolution, it
normally breaks the nerve fibers. In contrast, IDWT could completely recover
the neuronal data using the components of DWT decomposition.

Using the dual filters
$\tilde{\text{f}}_{c_{0}c_{1}c_{2}},~{}c_{0},c_{1},c_{2}\in\\{l,h\\}$, 3D IDWT
reconstructs the original data X based on the eight components,

X
$\displaystyle=\sum_{c_{0},c_{1},c_{2}\in\\{l,h\\}}\tilde{\text{f}}_{c_{0}c_{1}c_{2}}\ast(\uparrow
2)\textbf{\emph{X}}_{c_{0}c_{1}c_{2}},$ (5)

where $(\uparrow 2)$ denotes the naive up-sampling operation. For a 3D tensor
$\textbf{\emph{X}}=\\{X_{i,j,k}\\}\in\mathbb{R}^{d\times m\times n}$,

$(\uparrow 2)\textbf{\emph{X}}=\\{((\uparrow
2)\textbf{\emph{X}})_{i,j,k}\\}\in\mathbb{R}^{2d\times 2m\times 2n},$ (6)

and

$((\uparrow
2)\textbf{\emph{X}})_{i,j,k}=\left\\{\begin{array}[]{cc}X_{\frac{i}{2},\frac{j}{2},\frac{k}{2}}&\text{if}~{}\frac{i}{2},\frac{j}{2},\frac{k}{2}\in\mathbb{Z},\\\
0&\text{else}.\end{array}\right.$ (7)

The dual filters of orthogonal 3D wavelet are the same with the original
filters,

$\displaystyle\tilde{\text{f}}_{c_{0}c_{1}c_{2}}=\text{f}_{c_{0}c_{1}c_{2}},~{}c_{0},c_{1},c_{2}\in\\{l,h\\}.$
(8)

Generally, the 3D wavelet filters are tensor products of the two filters of 1D
wavelet, $\text{f}_{l},\text{f}_{h}$, i.e.,

$\displaystyle\text{f}_{c_{0}c_{1}c_{2}}$
$\displaystyle=\text{f}_{c_{0}}\otimes\text{f}_{c_{1}}\otimes\text{f}_{c_{2}},\quad
c_{0},c_{1},c_{2}\in\\{l,h\\},$ (9)

where $\otimes$ represents the tensor product. Take Haar wavelet for example,
the low-pass and high-pass filters of 1D Haar wavelet are

$\displaystyle\text{f}^{\text{H}}_{l}=\frac{1}{\sqrt{2}}(1,1)^{T},\quad\text{f}^{\text{H}}_{h}=\frac{1}{\sqrt{2}}(1,-1)^{T}.$
(10)

Then, via Eq. (9), we get the filters of the corresponding 3D Haar wavelet:

$\displaystyle\text{f}_{lll}^{\text{H}}$
$\displaystyle=\frac{1}{2\sqrt{2}}\left[\left[\begin{array}[]{cc}1&1\\\
1&1\end{array}\right];\left[\begin{array}[]{cc}1&1\\\
1&1\end{array}\right]\right],$ (15) $\displaystyle\text{f}_{llh}^{\text{H}}$
$\displaystyle=\frac{1}{2\sqrt{2}}\left[\left[\begin{array}[]{cc}1&-1\\\
1&-1\end{array}\right];\left[\begin{array}[]{cc}1&-1\\\
1&-1\end{array}\right]\right],$ (20) $\displaystyle\text{f}_{lhl}^{\text{H}}$
$\displaystyle=\frac{1}{2\sqrt{2}}\left[\left[\begin{array}[]{cc}1&1\\\
-1&-1\end{array}\right];\left[\begin{array}[]{cc}1&1\\\
-1&-1\end{array}\right]\right],$ (25) $\displaystyle\text{f}_{lhh}^{\text{H}}$
$\displaystyle=\frac{1}{2\sqrt{2}}\left[\left[\begin{array}[]{cc}1&-1\\\
-1&1\end{array}\right];\left[\begin{array}[]{cc}1&-1\\\
-1&1\end{array}\right]\right],$ (30) $\displaystyle\text{f}_{hll}^{\text{H}}$
$\displaystyle=\frac{1}{2\sqrt{2}}\left[\left[\begin{array}[]{cc}1&1\\\
1&1\end{array}\right];\left[\begin{array}[]{cc}-1&-1\\\
-1&-1\end{array}\right]\right],$ (35) $\displaystyle\text{f}_{hlh}^{\text{H}}$
$\displaystyle=\frac{1}{2\sqrt{2}}\left[\left[\begin{array}[]{cc}1&-1\\\
1&-1\end{array}\right];\left[\begin{array}[]{cc}-1&1\\\
-1&1\end{array}\right]\right],$ (40) $\displaystyle\text{f}_{hhl}^{\text{H}}$
$\displaystyle=\frac{1}{2\sqrt{2}}\left[\left[\begin{array}[]{cc}1&1\\\
-1&-1\end{array}\right];\left[\begin{array}[]{cc}-1&-1\\\
1&1\end{array}\right]\right],$ (45) $\displaystyle\text{f}_{hhh}^{\text{H}}$
$\displaystyle=\frac{1}{2\sqrt{2}}\left[\left[\begin{array}[]{cc}1&-1\\\
-1&1\end{array}\right];\left[\begin{array}[]{cc}-1&1\\\
1&-1\end{array}\right]\right].$ (50)

Eqs. (1) and (5) present the forward propagations of 3D DWT and IDWT. It is
onerous to deduce the gradients for the backward propagations from these
expressions. Fortunately, the modern deep learning framework PyTorch [41]
could automatically deduce the gradients for tensor arithmetics. We have
rewritten 3D DWT and IDWT as general network layers in PyTorch, which will be
publicly available for other researchers. One can flexibly design end-to-end
3D wavelet integrated deep networks using these layers.

For the noisy neuronal cube X, the random noise mostly show up in its high-
frequency components, while the basic nerve fiber structure is presented by
the low-frequency one. Therefore, 3D DWT and IDWT could be used to denoise the
neuronal cube while maintaining the structure of nerve fiber at the same time.
DWT halves the size of the neuronal data, and IDWT recovers it, which are good
substitutes for the commonly used sampling operations in the 3D deep networks.
They could be used to reduce the aliasing effects in the 3D deep networks, and
improve the performance of 3D neuron segmentation and reconstruction. Fig. 3
compares the results of 3D DWT/IDWT and 3D max-pooling/max-unpooling on a
neuronal cube.

###  IV-B The architecture of 3D WaveUNets

Figure 4: The encoder-decoder architecture.

In this section, we design 3D WaveUNet using 3D DWT and IDWT to increase
neuron segmentation performance for better neuron reconstruction.

As Fig. 4 shows, the encoder-decoder architecture could be decomposed into
several nested dual structures with a bottom block. The dual structure
consists of forward process and reverse process connected via a branch path.
While the forward process contains a serial of 3D convolutions followed by a
down-sampling operation, the reverse process is an up-sampling operation
followed by a number of 3D convolutions. The reverse process up-samples the
feature maps, and exploits the data transmitted from the forward process via
the branch path. The bottom block is variant in different encoder-decoder
architectures. 3D U-Net [42] takes a convolution block as the bottom block,
while 3D U-Net Plus [6] employs a 3D ASPP [43]. We denote the all connected
forward processes and reverse processes, together with the bottom block, as
main path, and denote the data flowing through it as mainstream.

(a) DS-PU, 3D version of dual structure used in SegNet [44]

(b) DS-PDc used in 3D U-Net Plus [6] and 3D U-Net [45]

(c) DS-ScIn, 3D version of dual structure used in DeepLabV3+ [43]

Figure 5: The 3D versions of the commonly used dual structures.

(a) WADS-DDc (DWT, Deconvolute)

(b) WADS-DIn (DWT, Interpolate)

(c) WADS-DI (DWT, IDWT)

(d) WADS-DIDn (DWT, IDWT; Denoise)

Figure 6: The 3D wavelet integrated dual structures (WADS).

Fig. 5 shows the 3D versions of dual structures used in previous encoder-
decoder networks [42, 44, 43, 45, 6]. DS-PU, Dual Structure with max-Pooling
and max-Unpooling, is a 3D version of dual structure used in SegNet [44]. As
Fig. 5(a) shows, DS-PU applies max-pooling in its forward process to reduce
the feature map resolution and transmits the pooling indices to the reverse
process for feature map up-sampling via max-unpooling. However, the max-
pooling operation would accumulate the data noise, and the max-unpooling
cannot restore the lost details, as Fig. 3 shows. Fig. 5(b) shows DS-PDc, Dual
Structure with max-Pooling and Deconvolution, used in 3D U-Net [45] and 3D
U-Net Plus [6]. While DS-PDc applies the max-pooling for the down-sampling in
its forward process, it uses deconvolution for the up-sampling in the reverse
process. DS-PDc copies the feature maps from its forward process to reverse
process via the branch path, then concatenates it with the up-sampled feature
map in reverse process to recover the low-level information in mainstream. DS-
ScIn, shown in Fig. 5(c), has the same structure with DS-PDc, while it applies
Strided-convolution and linear Interpolation for down-sampling and up-
sampling, respectively. In DS-PDc and DS-ScIn, the data tensor injected from
their forward processes to reverse processes contains redundant information
including noises, which could interfere with 3D neuron segmentation.

We design four 3D WAvelet based Dual Structure (WADS) by replacing the
sampling operations with 3D DWT/IDWT, as Fig. 6 shows. In comparison, WADS-DDc
(Fig. 6(a)) and WADS-DIn (Fig. 6(b)) are designed by replacing the down-
sampling operations in DS-PDc and DS-ScIn with 3D DWT, respectively. In their
forward processes, while the feature map is decomposed by 3D DWT into eight
components, only the low-frequency one is kept to extract the robust high-
level features, and the seven high-frequency components are abandoned. WADS-DI
(Fig. 6(c)) and WADS-DIDn (Fig. 6(d)) apply 3D DWT and 3D IDWT for down-
sampling and up-sampling in their forward and reverse processes, respectively.
In the forward process, while feature map is decomposed by 3D DWT, the low-
frequency component is kept to extract the robust high-level features, and the
seven high-frequency components are transmitted, via the branch path, to
reverse process for feature map up-sampling in 3D IDWT. WADS-DIDn utilizes a
Denoising block to filter the high-frequency components, which is implemented
by hard shrinkage with threshold $\lambda=0.25$:

$\displaystyle\text{HardShrink}(x)=\begin{cases}x,&\text{ if }x>\lambda,\\\
x,&\text{ if }x<-\lambda,\\\ 0,&\text{otherwise}.\end{cases}$ (51)

In the denoising block, we filter every coefficient $x$ in the seven high-
frequency components

$\textbf{\emph{X}}_{c},~{}c\in\\{llh,lhl,lhh,hll,hlh,hhl,hhh\\},$

according to Eq. (51).

TABLE I: Deep network configurations.

data size | channel number | 3D U-Netsa | 3D WaveUNetsb
---|---|---|---
encoder | decoder
$32\times 128\times 128$ | 1, 4 | 4, 4 |  DS-$x$ |  WADS-$y$
$16\times 64\times 64$ | 4, 8 | 8, 4 |  DS-$x$ |  WADS-$y$
$8\times 32\times 32$ | 8, 16 | 16, 8 |  DS-$x$ |  WADS-$y$
$4\times 16\times 16$ | 16, 32 | 32, 16 |  DS-$x$ |  WADS-$y$
1-5[2.25pt/3pt] $4\times 16\times 16$ | 32, 32 | bottom block

a The three 3D U-Nets are named as 3D U-Net($x$), $x\in\\{\text{PU, PDc,
ScIn}\\}$.

b The four 3D WaveUNets designed in this paper are named as 3D WaveUNet($y$),
$y\in\\{\text{DDc, DIn, DI, DIDn}\\}$.

Using the seven dual structures, we design seven 3D encoder-decoder networks
for neuron segmentation. While the first three networks are variants of 3D
U-Net, the last four are 3D WaveUNets designed in this paper. Each of them
contains four nested dual structures with one bottom block containing two
convolutions. Table I illustrates their configurations. In Table I, the first
column shows the input size. Every number in the table corresponds to a
convolutional layer with batch normalization (BN) and ReLU. While the number
in column “encoder” is the number of input channels of the convolution, the
number in column “decoder” is that of output channels. At the end of networks,
a convolution with kernel size of $1\times 1$ converts the output of decoder
into the predicted segmentation result.

##  V The neuronal cube dataset

Figure 7: A neuronal image and three cubes cut from it. _A_ : labeling, _B_ :
cutting. The cubes are corrupted by random noises; some weak nerve fibers in
the cubes are disconnected, as marked by the yellow arrows. The 3D label
matrices are shown below the cubes.

As we partition the large neuron images into small cubes for neuron
segmentation, it is crucial to collect an appropriate dataset to train and
evaluate 3D WaveUNet. We here introduce the _Neu_ ronal _Cu_ be _Da_ taset,
NeuCuDa, generated from the BigNeuron images. The procedure to produce NeuCuDa
is shown in Fig. 7 and summarized as below:

  1. 1)

_Selection_. From the 166 3D neuronal images in BigNeuron, we choose 61 images
to generate the neuronal cubes for the training of 3D deep networks, and 28
images to generate the testing cubes. The 28 testing images will also be used
to evaluate the performance of reconstruction method proposed in Sec. III.

  2. 2)

_Labeling_. For each of the chosen 89 (61 + 28) neuronal images, we generate a
3D 0-1 label matrix according to its standard digital reconstruction. Every
voxel in the images is assigned to 0 (background) or 1 (nerve fiber). An
example neuronal image with its 3D label matrix is shown in Fig. 7.

  3. 3)

_Cutting_. Neuronal cubes with size of $32\times 128\times 128$ are randomly
cut from the 3D image. Meanwhile, their label cubes are cut from the
corresponding 3D label matrix. Fig. 7 shows three example neuronal cubes.

NeuCuDa contains 19251 and 7132 neuronal cubes for the training and testing of
3D WaveUNet, respectively, which will be publicly available for other
researchers.

##  VI Experiments

TABLE II: Segmentation and reconstruction results.

Networks | Dual | Encoder | Decoder | wavelet | Segmentation (IoU) | Reconstruction | Parameters
---|---|---|---|---|---|---|---
Structure |  Pool |  Str-conv |  DWT |  Unpool |  Deconv |  Interpolate |  IDWT |  bga | ner_fib | mean | ESA | DSA | PDS
baseline (APP2) | – | – | – | – | – | – | – | – | – | – | – | – | 9.3942 | 14.3641 | 0.3573 | –
3D U-Net | DS-PU | $\surd$ |  |  | $\surd$ |  |  |  | none | 99.04 | 45.55 | 72.30 | 2.5444 | 7.3450 | 0.2200 |  0.17 $\times 10^{6}$
DS-PDc | $\surd$ |  |  |  | $\surd$ |  |  | none | 99.29 | 53.65 | 76.43 | 2.4046 | 7.1799 | 0.1950 |  0.20 $\times 10^{6}$
DS-ScIn |  | $\surd$ |  |  |  | $\surd$ |  | none | 99.21 | 52.96 | 76.09 | 2.3614 | 6.9439 | 0.1968 |  0.20 $\times 10^{6}$
3D WaveUNet | WADS-DDc |  |  | $\surd$ |  | $\surd$ |  |  | haar | 99.32 | 54.80 | 77.06 | 2.3272 | 6.8841 | 0.1962 | 0.20 $\times 10^{6}$
|  |  |  |  | ch2.2 | 99.32 | 54.58 | 76.95 | 2.2495 | 6.5712 | 0.1956
|  |  |  |  | db3 | 90.27 | 54.76 | 77.02 | 2.3391 | 7.0144 | 0.1931
WADS-DIn |  |  | $\surd$ |  |  | $\surd$ |  | haar | 99.20 | 52.59 | 75.90 | 2.5247 | 7.1783 | 0.2083 | 0.20 $\times 10^{6}$
|  |  |  |  | ch2.2 | 99.27 | 52.89 | 76.08 | 2.0288 | 6.2569 | 0.1922
|  |  |  |  | db2 | 99.20 | 52.67 | 75.94 | 2.2197 | 6.6692 | 0.1947
WADS-DI |  |  | $\surd$ |  |  |  | $\surd$ | haar | 99.30 | 54.31 | 76.81 | 2.0588 | 6.4043 | 0.1866 | 0.17 $\times 10^{6}$
|  |  |  |  | ch4.4 | 99.28 | 54.46 | 76.87 | 2.1726 | 6.3609 | 0.1938
|  |  |  |  | db4 | 99.22 | 53.06 | 76.14 | 2.0676 | 6.2566 | 0.1857
WADS-DIDnb |  |  | $\surd$ |  |  |  | $\surd$ | haar | 99.30 | 54.76 | 77.03 | 2.0336 | 6.0730 | 0.1868 | 0.17 $\times 10^{6}$
|  |  |  |  | ch4.4 | 99.26 | 54.78 | 77.02 | 2.2067 | 6.6038 | 0.1914
|  |  |  |  | db4 | 99.24 | 53.22 | 76.23 | 1.9973 | 6.0173 | 0.1897
3D Residual Net [5] | DS-ScDc |  | $\surd$ |  |  | $\surd$ |  |  | none | – | – | – | 3.2892 | 5.8900 | 0.3367 |  11.41 $\times 10^{6}$
3D U-Net Plus [6] | DS-PDc | $\surd$ |  |  |  | $\surd$ |  |  | none | – | – | – | 33.5020 | 40.3151 | 0.5539 |  3.60 $\times 10^{6}$

a “bg” and “ner_fib” are short for “background” and “nerve fiber”,
respectively.

b “Dn” stands for “Denoising block”.

We first train and evaluate the seven 3D deep networks on NeuCuDa. Then, using
the well-trained networks, we segment and reconstruct the neurons in the 28
test neuronal images with the proposed method in Sec. III.

###  VI-A 3D neuronal cube segmentation

On the 19251 training cubes, we train the 3D U-Nets and 3D WaveUNets for 30
epochs, using stochastic gradient descent (SGD). The training is driven by
cross-entropy loss with weights 1, 5 for “background” and “nerve fiber”
voxels, to address the imbalances in the cubes. We initially take a learning
rate of 0.1, and reduce it following a polynomial decay for each iteration.
The batch size, momentum, and weight decay are set as 32, 0.9, and 0.0001,
respectively. The segmentation performances on the 7132 test cubes are shown
in Table II. We train 3D WaveUNets when various wavelets are used, and for
each network architecture, the results of three wavelets are shown in Table
II. In the table, “haar” stands for the Haar wavelet, while “db$p$” and
“ch$p.\tilde{p}$” stand for Daubechies with approximation order $p$ and Cohen
wavelet with orders $p,\tilde{p}$, respectively. The filters of these wavelets
could be found in [28].

3D U-Net(PU), applying max-pooling/max-unpooling, performs the worst on
neuronal cube segmentation among the seven deep networks. This result suggests
that the pair of sampling operations is not suitable for neuron segmentation,
which matches our analysis. Comparing the segmentation results of 3D
U-Net(PDc) and 3D WaveUNet(DDc), one can find that DWT performs better than
max-pooling, when deconvolution is taken as up-sampling operation; 3D
WaveUNet(DDc) achieves the best segmentation performance (mIoU, $77.06\%$).

Among the seven networks, the segmentation performances of 3D U-Net(ScIn) and
3D WaveUNet(DIn) are only better than that of 3D U-Net(PU), which indicates
that the linear interpolation used in their decoder is not a good up-sampling
operation for 3D neuron segmentation.

3D WaveUNet(DI) and 3D WaveUNet(DIDn), which apply 3D DWT and IDWT for neuron
data down-sampling and up-sampling, perform also well for neuronal cube
segmentation. Comparing their results, we conclude that denoising on high-
frequency components could slightly improve the performance of neuronal cube
segmentation. The segmentation performance of 3D WaveUNet(DIDn) is close to
that of 3D WaveUNet(DDc), although the convolutions in decoder of 3D
WaveUNet(DDc) employs more parameters. In summary, the 3D wavelets improve the
performance of neuron segmentation in 3D noisy images.

Figure 8: The reconstructions of various methods. (a) The reconstruction
manually traced by experts. (b) - (c) Two reconstructions of APP2 on the
original neuronal image, with different parameters. (d) - (f) The
reconstructions of APP2 on the images segmented by the three 3D U-Nets. (g) -
(j) The reconstructions of APP2 on the images segmented by the four 3D
WaveUNets.

###  VI-B 3D neuron reconstruction

After segmenting the nerve fibers in the partitioned cubes, we assemble 3D
neurons in the 28 test neuronal images, and reconstruct them using APP2, as
described in Sec. III. We evaluate the accuracy of automatic reconstructions
by comparing them with the manual one, using three metrics (lower is better),
i.e., “entire structure average (ESA)”, “different structure average (DSA)”,
and “percentage of different structures (PDS)”, proposed in [46] by Peng et
al. Table II shows the three mean metric values of the 28 neurons
reconstructed using APP2 on the segmentation results of 3D U-Nets and 3D
WaveUNets.

The three mean metric values (9.3942, 14.3641, and 0.3573) of APP2 on the 28
original noisy images are taken as baseline. From Table II, one can find that
the reconstruction performance on the segmented images of the seven deep
networks is clearly superior to the baseline, which indicates that our method
significantly improve the 3D neuron reconstruction. Generally, with the
application of 3D DWT and IDWT in the encoder and decoder of the deep
networks, the reconstruction performance is getting better and better, and 3D
WaveUNet(DIDn) with wavelet “db4” achieves the best reconstruction performance
(1.9973, 6.0173, and 0.1897). In summary, our 3D wavelet and deep network
based method efficiently improve the automatic reconstruction performance for
3D neurons in the noisy images.

Table II also lists the reconstruction performance of two previous deep
network based 3D neuron segmentation methods [5, 6]. For “3D Residual Net”, we
compute the three mean metric values (3.2892, 5.8900, and 0.3367) of APP2
performance on 17 BigNeuron images published in [5]. For “3D U-Net Plus”, we
reconstruct the neurons in the 28 test neuronal images using the well-trained
3D U-Net Plus and reconstruction method released in [6], and the
reconstruction performance is 33.5020, 40.3151, and 0.5539. While the DSA
value (5.8900) of 3D Residual Net is slightly better than that of our method,
the network employs significantly more parameters (11.41 M) than our 3D
WaveUNets (0.17 M). The performance of 3D U-Net Plus is even inferior to the
baseline, because the method can not completely segment the neurons in large
size neuronal images.

Fig. 8 and Fig. 9 visually show the reconstructions of two noisy neuronal
image. In Fig. 8, the original size of the image is $291\times 3298\times
1881$ (z-y-x), while the previous deep network based 3D neuron segmentation
methods [5, 6, 27] cannot process the neuronal images with such a large size.
Fig. 8 shows the manual reconstruction of the neuron, which perfectly reflect
the neuron morphology. Fig. 8 and Fig. 8 show the reconstructions of APP2 with
different parameters on the original neuronal image. The following three
subfigures show the reconstructions of APP2 on the neurons segmented by the
three 3D U-Nets, and the last four subfigures show that of the four 3D
WaveUNets.

On the original noisy image, the automatic tracing algorithm APP2 either stop
tracing weak nerve fibers (Fig. 8) or over-reconstruct strong noises as nerve
fibers (Fig. 8). From Fig. 8, one can find that the reconstructions of APP2 on
the segmented image are superior to that on the original image, which
illustrates the efficiency of our 3D neuron reconstruction method. The 3D
U-Nets either classify “background” voxels as “nerve fiber” voxels or do not
extract complete nerve fibers. Therefore, their segmented neuronal images
contain much background noise (Fig. 8 and Fig. 8), or the reconstructions are
not complete (Fig. 8 and Fig. 8), as marked by the yellow arrows. The 3D
WaveUNets obtain cleaner segmentation and more complete reconstruction, which
justifies the effectiveness of 3D wavelets in suppressing noise and keeping
nerve fiber structure. Comprehensively, the 3D WaveUNet(DIDn), which utilizes
DWT, IDWT, and denoising block, achieves the best segmentation and
reconstruction results.

Figure 9: 3D reconstructed neuron surrounded by nerve fibers. (a) The
reconstruction manually traced by experts. (b) - (c) Two reconstructions of
APP2 on the original neuronal image. (d) The reconstruction of APP2 on the
images segmented by 3D U-Net(PDc). (e) - (f) The reconstructions of APP2 on
the images segmented by 3D WaveUNet(DI) and 3D WaveUNet(DIDn).

Fig. 9 visually show another example neuron reconstructed from a noisy image,
where lots of tangling nerve fibers are available. The original size of the
neuronal image is $37\times 803\times 928$, and a lot of nerve fibers surround
the target neuron or pass through the image, which result in gread challenges
to the neuron segmentation and reconstruction algorithms. Fig. 9 shows the
manual reconstruction of the target neuron, while Fig. 9 and Fig. 9 show the
reconstructions of APP2 with different parameters on the original neuronal
image. The following three subfigures in the second row show the
reconstructions of APP2 on the neuron segmented by 3D U-Net(PDc), 3D
WaveUNet(DI), and 3D WaveUNet(DIDn).

On the original image, the automatic tracing algorithm, APP2, either overlooks
the weak nerve fibers of the target neuron (Fig. 9) or incorrectly include the
fibers as part of the target neuron (Fig. 9). On the segmented images, the
reconstruction performance of APP2 is significantly improved. As marked by the
yellow arrow in Fig. 9, a fiber is incorrectly traced by APP2 on the segmented
image of 3D U-Net. The APP2 reconstructions (Fig. 9 and Fig. 9) on the images
segmented by 3D WaveUNets match better with the manual one. These results
illustrate that our segmentation method and 3D wavelet could suppress the
interference of surrounding nerve fibers in the image and efficiently improve
the results of 3D neuron reconstruction.

##  VII Conclusions

We rewrite 3D DWT/IDWT as general network layers, and design the first 3D
wavelet integrated encoder-decoder network, 3D WaveUNet, for 3D neuron
segmentation. The 3D wavelet transforms could suppress the noise propagation
and keep nerve fiber structure during the deep network inference, thus improve
the performance of 3D neuron segmentation and reconstruction in noisy neuronal
images.

In future, we will study the 3D neuron instance segmentation for images
containing multiple neurons, and extend the 3D wavelet integrated deep
networks to more 3D medical image applications.

## References

  * [1] A. Li, H. Gong, B. Zhang, Q. Wang, C. Yan, J. Wu, Q. Liu, S. Zeng, and Q. Luo, “Micro-optical sectioning tomography to obtain a high-resolution atlas of the mouse brain,” _Science_ , vol. 330, no. 6009, pp. 1404–1408, 2010.
  * [2] H. Xiao and H. Peng, “App2: automatic tracing of 3d neuron morphology based on hierarchical pruning of a gray-weighted image distance-tree,” _Bioinformatics_ , vol. 29, no. 11, pp. 1448–1454, 2013.
  * [3] S. Liu, D. Zhang, Y. Song, H. Peng, and W. Cai, “Automated 3-d neuron tracing with precise branch erasing and confidence controlled back tracking,” _IEEE transactions on medical imaging_ , vol. 37, no. 11, pp. 2441–2452, 2018\.
  * [4] T. Quan, H. Zhou, J. Li, S. Li, A. Li, Y. Li, X. Lv, Q. Luo, H. Gong, and S. Zeng, “Neurogps-tree: automatic reconstruction of large-scale neuronal populations with dense neurites,” _Nature methods_ , vol. 13, no. 1, p. 51, 2016.
  * [5] R. Li, T. Zeng, H. Peng, and S. Ji, “Deep learning segmentation of optical microscopy images improves 3-d neuron reconstruction,” _IEEE transactions on medical imaging_ , vol. 36, no. 7, pp. 1533–1541, 2017.
  * [6] Q. Li and L. Shen, “3d neuron reconstruction in tangled neuronal image with deep networks,” _IEEE transactions on medical imaging_ , vol. 39, no. 2, pp. 425–435, 2020.
  * [7] Y. Jiang, W. Chen, M. Liu, Y. Wang, and E. Meijering, “3d neuron microscopy image segmentation via the ray-shooting model and a dc-blstm network,” _IEEE Transactions on Medical Imaging_ , 2020.
  * [8] T. Klinghoffer, P. Morales, Y.-G. Park, N. Evans, K. Chung, and L. J. Brattain, “Self-supervised feature extraction for 3d axon segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2020, pp. 978–979.
  * [9] Q. Huang, Y. Chen, S. Liu, C. Xu, T. Cao, Y. Xu, X. Wang, G. Rao, A. Li, S. Zeng _et al._ , “Weakly supervised learning of 3d deep network for neuron reconstruction,” _Frontiers in Neuroanatomy_ , vol. 14, 2020.
  * [10] Q. Li, L. Shen, S. Guo, and Z. Lai, “Wavelet integrated cnns for noise-robust image classification,” in _The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2020.
  * [11] R. Zhang, “Making convolutional networks shift-invariant again,” in _Proceedings of the International Conference on Machine Learning_ , 2019, pp. 7324–7334.
  * [12] X. Zou, F. Xiao, Z. Yu, and Y. J. Lee, “Delving deeper into anti-aliasing in convnets,” in _Proceedings of the British Machine Vision Conference_ , 2020\.
  * [13] H. Peng, M. Hawrylycz, J. Roskams, S. Hill, N. Spruston, E. Meijering, and G. A. Ascoli, “Bigneuron: large-scale 3d neuron reconstruction from optical microscopy images,” _Neuron_ , vol. 87, no. 2, pp. 252–256, 2015.
  * [14] H. Gong, S. Zeng, C. Yan, X. Lv, Z. Yang, T. Xu, Z. Feng, W. Ding, X. Qi, A. Li _et al._ , “Continuously tracing brain-wide long-distance axonal projections in mice at a one-micron voxel resolution,” _Neuroimage_ , vol. 74, pp. 87–98, 2013.
  * [15] K. Chung and K. Deisseroth, “Clarity for mapping the nervous system,” _Nature methods_ , vol. 10, no. 6, p. 508, 2013.
  * [16] K. Chung, J. Wallace, S.-Y. Kim, S. Kalyanasundaram, A. S. Andalman, T. J. Davidson, J. J. Mirzabekov, K. A. Zalocusky, J. Mattis, A. K. Denisin _et al._ , “Structural and molecular interrogation of intact biological systems,” _Nature_ , vol. 497, no. 7449, pp. 332–337, 2013.
  * [17] T. Ku, J. Swaney, J.-Y. Park, A. Albanese, E. Murray, J. H. Cho, Y.-G. Park, V. Mangena, J. Chen, and K. Chung, “Multiplexed and scalable super-resolution imaging of three-dimensional protein localization in size-adjustable tissues,” _Nature biotechnology_ , vol. 34, no. 9, pp. 973–981, 2016.
  * [18] Y.-G. Park, C. H. Sohn, R. Chen, M. McCue, D. H. Yun, G. T. Drummond, T. Ku, N. B. Evans, H. C. Oak, W. Trieu _et al._ , “Protection of tissue physicochemical properties using polyfunctional crosslinkers,” _Nature biotechnology_ , vol. 37, no. 1, pp. 73–83, 2019.
  * [19] H. Peng, Z. Ruan, F. Long, J. H. Simpson, and E. W. Myers, “V3d enables real-time 3d visualization and quantitative analysis of large-scale biological image data sets,” _Nature biotechnology_ , vol. 28, no. 4, p. 348, 2010.
  * [20] A. K. Ai-Awami, J. Beyer, D. Haehn, N. Kasthuri, J. W. Lichtman, H. Pfister, and M. Hadwiger, “Neuroblocks–visual tracking of segmentation and proofreading for large connectomics projects,” _IEEE transactions on visualization and computer graphics_ , vol. 22, no. 1, pp. 738–746, 2015.
  * [21] C. Magliaro, A. L. Callara, N. Vanello, and A. Ahluwalia, “A manual segmentation tool for three-dimensional neuron datasets,” _Frontiers in neuroinformatics_ , vol. 11, p. 36, 2017.
  * [22] Y. Wang, Q. Li, L. Liu, Z. Zhou, Z. Ruan, L. Kong, Y. Li, Y. Wang, N. Zhong, R. Chai _et al._ , “Teravr empowers precise reconstruction of complete 3-d neuronal morphology in the whole brain,” _Nature communications_ , vol. 10, no. 1, pp. 1–9, 2019.
  * [23] G. A. Ascoli, D. E. Donohue, and M. Halavi, “Neuromorpho. org: a central resource for neuronal morphologies,” _Journal of Neuroscience_ , vol. 27, no. 35, pp. 9247–9251, 2007.
  * [24] Y. Wang, A. Narayanaswamy, C.-L. Tsai, and B. Roysam, “A broadly applicable 3-d neuron tracing method based on open-curve snake,” _Neuroinformatics_ , vol. 9, no. 2-3, pp. 193–217, 2011.
  * [25] H. Peng, F. Long, and G. Myers, “Automatic 3d neuron tracing using all-path pruning,” _Bioinformatics_ , vol. 27, no. 13, pp. i239–i247, 2011.
  * [26] S. Liu, D. Zhang, S. Liu, D. Feng, H. Peng, and W. Cai, “Rivulet: 3d neuron morphology tracing with iterative back-tracking,” _Neuroinformatics_ , vol. 14, no. 4, pp. 387–401, 2016.
  * [27] H. Wang, D. Zhang, Y. Song, S. Liu, H. Huang, M. Chen, H. Peng, and W. Cai, “Multiscale kernels for enhanced u-shaped network to improve 3d neuron tracing,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2019, pp. 0–0.
  * [28] I. Daubechies, _Ten lectures on wavelets_. Siam, 1992, vol. 61.
  * [29] D. L. Donoho, “De-noising by soft-thresholding,” _IEEE transactions on information theory_ , vol. 41, no. 3, pp. 613–627, 1995.
  * [30] D. L. Donoho and J. M. Johnstone, “Ideal spatial adaptation by wavelet shrinkage,” _biometrika_ , vol. 81, no. 3, pp. 425–455, 1994.
  * [31] B.-J. Kim and W. A. Pearlman, “An embedded wavelet video coder using three-dimensional set partitioning in hierarchical trees (spiht),” in _Proceedings DCC’97. Data Compression Conference_. IEEE, 1997, pp. 251–260.
  * [32] J. Wu, K. Jiang, Y. Fang, L. Jiao, and G. Shi, “Hyperspectral image compression using distributed source coding and 3d speck,” in _MIPPR 2009: Multispectral Image Acquisition and Processing_ , vol. 7494. International Society for Optics and Photonics, 2009, p. 74940Z.
  * [33] C. G. Bahce and U. Bayazit, “Compression of geometry videos by 3d-speck wavelet coder,” _VISUAL COMPUTER_ , 2020.
  * [34] H. Huang, R. He, Z. Sun, and T. Tan, “Wavelet-srnet: A wavelet-based cnn for multi-scale face super resolution,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 1689–1697.
  * [35] Y. Duan, F. Liu, L. Jiao, P. Zhao, and L. Zhang, “Sar image segmentation based on convolutional-wavelet neural network and markov random field,” _Pattern Recognition_ , vol. 64, pp. 255–267, 2017.
  * [36] P. Liu, H. Zhang, K. Zhang, L. Lin, and W. Zuo, “Multi-level wavelet-cnn for image restoration,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2018, pp. 773–782.
  * [37] T. Williams and R. Li, “Wavelet pooling for convolutional neural networks,” in _International Conference on Learning Representations_ , 2018.
  * [38] J. Guan, R. Lai, and A. Xiong, “Wavelet deep neural network for stripe noise removal,” _IEEE Access_ , vol. 7, pp. 44 544–44 554, 2019.
  * [39] J. Yoo, Y. Uh, S. Chun, B. Kang, and J.-W. Ha, “Photorealistic style transfer via wavelet transforms,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2019, pp. 9036–9045.
  * [40] L. Liu, J. Liu, S. Yuan, G. Slabaugh, A. Leonardis, W. Zhou, and Q. Tian, “Wavelet-based dual-branch network for image demoiréing,” _arXiv preprint arXiv:2007.07173_ , 2020.
  * [41] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” 2017.
  * [42] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _International Conference on Medical image computing and computer-assisted intervention_. Springer, 2015, pp. 234–241.
  * [43] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in _Proceedings of the ECCV_ , 2018, pp. 801–818.
  * [44] V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 39, no. 12, pp. 2481–2495, 2017.
  * [45] Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3d u-net: learning dense volumetric segmentation from sparse annotation,” in _International conference on medical image computing and computer-assisted intervention_. Springer, 2016, pp. 424–432.
  * [46] H. Peng, Z. Ruan, D. Atasoy, and S. Sternson, “Automatic reconstruction of 3d neuron structures using a graph-augmented deformable model,” _Bioinformatics_ , vol. 26, no. 12, pp. i38–i46, 2010.


# Entangling four logical qubits beyond break-even in a nonlocal code

Yifan Hong<EMAIL_ADDRESS>Department of Physics and Center for
Theory of Quantum Matter, University of Colorado, Boulder, CO 80309, USA
Elijah Durso-Sabina  Quantinuum, Broomfield, CO 80021, USA  David Hayes
Quantinuum, Broomfield, CO 80021, USA  Andrew Lucas  Department of Physics and
Center for Theory of Quantum Matter, University of Colorado, Boulder, CO
80309, USA

###### Abstract

Quantum error correction protects logical quantum information against
environmental decoherence by encoding logical qubits into entangled states of
physical qubits. One of the most important near-term challenges in building a
scalable quantum computer is to reach the break-even point, where logical
quantum circuits on error-corrected qubits achieve higher fidelity than
equivalent circuits on uncorrected physical qubits. Using Quantinuum’s H2
trapped-ion quantum processor, we encode the GHZ state in four logical qubits
with fidelity $99.5\pm 0.15\%\leq F\leq 99.7\pm 0.1\%$ (after postselecting on
over 98% of outcomes). Using the same quantum processor, we can prepare an
uncorrected GHZ state on four physical qubits with fidelity $97.8\pm 0.2\%\leq
F\leq 98.7\pm 0.2\%$. The logical qubits are encoded in a $\llbracket
25,4,3\rrbracket$ Tanner-transformed long-range-enhanced surface code. Logical
entangling gates are implemented using simple swap operations. Our results are
a first step towards realizing fault-tolerant quantum computation with logical
qubits encoded in geometrically nonlocal quantum low-density parity check
codes.

_Introduction._ — It is widely believed that fault tolerance will be necessary
for quantum computers to outperform classical computers on large-scale
problems. Decoherence, if left ignored, will eventually destroy entanglement
and any quantum advantage in computation. A critical milestone for the
development of a quantum computer will, therefore, be the protection of
logical qubits and the implementation of logical gates whose error rates are
below that of physical qubits and circuits which are not error-corrected. When
logical qubits are more robust than physical qubits, we have reached _break-
even_.

Since the discovery of topological quantum codes Kitaev (2003); Bombin and
Martin-Delgado (2006) and their promise to preserve quantum information for
arbitrarily long times with a relatively high tolerance for errors Dennis  _et
al._ (2002); Landahl  _et al._ (2011), there have been immense efforts to
implement such codes in current architectures Postler  _et al._ (2022); Ryan-
Anderson  _et al._ (2021); AI (2023); Bluvstein  _et al._ (2023); da Silva
_et al._ (2024). Unfortunately, the resource requirements for large-scale,
fault-tolerant quantum computation using topological codes with current
hardware specifications may be enormous Gidney and Ekerå (2021).

More generally, it is known that these overheads cannot be substantially
improved in planar architectures (codes) with only nearest-neighbor
connectivity Bravyi  _et al._ (2010). To avoid this fundamental challenge,
quantum low-density parity-check (LDPC) codes Breuckmann and Eberhardt
(2021a); Tillich and Zemor (2014); Hastings  _et al._ (2020); Panteleev and
Kalachev (2022a); Breuckmann and Eberhardt (2021b); Panteleev and Kalachev
(2022b); Dinur  _et al._ (2022); Leverrier and Zémor (2022) have been
developed to significantly reduce the resource overheads for error correction
Gottesman (2014); Fawzi  _et al._ (2018). To avoid the constraints of Ref.
Bravyi  _et al._ (2010), these codes will necessarily require long-range
connectivity Baspin and Krishna (2022); Baspin  _et al._ (2023). Platforms
with such capabilities may be able to significantly reduce the overhead for
quantum error correction, and a number of recent proposals have been made for
how to incorporate limited long-range connectivity into hardware Bravyi  _et
al._ (2024); Xu  _et al._ (2024); Hong  _et al._ (2024); Viszlai  _et al._
(2023) to ameliorate the constraints of spatial locality.

Thus far, we are not aware of any explicit implementation of a nonlocal LDPC
code in any quantum hardware. Other recent demonstrations of break-even
operations we are aware of include the use of a small 7-qubit Steane code
Ryan-Anderson  _et al._ (2022); Bluvstein  _et al._ (2023) and the use of
concatenated error detection codes da Silva  _et al._ (2024), neither of which
achieve the high-encoding rates of qLDPC codes. While such codes may improve
performance, to truly avoid large resource overheads in a scalable way, it is
crucial to use a nonlocal code which protects logical qubits in a more complex
manner, yet which is also amenable to quantum hardware.

This work provides a proof-of-principle demonstration that nonlocal codes can
improve the performance of present-day quantum computers. We have implemented
a $\llbracket 25,4,3\rrbracket$ quantum LDPC code (encoding 4 logical qubits
in 25 physical qubits, with the smallest logical operator acting on 3 physical
qubits) on Quantinuum’s H2 trapped-ion quantum computer Moses  _et al._
(2023), which was configured with 32 physical qubits at the time of the
experiments. This encoding enables us to prepare an error-corrected, logical
4-qubit entangled GHZ state beyond the break-even point, with mild
postselection overhead, for the first time. Using the existing hardware, it
would not have been possible to attempt such a demonstration, even in
principle, by using four surface code patches (with 36 total physical) to
encode the logical qubits. This work highlights the importance of choosing
good hardware-suited codes with efficient logical gates. The long-range-
enhanced surface code Hong  _et al._ (2024) underlying our methods will be
scalable to larger trapped-ion QCCD processors Kielpinski  _et al._ (2002);
Home  _et al._ (2009); Pino  _et al._ (2020); Kaushal  _et al._ (2020) as well
as to neutral atom arrays Saffman (2016); Browaeys and Lahaye (2020); Kaufman
and Ni (2021); Evered  _et al._ (2023); Bluvstein  _et al._ (2023) for the
foreseeable future.

_Quantum encoding._ — The quantum LDPC code we use is a Calderbank-Shor-Steane
(CSS) stabilizer code Gottesman (1997); Calderbank and Shor (1996); Steane
(1996), which is defined by a stabilizer group of Pauli strings that mutually
commute and act trivially on logical codewords. In a CSS code, these Pauli
strings are generated by products of strictly $X$ or $Z$ operators acting on
$n$ qubits. Given the Pauli (anti)commutation relations, it is helpful to
organize data about the stabilizer group in a pair of matrices $H_{X,Z}$ with
$\mathbb{F}_{2}=\\{0,1\\}$ coefficients. Each row of the matrix $H_{X}$, which
in turn has $n$ columns, fixes one of the stabilizers of the code to be a
product of Pauli $X$s on all qubits with a 1 in the corresponding column;
$H_{Z}$ is defined similarly. To be a valid code, we require $X$ and $Z$
stabilizers to commute, which means $H_{X}H_{Z}^{\mathsf{T}}=0$ (here
multiplication/addition are mod 2).

A convenient way to build a quantum code is the hypergraph product Tillich and
Zemor (2014) of a classical linear code with itself. For an input $m\times n$
classical parity-check matrix $H$, the resulting CSS stabilizer matrices take
the form

$\displaystyle H^{\vphantom{*}}_{X}$
$\displaystyle=\big{(}\,H\otimes\mathds{1}_{n}\;\;|\;\;\mathds{1}_{m}\otimes
H^{\mathsf{T}}\,\big{)}$ (1a) $\displaystyle H^{\vphantom{*}}_{Z}$
$\displaystyle=\big{(}\,\mathds{1}_{n}\otimes
H\;\;|\;\;H^{\mathsf{T}}\otimes\mathds{1}_{m}\,\big{)}\,.$ (1b)

If the classical input code has parameters $[n,k,d]$ with $m=n-k$ parity
checks (meaning that there are $n$ classical bits encoding $k$ logicals, with
code distance $d$), then so long as the classical code has no redundant parity
checks, the resulting quantum CSS hypergraph product code has parameters
$\llbracket n^{2}+(n-k)^{2},k^{2},d\rrbracket$. The geometrical layout of the
quantum code can be understood as a Cartesian graph product between those of
its classical input codes. While the hypergraph product can produce codes with
constant rate ($k\propto n$), the distance scaling is sublinear
($d\lesssim\sqrt{n}$). Nonetheless, this distance is preserved under standard
syndrome extraction techniques with circuit-level noise Manes and Claes
(2023), a feature not supported by other quantum codes such as the Steane code
Steane (1996).

Figure 1: The code construction is illustrated. Circles and red (blue) squares
denote qubits and $X$($Z$)-type stabilizers respectively, with edges
representing their connectivity. Magenta lines denote long-range interactions.
(a) A quantum Tanner transformation reduces the number of physical qubits from
34 to 25. (b) The support of the logical $X$ and $Z$ operators for logical
qubits 1-4 are drawn with shaded boxes, solid lines, dashed lines, and striped
boxes respectively. (c) The order of CNOT gates for distance-preserving
syndrome extraction is depicted for a selection of stabilizers.

We now summarize the construction of our quantum LDPC code, with a graphical
depiction in Fig. 1. For the classical input code, we first pick a $[3,2,2]$
parity code consisting of a single, global parity check on all three physical
bits. We then concatenate two of the physical bits with a $[2,1,2]$ repetition
code to obtain a $[5,2,3]$ code. We then take the hypergraph product of this
$[5,2,3]$ code with itself to obtain a $\llbracket 34,4,3\rrbracket$ CSS code.
Finally, we perform a quantum Tanner transformation Leverrier and Zémor (2022)
to reduce the number of physical qubits from 34 to 25, while preserving both
$k$ and $d$. We note that the quantum Tanner transformation does not preserve
the generic circuit-level distance property mentioned earlier; however, we can
still mitigate the effects of correlated errors by choosing a clever pattern
for syndrome extraction like that for the rotated surface code Tomita and
Svore (2014). The order of CNOTs is carefully chosen so that all correlated
ancilla (hook) errors end up “against the grain” of the logical operators, and
so $d=3$ errors on the ancilla qubits are necessary to affect the codespace.
This code is a minimal example of a more general family of long-range-enhanced
surface codes Hong  _et al._ (2024), which possess the fewest nonlocal
stabilizers necessary to avoid the constraints of locality Bravyi  _et al._
(2010), while adding O(1) logical qubits.

The standard protocol to prepare the GHZ state on 4 qubits is to initialize
the qubits in $\lvert+000\rangle$ and then apply CNOTs between the
$\lvert+\rangle$ qubit (control) and the rest (target); here
$Z|0\rangle=|0\rangle$ and $X|+\rangle=|+\rangle$. Since our code is CSS and
has a distance-preserving syndrome extraction circuit, we can perform a
transversal initialization into the logical
$\lvert\bar{0}\bar{0}\bar{0}\bar{0}\rangle$ state by initializing all physical
qubits in $\lvert 0\rangle^{\otimes n}$ and then measuring all $X$-type
stabilizers. The stabilizer measurement can be performed by initializing
ancillas in the $\lvert+\rangle$ state, performing CNOT gates between the
ancillas (control) and physical qubits (target) according to the pattern in
Fig. 1c, and finally measuring all ancillas in the $X$ basis. We then prepare
the third logical qubit in the $\lvert\bar{+}\rangle$ state by measuring one
of its logical $\bar{X}_{3}$ operators three times and postselecting on the
agreement of all three outcomes in order to mitigate the effects of readout
error. After postselection, a logical $\bar{Z}_{3}$ is then applied (offline)
if the agreed outcome is $-1$. After preparing the logical
$\lvert\bar{0}\bar{0}\bar{+}\bar{0}\rangle$ state, we perform the desired
logical CNOT gates by permuting the data qubits – a feature inherited from the
$[3,2,2]$ parity code. This relabeling step can be done in software in our
particular experiment. In particular, two combinations of “fold-swaps” are
sufficient: see Fig. 2. Finally, we perform a transversal measurement of all
logical qubits in either the $X$ or $Z$ basis by measuring all physical qubits
in that basis. This transversal measurement is then used to reconstruct a
final syndrome which is not subject to circuit-level noise and can be used to
correct errors. All error correction is performed “in software” by tracking
the Pauli frame: the final logical measurement outcomes are modified according
to a decoding algorithm. We use belief-propagation with ordered statistics
decoding (BP+OSD) Roffe  _et al._ (2020); Roffe (2022), specifically the
“minimum-sum” (10 iterations) and “combination sweep” ($\lambda=14$)
strategies of BP and OS respectively, to infer physical errors based on the
syndrome measurement outcomes.

Figure 2: The procedures for preparing the physical and logical 4-qubit GHZ
states are illustrated. (a) The quantum circuit which prepares the physical
GHZ state. (b) The circuit which prepares the logical GHZ state. (c-d) A
combination of fold swaps implements the above logical circuit while
preserving the stabilizer group.

We now discuss the fault tolerance (or lack thereof) of each of the
aforementioned steps in our logical GHZ protocol. As previously shown, our
syndrome extraction circuit is distance-preserving and so we do not need to
worry about ancilla hook errors. However, since we only perform a single round
of syndrome extraction, one may worry about the effects of ancilla measurement
errors. For example, in a large surface code, a single flipped check in the
middle of the code may cause a matching decoder to pair it to the boundary,
resulting in a large error string. Fortunately, for our distance three code,
any single flipped $X$-check can be caused by a single-qubit $Z$ error, and so
we do not suffer from this type of error propagation. For the logical
$\bar{X}_{3}$ measurement, a single $Z$ error on a physical qubit prior to the
measurements can spoil the result, and so this step is not fault tolerant. We
note that a fault-tolerant procedure of measuring $\bar{X}_{3}$ can be done
using generalized lattice surgery techniques Cohen  _et al._ (2022), though
this would add considerable overhead (and requires more physical qubits than
were available in H2 at the time) and so we leave it to future work. The
physical qubit permutations to implement the fold-swaps can be achieved by a
simple relabeling in software and so is fault tolerant by definition. Lastly,
the final measurement of all physical qubits is transversal and so is also
fault tolerant.

_Experimental results._ — The experiments described in this work used the H2
trapped-ion processor described in Ref. Moses  _et al._ (2023). The H2
processor was configured with 32 trapped-ion qubits, all of which can be
directly gated with each other using ion-transport operations to pair any two-
qubits in a single potential well — a crucial feature for our $\llbracket
25,4,3\rrbracket$ code. Physical-level entangling operations are implemented
using a Mølmer-Sørensen interaction Sørensen and Mølmer (1999) driven by
lasers directed to the four gating regions. Likewise, physical level single-
qubit gates, reset, and measurement all take place in these same regions using
additional lasers systems. At the time of these experiments, the typical error
rates in H2 were as follows: $<2\times 10^{-3}$ two-qubit gate error, $\approx
3\times 10^{-5}$ single-qubit gate error, and $\approx 2\times 10^{-3}$ state
preparation and measurement (SPAM) error.

For the physical GHZ experiment, we run the circuit in Fig. 2a for 10,000
shots, measuring all physical qubits in the $Z$ basis for 5000 shots and in
the $X$ basis for the other half. For the logical GHZ experiment, we run the
circuit in Fig. 2b for 5000 shots, with 2500 shots each dedicated to logical
$\bar{X}$ and $\bar{Z}$ readout. Uncertainties in the data are computed using
the standard error $\sigma=\sqrt{p(1-p)/N}$, where $N$ is the number of shots.
For a perfect GHZ state, we expect all $Z$ measurement outcomes to agree and
the product of all $X$ measurement outcomes to have even parity. The mismatch
rates in the experimental data from these theoretical predictions are
tabulated in Table 1. In addition to the error-corrected logical results, we
also give the uncorrected logical results which are obtained by simply
ignoring the syndrome values (i.e. no feedback).

GHZ experiment |  $\bar{z}$ mismatch (%) |  $\bar{x}$ mismatch (%)
---|---|---
Physical | $1.3\pm 0.2$ | $0.9\pm 0.1$
Logical (No QEC) | $5.7\pm 0.5$ | $2.9\pm 0.3$
Logical (with QEC) | $0.3\pm 0.1$ | $0.2\pm 0.1$
Table 1: Fidelity of GHZ state preparation in experiment. The second column
displays the percentage of runs when the four $Z$ measurement outcomes
disagreed. The third column displays the percentage of runs when the product
of the four $X$ measurements was $-1$. Standard errors for the data are also
shown.

Since the GHZ state is a stabilizer state with generators
$\\{Z_{1}Z_{2},Z_{2}Z_{3},Z_{3}Z_{4},X_{1}X_{2}X_{3}X_{4}\\}$, we can compute
simple upper and lower bounds on the GHZ fidelity of our experimental state
$\rho$. Because $\lvert\rm GHZ\rangle$ is a pure state, the fidelity Jozsa
(1994) takes the form

$\displaystyle F_{\rm GHZ}(\rho)=\langle\rm GHZ\rvert\rho\lvert\rm
GHZ\rangle\equiv\langle\rho\rangle_{\rm GHZ}\,.$ (2)

We now compute an upper bound for the experimental fidelities. First, notice
that the two projectors $P^{\pm}_{z}$ onto whether the $Z$ measurement
outcomes agree
($P^{+}_{z}=(\mathds{1}+Z_{1}Z_{2})(\mathds{1}+Z_{2}Z_{3})(\mathds{1}+Z_{3}Z_{4})/8$)
or disagree ($P^{-}_{z}=\mathds{1}-P^{+}_{z}$) form a projection-valued
measure that splits the Hilbert space into two orthogonal subspaces. Using the
fact that $\langle P^{+}_{z}\rangle_{\rm GHZ}=1$, i.e. the GHZ state strictly
lives in the $P^{+}_{z}$ subspace, we can upper bound the fidelity (2) by

$\displaystyle F_{\rm GHZ}(\rho)\leq{\rm tr}\big{(}P^{+}_{z}\rho\big{)}\,,$
(3)

which corresponds to the second column in Table 1. For the lower bound,
observe that the GHZ state also strictly resides in the subspace of
$P^{+}_{x}\equiv(\mathds{1}+X_{1}X_{2}X_{3}X_{4})/2$ such that its density
matrix takes the form
$\lvert\mathrm{GHZ}\rangle\langle\mathrm{GHZ}\rvert=P^{+}_{z}P^{+}_{x}$. We
can then use the union bound from probability theory to upper bound the
probabilities of being orthogonal to GHZ, which provides a lower bound to the
fidelity (2):

$\displaystyle F_{\rm GHZ}(\rho)\geq 1-{\rm
tr}\big{(}P^{-}_{z}\rho\big{)}-{\rm tr}\big{(}P^{-}_{x}\rho\big{)}\,.$ (4)

The first average corresponds to the fraction of runs where the $Z$ outcomes
disagree and is given by the second column in Table 1. The contribution from
the $XXXX$ generator is given by the third column. For the physical and
(error-corrected) logical states $\rho,\bar{\rho}$, we hence obtain fidelity
bounds of

$\displaystyle 97.8\pm 0.2\%$ $\displaystyle\leq F_{\rm GHZ}(\rho)\leq 98.7\pm
0.2\%$ (5a) $\displaystyle 99.5\pm 0.15\%$ $\displaystyle\leq F_{\rm
GHZ}(\bar{\rho})\leq 99.7\pm 0.1\%\,.$ (5b)

The statistical significance between the physical upper bound and the logical
lower bound is $\approx 3.6\sigma$, assuming statistically independent
uncertainties.

_Outlook._ — We have prepared a logical GHZ state on four qubits with a higher
fidelity than its physical counterpart. The key ingredient is a quantum LDPC
code whose long-range interactions enable (i) a compact encoding of logical
qubits and (ii) efficient logical entangling gates via qubit permutations.
Having demonstrated the ability to prepare specific quantum entangled logical
states beyond break-even, an immediate near-term experimental goal will be to
use our encoding (or others) to demonstrate break-even on larger circuits
involving multiple logical qubits, eventually including circuits with non-
Clifford gates.

Our work highlights an advantage of using quantum LDPC codes in existing
hardware over conventional topological codes like the surface and color codes.
Of course, a major feature of the topological codes is that they are equipped
with transversal implementations of the Clifford group. A big challenge in the
field is to construct nonlocal quantum LDPC codes with hardware-efficient
logical gates; some progress using hypergraph product codes has been
previously made Krishna and Poulin (2021); Quintavalle  _et al._ (2023). We
hope that this work stimulates further progress in this direction, especially
since it is already known that LDPC codes can significantly reduce the
overhead for fault-tolerant quantum computation when functioning just as a
memory block Cohen  _et al._ (2022). On a separate front, nonlocal codes
derived from concatenating quantum Hamming codes also show promise for low-
overhead fault tolerance Yamasaki and Koashi (2024); Yoshida  _et al._ (2024).
Since these codes are not LDPC, they require more complicated gadgets for
fault tolerance. A worthwhile future direction is to perform a detailed cost-
benefit analysis between LDPC and concatenated architectures on different
hardware.

_Acknowledgements._ — This work was supported in part by the Alfred P. Sloan
Foundation under Grant FG-2020-13795 (AL), by the Air Force Office of
Scientific Research under Grant FA9550-24-1-0120 (YH, AL), and by the Office
of Naval Research via Grant N00014-23-1-2533 (YH, AL). Additionally, we thank
the hardware team at Quantinuum for making these experiments possible.

## References

  * Kitaev (2003) A.Yu. Kitaev, “Fault-tolerant quantum computation by anyons,” Annals of Physics 303, 2–30 (2003).
  * Bombin and Martin-Delgado (2006) H. Bombin and M. A. Martin-Delgado, “Topological quantum distillation,” Phys. Rev. Lett. 97, 180501 (2006).
  * Dennis  _et al._ (2002) Eric Dennis, Alexei Kitaev, Andrew Landahl, and John Preskill, “Topological quantum memory,” Journal of Mathematical Physics 43, 4452–4505 (2002).
  * Landahl  _et al._ (2011) Andrew J. Landahl, Jonas T. Anderson, and Patrick R. Rice, “Fault-tolerant quantum computing with color codes,” (2011), arXiv:1108.5738 [quant-ph] .
  * Postler  _et al._ (2022) Lukas Postler, Sascha Heußen, Ivan Pogorelov, Manuel Rispler, Thomas Feldker, Michael Meth, Christian D. Marciniak, Roman Stricker, Martin Ringbauer, Rainer Blatt, Philipp Schindler, Markus Müller, and Thomas Monz, “Demonstration of fault-tolerant universal quantum gate operations,” Nature 605, 675–680 (2022), arXiv:2111.12654 [quant-ph] .
  * Ryan-Anderson  _et al._ (2021) C. Ryan-Anderson, J. G. Bohnet, K. Lee, D. Gresh, A. Hankin, J. P. Gaebler, D. Francois, A. Chernoguzov, D. Lucchetti, N. C. Brown, T. M. Gatterman, S. K. Halit, K. Gilmore, J. A. Gerber, B. Neyenhuis, D. Hayes, and R. P. Stutz, “Realization of real-time fault-tolerant quantum error correction,” Phys. Rev. X 11, 041058 (2021).
  * AI (2023) Google Quantum AI, “Suppressing quantum errors by scaling a surface code logical qubit,” Nature 614, 676–681 (2023).
  * Bluvstein  _et al._ (2023) Dolev Bluvstein, Simon J. Evered, Alexandra A. Geim, Sophie H. Li, Hengyun Zhou, Tom Manovitz, Sepehr Ebadi, Madelyn Cain, Marcin Kalinowski, Dominik Hangleiter, J. Pablo Bonilla Ataides, Nishad Maskara, Iris Cong, Xun Gao, Pedro Sales Rodriguez, Thomas Karolyshyn, Giulia Semeghini, Michael J. Gullans, Markus Greiner, Vladan Vuletić, and Mikhail D. Lukin, “Logical quantum processor based on reconfigurable atom arrays,” Nature 626, 58–65 (2023).
  * da Silva  _et al._ (2024) M. P. da Silva, C. Ryan-Anderson, J. M. Bello-Rivas, A. Chernoguzov, J. M. Dreiling, C. Foltz, F. Frachon, J. P. Gaebler, T. M. Gatterman, L. Grans-Samuelsson, D. Hayes, N. Hewitt, J. Johansen, D. Lucchetti, M. Mills, S. A. Moses, B. Neyenhuis, A. Paz, J. Pino, P. Siegfried, J. Strabley, A. Sundaram, D. Tom, S. J. Wernli, M. Zanner, R. P. Stutz, and K. M. Svore, “Demonstration of logical qubits and repeated error correction with better-than-physical error rates,” (2024), arXiv:2404.02280 [quant-ph] .
  * Gidney and Ekerå (2021) Craig Gidney and Martin Ekerå, “How to factor 2048 bit rsa integers in 8 hours using 20 million noisy qubits,” Quantum 5, 433 (2021).
  * Bravyi  _et al._ (2010) Sergey Bravyi, David Poulin, and Barbara Terhal, “Tradeoffs for reliable quantum information storage in 2d systems,” Phys. Rev. Lett. 104, 050503 (2010).
  * Breuckmann and Eberhardt (2021a) Nikolas P. Breuckmann and Jens Niklas Eberhardt, “Quantum low-density parity-check codes,” PRX Quantum 2 (2021a), 10.1103/prxquantum.2.040101.
  * Tillich and Zemor (2014) Jean-Pierre Tillich and Gilles Zemor, “Quantum ldpc codes with positive rate and minimum distance proportional to the square root of the blocklength,” IEEE Transactions on Information Theory 60, 1193–1202 (2014).
  * Hastings  _et al._ (2020) Matthew B. Hastings, Jeongwan Haah, and Ryan O’Donnell, “Fiber bundle codes: Breaking the $n^{1/2}\operatorname{polylog}(n)$ barrier for quantum ldpc codes,” (2020), arXiv:2009.03921 [quant-ph] .
  * Panteleev and Kalachev (2022a) Pavel Panteleev and Gleb Kalachev, “Quantum ldpc codes with almost linear minimum distance,” IEEE Transactions on Information Theory 68, 213–229 (2022a).
  * Breuckmann and Eberhardt (2021b) Nikolas P. Breuckmann and Jens N. Eberhardt, “Balanced product quantum codes,” IEEE Transactions on Information Theory 67, 6653–6674 (2021b).
  * Panteleev and Kalachev (2022b) Pavel Panteleev and Gleb Kalachev, “Asymptotically good quantum and locally testable classical ldpc codes,” (2022b), arXiv:2111.03654 [cs.IT] .
  * Dinur  _et al._ (2022) Irit Dinur, Min-Hsiu Hsieh, Ting-Chun Lin, and Thomas Vidick, “Good quantum ldpc codes with linear time decoders,” (2022), arXiv:2206.07750 [quant-ph] .
  * Leverrier and Zémor (2022) Anthony Leverrier and Gilles Zémor, “Quantum tanner codes,” (2022), arXiv:2202.13641 [quant-ph] .
  * Gottesman (2014) Daniel Gottesman, “Fault-tolerant quantum computation with constant overhead,” (2014), arXiv:1310.2984 [quant-ph] .
  * Fawzi  _et al._ (2018) Omar Fawzi, Antoine Grospellier, and Anthony Leverrier, “Constant overhead quantum fault-tolerance with quantum expander codes,” in  _2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS)_ (IEEE, 2018).
  * Baspin and Krishna (2022) Nouédyn Baspin and Anirudh Krishna, “Quantifying nonlocality: How outperforming local quantum codes is expensive,” Phys. Rev. Lett. 129, 050505 (2022).
  * Baspin  _et al._ (2023) Nouédyn Baspin, Omar Fawzi, and Ala Shayeghi, “A lower bound on the overhead of quantum error correction in low dimensions,” (2023), arXiv:2302.04317 [quant-ph] .
  * Bravyi  _et al._ (2024) Sergey Bravyi, Andrew W. Cross, Jay M. Gambetta, Dmitri Maslov, Patrick Rall, and Theodore J. Yoder, “High-threshold and low-overhead fault-tolerant quantum memory,” Nature 627, 778–782 (2024), arXiv:2308.07915 [quant-ph] .
  * Xu  _et al._ (2024) Qian Xu, J. Pablo Bonilla Ataides, Christopher A. Pattison, Nithin Raveendran, Dolev Bluvstein, Jonathan Wurtz, Bane Vasić, Mikhail D. Lukin, Liang Jiang, and Hengyun Zhou, “Constant-overhead fault-tolerant quantum computation with reconfigurable atom arrays,” Nature Physics (2024), 10.1038/s41567-024-02479-z.
  * Hong  _et al._ (2024) Yifan Hong, Matteo Marinelli, Adam M. Kaufman, and Andrew Lucas, “Long-range-enhanced surface codes,” (2024), arXiv:2309.11719 [quant-ph] .
  * Viszlai  _et al._ (2023) Joshua Viszlai, Willers Yang, Sophia Fuhui Lin, Junyu Liu, Natalia Nottingham, Jonathan M. Baker, and Frederic T. Chong, “Matching Generalized-Bicycle Codes to Neutral Atoms for Low-Overhead Fault-Tolerance,” (2023), arXiv:2311.16980 [quant-ph] .
  * Ryan-Anderson  _et al._ (2022) C. Ryan-Anderson, N. C. Brown, M. S. Allman, B. Arkin, G. Asa-Attuah, C. Baldwin, J. Berg, J. G. Bohnet, S. Braxton, N. Burdick, J. P. Campora, A. Chernoguzov, J. Esposito, B. Evans, D. Francois, J. P. Gaebler, T. M. Gatterman, J. Gerber, K. Gilmore, D. Gresh, A. Hall, A. Hankin, J. Hostetter, D. Lucchetti, K. Mayer, J. Myers, B. Neyenhuis, J. Santiago, J. Sedlacek, T. Skripka, A. Slattery, R. P. Stutz, J. Tait, R. Tobey, G. Vittorini, J. Walker, and D. Hayes, “Implementing fault-tolerant entangling gates on the five-qubit code and the color code,” arXiv:2208.01863 (2022).
  * Moses  _et al._ (2023) S. A. Moses, C. H. Baldwin, M. S. Allman, R. Ancona, L. Ascarrunz, C. Barnes, J. Bartolotta, B. Bjork, P. Blanchard, M. Bohn, J. G. Bohnet, N. C. Brown, N. Q. Burdick, W. C. Burton, S. L. Campbell, J. P. Campora, C. Carron, J. Chambers, J. W. Chan, Y. H. Chen, A. Chernoguzov, E. Chertkov, J. Colina, J. P. Curtis, R. Daniel, M. DeCross, D. Deen, C. Delaney, J. M. Dreiling, C. T. Ertsgaard, J. Esposito, B. Estey, M. Fabrikant, C. Figgatt, C. Foltz, M. Foss-Feig, D. Francois, J. P. Gaebler, T. M. Gatterman, C. N. Gilbreth, J. Giles, E. Glynn, A. Hall, A. M. Hankin, A. Hansen, D. Hayes, B. Higashi, I. M. Hoffman, B. Horning, J. J. Hout, R. Jacobs, J. Johansen, L. Jones, J. Karcz, T. Klein, P. Lauria, P. Lee, D. Liefer, S. T. Lu, D. Lucchetti, C. Lytle, A. Malm, M. Matheny, B. Mathewson, K. Mayer, D. B. Miller, M. Mills, B. Neyenhuis, L. Nugent, S. Olson, J. Parks, G. N. Price, Z. Price, M. Pugh, A. Ransford, A. P. Reed, C. Roman, M. Rowe, C. Ryan-Anderson, S. Sanders, J. Sedlacek, P. Shevchuk, P. Siegfried, T. Skripka, B. Spaun, R. T. Sprenkle, R. P. Stutz, M. Swallows, R. I. Tobey, A. Tran, T. Tran, E. Vogt, C. Volin, J. Walker, A. M. Zolot, and J. M. Pino, “A race-track trapped-ion quantum processor,” Phys. Rev. X 13, 041052 (2023).
  * Kielpinski  _et al._ (2002) D. Kielpinski, C. Monroe, and D. J. Wineland, “Architecture for a large-scale ion-trap quantum computer,” Nature 417, 709–711 (2002).
  * Home  _et al._ (2009) Jonathan P. Home, David Hanneke, John D. Jost, Jason M. Amini, Dietrich Leibfried, and David J. Wineland, “Complete methods set for scalable ion trap quantum information processing,” Science 325, 1227–1230 (2009), https://www.science.org/doi/pdf/10.1126/science.1177077 .
  * Pino  _et al._ (2020) J. M. Pino, J. M. Dreiling, C. Figgatt, J. P. Gaebler, S. A. Moses, M. S. Allman, C. H. Baldwin, M. Foss-Feig, D. Hayes, K. Mayer, C. Ryan-Anderson, and B. Neyenhuis, “Demonstration of the trapped-ion quantum-ccd computer architecture,” Nature (2020), 10.1038/s41586-021-03318-4, arXiv:2003.01293 [quant-ph] .
  * Kaushal  _et al._ (2020) V. Kaushal, B. Lekitsch, A. Stahl, J. Hilder, D. Pijn, C. Schmiegelow, A. Bermudez, M. Müller, F. Schmidt-Kaler, and U. Poschinger, “Shuttling-based trapped-ion quantum information processing,” AVS Quantum Science 2, 014101 (2020), https://pubs.aip.org/avs/aqs/article-pdf/doi/10.1116/1.5126186/19738817/014101_1_online.pdf .
  * Saffman (2016) M Saffman, “Quantum computing with atomic qubits and rydberg interactions: progress and challenges,” Journal of Physics B: Atomic, Molecular and Optical Physics 49, 202001 (2016).
  * Browaeys and Lahaye (2020) Antoine Browaeys and Thierry Lahaye, “Many-body physics with individually controlled Rydberg atoms,” Nature Physics 16, 132–142 (2020).
  * Kaufman and Ni (2021) Adam M Kaufman and Kang-Kuen Ni, “Quantum science with optical tweezer arrays of ultracold atoms and molecules,” Nature Physics 17, 1324–1333 (2021).
  * Evered  _et al._ (2023) Simon J. Evered, Dolev Bluvstein, Marcin Kalinowski, Sepehr Ebadi, Tom Manovitz, Hengyun Zhou, Sophie H. Li, Alexandra A. Geim, Tout T. Wang, Nishad Maskara, Harry Levine, Giulia Semeghini, Markus Greiner, Vladan Vuletić, and Mikhail D. Lukin, “High-fidelity parallel entangling gates on a neutral-atom quantum computer,” Nature 622, 268–272 (2023).
  * Gottesman (1997) Daniel Gottesman, “Stabilizer codes and quantum error correction,” (1997), arXiv:quant-ph/9705052 [quant-ph] .
  * Calderbank and Shor (1996) A. R. Calderbank and Peter W. Shor, “Good quantum error-correcting codes exist,” Phys. Rev. A 54, 1098–1105 (1996).
  * Steane (1996) Andrew Steane, “Multiple-particle interference and quantum error correction,” Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences 452, 2551–2577 (1996).
  * Manes and Claes (2023) Argyris Giannisis Manes and Jahan Claes, “Distance-preserving stabilizer measurements in hypergraph product codes,” (2023), arXiv:2308.15520 [quant-ph] .
  * Tomita and Svore (2014) Yu Tomita and Krysta M. Svore, “Low-distance surface codes under realistic quantum noise,” Phys. Rev. A 90, 062320 (2014).
  * Roffe  _et al._ (2020) Joschka Roffe, David R. White, Simon Burton, and Earl Campbell, “Decoding across the quantum low-density parity-check code landscape,” Phys. Rev. Res. 2, 043423 (2020).
  * Roffe (2022) Joschka Roffe, “LDPC: Python tools for low density parity check codes,”  (2022).
  * Cohen  _et al._ (2022) Lawrence Z. Cohen, Isaac H. Kim, Stephen D. Bartlett, and Benjamin J. Brown, “Low-overhead fault-tolerant quantum computing using long-range connectivity,” Science Advances 8 (2022), 10.1126/sciadv.abn1717.
  * Sørensen and Mølmer (1999) Anders Sørensen and Klaus Mølmer, “Quantum computation with ions in thermal motion,” Phys. Rev. Lett. 82, 1971–1974 (1999).
  * Jozsa (1994) Richard Jozsa, “Fidelity for Mixed Quantum States,” J. Mod. Opt. 41, 2315–2323 (1994).
  * Krishna and Poulin (2021) Anirudh Krishna and David Poulin, “Fault-tolerant gates on hypergraph product codes,” Phys. Rev. X 11, 011023 (2021).
  * Quintavalle  _et al._ (2023) Armanda O. Quintavalle, Paul Webster, and Michael Vasmer, “Partitioning qubits in hypergraph product codes to implement logical gates,” Quantum 7, 1153 (2023).
  * Yamasaki and Koashi (2024) Hayata Yamasaki and Masato Koashi, “Time-efficient constant-space-overhead fault-tolerant quantum computation,” Nature Physics 20, 247–253 (2024).
  * Yoshida  _et al._ (2024) Satoshi Yoshida, Shiro Tamiya, and Hayata Yamasaki, “Concatenate codes, save qubits,” (2024), arXiv:2402.09606 [quant-ph] .