2023

These authors contributed equally to this work. These authors contributed
equally to this work. [1,2]Sixian You

1]Research Laboratory of Electronics, MIT 2]Department of Electrical
Engineering and Computer Science, MIT 3]Newton-Wellesley Hospital, Mass
General Brigham 4]Department of Biological Engineering, MIT 5]Department of
Mechanical Engineering, MIT

# System- and Sample-agnostic Isotropic 3D Microscopy by Weakly Physics-
informed, Domain-shift-resistant Axial Deblurring

Jiashu Han Kunzan Liu Keith B. Isaacson Kristina Monakhova Linda G. Griffith
<EMAIL_ADDRESS>[  [  [  [  [

###### Abstract

Three-dimensional (3D) subcellular imaging is essential for biomedical
research, but the diffraction limit of optical microscopy compromises axial
resolution, hindering accurate 3D structural analysis. This challenge is
particularly pronounced in label-free imaging of thick, heterogeneous tissues,
where assumptions about data distribution (e.g. sparsity, label-specific
distribution, and lateral-axial similarity) and system priors (e.g.
independent and identically distributed (i.i.d.) noise and linear shift-
invariant (LSI) point-spread functions (PSFs)) are often invalid. Here, we
introduce SSAI-3D, a weakly physics-informed, domain-shift-resistant framework
for robust isotropic 3D imaging. SSAI-3D enables robust axial deblurring by
generating a PSF-flexible, noise-resilient, sample-informed training dataset
and sparsely fine-tuning a large pre-trained blind deblurring network. SSAI-3D
was applied to label-free nonlinear imaging of living organoids, freshly
excised human endometrium tissue, and mouse whisker pads, and further
validated in publicly available ground-truth-paired experimental datasets of
3D heterogeneous biological tissues with unknown blurring and noise across
different microscopy systems.

##  1 Introduction

Three-dimensional (3D) sub-cellular imaging is highly desirable in biomedical
research, as it reveals the intricate spatial organization of organelles,
cells, and biological networks. Achieving high-resolution sub-cellular imaging
across all three dimensions is crucial for accurately understanding complex
biological processes, such as neural circuits, disease pathogenesis, and
cellular responses to drugs gobel2007imaging ; wang2018three ;
cruz2019neutrophil ; olivier2010cell ; pulous2022cerebrospinal ; zhao2023two .
However, the fundamental diffraction limit of optical microscopy results in
axial resolution that is 2–5 times worse than lateral resolution. This
resolution disparity severely hinders isotropic 3D imaging, limiting our
ability to faithfully represent the true 3D structure of biological samples
and fully realize the potential of 3D imaging in scientific research
boden2021volumetric ; verveer2007high ; zipfel2003nonlinear .

While recent advances in microscopy theory and instrumentation enable
isotropic 3D imaging with certain specialized systems, their reliance on
specific hardware, contrast mechanisms, and/or sample preparation limits
widespread adoption, particularly for thick, living, and intact biosystems
planchon2011rapid ; aquino2011two ; jia2014isotropic ; chhetri2015whole ;
wu2021multiview ; zhao2022isotropic ; dean2022isotropic ; li2023three . To
make isotropic imaging more accessible across different microscopy modalities,
algorithmic approaches seek to address the inverse problem of axial
deblurring. In recent years, deconvolution methods based on deep neural
networks (DNN), given their effective and flexible learning of data priors,
have shown unprecedented performance compared to classical deconvolution
methods, such as Richardson-Lucy and fast iterative shrinkage thresholding
algorithm (FISTA), on two-dimensional (2D) deconvolution ren2020neural ;
khan2020flatnet ; whang2022deblurring ; yanny2022deep ; kohli2022ring .
However, axial resolution restoration is significantly more challenging since
acquiring isotropic 3D imaging data as ground truth is impractical for most
microscopes, especially in living tissues, making supervised learning methods
unsuitable for general isotropic 3D imaging. Despite the existence of a few
systems capable of collecting paired isotropic 3D imaging data and generously
sharing it wu2021multiview ; voigt2019mesospim , the potential for pre-
training a network and generalizing to diverse microscopes and samples is
hampered by the significant domain shifts between training and target domains
inherent in biomedical imaging moor2023foundation ; ma2024pretraining .

Domain shift is particularly pronounced in scientific imaging compared to
general computer vision hendrycks2016baseline ; hendrycks2021many ;
andreassen2022evolution , due to the high heterogeneity of microscopy
modalities, system properties, sample variation across species and tissues,
and the exploratory nature (unseen anomaly detection) of microscopy data. To
address these challenges and enable general isotropic 3D imaging, unsupervised
generative adversarial network (GAN)-based methods have been proposed to
enable deblurring without matched axial image pairs li2021unsupervised ;
park2022deep ; ning2023deep . However, the inherent instability of saddle-
point optimization in GAN-based methods leaves them vulnerable to network
collapse, potentially generating hallucinations and inconsistencies
yadav2018stabilizing ; cohen2018distribution ; de2021deep . Supervised methods
leverage similarities between lateral and axial data distributions and employ
explicit blurring forward models to achieve high-fidelity isotropic imaging
without paired in-distribution data weigert2018content ; chen2021three .
However, the practical applications of single-stack isotropic recovery using
supervised methods are limited for two primary reasons. First, these methods
rely on an explicit point-spread function (PSF) model that is assumed to be
precisely matched and consistent across the entire imaging volume. In reality,
factors such as field curvature, misalignment, and tissue scattering
contribute to optical aberrations, resulting in spatially varying PSFs that
are sample- and system-dependent, making real-time calibration difficult. This
modeling challenge is further exacerbated by noise, especially in low-light
conditions guo2020rapid . Second, self-supervised methods require assumptions
of lateral-axial similarity within single-stack data to learn axial deblurring
from synthetic lateral deblurring. While this assumption holds for certain
cell and tissue structures (e.g., neurons in the cortex and sinusoids in the
liver, as shown in prior axial deblurring works weigert2018content ;
park2022deep ; ning2023deep ), it breaks down in tissues with high
directionality, polarity, or anisotropy, which are common in biological
systems such as developing embryos, epithelial glands, and collagen fibers in
the extracellular matrix guo2023deep .

These challenges are particularly pronounced in label-free imaging of thick,
heterogeneous tissues liu2024deep , where assumptions about data distribution
(e.g. sparsity, label-specific distribution, and lateral-axial similarity) and
system priors (e.g. independent and identically distributed (i.i.d.) noise,
and linear shift-invariant (LSI) PSFs) are often invalid. To overcome these
challenges, we propose SSAI-3D (System- and Sample-agnostic Isotropic 3D
Microscopy), a weakly physics-informed, domain-shift-resistant axial
deblurring framework for robust isotropic recovery across diverse systems and
samples. First, we formulate isotropic recovery as a semi-blind deblurring
problem and leverage a pre-trained blind deblurring network developed for
natural scene images as the initial network. Then, we create a synthetic
training dataset by blurring denoised lateral images with various PSFs of
different sizes and orientations to adapt the deblurring network to the
current 3D imaging stack. This allows us to train the network in a self-
supervised manner, eliminating the need for pixel-wise co-registered data
pairs. We demonstrate that the combination of a denoised and PSF-varying
training dataset, coupled with leveraging existing knowledge from the
experimentally pre-trained blind deblurring network, significantly alleviates
susceptibility to spatially varying 3D blurring kernels and noise.

Secondly, we sparsely fine-tune the large pre-trained deblurring network using
this synthetic dataset. A surgeon network is employed to identify which layers
are most critical for adapting to microscopic images, while other layers
remain frozen to preserve the original deblurring capabilities learned from
the much larger dataset of natural scene image pairs. This approach leverages
prior knowledge of blind deblurring while efficiently adapting to microscopy
data and minimizing assumptions about lateral-axial similarity, thus reducing
overfitting and computational costs commonly associated with microscopy
deconvolution problems. We demonstrate high-fidelity single-stack axial
deblurring performance of SSAI-3D on open-source experimental datasets from
various microscopy systems: confocal (with near-isotropic validation through
hardware modification voigt2019mesospim ), light-sheet (with near-isotropic
validation through hardware modification wu2021multiview ), wide-field, and
nonlinear microscope you2018intravital ; liu2024deep . The high-fidelity
recovery across diverse microscopy systems (varying in aberrations, noise, and
contrasts) and samples (with different degrees of lateral-axial dissimilarity)
indicates that SSAI-3D can be a widely accessible and reliable tool for
isotropic imaging, facilitating precise analysis of complex 3D biological
structures and processes.

##  2 Results

###  2.1 Isotropic resolution recovery by self-supervised sparsely fine-tuned
deblurring

Figure 1: Principles of SSAI-3D. a, SSAI-3D enables robust isotropic
resolution recovery across diverse 3D imaging systems (confocal, light-sheet,
wide-field, and nonlinear) and diverse 3D biological samples (organelles,
cells, tissues, and organs). b, The deblurring network is initialized with a
large pre-trained network for blind deconvolution on a large dataset of
natural image pairs. c, Starting with a single microscopy-specific image stack
where the axial resolution is worse than the lateral resolution, lateral
images are blurred with a series of PSFs of different sizes and orientations
to generate a self-supervised dataset. Then, generating zero-shot metrics
using $\sim$1% of this dataset, a surgeon network is employed to select the
critical layers to fine-tune in the large pre-trained deblurring network. Only
$\sim$10% layers are selected and sparsely fine-tuned according to the
generated self-supervised dataset. Given unseen axial images, the fine-tuned
deblurring network predicts deblurred images with isotropic resolution. Insets
in the images represent the corresponding Fourier spectrums.

The SSAI-3D framework for single-stack isotropic resolution recovery consists
of two steps (Fig. 1c). First, a self-supervised training data is generated by
applying a series of PSFs with varying sizes and orientations to denoised
lateral images (see Methods for details). Unlike methods that rely on an exact
PSF model and thus have strong physics-based assumptions about the image
generation forward model, these sets of weakly-physics-based synthetic pairs
emphasize the general blurring action of the optical microscope rather than
the precise forward model, thereby increasing the network’s robustness against
unknown optical aberrations and noise.

Secondly, we leverage knowledge learned from natural scene deblurring tasks
via a large pre-trained deblurring network (Fig. 1b and Supplementary Fig.
S1a). We then sparsely fine-tune it using the created self-supervised dataset,
selecting and fine-tuning only $\sim$10% of the layers while freezing the
rest. Layer selection is guided by a surgeon network, which evaluates each
layer’s contribution to the adaptation task using zero-shot metrics, including
statistics related to the activation, gradient, loss function, and pre-trained
weights (see Methods for details and Supplementary Fig. S1b for architecture).
This surgeon network mitigates the stochastic instability of zero-shot
metrics, while sparse fine-tuning avoids performance degradation in out-of-
distribution data and the computational burden of full fine-tuning
lee2022surgical . Our approach preserves the deblurring capability learned
from the larger natural scene dataset, making only necessary adaptations to
the microscopy image stack. This results in a computationally and data-
efficient process that is less susceptible to overfitting to the lateral data
distribution or potential mismatches in the forward model. Finally, with these
two steps, in the inference stage, the axial images are sent to the adapted
customized deblurring network that predicts deblurred images with isotropic
resolution. Without strong assumptions about the imaging system or sample data
distribution, SSAI-3D is anticipated to be a widely accessible and robust tool
for isotropic 3D imaging across diverse microscopy modalities and a broad
spectrum of biological applications (Fig. 1a).

Figure 2: Performance of SSAI-3D on synthetic samples. Simulation of bead (a)
and strand structures (b) reveals the performance of SSAI-3D and existing
methods (c, d). The simulation objects are visualized using depth-coded
projections with the colorbar representing the actual depth. Reconstructed
axial resolution (mean$\pm$standard deviation) is labeled in c. e, f,
Characterization of reconstruction fidelity for beads (a) and strands (b). g,
h, Ablation study on the performance of different levels of blurring for beads
(a) and strands (b). i, j, Robustness of reconstruction against PSF mismatch
for beads (a) and strands (b). A blurring with a standard deviation of 5 is
assumed to be known for CARE. The standard deviation of the axial PSF on the
x-axis of (g–j) is also the times worse it is relative to the lateral PSF. k,
Comparison of the training time. Scale bars: 6 µm (a, c); 60 µm (b, d).

We first compared SSAI-3D to the state-of-the-art unsupervised (Self-Net
ning2023deep ) and supervised (CARE weigert2018content ) methods using two
distinct synthetic objects, assuming a known and LSI PSF across the imaging
volume. The first synthetic object comprised 50 randomly seeded sub-
diffraction-limited fluorescent beads within a 30 µm$\times$30 µm$\times$30 µm
imaging volume (Fig. 2a). Assuming a lateral resolution of 200 nm and axial
resolution of 700 nm for the imaging system, all three methods demonstrated
significant improvement in axial resolution (Fig. 2c). While Self-Net
exhibited greater variability in bead shape and intensity, both CARE and
SSAI-3D achieved reliable axial resolution recovery (201.4$\pm$2.7 nm and
200.7$\pm$2.9 nm, respectively).

However, many biological structures do not exhibit perfect lateral-axial
similarity like round beads. Therefore, we conducted a second simulation on a
more heterogeneous 3D synthetic structure, comprising 5000 directional strands
within a 300 µm$\times$300 µm$\times$300 µm imaging volume (Fig. 2b; see
Methods for details). This denser object with directionality better simulates
the complex 3D structures found in real biological samples, such as dendrites,
glands, and fibers (neural, elastin, and collagen). In contrast to the bead
simulation, existing methods exhibited significant performance degradation,
while SSAI-3D maintained high-fidelity resolution recovery with accurate shape
and intensity preservation (Fig. 2d and Supplementary Fig. S2). Quantitative
analysis further confirmed that, even with a known and LSI PSF, existing
algorithms significantly deteriorate in performance when applied to
directional, 3D asymmetric structures, as a result of their reliance on the
assumption of lateral-axial similarity. In contrast, SSAI-3D produced high-
fidelity restorations with low mean square error (MSE) (Fig. 2e and f) and
high structural similarity (SSIM) (Supplementary Fig. S3) across both
simulations. Ablation studies confirmed the consistent axial deblurring
fidelity of SSAI-3D across varying levels of blurring (Fig. 2g and h) and its
robustness to PSF mismatch (Fig. 2i and j; see Methods for details). In
addition, benefiting from sparse fine-tuning with only $\sim$15 million
parameters modified, SSAI-3D exhibited a 2.5$\sim$3.5-fold reduction in
training time compared to existing methods, thereby enhancing its
accessibility and reducing computational requirements (Fig. 2k).

###  2.2 Sparse fine-tuning on large pre-trained model relaxes assumptions on
lateral-axial similarity of 3D data

Figure 3: Effect of lateral-axial similarity of 3D data using simulations and
tissues. Simulation of sphere (high lateral-axial similarity, a) and cylinder
(low lateral-axial similarity, d) reveals the performance of SSAI-3D and
existing methods (c, f). b, e, Relationship between training MSE on lateral
images and inference MSE on axial images indicates robustness of different
methods against distribution shifts. In label-free nonlinear imaging,
biological tissues exhibit different levels of lateral-axial similarity (high
in living and intact human blood-brain barrier microfludic model (g) and low
in freshly excised human endometrial tissue (i)). h, j, Raw lateral and axial
images of g and i, as well as restored axial images using SSAI-3D. Arrows:
symmetric endothelial cells (h) and polarized epithelial glands (j). Scale
bars: 50 µm.

To understand the performance differences between the two data types (beads
vs. strands), we further investigated how lateral-axial data similarity
influences algorithm performance. We quantified this relationship by plotting
the accuracy on the reference lateral distribution against the accuracy on the
potentially shifted axial distribution ($n=9$) wortsman2022robust . In cases
of high lateral-axial similarity, existing methods and SSAI-3D aligned with
the identity line ($y=x$, Fig. 3b), indicating successful deblurring when the
distribution shift in the test axial datasets is negligible. However, when
lateral-axial similarity is low within the sample, CARE and SSAI-3D with full
fine-tuning exhibited a significant gap between training MSE on lateral images
and inference MSE on axial images. In contrast, SSAI-3D with sparse fine-
tuning demonstrated robustness to distribution shifts, resulting in high-
fidelity axial deblurring even in 3D heterogeneous samples
hendrycks2016baseline ; hendrycks2021many ; andreassen2022evolution  (Fig. 3e;
see methods and Supplementary Fig. S4 for details in full fine-tuning).

In this experiment, we considered a simulation of imaging spheres and
cylinders in 3D, representing cases of high (circles in both lateral and axial
planes, Fig. 3a) and low (circles in lateral, rectangles in axial planes, Fig.
3d) lateral-axial similarity, respectively. Consistent with the results in
Fig. 2, Self-Net, CARE, and SSAI-3D achieved comparable reconstructions under
high lateral-axial similarity (Fig. 3c). However, in the low similarity case,
performance diverged significantly, with SSAI-3D outperforming existing
methods (Fig. 3f). This divergence arises because existing methods heavily
rely on lateral-axial similarity, making them susceptible to significant
performance decreases when distribution shifts occur in real axial test data.
In contrast, the sparse fine-tuning in SSAI-3D preserves deblurring
capabilities learned from the natural scene dataset while adapting minimally
to the microscopy image stack, resulting in superior axial deblurring
performance for 3D heterogeneous samples under lateral-axial distribution
shifts.

In real-world biological imaging, lateral-axial similarity can vary
significantly depending on the sample’s structure and imaging scale. For
instance, neurons within a small volume may exhibit high lateral-axial
similarity, whereas glands, dendrites, and fibers (neural, elastin, collagen)
with high directionality, polarity, or anisotropy typically exhibit low
lateral-axial similarity. We demonstrated this sample-dependent 3D
heterogeneity using two thick and living samples imaged with label-free
nonlinear imaging liu2024deep . First, we imaged a living human blood-brain
barrier microfluidic model where vascularized endothelial cells formed lumen
structures with pericytes and astrocytes hajal2022engineered . This 3D self-
assembled multicellular model displayed high lateral-axial similarity (Fig.
3g). Secondly, we imaged a freshly excised human endometrial tissue, where
directional collagen growth and the inherent polarity of the epithelial glands
resulted in low lateral-axial similarity (Fig. 3i). SSAI-3D successfully
restored fine structures such as endothelial cells (Fig. 3h) and epithelial
glands (Fig. 3j) in axial views, even in cases of low lateral-axial
similarity. This robustness across samples with varying structural properties
highlights the potential of SSAI-3D as a widely applicable tool for 3D
biological imaging.

###  2.3 Generalizability to varying system imperfections and sample
heterogeneity in simulation and experiments

Figure 4:  Effect of optical aberrations and noise on deblurring performance
using simulations. a, Example of optical aberrations in real microscopy system
(spatially-varying PSF in mesoSPIM voigt2019mesospim ). b, Example of noise in
real microscopy system (low-light condition in dSLAM ye2023learned ). c,
Ground truth image for simulation on effect of optical aberrations and noise.
d–g, Four different levels of optical aberrations and noise with the
corresponding raw images (d, Shift-invariant PSF (ideal case); e, Shift-
variant PSF; f, Shift-variant PSF with rotations along $z$-direction; g,
Shift-variant PSF with rotations along $z$-direction and noise across the
entire imaging volume). h–k, Comparison of Self-Net, CARE, and SSAI-3D on
deblurring performance using SSIM in scenarios d–g. Bottom: Blow-ups of the
co-registered images within the white box. Scale bars: 2 mm (a); 25 µm (b–k).

Next, we investigate the robustness of SSAI-3D against various system
imperfections compared to existing methods. In practical 3D biomedical
imaging, imaging systems exhibit diverse PSFs in shape and size due to varying
signal generation mechanisms and customized instrumentation and acquisition
parameters. Furthermore, PSFs can be spatially varying across the imaging
volume due to factors such as aberration, misalignment, and tissue scattering
(Fig. 4a). Quantifying the precise PSFs for an arbitrary image stack is
infeasible since they are system- and sample-dependent. Additionally, many
imaging systems operate under photon-limited conditions due to biophysical and
biochemical constraints, such as the need for high imaging speed or to
minimize photobleaching, phototoxicity, and tissue heating li2023real . The
resulting low signal-to-noise ratio (SNR) images can significantly affect the
accuracy of deconvolution methods guo2020rapid  (Fig. 4b). Given these
challenges, we demonstrated the robustness of SSAI-3D against a spectrum of
system imperfections, including spatially varying PSFs and noise, by
leveraging a self-supervised dataset generated from PSFs with diverse sizes
and orientations.

We simulated 5000 randomly oriented tubular objects (high lateral-axial
similarity) within a 300 µm$\times$300 µm$\times$300 µm imaging volume (Fig.
4c). To evaluate axial deblurring performance, we initially applied a shift-
invariant axial blur, assumed to be known for CARE (Fig. 4d). In this ideal
scenario, Self-Net, CARE, and SSAI-3D all achieved high-quality resolution
recovery with low MSE (Fig. 4h). However, in practice, obtaining an accurate
PSF profile for the system is often challenging, and PSF mismatch can degrade
the performance of CARE (Fig. 2i and j). Next, we progressively introduced
system imperfections. Fig. 4e and f incorporated variations in the extent and
orientation of the axial blur. The non-uniform PSF profiles, combined with
asymmetry between lateral and axial aberrations, led to decreased accuracy for
Self-Net and CARE, particularly in regions with pronounced aberrations (Fig.
4i and j). To incorporate the effect of noise, we introduced both Gaussian and
Poisson noise to the images after artificial axial blurring (Fig. 4g). To
mitigate noise corruption and amplification during the forward blurring and
inverse deblurring processes, SSAI-3D incorporates a denoising step prior to
self-supervised dataset generation and the deblurring model, enhancing its
robustness against noise in low-light conditions (Fig. 4k and Supplementary
Fig. S5).

Next, we investigate the performance of SSAI-3D on real-life biological data
across various microscopy modalities, system properties, and sample types. As
demonstrated in Fig. 5, SSAI-3D achieved consistent isotropic resolution
recovery across a broad range of samples (living/fixed, human/animal, varying
degrees of lateral-axial similarity) and imaging systems (diverse contrast
mechanisms, commercial/custom-built, low/high resolution, low/high SNR). 3D
images with varying degrees of anisotropy, acquired using light-sheet,
confocal, wide-field, and nonlinear microscopy, all exhibited substantial
axial resolution enhancement after restoration with SSAI-3D within 0.5 GPU
hours of training (see Supplementary Table S1 and Methods for details).
Fourier spectrum analysis quantified this improvement, revealing previously
indistinguishable fine structures in low-resolution raw images, such as
dendrites in mouse brain neurons (Fig. 5c) and nucleoli in human endometrial
epithelial cells (Fig. 5f). Notably, the surgeon network selected different
layers for fine-tuning in each imaging stack (Supplementary Fig. S6). These
results highlight the potential of SSAI-3D as a generalizable and accessible
tool for 3D imaging, enhancing axial resolution to facilitate the study of 3D
tissue architecture and subcellular features.

Figure 5: SSAI-3D allows isotropic resolution recovery across different
imaging systems and samples. Each example includes the raw and restored images
(left), along with their respective blow-ups (highlighted in the white boxes,
middle) and Fourier spectrums (right, labeled numbers represent the lateral
and axial resolutions). a, Cleared and stained mouse brain vasculature from
light-sheet microscopy. b, Freshly excised human endometrial tissue from
multiphoton autofluorescence microscopy. c, Cleared and stained mouse brain
neurons from confocal microscopy. d, Fixed mouse whisker follicles from second
harmonic generation (SHG) microscopy. e, Cleared mouse liver from wide-field
microscopy. f, Freshly excised human endometrial tissue from third harmonic
generation (THG) microscopy. Scale bars: 20 µm.

### Validation of SSAI-3D with hardware-corrected isotropic imaging

Figure 6: Validated isotropic resolution recovery of SSAI-3D facilitates
downstream biological analysis. a, Comparison of software-corrected (SSAI-3D)
and hardware-corrected (mesoSPIM) axial image of mouse brain from light-sheet
microscopy. b, Neuron detection accuracy using raw, software-corrected, and
hardware-corrected images. Arrows in a and b are pointing to a same neuron. c,
Comparison of software-corrected (SSAI-3D) and hardware-corrected (multiview
confocal) axial image of mitochondria from confocal microscopy. d, Statistics
of the volume of DNA puncta in mitochondria. Inset: overall statistics of raw
data. Arrows in c represent the same DNA puncta with the volume marked in d.
Scale bars: 1 mm (a, b); 1 µm (c, d).

To experimentally validate the high-fidelity axial deblurring of SSAI-3D and
demonstrate its robustness to high sample heterogeneity and unknown system
imperfections for downstream biological analysis, we presented two comparative
studies against near-isotropic imaging references obtained via hardware
advancements. First, we assessed SSAI-3D’s isotropic resolution recovery in
light-sheet microscopy using co-registered raw images and a near-isotropic
reference acquired by mesoscale selective plane-illumination microscopy
(mesoSPIM) voigt2019mesospim  (Fig. 6a). A mouse brain was cleared, stained,
and imaged with mesoSPIM to visualize the vasoactive intestinal peptide
(VIP)-expressing interneurons pi2013cortical . At the mesoscale, the mouse
brain exhibited high lateral-axial dissimilarity, and system aberrations
varied significantly across the large field of view (FOV) (Fig. 4a). By
comparing against the near-isotropic hardware-corrected images, SSAI-3D
demonstrated high-fidelity 3D isotropic resolution recovery despite the high
lateral-axial dissimilarity and the spatially varying PSFs, which degraded the
performance of existing methods (please refer to Supplementary Fig. S7 for
method comparison). To evaluate the effect of SSAI-3D on downstream biological
analysis, we quantified the accuracy of neuron extraction and observed the
segmentation accuracy of SSAI-3D closely matched that of the near-isotropic
hardware-corrected mesoSPIM dataset (Fig. 6b; see Methods for detection
algorithms).

To further evaluate generalizability across diverse system imperfections and
sample geometries, we assessed SSAI-3D on a multi-modal confocal microscopy
dataset, in addition to the light-sheet microscopy data. This dataset
comprised co-registered raw images and a near-isotropic reference acquired by
multiview confocal super-resolution microscopy wu2021multiview  (Fig. 6c). An
expanded mitochondrion was imaged, with its outer membrane and DNA
immunolabeled by two distinct dyes. Notably, axial and lateral data
distributions, as well as those between channels, exhibited significant
dissimilarity. Applying SSAI-3D to each channel individually with customized
sparse fine-tuning, we found that only SSAI-3D achieved high-fidelity
restoration compared to existing methods (please refer to Supplementary Fig.
S8 for method comparison). Analysis of the detected DNA volume within the
image stack revealed that volume statistics from the SSAI-3D reconstruction
were highly consistent with those derived from the near-isotropic reference
(Fig. 6d).

## Discussion

SSAI-3D demonstrated robust isotropic resolution recovery across diverse real-
world microscopy systems and samples (Fig. 5). In particular, this approach
was applied to label-free nonlinear microscopy of intact, thick, and
heterogeneous tissues, where assumptions about data distribution (e.g.,
sparsity, label-specific distribution, axial-lateral similarity) and system
priors (e.g., i.i.d. noise, LSI PSFs) often fail. The axial deblurring
pipeline was further validated using publicly available, ground-truth-paired
experimental datasets featuring potentially high lateral-axial dissimilarity
in biological tissues, as well as unknown blurring and noise characteristics
across different microscopy systems (Fig. 6). SSAI-3D achieved this robust
axial deblurring by generating a PSF-varying, noise-resilient, sample-informed
training dataset and sparsely fine-tuning a large pre-trained blind deblurring
network. Compared to existing computational methods, SSAI-3D does not require
strong assumptions about the imaging system or sample data distribution, while
maintaining low computational cost. Complementary to hardware-corrected
isotropic imaging, SSAI-3D avoids the need for additional complex
instrumentation and sample pre-processing, thus enabling broader applicability
across biomedical applications, particularly for thick, intact, and living
biosystems.

Despite the robustness, the performance of SSAI-3D will likely degrade in
these scenarios. First, if the original axial resolution is more than 7 times
worse than the lateral resolution (roughly 2.5 µm lateral resolution assuming
a Gaussian beam), the recovery fidelity deteriorates gradually, as the
ablation experiment shown in Fig. 2i, j. Secondly, the denoising step has been
critical for high-fidelity axial deblurring, likely similar to the noise
amplification in the classical deconvolution problem. The use of SSAI-3D
should be cautionary when it comes to highly noisy datasets, where either more
acquisition time or a more effective denoising algorithm could help.

## Methods

### Self-supervised dataset generation

SSAI-3D employed a self-supervision strategy for learning isotropic
restoration using the raw 3D anisotropic data itself. The self-supervised
dataset was generated from the high-resolution lateral images, which were
first denoised and then artificially blurred by convolving with 25 PSFs that
combined five different sizes and five different orientations. For the
denoising step, we adopted the framework presented in our previous work
ye2023learned . To simulate an array of PSFs with varying sizes, Gaussian blur
was applied to the lateral images using kernels with standard deviations
ranging from 3 to 7 pixels. To simulate an array of PSFs with different
orientations, the kernels were rotated within the range of
$[-45^{\circ},45^{\circ}]$.

### Contribution of different layers in fine-tuning

To determine which layers to fine-tune for adapting to the data distribution
of a customized dataset, we assessed each layer’s contribution to the
performance of the network using zero-shot metrics. In this paper, we employed
the following 14 such metrics, obtainable via a single forward and backward
pass of the pre-trained network, to predict the performance of fine-tuning
each specific layer.

We denote the weights of the pre-trained network as $\bm{\Theta}$ and the loss
function as $\mathcal{L}$. For layer $l$, we denote the single pass forward
activation and the single pass backward gradient at this layer as
$\bm{A}^{(l)}$ and $\bm{G}^{(l)}$, respectively. Then, the 14 zero-shot
metrics $\bm{U}^{(l)}=[u_{1}^{(l)},...,u_{14}^{(l)}]$ of layer $l$ can be
represented as follows.

  * •

Metrics evaluating forward activation. These metrics aim to capture the
compatibility of pre-trained parameters with the customized dataset. The
metrics relate to activation values during the forward pass and include the
average, standard deviation, average of absolute values, and standard
deviation of absolute activation values:

$u_{1}^{(l)}=\mathbb{E}\left\\{\bm{A}^{(l)}\right\\},u_{2}^{(l)}=\text{std}\left\\{\bm{A}^{(l)}\right\\},u_{3}^{(l)}=\mathbb{E}\left\\{\left|\bm{A}^{(l)}\right|\right\\},u_{4}^{(l)}=\text{std}\left\\{\left|\bm{A}^{(l)}\right|\right\\}.$

  * •

Metrics evaluating backward gradient. These metrics aim to determine which
pre-trained model parameters are most in need of adjustment. The metrics
relate to the gradient of the model and include the average, standard
deviation, average of absolute values, and standard deviation of absolute
gradient values:

$u_{5}^{(l)}=\mathbb{E}\left\\{\bm{G}^{(l)}\right\\},u_{6}^{(l)}=\text{std}\left\\{\bm{G}^{(l)}\right\\},u_{7}^{(l)}=\mathbb{E}\left\\{\left|\bm{G}^{(l)}\right|\right\\},u_{8}^{(l)}=\text{std}\left\\{\left|\bm{G}^{(l)}\right|\right\\}.$

To evaluate the importance of each layer based on channel saliency, we also
employed the $\mathtt{fisher}$ metric theis2018faster . This metric assesses
the impact of removing activation channels (and their corresponding
parameters) that are estimated to have the least effect on the loss:

$u_{9}^{(l)}=\left(\bm{A}^{(l)}\odot\bm{G}^{(l)}\right)^{2}.$

  * •

Metrics related to the loss function. To evaluate the importance of each layer
based on their sensitivity to the loss with respect to the network inputs, we
employed the metric $\mathtt{snip}$ lee2018snip  as

$u_{10}^{(l)}=\left|{\frac{\partial\mathcal{L}}{\partial\bm{\Theta}}\odot\bm{\Theta}}\right|.$

To evaluate the importance of each layer based on their sensitivity to the
gradient (as opposed to loss in $\mathtt{snip}$) with respect to the network
inputs, we employed the metric $\mathtt{grasp}$ wang2020picking  as

$u_{11}^{(l)}=-\mathbb{H}\left\\{\frac{\partial\mathcal{L}}{\partial\bm{\Theta}}\right\\}\odot\bm{\Theta},$

where $\mathbb{H}\\{\cdot\\}$ denotes the Hessian matrix. To evaluate the
importance of each layer based on a loss function derived straightforwardly
from the product of all the layer parameters (no need for input data to
compute), we employed $\mathtt{synflow}$ tanaka2020pruning  as

$u_{12}^{(l)}=\frac{\partial\mathcal{L}}{\partial\bm{\Theta}}\odot\bm{\Theta}.$

  * •

Metrics related to the pre-trained network. We also included the metrics aim
to capture the inherent knowledge from the parameters of the pre-trained
network, including the average and the standard deviation:

$u_{13}^{(l)}=\mathbb{E}\left\\{\bm{\Theta}^{(l)}\right\\},u_{14}^{(l)}=\text{std}\left\\{\bm{\Theta}^{(l)}\right\\}.$

### Sparse fine-tuning with a surgeon network

Sparse fine-tuning was achieved by selecting specific layers in the large pre-
trained network for modification while freezing the remaining layers. To
efficiently select these layers based on the aforementioned zero-shot metrics,
we employed a small surgeon network. The surgeon network was a 6-layer multi-
layer perceptron (MLP) with 200k parameters (Supplementary Fig. S1). This
network took the zero-shot metrics $\bm{U}^{(l)}$, along with the positional
embedding $l$, as inputs and outputs a score $s^{(l)}$, representing the
contribution of layer $l$ in the pre-trained deblurring network. The layers
with the highest 10% scores were then selected for sparse fine-tuning. To pre-
train the surgeon network using ground truth labels, we created a meta-dataset
($\sim$1 GB) comprising 5 different stacks from a custom-built nonlinear
microscope liu2024deep . Using this meta-dataset, 300 input-output data pairs
were generated, following the format
$\left\\{\left[\bm{U}^{(l)},l\right],s^{(l)}\right\\}$ as defined above. In
training the surgeon network, we employed the Adam optimizer with a learning
rate of $10^{-4}$, a weight decay of $10^{-6}$, and the pairwise ranking loss,
as we were only interested in the relative performance of the layers. The pre-
trained surgeon network is released along with the codes.

### Architecture, training, and inference of deblurring network

The deblurring network employed a NAFNet topology with an encoder-decoder
architecture for end-to-end deconvolution chen2022simple . NAFNet is a pre-
trained network on real-world deblurring tasks, such as those in the GoPro
park2019down  and REDS liu2022video  datasets. Due to the inherent nature of
the method, the network was implemented in 2D, with input patches of
256$\times$256 and a batch size of 8. For all presented experiments, network
parameters were optimized using the Adam optimizer with a learning rate of
$10^{-3}$ and weight decay of $10^{-4}$. Training SSAI-3D took approximately
30 minutes on a single Nvidia GeForce RTX 3080 card, while inference for a
300$\times$300$\times$300 tensor took roughly 50 seconds. To minimize memory
consumption, the model was quantized to float16 precision. The employed
deblurring network contained approximately 100 million parameters. Only the
deconvolution network was trained, with $\sim$10% of the layers fine-tuned and
the remaining $\sim$90% frozen. In the inference stage, all layers were fixed,
and axial images were first denoised using the same denosing network and then
deconvolved to maintain high data resemblance with the training data.

### Simulation data generation

For the simulation of beads in Fig. 2a, the ground truth image volumes were
generated with 50 randomly located beads of random radius and intensity. The
isotropic beads were generated according to a 3D Gaussian distribution with
fixed standard deviations of 2.0 in all three dimensions. The volume for
training the model contained 50 beads in a 30 µm$\times$30 µm$\times$30 µm
volume, with 100 nm pixel size. For the simulation of symmetric tubular
strands in Fig. 4c, the ground truth image volumes contained strands facing
random directions at random locations, with a mean thickness of 4 pixels and
intensity values ranging from 15000 to 65535. The volume for training the
model contained 5000 tubes in a 300 µm$\times$300 µm$\times$300 µm volume. A
3D elastic grid-based deformation field was then applied to deform the volume,
creating the ground truth. Different blur levels were generated using a
51$\times$51$\times$51 3D Gaussian kernel, elongated axially to serve as raw
input, with a standard deviation of 0.5 in lateral directions and 4.0 in the
axial direction. For the simulation of asymmetric tubular strands in Fig. 2b,
the ground truth volume was generated via 3D Bézier curves, shaped as curves
in the lateral direction and straight horizontal lines axially. This created
semi-synthetic data with vastly different distributions across views. A total
of 5000 strands were generated in a 300 µm$\times$300 µm$\times$300 µm volume,
with a thickness of 1 to 4 pixels, a total length of around 1200 pixels, and
binary brightness values. Similar to the previous simulation, blur was
introduced using a 51$\times$51$\times$51 3D Gaussian kernel with anisotropic
standard deviations in lateral and axial directions.

### Sample preparation and imaging acquisition

3D imaging in living tissues was achieved using a custom-built nonlinear
microscope, as detailed in qiu2024spectral . The preparation of the living and
intact human blood-brain barrier microfluidic model (used in Fig. 3e) was
described in liu2024deep  and the preparation of freshly excised human
endometrial tissue (used in Fig. 5b and f) was described in ye2023learned . A
custom-built multimode fiber source at 1100 nm was employed for simultaneous
label-free imaging of THG, NAD(P)H, SHG, and FAD you2018intravital ;
liu2024deep . To achieve deep imaging with uniform SNR across different
depths, pulse energy was varied according to imaging depth, maintaining
approximately 3 nJ pulse energy at the excitation focus.

### Method comparison

We compared the performance of SSAI-3D with four baseline methods: Richardson-
Lucy, OT-CycleGAN park2022deep , Self-Net ning2023deep , and CARE
weigert2018content . These methods were all implemented by open-source codes
released by the cited papers. For CARE, the semi-synthetic dataset was created
by applying an explicit blurring and down-sampling model to degrade the high-
resolution lateral images to resemble the low-resolution axial images. In Fig.
2i and j, a Gaussian blur with a standard deviation of 5 was assumed to be
known for CARE. To assess robustness against PSF mismatch, the model trained
with this known blur was applied to different levels of blurring for CARE,
while a single model trained on varying blurs was used for SSAI-3D. In Fig. 6a
and c, where the blurring was spatially varying, a single blur within the
range of the axial degradation was selected. In Fig. 3b, Fig. 3e, and
Supplementary Fig. S4, SSAI-3D with full fine-tuning was implemented by fine-
tuning the entire pre-trained deblurring network without freezing any layers.
For all the experiments presented, the hyper-parameters and training settings
were universal.

### Evaluation metrics

The MSE used in the paper is defined as

$\text{MSE}=\frac{1}{N}\|\bm{X}-\bm{X}_{0}\|^{2},$

where $\bm{X}$ denotes the restored image, $\bm{X}_{0}$ denotes the
corresponding ground truth, and $N$ denotes the number of pixels. The SSIM
used in the paper is defined as

$\text{SSIM}=\frac{(2\mu\mu_{0}+C_{1})(2\sigma^{\prime}+C_{2})}{(\mu^{2}+\mu_{0}^{2}+C1)(\sigma^{2}+\sigma_{0}^{2}+C_{2})},$

where $\\{\mu,\mu_{0}\\}$ and $\\{\sigma,\sigma_{0}\\}$ are the means and
variances of $\bm{X}$ and $\bm{X}_{0}$, respectively. $\sigma^{\prime}$ is the
covariance of $\bm{X}$ and $\bm{X}_{0}$. The two constants
$C_{1}=(k_{1}L)^{2}$ and $C_{2}=(k_{2}L)^{2}$ are chosen with $k_{1}=0.01$ and
$k_{2}=0.03$. In Fig. 2c, the axial resolution after isotropic recovery was
measured by the full width at half maximum (FWHM) of the sub-diffraction-
limited beads in $z$-direction (mean$\pm$standard deviation, $n=50$). In Fig.
5, the resolution of the image was characterized by the width in its Fourier
spectrum. In Fig. 6b and d, the individual granules were detected by an
universal threshold of pixel value in raw, software-corrected, and hardware-
corrected images. The detection accuracy of image $\bm{X}$ was characterized
by the normalized cross-correlation (NCC) against hardware-corrected image
$\bar{\bm{X}}$ as

$\text{NCC}=\frac{\left(\bm{X}-\mathbb{E}\\{\bm{X}\\}\right)\odot\left(\bar{\bm{X}}-\mathbb{E}\\{\bar{\bm{X}}\\}\right)}{\sqrt{\left\|\bm{X}-\mathbb{E}\\{\bm{X}\\}\right\|^{2}\left\|\bar{\bm{X}}-\mathbb{E}\\{\bar{\bm{X}}\\}\right\|^{2}}}.$

## Data availability

The open-source datasets used in this paper can be found at
https://doi.org/10.5281/zenodo.7882519 ning2023deep  (Fig. 5a, c, and e),
https://idr.openmicroscopy.org/ voigt2019mesospim  (under accession number
idr0066, Fig. 6a), and https://zenodo.org/record/5495955 wu2021multiview
(Fig. 6b). Other datasets are available from the corresponding author upon
reasonable request.

## Code availability

The implementations of SSAI-3D, the pre-trained surgeon network, as well as
some examples will be publicly available.

## References

  * (1) Göbel, W., Kampa, B. M. & Helmchen, F.  Imaging cellular network dynamics in three dimensions using fast 3D laser scanning.  _Nat. Methods_ 4, 73–79 (2007).
  * (2) Wang, T. _et al._ Three-photon imaging of mouse brain structure and function through the intact skull.  _Nat. Methods_ 15, 789–792 (2018).
  * (3) Cruz Hernández, J. C. _et al._ Neutrophil adhesion in brain capillaries reduces cortical blood flow and impairs memory function in Alzheimer’s disease mouse models.  _Nat. Neurosci._ 22, 413–420 (2019).
  * (4) Olivier, N. _et al._ Cell lineage reconstruction of early zebrafish embryos using label-free nonlinear microscopy.  _Science_ 329, 967–971 (2010).
  * (5) Pulous, F. E. _et al._ Cerebrospinal fluid can exit into the skull bone marrow and instruct cranial hematopoiesis in mice with bacterial meningitis.  _Nat. Neurosci._ 25, 567–576 (2022).
  * (6) Zhao, Z. _et al._ Two-photon synthetic aperture microscopy for minimally invasive fast 3D imaging of native subcellular behaviors in deep tissue.  _Cell_ 186, 2475–2491 (2023).
  * (7) Boden, A. _et al._ Volumetric live cell imaging with three-dimensional parallelized RESOLFT microscopy.  _Nat. Biotechnol._ 39, 609–618 (2021).
  * (8) Verveer, P. J. _et al._ High-resolution three-dimensional imaging of large specimens with light sheet-based microscopy.  _Nat. Methods_ 4, 311–313 (2007).
  * (9) Zipfel, W. R., Williams, R. M. & Webb, W. W.  Nonlinear magic: Multiphoton microscopy in the biosciences.  _Nat. Biotechnol._ 21, 1369–1377 (2003).
  * (10) Planchon, T. A. _et al._ Rapid three-dimensional isotropic imaging of living cells using Bessel beam plane illumination.  _Nat. Methods_ 8, 417–423 (2011).
  * (11) Aquino, D. _et al._ Two-color nanoscopy of three-dimensional volumes by 4Pi detection of stochastically switched fluorophores.  _Nat. Methods_ 8, 353–359 (2011).
  * (12) Jia, S., Vaughan, J. C. & Zhuang, X.  Isotropic three-dimensional super-resolution imaging with a self-bending point spread function.  _Nat. Photon._ 8, 302–306 (2014).
  * (13) Chhetri, R. K. _et al._ Whole-animal functional and developmental imaging with isotropic spatial resolution.  _Nat. Methods_ 12, 1171–1178 (2015).
  * (14) Wu, Y. _et al._ Multiview confocal super-resolution microscopy.  _Nature_ 600, 279–284 (2021).
  * (15) Zhao, Y. _et al._ Isotropic super-resolution light-sheet microscopy of dynamic intracellular structures at subsecond timescales.  _Nat. Methods_ 19, 359–369 (2022).
  * (16) Dean, K. M. _et al._ Isotropic imaging across spatial scales with axially swept light-sheet microscopy.  _Nat. Protoc._ 17, 2025–2053 (2022).
  * (17) Li, X. _et al._ Three-dimensional structured illumination microscopy with enhanced axial resolution.  _Nat. Biotechnol._ 41, 1307–1319 (2023).
  * (18) Ren, D., Zhang, K., Wang, Q., Hu, Q. & Zuo, W.  Neural blind deconvolution using deep priors.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 3341–3350 (2020).
  * (19) Khan, S. S., Sundar, V., Boominathan, V., Veeraraghavan, A. & Mitra, K.  Flatnet: Towards photorealistic scene reconstruction from lensless measurements.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ 44, 1934–1948 (2020).
  * (20) Whang, J. _et al._ Deblurring via stochastic refinement.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 16293–16303 (2022).
  * (21) Yanny, K., Monakhova, K., Shuai, R. W. & Waller, L.  Deep learning for fast spatially varying deconvolution.  _Optica_ 9, 96–99 (2022).
  * (22) Kohli, A. _et al._ Ring deconvolution microscopy: An exact solution for spatially-varying aberration correction.  _arXiv preprint arXiv:2206.08928_ (2022).
  * (23) Voigt, F. F. _et al._ The mesoSPIM initiative: Open-source light-sheet microscopes for imaging cleared tissue.  _Nat. Methods_ 16, 1105–1108 (2019).
  * (24) Moor, M. _et al._ Foundation models for generalist medical artificial intelligence.  _Nature_ 616, 259–265 (2023).
  * (25) Ma, C., Tan, W., He, R. & Yan, B.  Pretraining a foundation model for generalizable fluorescence microscopy-based image restoration.  _Nat. Methods_ 1–10 (2024).
  * (26) Hendrycks, D. & Gimpel, K.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  In _International Conference on Learning Representations (ICLR)_ (2016).
  * (27) Hendrycks, D. _et al._ The many faces of robustness: A critical analysis of out-of-distribution generalization.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , 8340–8349 (2021).
  * (28) Andreassen, A. J., Bahri, Y., Neyshabur, B. & Roelofs, R.  The evolution of out-of-distribution robustness throughout fine-tuning.  _Transactions on Machine Learning Research_ (2022).
  * (29) Li, X. _et al._ Unsupervised content-preserving transformation for optical microscopy.  _Light: Science & Applications_ 10, 44 (2021).
  * (30) Park, H. _et al._ Deep learning enables reference-free isotropic super-resolution for volumetric fluorescence microscopy.  _Nat. Commun._ 13, 3297 (2022).
  * (31) Ning, K. _et al._ Deep self-learning enables fast, high-fidelity isotropic resolution restoration for volumetric fluorescence microscopy.  _Light: Science & Applications_ 12, 204 (2023).
  * (32) Yadav, A., Shah, S., Xu, Z., Jacobs, D. & Goldstein, T.  Stabilizing adversarial nets with prediction methods.  In _International Conference on Learning Representations (ICLR)_ (2018).
  * (33) Cohen, J. P., Luck, M. & Honari, S.  Distribution matching losses can hallucinate features in medical image translation.  In _Medical Image Computing and Computer Assisted Intervention (MICCAI)_ , 529–536 (Springer, 2018).
  * (34) de Haan, K. _et al._ Deep learning-based transformation of H&E stained tissues into special stains.  _Nat. Commun._ 12, 1–13 (2021).
  * (35) Weigert, M. _et al._ Content-aware image restoration: Pushing the limits of fluorescence microscopy.  _Nat. Methods_ 15, 1090–1097 (2018).
  * (36) Chen, J. _et al._ Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes.  _Nat. Methods_ 18, 678–687 (2021).
  * (37) Guo, M. _et al._ Rapid image deconvolution and multiview fusion for optical microscopy.  _Nat. Biotechnol._ 38, 1337–1346 (2020).
  * (38) Guo, M. _et al._ Deep learning-based aberration compensation improves contrast and resolution in fluorescence microscopy.  _bioRxiv_ (2023).
  * (39) Liu, K. _et al._ Deep and dynamic metabolic and structural imaging in living tissues.  _arXiv preprint arXiv:2404.11901_ (2024).
  * (40) You, S. _et al._ Intravital imaging by simultaneous label-free autofluorescence-multiharmonic microscopy.  _Nat. Commun._ 9, 2125 (2018).
  * (41) Lee, Y. _et al._ Surgical fine-tuning improves adaptation to distribution shifts.  In _The Eleventh International Conference on Learning Representations_ (2022).
  * (42) Wortsman, M. _et al._ Robust fine-tuning of zero-shot models.  In _Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition (CVPR)_ , 7959–7971 (2022).
  * (43) Hajal, C. _et al._ Engineered human blood-brain barrier microfluidic model for vascular permeability analyses.  _Nat. Protoc._ 17, 95–128 (2022).
  * (44) Ye, C. T. _et al._ Learned, uncertainty-driven adaptive acquisition for photon-efficient multiphoton microscopy.  _arXiv preprint arXiv:2310.16102_ (2023).
  * (45) Li, X. _et al._ Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit.  _Nat. Biotechnol._ 41, 282–292 (2023).
  * (46) Pi, H.-J. _et al._ Cortical interneurons that specialize in disinhibitory control.  _Nature_ 503, 521–524 (2013).
  * (47) Theis, L., Korshunova, I., Tejani, A. & Huszár, F.  Faster gaze prediction with dense networks and fisher pruning.  _arXiv preprint arXiv:1801.05787_ (2018).
  * (48) Lee, N., Ajanthan, T. & Torr, P.  SNIP: Single-shot network pruning based on connection sensitivity.  In _International Conference on Learning Representations_ (2018).
  * (49) Wang, C., Zhang, G. & Grosse, R.  Picking winning tickets before training by preserving gradient flow.  _arXiv preprint arXiv:2002.07376_ (2020).
  * (50) Tanaka, H., Kunin, D., Yamins, D. L. & Ganguli, S.  Pruning neural networks without any data by iteratively conserving synaptic flow.  _Advances in Neural Information Processing Systems (NIPS)_ 33, 6377–6389 (2020).
  * (51) Chen, L., Chu, X., Zhang, X. & Sun, J.  Simple baselines for image restoration.  In _European Conference on Computer Vision (ECCV)_ , 17–33 (2022).
  * (52) Park, D., Kim, J. & Chun, S. Y.  Down-scaling with learned kernels in multi-scale deep neural networks for non-uniform single image deblurring.  _arXiv preprint arXiv:1903.10157_ (2019).
  * (53) Liu, H. _et al._ Video super-resolution based on deep learning: A comprehensive survey.  _Artificial Intelligence Review_ 55, 5981–6035 (2022).
  * (54) Qiu, T. _et al._ Spectral-temporal-spatial customization via modulating multimodal nonlinear pulse propagation.  _Nat. Commun._ 15, 2031 (2024).
  * (55) Todorov, M. I. _et al._ Machine learning analysis of whole mouse brain vasculature.  _Nat. Methods_ 17, 442–449 (2020).

## Acknowledgements

We thank Honghao Cao and Tong Qiu for assistance in nonlinear imaging, as well
as helpful discussions from Matthew Yeung, Steven F. Nagle, and Haidong Feng.
We thank Song Han for insights into sparse fine tuning. We thank Roger D.
Kamm, Sarah Spitz, and Francesca Michela Pramotton for providing the human
blood-brain barrier microfludic model for nonlinear imaging. We thank Ellen L.
Kan for assistance in preparing the human endometrium tissue for nonlinear
imaging. We thank Fan Wang, Manuel Levy, and Eva Lendaro for providing the
mouse whisker pad tissue for nonlinear imaging. We acknowledge that Fig. 1a
contains materials from BioRender (https://biorender.com/). The work has been
supported by MIT startup funds and CZI Dynamic Imaging via Chan Zuckerberg
Donor Advised Fund (DAF) through the Silicon Valley Community Foundation
(SVCF). K.L. acknowledges support from the MIT Irwin Mark Jacobs (1957) and
Joan Klein Jacobs Presidential Fellowship.

## Author contributions

J.H., K.L., and S.Y. conceived the idea of the project. J.H. designed detailed
implementations in network training and inference. K.L. built the optical
setup and performed the imaging experiments. J.H. and K.L. analyzed the data
and prepared the figures with input from K.B.I., K.M., and L.G.G.. K.B.I. and
L.G.G. provided the samples for imaging and insights into the biomedical
questions. J.H., K.L., and S.Y. wrote the manuscript with the input from all
authors. S.Y. obtained the funding and supervised the research.

## Competing interests

The authors declare no competing interests.

## Supplementary information

Figure S1: Network architecture of the deblurring network (intra-block structure, a) and the surgeon network (b). The deblurring network is a large convolutional neural network (NAFNet chen2022simple ). The surgeon network is a small multi-layer perceptron (MLP) that takes the zero-shot metrics $\bm{U}^{(l)}$ and positional embedding $l$ and outputs a score $s^{(l)}$.  Figure S2: Comparison of co-registered intensity profiles in the ground truth image (e) with the raw axial image (a) as well as the resolution recovered image using Self-Net (b), CARE (c), and SSAI-3D (d). Scale bars: 60 µm. Figure S3: Fidelity of resolution recovery characterized by SSIM. a, b, SSIM of resolution recovery in Fig. 2c and d. c, d, Ablation study on the size of PSF for resolution recovery in Fig. 2c and d. Figure S4: Comparison of full fine-tuning and sparse fine-tuning in lateral training MSE against axial inference MSE in the same number of training epochs. a and b correspond to the simulation of spheres and cylinders in Fig. 3a and b, respectively. Figure S5: Denoising in SSAI-3D improves the robustness of deblurring against noise. a, Simulation of 3D imaging stack with noise. b, Deblurring result of CARE. c, Denoised image of a. d, Deblurring result of SSAI-3D after denoising. e, Ground truth image. Scale bars: 10 µm.  Figure S6: Contribution analysis shows the impact of fine-tuning different layers in the deblurring network. Different imaging stacks have different preferences on the layers to fine-tune, evidenced by the scores (output $s^{(l)}$ of the surgeon network). SHG, THG, and multiphoton autofluorescence correspond to images presented in Fig. 5b, d, and f, respectively.  Figure S7: Comparison of existing methods on the mesoSPIM dataset voigt2019mesospim . SSAI-3D exhibited higher fidelity in reconstruction and higher accuracy in downstream analysis (Neuron extraction in Fig. 6a and b). Scale bars: 1 mm.  Figure S8: Comparison of existing methods on the multiview confocal dataset wu2021multiview . SSAI-3D exhibited higher fidelity in reconstruction and higher accuracy in downstream analysis (Statistics of DNA volume in Fig. 6c and d). Scale bars: 5 µm. Table S1: Summary of 3D imaging data used in this work. Figure | System | Sample |  | Pixel number
---
($X,Y,Z$)
| Pixel size
---
($\text{d}x,\text{d}y,\text{d}z$) µm
2a | Simulation | Symmetric beads | (300, 300, 300) | (0.1, 0.1, 0.1)
2b | Simulation | Asymmetric strands | (900, 900, 900) | (0.33, 0.33, 0.33)
3a | Simulation | Symmetric spheres | (300, 300, 300) | (1.0, 1.0, 1.0)
3b | Simulation | Asymmetric cylinders | (300, 300, 300) | (1.0, 1.0, 1.0)
3e | Multiphoton |  | Living and intact
---
human blood-brain
barrier model liu2024deep
(700, 700, 250) | (0.5, 0.5, 2.0)
3f | Nonlinear |  | Freshly excise human
---
endometrium tissue
(512, 512, 60) | (0.4, 0.4, 1.2)
4 | Simulation | Symmetric strands | (300, 300, 300) | (1.0, 1.0, 1.0)
5a | Light sheet |  | Cleared and stained
---
mouse brain
vasculature todorov2020machine
(916, 916, 498) | (1.63, 1.63, 3)
5b | Multiphoton |  | Freshly excise human
---
endometrium tissue
(512, 512, 60) | (0.4, 0.4, 1.2)
5c | Confocal |  | Cleared and stained
---
mouse brain
neurons ning2023deep
(1024, 1024, 90) | (0.21, 0.21, 1.00)
5d | SHG |  | Fixed mouse whisker
---
pad tissue qiu2024spectral
(500, 500, 50) | (0.5, 0.5, 1.0)
5e | Wide-field |  | Cleared
---
mouse liver ning2023deep
(1800, 1700, 246) | (0.32, 0.32, 1.00)
5f | THG |  | Freshly excise human
---
endometrium tissue
(512, 512, 60) | (0.4, 0.4, 1.2)
6a | Light sheet |  | Cleared and stained
---
mouse brain voigt2019mesospim
(2048, 2048, 1937) | (6.4, 6.4, 4.8)
6c | Confocal |  | Expanded
---
mitochondria wu2021multiview
(530, 1396, 1396) | (0.013, 0.013, 0.016)