# 2.5D Multi-view Averaging Diffusion Model for 3D Medical Image Translation:
Application to Low-count PET Reconstruction with CT-less Attenuation
Correction

Tianqi Chen  Jun Hou  Yinchi Zhou  Huidong Xie  Xiongchao Chen  Qiong Liu
Xueqi Guo  Menghua Xia  James S. Duncan  Chi Liu  Bo Zhou
<EMAIL_ADDRESS>Department of Biomedical Engineering, Yale
University, New Haven, CT, USA  Department of Radiology and Biomedical
Imaging, Yale School of Medicine, New Haven, CT, USA  Department of Electrical
Engineering, Yale University, New Haven, CT, USA.  Department of Computer
Science, University of California Irvine, Irvine, CA, USA  Department of
Radiology, Northwestern University, Chicago, IL, USA.

###### Abstract

Positron Emission Tomography (PET) is an important clinical imaging tool but
inevitably introduces radiation hazards to patients and healthcare providers.
Reducing the tracer injection dose and eliminating the CT acquisition for
attenuation correction can reduce the overall radiation dose, but often
results in PET with high noise and bias. Thus, it is desirable to develop 3D
methods to translate the non-attenuation-corrected low-dose PET (NAC-LDPET)
into attenuation-corrected standard-dose PET (AC-SDPET). Recently, diffusion
models have emerged as a new state-of-the-art deep learning method for image-
to-image translation, better than traditional CNN-based methods. However, due
to the high computation cost and memory burden, it is largely limited to 2D
applications. To address these challenges, we developed a novel 2.5D Multi-
view Averaging Diffusion Model (MADM) for 3D image-to-image translation with
application on NAC-LDPET to AC-SDPET translation. Specifically, MADM employs
separate diffusion models for axial, coronal, and sagittal views, whose
outputs are averaged in each sampling step to ensure the 3D generation quality
from multiple views. To accelerate the 3D sampling process, we also proposed a
strategy to use the CNN-based 3D generation as a prior for the diffusion
model. Our experimental results on human patient studies suggested that MADM
can generate high-quality 3D translation images, outperforming previous CNN-
based and Diffusion-based baseline methods.

###### keywords:

Low-count PET, Attenuation Correction, Denoising, Diffusion Model, 3D Image
Translation

##  1 Introduction

Positron emission tomography (PET) is a vital functional imaging tool with
wide applications in oncology, cardiology, neurology, and biomedical research.
In clinical practice, PET imaging involves radioactive tracer injection to the
patients, with the dosage level carefully guided by the principle of As Low As
Reasonably Achievable (ALARA) [40]. In order to correct the attenuation of the
PET signal, PET is also acquired along with CT to provide an attenuation map
($\mu$-map) for PET attenuation correction (AC). This multi-modal imaging
procedure inevitably introduces radiation hazards to the patient and
healthcare providers. Thus, it is highly desirable to reduce the overall
radiation dose in this important imaging procedure. On one hand, the PET
radiation dose can be directly reduced by reducing the tracer injection dose.
However, the PET image will suffer from high noise and bias with the low-count
signal. On the other hand, while PET can be acquired alone without CT, the PET
image would inevitably suffer further increased quantification errors if CT-
based AC is not included. Thus, reconstruction of standard-count PET from low-
count PET and CT-less AC for PET are both important topics for reducing the
overall radiation dose in PET imaging, as illustrated in Figure 1.

Figure 1: Illustration of low-count/dose PET reconstruction with CT-less
Attenuation Correction (AC) for reducing the overall radiation dose in PET.

There are many previous works on low-count PET image denoising, and can be
summarized into two categories, including conventional image post-processing
[10, 29, 30] and deep learning (DL) based methods [44, 43, 28, 19, 14, 12, 52,
32, 3, 26, 45, 33]. Conventional image post-processing techniques, such as
Gaussian filtering, are standard techniques in PET reconstruction but hard to
preserve local structures. Non-local mean filter [10] and block-matching 4D
filter [29] were proposed to denoise low-dose PET while better preserving the
structural information. While these conventional image post-processing methods
may substantially improve the image quality, over-smoothing is commonly
observed in ultra-low-count settings. As the statistical characteristics of
noise in medical imaging are complex and hard to model, DL models can learn
the highly non-linear relationship from data and recover the original signal
from noise. Previous DL-based methods can be further divided into two
categories. The first category only uses the low-count PET data as input. For
example, Kaplan et al. [19] proposed using a GAN [13] with UNet [35] as the
generator to predict standard-count PET from low-count PET. Similarly, Wang et
al. [43] proposed using a 3D-conditional-GAN [16] also with UNet as the
generator to translate low-count PET to standard-count PET. Zhou et al. [52]
and Gong et al. [12] found incorporating Wasserstein GAN [1] can also achieve
promising low-count PET denoising performance. The second category uses low-
count PET and MR/CT as input. For example, Xiang et al. [44] proposed a deep
auto-context CNN that takes low-count PET image and T1 MR image as input for
prediction of standard-count PET. Similarly, Chen et al. [3] proposed to input
low-count PET along with multi-contrast MR images into a UNet [35] for ultra-
low-dose PET denoising. Compared to conventional PET denoising methods, all
these DL-based methods achieved superior denoising performance. However,
almost all of these previous methods were developed based on U-Net or its
variants[35, 16, 55]. Most importantly, they assumed CT-based AC is already
done for the low-count PET, thus CT-less AC for further reducing the radiation
was not considered.

Figure 2: The overall workflow of 2.5D Multi-view Averaging Diffusion Model
(MADM). MADM contains a one-step inference generative model(orange) and MAD-
BLK(grey). The MAD-BLK contains Three models in axial, sagittal, and coronal
view, and the output of each model will be averaged in the average block
before output.

[htb!]  $\textbf{Input: }x\in N^{d_{1}\times d_{2}\times d_{3}}$
$\textbf{Initialize \\#1: }t_{s}\in[0,T]\text{: the start timestep of
denoising process}$ $\textbf{Initialize \\#2: }s\text{: the number of adjacent
slices loaded as 2.5D condition input}$ $\textbf{Initialize \\#3:
}\epsilon\sim N(0,I)$ $y_{\text{prior}}=f_{\text{prior}}(x)$ ;

$\triangleright$ One-step inference to generate prior image
$\bar{y}_{\text{prior}}=\sqrt{\bar{a}_{t_{0}}}y_{\text{prior}}+\sqrt{1-\bar{a}_{t_{0}}}\epsilon$
;

$\triangleright$ Diffuse, i.e. add noise, to the prior image for
_$t=t_{s},t_{s}-1,...,0$_ do

$y^{\text{avg}}_{t}=\bar{y}_{\text{prior}}\text{ if }t==t_{s}$ ;

$\triangleright$ Start the reverse process at $t_{s}$ based on diffused prior
image $z\sim\mathcal{N}(0,\mathcal{I})\text{ if }t>1\text{, else }z=0$
for  _$i=1,2,...,d_{1}$_ do

$y^{\text{co}}_{t-1}[i,:,:]=\frac{1}{\sqrt{\alpha_{t}}}(y^{\text{avg}}_{t}[i,:,:]-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha_{t}}}}f^{co}_{\theta}(y^{\text{avg}}_{t}[i,:,:],x[i-s:i+s,:,:],t))+\sigma_{t}z$
;

$\triangleright$ Coronal view 2.5D reverse

for  _$i=1,2,...,d_{2}$_ do

$y^{\text{sa}}_{t-1}[:,i,:]=\frac{1}{\sqrt{\alpha_{t}}}(y^{\text{avg}}_{t}[:,i,:]-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha_{t}}}}f^{sa}_{\theta}(y^{\text{avg}}_{t}[:,i,:],x[:,i-s:i+s,:],t))+\sigma_{t}z$
;

$\triangleright$ Sagittal view 2.5D reverse

for  _$i=1,2,...,d_{3}$_ do

$y^{\text{ax}}_{t-1}[:,:,i]=\frac{1}{\sqrt{\alpha_{t}}}(y^{\text{avg}}_{t}[:,:,i]-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha_{t}}}}f^{ax}_{\theta}(y^{\text{avg}}_{t}[:,:,i],x[:,:,i-s:i+s],t))+\sigma_{t}z$
;

$\triangleright$ Axial view 2.5D reverse

$y^{\text{avg}}_{t-1}=\frac{y^{\text{co}}_{t-1}+y^{\text{ax}}_{t-1}+y^{\text{sa}}_{t-1}}{3}$
;

$\triangleright$ Averaging volumes from multiple 2.5D view models

On the other hand, there are also many previous efforts for CT-less AC for
PET. For time-of-flight (TOF) PET imaging data, Maximum Likelihood
Reconstruction of Activity and Attenuation (MLAA) algorithms [34] were first
developed to simultaneously reconstruct the tracer activity ($\lambda$-MLAA)
and the attenuation map ($\mu$-MLAA), based on the TOF PET raw data only.
Ideally, MLAA would solve the AC problem for TOF PET. However, the direct use
of $\mu$-MLAA still suffers from significant quantification error as compared
to the CT-based ($\mu$-CT) OSEM reconstruction [34, 37, 31], even in the
standard-count PET. With recent advancements in deep learning (DL), DL-based
CT-less AC methods have been extensively investigated to address this
challenge [20, 4]. The previous DL-based AC methods can be summarized into two
classes. The first category of methods uses either non-attenuation corrected
(NAC) PET or MLAA reconstructions as deep network input to generate CT-based
$\mu$-map [37, 41, 24, 25, 15, 9]. Then, AC is performed based on the
generated $\mu$-map. Similarly, the second category of methods also uses
either NAC PET or MLAA reconstructions as deep network input, but to directly
generate attenuation-corrected PET [38, 8, 39]. However, almost all of these
previous methods were developed based on U-Net or its variants[35, 16, 55] and
only validated on standard-count PET. Even though these approaches achieved
reasonable AC performance on standard-count PET, directly deploying these
methods for low-count PET AC may result in non-ideal performance given the
low-quality PET input. Additional steps for low-count PET denoising also need
to be considered even after AC is performed.

While reconstruction of standard-count PET from low-count PET task and CT-less
AC for standard-count PET task were both extensively studied in prior works,
low-count PET reconstruction with CT-less AC that can holistically reduce
radiation dose has not been investigated before. Directly deploying previous
DL models may lead to non-ideal translation results since the low-count PET
image without AC not only suffers from high noise levels but also significant
bias due to attenuation. In recent years, the Diffusion Model (DM) has emerged
as the new state-of-the-art image generation method and has shown promising
results in image-to-image translation. However, there are two major issues
with DM. First, these methods were mainly developed for 2D image data, but not
3D data like PET. Translating 3D images in a slice-by-slice 2D manner will
lead to inconsistency from another view [45, 21, 6]. Even though it is
possible to directly extend the 2D network into the 3D network in the
diffusion model for 3D translation, the computation burden and memory
requirement make it highly infeasible with the current hardware technique,
especially when it is applied to high-resolution 3D imaging data like PET.
Second, the translation speed is significantly slower than previous CNN-based
approaches, due to the large number of diffusion sampling steps required. This
is further exacerbated if DM is directly extended to 3D imaging data like PET.

To address these issues, we developed a novel 2.5D Multi-view Averaging
Diffusion Model (MADM) for 3D medical image translation with the first
application on low-count PET reconstruction with CT-less AC. There are three
main contributions of MADM. First, to reduce the computational demands
associated with 3D models and to mitigate the discontinuities inherent in 2D
models, we proposed to use a 2.5D model that incorporates several adjacent
slices in addition to the target slice for prediction. Second, to further
minimize discontinuities and enhance prediction accuracy, we proposed to train
separate 2.5D models for each view in 3D, i.e. axial, coronal, and sagittal
views. During the inference, we average the result in each denoising step of
the diffusion process. Third, to expedite the translation process and refine
generated image accuracy, we proposed to utilize 3D predictions from a
Convolutional Neural Network (CNN)-based model as a prior for our diffusion
model. Our experimental results show that MADM can generate high-quality AC
standard-count PET(AC-SDPET) directly from the NAC low-count PET(NAC-LDPET) in
3D with reasonable computation resources, and also surpassing several previous
baselines.

##  2 Methods

###  2.1 Cascaded Multi-path Shortcut Diffusion Model

The general pipeline of the 2.5D Multi-view Averaging Diffusion Model (MADM)
is depicted in Figure 2. During both training and sampling processes, MADM
employs a multifaceted approach: first, we use a CNN-based one-step inference
generative model to create a 3D prior image, then we use multiple conditional
2.5D diffusion models in axial, coronal, and sagittal view and average the
outputs from different views in each denoising step to further refine the 3D
prior image. The following section details the training and inference
methodologies.

###  2.2 2.5D Multi-view Averaging Diffusion

We denote the NAC-LDPET input image as $x\in N^{d_{1}\times d_{2}\times
d_{3}}$ and the target AC-SDPET image as $y\in N^{d_{1}\times d_{2}\times
d_{3}}$ where $d_{1},d_{2},d_{3}$ is the width, depth, and hight of the 3D
image.
Training: To generate the prior image for MADM, we first need to train a one-
step inference generative model $f_{\text{prior}}(\cdot)$, i.e. U-Net [35]
here, which predicts $y_{0}$ from $x$. This model can be trained using the L2
loss

$\mathcal{L}_{v}=||f_{\text{prior}}(x)-y||^{2}_{2},$ (1)

On the other hand, the diffusion process tries to predict the distribution
$y_{0}\sim q(y)$. Then we have a forward diffusion process which adds noise
$\epsilon$ to $y_{0}$ for $T$ steps:

$q(y_{t}|y_{t-1})=\mathcal{N}(y_{t};\sqrt{1-\beta_{t}}y_{t-1},\beta_{t}I),$
(2)

and

$q(y_{1:T}|y_{0})=\prod^{T}_{t=1}q(y_{t}|y_{t-1}),$ (3)

where we let ${\beta_{t}\in(0,1)}^{T}_{t=1}$ and $\beta_{t}<\beta_{t+1}$.
Following that, we can sample $y_{t}$ for any $t\in[0,T]$ using:

$y_{t}=\sqrt{\bar{\alpha}_{t}}y_{0}+\sqrt{1-\bar{\alpha}_{t}}\epsilon,$ (4)
$q(y_{t}|y_{0})=\mathcal{N}(y_{t};\sqrt{\bar{\alpha}_{t}}y_{0},(1-\bar{\alpha}_{t})I),$
(5)

where $\epsilon\sim N(0,I)$, $\alpha_{t}=1-\beta_{t}$, and
$\bar{\alpha_{t}}=\prod^{t}_{i=1}\alpha_{i}$. Then in the reverse diffusion
process(denoising process), we want to sample $y_{0}$ from
$y_{T}\sim\mathcal{N}(0,I)$, thus we need a model $p_{\theta}$ to run this
reverse diffusion process which

$p_{\theta}(y_{t-1}|y_{t})=\mathcal{N}(y_{t-1};\mu_{\theta}(y_{t},x,t),\beta_{t}I),$
(6)

and

$p_{\theta}(y_{0:T})=p(y_{T})\prod^{T}_{t=1}p_{\theta}(y_{t-1}|y_{t}),$ (7)

where

$\mu_{\theta}(y_{t},x,t)=\frac{1}{\sqrt{\alpha_{t}}}(y_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha}_{t}}}f_{\theta}(y_{t},x,t)),$
(8)

Therefore we can use model $f_{\theta}(\cdot)$ to predict $y_{t-1}$ from
$y_{t}$ given the NAC-LDPET input $x$. In our MADM, instead of using a whole
3D $x$ as the conditional input, we use 2.5D slices $x^{v}$ cropped $(2s+1)$
adjacent slices from $x$ in different views
$v\in\\{\text{coronal},\text{sagittal},\text{axial}\\}$.

To train the conditional diffusion model $f^{v}_{\theta}(\cdot)$ separately in
coronal, sagittal, and axial view, we use pixel-wise L2 loss

$\mathcal{L}_{v}=||f_{\theta}^{v}(y_{t}^{v},x^{v},t)-\epsilon||^{2}_{2},$ (9)

where $t=0,1,...,T$ is the timestep in diffusion process.
Sampling:After the loss of conditional diffusion models $f_{\theta}^{v}$ and
$f_{\text{prior}}$ converged in training, we use $f_{\text{prior}}$ to predict
the $y_{\text{prior}}$ from $x$

$y_{\text{prior}}=f_{\text{prior}}(x).$ (10)

Then we add noise to the $y_{\text{{prior}}}$ to the same noise level $t_{s}$
in diffusion process by

$\bar{y}_{\text{prior}}=\sqrt{\bar{a}_{t_{s}}}y_{\text{prior}}+\sqrt{1-\bar{a}_{t_{s}}}\epsilon,$
(11)

where $t_{s}\in[0,T]$. Following the DDPM process, the noisy image in $t-1$
for the step $t$ can be predicted using the conditional diffusion models

$y^{\text{v}}_{t-1}=\frac{1}{\sqrt{\alpha_{t}}}(y^{v}_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha_{t}}}}f^{v}_{\theta}(y^{v}_{t},x^{v},t))+\sigma_{t}z,$
(12)

where $y_{t_{s}}=\bar{y}_{\text{prior}}$ when $t=t_{s}$. After predicting the
$y_{t-1}^{v}$ in three views, our method averages the three predictions using

$y^{\text{avg}}_{t-1}=\frac{y^{\text{co}}_{t-1}+y^{\text{ax}}_{t-1}+y^{\text{sa}}_{t-1}}{3},$
(13)

to get the final prediction for step $t-1$ then repeat the denoise process
using equation 12 and 13 from step $t_{s}$ to $0$ to get the final result
$y_{0}=y_{0}^{avg}$. The algorithm is summarized in Algorithm 1.

Figure 3: Visual comparison of AC-SDPET generation from different methods
under 5% NAC-LDPET settings. The coronal view image(top) and error map(bottom)
are shown. RMSE and PSNR values are calculated for each individual volume. The
CT-based AC-SDPET(top) and image index in to the color map of the error
map(bottom) are shown in the first column. Figure 4: Visual comparison of AC-
SDPET generation from different methods under 10% NAC-LDPET settings. The
coronal view image(top) and error map(bottom) are shown. RMSE and PSNR values
are calculated for each individual volume. The CT-based AC-SDPET(top) and
image index in to the color map of the error map(bottom) are shown in the
first column. Table 1: Quantitative comparison of AC-SDPET generation from
different methods under two different low-count settings using PSNR, SSIM, and
RMSE. Best results are marked in bold.

Evaluation | 5% Low-count | 10% Low-count |  GPU Memory Usage
---|---|---|---
PSNR | SSIM | RMSE | PSNR | SSIM | RMSE | Training(MB) |  Sampling(MB)
NAC-LD | $29.385\pm 1.440$ | $0.9489\pm 0.0147$ | $0.2666\pm 0.0621$ | $30.049\pm 1.442$ | $0.9544\pm 0.0133$ | $0.2293\pm 0.0561$ | - |  \-
UNet[36] | $38.101\pm 1.423$ | $0.9936\pm 0.0021$ | $0.0360\pm 0.0096$ | $38.869\pm 1.384$ | $0.9946\pm 0.0018$ | $0.0301\pm 0.0082$ | 10014 |  10460
cGAN[17] | $38.295\pm 1.467$ | $0.9938\pm 0.0021$ | $0.0344\pm 0.0087$ | $39.045\pm 1.454$ | $0.9948\pm 0.0018$ | $0.0289\pm 0.0076$ | 11440 |  13020
3D-DDPM[2] | $37.197\pm 1.271$ | $0.9924\pm 0.0026$ | $0.0437\pm 0.0085$ | $37.541\pm 1.189$ | $0.9930\pm 0.0022$ | $0.0403\pm 0.0079$ | 68874 |  23798
TPDM[21] | $36.818\pm 1.461$ | $0.9916\pm 0.0029$ | $0.0485\pm 0.0150$ | $37.463\pm 1.478$ | $0.9925\pm 0.0027$ | $0.0419\pm 0.0137$ | 6442 |  3012
MADM | $\mathbf{38.920\pm 1.503}$ | $\mathbf{0.9944\pm 0.0020}$ | $\mathbf{0.0299\pm 0.0088}$ | $\mathbf{39.453\pm 1.487}$ | $\mathbf{0.9952\pm 0.0017}$ | $\mathbf{0.0265\pm 0.0083}$ | 6442 |  4464

###  2.3 Data Preparation and Implementation Details

We collected a real patient dataset from the Yale New Haven Hospital (YNHH)
for training and evaluation of our method. The data was acquired using a
Siemens Biograph mCT scanner at YNHH. PET scans were performed approximately
60 mins after intravenous injection of about 10 mCi 18F-FDG tracer, with
whole-body continuous bed scanning protocol. We used uniform down-sampling of
the PET list-mode data with down-sampling ratios of 5% and 10% to generate
low-count PET data at two different low-count levels. To generate the non-AC
low-count PET image (i.e. NAC-LDPET), we used the down-sampled PET list-mode
data and performed reconstruction using the ordered-subsets expectation
maximization (OSEM) algorithm with 2 iterations and 21 subsets without AC. On
the other hand, to generate the AC full-count PET image (i.e. AC-SDPET), we
used the original PET list-mode data without down-sampling and performed the
same OSEM protocol but with AC. For both reconstructions, a post-
reconstruction Gaussian filter with $5mm$ full width at half maximum (FWHM)
was used. The voxel size of the reconstructed image was $4.08\times 4.08\times
4.06mm^{3}$. The image size was $200\times 200$ in the transverse plane and
varied in the axial direction depending on the patient’s height. The 147
subjects were split into 120 subjects for training and 27 subjects for
evaluation. We trained and implemented all models on the NVIDIA H100 GPUs. We
trained our MADM model with a batch size of 50 for 200K training step with the
$lr=1\times 10^{-4}$ and linearly decrease to 0. We used the EMA rate of
0.9999. We used the linear schedule of 1000 time steps.

###  2.4 Evaluation Strategies and Baselines

Using the AC-SDPET as the reference, we evaluated the 3D translation
performance both quantitatively and qualitatively. First, the quality of the
translated images was evaluated at the image level using three image-quality
metrics, including Peak Signal-to-Noise Ratio (PSNR), Structural Similarity
Index (SSIM), and Root Mean Square Error (RMSE). SSIM focuses on the
evaluation of structural recovery, while RMSE with a unit of SUV and PSNR with
a unit of dB stress the evaluation of intensity profile recovery. Second, the
quality of the translated PET was evaluated at clinically important local
lesion regions throughout the whole body. Specifically, the difference in
tumors’ SUV mean was analyzed. For comparative evaluation, we compared our
results against previous state-of-the-art DL-based image translation methods,
including UNet-based methods [35], GAN-based methods [16], and Diffusion-based
methods [21, 2]. For both quantitative and qualitative evaluations, we
evaluated the performance under two different ultra-low-dose/count settings,
i.e. 10% and 5% of PET full counts.

##  3 Experimental Results

In Figure 3, we show the qualitative comparison between our method and
previous state-of-the-art methods for a patient example under the $5\%$ low-
count PET scenario. As we can see, the original NAC-LDPET not only suffers
from high noise levels but also highly biased quantification errors due to the
attenuation effect, thus resulting in a low PSNR of $28.531$ dB. The error map
computing the difference between it and the AC-SDPET, i.e. ground truth,
further demonstrates this issue. Using previous CNN methods, i.e. UNet [36]
and GAN [16], we can see that these methods can generate reasonable AC-SDPET
with PSNR reach to $36.73$ dB. However, relatively high errors were found in
the liver and rib regions. On the other hand, we can see that previous
diffusion methods, i.e. 3D-DDPM [2] and TPDM[21], can also generate reasonable
AC-SDPET from NAC-LDPET. However, the performance in terms of PSNR and RMSE is
slightly lower than the CNN-based methods. In the right column, our MADM that
uses the CNN-based method’s generation as prior and further refined using a
multi-view 2.5D diffusion model generates the most consistent AC-SDPET as
compared to ground truth, with further reduced error in major organs, e.g.
liver and bone.

In Figure 4, we present another example of AC-SDPET generated from $10\%$ low-
count PET, comparing our method with previous methods. This example focuses on
the tumor regions, which are zoomed in and highlighted with an orange arrow in
Figure 4. Compared to other CNN-based and diffusion-based methods, our MADM
not only achieves higher PSNR and lower NMSE quantitatively, but also shows a
smaller error in the tumor region, as evidenced by the error map. Although
both state-of-the-art methods and our MADM exhibit larger errors in the tumor
region, our MADM significantly reduces errors in the surrounding areas. This
reduction in error provides a substantial benefit for lesion masking and
further analysis.

The quantitative comparisons were summarized in Table 1. Similar to the
observations from the visualizations, we can see that the traditional CNN-
based approaches can slightly outperform the diffusion-based approach in our
3D scenario. For example, the cGAN[17] achieved the best performance among all
the baselines with PSNR = $38.295$ dB under the $5\%$ low-count scenario,
while the best previous diffusion method, i.e. 3D-DDPM[2], only achieved PSNR
= $37.197$ dB. Our method that combines the CNN and Diffusion achieved
consistently better results with PSNR = $38.920$ dB than all the previous
methods.

Table 2: Quantitative comparison of AC-SDPET generated from NAC-LDPET data.
Best results are marked in bold.

Lesion |  Error - SUV mean
---|---
Quantification | 5% Low-count |  10% Low-count
NAC-LD | $0.1540\pm 0.0710$ |  $0.1517\pm 0.0740$
UNet [36] | $0.0989\pm 0.0394$ |  $0.0855\pm 0.0451$
cGAN [17] | $0.0894\pm 0.0356$ |  $0.0805\pm 0.0393$
3D-DDPM [2] | $0.0907\pm 0.0363$ |  $0.0874\pm 0.0368$
TPDM [21] | $0.0922\pm 0.0359$ |  $0.0911\pm 0.0399$
MADM | $\mathbf{0.0787\pm 0.0391}$ |  $\mathbf{0.0736\pm 0.0403}$

The quantitative comparisons of local tumor regions were summarized in Table
2. Similar to the observations from the above global quality analysis, we
found that NAC-LDPET suffers from significant lesion quantification errors
thus resulting in an averaged SUV mean error over $0.15$ in terms of RMSE
under the $5\%$ condition. Even though previous methods can significantly
reduce lesion quantification errors, the performance reaches its limit at
approximately $0.09$. On the other hand, our MADM can further reduce this
error to below $0.08$ which significantly improves the lesion quantification.

###  3.1 Ablative Studies

Multi-view Averaging Strategies in MADM was studied and summarized in Table 3.
Specifically, we investigated MADM’s performance when different single view or
combination of views strategies were implemented in MADM. As we can see,
single-view strategies (first three rows) show consistently lower performance
than the two-view combination strategies (fourth to seventh rows). MADM with
all three views, i.e. axial, coronal, and sagittal, yields superior
performance as compared to MADM with only one or two views.

Table 3: Ablative studies on the inclusion of multi-view averaging strategies in MADM. ✓ and ✗ means used and not used the model of the view in MADM, respectively. 5% NAC-LDPET is considered here. Axial | Coronal | Saggital | PSNR |  RMSE
---|---|---|---|---
✓ | ✗ | ✗ | $37.884\pm 1.460$ |  $0.0378\pm 0.0104$
✗ | ✓ | ✗ | $37.801\pm 1.459$ |  $0.0385\pm 0.0098$
✗ | ✗ | ✓ | $38.066\pm 1.458$ |  $0.0362\pm 0.0095$
✓ | ✓ | ✗ | $38.350\pm 1.505$ |  $0.0341\pm 0.0097$
✗ | ✓ | ✓ | $38.465\pm 1.468$ |  $0.0331\pm 0.0089$
✓ | ✗ | ✓ | $38.747\pm 1.486$ |  $0.0311\pm 0.0091$
✓ | ✓ | ✓ | $\mathbf{38.920\pm 1.503}$ |  $\mathbf{0.0299\pm 0.0088}$

Multi-view Averaging vs Sequential-based Strategies was investigated and
reported in Table 4. Specifically, we investigated a variant of MADM, called
2.5D Multi-View Sequential Diffusion (MVSD) that uses the same sets of 2.5D
diffusion models as in MADM but was applied sequentially at multiple views
during the sampling process. As we can see from the table, the MVSD approach,
regardless of the order of axial, coronal, and sagittal views processed,
exhibits negligible differences in performance, with PSNR values hovering
around $38.61$ dB and RMSE at approximately $0.0321$. Our MADM on the last row
achieved superior performance than all MVSD with different settings.

Table 4: Quantitative comparison of AC-SDPET generated from NAC-LDPET data. Best results are marked in bold. 5% NAC-LDPET is considered here. 2.5D Multi-View Sequential | PSNR |  RMSE
---|---|---
Axial $\rightarrow$ Coronal $\rightarrow$ Saggital | $38.613\pm 1.500$ |  $0.0321\pm 0.0092$
Coronal $\rightarrow$ Saggital $\rightarrow$ Axial | $38.609\pm 1.500$ |  $0.0321\pm 0.0092$
Saggital $\rightarrow$ Axial $\rightarrow$ Coronal | $38.613\pm 1.499$ |  $0.0321\pm 0.0092$
MADM | $\mathbf{38.920\pm 1.503}$ |  $\mathbf{0.0299\pm 0.0088}$
Table 5: Quantitative comparison of AC-SDPET generated from NAC-LDPET data. Best results are marked in bold. 5% NAC-LDPET is considered here. Eval | PSNR |  RMSE
---|---|---
2D MADM | $37.816\pm 1.541$ |  $0.0386\pm 0.0110$
2.5D MADM s = 4 | $38.759\pm 1.450$ |  $0.0309\pm 0.0080$
2.5D MADM s = 8 | $\mathbf{38.920\pm 1.503}$ |  $\mathbf{0.0299\pm 0.0088}$
2.5D MADM s = 12 | $38.772\pm 1.478$ |  $0.0309\pm 0.0087$

Impact of 2.5D Setting in MADM was investigated and summarized in Table 5. As
we can see, transitioning from a 2D to a 2.5D in MADM elevates the PSNR from
$37.82$ dB to $38.92$ dB and reduces the RMSE from $0.0386$ to $0.0299$. This
enhancement in model performance is attributed to the 2.5D model’s capability
to integrate contextual information from adjacent slices. Figure 5 exemplifies
that the 2.5D MADM more accurately predicts the shape and intensity in lesion
areas (red arrows). By incorporating $s$ upper and lower slices relative to
the target slice into its input, the 2.5D model achieves a more comprehensive
understanding and precise reconstruction of abnormal areas. We found that
setting the number of adjacent slices to $s=8$ yields the best performance,
achieving a PSNR of $38.92$ dB and an RMSE of $0.0299$.

Figure 5: Visual comparison of AC-SDPET generation form 2D MADM and 2.5D MADM
under 5% NAC-LDPET settings. The coronal view image(top) and error map(bottom)
are shown. RMSE and PSNR values are calculated for each volume. The CT-based
AC-SDPET(top) and image index into the color map of the error map(bottom) are
shown in the first column. Figure 6: Ablative study on the starting time step
of the diffusion sampling process. NMSE evaluation under the $5\%$ NAC-LDPET
setting is used here. Best performance is reached when $t_{s}=200$.

Impact of CNN-Prior of MADM was investigated as well. A visual example is
shown in Figure 7. As we can see, implementing a prior strategy within the
MADM framework significantly enhances the accuracy of the translated 3D image
and reduces the intensity errors here. In parallel, we also studied the impact
of starting timestep $t_{s}$ of using the prior image in the sampling process,
and the results are summarized in Figure 6, where we plotted the translation’s
NMSE against $t_{s}$ at an interval of $100$. As we observe from this result,
the NMSE first decreases as $t_{s}$ increases. It reached the minimal NMSE of
$0.0299$ at $t_{s}=200$ before the NMSE started to increase again. In contrast
to previous diffusion methods that typically require $t_{s}=1000$, our method
with optimal performance at $t_{s}=200$ outperforms them in terms of both
speed and image quality. Please note that the result reported at $t_{s}=0$ is
c-GAN [17] without diffusion process.

Figure 7: Visual comparison of AC-SDPET generation with and without UNet Prior
in MADM. under 5% NAC-LDPET settings. The coronal view image(top) and error
map(bottom) are shown. RMSE and PSNR values are calculated for each individual
volume. The CT-based AC-SDPET(top) and image index in to the color map of the
error map(bottom) are shown in the first column.

##  4 Discussion

In this work, we develop a novel 3D image-to-image translation diffusion
model, called 2.5D Multi-view Averaging Diffusion Model (MADM), for reducing
the radiation in PET acquisition by directly translating NAC-LDPET to AC-SDPET
in 3D. During the training, MADM trains multiple lightweight 2.5D models from
axial, coronal, and sagittal views. In parallel, a conventional 3D CNN-based
network (i.e. 3D UNet here) is also trained. During the inference process, we
first use the trained 3D UNet for generating a prior 3D image. Then, we
utilize this 3D prior image with scheduled noise as a starting point in our
diffusion process to further refine it. During the subsequent diffusion, all
views’ 2.5D models are applied in each sampling step and ensemble before
proceeding to the next step. There are several key advantages of MADM. First,
directly extending the previous 2D diffusion model into 3D is highly
infeasible due to the challenge of large GPU memory requirements. MADM
utilizes multiple lightweight 2.5D diffusion models instead of a 3D diffusion
model, which makes it trainable on limited GPU memory settings, especially
when dealing with high-resolution 3D image-to-image generation, e.g. NAC-LDPET
to AC-SDPET 3D translation that explored in this work. As we can observe from
the last two columns in Table 1, MADM training memory consumption (per batch)
is nearly 10 times less than 3D-DDPM. Second, MADM uses two simple and
efficient strategies to enable high-quality 3D translation. The first is to
use a 2.5D model that can learn the translation with contextual information
from adjacent slices. The second is to use 2.5D models from three different
views, where we average their outputs during the sampling process to ensure
the slice-by-slice inconsistencies are minimized thus improving the 3D
translation. Third, training diffusion models often require large-scale
datasets to achieve robust performance. In contrast to directly feeding
limited amounts of whole 3D volumes into a 3D diffusion model for training,
2.5D inputs from multiple views extracted from 3D volumes naturally improve
the number and diversity of the training data. In fact, as we can observe from
Table 1 and Table 2, MADM achieved significantly better translation in terms
of both global and local metrics, and we believe this is one of the main
reasons. Lastly, we utilize the one-step inference 3D CNN model to generate a
prior image as the starting point of the diffusion model, which not only
reduces the number of required sampling steps but improves the translation
accuracy.

However, our current work also has potential limitations, suggesting
interesting directions for future studies. First, even though we reduce the
memory requirement (Table 1) for diffusion-based 3D image-to-image
translation, the inference speed of MADM per 3D volume in our applications
takes about 10 minutes for the volume $192\times 192\times 192$ and up to 30
minutes for the volume $192\times 192\times 460$ which one would consider
long, especially when comparing with CNN-based methods. Depending on the
user’s time budget, MADM can adjust the starting time point of the diffusion
process (Figure 6), trading the inference speed with the performance. As we
can see from Figure 6, the optimal performance is achieved when $t_{s}=200$.
In this case, we can adjust the $t_{s}=100$ to reduce the inference steps by
two times, thus significantly reducing the inference time. While the
performance reduces from NMSE of $0.0299$ to $0.0319$, the performance is
still better than all the previous baselines in Table 1. In parallel, we can
adjust the number of views of MADM to improve the inference speed. As we can
see from Table 3, even though optimal performance was achieved when all three
views were included, we can use the two-view setting, i.e. axial + saggital
views, to reduce the inference time but still generate superior 3D translation
performance as compared to previous baseline methods (Table 1). On the other
hand, we could also replace the current 3D prior image generation network,
i.e. 3D UNet, with more advanced network architecture, such as Transformer
[42, 7] and Multi-stage Network [47, 48] to improve the prior 3D image
generation, which may shorten the diffusion sampling process for optimal
performance. Second, even though our results (Table 1) show that MADM achieved
superior 3D translation performance as compared to previous state-of-the-art
diffusion and CNN-based methods, one of the key limitations is that we don’t
know how confident we are with our translation. 3D pixel-wise uncertainty
estimation is highly desirable. In fact, uncertainty estimation for the I2I
diffusion model is already well-established in many previous works [53, 11].
In general, pixel-wise uncertainty can be obtained by running the diffusion
reverse pass multiple times and then computing the pixel-wise standard
deviation of the translated results. Similar here for MADM, we could also run
multiple reverse passes by adding different noise to the same prior image,
thus generating multiple 3D translation results. Then, the 3D pixel-wise
uncertainty can be obtained by calculating the standard deviation of the
translations. With the uncertainty map, one can identify areas where the model
is confident and areas where it is not. One could use this information to
fine-tune the model to reinforce the learning in the uncertain regions, thus
potentially further improving the performance. In parallel to uncertainty
estimation, averaging multiple runs’ transition results as final translation
also presents potential opportunities for further improving the performance
[53]. However, please note that utilizing this simple and efficient strategy
for uncertainty estimation and performance gain comes at the cost of further
elevated inference time, where the MADM inference time would increase linearly
as the number of reverse runs increases. We recommend that users balance this
trade-off in their application, especially in time-sensitive applications.
Third, we only validated our 3D translation method for the application of PET
dose reduction in this work. While we validate our method on clinically
significant regions, i.e. tumors (Table 2), further analysis on the impacts on
downstream analysis, such as prognosis and treatment planning, needs to be
carried out in the future. In addition, interesting future directions also
include validating MADM to other 3D medical imaging datasets and applications,
such as 3D cross-modal MRI synthesis [22, 18], 3D contrast image synthesis in
CT/MRI [23, 5, 27], Low-dose CT denoising [46, 49, 48], and accelerated MRI
reconstruction [54, 51, 50]. Finally, we would also like to add that - while
we framed the MADM as an independent method for 3D translation here, we
believe this 2.5D-based method could be used for assisting the 3D network-
based diffusion translation once the computation barrier for it is not an
issue in the future. Specifically, during the diffusion inference process, we
could average the 3D results with the multi-view 2.5D results, and it could
potentially further improve the translation performance. This is an
interesting direction that we will investigate in our future works.

##  5 Conclusion

In this work, we developed a novel 2.5D Multi-view Averaging Diffusion Model
(MADM) for 3D medical image translation with the first application on low-
count PET reconstruction with CT-less AC. Our method has three key components.
First, to reduce the computational demands associated with 3D models and to
mitigate the discontinuities inherent in 2D models, we proposed to use a 2.5D
model that incorporates several adjacent slices in addition to the target
slice for prediction. Second, to further minimize discontinuities and enhance
prediction accuracy, we proposed to train separate 2.5D models for each view
in 3D. During the inference, we average the result in each denoising step of
the diffusion process. Third, to accelerate the translation process and refine
generated image accuracy, we proposed to utilize 3D predictions from a CNN-
based model as a prior for our diffusion model. Our experimental results show
that MADM can generate high-quality AC standard-count PET(AC-SDPET) directly
from the NAC low-count PET(NAC-LDPET) with limited computation memory
resources, and outperforming several previous baselines. We believe our method
can be extended to other 3D imaging applications, and potentially useful in
other clinical applications.

## Acknowledgments

This work was supported by the National Institutes of Health (NIH) grant
R01EB025468 and R01CA275188.

## Declaration of Competing Interest

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

## Credit authorship contribution statement

Tianqi Chen: Conceptualization, Methodology, Software, Visualization,
Validation, Formal analysis, Writing original draft. Jun Hou:
Conceptualization, Methodology, Software, Visualization, Validation, Formal
analysis, Writing - review and editing. Yinchi Zhou: Results analysis, Writing
- review and editing. Huidong Xie: Conceptualization, Methodology, Software,
Writing - review and editing. Xiongchao Chen: Writing - review and editing.
Qiong Liu: Writing - review and editing. Xueqi Guo: Writing - review and
editing. Menghua Xia: Writing - review and editing. James S. Duncan: Writing -
review and editing Chi Liu: Writing - review and editing, Co-supervision. Bo
Zhou: Conceptualization, Methodology, Software, Visualization, Validation,
Formal analysis, Writing original draft, Supervision.

## References

  * Arjovsky et al. [2017] Arjovsky, M., Chintala, S., Bottou, L., 2017.  Wasserstein gan.  arXiv preprint arXiv:1701.07875 .
  * Bieder et al. [2023] Bieder, F., Wolleb, J., Durrer, A., Sandkühler, R., Cattin, P.C., 2023.  Diffusion models for memory-efficient processing of 3d medical images.  arXiv preprint arXiv:2303.15288 .
  * Chen et al. [2019] Chen, K.T., Gong, E., de Carvalho Macruz, F.B., Xu, J., Boumis, A., Khalighi, M., Poston, K.L., Sha, S.J., Greicius, M.D., Mormino, E., et al., 2019.  Ultra–low-dose 18f-florbetaben amyloid pet imaging using deep learning with multi-contrast mri inputs.  Radiology 290, 649–656.
  * Chen and Liu [2023] Chen, X., Liu, C., 2023.  Deep-learning-based methods of attenuation correction for spect and pet.  Journal of Nuclear Cardiology 30, 1859–1878.
  * Choi et al. [2021] Choi, J.W., Cho, Y.J., Ha, J.Y., Lee, S.B., Lee, S., Choi, Y.H., Cheon, J.E., Kim, W.S., 2021.  Generating synthetic contrast enhancement from non-contrast chest computed tomography using a generative adversarial network.  Scientific reports 11, 20403.
  * Chung et al. [2023] Chung, H., Ryu, D., McCann, M.T., Klasky, M.L., Ye, J.C., 2023.  Solving 3d inverse problems using pre-trained 2d diffusion models, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22542–22551.
  * Dalmaz et al. [2022] Dalmaz, O., Yurt, M., Çukur, T., 2022.  Resvit: residual vision transformers for multimodal medical image synthesis.  IEEE Transactions on Medical Imaging 41, 2598–2614.
  * Dong et al. [2020] Dong, X., Lei, Y., Wang, T., Higgins, K., Liu, T., Curran, W.J., Mao, H., Nye, J.A., Yang, X., 2020.  Deep learning-based attenuation correction in the absence of structural information for whole-body positron emission tomography imaging.  Physics in Medicine & Biology 65, 055011.
  * Dong et al. [2019] Dong, X., Wang, T., Lei, Y., Higgins, K., Liu, T., Curran, W.J., Mao, H., Nye, J.A., Yang, X., 2019.  Synthetic ct generation from non-attenuation corrected pet images for whole-body pet imaging.  Physics in Medicine & Biology 64, 215016.
  * Dutta et al. [2013] Dutta, J., Leahy, R.M., Li, Q., 2013.  Non-local means denoising of dynamic pet images.  PloS one 8, e81390.
  * Gong et al. [2024] Gong, K., Johnson, K., El Fakhri, G., Li, Q., Pan, T., 2024.  Pet image denoising based on denoising diffusion probabilistic model.  European Journal of Nuclear Medicine and Molecular Imaging 51, 358–368.
  * Gong et al. [2020] Gong, Y., Shan, H., Teng, Y., Tu, N., Li, M., Liang, G., Wang, G., Wang, S., 2020.  Parameter-transferred wasserstein generative adversarial network (pt-wgan) for low-dose pet image denoising.  IEEE Transactions on Radiation and Plasma Medical Sciences .
  * Goodfellow et al. [2014] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y., 2014.  Generative adversarial nets, in: Advances in neural information processing systems, pp. 2672–2680.
  * Hu et al. [2020] Hu, Z., Xue, H., Zhang, Q., Gao, J., Zhang, N., Zou, S., Teng, Y., Liu, X., Yang, Y., Liang, D., et al., 2020.  Dpir-net: Direct pet image reconstruction based on the wasserstein generative adversarial network.  IEEE Transactions on Radiation and Plasma Medical Sciences .
  * Hwang et al. [2019] Hwang, D., Kang, S.K., Kim, K.Y., Seo, S., Paeng, J.C., Lee, D.S., Lee, J.S., 2019.  Generation of pet attenuation map for whole-body time-of-flight 18f-fdg pet/mri using a deep neural network trained with simultaneously reconstructed activity and attenuation maps.  Journal of Nuclear Medicine 60, 1183–1189.
  * Isola et al. [2017a] Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A., 2017a.  Image-to-image translation with conditional adversarial networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134.
  * Isola et al. [2017b] Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A., 2017b.  Image-to-image translation with conditional adversarial networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  * Ji et al. [2022] Ji, S., Yang, D., Lee, J., Choi, S.H., Kim, H., Kang, K.M., 2022.  Synthetic mri: technologies and applications in neuroradiology.  Journal of Magnetic Resonance Imaging 55, 1013–1025.
  * Kaplan and Zhu [2019] Kaplan, S., Zhu, Y.M., 2019.  Full-dose pet image estimation from low-dose pet image using deep learning: a pilot study.  Journal of digital imaging 32, 773–778.
  * Lee [2020] Lee, J.S., 2020.  A review of deep-learning-based approaches for attenuation correction in positron emission tomography.  IEEE Transactions on Radiation and Plasma Medical Sciences 5, 160–184.
  * Lee et al. [2023] Lee, S., Chung, H., Park, M., Park, J., Ryu, W.S., Ye, J.C., 2023.  Improving 3d imaging with pre-trained perpendicular 2d diffusion models, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10710–10720.
  * Li et al. [2023] Li, H.B., Conte, G.M., Anwar, S.M., Kofler, F., Ezhov, I., van Leemput, K., Piraud, M., Diaz, M., Cole, B., Calabrese, E., et al., 2023.  The brain tumor segmentation (brats) challenge 2023: Brain mr image synthesis for tumor segmentation (brasyn).  ArXiv .
  * Li et al. [2022] Li, W., Lam, S., Li, T., Cheung, A.L.Y., Xiao, H., Liu, C., Zhang, J., Teng, X., Zhi, S., Ren, G., et al., 2022.  Multi-institutional investigation of model generalizability for virtual contrast-enhanced mri synthesis, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 765–773.
  * Liu et al. [2018a] Liu, F., Jang, H., Kijowski, R., Bradshaw, T., McMillan, A.B., 2018a.  Deep learning mr imaging-based attenuation correction for pet/mr imaging.  Radiology 286, 676–684.
  * Liu et al. [2018b] Liu, F., Jang, H., Kijowski, R., Zhao, G., Bradshaw, T., McMillan, A.B., 2018b.  A deep learning approach for 18f-fdg pet attenuation correction.  EJNMMI physics 5, 1–15.
  * Liu et al. [2020a] Liu, H., Wu, J., Lu, W., Onofrey, J.A., Liu, Y.H., Liu, C., 2020a.  Noise reduction with cross-tracer and cross-protocol deep transfer learning for low-dose pet.  Physics in Medicine & Biology 65, 185006.
  * Liu et al. [2020b] Liu, J., Tian, Y., Ağıldere, A.M., Haberal, K.M., Coşkun, M., Duzgol, C., Akin, O., 2020b.  Dyefreenet: Deep virtual contrast ct synthesis, in: Simulation and Synthesis in Medical Imaging: 5th International Workshop, SASHIMI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Proceedings 5, Springer. pp. 80–89.
  * Lu et al. [2019] Lu, W., Onofrey, J.A., Lu, Y., Shi, L., Ma, T., Liu, Y., Liu, C., 2019.  An investigation of quantitative accuracy for deep learning based denoising in oncological pet.  Physics in Medicine & Biology 64, 165019.
  * Maggioni et al. [2012] Maggioni, M., Katkovnik, V., Egiazarian, K., Foi, A., 2012.  Nonlocal transform-domain filter for volumetric data denoising and reconstruction.  IEEE transactions on image processing 22, 119–133.
  * Mejia et al. [2016] Mejia, J., Mederos, B., Mollineda, R.A., Maynez, L.O., 2016.  Noise reduction in small animal pet images using a variational non-convex functional.  IEEE Transactions on Nuclear Science 63, 2577–2585.
  * Nuyts et al. [2018] Nuyts, J., Rezaei, A., Defrise, M., 2018.  The validation problem of joint emission/transmission reconstruction from tof-pet projections.  IEEE Transactions on Radiation and Plasma Medical Sciences 2, 273–278.
  * Ouyang et al. [2019] Ouyang, J., Chen, K.T., Gong, E., Pauly, J., Zaharchuk, G., 2019.  Ultra-low-dose pet reconstruction using generative adversarial network with feature matching and task-specific perceptual loss.  Medical physics 46, 3555–3564.
  * Pan et al. [2024] Pan, S., Abouei, E., Peng, J., Qian, J., Wynne, J.F., Wang, T., Chang, C.W., Roper, J., Nye, J.A., Mao, H., et al., 2024.  Full-dose whole-body pet synthesis from low-dose pet using high-efficiency denoising diffusion probabilistic model: Pet consistency model.  Medical Physics .
  * Rezaei et al. [2016] Rezaei, A., Michel, C., Casey, M.E., Nuyts, J., 2016.  Simultaneous reconstruction of the activity image and registration of the ct image in tof-pet.  Physics in Medicine & Biology 61, 1852.
  * Ronneberger et al. [2015a] Ronneberger, O., Fischer, P., Brox, T., 2015a.  U-net: Convolutional networks for biomedical image segmentation, in: International Conference on Medical image computing and computer-assisted intervention, Springer. pp. 234–241.
  * Ronneberger et al. [2015b] Ronneberger, O., Fischer, P., Brox, T., 2015b.  U-net: Convolutional networks for biomedical image segmentation, in: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (Eds.), Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Springer International Publishing, Cham. pp. 234–241.
  * Shi et al. [2023] Shi, L., Zhang, J., Toyonaga, T., Shao, D., Onofrey, J.A., Lu, Y., 2023.  Deep learning-based attenuation map generation with simultaneously reconstructed pet activity and attenuation and low-dose application.  Physics in Medicine & Biology 68, 035014.
  * Shiri et al. [2019] Shiri, I., Ghafarian, P., Geramifar, P., Leung, K.H.Y., Ghelichoghli, M., Oveisi, M., Rahmim, A., Ay, M.R., 2019.  Direct attenuation correction of brain pet images using only emission data via a deep convolutional encoder-decoder (deep-dac).  European radiology 29, 6867–6879.
  * Shiri et al. [2023] Shiri, I., Salimi, Y., Maghsudi, M., Jenabi, E., Harsini, S., Razeghi, B., Mostafaei, S., Hajianfar, G., Sanaat, A., Jafari, E., et al., 2023.  Differential privacy preserved federated transfer learning for multi-institutional 68ga-pet image artefact detection and disentanglement.  European journal of nuclear medicine and molecular imaging , 1–14.
  * Strauss and Kaste [2006] Strauss, K.J., Kaste, S.C., 2006.  The alara (as low as reasonably achievable) concept in pediatric interventional and fluoroscopic imaging: striving to keep radiation doses as low as possible during fluoroscopy of pediatric patients—a white paper executive summary.  Radiology 240, 621–622.
  * Toyonaga et al. [2022] Toyonaga, T., Shao, D., Shi, L., Zhang, J., Revilla, E.M., Menard, D., Ankrah, J., Hirata, K., Chen, M.K., Onofrey, J.A., et al., 2022.  Deep learning–based attenuation correction for whole-body pet—a multi-tracer study with 18f-fdg, 68 ga-dotatate, and 18f-fluciclovine.  European Journal of Nuclear Medicine and Molecular Imaging 49, 3086–3097.
  * Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I., 2017.  Attention is all you need.  Advances in neural information processing systems 30.
  * Wang et al. [2018] Wang, Y., Yu, B., Wang, L., Zu, C., Lalush, D.S., Lin, W., Wu, X., Zhou, J., Shen, D., Zhou, L., 2018.  3d conditional generative adversarial networks for high-quality pet image estimation at low dose.  Neuroimage 174, 550–562.
  * Xiang et al. [2017] Xiang, L., Qiao, Y., Nie, D., An, L., Lin, W., Wang, Q., Shen, D., 2017.  Deep auto-context convolutional neural networks for standard-dose pet image estimation from low-dose pet/mri.  Neurocomputing 267, 406–416.
  * Xie et al. [2023] Xie, H., Gan, W., Zhou, B., Chen, X., Liu, Q., Guo, X., Guo, L., An, H., Kamilov, U.S., Wang, G., Liu, C., 2023.  DDPET-3D: Dose-aware Diffusion Model for 3D Ultra Low-dose PET Imaging.  URL: http://arxiv.org/abs/2311.04248, doi:10.48550/arXiv.2311.04248. arXiv:2311.04248 [eess].
  * Yin et al. [2019] Yin, X., Zhao, Q., Liu, J., Yang, W., Yang, J., Quan, G., Chen, Y., Shu, H., Luo, L., Coatrieux, J.L., 2019.  Domain progressive 3d residual convolution network to improve low-dose ct imaging.  IEEE transactions on medical imaging 38, 2903–2913.
  * Zamir et al. [2021] Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H., Shao, L., 2021.  Multi-stage progressive image restoration, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14821–14831.
  * Zhou et al. [2022a] Zhou, B., Chen, X., Xie, H., Zhou, S.K., Duncan, J.S., Liu, C., 2022a.  Dudoufnet: Dual-domain under-to-fully-complete progressive restoration network for simultaneous metal artifact reduction and low-dose ct reconstruction.  IEEE Transactions on Medical Imaging 41, 3587–3599.
  * Zhou et al. [2021] Zhou, B., Chen, X., Zhou, S.K., Duncan, J.S., Liu, C., 2021.  Dudodr-net: Dual-domain data consistent recurrent network for simultaneous sparse view and metal artifact reduction in computed tomography.  Medical Image Analysis , 102289.
  * Zhou et al. [2022b] Zhou, B., Schlemper, J., Dey, N., Salehi, S.S.M., Liu, C., Duncan, J.S., Sofka, M., 2022b.  Dsformer: A dual-domain self-supervised transformer for accelerated multi-contrast mri reconstruction.  arXiv preprint arXiv:2201.10776 .
  * Zhou and Zhou [2020] Zhou, B., Zhou, S.K., 2020.  Dudornet: Learning a dual-domain recurrent network for fast mri reconstruction with deep t1 prior, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 298–313.
  * Zhou et al. [2020] Zhou, L., Schaefferkoetter, J.D., Tham, I.W., Huang, G., Yan, J., 2020.  Supervised learning with cyclegan for low-dose fdg pet image denoising.  Medical Image Analysis 65, 101770.
  * Zhou et al. [2024] Zhou, Y., Chen, T., Hou, J., Xie, H., Dvornek, N.C., Zhou, S.K., Wilson, D.L., Duncan, J.S., Liu, C., Zhou, B., 2024.  Cascaded multi-path shortcut diffusion model for medical image translation.  arXiv preprint arXiv:2405.12223 .
  * Zhou et al. [2023] Zhou, Y., Wang, H., Liu, C., Liao, B., Li, Y., Zhu, Y., Hu, Z., Liao, J., Liang, D., 2023.  Recent advances in highly accelerated 3d mri.  Physics in Medicine & Biology 68, 14TR01.
  * Zhu et al. [2017] Zhu, J.Y., Park, T., Isola, P., Efros, A.A., 2017.  Unpaired image-to-image translation using cycle-consistent adversarial networks, in: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232.