# Self-Supervision Improves Diffusion Models
for Tabular Data Imputation

Yixin Liu<EMAIL_ADDRESS>0000-0002-4309-5076 Monash
UniversityMelbourneAustralia ,  Thalaiyasingam Ajanthan<EMAIL_ADDRESS>0000-0002-6431-0775 AmazonCanberraAustralia ,  Hisham Husain
<EMAIL_ADDRESS>0009-0007-3736-3410 AmazonMelbourneAustralia and  Vu
Nguyen<EMAIL_ADDRESS>0000-0002-0294-4561 AmazonAdelaideAustralia

(2024)

###### Abstract.

Incomplete tabular datasets are ubiquitous in many applications for a number
of reasons such as human error in data collection or privacy considerations.
One would expect a natural solution for this is to utilize powerful generative
models such as diffusion models, which have demonstrated great potential
across image and continuous domains. However, vanilla diffusion models often
exhibit sensitivity to initialized noises. This, along with the natural
sparsity inherent in the tabular domain, poses challenges for diffusion
models, thereby impacting the robustness of these models for data imputation.
In this work, we propose an advanced diffusion model named Self-supervised
imputation Diffusion Model (SimpDM for brevity), specifically tailored for
tabular data imputation tasks. To mitigate sensitivity to noise, we introduce
a self-supervised alignment mechanism that aims to regularize the model,
ensuring consistent and stable imputation predictions. Furthermore, we
introduce a carefully devised state-dependent data augmentation strategy
within SimpDM, enhancing the robustness of the diffusion model when dealing
with limited data. Extensive experiments demonstrate that SimpDM matches or
outperforms state-of-the-art imputation methods across various scenarios.

Tabular Data, Data Imputation, Incomplete Data, Diffusion Model, Self-
Supervised Learning

††journalyear: 2024††copyright: rightsretained††conference: Proceedings of the
33rd ACM International Conference on Information and Knowledge Management;
October 21–25, 2024; Boise, ID, USA††booktitle: Proceedings of the 33rd ACM
International Conference on Information and Knowledge Management (CIKM ’24),
October 21–25, 2024, Boise, ID, USA††doi: 10.1145/3627673.3679829††isbn:
979-8-4007-0436-9/24/10††ccs: Information systems Incomplete data††ccs:
Computer systems organization Neural networks

##  1\. Introduction

Tabular data are ubiquitous across industries and disciplines, including but
not limited to finance (Assefa et al., 2020), healthcare (Hernandez et al.,
2022), and environmental sciences (Judson et al., 2009). In real-world
scenarios, tabular data often contain missing values for many reasons, such as
human error, privacy considerations, and the inherent challenges associated
with data collection processes (Muzellec et al., 2020). For instance, some
characteristics of a patient might not be accurately documented during their
visit (Alaa et al., 2017; Jarrett et al., 2022). The presence of missing data
significantly impacts the quality of tabular datasets, introducing bias and
rendering a majority of machine learning methods inapplicable.

(a) With different Gaussian noises (red) as initialization, the diffusion
model outputs diverse and inaccurate imputation (blue).

(b) The limited data samples lead to poor imputation results (higher RMSE) and
unstable performance (higher variance).

Figure 1. Motivating experiments on UCI Power dataset. (a) Given a data
sample, the imputation results by diffusion model with different Gaussian
initialization at the first diffusion step. (b) The imputation performance
under different numbers of training samples.

To handle the missing data problem, data imputation is a promising solution
that aims to estimate missing values based on the observed data (Osman et al.,
2018). Existing data imputation methods usually use statistical algorithms
(Farhangfar et al., 2007), shallow machine learning algorithms (Troyanskaya et
al., 2001; Van Buuren and Groothuis-Oudshoorn, 2011), and deep neural networks
(Muzellec et al., 2020; Kyono et al., 2021) to complete missing data. Among
them, deep generative model-based imputation models exhibit competitive
performance owing to their capability in modeling the data manifold, which
helps complete the missing elements (Yoon et al., 2018; Mattei and Frellsen,
2019). Presently, diffusion models (Ho et al., 2020; Yang et al., 2022), the
leading deep generation models, have shown remarkable capabilities in
generating data across diverse types, including images (Croitoru et al.,
2023), audio (Kong et al., 2021), and time-series data (Rasul et al., 2021).
Building on these achievements, recent studies have used diffusion models for
tabular data generation (Kotelnikov et al., 2023; Lee et al., 2023; Kim et
al., 2023), as well as data imputation (Zheng and Charoenphakdee, 2022;
Jolicoeur-Martineau et al., 2023).

In this research, we identify that vanilla diffusion models are suboptimal for
tabular data imputation. Specifically, we pinpoint two key mismatches, namely
objective mismatch and data scale mismatch, between tabular data imputation
tasks and other scenarios where diffusion models excel. These mismatches
potentially contribute to the degradation in data imputation performance of
diffusion models.

Firstly, the inherent mismatch in learning objective between generation and
imputation problems can severely impede the imputation performance of
diffusion models. Specifically, diversity is a key objective of the generation
problem, which requires the generated data to vary significantly while
maintaining relevance to the given context. Diffusion models being sensitive
to the initial noise ($x_{T}$) at the generation stage helps generate diverse
samples – different noise usually leads to different generated samples.
Conversely, the objective of the imputation task is accuracy rather than
diversity, requiring the imputed data to closely resemble the singular ground-
truth values. In this case, the sensitivity to the initial noise results in a
large variance in imputed results, hurting accuracy. As shown in the
motivating example in Fig. 1(a), given identical observation ($-0.03$ and
$0.13$), the vanilla diffusion model generates diverse imputed values, leading
to substantial gaps between imputed values and the ground-truth.

Moreover, the much smaller data scale of tabular data compared to other
domains (e.g., image) also hinders diffusion models from comprehending the
data manifold, yielding subpar data imputation models. For example, CIFAR-10
is a relatively small image dataset, but it still has 60k samples, supporting
the diffusion models to capture data patterns (Kulikov et al., 2023). In
contrast, tabular data usually have only a few thousand or even hundreds of
samples, making it much more difficult for diffusion models to capture the
true data distribution, resulting in overfitting. An example in Fig. 1(b)
shows the negative impact of data samples on imputation effectiveness.

To improve the performance of diffusion models on tabular data imputation
tasks, we introduce an advanced diffusion model termed Self-supervised
imputation Diffusion Model (SimpDM for short). By integrating self-supervised
learning techniques into tabular diffusion model, we address the
aforementioned mismatch issues. Specifically, to tackle objective mismatch, we
introduce a self-supervised alignment mechanism to regularize the output of
the diffusion model. The key idea is to encourage the diffusion model to
provide consistent and accurate imputation for the same observed data,
enhancing the stability of imputation results. Furthermore, to handle data
scale mismatch, we introduce state-dependent augmentation, a perturbation-
based data augmentation strategy that is carefully designed for tabular data
imputation tasks. With data augmentation, we extend the training set,
improving the robustness of the diffusion model. Meanwhile, an effective
augmentation can also ensure the effectiveness of the self-supervised learning
framework. To verify the imputation capability of our SimpDM model, we
conducted extensive experiments on real-world benchmark datasets across
multiple missing data scenarios. The empirical results highlight the strong
imputation performance of SimpDM compared to state-of-the-art methods. To sum
up, the contributions of this paper are three-fold:

  * •

Finding. We identify two key challenges that hinder diffusion models in
solving tabular data imputation tasks: target mismatch and data scale
mismatch.

  * •

Solution. We propose a novel diffusion model termed SimpDM. The proposed
method integrates two effective techniques, i.e., self-supervised alignment
and state-dependent augmentation, to address the aforementioned challenges.

  * •

Experiments. We conduct extensive experiments on 17 benchmark datasets in
multiple missing scenarios, and the results illustrate the effectiveness of
SimpDM.

##  2\. Related Works

In this section, we briefly review two related research directions, i.e.,
tabular missing data imputation and diffusion models.

###  2.1. Tabular Missing Data Imputation

Tabular data imputation is an essential research topic to handle the missing
data problem. Early solutions use statistical algorithms to estimate missing
entries according to the mean, median, or mode estimation of observed data
(Farhangfar et al., 2007). Besides statistical methods, shallow machine
learning methods such as kNN imputation (Troyanskaya et al., 2001), MICE (Van
Buuren and Groothuis-Oudshoorn, 2011), and missForest (Stekhoven and Bühlmann,
2012) are also effective to complete the missing data.

To further exploit the potential of deep learning techniques, recent studies
use deep neural networks for data imputation (Jarrett et al., 2022). For
instance, Muzellec et al. (2020) propose training deep imputation model by
minimizing the optimal transport distance between two groups of data, while
Kyono et al. (2021) use deep models to discover the causal structure
underlying data for data imputation. Graph neural networks (Liu et al., 2024,
2023; Zheng et al., 2023b; Liang et al., 2024; Zheng et al., 2023a) are also
adapted to data imputation since they are capable of modeling inter-sample
correlation (You et al., 2020; Telyatnikov and Scardapane, 2023; Zhong et al.,
2023). Within deep methodologies, generative model-based approaches stand out
for their remarkable performance. Specifically, GAIN (Yoon et al., 2018)
harnesses GAN while MIWAE (Mattei and Frellsen, 2019) utilizes VAE as their
backbone, enhancing their imputation capabilities. Several recent studies also
attempted to apply diffusion models to tabular data imputation tasks (Zheng
and Charoenphakdee, 2022; Jolicoeur-Martineau et al., 2023; Ouyang et al.,
2023). Nevertheless, these approaches tend to underestimate the inherent
disparity between data imputation and diffusion models, resulting in
suboptimal imputation performance.

###  2.2. Diffusion Models

Diffusion models (Sohl-Dickstein et al., 2015) are a generative paradigm that
strives to approximate the target distribution through the endpoint of a
Markov chain. This chain initiates from a specified parametric distribution,
usually a standard Gaussian. Each step of this Markov process is executed by a
deep neural network that learns the inversion of the diffusion process. DDPM
(Ho et al., 2020) bridges the gap between diffusion models and score matching
approaches (Song and Ermon, 2020), showcasing the powerful capability of
diffusion models in image generation. So far, diffusion models have made great
success in the generation tasks on various domains, such as image (Dhariwal
and Nichol, 2021), text (Hoogeboom et al., 2021), audio (Kong et al., 2021),
time-series (Rasul et al., 2021), and graph (Xu et al., 2022; Pan et al.,
2024).

Given the formidable capabilities of diffusion models, recent studies attempt
to leverage diffusion models to handle learning tasks on tabular data. For
instance, TabDDPM (Kotelnikov et al., 2023) is a representative method that
combines Gaussian and Multinomial diffusion models together to generate mixed-
type tabular data. CoDi (Lee et al., 2023) and STaSy (Kim et al., 2023) also
show impressive capability in tabular data synthesis. Inspired by the success
of diffusion models in image inpainting (Lugmayr et al., 2022) and time-series
imputation (Tashiro et al., 2021), researchers start to discover the potential
of diffusion models in tabular data imputation (Zheng and Charoenphakdee,
2022; Jolicoeur-Martineau et al., 2023; Ouyang et al., 2023). This paper
further delves into this research direction, enhancing diffusion models for
imputation from a novel perspective.

##  3\. Preliminary

###  3.1. Problem Definition

Let $\overline{\mathbf{X}}=\\{\overline{\mathbf{x}}_{[i]}\\}^{n}_{i=1}$
denotes the feature matrix of a complete tabular dataset, where the $i$-th row
$\overline{\mathbf{x}}_{[i]}$ is a $d$-dimensional feature vector of the
$i$-th sample. Each feature can be numerical or categorical. The missing data
problem can be modeled by a binary mask matrix
$\mathbf{M}\in\\{0,1\\}^{n\times d}$, where $\mathbf{M}_{[i,j]}=1$ indicates
the entry $\overline{\mathbf{X}}_{[i,j]}$ is missing, otherwise
$\mathbf{M}_{[i,j]}=0$. The observed incomplete data matrix can be represented
by

(1) $\mathbf{X}=\mathbf{X}^{(obs)}\odot\left(\mathbbm{1}_{n\times
d}-\mathbf{M}\right)+\varnothing_{n\times d}\odot\mathbf{M},$

where $\varnothing_{n\times d}$ is an $n\times d$ matrix of null (missing)
values, $\mathbf{X}^{(obs)}$ contains the observed entries that are from
$\overline{\mathbf{X}}$, $\odot$ is the element-wise product and
$\mathbbm{1}_{n\times d}$ is an $n\times d$ matrix containing $1$ for every
entry. Given $\mathbf{X}$, the goal of tabular data imputation is to estimate
an imputed data matrix $\hat{\mathbf{X}}$ where the missing entries of
$\mathbf{X}$ are filled, which can be written by

(2) $\hat{\mathbf{X}}=\mathbf{X}^{(obs)}\odot\left(\mathbbm{1}_{n\times
d}-\mathbf{M}\right)+\hat{\mathbf{X}}^{(imp)}\odot\mathbf{M},$

where $\hat{\mathbf{X}}^{(imp)}$ contains the imputed entries. The objective
is to learn $\hat{\mathbf{X}}$ that is as close as possible to
$\overline{\mathbf{X}}$.

###  3.2. Diffusion Models

Diffusion models (Sohl-Dickstein et al., 2015) are deep generative models
derived from a forward and reverse Markov process. The forward process $q$ is
to gradually disturb a sample $\mathbf{x}_{0}$ into a noisy sample
$\mathbf{x}_{T}$, while the reverse process $p$ is to denoise and generate the
sample $\hat{\mathbf{x}}_{0}$ from $\mathbf{x}_{T}$:

(3) $\displaystyle
q\left(\mathbf{x}_{1:T}\mid\mathbf{x}_{0}\right):=\prod_{t=1}^{T}q\left(\mathbf{x}_{t}\mid\mathbf{x}_{t-1}\right),$
$\displaystyle
p_{\theta}\left(\mathbf{x}_{0:T}\right):=p\left(\mathbf{x}_{T}\right)\prod_{t=1}^{T}p_{\theta}\left(\mathbf{x}_{t-1}\mid\mathbf{x}_{t}\right),$

where $p_{\theta}\left(\mathbf{x}_{t-1}\mid\mathbf{x}_{t}\right)$ is
parametrized by a neural network whose parameter is $\theta$. Here $\theta$
can be optimized by minimizing the variational upper bound on the negative
log-likelihood:

(4) $\displaystyle\mathcal{L}_{\mathrm{vb}}$
$\displaystyle=\mathbb{E}_{q}[\underbrace{D_{\mathrm{KL}}\left[q\left(\mathbf{x}_{T}\mid\mathbf{x}_{0}\right)||p\left(\mathbf{x}_{T}\right)\right]}_{L_{T}}\underbrace{-\log
p_{\theta}\left(\mathbf{x}_{0}\mid\mathbf{x}_{1}\right)}_{L_{0}}$
$\displaystyle+\sum_{t=2}^{T}\underbrace{D_{\mathrm{KL}}\left(q\left(\mathbf{x}_{t-1}\mid\mathbf{x}_{t},\mathbf{x}_{0}\right)||p_{\theta}\left(\mathbf{x}_{t-1}\mid\mathbf{x}_{t}\right)\right)}_{L_{t-1}}].$

We can describe the evolution of the distributions with the following forward
and reverse process equations

(5) $\displaystyle d\mathbf{x}_{t}=\mathbf{x}_{t}dt+\sqrt{2}d\mathbf{w},$ (6)
$\displaystyle d\mathbf{x}_{t}=\left(\mathbf{x}_{t}-2\nabla_{x}\log
p_{t}\right)+\sqrt{2}d\mathbf{w},$

where $\mathbf{w}$ corresponds to Brownian motion and $p_{t}$ is the law of
the random variable $\mathbf{x}_{t}$. Gaussian diffusion models (Ho et al.,
2020) have forward and reverse processes characterized by Gaussian
distributions:

(7) $\displaystyle
q\left(\mathbf{x}_{t}\mid\mathbf{x}_{t-1}\right)=\mathcal{N}\left(\mathbf{x}_{t};\sqrt{1-\beta_{t}}\mathbf{x}_{t-1},\beta_{t}I\right),$
$\displaystyle
q\left(\mathbf{x}_{T}\right)=\mathcal{N}\left(\mathbf{x}_{T};0,I\right),$
$\displaystyle
p_{\theta}\left(\mathbf{x}_{t-1}\mid\mathbf{x}_{t}\right)=\mathcal{N}\left(x_{t-1};\mu_{\theta}\left(\mathbf{x}_{t},t\right),\Sigma_{\theta}\left(\mathbf{x}_{t},t\right)\right),$

where Gaussian noise is added to the sample following a variance schedule
$\beta_{t}\in(0,1)$. Using equivalences between score matching and error
matching in (Hyvärinen and Dayan, 2005), (Ho et al., 2020) further propose a
simplified objective function for model optimization:

(8) $\mathcal{L}_{\text{ddpm
}}=\mathbb{E}_{t,\mathbf{x}_{0},\boldsymbol{\epsilon}}\left[\left\|\boldsymbol{\epsilon}-\boldsymbol{\epsilon}_{\theta}\left(\mathbf{x}_{t},t\right)\right\|^{2}\right],$

where $\boldsymbol{\epsilon}_{\theta}$ is a neural network aiming to predict
the Gaussian noise $\boldsymbol{\epsilon}$, which can be used to generate
$\hat{\mathbf{x}}_{0}$ during inference.

To deal with categorical data, multinomial diffusion models (Hoogeboom et al.,
2021) define a categorical distribution that perturbs the data with noise over
$K$ classes:

(9) $\displaystyle
q\left(\mathbf{x}_{t}\mid\mathbf{x}_{t-1}\right)=\operatorname{Cat}\left(\mathbf{x}_{t};\left(1-\beta_{t}\right)\mathbf{x}_{t-1}+\beta_{t}/K\right),$
$\displaystyle
p_{\theta}\left(\mathbf{x}_{t-1}\mid\mathbf{x}_{t}\right)=\sum^{K}_{\hat{\mathbf{x}}_{0}=1}q\left(\mathbf{x}_{t-1}\mid\mathbf{x}_{t},\hat{\mathbf{x}}_{0}\right)p_{\theta}\left(\hat{\mathbf{x}}_{0}\mid\mathbf{x}_{t}\right),$

where $\operatorname{Cat}(\cdot)$ is a categorical distribution.

##  4\. Methodology

Figure 2. The overall pipeline of the training procedure of SimpDM. Given a
training sample $\mathbf{x}$ and its missing mask $\mathbf{m}$, the first step
is to apply average padding for the missing entries and sample the pseudo mask
$\mathbf{m}_{p}$ and condition mask $\mathbf{m}_{c}$. In self-supervised
alignment, we sample different $t$ and $\epsilon$ at two channels, and then
run the diffusion model at each channel. Apart from the diffusion model loss
$\mathcal{L}_{dm}$, we use a self-supervised alignment loss $\mathcal{L}_{sa}$
to minimize the distance of the predictions at two channels. We further use a
state-dependent augmentation strategy to perturb the model’s input according
to the states (GT, MS, or PM) of each entry.

In this section, we present SimpDM, a self-supervision improved diffusion
model for tabular data imputation. The overall training pipeline of SimpDM is
demonstrated in Fig. 2. In Sec. 4.1, we first introduce the base diffusion
model specially constructed for tabular data imputation. We then extend the
base model to SimpDM by introducing two pivot designs, i.e., state-dependent
self-supervised alignment (Sec. 4.2) and data augmentation (Sec. 4.3).
Finally, we discuss how to apply SimpDM to mixed-type tabular data in Sec.
4.4.

###  4.1. Diffusion Model for Data Imputation

We first introduce a Gaussian diffusion model for imputing tabular data with
numerical features. Note that the vanilla diffusion models (Ho et al., 2020)
are mainly designed for data generation, making it difficult to apply them
directly to imputation tasks (Lugmayr et al., 2022). Therefore, we implement a
series of modifications to adapt the diffusion model effectively for tabular
data imputation.

Hybrid input data. A major characteristic of the data imputation problem is
that the data is partly accessible. In other words, we have the observed
entries to guide the estimation of the missing ones. To involve the observed
entries in model input, we use a “hybrid input data” design in SimpDM. More
precisely, the model input consists of both the ground-truth values from
observed entries and the initialized/estimated values for the missing entries,
combined together. Such a design, similar to RePaint (Lugmayr et al., 2022)
for image inpainting, allows the diffusion model to effectively leverage known
information to infer missing data while preserving consistency across various
samples in the input space.

MLP-based diffusion model. As tabular data is usually low-dimensional and with
few data samples, complex network model architectures are prone to overfitting
and rather unnecessary. Since previous study (Kotelnikov et al., 2023) shows
that a shallow MLP-based diffusion model is adequate for tabular data
generation, in SimpDM, we use a plain MLP as our model, without U-net
architecture and cross-layer shortcut. The simple model architecture not only
substantially reduces the computational cost of our method but also guards
against overfitting the limited data. The learnable embedding of diffusion
step $t$ is added to the latent representation at the first layer. Notably, we
further introduce the missing mask as an extra input condition of the
diffusion model. In particular, we use a learnable projection layer to map the
missing mask vector to a mask embedding, and then add the mask embedding into
the first-layer latent representation. The mask embedding enables the
diffusion model to discern the status of the hybrid input, aiding in making
accurate imputations.

Pseudo missing training strategy. In the setting of data imputation problem, a
key challenge is that the ground-truth values of missing entries are unknown
during training procedure (Kyono et al., 2021). That is to say, we cannot
acquire supervision signals from these entries for model training. To address
this challenge, we develop a “pseudo missing” training strategy inspired by
CSDI (Tashiro et al., 2021). Firstly, during the training phase, we pad the
values of missing entries to their respective column-wise average values,
which can be regarded as an estimation via mean imputation. Then, in each
training iteration, we sample a pseudo missing mask to remask a partition of
observed entries. Subsequently, we merge the original missing mask and pseudo
mask to create the condition mask of the current training iteration. For the
entries where the condition mask equals $1$, we add Gaussian noise $\epsilon$
to them based on the sampled diffusion time step $t$. As for the remaining
entries, which act as “observed data” in the current iteration, we maintain
their original values. In this case, we can train the model by denoising the
pseudo missing entries since their ground-truth values are accessible.

Value prediction and loss calculation. In vanilla Gaussian diffusion models
(Ho et al., 2020), predicting the Gaussian noises of diffusion (Eq. (8)) has
been shown to be effective for data generation. However, in the context of
tabular data imputation, our empirical findings suggest that predicting the
missing values can result in improved imputation performance compared to
predicting noises. Hence, we directly let the model predict the ground-truth
values of the missing entries. When computing the loss function, we solely
calculate the loss on the pseudo missing terms, directing the model’s
attention toward completing the missing data.

To sum up, the training process of Gaussian diffusion model for tabular data
imputation can be expressed by:

(10)
$\mathcal{L}_{\text{dm}}=\mathbb{E}_{t,\mathbf{x}_{0},\boldsymbol{\epsilon},\mathbf{m}_{p}}\left[\left\|(\mathbf{x}_{0}-\mathbf{x}_{\theta}\left(\tilde{\mathbf{x}}_{t},t,\mathbf{m}_{c}\right))\odot\mathbf{m}_{p}\right\|^{2}\right],$
(11)
$\tilde{\mathbf{x}}_{t}={\mathbf{x}}_{0}\odot\left(\mathbbm{1}_{d}-\mathbf{m}_{c}\right)+\mathbf{x}_{t}\odot\mathbf{m}_{c},$

where $\mathbf{m}_{p}$ and $\mathbf{m}_{c}=\mathbf{m}+\mathbf{m}_{p}$ are the
pseudo mask and condition mask respectively, $\tilde{\mathbf{x}}_{t}$ is the
hybrid model input, and $\mathbf{x}_{\theta}$ is the MLP-based model. During
inference time, we can initialize the missing entries by $q(\mathbf{x}_{T})$
and set condition mask $\mathbf{m}_{c}=\mathbf{m}$ directly. Before each
inference denoising step, we execute Eq. (11) to ensure the observed data is
embedded in the model input.

###  4.2. Self-Supervised Alignment

With the diffusion model introduced in Sec. 4.1 (we denote it as “base model”
in the rest of this paper), we can complete missing entries with recurrent
denoising processes. Nevertheless, the base model struggles to achieve optimal
imputation performance due to the inherent target mismatch between imputation
and generation tasks. Specifically, to achieve the diversity target in
generation tasks, diffusion models are sensitive to the initial noise $x_{T}$.
With varying initial noises, the generated data should exhibit diversity.
However, this sensitivity contrasts with the goals of data imputation tasks,
which require precise prediction of missing values rather than diversity. To
further improve the imputation performance, we design a self-supervised
alignment mechanism to suppress this sensitivity.

Concretely, in the training procedure of SimpDM, we construct two parallel
channels to run the diffusion model for each sample. The two channels share
the same pseudo mask and condition mask. For each channel, we sample its
diffusion step (denoted by $t_{1}$ and $t_{2}$) and diffusion noise (denoted
by $\epsilon_{1}$ and $\epsilon_{2}$), respectively. With different $t$ and
$\epsilon$, we can execute the forward diffusion process to generate
$x_{t_{1}}$ and $x_{t_{2}}$, and finally acquire the corresponding hybrid
inputs $\tilde{x}_{t_{1}}$ and $\tilde{x}_{t_{2}}$ via Eq. (11). Since they
have a shared condition mask, the observed data in $\tilde{x}_{t_{1}}$ and
$\tilde{x}_{t_{2}}$ are identical. As the example shown in Fig. 2, both of the
two channels have an observed value $-0.45$. Differently, due to the disparity
between $x_{t_{1}}$ and $x_{t_{2}}$, the pseudo missing and real missing
entries in $\tilde{x}_{t_{1}}$ and $\tilde{x}_{t_{2}}$ exhibit notable
differences.

Recalling that our goal is to suppress the sensitivity to diverse noisy
inputs. In particular, given two input data with the same observed entries
(i.e., $\tilde{x}_{t_{1}}$ and $\tilde{x}_{t_{2}}$), the outputs of the
diffusion model should be close to each other. Motivated by this, we acquire
their corresponding outputs
$\hat{\mathbf{x}}_{1}=\mathbf{x}_{\theta}\left(\tilde{\mathbf{x}}_{t_{1}},{t_{1}},\mathbf{m}_{c}\right)$
and
$\hat{\mathbf{x}}_{2}=\mathbf{x}_{\theta}\left(\tilde{\mathbf{x}}_{t_{2}},{t_{2}},\mathbf{m}_{c}\right)$,
and try to minimize the difference between them with a self-supervised
alignment loss $\mathcal{L}_{sa}(\hat{\mathbf{x}}_{1},\hat{\mathbf{x}}_{2})$.
At the same time, we calculate the basic loss of the diffusion model (Eq.
(10)) at two channels, writing the final loss function of SimpDM as:

(12)
$\mathcal{L}=\mathcal{L}_{dm1}(\hat{\mathbf{x}}_{1})+\mathcal{L}_{dm2}(\hat{\mathbf{x}}_{2})+\gamma\mathcal{L}_{sa}(\hat{\mathbf{x}}_{1},\hat{\mathbf{x}}_{2}),$

where $\gamma$ is a tunable trade-off hyper-parameter for $\mathcal{L}_{sa}$.
In practice, we have several options for $\mathcal{L}_{sa}$, such as MSE loss,
contrastive loss (Chen et al., 2020), and Sinkhorn divergence (Muzellec et
al., 2020). Considering its empirical performance (see Sec. 5.4) and high
efficiency, we employ MSE loss for self-supervised alignment in SimpDM.

###  4.3. State-Dependent Data Augmentation

Another significant challenge in tabular data imputation is the limited size
of the dataset, which might not offer adequate information for diffusion
models to learn the data manifold effectively. Consequently, the model can
lean towards overfitting due to data scarcity, diminishing the overall
robustness of diffusion models. To address this problem, data augmentation
emerges as a promising solution, creating extra synthetic samples from the
original ones (Shorten and Khoshgoftaar, 2019; Kulikov et al., 2023). Despite
various data augmentation methods designed for image data, most of them cannot
be applied to tabular data, which motivates us to produce a well-crafted
augmentation strategy for our target scenario.

To augment tabular data, data perturbation with random noises (e.g., uniform,
Gaussian, or others) can be a simple yet effective strategy (Sathianarayanan
et al., 2022; Cai and Fan, 2022; Wang et al., 2024). Nevertheless, this simple
strategy may not fully adapt to the diffusion model for SimpDM where input
entries are missing or already noisy. Specifically, weak perturbations might
minimally affect the missing or uncertain input entries since they are already
noisy. However, increasing the perturbation strength significantly shifts the
augmented data away from the original data distribution.

Table 1. Imputation performance comparison in terms of RMSE. The best and
runner-up performances are highlighted by bold and underline, respectively.
“OOM” indicates Out-Of-Memory on a 16GB GPU.

Method | Iris | Yacht | Housing | Diabetes | Blood | Energy | German | Concrete | Yeast
---|---|---|---|---|---|---|---|---|---
mean | .2634$\pm$.0054 | .2970$\pm$.0072 | .2471$\pm$.0140 | .4595$\pm$.0134 | .1654$\pm$.0036 | .3535$\pm$.0019 | .3163$\pm$.0185 | .2255$\pm$.0017 | .1185$\pm$.0009
KNN | .1428$\pm$.0057 | .2630$\pm$.0057 | .1352$\pm$.0123 | .3339$\pm$.0084 | .1287$\pm$.0033 | .2408$\pm$.0022 | .2968$\pm$.0217 | .1746$\pm$.0032 | .1164$\pm$.0036
MF | .1428$\pm$.0139 | .2472$\pm$.0086 | .1458$\pm$.0150 | .4579$\pm$.0134 | .1201$\pm$.0047 | .2291$\pm$.0039 | .3106$\pm$.0261 | .1776$\pm$.0032 | .1122$\pm$.0011
MICE | .1454$\pm$.0142 | .2805$\pm$.0080 | .1852$\pm$.0162 | .4470$\pm$.0144 | .1337$\pm$.0045 | .2606$\pm$.0024 | .3374$\pm$.0207 | .2004$\pm$.0018 | .1286$\pm$.0003
OT | .1393$\pm$.0108 | .2742$\pm$.0051 | .1657$\pm$.0174 | .3255$\pm$.0096 | .1442$\pm$.0033 | .2466$\pm$.0025 | .3078$\pm$.0197 | .1747$\pm$.0043 | .1164$\pm$.0008
MIRACLE | .1364$\pm$.0183 | .2560$\pm$.0084 | .1633$\pm$.0161 | .3573$\pm$.0488 | .1157$\pm$.0028 | .2294$\pm$.0023 | .3027$\pm$.0214 | .1719$\pm$.0028 | .1118$\pm$.0013
GRAPE | .1343$\pm$.0206 | .2450$\pm$.0104 | .1382$\pm$.0172 | .3187$\pm$.0097 | .1194$\pm$.0067 | .2365$\pm$.0071 | .2981$\pm$.0191 | .1380$\pm$.0064 | .1144$\pm$.0012
IGRM | .1197$\pm$.0123 | .2391$\pm$.0077 | .1363$\pm$.0175 | .3235$\pm$.0127 | .1182$\pm$.0047 | .1863$\pm$.0049 | .3032$\pm$.0206 | .1240$\pm$.0038 | .1161$\pm$.0013
MIWAE | .1323$\pm$.0160 | .2669$\pm$.0027 | .1572$\pm$.0111 | .3750$\pm$.0223 | .1280$\pm$.0047 | .2528$\pm$.0012 | .3515$\pm$.0185 | .1894$\pm$.0068 | .1236$\pm$.0012
GAIN | .1353$\pm$.0131 | .2526$\pm$.0082 | .1601$\pm$.0167 | .3942$\pm$.0166 | .1537$\pm$.0099 | .2702$\pm$.0097 | .3242$\pm$.0199 | .2203$\pm$.0021 | .1241$\pm$.0033
TabCSDI | .1365$\pm$.0058 | .2588$\pm$.0062 | .1577$\pm$.0153 | .3636$\pm$.0132 | .1374$\pm$.0058 | .2611$\pm$.0047 | .2997$\pm$.0223 | .2477$\pm$.0036 | .1198$\pm$.0009
FD | .1392$\pm$.0058 | .2509$\pm$.0081 | .1592$\pm$.0106 | .3789$\pm$.0044 | .1346$\pm$.0040 | .2592$\pm$.0053 | .3024$\pm$.0309 | .1927$\pm$.0016 | .1192$\pm$.0017
SimpDM | .1083$\pm$.0087 | .2324$\pm$.0141 | .1338$\pm$.0119 | .2937$\pm$.0031 | .1088$\pm$.0046 | .2194$\pm$.0053 | .2889$\pm$.0194 | .1366$\pm$.0008 | .1107$\pm$.0010
Method | Airfoil | Wine-red | Abalone | Wine-white | Phoneme | Power | Ecommerce | California | Average Rank
mean | .2898$\pm$.0024 | .1298$\pm$.0060 | .1952$\pm$.0011 | .1051$\pm$.0087 | .1622$\pm$.0007 | .1966$\pm$.0007 | .3193$\pm$.0066 | .1463$\pm$.0003 | 11.7
KNN | .2513$\pm$.0018 | .0931$\pm$.0013 | .1305$\pm$.0006 | .0811$\pm$.0042 | .1344$\pm$.0009 | .1571$\pm$.0015 | .2959$\pm$.0113 | .1419$\pm$.0004 | 5.2
MF | .2379$\pm$.0023 | .0980$\pm$.0025 | .1461$\pm$.0021 | .0858$\pm$.0066 | .1369$\pm$.0010 | .1512$\pm$.0012 | .3654$\pm$.0105 | .1186$\pm$.0005 | 6.1
MICE | .2929$\pm$.0015 | .1071$\pm$.0028 | .1420$\pm$.0010 | .0920$\pm$.0076 | .1780$\pm$.0006 | .1631$\pm$.0014 | .3606$\pm$.0079 | .1292$\pm$.0007 | 10.2
OT | .2726$\pm$.0039 | .0998$\pm$.0040 | .1499$\pm$.0018 | .0865$\pm$.0070 | .1573$\pm$.0008 | .1871$\pm$.0011 | .3177$\pm$.0067 | .1426$\pm$.0004 | 8.2
MIRACLE | .2577$\pm$.0024 | .0942$\pm$.0027 | .1293$\pm$.0012 | .0865$\pm$.0070 | .1559$\pm$.0006 | .1442$\pm$.0010 | .3142$\pm$.0056 | .1175$\pm$.0005 | 4.9
GRAPE | .2289$\pm$.0045 | .0897$\pm$.0033 | .1314$\pm$.0061 | .0982$\pm$.0198 | .1237$\pm$.0012 | .1338$\pm$.0016 | .3071$\pm$.0082 | .0997$\pm$.0006 | 3.5
IGRM | .1797$\pm$.0033 | .0860$\pm$.0027 | .1358$\pm$.0063 | .0981$\pm$.0206 | .1228$\pm$.0011 | OOM | OOM | OOM | 3.2
MIWAE | .2517$\pm$.0020 | .1138$\pm$.0000 | .1302$\pm$.0023 | .0864$\pm$.0074 | .1633$\pm$.0015 | .1578$\pm$.0028 | .3405$\pm$.0104 | .1285$\pm$.0005 | 7.6
GAIN | .2581$\pm$.0019 | .1140$\pm$.0044 | .1493$\pm$.0113 | .1024$\pm$.0076 | .1685$\pm$.0027 | .1626$\pm$.0050 | .3450$\pm$.0119 | .1311$\pm$.0007 | 9.9
TabCSDI | .2473$\pm$.0026 | .0994$\pm$.0078 | .1487$\pm$.0035 | .0952$\pm$.0079 | .1600$\pm$.0015 | .1526$\pm$.0020 | .3250$\pm$.0112 | .1287$\pm$.0010 | 7.8
FD | .2458$\pm$.0020 | .1026$\pm$.0026 | .1468$\pm$.0013 | .0976$\pm$.0118 | .1596$\pm$.0014 | .1633$\pm$.0011 | .3357$\pm$.0087 | .1289$\pm$.0004 | 7.7
SimpDM | .2024$\pm$.0046 | .0873$\pm$.0016 | .1204$\pm$.0012 | .0727$\pm$.0056 | .1165$\pm$.0004 | .1320$\pm$.0009 | .3051$\pm$.0111 | .1056$\pm$.0006 | 1.4

On the basis of Gaussian noise-based data perturbation, we propose a state-
dependent data augmentation. The core idea is to perturb entries in different
states with different strengths (i.e., Gaussian variance). Concretely, given
an input vector of SimpDM, each entry can be in three “states”: ground-truth
state (GT) where $\mathbf{m}_{c}=0$, pseudo missing state (PM) where
$\mathbf{m}_{p}=0$, and missing state (MS) where $\mathbf{m}=1$. For entries
in different states, their certainties can be quite different: for a GT entry,
the data is fully reliable; for a PM entry, its certainty is moderate, since
this entry is generated by adding ground-truth data with $t$-related noise;
for a MS entry, it has the lowest certainty because we know nothing about the
truth value of this entry. In our state-dependent data augmentation strategy,
we propose to define the perturbation strength $p$ according to states.
Concretely, for entries with lower certainty, we will allocate a higher
perturbation strength to it. Formally, in training phase, Eq. (11) can be
rewritten by:

(13)
$\displaystyle\tilde{\mathbf{x}}_{t}={\mathbf{x}}_{0}\odot\left(\mathbbm{1}_{d}-\mathbf{m}_{c}\right)+\mathbf{x}_{t}\odot\mathbf{m}_{c}+\boldsymbol{\xi}\odot\mathbf{p},$
$\displaystyle\mathbf{p}=\left(\mathbbm{1}_{d}-\mathbf{m}\right)\times
p_{GT}+\mathbf{m}_{p}\times p_{PM}+\mathbf{m}\times p_{MS},$

where $\boldsymbol{\xi}$ is a zero-centered perturbation noise vector,
$\mathbf{p}$ is the perturbation strength vector, $p_{GT}$, $p_{PM}$, and
$p_{MS}$ are the perturbation strengths for the corresponding states that
satisfy $p_{GT}<p_{PM}<p_{MS}$. With state-dependent augmentation, we can
generate more reliable training samples for model training without disrupting
the original data distribution. As a result, the model robustness can be
boosted; Also, data augmentation can further facilitate self-supervised
learning by amplifying the diversity within the data across two channels. The
overall algorithm and complexity analysis of SimpDM is demonstrated in
Appendix A and B, respectively.

###  4.4. Extending to Mixed-Type Data

Previous subsections introduce the Gaussian diffusion version of SimpDM for
pure numerical tabular data. In practice, we can easily extend SimpDM to
mixed-type data via the following simple modifications for categorical
features.

At the diffusion model stage, we introduce multinomial diffusion process for
categorical features, while keeping Gaussian diffusion process for numerical
features. Similar to TabDDPM (Kotelnikov et al., 2023), each categorical
feature can be assigned a $K$-dimensional prediction head (where $K$ is the
number of categories), followed by a Softmax function. The cross-entropy loss
is used to optimize the categorical output. For self-supervised alignment, we
minimize the output scores of each prediction head. For state-dependent data
augmentation, we use random state transformation (Hoogeboom et al., 2021) as
the perturbation for categorical data.

##  5\. Experiments

###  5.1. Experimental Settings

####  5.1.1. Dataset

We evaluate the imputation performance on 17 real-world datasets across
various domains from the UCI Machine Learning repository (Asuncion and Newman,
2007) and Kaggle, including Iris, Yacht, Housing, Diabetes, Blood, Energy,
German, Concrete, Yeast, Airfoil, Wine-red, Abalone, Wine-white, Phoneme,
Power, Ecommerce, and California. We use the first two letters of each dataset
name to indicate it for simplification purposes. The statistics of datasets
are given in Appendix C.

####  5.1.2. Baselines

We compare SimpDM with three groups of methods: 1) shallow methods, including
mean imputation, kNN imputation (Troyanskaya et al., 2001), miss forest (MF)
(Stekhoven and Bühlmann, 2012), and MICE (Van Buuren and Groothuis-Oudshoorn,
2011); 2) deep methods, including OT (Muzellec et al., 2020), MIRACLE (Kyono
et al., 2021), GRAPE (You et al., 2020), and IGRM (Zhong et al., 2023); 3)
deep generative methods, including MIWAE (Mattei and Frellsen, 2019), GAIN
(Yoon et al., 2018), TabCSDI (Zheng and Charoenphakdee, 2022), and
ForestDiffusion (FD) (Jolicoeur-Martineau et al., 2023). Among them, TabCSDI
and FD are also based on diffusion models.

####  5.1.3. Implementation Details

In our major experiments, we simulate the default data missing setting as
missing with complete random (MCAR) scenario with $30\%$ missing ratio. We use
Root Mean Squared Error (RMSE) as our evaluation metric, which is commonly
used in previous works (Kyono et al., 2021; Muzellec et al., 2020). We report
the averaged test accuracy over 5 runs of experiments. We reproduce baselines
based on HyperImpute package (Jarrett et al., 2022) or their corresponding
official source codes. We select some important hyper-parameters through grid
search, and keep the rest insensitive hyper-parameters to be fixed values.
Concretely, the grid search is carried out on the following search space:

  * •

Diffusion steps: {10, 50, 100}

  * •

Training epochs: {10000, 20000, 30000}

  * •

Learning rate: {0.001, 0.0001}

  * •

The number of layers: {3,4,5}

  * •

Hidden dimensions: {256, 512, 1024}

  * •

Trade-off parameter of self-supervised alignment loss $\gamma$: {0.2, 0.5, 1,
3, 5, 10}

  * •

Perturbation strengths [$p_{GT}$,$p_{PM}$,$p_{MS}$]:
{ [0.0001,0.08,0.1], [0.0001,0.4,0.5], [0.001,0.8,1], [0.001,2,3] }

####  5.1.4. Computing Infrastructures

We run all experiments with an Amazon Web Service (AWS) EC2 instance with the
instance size g4dn.xlarge, which features a 4-core CPU, 16 GB Memory, and a
Nvidia T4 GPU with 16 GB GPU Memory. We implement the proposed SimpDM with
Python 3.10 and PyTorch 1.11.0 (Paszke et al., 2019).

####  5.1.5. Source Code

The source code of SimpDM is available at:
https://github.com/yixinliu233/SimpDM.

###  5.2. Performance Comparison

The performance comparison in the default setting is illustrated in Table 1.
From the results, we have the following observations.

  1. (1)

SimpDM outperforms all the baselines on 11 datasets while achieving runner-up
results on the rest 6 datasets. The consistent superiority of a majority of
datasets highlights the adaptability and robustness of our method in
addressing missing data imputation challenges.

  2. (2)

Compared to diffusion model-based methods (i.e., TabCSDI and FD), SimpDM has
significantly better performance, indicating the effectiveness of self-
supervised alignment and state-dependent augmentation.

  3. (3)

Among generative methods, two diffusion model-based approaches perform
slightly better than the GAN/VAE-based methods. This observation demonstrates
the potential of diffusion models in addressing imputation problems.

  4. (4)

Deep learning-based methods, such as MIRACLE, GRAPE, and IGRM, show
competitive performance, indicating the superior capability of deep neural
networks in data completion.

  5. (5)

In the realm of shallow methods, kNN and MF exhibit strong performance,
underscoring their superior capability in tabular data imputation.

  6. (6)

Compared to the strongest baseline IGRM, SimpDM requires less memory,
indicating the efficiency and scalability of our approach.

Figure 3. Imputation performance on MAR and MNAR scenarios. Figure 4.
Imputation performance under different missing ratios.

###  5.3. Different Missing Scenarios and Ratios

To investigate the generalization ability in various missing situations, we
conduct experiments on different missing scenarios (i.e., missing at random
(MAR) and missing not at random (MNAR)) and missing ratios. For the sake of
space, we compare SimpDM with five highly competitive baselines, namely kNN,
MF, MIRACLE, GRAPE, and IGRM. The experiments are conducted on four datasets,
namely, Yeast (YE), Abalone (AB), Wine-white (WW), and Power (PO).

####  5.3.1. Different missing scenarios

We conduct experiments to assess the effectiveness of SimpDM in both MAR and
MNAR scenarios, with results detailed in Fig. 3. The figures demonstrate that
SimpDM consistently surpasses all representative baselines in both scenarios,
highlighting its robust generalization capabilities. Conversely, certain
baselines exhibit suboptimal and unstable performance in certain cases.

####  5.3.2. Different missing ratios

To verify the robustness of SimpDM across varying degrees of data missingness,
we alter the missing ratios from $0.1$ to $0.7$, assessing its imputation
performance across diverse scenarios. As depicted in Fig. 4, SimpDM
consistently demonstrates optimal or competitive performance across various
missing ratios. Notably, in situations of severe data absence, SimpDM exhibits
more substantial performance gains compared to the deep baselines.

###  5.4. Ablation Studies

####  5.4.1. Effect of key components

To examine the contributions of two key components in SimpDM, namely self-
supervised alignment (SA) and state-dependent augmentation (AUG), we conduct
ablation experiments on our base model and its two variants, each
incorporating one of these components. From the results in Table 2, we can
find that both SA and AUG yield substantial performance enhancements for the
base model. Furthermore, SA demonstrates a more pronounced impact on
performance across the majority of datasets. Finally, we can witness that
SimpDM, which incorporates both of these techniques, exhibits the best
performance.

(a) Runtime Analysis

(b) Case study

(c) Visualization

Figure 5. (a) Runtime per training epoch on different datasets (with size
$n\times d$). (b) The imputation results from different initialization. (c)
t-SNE visualization of ground-truth data and imputed data.

####  5.4.2. Effect of different self-supervised losses

In SimpDM, self-supervised alignment loss is a critical component to eliminate
the unexpected diversity of imputation results. We explore three self-
supervised alignment loss types: MSE loss, contrastive loss (CL) (Chen et al.,
2020), and Sinkhorn divergence (SK) (Muzellec et al., 2020) (refer to Appendix
xx for definitions), evaluating their respective effectiveness. As illustrated
in Table 3, MSE loss attains the optimal results on 7 out of 8 datasets and
exhibits competitive performance on the remaining one. We attribute the
superior performance of MSE loss to its consistency with the objective of
aligning the imputation results of two channels. Simultaneously, CL and SK
losses also yield improvements, underscoring the effectiveness of self-
supervised alignment.

Table 2. Ablation study for the key components in SimpDM.

Variant | YA | DI | EN | YE | WR | AB | WW | PO
---|---|---|---|---|---|---|---|---
Base model | .294 | .307 | .286 | .149 | .100 | .163 | .089 | .169
+SA | .243 | .298 | .232 | .118 | .091 | .130 | .076 | .140
+AUG | .241 | .297 | .241 | .123 | .092 | .132 | .077 | .149
SimpDM | .232 | .294 | .219 | .111 | .087 | .120 | .073 | .132

Table 3. Ablation study for self-supervised losses.

Variant | YA | DI | EN | YE | WR | AB | WW | PO
---|---|---|---|---|---|---|---|---
Base model | .294 | .307 | .286 | .149 | .100 | .163 | .089 | .169
+SA (CL) | .261 | .307 | .251 | .121 | .095 | .138 | .081 | .151
+SA (SK) | .259 | .295 | .259 | .134 | .092 | .148 | .080 | .154
+SA (MSE) | .243 | .298 | .232 | .118 | .091 | .130 | .076 | .140

Table 4. Ablation study for augmentation strategies.

Variant | YA | DI | EN | YE | WR | AB | WW | PO
---|---|---|---|---|---|---|---|---
Base model | .294 | .307 | .286 | .149 | .100 | .163 | .089 | .169
+AUG (strong) | .298 | .384 | .280 | .139 | .127 | .172 | .099 | .184
+AUG (weak) | .296 | .306 | .290 | .149 | .099 | .165 | .090 | .171
+AUG (SD) | .241 | .297 | .241 | .123 | .092 | .132 | .077 | .149

####  5.4.3. Effect of different augmentation strategies

To verify the effectiveness of state-dependent (SD) augmentation, we compare
it with two plain data augmentation strategies that perturb all entries with
strong or weak strengths. From Table 4, we can see that weak data augmentation
does not deeply affect the performance, whereas strong data augmentation tends
to have a negative impact since it distorts the distribution of the original
data. Conversely, state-dependent augmentation brings improvement by applying
customized perturbation strengths to each entry according to its state and
certainty.

###  5.5. Runtime Analysis

We present the runtime of each training epoch of SimpDM on different datasets.
All the experiments are conducted on an Amazon EC2 server (see Sec. 5.1) and
with a fixed-size MLP diffusion model (with 3 layers and 256 hidden units).
The experimental results are illustrated in Fig. 5(a). The figure clearly
illustrates that the training time of SimpDM exhibits a linear relationship
with both the number of samples $n$ and the number of dimensions $d$. This
observation aligns seamlessly with our analysis detailed in Appendix B. Also,
the training cost of SimpDM is quite small (with few milliseconds for each
epoch), indicating the high running efficiency and scalability of our method.

###  5.6. Qualitative Analysis

####  5.6.1. Case study

We perform a case study experiment on a sample from the Power dataset (the
same one as Fig. 1(a)) to investigate whether SimpDM can deliver stable
imputation results from diverse initial noises. From Fig. 5(b), it is evident
that SimpDM yields both increased stability and enhanced accuracy. This
improvement can be attributed to the integration of the self-supervised
alignment mechanism.

####  5.6.2. Visualization

Using t-SNE algorithm (Van der Maaten and Hinton, 2008), we visualize the
distribution of the original Iris dataset and the data imputed by the base
model and SimpDM, respectively. In Fig. 5(c), we observe a significant overlap
between the distribution of data imputed by SimpDM and the original data
distribution, which suggests that SimpDM adeptly captures the data manifold.
In contrast, the base diffusion model struggles to match the data
distribution.

##  6\. Conclusion

In this paper, we introduce a novel variant of diffusion models, termed
SimpDM, designed for tabular data imputation. To enhance the imputation
capabilities of the diffusion model, we propose a self-supervised alignment
mechanism aimed at reducing its sensitivity to noise, thereby improving the
stability of imputation results. Simultaneously, to address discrepancies in
data scale, we present a state-dependent augmentation strategy that generates
synthetic data during model training, which aims to bolster the robustness of
SimpDM. Extensive experiments showcase the superior performance of SimpDM
across various scenarios.

##  Appendix A Algorithm of SimpDM

Algorithm 1 SimpDM Training

Input: Sample $\mathbf{x}$, missing mask $\mathbf{m}$

1:
$\mathbf{m}_{p}\sim\operatorname{Bernoulli}(\left(\mathbbm{1}_{n}-\mathbf{m}\right)\times
r_{m})$

2: $\mathbf{m}_{c}=\mathbf{m}_{p}+\mathbf{m}$

3: for $k=1,2$ do

4: $t_{k}\sim\operatorname{Uniform}(\\{0,\ldots,T\\})$

5: $\epsilon_{k}\sim\mathcal{N}(0,\boldsymbol{I})$

6: $\xi_{k}\sim\mathcal{N}(0,\boldsymbol{I})$

7: Calculate $\tilde{\mathbf{x}}_{t_{k}}$ via Eq. (13)

8:
$\hat{\mathbf{x}}_{k}=\mathbf{x}_{\theta}\left(\tilde{\mathbf{x}}_{t_{k}},{t_{k}},\mathbf{m}_{c}\right)$

9: Calculate $\mathcal{L}_{dm_{k}}$ via Eq. (10)

10: end for

11: Calculate
$\mathcal{L}_{sa}=\operatorname{MSE}(\hat{\mathbf{x}}_{1},\hat{\mathbf{x}}_{2})$

12: Calculate overall loss $\mathcal{L}$ via Eq. (12)

13: Update model $\mathbf{x}_{\theta}$ by taking gradient descent step on
$\mathcal{L}$

Algorithm 2 SimpDM Imputing

Input: Sample $\mathbf{x}$, missing mask $\mathbf{m}$

1: $\mathbf{x}_{T}\sim\mathcal{N}(0,\boldsymbol{I})$

2: for $t=T,\cdots,1$ do

3: $\mathbf{z}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$ if $t>1$, else
$\mathbf{z}=\mathbf{0}$

4: Calculate $\tilde{\mathbf{x}}_{t}$ via Eq. (11)

5:
$\mathbf{x}_{t-1}=\frac{1}{\sqrt{\alpha_{t}}}\big{(}\mathbf{x}_{\theta}\left(\tilde{\mathbf{x}}_{t},{t},\mathbf{m}\right)\big{)}+\sigma_{t}\mathbf{z}$

6: end for

7: $\hat{\mathbf{x}}=\mathbf{x}\odot\left(\mathbbm{1}_{n\times
d}-\mathbf{m}\right)+{\mathbf{x}}_{0}\odot\mathbf{m}$

8: return $\hat{\mathbf{x}}$

Table 5. The statistics of datasets, including the number of samples,
numerical features, and categorical features.

Dataset | #Samples | #Numerical Feat | #Categorical Feat
---|---|---|---
Iris | 150 | 4 | 0
Yacht | 308 | 6 | 0
Housing | 506 | 12 | 1
Diabetes | 520 | 1 | 15
Blood | 748 | 4 | 0
Energy | 767 | 8 | 0
German | 1,000 | 7 | 13
Concrete | 1,030 | 8 | 0
Yeast | 1,484 | 8 | 0
Airfoil | 1,503 | 5 | 0
Wine-red | 1,599 | 11 | 0
Abalone | 4,177 | 7 | 1
Wine-white | 4,898 | 11 | 0
Phoneme | 5,404 | 5 | 0
Power | 9,568 | 4 | 0
Ecommerce | 10,999 | 3 | 7
California | 20,640 | 8 | 0

In this section, we provide the training and inference algorithms of SimpDM
with the example of a single data. In practice, we employ mini-batch-based
training. Here we use Gaussian noise as the example of perturbation and employ
MSE loss as the self-supervised alignment loss.

The training algorithm of SimpDM is summarized in Algorithm 1. In the first
step, we sample the pseudo mask $\mathbf{m}_{p}$ for the missing training
strategy. The condition mask can be computed by adding $\mathbf{m}_{p}$ and
the original missing mask $\mathbf{m}$. After that, we run the diffusion model
on two channels ($k=1,2$) respectively. In each channel, we sample the
diffusion time step $t_{k}$ and noise $\epsilon_{k}$ first. Then, we conduct
the state-dependent augmentation with an extra sampled perturbation $\xi_{k}$
(Line 6-7). With the perturbed input, we obtain the imputed data with the
diffusion model (Line 8) and calculate the diffusion model loss
$\mathcal{L}_{{dm}_{k}}$ accordingly (Line 9). Once we obtain the imputed data
by two channels ($\hat{\mathbf{x}}_{1}$ and $\hat{\mathbf{x}}_{2}$), we
calculate the self-supervised alignment loss $\mathcal{L}_{sa}$. Finally, we
can add the losses together and train the diffusion model
$\mathbf{x}_{\theta}$ via gradient descent.

The inference (imputation) of SimpDM is demonstrated in Algorithm 2. Similar
to the vanilla diffusion model, we first initialize $\mathbf{x}_{T}$ with
random Gaussian noise, and then recursively run the denoising iteration with
the diffusion model. Differently, in each iteration, we set the observed
entries to the ground-truth values (Line 4) before each denoising step, which
ensures the observed values can well guide the imputation process. Finally,
the imputed data $\hat{\mathbf{x}}$ can be obtain by combining $\mathbf{x}$
and the final estimation $\mathbf{x}_{0}$ (Line 7).

##  Appendix B Complexity Analysis

We study the time complexity of a single training epoch of SimpDM, given a
tabular dataset with $n$ samples and $d$ dimensions. For the sampling of
pseudo mask, Gaussian noise, and augmented perturbation, their complexities
are all $\mathcal{O}(nd)$. For the diffusion model, the time complexity is
$\mathcal{O}(nd_{h}(d+d_{h}L))$, where $d_{h}$ and $L$ are the latent
dimensions and the layer number of MLP, respectively. For the diffusion model
loss and the MSE self-supervised alignment loss, the complexities are also
$\mathcal{O}(nd)$. Note that if we use more complex self-supervised alignment
loss (e.g., contrastive loss or Sinkhorn loss), the complexity can be higher
to $\mathcal{O}(n^{2}d)$, which severely reduces the running efficiency. After
eliminating the smaller terms, the training time complexity of SimpDM becomes
$\mathcal{O}(nd_{h}(d+d_{h}L))$. For the testing phase of SimpDM, the
complexity is also similar to vanilla diffusion models. In specific, the
complexity is $\mathcal{O}(ndT)$ for the whole process, since we require $T$
diffusion time steps for data refinement. To sum up, this complexity of SimpDM
scales linearly with both $n$ and $d$, resembling the computational costs of
vanilla diffusion models.

##  Appendix C Dataset

We conduct the experiments on 17 datasets from the UCI Machine Learning
repository (Asuncion and Newman, 2007) and Kaggle, including Iris, Yacht,
Housing, Diabetes, Blood, Energy, German, Concrete, Yeast, Airfoil, Wine-red,
Abalone, Wine-white, Phoneme, Power, Ecommerce, and California. The statistics
of datasets are provided in Table 5.

## References

  * (1)
  * Alaa et al. (2017) Ahmed M Alaa, Jinsung Yoon, Scott Hu, and Mihaela Van der Schaar. 2017.  Personalized risk scoring for critical care prognosis using mixtures of gaussian processes.  _IEEE Transactions on Biomedical Engineering_ 65, 1 (2017), 207–218.
  * Assefa et al. (2020) Samuel A Assefa, Danial Dervovic, Mahmoud Mahfouz, Robert E Tillman, Prashant Reddy, and Manuela Veloso. 2020.  Generating synthetic data in finance: opportunities, challenges and pitfalls. In _Proceedings of the First ACM International Conference on AI in Finance_. 1–8.
  * Asuncion and Newman (2007) Arthur Asuncion and David Newman. 2007.  UCI machine learning repository.
  * Cai and Fan (2022) Jinyu Cai and Jicong Fan. 2022.  Perturbation learning based anomaly detection. In _Advances in Neural Information Processing Systems_. 14317–14330.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020.  A simple framework for contrastive learning of visual representations. In _International conference on machine learning_. PMLR, 1597–1607.
  * Croitoru et al. (2023) Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. 2023.  Diffusion models in vision: A survey.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ (2023).
  * Dhariwal and Nichol (2021) Prafulla Dhariwal and Alexander Nichol. 2021.  Diffusion models beat gans on image synthesis.  _Advances in neural information processing systems_ 34 (2021), 8780–8794.
  * Farhangfar et al. (2007) Alireza Farhangfar, Lukasz A Kurgan, and Witold Pedrycz. 2007.  A novel framework for imputation of missing values in databases.  _IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans_ 37, 5 (2007), 692–709.
  * Hernandez et al. (2022) Mikel Hernandez, Gorka Epelde, Ane Alberdi, Rodrigo Cilla, and Debbie Rankin. 2022.  Synthetic data generation for tabular health records: A systematic review.  _Neurocomputing_ 493 (2022), 28–45.
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020.  Denoising diffusion probabilistic models.  _Advances in neural information processing systems_ 33 (2020), 6840–6851.
  * Hoogeboom et al. (2021) Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. 2021.  Argmax flows and multinomial diffusion: Learning categorical distributions.  _Advances in Neural Information Processing Systems_ 34 (2021), 12454–12465.
  * Hyvärinen and Dayan (2005) Aapo Hyvärinen and Peter Dayan. 2005.  Estimation of non-normalized statistical models by score matching.  _Journal of Machine Learning Research_ 6, 4 (2005).
  * Jarrett et al. (2022) Daniel Jarrett, Bogdan C Cebere, Tennison Liu, Alicia Curth, and Mihaela van der Schaar. 2022.  Hyperimpute: Generalized iterative imputation with automatic model selection. In _International Conference on Machine Learning_. 9916–9937.
  * Jolicoeur-Martineau et al. (2023) Alexia Jolicoeur-Martineau, Kilian Fatras, and Tal Kachman. 2023.  Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Trees.  _arXiv preprint arXiv:2309.09968_ (2023).
  * Judson et al. (2009) Richard Judson, Ann Richard, David J Dix, Keith Houck, Matthew Martin, Robert Kavlock, Vicki Dellarco, Tala Henry, Todd Holderman, Philip Sayre, et al. 2009\.  The toxicity data landscape for environmental chemicals.  _Environmental health perspectives_ 117, 5 (2009), 685–695.
  * Kim et al. (2023) Jayoung Kim, Chaejeong Lee, and Noseong Park. 2023.  STaSy: Score-based Tabular data Synthesis. In _The Eleventh International Conference on Learning Representations_.
  * Kong et al. (2021) Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. 2021.  DiffWave: A Versatile Diffusion Model for Audio Synthesis. In _International Conference on Learning Representations_.
  * Kotelnikov et al. (2023) Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. 2023.  Tabddpm: Modelling tabular data with diffusion models. In _International Conference on Machine Learning_. PMLR, 17564–17579.
  * Kulikov et al. (2023) Vladimir Kulikov, Shahar Yadin, Matan Kleiner, and Tomer Michaeli. 2023.  Sinddm: A single image denoising diffusion model. In _International Conference on Machine Learning_. PMLR, 17920–17930.
  * Kyono et al. (2021) Trent Kyono, Yao Zhang, Alexis Bellot, and Mihaela van der Schaar. 2021.  Miracle: Causally-aware imputation via learning missing data mechanisms.  _Advances in Neural Information Processing Systems_ 34 (2021), 23806–23817.
  * Lee et al. (2023) Chaejeong Lee, Jayoung Kim, and Noseong Park. 2023.  CoDi: Co-Evolving Contrastive Diffusion Models for Mixed-Type Tabular Synthesis. In _International Conference on Machine Learning_. PMLR.
  * Liang et al. (2024) Ke Liang, Lingyuan Meng, Sihang Zhou, Wenxuan Tu, Siwei Wang, Yue Liu, Meng Liu, Long Zhao, Xiangjun Dong, and Xinwang Liu. 2024.  MINES: Message Intercommunication for Inductive Relation Reasoning over Neighbor-Enhanced Subgraphs. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 38. 10645–10653.
  * Liu et al. (2024) Yixin Liu, Shiyuan Li, Yu Zheng, Qingfeng Chen, Chengqi Zhang, and Shirui Pan. 2024.  ARC: A Generalist Graph Anomaly Detector with In-Context Learning.  _arXiv preprint arXiv:2405.16771_ (2024).
  * Liu et al. (2023) Yue Liu, Ke Liang, Jun Xia, Sihang Zhou, Xihong Yang, Xinwang Liu, and Stan Z Li. 2023.  Dink-net: Neural clustering on large graphs. In _International Conference on Machine Learning_. PMLR, 21794–21812.
  * Lugmayr et al. (2022) Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. 2022.  Repaint: Inpainting using denoising diffusion probabilistic models. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 11461–11471.
  * Mattei and Frellsen (2019) Pierre-Alexandre Mattei and Jes Frellsen. 2019.  MIWAE: Deep generative modelling and imputation of incomplete data sets. In _International conference on machine learning_. PMLR, 4413–4423.
  * Muzellec et al. (2020) Boris Muzellec, Julie Josse, Claire Boyer, and Marco Cuturi. 2020.  Missing data imputation using optimal transport. In _International Conference on Machine Learning_. PMLR, 7130–7140.
  * Osman et al. (2018) Muhammad S Osman, Adnan M Abu-Mahfouz, and Philip R Page. 2018.  A survey on data imputation techniques: Water distribution system as a use case.  _IEEE Access_ 6 (2018), 63279–63291.
  * Ouyang et al. (2023) Yidong Ouyang, Liyan Xie, Chongxuan Li, and Guang Cheng. 2023.  MissDiff: Training Diffusion Models on Tabular Data with Missing Values.  _arXiv preprint arXiv:2307.00467_ (2023).
  * Pan et al. (2024) Shirui Pan, Yizhen Zheng, and Yixin Liu. 2024.  Integrating graphs with large language models: Methods and prospects.  _IEEE Intelligent Systems_ 39, 1 (2024), 64–68.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019\.  Pytorch: An imperative style, high-performance deep learning library.  _Advances in Neural Information Processing Systems_ 32 (2019), 8026–8037.
  * Rasul et al. (2021) Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. 2021.  Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In _International Conference on Machine Learning_. PMLR, 8857–8868.
  * Sathianarayanan et al. (2022) Balachander Sathianarayanan, Yogesh Chandra Singh Samant, Prahalad S Conjeepuram Guruprasad, Varshin B Hariharan, and Nirmala Devi Manickam. 2022.  Feature-based augmentation and classification for tabular data.  _CAAI Transactions on Intelligence Technology_ 7, 3 (2022), 481–491.
  * Shorten and Khoshgoftaar (2019) Connor Shorten and Taghi M Khoshgoftaar. 2019.  A survey on image data augmentation for deep learning.  _Journal of big data_ 6, 1 (2019), 1–48.
  * Sohl-Dickstein et al. (2015) Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015.  Deep unsupervised learning using nonequilibrium thermodynamics. In _International conference on machine learning_. PMLR, 2256–2265.
  * Song and Ermon (2020) Yang Song and Stefano Ermon. 2020.  Improved techniques for training score-based generative models.  _Advances in neural information processing systems_ 33 (2020), 12438–12448.
  * Stekhoven and Bühlmann (2012) Daniel J Stekhoven and Peter Bühlmann. 2012.  MissForest—non-parametric missing value imputation for mixed-type data.  _Bioinformatics_ 28, 1 (2012).
  * Tashiro et al. (2021) Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. 2021.  Csdi: Conditional score-based diffusion models for probabilistic time series imputation.  _Advances in Neural Information Processing Systems_ 34 (2021), 24804–24816.
  * Telyatnikov and Scardapane (2023) Lev Telyatnikov and Simone Scardapane. 2023.  EGG-GAE: scalable graph neural networks for tabular data imputation. In _International Conference on Artificial Intelligence and Statistics_. PMLR, 2661–2676.
  * Troyanskaya et al. (2001) Olga Troyanskaya, Michael Cantor, Gavin Sherlock, Pat Brown, Trevor Hastie, Robert Tibshirani, David Botstein, and Russ B Altman. 2001.  Missing value estimation methods for DNA microarrays.  _Bioinformatics_ 17, 6 (2001), 520–525.
  * Van Buuren and Groothuis-Oudshoorn (2011) Stef Van Buuren and Karin Groothuis-Oudshoorn. 2011.  mice: Multivariate imputation by chained equations in R.  _Journal of statistical software_ 45 (2011), 1–67.
  * Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton. 2008.  Visualizing data using t-SNE.  _Journal of machine learning research_ 9, 11 (2008).
  * Wang et al. (2024) Yili Wang, Yixin Liu, Xu Shen, Chenyu Li, Kaize Ding, Rui Miao, Ying Wang, Shirui Pan, and Xin Wang. 2024.  Unifying Unsupervised Graph-Level Anomaly Detection and Out-of-Distribution Detection: A Benchmark.  _arXiv preprint arXiv:2406.15523_ (2024).
  * Xu et al. (2022) Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. 2022.  Geodiff: A geometric diffusion model for molecular conformation generation.  _arXiv preprint arXiv:2203.02923_ (2022).
  * Yang et al. (2022) Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. 2022.  Diffusion models: A comprehensive survey of methods and applications.  _Comput. Surveys_ (2022).
  * Yoon et al. (2018) Jinsung Yoon, James Jordon, and Mihaela Schaar. 2018.  Gain: Missing data imputation using generative adversarial nets. In _International conference on machine learning_. PMLR, 5689–5698.
  * You et al. (2020) Jiaxuan You, Xiaobai Ma, Yi Ding, Mykel J Kochenderfer, and Jure Leskovec. 2020.  Handling missing data with graph representation learning.  _Advances in Neural Information Processing Systems_ 33 (2020), 19075–19087.
  * Zheng and Charoenphakdee (2022) Shuhan Zheng and Nontawat Charoenphakdee. 2022.  Diffusion models for missing value imputation in tabular data.  _arXiv preprint arXiv:2210.17128_ (2022).
  * Zheng et al. (2023a) Xin Zheng, Yixin Liu, Zhifeng Bao, Meng Fang, Xia Hu, Alan Wee-Chung Liew, and Shirui Pan. 2023a.  Towards data-centric graph machine learning: Review and outlook.  _arXiv preprint arXiv:2309.10979_ (2023).
  * Zheng et al. (2023b) Yizhen Zheng, He Zhang, Vincent Lee, Yu Zheng, Xiao Wang, and Shirui Pan. 2023b.  Finding the missing-half: Graph complementary learning for homophily-prone and heterophily-prone graphs. In _International Conference on Machine Learning_. PMLR, 42492–42505.
  * Zhong et al. (2023) Jiajun Zhong, Ning Gui, and Weiwei Ye. 2023.  Data imputation with iterative graph reconstruction. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 37. 11399–11407.


# Hydrogen Atom in the Cosmic Microwave Background

Jose A. Magpantay<EMAIL_ADDRESS>Quezon City, Philippines

###### Abstract

The cosmic microwave background covers the entire universe which suggests the
absence of any closed system, except the universe itself. In this paper, I
consider the effect of the cosmic microwave background on the hydrogen atom,
which must be very small otherwise changes in energy levels would have been
measurable. But how small is small? This I compute by considering a system in
an environment or bath. I derive the bath’s (in this case the CMB) effect on
the hydrogen atom in the Feynman-Vernon approach to an open system. The effect
is small and quantified in terms of a correction to the hydrogen atom that
breaks time-reversal symmetry, as expected of memory effects. This is
significant, there are imperceptible changes in the state of the hydrogen
atom, which means that the pervasive CMB must have similar small effect on
other atoms thus breaking time-reversal symmetry in all physical systems.

###### pacs:

Valid PACS appear here

††preprint: APS/123-QED

1\. Quantum theory started from two outstanding problems in the early 20th
century - the blackbody radiation pattern and the spectral lines of the
hydrogen atom - see any introductory atomic physics book, for example Eisberg
. In this paper, I consider the effect of the cosmic microwave background on
the hydrogen atom, which we know from cosmology started at 380,000 years after
the Big Bang when the Universe has cooled to temperature of the order of
$10^{3}$ K and has expanded to the size of the order of $10^{7}$ light years,
reducing the energy of the electromagnetic radiation to the order of $10^{-1}$
ev Wikipedia1 , which is too little to kick out the electron from the clutches
of the proton (ionization energy is of the order of ev). At 13.8 B years of
age, the universe is bathed by a microwave background radiation of cosmic
origin, the electromagnetic radiation that became transparent to the universe
at 380,000 years age, now has a much lower energy of the order of $10^{-3}$
ev. The microwave background follows extremely well the blackbody radiation
pattern derived by Planck at the turn of the century with temperature equal to
2.73 K Wikipedia2 . About a decade later, Bohr explained the spectral lines of
hydrogen atom by considering an isolated proton and electron system by
assuming classical physics - planetary orbits, but proposing quantized angular
momentum states. Still another decade later, the hydrogen atom was treated
using the new quantum mechanics formulated by Heisenberg and Schroedinger,
spin was later added and the electron made relativistic by Dirac, showing a
richer structure with corresponding spectral lines, see for example Schiff .

But up to this time, the treatment of the hydrogen atom neglects the cosmic
microwave background. Although each individual photon does not have enough
energy to alter substantially the energy of the electron, however, there are
around $10^{2}$ per $cm^{3}$ of CMB photons in the universe today, every
electron and proton must then be hit by CMB photons every now and then because
the number of particles in the universe today is of the order of $10^{-7}$ per
$cm^{3}$, thus giving around $10^{9}$ CMB photons per hydrogen atom Wikipedia1
. This suggests a situation of a Brownian particle in a very weak and dilute
(typically water has $10^{22}$ molecules per $cm^{3}$ that can hit a pollen)
’fluid’.

The question now is how to compute the effect of the CMB on the hydrogen atom?
The Brownian particle analogy is not too useful. Fortunately, the quantum
formalism for dealing with any system in bath or environment, called an open
system, is established. The density matrix of such a system, after taking care
of the bath degrees of freedom is given by the Feynman/Vernon method Feynman
in non-relativistic quantum case. Thus, I will I only consider the non-
relativistic hydrogen atom. The fully relativistic quantum theory of electron
and photon is the realm of QED.

2\. The CMB is the remnant of the EM radiation that decoupled from matter at
380 K after the Big Bang. It is described then by a free EM field with
Lagrangian (SI units)

$S_{em}=-\frac{1}{4\mu_{0}}\int
d^{4}x(\partial_{\mu}A_{\nu}-\partial_{\nu}A_{\mu})(\partial^{\mu}A^{\nu}-\partial^{\nu}A^{\mu}),$
(1)

where $\mu_{0}$ is the magnetic permeability of free space. I use the metric
with signature (-+++). The use of the Coulomb gauge
$\vec{\partial}\cdot\vec{A}=0$ results in

$S_{em}=\frac{1}{2\mu_{0}}\int
d^{4}x\left[(\partial_{0}A_{i})^{2}-(\partial_{i}A_{j})^{2}\right],$ (2)

giving the field equation

$(\partial_{0}^{2}-\vec{\partial}\cdot\vec{\partial})\vec{A}=0.$ (3)

The solution to equation (3) is given by

$\displaystyle\vec{A}(\vec{x},t)=\sum_{\vec{k},\alpha}\left[q_{\vec{k}}(t)\vec{\varphi}_{\vec{k},\alpha}(\vec{x})+q_{\vec{k}}(t)\vec{\varphi}^{*}_{\vec{k},\alpha}(\vec{x})\right],$
(4a)
$\displaystyle\vec{\varphi}_{\vec{k}\alpha}(\vec{x})=(\frac{1}{2\pi})^{\frac{3}{2}}\vec{\epsilon}(\vec{k},\alpha)\exp{(i\vec{k}\cdot\vec{x})},$
(4b)

where $\vec{\epsilon}(\vec{k},\alpha)$ with $\alpha=1,2$ are the polarization
vectors of unit length and
$(\vec{\epsilon}(\vec{k},1),\vec{\epsilon}(\vec{k},2),\vec{k})$ form a right-
handed triad of vectors. Also, the $\vec{k}$ summation will go into continuum
integration when the CMB nature of the EM waves is accounted for later. For
equation (4) to be a solution to equation (3), the $q_{\vec{k}}(t)$ must
satisfy

$\dfrac{d^{2}q_{\vec{k}}}{dt^{2}}+(ck)^{2}q_{\vec{k}}=0,$ (5)

which means $q_{\vec{k}}$ is a harmonic oscillator of ’mass’
$\frac{1}{\mu_{0}c}$ and ’frequency’ ck. It is assumed in equation (4) that
the harmonic oscillator solutions are all real (sine and cosine solutions will
be used). Substituting equations(4;a,b) in equation (2) results in

$S_{em}=\int
dt\sum_{\vec{k}}\frac{1}{2\mu_{0}c}\left[\left(\dfrac{dq_{\vec{k}}}{dt}\right)^{2}-(ck)^{2}q_{\vec{k}}^{2}\right].$
(6)

In this form, the free EM wave is written as a sum of harmonic oscillators.

3\. A charge particle of charge q interacts with an electromagnetic wave as
given in the interaction Lagrangian term

$L_{int}=q\vec{v}\cdot\vec{A}(\vec{x},t),$ (7)

A hydrogen atom in a CMB has both proton and electron interacting with the CMB
EM wave. Since the proton has much smaller velocity than the electron in the
hydrogen atom (typically of the order of $10^{-4}$ slower), I will neglect the
CMB interaction with the proton and just consider the effect on the electron.
For simplicity, I will just put the proton at the origin. The full Lagrangian
being considered is now

$\displaystyle L=L_{hy}+L_{em}+L_{int},$ (8a) $\displaystyle
L_{hy}=\frac{1}{2}m(\dfrac{d\vec{x}}{dt})^{2}+\dfrac{e^{2}}{4\pi\epsilon_{0}}\frac{1}{x},$
(8b) $\displaystyle
L_{em}=\sum_{\vec{k}}\frac{1}{2\mu_{0}c}\left[(\dfrac{dq_{\vec{k}}}{dt})^{2}-(ck)^{2}q_{\vec{k}}^{2}\right],$
(8c) $\displaystyle
L_{int}=-e\dot{\vec{x}}\cdot\sum_{\vec{k},\alpha}\left[q_{\vec{k}}(t)\vec{\varphi}_{\vec{k}\alpha}(\vec{x})+q_{\vec{k}}(t)\vec{\varphi}^{*}_{\vec{k}\alpha}(\vec{x})\right],$
(8d)

where e is the electron charge. Using equation (4b), the interaction of the
electron with the EM waves can be written as

$L_{int}=-e(\frac{1}{(2\pi})^{\frac{3}{2}}\sum_{\vec{k}}\dot{\vec{x}}(t)\cdot\vec{\epsilon}\cos(\vec{k}\cdot\vec{x})q_{\vec{k}}(t),$
(9)

where $\vec{\epsilon}=[\vec{\epsilon}(\vec{k},1)+\vec{\epsilon}(\vec{k},2)]$.
Equations (8b) and (9) show the similarity with the point particle in a bath
of harmonic oscillator discussed in F/V and Caldeira-Leggett (C/L) Caldeira
with a crucial difference of the electron having a bit more complicated
interaction - it is velocity dependent and the coordinate is not factored out
of the ’frequency’ term as shown in the cosine. But still for every $\vec{k}$,
the calculations done by F/V and C/L are applicable.

4\. I give the results of the F/V and C/L calculations right up to the point
before summing over the different ’frequency’ contributions, which I will do
in 5. But before I go into that, an admission of error. In a previous paper
Magpantay , I argued the difference between my calculation’s equation (25) and
C/L’s equation (3.9) and (3.10). First, I missed a factor of $\frac{1}{2}$ in
the second line of equation (25) and second, I missed a simple evaluation that
would have shown
$(\coth\theta+\dfrac{1}{\sinh\theta})=\coth\frac{\theta}{2})$. Taking these
corrections, I find the C/L results and there is no basis for claiming any
difference with their result.

The C/L quantities, as given in formulas (3.9) to (3.11), translate to this
paper’s quantities via

$\displaystyle m\rightarrow\frac{1}{\mu_{0}c},$ (10a)
$\displaystyle\omega_{i}\rightarrow ck,$ (10b) $\displaystyle c_{i}\rightarrow
e(\frac{1}{(2\pi})^{\frac{3}{2}},$ (10c) $\displaystyle
x\rightarrow\dot{\vec{x}}(t)\cdot\vec{\epsilon}\cos(\vec{k}\cdot\vec{x}).$
(10d)

The propagator for the density matrix is given by

$\begin{split}J(\vec{x},\vec{y};\vec{x}^{\prime},\vec{y}^{\prime})&=\int_{endpoints}(d\tilde{\vec{x}})(d\tilde{\vec{y}})\exp\frac{i}{\hbar}\big{\\{}\int_{0}^{t}dt^{\prime}\left[L_{hy}(\tilde{\vec{x}})-L_{hy}(\tilde{\vec{y}})\right]\\\
&\quad+\int_{0}^{t}dt^{\prime}\int_{0}^{t^{\prime}}dt^{\prime\prime}\sum_{\vec{k}}\frac{\mu_{0}e^{2}}{16\pi^{3}}\frac{1}{k}\Big{\\{}\left[\dot{\tilde{\vec{x}}}(t^{\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{x}}(t^{\prime}))-\dot{\tilde{\vec{y}}}(t^{\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{y}}(t^{\prime})\right]\\\
&\quad Xi\coth(\frac{\beta\hbar
ck}{2})\cos(ck(t^{\prime}-t^{\prime\prime}))\left[\dot{\tilde{\vec{x}}}(t^{\prime\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{x}}(t^{\prime\prime}))-\dot{\tilde{\vec{y}}}(t^{\prime\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{y}}(t^{\prime\prime}))\right]\\\
&\quad+\Big{[}\dot{\tilde{\vec{x}}}(t^{\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{x}}(t^{\prime}))-\dot{\tilde{\vec{y}}}(t^{\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{y}}(t^{\prime})\Big{]}\sin(ck(t^{\prime}-t^{\prime\prime}))\\\
&\quad
X\Big{[}\dot{\tilde{\vec{x}}}(t^{\prime\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{x}}(t^{\prime\prime}))+\dot{\tilde{\vec{y}}}(t^{\prime\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{y}}(t^{\prime\prime}))\Big{\\}}\big{\\}}.\end{split}$
(11)

This propagator is more involved because when we sum over $\vec{k}$, the
electron coordinate $\vec{x}$ is entwined with it inside the cosine term and
the electron velocity is entwined with the sum of the two polarization
vectors. But the more important question is what weighs the contribution of
the different $\vec{k}$?

5\. Since the electromagnetic waves are supposed to represent the CMB, the sum
over $\vec{k}$ will naturally go to the integral

$\sum_{\vec{k}}\rightarrow\int k^{2}dkd\Omega(1)$ (12)

which means all ’frequencies’ have equal weight because of the weight (1),
i.e., the EM waves represent a ’white’ light (note, the $k^{2}$ and $d\Omega$
are geometrical factors in 3D). This is not representative of CMB. To ensure
that the EM waves in equation (11) is a CMB, it must be weighted on by the
blackbody radiation pattern given by Planck. The Planck distribution given by

$\rho_{T}(\lambda)d\lambda=\frac{8\pi
hc}{\lambda^{5}}\dfrac{1}{\exp{\frac{\beta hc}{\lambda}}-1}d\lambda,$ (13)

where $\beta=\frac{1}{k_{B}T}$, $T=2.7K$ and $k_{B}$ is Boltzmann’s constant.
The distribution accounts for the number of waves and the average energy per
wave between wavelengths $\lambda$ and $\lambda+d\lambda$. Since the
temperature is 2.7 K, I am not considering time starting from 380 K years
after the Big Bang (temperature then is of the order of $10^{3}$ K but only
when the Universe has cooled down to 2.7 K. Current ideas do not know when
this exactly happened but present ideas speculate that the Universe has cooled
down to 10 to 20 K after a billion years so maybe a few more billion years
will cool down the Universe to about 2.7 K.

In terms of the ’frequency’ k, which is actually the wave number
$k=\frac{2\pi}{\lambda}$, the Planck distribution is given by

$\rho_{T}(\lambda)d\lambda=-\rho_{T}(k)dk,$ (14)

giving

$\rho_{T}(k)=\frac{hc}{2\pi^{3}}k^{3}\dfrac{1}{\exp{\beta\hbar ck}-1}.$ (15)

To have a normalized distribution, so that the contribution of each
’frequency’ k is properly weighted, the distribution $\rho_{T}(k)$ must be
divided by the norm

$\begin{split}\rho_{T}&=\int_{0}^{\infty}dk\rho_{T}(k)\\\
&=(\dfrac{2\pi}{\beta\hbar c})^{4}\frac{1}{240}\end{split}$ (16)

where the integral is given in page 325 of Gradshteyn .

The sum over $\vec{k}$ in equation(11) should now be replaced by

$\begin{split}\sum_{\vec{k}}(eqn11term)&=\int
d\Omega\int_{0}^{\infty}dkOF\dfrac{1}{\exp{\beta\hbar
ck}-1}k^{4}\big{\\{}\Big{[}\dot{\tilde{\vec{x}}}(t^{\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{x}}(t^{\prime}))-\dot{\tilde{\vec{y}}}(t^{\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{y}}(t^{\prime})\Big{]}\\\
&\quad Xi\coth(\frac{\beta\hbar
ck}{2})\cos(ck(t^{\prime}-t^{\prime\prime}))\Big{[}\dot{\tilde{\vec{x}}}(t^{\prime\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{x}}(t^{\prime\prime}))-\dot{\tilde{\vec{y}}}(t^{\prime\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{y}}(t^{\prime\prime}))\Big{]}\\\
&\quad+\Big{[}\dot{\tilde{\vec{x}}}(t^{\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{x}}(t^{\prime}))-\dot{\tilde{\vec{y}}}(t^{\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{y}}(t^{\prime})\Big{]}\sin(ck(t^{\prime}-t^{\prime\prime}))\\\
&\quad
X\Big{[}\dot{\tilde{\vec{x}}}(t^{\prime\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{x}}(t^{\prime\prime}))+\dot{\tilde{\vec{y}}}(t^{\prime\prime})\cdot\vec{\epsilon}\cos(\vec{k}\cdot\tilde{\vec{y}}(t^{\prime\prime}))\big{\\}},\end{split}$
(17)

where the constant overall factor OF is given by

$\begin{split}OF&=e^{2}\mu_{0}hc(\beta\hbar c)^{4}\frac{240}{256\pi^{9}}\\\
&\propto 10^{-82}.\end{split}$ (18)

The SI values of the physical constants and $T=2.7$ K are used in above. Note,
the overall factor is fantastically small. It gives a sense that the CMB
memory correction to the action must really be small.

6\. The above integral over k is rather complicated and the focus of this
part. To make the integration doable, define

$u=ck.$ (19)

Using this in the $\coth(\frac{\beta\hbar ck}{2})$ term, the term can be
approximated by

$\coth(\frac{\beta\hbar ck}{2})\approx\frac{2}{\beta\hbar}(\frac{1}{u}).$ (20)

The reason for this is that substituting the values for the Boltzmann constant
$k_{B}$ and Planck constant $\hbar$, the RHS of (20) is $\propto
10^{11}\frac{1}{u}$ and the neglected terms are of the order of
$(10^{-11}u)^{n}$, where $n=1,2,...$.

Another term in equation (17) is
$\cos(\vec{k}\cdot\tilde{\vec{x}}(t^{\prime}))=\cos(\frac{1}{c}x(t^{\prime})u\cos\Theta_{x}(t^{\prime}))$,
where $\Theta_{x}(t^{\prime})$ is the angle between the electron’s position
vector at t’ and the CMB’s wave number vector $\vec{k}$, with an explicit time
dependence as shown later. Since $\frac{1}{c}\approx 10^{-8}$, then the
approximation of the cosine term can be used, i.e.,

$\cos(\frac{1}{c}x(t^{\prime})u\cos\Theta_{x}(t^{\prime}))\approx
1-\frac{1}{2}(\dfrac{{x}(t^{\prime})\cos\Theta_{x}(t^{\prime})u}{c})^{2}.$
(21)

Another angle that appears in equation (17) is in the term
$\dot{\tilde{\vec{x}}}(t^{\prime})\cdot\vec{\epsilon}=|\dot{\tilde{\vec{x}}}(t^{\prime})|\cos\Phi_{x}(t^{\prime})$,
the angle between the electron’s velocity vector at t’ and the CMB’s
polarization vector $\vec{\epsilon}$. This is also time dependent and the
explicit time dependence will also be shown later.

Using equations (19), (20) and (21) in (17) results in

$\begin{split}Equation(17)&=(10^{-122})\int
d\Omega\int_{0}^{\infty}du\dfrac{u^{4}}{\exp{\gamma
u}-1}\big{\\{}i\dfrac{2}{\beta\hbar}\dfrac{\cos
u(t^{\prime}-t^{\prime\prime})}{u}\\\ &\quad
X\left[|\dot{\tilde{\vec{x}}}(t^{\prime})|\cos\Phi_{x}(t^{\prime})\left(1-\frac{1}{2}(\dfrac{x(t^{\prime})\cos\Theta_{x}(t^{\prime})u}{c})^{2}\right)-|\dot{\tilde{\vec{y}}}(t^{\prime})|\cos\Phi_{y}(t^{\prime})\left(1-\frac{1}{2}(\dfrac{y(t^{\prime})\cos\Theta_{y}(t^{\prime})u}{c})^{2}\right)\right]\\\
&\quad
X\left[|\dot{\tilde{\vec{x}}}(t^{\prime\prime})|\cos\Phi_{x}(t^{\prime\prime})\left(1-\frac{1}{2}(\dfrac{x(t^{\prime\prime})\cos\Theta_{x}(t^{\prime\prime})u}{c})^{2}\right)-|\dot{\tilde{\vec{y}}}(t^{\prime\prime})|\cos\Phi_{y}(t^{\prime\prime})\left(1-\frac{1}{2}(\dfrac{y(t^{\prime\prime})\cos\Theta_{y}(t^{\prime\prime})u}{c})^{2}\right)\right]\\\
&\quad+\sin
u(t^{\prime}-t^{\prime\prime})\left[|\dot{\tilde{\vec{x}}}(t^{\prime})|\cos\Phi_{x}(t^{\prime})\left(1-\frac{1}{2}(\dfrac{x(t^{\prime})\cos\Theta_{x}(t^{\prime})u}{c})^{2}\right)-|\dot{\tilde{\vec{y}}}(t^{\prime})|\cos\Phi_{y}(t^{\prime})\left(1-\frac{1}{2}(\dfrac{y(t^{\prime})\cos\Theta_{y}(t^{\prime})u}{c})^{2}\right)\right]\\\
&\quad
X\left[|\dot{\tilde{\vec{x}}}(t^{\prime\prime})|\cos\Phi_{x}(t^{\prime\prime})\left(1-\frac{1}{2}(\dfrac{x(t^{\prime\prime})\cos\Theta_{x}(t^{\prime\prime})u}{c})^{2}\right)+|\dot{\tilde{\vec{y}}}(t^{\prime\prime})|\cos\Phi_{y}(t^{\prime\prime})\left(1-\frac{1}{2}(\dfrac{y(t^{\prime\prime})\cos\Theta_{y}(t^{\prime\prime})u}{c})^{2}\right)\right]\big{\\}},\end{split}$
(22)

where $\gamma=\dfrac{\hbar}{k_{B}T}$ and $T=2.7K$. Note the considerable
reduction of the OF by $\frac{1}{c^{5}}\propto 10^{-40}$.

From the Table of Integrals Gradshteyn

$\displaystyle\int_{0}^{\infty}\dfrac{\sin au}{\exp{\gamma
u}-1}du=\frac{\pi}{2\gamma}\coth(\frac{\pi
a}{\gamma})-\frac{1}{2a},[a>0,Re\gamma>0]$ (23a)
$\displaystyle\int_{0}^{\infty}\dfrac{u\cos bu}{\exp{\gamma
u}-1}du=\frac{1}{2b^{2}}-\frac{\pi^{2}}{2\gamma^{2}}(\sinh(\frac{b\pi}{\gamma}))^{-2},[Re\gamma>0],$
(23b)

where $a=b=(t^{\prime}-t^{\prime\prime})$. The u integrals in equation (22)
are arrived at by differentiating equations (23) w.r.t. a or b a number of
times to arrive at the power of u in each integral. The result of each
integration is also simplified by taking note that
$\gamma=\frac{\hbar}{k_{B}T}\approx 10^{-11}$. With these in mind, I list down
the three least small among all the small terms. They are, in decreasing
strength, and following equation (11), to include the time integrals:

$\displaystyle\begin{split}First&\propto(10^{-111})\int_{0}^{t}dt^{\prime}\int_{0}^{t^{\prime}}dt^{\prime\prime}\int
d\Omega\left\\{\dfrac{3}{(t^{\prime}-t^{\prime\prime})^{4}}-\dfrac{8\pi^{2}}{\gamma^{4}}\exp{(\dfrac{-2\pi(t^{\prime}-t^{\prime\prime})}{\gamma})}\right\\}\\\
&\quad
X\left[|\dot{\tilde{\vec{x}}}(t^{\prime})|\cos\Phi_{x}(t^{\prime})-|\dot{\tilde{\vec{y}}}(t^{\prime})|\cos\Phi_{y}(t^{\prime})\right]\left[|\dot{\tilde{\vec{x}}}(t^{\prime\prime})|\cos\Phi_{x}(t^{\prime\prime})-|\dot{\tilde{\vec{y}}}(t^{\prime\prime})|\cos\Phi_{y}(t^{\prime\prime})\right],\\\
Second&\propto(10^{-122})\int_{0}^{t}dt^{\prime}\int_{0}^{t^{\prime}}dt^{\prime\prime}\int
d\Omega\left\\{\dfrac{-12}{(t^{\prime}-t^{\prime\prime})^{5}}+4(\frac{\pi}{\gamma})^{5}\exp{(\dfrac{-2\pi(t^{\prime}-t^{\prime\prime})}{\gamma})}\right\\}\\\
&\quad
X\left[|\dot{\tilde{\vec{x}}}(t^{\prime})|\cos\Phi_{x}(t^{\prime})-|\dot{\tilde{\vec{y}}}(t^{\prime})|\cos\Phi_{y}(t^{\prime})\right]\left[|\dot{\tilde{\vec{x}}}(t^{\prime\prime})|\cos\Phi_{x}(t^{\prime\prime})+|\dot{\tilde{\vec{y}}}(t^{\prime\prime})|\cos\Phi_{y}(t^{\prime\prime})\right],\\\
Third&\propto(10^{-127})\int_{0}^{t}dt^{\prime}\int_{0}^{t^{\prime}}dt^{\prime\prime}\int
d\Omega\left\\{\dfrac{60}{(t^{\prime}-t^{\prime\prime})^{6}}-\dfrac{32\pi^{5}}{\gamma^{6}}\exp{(\dfrac{-2\pi(t^{\prime}-t^{\prime\prime})}{\gamma})}\right\\}\\\
&\quad\Big{\\{}\left[|\dot{\tilde{\vec{x}}}(t^{\prime})|\cos\Phi_{x}(t^{\prime})-|\dot{\tilde{\vec{y}}}(t^{\prime})|\cos\Phi_{y}(t^{\prime})\right]\left[|\dot{\tilde{\vec{x}}}(t^{\prime\prime})|\cos\Phi_{x}(t^{\prime\prime})x^{2}(t^{\prime\prime})\cos^{2}\Theta_{x}(t^{\prime\prime})-|\dot{\tilde{\vec{y}}}(t^{\prime\prime})|\cos\Phi_{y}(t^{\prime\prime})y^{2}(t^{\prime\prime})\cos^{2}\Theta_{y}(t^{\prime\prime})\right]\\\
&\quad+\left[|\dot{\tilde{\vec{x}}}(t^{\prime})|\cos\Phi_{x}(t^{\prime})x^{2}(t^{\prime})\cos^{2}\Theta_{x}(t^{\prime})-|\dot{\tilde{\vec{y}}}(t^{\prime})|\cos\Phi_{y}(t^{\prime})y^{2}(t^{\prime})\cos^{2}\Theta_{y}(t^{\prime})\right]\left[|\dot{\tilde{\vec{x}}}(t^{\prime\prime})|\cos\Phi_{x}(t^{\prime\prime})-|\dot{\tilde{\vec{y}}}(t^{\prime\prime})|\cos\Phi_{y}(t^{\prime\prime})\right]\Big{\\}}.\end{split}$
(24a)

In the First, the OF is multiplied by $\frac{1}{\beta\hbar}\propto 10^{11}$
giving a factor $\propto 10^{-111}$. In the Second, there is no modifier to
OF. In the Third, the OF is multiplied by $\frac{1}{\beta\hbar}\propto
10^{11}$ then by $\frac{1}{c^{2}}$, which comes from equation (21). resulting
in the factor $10^{-127}$.

7\. Equation (24) gives the correction to the hydrogen atom action terms in
the propagator for the density matrix as shown in equation (11). Evaluating
the path integrals in (11) with the time delayed and non-quadratic terms in
(24) is clearly not possible. Still, the angular integrations can be done and
should be finite (see example in the next paragraphs) even for an arbitrary
$\tilde{\vec{x}}$ and $\tilde{\vec{y}}$ and time derivatives. Equation (24)
then gives memory terms with extremely small factors showing that the time-
reversal symmetry violating terms are extremely small.

Separating out the x from y contributions to have a clearly
$L_{eff}(\vec{x},\dot{\vec{x}})$ does not seem to be possible also. However,
the structure of the terms of equation (24) shows the separate x, y and cross
terms have about the same contribution. This will be shown in this section by
considering the effect of the CMB on the Bohr model. This will also make
concrete how small the effect of the CMB is on the energy levels of the atom.

The next problem is to integrate the angles of the CMB EM waves to complete
the sum over CMB waves contribution. This is done by setting up the hydrogen
atom geometry, as treated by Bohr, and superimposing the CMB vector $\vec{k}$.
It will be clear later in the calculations that I will only provide an
estimate of the CMB correction to the Bohr energy levels, which is the same as
the Schroedinger treatment without spin and relativistic corrections.

Figure 1: The electron’s circular orbit on (x,y) plane superimposed with the
CMB propagation vector $\vec{k}$, which is along $z^{\prime\prime}$ axis and
the polarization vectors along the $x^{\prime\prime}$ and $y^{\prime\prime}$
axes.

The electron is doing a circular motion with the Coulomb attraction from the
proton providing the centripetal acceleration. The Bohr condition of the
quantization of the angular momentum is then added. The relevant equations are

$\displaystyle\frac{1}{4\pi\epsilon_{0}}\frac{e^{2}}{r^{2}}=m\frac{v^{2}}{r},$
(25a) $\displaystyle\begin{split}L&=mvr\\\ &=n\hbar,\end{split}$ (25b)

where n is an integer. These equations give the quantized values

$\displaystyle r=\dfrac{4\pi\epsilon_{0}\hbar^{2}}{me^{2}}n^{2},$ (26a)
$\displaystyle v=\dfrac{e^{2}}{4\pi\epsilon_{0}\hbar}\frac{1}{n},$ (26b)
$\displaystyle\omega=\dfrac{me^{4}}{(4\pi\epsilon_{0})^{2}\hbar^{3}}\frac{1}{n^{3}},$
(26c)
$\displaystyle\begin{split}E&=-\dfrac{me^{4})}{2(4\pi\epsilon_{0})^{2}\hbar^{2}}\frac{1}{n^{2}}\\\
&=-\frac{13.6}{n^{2}}ev,\end{split}$ (26d)

where $\omega$ is the angular velocity and E is the energy of the hydrogen
atom.

The position and velocity of the electron, assuming it starts at $\phi=0$ are
given by

$\displaystyle\vec{x}(t^{\prime})=r\cos\omega t^{\prime}\textbf{i}+r\sin\omega
t^{\prime}\textbf{j},$ (27a)
$\displaystyle\dot{\vec{x}}(t^{\prime})=-r\omega\sin\omega
t^{\prime}\textbf{i}+r\omega\cos\omega t^{\prime}\textbf{j}.$ (27b)

If the y terms are also considered in equation (24), the end point conditions
in equation (11) shows different $\tilde{\vec{y}}$ end points (
$(\vec{y^{\prime}},\vec{y})$ ). In the context of the Bohr atom, it means the
y trajectory can have a different quantum number n, say n’.

To specify the CMB arbitrary EM wave vector $\vec{k}$, start with the unit
vectors $(\textbf{i},\textbf{j},\textbf{k})$. Then rotate about k by angle
$\varphi$ to get the unit vectors
$(\textbf{i'},\textbf{j'},\textbf{k'}=\textbf{k})$, then follow up with a
counter clockwise rotation by angle $\theta$ about the i’ axis to get the unit
vectors $(\textbf{i''}=\textbf{i'},\textbf{j''},\textbf{k''})$. Now I can
identify these RHT of vectors with the RHT of vectors of the CMB EM wave given
by $(\vec{\epsilon}(\vec{k},1),\vec{\epsilon}(\vec{k},2),\vec{k})$. For
completeness, the vector $\vec{k}$ makes an angle $\theta$ with the k unit
vector and angle $\varphi+\frac{3\pi}{2}$ with the i unit vector (if I do the
second rotation clockwise, the vector $\vec{k}$ makes an angle
$\varphi+\frac{1\pi}{2}$ with the i unit vector . This operations will
identify the terms in equation (24) as

$\displaystyle\tilde{x}(t^{\prime})=r,$ (28a)
$\displaystyle|\dot{\tilde{\vec{x}}}(t^{\prime})|=r\omega,$ (28b)
$\displaystyle\cos\Phi_{x}(t^{\prime})=\cos\omega
t^{\prime}(\sin\varphi+\cos\varphi\cos\theta)-\sin\omega
t^{\prime}(\cos\varphi-\sin\varphi\cos\theta),$ (28c)
$\displaystyle\cos\Theta_{x}(t^{\prime})=\cos\omega
t^{\prime}\sin\varphi\sin\theta-\sin\omega t^{\prime}\cos\varphi\sin\varphi.$
(28d)

Now, the solid angle averaging can be done and all are finite integrals. The
interesting part is the term

$\begin{split}|\dot{\tilde{\vec{x}}}(t^{\prime})||\dot{\tilde{\vec{x}}}(t^{\prime\prime})|&=v^{2},\\\
&=-\frac{2}{m}E_{n},\\\ &\approx 10^{+31}(\frac{13.6}{n^{2}})ev,\end{split}$
(29)

This will strengthen the really small terms given by equation (24) by 31
orders of magnitude but still resulting in very weak contributions. The change
in action $\int_{0}^{t}dt^{\prime}L_{hy}(t^{\prime})$ from the different
contributions are now

$\displaystyle First\propto
10^{-80}\left[\int_{0}^{t}dt^{\prime}\left(\int_{0}^{t^{\prime}}d\tau\left\\{\dfrac{3}{\tau^{4}}-\dfrac{8\pi^{2}}{\gamma^{4}}\exp{(\dfrac{-2\pi\tau}{\gamma})}\right\\}\cos\omega\tau\right)\right]E_{n},$
(30a) $\displaystyle Second\propto
10^{-91}\left[\int_{0}^{t}dt^{\prime}\left(\int_{0}^{t^{\prime}}d\tau\left\\{\dfrac{-12}{\tau^{5}}+4(\frac{\pi}{\gamma})^{5}\exp{(\dfrac{-2\pi\tau}{\gamma})}\right\\}\cos\omega\tau\right)\right]E_{n},$
(30b) $\displaystyle Third\propto
10^{-116}\left[\int_{0}^{t}dt^{\prime}\left(\int_{0}^{t^{\prime}}d\tau\left\\{\dfrac{60}{\tau^{6}}-\dfrac{32\pi^{5}}{\gamma^{6}}\exp{(\dfrac{-2\pi\tau}{\gamma})}\right\\}\cos\omega\tau\right)\right]E_{n},$
(30c)

where a change of variables, from $t^{\prime\prime}$ to
$\tau=t^{\prime}-t^{\prime\prime}$ was made in the integration. The much
smaller Third term comes from the fact that this term goes with $r^{2}$, which
by equation (26a), is $\propto 10^{-20}$.

All the contributions are small, modulo the time factor given by the two time
integrals, which shows memory effects. These time integrals are rapidly
oscillating integrals because $\omega\propto 10^{+14}$ by equation (26c) and
modified by the damping polynomial term in the denominator and also by an
exponential decaying term. The exponential damping terms are straightforward
to evaluate. The problematic terms are the polynomial terms in the
denominator. Still it is expected that the $\tau$ integrations would lead to a
finite integral even though the closed form in terms of $t^{\prime}$ is not
known. Then the $t^{\prime}$ integration expression should give an overall
effect of the CMB on the hydrogen atom, which must be small because of the
very small factors that go with each term in equation (30).

8\. This paper showed that the effect of the CMB on the hydrogen atom is
almost negligible, the energy levels are hardly changed, the memory effects
are imperceptible but break time-reversal symmetry. Thus, this paper showed
that since everything is made up of atoms, as a matter of principle, time-
reversal symmetry is no longer a symmetry of atomic physics because of the CMB
effect on these atoms.

If the pervasive CMB breaks time-reversal symmetry, surely the other long-
range force in the universe, gravity, can also break time-reversal symmetry.
The gravitational wave generated during the Big Bang, although much weaker
than the CMB effect now, must also produce time-reversal symmetry violation.
In this sense, time-reversal symmetry is not really present in principle but
only ’observed’ because we are not sensitive to the violations.

###### Acknowledgements.

I am grateful to Felicia Magpantay for putting my Latex file right. I would
like to thank Maribel Garcia for doing the figure. A special thanks to Gravity
for the constant company during this pandemic while I write this paper.

## References

  * (1) Eisberg, Robert M., Fundamentals of Modern Physics, John Wiley and Sons, Inc., 1961.
  * (2) Wikipedia, Chronology of the Universe, last edited 09 January 2021.
  * (3) Wikipedia, Cosmic Microwave Background, last edited 20 January 2021.
  * (4) Schiff, Leonard I., Quantum Mechanics, McGraw-Hill, Inc., 1968.
  * (5) Feynman, R. P. and Vernon, F., The Theory of a General Quantum System Interacting with a Linear Dissipative System, Annals of Physics, 24, 118, 1963.
  * (6) Caldeira, A. and Leggett, A. J., Path Integral Approach To Quantum Brownian Motion, Physica, 121A, 587, 1983.
  * (7) Magpantay, Jose A., Lindbald Plus From Feynman-Vernon, arxiv:200300194.
  * (8) Gradshteyn, I. S. and Ryzhik, I. M., Table of Integrals, Series and Products, Academic Press, Inc., 1965.