#  $CP$ violation due to a Majorana phase in two flavor neutrino oscillations
with decays

Khushboo Dixit<EMAIL_ADDRESS>Department of Physics, Indian Institute of
Technology Bombay, Mumbai 400076, India  Akhila Kumar Pradhan
<EMAIL_ADDRESS>Department of Physics, Indian Institute of Technology
Bombay, Mumbai 400076, India  S. Uma Sankar<EMAIL_ADDRESS>Department of
Physics, Indian Institute of Technology Bombay, Mumbai 400076, India

###### Abstract

We study the conditions under which the Majorana phase of the two flavor
neutrino mixing matrix appears in the oscillation probabilities and causes
$CP$ violation. We find that the Majorana phase remains in the neutrino
evolution equation if the neutrino decay eigenstates are not aligned with the
mass eigenstates. We show that, in general, two kinds of $CP$ violation are
possible: one due to the Majorana phase and the other due to the phase of the
off-diagonal element of the neutrino decay matrix. We find that the $CP$
violating terms in the oscillation probabilities are also sensitive to
neutrino mass ordering.

##  I Introduction

Neutrinos are the most intriguing particles in nature. They are the only known
elementary neutral fermions. Even their fundamental nature, whether they are
Dirac or Majorana fermions, is an open question. In the Standard Model (SM),
neutrinos are massless. However, the discovery of neutrino oscillations showed
that different neutrino flavors mix to form mass eigenstates and these states
have tiny, nondegenerate masses, which are more than a million times smaller
than the electron mass. The mechanism that gives rise to such tiny masses is
also an open problem in particle physics.

Since neutrinos are neutral, it is possible for them to be their own
antiparticle, i.e., they can be Majorana fermions. The mass term for Majorana
neutrinos has a very different form compared to that of Dirac neutrinos and
can be naturally made small via seesaw mechanism. Majorana masses violate
lepton number by two units and lead to interesting signals such as
neutrinoless double beta decay or same sign lepton pairs at colliders. If the
neutrinos have Majorana masses, the mixing matrix connecting the flavor
eigenstates to mass eigensates has extra phases (called Majorana phases).

Neutrino oscillation probabilities depend on the elements of the mixing matrix
and the mass-squared differences, in general. It is well established that
vacuum oscillation probabilities do not depend on the Majorana phases
Bilenky:1980cx ; Schechter:1980gr ; Doi:1980yb ; Giunti:2010ec . Matter
effects arise due to neutrino propagation in dense matter and they modify the
neutrino evolution and hence, neutrino oscillation probabilities. The matter
effects can be due to pure SM interactions or they can include nonstandard
interactions also. In both cases, it can be shown that the Majorana phases do
not appear in the oscillation probabilities.

There are, however, some neutrino evolution equations for which the Majorana
phases appear in neutrino oscillation probabilities. A new form of neutrino
decoherence, with an off-diagonal term in the decoherence matrix was
considered in Ref. Benatti:2001fa . It was shown that the neutrino oscillation
probabilities depend on Majorana phases in such a case. It was also shown that
these probabilities are $CP$ violating Capolupo:2018hrp . This leads us to the
question of what other possibilities are there under which the Majorana phases
appear in neutrino oscillation probabilities and lead to $CP$ violation.

In this paper, we address the above question for the case of two flavor
oscillations. Extension of the discussion to three flavor oscillations is
straightforward. We consider the most general neutrino evolution Hamiltonian
including decay terms. Then we identify the terms in this Hamiltonian which
lead to the appearance of the Majorana phase in oscillation probabilities and
discuss their $CP$ and $CPT$ properties.

##  II Vacuum neutrino oscillations

In this section, we briefly discuss the dynamics of two flavor neutrino
oscillations in vacuum. In general, neutrino mass eigenstates $\nu_{i}$ mix
via a unitary matrix to introduce flavor states $\nu_{\alpha}$ of neutrinos

$\mathbf{\nu}_{\alpha}=U~{}\mathbf{\nu}_{i}=O~{}U_{ph}~{}\mathbf{\nu}_{i},$

where $\nu_{\alpha}=(\nu_{e}~{}~{}~{}\nu_{\mu})^{T}$ and
$\nu_{i}=(\nu_{1}~{}~{}~{}\nu_{2})^{T}$, and

$O=\begin{pmatrix}\cos\theta&\sin\theta\\\
-\sin\theta&\cos\theta\end{pmatrix}~{}~{}~{}~{}~{}~{}U_{ph}=\begin{pmatrix}1&0\\\
0&e^{i\phi}\end{pmatrix}.$ (1)

The mixing matrix $U$ is parameterized in terms of the mixing angle $\theta$
and the Majorana phase $\phi$. We assume that two other phases are pulled out
on the left and are absorbed in the flavor states. In the case of Dirac
neutrinos, the phase $\phi$ gets absorbed in the neutrino mass eigenstates
through rephasing and we are left with the orthogonal mixing matrix. Such
rephasing cannot be done for Majorana neutrinos.

Let us now consider the traditional diagonal Hamiltonian in mass basis that
governs the time evolution of neutrino mass eigenstates,

$\displaystyle\mathcal{H}$ $\displaystyle=\begin{pmatrix}a_{1}&0\\\
0&a_{2}\end{pmatrix}$
$\displaystyle=\frac{(a_{1}+a_{2})}{2}\begin{pmatrix}1&0\\\
0&1\end{pmatrix}+\frac{(a_{2}-a_{1})}{2}\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},$

where $a_{1}=m_{1}^{2}/2E$ and $a_{2}=m_{2}^{2}/2E$ where $m_{i}$ are the mass
eigenvalues and $E$ is the energy of the neutrinos. Evolution equations in
mass eigenbasis are

$i\frac{d}{dt}\nu_{i}(t)=\left[\frac{(a_{1}+a_{2})}{2}\sigma_{0}-\frac{(a_{2}-a_{1})}{2}\sigma_{z}\right]\nu_{i}(t),$
(2)

where $\sigma_{0}$ is the $2\times 2$ identity matrix and $\sigma_{z}$ is the
diagonal Pauli matrix. In the flavor basis, Eq. (2) has the form

$i\frac{d}{dt}\nu_{\alpha}(t)=\left[\frac{(a_{1}+a_{2})}{2}\sigma_{0}-\frac{(a_{2}-a_{1})}{2}OU_{ph}\sigma_{z}U_{ph}^{\dagger}O^{T}\right]\nu_{\alpha}(t).$
(3)

The term proportional to $\sigma_{0}$ makes no distinction between the flavors
and hence is absent from the probabilities. Since, $U_{ph}$ and $\sigma_{z}$
are diagonal matrices, they commute; the $\sigma_{z}$ term simplifies to
$O\sigma_{z}O^{T}$ and the phase $\phi$ disappears from the evolution
equation. The vacuum neutrino flavor transition probability is obtained as

$P({\nu_{e}\rightarrow\nu_{\mu}})=\sin^{2}2\theta\sin^{2}\bigg{(}\frac{(a_{2}-a_{1})t}{2}\bigg{)}\equiv
P_{e\mu}^{\rm vac}.$ (4)

The relations between different probabilities, $P_{ee}^{\rm
vac}=1-P_{e\mu}^{\rm vac}=P_{\mu\mu}^{\rm vac}$ and $P_{\mu e}^{\rm
vac}=P_{e\mu}^{\rm vac}$, follow trivially. Expressing $a_{i}$ as
$m_{i}^{2}/2E$ we get the standard expression for transition probability,

$P_{e\mu}^{\rm vac}=\sin^{2}2\theta\sin^{2}\bigg{(}\frac{\Delta
m^{2}L}{4E}\bigg{)},$ (5)

where $L$ is the distance traveled by neutrinos.

##  III Oscillations with general decay Hamiltonian

We now consider a neutrino Hamiltonian including decay terms. Such a
Hamiltonian has the form

$\mathcal{H}=M-i\Gamma/2,$ (6)

where $M$ is the mass matrix and $\Gamma$ is the decay matrix. For a general
two particle system these matrices are diagonal, i.e., the mass eigenstates
are also the decay eigenstates. However, for a system of two particles that
can oscillate into each other, these matrices can have off-diagonal terms, as
in the case of neutral meson system PKKabir ; BigiSanda ; Branco:1999fs . In
this work, we choose the matrices $M$ and $\Gamma/2$ to be of the form
Berryman:2014yoa ; Chattopadhyay:2021eba

$\mathcal{M}=\begin{pmatrix}a_{1}&0\\\
0&a_{2}\end{pmatrix},~{}~{}~{}\Gamma/2=\begin{pmatrix}b_{1}&\frac{1}{2}\eta
e^{i\xi}\\\ \frac{1}{2}\eta e^{-i\xi}&b_{2}\end{pmatrix}.$ (7)

In Eq. (7), the parameters $a_{1},a_{2},b_{1},b_{2},\eta$ and $\xi$ are real,
with $a_{2}-a_{1}=\Delta m^{2}/2E$ depicting the frequency of neutrino
oscillations and the rest causing decay of neutrino mass eigenstates. The
matrix $\Gamma$ needs to be positive semidefinite, i.e., non-negative, which
leads to the following constraints: $b_{1},b_{2}\geq 0$, and $\eta^{2}\leq
4b_{1}b_{2}$. A beam of oscillating neutrinos has a spread of energies. For
oscillating neutrinos, this spread is large enough such that both $\nu_{1}$
and $\nu_{2}$ are on mass shell. Because of this spread, both $\nu_{1}$ and
$\nu_{2}$ can decay into the same set of final states with the same energies.
This possibility gives rise to the off-diagonal term $\Gamma_{12}$.

Neutrino evolution through this Hamiltonian describes both neutrino
oscillation as well as neutrino decay. If $\Gamma$ is diagonal ($\eta=0$),
i.e., the decay eigenbasis is the same as the mass eigenbasis. In such a case
it is straightforward to show that the Majorana phase $\phi$ disappears from
neutrino evolution equations through a discussion similar to that of the
previous section. We now consider the case where $\Gamma$ is non-diagonal
($\eta\neq 0$), i.e., the mass eigenstates are not decay eigenstates. In this
case, the evolution equation in the mass eigenbasis takes the form

$\displaystyle i\frac{d}{dt}\nu_{i}(t)$ $\displaystyle=$
$\displaystyle\left[\frac{(a_{1}+a_{2})}{2}\sigma_{0}-\frac{(a_{2}-a_{1})}{2}\sigma_{z}\right.$
(8)
$\displaystyle-\left.\frac{i}{2}\left((b_{1}+b_{2})\sigma_{0}+\vec{\sigma}.\vec{\Gamma}\right)\right]~{}\nu_{i}(t),$

where $\vec{\Gamma}=[\eta\cos\xi,-\eta\sin\xi,-(b_{2}-b_{1})]$. Transforming
this equation to flavor basis we get

$\displaystyle i\frac{d}{dt}\nu_{\alpha}(t)$ $\displaystyle=$
$\displaystyle\left[\frac{(a_{1}+a_{2})}{2}\sigma_{0}-\frac{(a_{2}-a_{1})}{2}O\sigma_{z}O^{T}\right.$
$\displaystyle-\left.\frac{i}{2}(b_{1}+b_{2})\sigma_{0}-\frac{i}{2}OU_{ph}(\vec{\sigma}.\vec{\Gamma})U_{ph}^{\dagger}O^{T}\right]\nu_{\alpha}(t).$

Since $\sigma_{x}$ and $\sigma_{y}$ do not commute with $U_{ph}$ matrix, the
phase $\phi$ remains in the evolution equation.

The time evolution operator for neutrinos in the mass eigenbasis is
$\mathcal{U}=e^{-i\mathcal{H}t}$. This matrix can be expanded in the basis
spanned by $\sigma_{0}$ and Pauli matrices Nielsen . This expansion is
parameterized by a complex four-vector $n_{\mu}\equiv(n_{0},\vec{n})$, whose
components are given by $n_{\mu}=Tr[(-i\mathcal{H}t).\sigma_{\mu}]/2$.
Explicitly, they are expressed in terms of the parameters of $\mathcal{H}$ as

$\displaystyle n_{0}=-\frac{i}{2}(a_{1}+a_{2})t-\frac{1}{2}(b_{1}+b_{2})t,$
$\displaystyle n_{x}=-\frac{1}{2}(\eta\cos\xi)t,$ $\displaystyle
n_{y}=\frac{1}{2}(\eta\sin\xi)t,$ $\displaystyle
n_{z}=\frac{i}{2}(a_{2}-a_{1})t+\frac{1}{2}(b_{2}-b_{1})t.$ (10)

In terms of these components the evolution matrix $\mathcal{U}$ is

$\mathcal{U}=e^{n_{0}}\bigg{[}\cosh
n~{}\sigma_{0}+\frac{\vec{n}.\vec{\sigma}}{n}\sinh n\bigg{]},$ (11)

where

$n=\sqrt{n_{x}^{2}+n_{y}^{2}+n_{z}^{2}}=\frac{t}{2}\sqrt{\eta^{2}-(a_{2}-a_{1}-i(b_{2}-b_{1}))^{2}}.$
(12)

The evolution matrix in flavor basis can be obtained through the
transformation $\mathcal{U}_{f}=U\mathcal{U}U^{-1}$, where $U$ is defined in
Eq. (1). Oscillation probabilities can be obtained as

$P_{\alpha\beta}=|(\mathcal{U}_{f})_{\alpha\beta}|^{2}.$

The general probability expressions with all the decay parameters nonzero are
quite complicated. In this article, we are interested in how the probabilities
depend on the Majorana phase $\phi$. To illustrate this, we consider the
probability expressions in the limit $b_{1}=b=b_{2}$ and
$\eta\ll|a_{2}-a_{1}|$. For convenience we define

$\displaystyle\mathcal{A}$ $\displaystyle=$
$\displaystyle\frac{\sin(2\theta)\sin\left[\left(a_{2}-a_{1}\right)t\right]}{(a_{2}-a_{1})},~{}~{}$
$\displaystyle\mathcal{B}$ $\displaystyle=$
$\displaystyle\frac{\sin(2\theta)\sin^{2}\left[\frac{1}{2}t(a_{2}-a_{1})\right]}{(a_{2}-a_{1})}.$
(13)

Neglecting terms of $\mathcal{O}(\eta^{2})$ and higher order, we get the
survival probabilities as

$\displaystyle P_{ee}=e^{-2bt}\left(P_{ee}^{\rm
vac}-\eta\cos(\xi-\phi)\mathcal{A}\right)~{}~{}{\rm and}$ $\displaystyle
P_{\mu\mu}=e^{-2bt}\left(P_{\mu\mu}^{\rm
vac}+\eta\cos(\xi-\phi)\mathcal{A}\right)~{}~{}~{}~{}~{}~{}$ (14)

and the oscillation probabilities as

$\displaystyle P_{e\mu}=e^{-2bt}\left(P_{e\mu}^{\rm
vac}+2\eta\sin(\xi-\phi)\mathcal{B}\right)~{}~{}{\rm and}$ $\displaystyle
P_{\mu e}=e^{-2bt}\left(P_{\mu e}^{\rm
vac}-2\eta\sin(\xi-\phi)\mathcal{B}\right).~{}~{}~{}~{}~{}~{}$ (15)

Hence, we see that the Majorana phase $\phi$ appears in the probability
expressions if the neutrino evolution equation contains the off-diagonal term
of the decay matrix $\Gamma_{12}\propto\eta$. The presence of this term also
violates the equalities $P_{\mu\mu}=P_{ee}$ and $P_{\mu e}=P_{e\mu}$ that we
see in the case of two flavor vacuum oscillations. In addition, we note that
the terms with $\mathcal{B}$, present in oscillation probabilities, have
opposite signs for the two cases $a_{2}>a_{1}$ ($m_{2}>m_{1}$) and
$a_{2}<a_{1}$ ($m_{2}<m_{1}$); that is, the oscillation probability is
sensitive to the mass hierarchy.

We now consider the oscillations of antineutrinos. We assume $CPT$
conservation which implies the following relations for the mass and decay
matrices Berryman:2014yoa

$\bar{M}=M~{}~{}{\rm and}~{}~{}\bar{\Gamma}=\Gamma^{\ast}.$ (16)

Hence, antineutrino probabilities expressions can be obtained by making the
substitutions $\phi\rightarrow-\phi$ and $\xi\rightarrow-\xi$ in the neutrino
probability expressions. Explicitly these expressions are

$\displaystyle P_{\bar{e}\bar{e}}=e^{-2bt}\left(P_{\bar{e}\bar{e}}^{\rm
vac}-\eta\cos(\xi-\phi)\mathcal{A}\right)$ $\displaystyle
P_{\bar{\mu}\bar{\mu}}=e^{-2bt}\left(P_{\bar{\mu}\bar{\mu}}^{\rm
vac}+\eta\cos(\xi-\phi)\mathcal{A}\right)$ (17)

and

$\displaystyle P_{\bar{e}\bar{\mu}}=e^{-2bt}\left(P_{\bar{e}\bar{\mu}}^{\rm
vac}-2\eta\sin(\xi-\phi)\mathcal{B}\right)$ $\displaystyle
P_{\bar{\mu}\bar{e}}=e^{-2bt}\left(P_{\bar{\mu}\bar{e}}^{\rm
vac}+2\eta\sin(\xi-\phi)\mathcal{B}\right).$ (18)

Since we assumed $CPT$ invariance we find $P_{\bar{e}\bar{e}}=P_{ee}$,
$P_{\bar{\mu}\bar{\mu}}=P_{\mu\mu}$ and $P_{\bar{\mu}\bar{e}}=P_{e\mu}$.
However, there is $CP$ violation ($P_{\bar{e}\bar{\mu}}\neq P_{e\mu}$) and $T$
violation ($P_{\mu e}\neq P_{e\mu}$).

##  IV Results and Discussions

In this section, we discuss our results. In transforming the evolution
equation from the mass eigenbasis to the flavor eigenbasis, we get the matrix
product $U_{ph}\mathcal{H}U_{ph}^{\dagger}$, where $\mathcal{H}$ is the
Hamiltonian in mass eigenbasis. The diagonal phase matrix $U_{ph}$ commutes
with $\mathcal{H}$ whenever $\mathcal{H}$ is diagonal. This is true for the
usual neutrino oscillations and for the case where the mass eigenstates are
also the decay eigenstates. In such situations, $U_{ph}$ matrix drops out from
the neutrino evolution equation, which in turn leads to the oscillation
probabilities being independent of the Majorana phase $\phi$. However, when
the decay eigenstates are not aligned with the mass eigenstates, there is an
off-diagonal term in the decay matrix $\Gamma$ of $\mathcal{H}$. Since
$\mathcal{H}$ is no longer diagonal, it does not commute with $U_{ph}$, thus
leading to the presence of $\phi$ in the evolution equation and in the
probabilities. In addition to the off-diagonal dissipator discussed in
Benatti:2001fa , the off-diagonal decay matrix is another possible source for
the appearance of Majorana phases in the oscillation probabilities and the
corresponding $CP$ violation.

The off-diagonal term of the decay matrix $\Gamma$ has the general form $\eta
e^{i\xi}$ and the $CP$ violating term in the oscillation probabilities is
proportional to $\eta\sin(\xi-\phi)$. We distinguish different forms of $CP$
violation based on the values of the phases $\xi$ and $\phi$.

  * •

For $\eta\neq 0$ and $\xi=0$ the decay matrix $\Gamma$ is real and is $CP$
conserving. In such a case, we need $\phi\neq 0$ for $CP$ violation. We call
this $CP$ violation in mass because $\phi$ arises due to the diagonalization
of the complex mass matrix.

  * •

If $\phi=0$, it is still possible to have $CP$ violation if $\eta\neq 0$ and
$\xi\neq 0$. We call this $CP$ violation in decay because the $CP$ violating
phase $\xi$ comes from the decay matrix.

  * •

The most general possibility is $\eta\neq 0$, $\xi\neq 0$ and $\phi\neq 0$. In
this case, we have $CP$ violation due to both mass and decay provided
$\phi\neq\xi$.

We see that $\eta\neq 0$ in all the above three cases. However, a nonzero
value of $\eta$ is a necessary condition for $CP$ violation but not a
sufficient condition. For the two special cases, (a) $\phi=0=\xi$ and (b)
$\phi=\xi$, there is no $CP$ violation even when $\eta\neq 0$. In these two
cases, the $CP$ violating terms vanish and the flavor conversion probabilities
are the same as the vacuum probabilities multiplied by the decay term.
However, the presence of a nonzero value of $\eta$ is discernible in the
survival probabilities.

We now briefly discuss the values of parameters for which the effects
described in this work are likely to be observable. A bound of $\tau_{\nu}\geq
5.7\times 10^{5}$ s ($m_{\nu}/{\rm eV}$) is derived from the neutrino data of
Supernova 1987A Frieman:1987as , which leads to $\Gamma_{\nu}\equiv b\approx
10^{-21}$ eV for a neutrino of mass 1 eV. We take $\eta=b$, which satisfies
the semipositivity constraint $\eta\leq 2b$. The new effects considered in
this work are of order $\eta/(a_{2}-a_{1})=\eta E/\Delta m^{2}$. These effects
are of order 10% for $\Delta m^{2}\approx 10^{-4}\,{\rm eV^{2}}$ if $E\approx
10^{16}$ eV or $10^{7}$ GeV. That is, ultrahigh energy neutrinos from
astrophysical sources provide a platform to study the effect of the off-
diagonal decay term considered here.

##  V Summary and Conclusions

In this article, we point out scenarios in which the Majorana phase can appear
in neutrino oscillation probabilities that also cause $CP$ violation. We did
this analysis for two flavor oscillations, but the extension of this work for
three flavor oscillations is straightforward. $CP$ violation in neutrino
oscillations requires complex values of neutrino mixing matrix. In the case of
standard two flavor oscillations, the phases of this matrix, including the
Majorana phase, drop out of the evolution equation and there is no $CP$
violation. In this work, we have shown that the Majorana phase in two flavor
mixing remains in the evolution equation and causes $CP$ violation provided
the neutrinos decay and the decay eigenstates are not the same as mass
eigenstates. This requires the decay matrix $\Gamma$ to have an off-diagonal
term $\Gamma_{12}$. We have shown that two types of $CP$ violation are
possible: (a) that due to the Majorana phase $\phi$, which we call $CP$
violation in mass, and (b) that due to the phase $\xi$ of $\Gamma_{12}$, which
we call $CP$ violation in decay. The $CP$ violating term in the oscillation
probability is also sensitive to the neutrino mass ordering. In the two
special cases, when $\phi$ and $\xi$ are equal to each other or when both are
zero, there is no $CP$ violation even if the decay eigenstates are different
from the mass eigenstates. In such a situation, the flavor conversion
probabilities are insensitive to off-diagonal elements of $\Gamma$ but the
flavor survival probabilities do depend on them.

## Acknowledgments

K. D. and S. U. S. thank the Ministry of Education, Government of India, for
financial support through Institute of Eminence funding.

## References

  * (1) S. M. Bilenky, J. Hosek and S. T. Petcov, Phys. Lett. B 94 (1980), 495-498.
  * (2) J. Schechter and J. W. F. Valle, Phys. Rev. D 22 (1980), 2227.
  * (3) M. Doi, T. Kotani, H. Nishiura, K. Okuda and E. Takasugi, Phys. Lett. B 102 (1981), 323-326.
  * (4) C. Giunti, Phys. Lett. B 686 (2010), 41-43.
  * (5) F. Benatti and R. Floreanini, Phys. Rev. D 64 (2001), 085015.
  * (6) A. Capolupo, S. M. Giampaolo and G. Lambiase, Phys. Lett. B 792 (2019), 298-303.
  * (7) P. K. Kabir, “The CP puzzle: strange decays of the neutral kaon”. Academic Press (1968), Appendix A.
  * (8) G. C. Branco, L. Lavoura and J. P. Silva, “CP Violation,” Oxford University Press (1999), Section 6.2, pp 62-64.
  * (9) I. I. Bigi and A. I. Sanda, “CP violation,” Cambridge University Press (2009), Section 6.1, pp 90-92.
  * (10) J. M. Berryman, A. de Gouvêa, D. Hernández and R. L. N. Oliveira, Phys. Lett. B 742 (2015), 74-79
  * (11) D. S. Chattopadhyay, K. Chakraborty, A. Dighe, S. Goswami and S. M. Lakshmi, Phys. Rev. Lett. 129 (2022) no.1, 011802.
  * (12) Nielsen, M., Chuang, I. (2010). Quantum Computation and Quantum Information: 10th Anniversary Edition. Cambridge: Cambridge University Press.
  * (13) J. A. Frieman, H. E. Haber and K. Freese, Phys. Lett. B 200 (1988), 115-121 doi:10.1016/0370-2693(88)91120-3


# DCT Perceptron Layer: A Transform Domain Approach for Convolution Layer

Hongyi Pan
Xin Zhu
Salih Atici  Ahmet Enis Cetin
Department of Electrical and Computer Engineering
University of Illinois Chicago
{hpan21, xzhu61, satici2<EMAIL_ADDRESS>

###### Abstract

In this paper, we propose a novel Discrete Cosine Transform (DCT)-based neural
network layer which we call DCT-perceptron to replace the $3\times 3$ Conv2D
layers in the Residual neural Network (ResNet). Convolutional filtering
operations are performed in the DCT domain using element-wise multiplications
by taking advantage of the Fourier and DCT Convolution theorems. A trainable
soft-thresholding layer is used as the nonlinearity in the DCT perceptron.
Compared to ResNet’s Conv2D layer which is spatial-agnostic and channel-
specific, the proposed layer is location-specific and channel-specific. The
DCT-perceptron layer reduces the number of parameters and multiplications
significantly while maintaining comparable accuracy results of regular ResNets
in CIFAR-10 and ImageNet-1K. Moreover, the DCT-perceptron layer can be
inserted with a batch normalization layer before the global average pooling
layer in the conventional ResNets as an additional layer to improve
classification accuracy.

##  1 Introduction

Convolutional neural networks (CNNs) have produced remarkable results in image
classification [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], object detection [13,
14, 15, 16, 17] and semantic segmentation [18, 19, 20, 21, 22]. One of the
most widely used and successful CNNs is ResNet [6], which can build very deep
networks. However, the number of parameters significantly increases as the
network goes deeper to obtain better performance in accuracy. However, the
huge amount of parameters increases the computational load for the devices,
especially for edge devices with limited computational resources.

Figure 1: ImageNet-1K classification benchmark of the proposed methods
(orange) versus some other transform-based methods (green). The size of a
circle represents its computational cost. ResNet-50 (blue) is used as the
backbone.

(a)

(b)

Figure 2: ResNet’s convolutional residual blocks (a) versus the proposed DCT-
Perceptron (DCT-P) residual blocks (b). BN stands for batch normalization. One
$3\times 3$ Conv2D layer in each block is replaced by one proposed DCT-
perceptron layer. More than 21% parameters and 32% MACs are reduced for
ResNet-50 with reaching comparable accuracy results on ImageNet-1K.

Fourier convolution theorem states that the convolution
$\mathbf{y}=\mathbf{x}*\mathbf{w}$ of two vectors x and w in the space domain
can be implemented in the Discrete Fourier Transform (DFT) domain by
elementwise multiplication:

$\mathbf{Y}(k)=\mathbf{X}(k)\cdot\mathbf{W}(k),$ (1)

where $\mathbf{Y}=\mathscr{F}(\mathbf{y}),\mathbf{X}=\mathscr{F}(\mathbf{x})$,
and $\mathbf{W}=\mathscr{F}(\mathbf{w})$ are the DFTs of
$\mathbf{y},\mathbf{x}$, and $\mathbf{w}$, respectively, and $\mathscr{F}$ is
the Fourier transform operator. Equation 1 holds when the size of the DFT is
larger than the size of the convolution output $\mathbf{y}$. In this paper, we
use the Fourier convolution theorem to develop novel network layers. Since
there is one-to-one relationship between the kernel coefficients $\mathbf{w}$
and the Fourier transform $\mathbf{W}$, kernel weights can be learned in the
Fourier transform domain using backpropagation-type algorithms.

Since the DFT is a complex-valued transform, we replace the DFT with the real-
valued Discrete Cosine Transform (DCT). Our DCT-based layer replaces the
$3\times 3$ convolutional layer of ResNet, as shown in Figure 2. DCT can
express a vector in terms of a sum of cosine functions oscillating at
different frequencies the DCT coefficients contain frequency domain
information. DCT is the main enabler of a wide range of image and video coding
standards including JPEG and MPEG [23, 24]. With our proposed layer, our
models can reduce the number of parameters in ResNets significantly while
producing comparable accuracy results as the original ResNets. This is
possible because we can implement convolution-like operations in the DCT
domain using only elementwise multiplications as in the Fourier transform.
Both the DCT and the Inverse-DCT (IDCT) have fast $O(Nlog_{2}N)$ algorithms
which are actually faster than FFT because of the real-valued nature of the
transform. An important property of the DCT is that convolutions can be
implemented in the DCT domain using a modified version of the Fourier
Convolution Theorem [25].

##  2 Related transform domain methods

FFT-based methods The Fast Fourier transform (FFT) algorithm is the most
important signal and image processing method. Convolutions in time and image
domains can be performed using elementwise multiplications in the Fourier
domain. However, the Discrete Fourier Transform (DFT) is a complex transform.
In [26, 27], the fast Fourier Convolution (FFC) method is proposed in which
the authors designed the FFC layer based on the so-called Real-valued Fast
Fourier transform (RFFT). In RFFT-based methods, they concatenate the real and
the imaginary parts of the FFT outputs, and they apply convolutions with ReLU
in the concatenated Fourier domain. They do not take advantage of the Fourier
domain convolution theorem. Concatenating the real part and the imaginary part
of the complex tensors increases the number of parameters and their model
requires more parameters than the original ResNets because the number of
channels is doubled after concatenating. Our method takes advantage of the
convolution theorem and it can reduce the number of parameters significantly
while producing comparable accuracy results.

Wavelet-based methods Wavelet transform (WT) is a well-established signal and
image processing method and WT-based neural networks include [28, 29].
However, convolutions cannot be implemented in the wavelet domain.

DCT-based methods Other DCT-based methods include [30, 31, 32, 33, 34]. Since
the images are stored in the DCT domain in JPEG format, the authors use DCT
coefficients of the JPEG images in [30, 31, 32] but they did not use the
transform domain convolution theorem to train the network. In [33], transform
domain convolutional operations are used only during the testing phase. The
authors did not train the kernels of the CNN in the DCT domain. They only take
advantage of the fast DCT computation and reduce parameters by changing
$3\times 3$ Conv2D layers to $2\times 2$. In our implementation, we train the
filter parameters in the DCT domain and we use the soft-thresholding operator
as the nonlinearity. It is not possible to train a network using DCT without
soft-thresholding because both positive and negative amplitudes are important
in the DCT domain. In [34], Harmonic convolutional networks based on DCT were
proposed. Only forward DCT without inverse DCT computation is employed to
obtain Harmonic blocks for feature extraction. In contrast with spatial
convolution with learned kernels, this study proposes feature learning by
weighted combinations of responses of predefined filters. The latter extracts
harmonics from lower-level features in a region, and the latter layer applies
DCT on the outputs from the previous layer which are already encoded by DCT.

Trainable soft-thresholding The soft-thresholding function is widely used in
wavelet transform domain denoising [35] and as a proximal operator for the
$\ell_{1}$ norm [36]. With trainable threshold parameters, soft-thresholding
and its variants can be employed as the nonlinear function in the frequency
domain-based networks [37, 38, 39, 40]. In the frequency domain, ReLU and its
variants are not good choices because they remove negative valued frequency
components. This may cause issues when applying the inverse transform, as the
negative valued components are as equivalently important as the positive
components. On the contrary, the computational cost of the soft-thresholding
is similar to the ReLU, and it kepdf both positive and negative valued
frequency components exceeding a learnable threshold.

##  3 Methodology

In this section, we first review the discrete cosine transform (DCT). Then, we
introduce how we design the DCT-perceptron layer. Finally, we compare the
proposed DCT-perceptron layer with the conventional convolutional layer.

###  3.1 Background: DCT Convolution Theorem

Discrete cosine transform (DCT) is widely used for frequency analysis [41,
42]. The type-II one-dimensional (1D) DCT and inverse DCT (IDCT), which are
most generally used, are defined as follows:

$\mathbf{X}_{k}=\sum_{n=0}^{N-1}\mathbf{x}_{n}\text{cos}\left[\frac{\pi}{N}\left(n+\frac{1}{2}\right)k\right],$
(2)
$\mathbf{x}_{n}=\frac{1}{N}\mathbf{X}_{0}+\frac{2}{N}\sum_{k=1}^{N-1}\mathbf{X}_{k}\text{cos}\left[\frac{\pi}{N}\left(n+\frac{1}{2}\right)k\right],$
(3)

for $0\leq k\leq N-1,0\leq n\leq N-1,$ respectively.

DCT and IDCT can be implemented using butterfly operations. For an $N$-length
vector, the complexity of each 1D transform is $O(N\log_{2}N)$. The 2D DCT is
obtained from 1D DCT in a separable manner and a two-dimensional (2D) DCT can
be implemented in a separable manner using 1D-DCTs for computational
efficiency. The complexity of 2D-DCT and 2D-IDCT on an $N\times N$ image is
$O(N^{2}\log_{2}N)$ [43].

Compared to the Discrete Fourier transform (DFT), DCT is a real-valued
transform and it also decomposes a given signal or image according to its
frequency content. Therefore, DCT is more suitable for deep neural networks in
terms of computational cost.

DFT convolutional theorem states that an input feature map
$\mathbf{x}\in\mathbb{R}^{N}$ and a kernel $\mathbf{w}\in\mathbb{R}^{K}$ can
be convolved as:

$\mathbf{x}*_{c}\mathbf{w}=\mathscr{F}^{-1}\left(\mathscr{F}(\mathbf{x})\cdot\mathscr{F}(\mathbf{w})\right),$
(4)

where, $\mathscr{F}$ stands for DFT and $\mathscr{F}^{-1}$ stands for IDFT.
“$*_{c}$” is the circular convolution operator and “$\cdot$” is the element-
wise multiplication. In practice, the circular convolution “$*_{c}$” can be
converted to regular convolution “*” by padding zeros to input signals
properly and making the size of the DFT larger than the size of the
convolution. Similarly, the DCT convolution theorem is

$\mathbf{x}*^{s}\mathbf{w}=\mathscr{D}^{-1}\left(\mathscr{D}(\mathbf{x})\cdot\mathscr{D}(\mathbf{w})\right),$
(5)

where, $\mathscr{D}$ stands for DCT and $\mathscr{D}^{-1}$ stands for IDCT.
“$*^{s}$” stands for symmetric convolution and its relationship to the linear
convolution is

$\mathbf{x}*^{s}\mathbf{w}=\mathbf{x}*\mathbf{\tilde{w}}$ (6)

where $\mathbf{\tilde{w}}$ is the symmetrically extended kernel in
$\mathbb{R}^{2K-1}$:

$\mathbf{\tilde{w}}_{k}=\mathbf{w}_{|K-1-k|},$ (7)

for $k=0,1,...,2K-2$. DCT and DFT convolution theorems can be extended to two
or three dimensions in a straightforward manner. In the next subsection, we
will describe how we will use DCT as a deep-neural network layer. We call this
new layer the DCT-Perceptron layer.

###  3.2 DCT-Perceptron Layer

An overview of the DCT-perceptron layer architecture is presented in Figure 3.
We first compute $c$ 2D-DCTs of the input feature map tensor along the width
($W$) and height ($H$) axes. In the DCT domain, a scaling layer performs the
filtering operation by element-wise multiplication. After this step, we have
$1\times 1$ Conv2D layer, and a trainable soft-thresholding layer which
denoises the data and acts as the nonlinearity of the DCT-perceptron as shown
in Figure 3.

(a)

(b)

Figure 3: (a) DCT-perceptron and (b) tripod DCT-perceptron. Since the DCT2D
and the IDCT2D are linear transforms, summation in the DCT domain and in the
space domain are equivalent. For computational efficiency, the DCT2D and
IDCT2D are implemented as two 1D transforms along the weight and the height,
respectively. In our CIFAR-10 experiment the tripod structure reaches the
highest accuracy among the multiple-pod structures.

The scaling layer is derived from the property that the convolution in the
space domain is equivalent to the elementwise multiplication in the transform
domain. In detail, given an input tensor in $\mathbb{R}^{C\times W\times H}$,
we element-wise multiply it with a weight matrix in $\mathbb{R}^{W\times H}$
to perform an operation equivalent to the spatial domain convolutional
filtering.

The trainable soft-thresholding layer is applied to remove small entries in
the DCT domain. It is similar to image coding and transform domain denoising.
It is defined as:

$\mathbf{y}=\mathcal{S}_{T}(\mathbf{x})=\text{sign}(\mathbf{x})(|\mathbf{x}|-T)_{+},$
(8)

where, $T$ is a non-negative trainable threshold parameter. For an input
tensor in $\mathbb{R}^{C\times W\times H}$, there are $W\times H$ different
threshold parameters. The threshold parameters are determined using the back-
propagation algorithm. Our CIFAR-10 experiments show that the soft-
thresholding is superior to the ReLU as the non-linear function in DCT
analysis.

Overall, the DCT-perceptron layer is computed as:

$\mathbf{y}=\mathscr{D}^{-1}\left(\mathcal{S}_{\mathbf{T}}\left(\mathscr{D}(\mathbf{x})\cdot\mathbf{V})\otimes\mathbf{H}\right)\right)+\mathbf{x},$
(9)

where, $\mathscr{D}$ and $\mathscr{D}^{-1}$ stand for 2D-DCT and 2D-IDCT,
$\mathbf{V}$ is the scaling matrix, $\mathbf{H}$ are the kernels in the
$1\times 1$ Conv2D layer, $\mathbf{T}$ is the threshold parameter matrix in
the soft-thresholding, “$\cdot$” stands for the element-wise multiplication,
and $\otimes$ represents a $1\times 1$ 2D convolution over an input composed
of several input planes performed using PyTorch’s Conv2D API.

We further extend the DCT-perceptron layer to a tripod structure to get higher
accuracy with more trainable parameters (still significantly fewer than
Conv2D) as shown in Figure LABEL:fig:_DCT-Px3. The process of layers for the
DCT domain analysis (scaling, $1\times 1$ Conv2D, and soft-thresholding) is
depicted in Figure 4.

Figure 4: Processing in the DCT domain. Multiplications in the soft-
thresholding block are efficiently implemented using sign-bit operations.

In this section, we compare the difference between the DCT-perceptron layer
and the Conv2D layer of ResNet. To compare the computational cost and the
number of parameters, we specificity stride to 1 and apply padding=“SAME” on
the Conv2D layers.

As Figure 5 shows, convolution has spatial-agnostic and channel-specific
characteristics in ResNet’s Conv2D layer. Because of the spatial-agnostic
characteristics of a Conv2D layer, the network cannot adapt different visual
patterns corresponding to different spatial locations. On the contrary, the
DCT-perceptron layer is location-specific and channel-specific. 2D-DCT is
location-specific but channel-agnostic, as the 2D-DCT is computed using the
entire block as a weighted summation on the spatial feature map. The scaling
layer is also location-specific but channel-agnostic, as different scaling
parameters (filters) are applied on different entries, and weights are shared
in different channels. That is why we also use Pytorch’s $1\times 1$ Conv2D to
make the DCT-perceptron layer channel-specific.

(a)

(b)

Figure 5: To compute different entries of an output slice, the Conv2D layer uses same weights while the DCT-perceptron layer uses different. Thus, the Conv2D layer is spatial-agnostic while the DCT-Perceptron layer is location-specific. Layer (Operation) | Parameters | MACs
---|---|---
$K\times K$ Conv2D | $K^{2}C^{2}$ | $K^{2}N^{2}C^{2}$
$3\times 3$ Conv2D | $9C^{2}$ | $9N^{2}C^{2}$
DCT2D | $0$ | $(\frac{5N^{2}}{2}\log_{2}N+\frac{N^{2}}{3}-6N+\frac{62}{3})C$
Scaling, Soft-Thresholding | $2N^{2}$ | $N^{2}C$
$1\times 1$ Conv2D | $C^{2}$ | $N^{2}C^{2}$
IDCT2D | $0$ | $(\frac{5N^{2}}{2}\log_{2}N+\frac{N^{2}}{3}-6N+\frac{62}{3})C$
DCT-Perceptron | $2N^{2}+C^{2}$ | $(5N^{2}\log_{2}N+\frac{5N^{2}}{3}-6N+\frac{124}{3})C+N^{2}C^{2}$
Tripod DCT-Perceptron | $6N^{2}+3C^{2}$ | $(5N^{2}\log_{2}N+\frac{11N^{2}}{3}-6N+\frac{124}{3})C+3N^{2}C^{2}$
Table 1: Parameters and Multiply–Accumulate (MACs) for a $C$-channel $N\times
N$ image.

The comparison of the number of parameters and Multiply–Accumulate (MACs) is
presented in Table 1. In a Conv2D layer, we need to compute $N^{2}K^{2}$
multiplications for a $C$-channel feature map of size $N\times N$ with a
kernel size of $K\times K$. In the DCT-perceptron layer, there are $N^{2}$
multiplications from the scaling and $N^{2}$ multiplications from the $1\times
1$ Conv2D layer. There is no multiplication in the soft-thresholding because
the product between the sign of the input and the subtraction result can be
implemented using the sign-bit operations only. There are
$\frac{N^{2}}{2}\log_{2}N+\frac{N^{2}}{3}-2N+\frac{8}{3}$ multiplications with
$\frac{5N^{2}}{2}\log_{2}N+\frac{N^{2}}{3}-6N+\frac{62}{3}$ additions to
implement a 2D-DCT or a 2D-IDCT using the fast DCT algorithms with butterfly
operations [43]. Therefore, there are totally
$(5N^{2}\log_{2}N+\frac{5N^{2}}{3}-6N+\frac{124}{3})C+N^{2}C^{2}$ MACs in a
single-pod DCT-perceptron layer and
$(5N^{2}\log_{2}N+\frac{11N^{2}}{3}-6N+\frac{124}{3})C+3N^{2}C^{2}$ MACs in a
tripod DCT-perceptron layer. Compared to the ResNet’s Conv2D layer with
$9N^{2}C^{2}$ MACs, the DCT-perceptron layer reduces $6N^{2}C^{2}$ MACs to
$(5N^{2}\log_{2}N+\frac{11N^{2}}{3}-6N+\frac{124}{3})C$ MACs. In other words,
it reduces $O(N^{2}C^{2})$ to $O((N^{2}\log_{2}N)C)$ if we omit $3N^{2}C^{2}$
MACs. This significantly reduces the computation cost because $C$ is much
larger than $N$ in most of the hidden layers of ResNets. A tripod DCT-
perceptron layer only has $2N^{2}C+2N^{2}C^{2}$ more MACs than a single-pod
DCT-perceptron layer because the 2D-DCT in each pod is equivalent to each
other.

###  3.3 Introducing DCT-Perceptron to ResNets

ResNet-20 and ResNet-18 use convolutional residual block V1 in Fig. 2, while
ResNet-50 uses block V2 as shown in Fig. 2. We replace some $3\times 3$ Conv2D
layers whose input and output shapes are the same in the ResNets’
convolutional residual blocks by the proposed DCT-perceptron layer. In
ResNet-20 and ResNet-18, we replace the second $3\times 3$ Conv2D layer in
each convolution block. In ResNet-50, we replace $3\times 3$ Conv2D layers in
convolution blocks except for Conv2_1, Conv3_1, Conv4_1, and Conv5_1, as the
downsampling is performed by them. We call those revised using the DCT-
perceptron layer as DCT-ResNets and those revised using the tripod DCT-
perceptron layer as Tripod-DCT-ResNets. Details of the DCT-ResNets are
presented in Tables 2, 3 and 4, respectively. In our ablation study on
CIFAR-10, we also replace more $3\times 3$ Conv2D layers, but the experiments
show that replacing more layers leads to drops in accuracy as these models
have fewer parameters.

Layer | Output Shape | Implementation Details
---|---|---
Conv1 | $16\times 32\times 32$ | $3\times 3,16$
Conv2_x | $16\times 32\times 32$ | $\left[\begin{array}[]{c}3\times 3,16\\\ \bf{\text{DCT-P},16}\end{array}\right]\times 3$
Conv3_x | $32\times 16\times 16$ | $\left[\begin{array}[]{c}3\times 3,32\\\ \bf{\text{DCT-P},32}\end{array}\right]\times 3$
Conv4_x | $64\times 8\times 8$ | $\left[\begin{array}[]{c}3\times 3,32\\\ \bf{\text{DCT-P},64}\end{array}\right]\times 3$
GAP | $64$ | Global Average Pooling
Output | $10$ | Linear
Table 2: Structure of DCT-ResNet-20 for the CIFAR-10 classification task. Building blocks are shown in brackets, with the numbers of blocks stacked. Downsampling is performed by Conv3_1 and Conv4_1 with a stride of 2. We name layers following Table 1 in[6] for the comparison of structures easily. Layer | Output Shape | Implementation Details
---|---|---
Conv1 | $64\times 112\times 112$ |  $7\times 7,64$, stride 2
MaxPool | $64\times 56\times 56$ |  $2\times 2$, stride 2
Conv2_x | $64\times 56\times 56$ | $\left[\begin{array}[]{c}3\times 3,64\\\ \bf{\text{DCT-P},64}\end{array}\right]\times 2$
Conv3_x | $128\times 28\times 28$ | $\left[\begin{array}[]{c}3\times 3,128\\\ \bf{\text{DCT-P},128}\end{array}\right]\times 2$
Conv4_x | $256\times 14\times 14$ | $\left[\begin{array}[]{c}3\times 3,256\\\ \bf{\text{DCT-P},256}\end{array}\right]\times 2$
Conv5_x | $512\times 7\times 7$ | $\left[\begin{array}[]{c}3\times 3,512\\\ \bf{\text{DCT-P},512}\end{array}\right]\times 2$
GAP | $512$ | Global Average Pooling
Output | $1000$ | Linear
Table 3: Structure of DCT-ResNet-18 for the ImageNet-1K classification task. Building blocks are shown in brackets, with the numbers of blocks stacked. Downsampling is performed by Conv3_1, Conv4_1 and Conv5_1 with a stride of 2. Layer | Output Shape | Implementation Details
---|---|---
Conv1 | $64\times 112\times 112$ |  $7\times 7,64$, stride 2
MaxPool | $64\times 56\times 56$ |  $2\times 2$, stride 2
Conv2_1 | $256\times 56\times 56$ | $\left[\begin{array}[]{c}1\times 1,64\\\ 3\times 3,64,\text{stride }2\\\ 1\times 1,256\end{array}\right]$
Conv2_x | $256\times 56\times 56$ | $\left[\begin{array}[]{c}1\times 1,64\\\ \bf{\text{DCT-P},64}\\\ 1\times 1,256\end{array}\right]\times 2$
Conv3_1 | $512\times 28\times 28$ | $\left[\begin{array}[]{c}1\times 1,128\\\ 3\times 3,128,\text{stride }2\\\ 1\times 1,512\end{array}\right]$
Conv3_x | $512\times 28\times 28$ | $\left[\begin{array}[]{c}1\times 1,128\\\ \bf{\text{DCT-P},128}\\\ 1\times 1,512\end{array}\right]\times 3$
Conv4_1 | $1024\times 14\times 14$ | $\left[\begin{array}[]{c}1\times 1,256\\\ 3\times 3,256,\text{stride }2\\\ 1\times 1,1024\end{array}\right]$
Conv4_x | $1024\times 14\times 14$ | $\left[\begin{array}[]{c}1\times 1,256\\\ \bf{\text{DCT-P},256}\\\ 1\times 1,1024\end{array}\right]\times 5$
Conv5_1 | $2048\times 7\times 7$ | $\left[\begin{array}[]{c}1\times 1,512\\\ 3\times 3,512,\text{stride }2\\\ 1\times 1,2048\end{array}\right]$
Conv5_x | $2048\times 7\times 7$ | $\left[\begin{array}[]{c}1\times 1,512\\\ \bf{\text{DCT-P},512}\\\ 1\times 1,2048\end{array}\right]\times 2$
GAP | $2048$ | Global Average Pooling
Output | $1000$ | Linear
Table 4: Structure of DCT-ResNet-50 for the ImageNet-1K classification task.
Building blocks are shown in brackets, with the numbers of blocks stacked.

##  4 Experimental Results

Our experiments are carried out on a workstation computer with an NVIDIA RTX
3090 GPU. The code is written in PyTorch in Python 3. First, we experiment on
the CIFAR-10 dataset with ablation studies. Then, we compare DCT-ResNet-18 and
DCT-ResNet-50 with other related works on the ImageNet-1K dataset. We also
insert an additional DCT-perceptron layer with batch normalization before the
global average pooling layer. The additional DCT-perceptron layer improves the
accuracy of the original ResNets.

Method | Parameters | Accuracy
---|---|---
ResNet-20 [6] (official) | 0.27M | 91.25%
ResNet-20 (our trial, baseline) | 272,474 | 91.66%
DCT-ResNet-20 | 151,514 (44.39%$\downarrow$) | 91.59%
DCT-ResNet-20 (without shortcut connection in DCT-Perceptron) | 151,514 (44.39%$\downarrow$) | 91.12%
DCT-ResNet-20 (without scaling in DCT-Perceptron) | 147,482 (45.87%$\downarrow$) | 90.46%
DCT-ResNet-20 (using ReLU with thresholds in DCT-Perceptron) | 151,514 (44.39%$\downarrow$) | 91.30%
DCT-ResNet-20 (using ReLU in DCT-Perceptron) | 147,818 (45.75%$\downarrow$) | 91.06%
DCT-ResNet-20 (replacing all Conv2D) | 51,034 (81.31%$\downarrow$) | 85.74%
Bipod-DCT-ResNet-20 | 175,706 (35.51%$\downarrow$) | 91.42%
TriPod-DCT-ResNet-20 | 199,898 (26.64%$\downarrow$) | 91.75%
Tripod-DCT-ResNet-20 (with shortcut connection in tripod DCT-Perceptron) | 199,898 (26.64%$\downarrow$) | 91.50%
Quadpod-DCT-ResNet-20 | 224,090 (17.76%$\downarrow$) | 91.48%
Quintpod-DCT-ResNet-20 | 248,282 (8.88%$\downarrow$) | 91.47%
ResNet-20+1DCT-P | 276,826 (1.60%$\uparrow$) | 91.82%
Table 5: Ablation study: CIFAR-10 classification Results. Method | Parameters (M) | MACs (G) | Center-Crop Top1 Acc. | Top5 Acc.
---|---|---|---|---
ResNet-18 [6] (Torchvision [44], baseline) | 11.69 | 1.822 | 69.76% | 89.08%
DCT-based ResNet-18 [33] | 8.74 | - | 68.31% | 88.22%
DCT-ResNet-18 | 6.14 (47.5%$\downarrow$) | 1.374 (24.6%$\downarrow$) | 67.84% | 87.73%
Tripod-DCT-ResNet-18 | 7.56 (35.3%$\downarrow$) | 1.377 (24.4%$\downarrow$) | 69.55% | 89.04%
Tripod-DCT-ResNet-18+1DCT-P | 7.82 (33.1%$\downarrow$) | 1.378 (24.4%$\downarrow$) | 69.88% | 89.00%
ResNet-50 [6] (official [45]) | 25.56 | 3.86 | 75.3% | 92.2%
ResNet-50 [6] (Torchvision [44]) | 25.56 | 4.122 | 76.13% | 92.86%
Order 1 + ScatResNet-50 [29] | 27.8 | - | 74.5% | 92.0%
JPEG-ResNet-50 [30] | 28.4 | 5.4 | 76.06% | 93.02%
ResNet-50+DCT+FBS (3x32) [31] | 26.2 | 3.68 | 70.22% | -
ResNet-50+DCT+FBS (3x16) [31] | 25.6 | 3.18 | 67.03% | -
Faster JPEG-ResNet-50 [32] | 25.1 | 2.86 | 70.49% | -
DCT-based ResNet-50 [33] | 21.30 | - | 74.73% | 92.30%
Harm-ResNet-50 [34] | 25.6 | - | 75.94% | 92.88%
FFC-ResNet-50 (+LFU, $\alpha=0.25$) [26] | 26.7 | 4.3 | 77.8% | -
FFC-ResNet-50 ($\alpha=1$) [26] | 34.2 | 5.6 | 75.2% | -
ResNet-50 (our trial, baseline) | 25.56 | 4.122 | 76.06% | 92.85%
DCT-ResNet-50 | 18.28 (28.5%$\downarrow$) | 2.772 (32.8%$\downarrow$) | 75.17% | 92.47%
Tripod-DCT-ResNet-50 | 20.15 (21.1%$\downarrow$) | 2.780 (32.6%$\downarrow$) | 75.52% | 92.56%
Tripod-DCT-ResNet-50+1DCT-P | 24.35 (4.7%$\downarrow$) | 2.783 (32.5%$\downarrow$) | 75.82% | 92.76%
Table 6: ImageNet-1K center-crop results. We use Torchvision’s official ResNet-18 model as the ResNet-18 baseline since we use PyTorch official training code with the default setting to train revised ResNet-18s. Other frequency-analysis-based methods are listed for comparison. In FFC-ResNet-50 (+LFU, $\alpha=0.25$) [26], they use their proposed Fourier Units (FU) and local Fourier units (LFU) to process only 25% channels of each input tensor and use regular $3\times 3$ Conv2D to process the remaining to get the accuracy of 77.8% but top 5 accuracy result was not reported. We can also use a combination of regular $3\times 3$ Conv2D layers and DCT-perceptron layers to increase the accuracy by increasing the parameters. According to the ablation study in [26], if all channels are processed by their proposed FU, FFC-ResNet-50 ($\alpha=1$) only gets the accuracy of 75.2%. The reported number of parameters and MACs are much higher than the baseline in [26] even though MACs from the FFT and IFFT are not counted. Our methods obtain comparable accuracy to regular ResNets with significantly reduced parameters and MACs.  Method | Parameters (M) | MACs (G) | 10-Crop Top1 Acc. | Top5 Acc.
---|---|---|---|---
ResNet-18 (Torchvision [44], baseline) | 11.69 | 1.822 | 71.86% | 90.60%
DCT-ResNet-18 | 6.14 (47.5%$\downarrow$) | 1.374 (24.6%$\downarrow$) | 70.09% | 89.40%
Tripod-DCT-ResNet-18 | 7.56 (35.3%$\downarrow$) | 1.377 (24.4%$\downarrow$) | 71.91% | 90.55%
Tripod-DCT-ResNet-18+1DCT-P | 7.82 (33.1%$\downarrow$) | 1.378 (24.4%$\downarrow$) | 72.20% | 90.51%
ResNet-50 [6] (official [45]) | 25.56 | 3.86 | 77.1% | 93.3%
ResNet-50 [6] (Torchvision [44]) | 25.56 | 4.122 | 77.43% | 93.75%
ResNet-50 (our trial, baseline) | 25.56 | 4.122 | 77.53% | 93.75%
DCT-ResNet-50 | 18.28 (28.5%$\downarrow$) | 2.772 (32.8%$\downarrow$) | 76.75% | 93.44%
Tripod-DCT-ResNet-50 | 20.15 (21.1%$\downarrow$) | 2.780 (32.6%$\downarrow$) | 77.36% | 93.62%
Tripod-DCT-ResNet-50+1DCT-P | 24.35 (4.7%$\downarrow$) | 2.783 (32.5%$\downarrow$) | 77.41% | 93.66%
Table 7: ImageNet-1K 10-crop results. Our models obtain comparable accuracy with significantly reduced parameters and MACs. Method | Parameters (M) | MACs (G) | Center-Crop Top-1 | Top-5 | 10-Crop Top-1 | Top-5
---|---|---|---|---|---|---
ResNet-18 | 11.69 | 1.822 | 69.76% | 89.08% | 71.86% | 90.60%
ResNet-18+1DCT-P | 11.95 | 1.823 | 70.50% | 89.29% | 72.89% | 90.98%
ResNet-50 | 25.56 | 4.122 | 76.06% | 92.85% | 77.53% | 93.75%
ResNet-50+1DCT-P | 29.76 | 4.124 | 76.09% | 93.80% | 77.67% | 93.84%
Table 8: ImageNet-1K results of inserting an additional DCT-perceptron layer
before the global average pooling layer. The additional DCT-perceptron layer
improves the accuracy of ResNets without changing the main structure of the
ResNets.

###  4.1 Ablation Study: CIFAR-10 Classification

Training ResNet-20 and DCT-ResNet-20s follows the implementation in [6]. We
use an SGD optimizer with a weight decay of 0.0001 and momentum of 0.9. These
models are trained with a mini-batch size of 128. The initial learning rate is
0.1 for 200 epochs, and the learning rate is reduced by a factor of 1/10 at
epochs 82, 122, and 163, respectively. Data augmentation is implemented as
follows: First, we pad 4 pixels on the training images. Then, we apply random
cropping to get 32 by 32 images. Finally, we randomly flip images
horizontally. We normalize the images with the means of [0.4914, 0.4822,
0.4465] and the standard variations of [0.2023, 0.1994, 02010]. During the
training, the best models are saved based on the accuracy of the CIFAR-10 test
dataset, and their accuracy numbers are reported in Table 5.

Figure 6 shows the CIFAR-10 test error history during the training. As shown
in Table 5, the single-pod DCT-perceptron layer-based ResNet (DCT-ResNet-20)
has 44.39% lower parameters than regular ResNet-20, but the model only suffers
an accuracy loss of 0.07%. On the other hand, the tripod DCT-perceptron layer-
based ResNet (TriPod-DCT-ResNet-20) achieves 0.09% higher accuracy than the
baseline ResNet-20 model. TriPod-DCT-ResNet-20 has 26.64% lower parameters
than regular ResNet-20. When we add an extra DCT-perceptron layer to the
regular ResNet-20 (ResNet-20+1DCT-P) we even get a higher accuracy (91.82%) as
shown in Table 5.

(a)

(b)

Figure 6: Training on CIFAR-10. Curves denote test error. Left: ResNet-20 vs
DCT-ResNet-20. Right: ResNet-20 vs Tripod DCT-ResNet-20. More than 26%
parameters are reduced using the DCT-perceptron layers, while accuracy results
are comparable.

In the ablation study on the DCT-perceptron layer, first, we first remove the
residual design (the shortcut connection) in the DCT-perceptron layers and get
a worse accuracy (91.12%). Then, we remove scaling which is convolutional
filtering in the DCT-perceptron layers. In this case, the multiplication
operations in the DCT-perceptron layer are only implemented in the $1\times 1$
Conv2D. The accuracy drops from 91.59% to 90.46%. Therefore, scaling is
necessary to maintain accuracy. Next, we apply ReLU instead of soft-
thresholding in the DCT-perceptron layers. We first apply the same thresholds
as the soft-thresholding thresholds on ReLU. In this case, the number of
parameters is the same as in the proposed DCT-ResNet-20 model, but the
accuracy drops to 91.30%. We then try regular ReLU with a bias term in the
$1\times 1$ Conv2D layers in the DCT-perceptron layers, and the accuracy drops
further to 91.06%. These experiments show that the soft-thresholding is
superior to the ReLU in the DCT analysis, as the soft-thresholding retains
negative DCT components with high amplitudes, which are also used in denoising
and image coding applications. Furthermore, we replace all Conv2D layers in
the ResNet-20 model with the DCT-perceptron layers. We implement the
downsampling (Conv2D with the stride of 2) by truncating the IDCT2D so that
each dimension of the reconstructed image is one-half the length of the
original. In this method, 81.31% parameters are reduced, but the accuracy
drops to 85.74% due to lacking parameters.

In the ablation study on the tripod DCT-perceptron layer, we also try the
bipod, the quadpod, and the quintpod structures, but their accuracy is lower
than the tripod. Moreover, we also try adding the residual design in the
tripod DCT-perception layer, but our experiments show that the shortcut
connection is redundant. This may be because there are three channels in the
tripod structure replacing the shortcut. The multiple-channeled structure
allows the derivatives to propagate to earlier channels better than a single
channel.

###  4.2 ImageNet-1K Classification

We employ PyTorch’s official ImageNet-1K training code [46] in this section.
Since we use the PyTorch official training code with the default training
setting to train the revised ResNet-18s, we use the official trained ResNet-18
model from the PyTorch Torchvision as the baseline network. We use an NVIDIA
RTX3090 to train the ResNet-50 and the corresponding DCT-perceptron versions.
The default training needs about 26–30 GB GPU memory, while an RTX3090 only
has 24GB. Therefore, we halve the batch size and the learning rate
correspondingly. We use an SGD optimizer with a weight decay of 0.0001 and
momentum of 0.9. DCT-perceptron-based ResNet-18’s are trained with the default
setting: a mini-batch size of 256, an initial learning rate of 0.1 for 90
epochs. ResNet-50 and DCT-Perceptron-based models are trained with a mini-
batch size of 128, the initial learning rate is 0.05 for 90 epochs. The
learning rate is reduced by a factor of 1/10 after every 30 epochs. For data
argumentation, we apply random resized crops on training images to get 224 by
224 images, then we randomly flip images horizontally. We normalize the images
with the means of [0.485, 0.456, 0.406] and the standard variations of [0.229,
0.224, 0.225], respectively. We evaluate our models on the ImageNet-1K
validation dataset and compare them with the state-of-art papers. During the
training, the best models are saved based on the center-crop top-1 accuracy on
the ImageNet-1K validation dataset, and their accuracy numbers are reported in
Tables 6 and 7.

Figure 7 shows the test error history on the ImageNet-1K validation dataset
during the training phase. As shown in Table 6, we reduce 35.3% parameters and
22.4% MACs of regular ResNet-18 using the tripod DCT-perceptron layer, and the
center-crop top-1 accuracy only drops from 69.76% to 69.55%. It improves the
10-Crop top-1 accuracy from 71.86% to 71.91%. The single-pod DCT-ResNet-18
achieves a relatively poor accuracy (67.84%) because the parameters are
insufficient (47.5% reduced) for the ImageNet-1K Task. The DCT-ResNet-50 model
contains 28.5% less parameters with 32.8% less MACs than the baseline
ResNet-50 model, and its center-crop top-1 accuracy only drops from 76.06% to
75.17%. The tripod DCT-ResNet-50 model contains 21.1% less parameters with
32.6% less MACs than the baseline ResNet-50 model, while its center-crop top-1
accuracy only drops from 76.06% to 75.52%.

(a)

(b)

Figure 7: Training on ImageNet-1K. Curves denote the validation error of the
center crops. Left: ResNet-50 vs DCT-ResNet-50. Right: ResNet-50 vs Tripod
DCT-ResNet-50. More than 21% parameters and 32% MACs are reduced using the
DCT-perceptron layers with comparable accuracy to regular ResNet-50.

We can also increase the accuracy of regular ResNets by inserting an
additional DCT-perceptron layer with batch normalization after the global
average pooling layer of ResNets. We call these models as ResNets+1DCT-P. This
additional layer leads to a negligible increase in MACs. As shown in Table 5,
this additional layer increases the accuracy of ResNet-20 on CIFAR-10 from
91.66% to 91.82% with only 1.60% extra parameters. An additional DCT-
Perceptron layer increases the center-crop top-1 accuracy on ResNet-18 from
69.76% to 70.50% only with 2.3% extra parameters with 0.05% higher MACs, and
it increases ResNet-50’s center-crop top-5 accuracy from 92.85% to 93.80% with
16.4% extra parameters with 0.05% higher MACs. The additional layer can also
be applied to the Tripod-DCT-ResNets to further improve the accuracy.

##  5 Conclusion

In this paper, we proposed a novel layer based on DCT to replace $3\times 3$
Conv2D layers in convolutional neural networks. The proposed layer is derived
from the DCT convolution theorem. Processing in the DCT domain consists of a
scaling layer, which is equivalent to convolutional filtering, a $1\times 1$
Conv2D layer, and a trainable soft-thresholding function. In our CIFAR-10
experiments, we replaced the $3\times 3$ Conv2D layers of ResNet-20 using the
proposed DCT-perceptron layer. We reduced 44.39% parameters by obtaining a
comparable result (accuracy -0.07%), or 26.64% parameters by obtaining even a
slightly better result (accuracy + 0.09%), compared to the baseline ResNet-20
model. In our ImageNet-1K experiments, we also obtain comparable results
(accuracy drops less than 0.8%) compared to regular ResNet-50 while reducing
the number of parameters by more than 21% and MACs by more than 32%.
Furthermore, we improve the results of regular ResNets with an additional
layer of DCT-perceptron. We insert the proposed DCT-perceptron layer into
regular ResNets with batch normalization before the global average pooling
layer to improve the accuracy with a slight increase of MACs.

## References

  * [1] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  Advances in neural information processing systems, 25:1097–1105, 2012.
  * [2] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [3] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich.  Going deeper with convolutions.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015.
  * [4] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Identity mappings in deep residual networks.  In European conference on computer vision, pages 630–645. Springer, 2016.
  * [5] Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xiaogang Wang, and Xiaoou Tang.  Residual attention network for image classification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3156–3164, 2017.
  * [6] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [7] Diaa Badawi, Hongyi Pan, Sinan Cem Cetin, and A Enis Cetin.  Computationally efficient spatio-temporal dynamic texture recognition for volatile organic compound (voc) leakage detection in industrial plants.  IEEE Journal of Selected Topics in Signal Processing, 14(4):676–687, 2020.
  * [8] Hongyi Pan, Diaa Badawi, and Ahmet Enis Cetin.  Computationally efficient wildfire detection method using a deep convolutional network pruned via fourier analysis.  Sensors, 20(10):2891, 2020.
  * [9] Chirag Agarwal, Shahin Khobahi, Dan Schonfeld, and Mojtaba Soltanalian.  Coronet: a deep network architecture for enhanced identification of covid-19 from chest x-ray images.  In Medical Imaging 2021: Computer-Aided Diagnosis, volume 11597, page 1159722. International Society for Optics and Photonics, 2021.
  * [10] Harris Partaourides, Kostantinos Papadamou, Nicolas Kourtellis, Ilias Leontiades, and Sotirios Chatzis.  A self-attentive emotion recognition network.  In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7199–7203. IEEE, 2020.
  * [11] Dimitrios Stamoulis, Ting-Wu Chin, Anand Krishnan Prakash, Haocheng Fang, Sribhuvan Sajja, Mitchell Bognar, and Diana Marculescu.  Designing adaptive neural networks for energy-constrained image classification.  In Proceedings of the International Conference on Computer-Aided Design, pages 1–8, 2018.
  * [12] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.  A convnet for the 2020s.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11976–11986, 2022.
  * [13] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You only look once: Unified, real-time object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016.
  * [14] Süleyman Aslan, Uğur Güdükbay, B Uğur Töreyin, and A Enis Çetin.  Deep convolutional generative adversarial networks for flame detection in video.  In International Conference on Computational Collective Intelligence, pages 807–815. Springer, 2020.
  * [15] Guglielmo Menchetti, Zhanli Chen, Diana J Wilkie, Rashid Ansari, Yasemin Yardimci, and A Enis Çetin.  Pain detection from facial videos using two-stage deep learning.  In 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pages 1–5. IEEE, 2019.
  * [16] Süleyman Aslan, Uğur Güdükbay, B Uğur Töreyin, and A Enis Çetin.  Early wildfire smoke detection based on motion-based geometric image transformation and deep convolutional generative adversarial networks.  In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8315–8319. IEEE, 2019.
  * [17] Xianpeng Liu, Nan Xue, and Tianfu Wu.  Learning auxiliary monocular contexts helps monocular 3d object detection.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 1810–1818, 2022.
  * [18] Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang.  Bisenet: Bilateral segmentation network for real-time semantic segmentation.  In Proceedings of the European conference on computer vision (ECCV), pages 325–341, 2018.
  * [19] Zilong Huang, Xinggang Wang, Lichao Huang, Chang Huang, Yunchao Wei, and Wenyu Liu.  Ccnet: Criss-cross attention for semantic segmentation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 603–612, 2019.
  * [20] Jonathan Long, Evan Shelhamer, and Trevor Darrell.  Fully convolutional networks for semantic segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431–3440, 2015.
  * [21] Rudra PK Poudel, Stephan Liwicki, and Roberto Cipolla.  Fast-scnn: fast semantic segmentation network.  arXiv preprint arXiv:1902.04502, 2019.
  * [22] Yinli Jin, Wenbang Hao, Ping Wang, and Jun Wang.  Fast detection of traffic congestion from ultra-low frame rate image based on semantic segmentation.  In 2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA), pages 528–532. IEEE, 2019.
  * [23] Gregory K Wallace.  The jpeg still picture compression standard.  Communications of the ACM, 34(4):30–44, 1991.
  * [24] Didier Le Gall.  Mpeg: A video compression standard for multimedia applications.  Communications of the ACM, 34(4):46–58, 1991.
  * [25] Bo Shen, Ishwar K Sethi, and Vasudev Bhaskaran.  Dct convolution and its application in compressed domain.  IEEE transactions on circuits and systems for video technology, 8(8):947–952, 1998.
  * [26] Lu Chi, Borui Jiang, and Yadong Mu.  Fast fourier convolution.  Advances in Neural Information Processing Systems, 33:4479–4488, 2020.
  * [27] Umar Farooq Mohammad and Mohamed Almekkawy.  A substitution of convolutional layers by fft layers-a low computational cost version.  In 2021 IEEE International Ultrasonics Symposium (IUS), pages 1–3. IEEE, 2021.
  * [28] Pengju Liu, Hongzhi Zhang, Wei Lian, and Wangmeng Zuo.  Multi-level wavelet convolutional neural networks.  IEEE Access, 7:74973–74985, 2019.
  * [29] Edouard Oyallon, Eugene Belilovsky, Sergey Zagoruyko, and Michal Valko.  Compressing the input for cnns with the first-order scattering transform.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 301–316, 2018.
  * [30] Lionel Gueguen, Alex Sergeev, Ben Kadlec, Rosanne Liu, and Jason Yosinski.  Faster neural networks straight from jpeg.  Advances in Neural Information Processing Systems, 31, 2018.
  * [31] Samuel Felipe dos Santos, Nicu Sebe, and Jurandy Almeida.  The good, the bad, and the ugly: Neural networks straight from jpeg.  In 2020 IEEE International Conference on Image Processing (ICIP), pages 1896–1900. IEEE, 2020.
  * [32] Samuel Felipe dos Santos and Jurandy Almeida.  Less is more: Accelerating faster neural networks straight from jpeg.  In Iberoamerican Congress on Pattern Recognition, pages 237–247. Springer, 2021.
  * [33] Yuhao Xu and Hideki Nakayama.  Dct-based fast spectral convolution for deep convolutional neural networks.  In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2021.
  * [34] Matej Ulicny, Vladimir A Krylov, and Rozenn Dahyot.  Harmonic convolutional networks based on discrete cosine transform.  Pattern Recognition, 129:108707, 2022.
  * [35] David L Donoho.  De-noising by soft-thresholding.  IEEE transactions on information theory, 41(3):613–627, 1995.
  * [36] Oktay Karakuş, Igor Rizaev, and Alin Achim.  A simulation study to evaluate the performance of the cauchy proximal operator in despeckling sar images of the sea surface.  In IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium, pages 1568–1571. IEEE, 2020.
  * [37] Diaa Badawi, Agamyrat Agambayev, Sule Ozev, and A Enis Cetin.  Discrete cosine transform based causal convolutional neural network for drift compensation in chemical sensors.  In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8012–8016. IEEE, 2021.
  * [38] Hongyi Pan, Diaa Badawi, and Ahmet Enis Cetin.  Fast walsh-hadamard transform and smooth-thresholding based binary layers in deep neural networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4650–4659, 2021.
  * [39] Hongyi Pan, Diaa Badawi, and Ahmet Enis Cetin.  Block walsh-hadamard transform based binary layers in deep neural networks.  ACM Transactions on Embedded Computing Systems (TECS), 2022.
  * [40] Hongyi Pan, Diaa Badawi, Chang Chen, Adam Watts, Erdem Koyuncu, and Ahmet Enis Cetin.  Deep neural network with walsh-hadamard transform layer for ember detection during a wildfire.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 257–266, 2022.
  * [41] Nasir Ahmed, T_ Natarajan, and Kamisetty R Rao.  Discrete cosine transform.  IEEE transactions on Computers, 100(1):90–93, 1974.
  * [42] Gilbert Strang.  The discrete cosine transform.  SIAM review, 41(1):135–147, 1999.
  * [43] Martin Vetterli.  Fast 2-d discrete cosine transform.  In ICASSP’85. IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 10, pages 1538–1541. IEEE, 1985.
  * [44] Models and pre-trained weights.  https://pytorch.org/vision/stable/models.html, 2022.  Accessed: 2022-09-27.
  * [45] Deep residual networks official code.  https://github.com/KaimingHe/deep-residual-networks, 2015.  Accessed: 2022-09-29.
  * [46] Imagenet training in pytorch.  https://github.com/pytorch/examples/tree/main/imagenet, 2022.  Accessed: 2022-09-27.