[table]capposition=top

# Robustness to distribution shifts of compressed networks for edge devices

Lulan Shen, Ali Edalati, Brett Meyer, Warren Gross, James J. Clark
McGill University
Montreal, Quebec, Canada
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

It is necessary to develop efficient DNNs deployed on edge devices with
limited computation resources. However, the compressed networks often execute
new tasks in the target domain, which is different from the source domain
where the original network is trained. It is important to investigate the
robustness of compressed networks in two types of data distribution shifts:
domain shifts and adversarial perturbations. In this study, we discover that
compressed models are less robust to distribution shifts than their original
networks. Interestingly, larger networks are more vulnerable to losing
robustness than smaller ones, even when they are compressed to a similar size
as the smaller networks. Furthermore, compact networks obtained by knowledge
distillation are much more robust to distribution shifts than pruned networks.
Finally, post-training quantization is a reliable method for achieving
significant robustness to distribution shifts, and it outperforms both pruned
and distilled models in terms of robustness.

##  1 Introduction

Over the past few years, the field of deep learning has witnessed a remarkable
surge in the development of large-scale models [1, 2]. The deeper and wider
architectures of neural networks improve their performance but demand
considerable storage and computational resources, preventing them from being
deployed on edge devices [3, 4], _e.g_., mobile phones, smartwatches, IoT
devices, _etc_. This is because these devices often have limited resources in
terms of memory, processing power, and energy consumption, which makes it
challenging to run large models efficiently.

To overcome this challenge, various network compression methods have been
developed to reduce the memory footprint and computational complexity while
preserving the model’s accuracy. These techniques can be categorized into
different main approaches, including network pruning (_i.e_., weight pruning
[5], filter pruning [6, 7]), knowledge distillation (KD) [8], quantization [9,
10, 11, 12], low-rank approximation [13, 14, 15] and compact network design
[16, 17].

However, it is challenging to make compressed neural networks perform well in
the target domain. Typically, the original networks are trained on a source
domain, _e.g_., ImageNet dataset [18], but compressed networks execute new
tasks in a target domain, which has the same classes as the source domain but
lacks labeled training data. Therefore, there is always a distribution shift
between the source and target domains due to various factors, _e.g_., pose
positions, image quality, sensor noise, domain-specific characteristics,
_etc_. [19]. In this paper, we specifically define data distribution shifts as
situations where data samples used for training and testing are sourced from
different datasets or domains.

Iofinova _et al_. [20] investigated the transfer performance of pruned
convolutional neural networks (CNNs) by evaluating the image classification
accuracy of these networks in transfer learning tasks. Specifically, they
fine-tuned pre-trained and compressed models on ImageNet on different
downstream tasks, which contained classes different from those in ImageNet.
However, the study did not evaluate the performance of pruned CNNs or other
types of compressed models under distribution shifts between training and
application distributions with the same classes.

Further research is required to investigate the performance of compressed
models under distribution shifts, particularly when target domain data is
scarce and unlabeled. In such cases, it is challenging to train a standalone
network, which emphasizes the need to address distribution shifts to ensure
the model’s performance in the target domain. Furthermore, deploying neural
networks on edge devices is prevalent in real-life scenarios, and the effect
of distribution shifts on compressed neural networks cannot be ignored.
Therefore, it is crucial to study the effect of different compression
techniques on neural network robustness to distribution shifts, enabling
people to prioritize a compression technique that prioritizes model
robustness.

We empirically investigate the robustness of compressed object classification
networks to input data distribution shift, using three commonly-used network
compression techniques: filter pruning, KD, and post-training quantization.
Our objective is to identify if any of them have an advantage over the others
in terms of handling distribution shifts. This study examines two distribution
shifts: domain shifts due to input images acquired under varying environmental
conditions, and adversarial attacks, where input data is perturbed to alter
the network’s classification results.

Our findings reveal that the performance of compressed models in target
domains degrades as their compression ratio increases. Additionally,
compressing from a larger base model instead of a smaller one to achieve a
particular size of the compact model results in a less robust performance in
distribution shifts. Importantly, we conclude that post-training quantization
is the most effective compression technique for handling distribution shifts,
especially in the case of domain shifts. Particularly, when compressing the
model to one-quarter of its original size is sufficient, post-training
quantization proves to be a favorable approach.

To summarize, our contributions are:

  * •

Evaluating adversarial and domain shift robustness of compressed ResNets using
pruning, KD, and quantization on the Office-31 dataset.

  * •

Investigating the effect of compression rate on the robustness of pruned and
distilled models.

  * •

Evaluating the robustness of compressed models under a wide range of attacks
in different strengths.

##  2 Related Works

###  2.1 Model Compression

Network Pruning: The comprehensive literature reviews on compressing DNNs are
presented in [21, 22]. Weight pruning is a simple compression method that
removes redundant weights of neural networks [23], which converts a dense
network into a sparse one [5]. Filter pruning is a structured way to reduce
computation costs in CNNs without sparse networks or special hardware [24]
which is usually required for compressing and accelerating convolutional
layers [25]. In addition, filter pruning is faster and more efficient than
weight pruning by removing entire channels instead of pruning a single neuron
connection.

Knowledge Distillation: Knowledge distillation [8, 26, 27] is a popular
compression method where knowledge is transferred from a large “teacher” model
to a smaller “student” model. Mimicking the teacher’s performance, the
lightweight student model achieves competitive results in tasks like visual
recognition and natural language processing. The knowledge type, distillation
algorithm, and teacher-student architectures are crucial in this process. The
vanilla KD [8] uses the teacher model’s logits to train the student model with
a distillation loss, which aligns the predictions between the two models.

Quantization: Neural network quantization is a technique that can reduce the
model size and lower computation overheads by decreasing the precision of
parameter representation without sacrificing accuracy [28], _i.e_., converting
a 32-bit float precision to an 8-bit integer representation. Han _et al_. [14]
introduced an influential technique that combines pruning, quantization, and
Huffman coding to compress DNNs. Network binarization is another popular
quantization approach, which constrains the weights and activations of neural
networks to be binary values [9, 10, 11].

###  2.2 Adversarial Attacks

(a) Original: Speaker

(b) C&W: computer

(c) DeepFool: chair

(d) PGD: lamp

(e) No perturbation

(f) C&W perturbation

(g) DeepFool perturbation

(h) PGD perturbation

Figure 1: Subplots (a)-(d) show misclassified images of a speaker from the
Amazon domain by ResNet-50 under C&W, DeepFool, and PGD adversarial attacks.
Subplots (e)-(h) show the corresponding perturbations generated under attacks,
magnified by a factor of 500.

In recent years, there has been an increasing amount of research aimed at
developing techniques to deceive neural networks. These techniques, known as
adversarial attacks, involve making malicious yet subtle changes in the input
to fool the network (See Fig. 1) [29, 30, 31, 32, 33]. Alternately, various
techniques have been developed to enhance the robustness of neural networks
against such attacks [30, 34, 35, 36]. Also, several works have introduced
techniques to improve the adversarial robustness of compressed models by
combining adversarial training [30] and compression [37, 38, 39, 40].

Moreover, empirical and theoretical analyses presented in [41] suggested that
the sparsity and non-linearity of DNNs improve their adversarial robustness.
Later, Jordao _et al_. [42] empirically showed that pruning improves
adversarial robustness without investigating the effect of the pruning rate.
Shumailov _et al_. [43] studied the transferability of adversarially generated
data samples between compressed and uncompressed models in addition to the
adversarial robustness of the compressed models. However, they only
investigated pruning, quantization, and a few gradient-based adversarial
attacks.

By measuring the adversarial robustness of binarized neural networks, Galloway
_et al_. [44] revealed that quantization causes gradient masking [45], which
improves the robustness of quantized models against gradient-based attacks. In
addition, Lin _et al_. [46] concluded that the robustness of quantized models
is improved against small perturbations but reduced against large
perturbations. They also introduced defensive quantization to improve the
adversarial robustness of quantized models.

##  3 Methodology

###  3.1 Model Compression

We use three primary compression techniques to reduce the size of various
ResNets [47], discussed below.

####  3.1.1 Pruning

We apply $\mathcal{L}1$-norm Filter Pruning ($\mathcal{L}1$-FP) [24] for
pruning. This technique assumes filters or neurons with smaller weights are
less important in the model. Each filter, denoted as
$\mathcal{F}_{i\in[1,I]}$, has $J$ weight elements in a model with $I$
filters. The importance score for each filter is calculated using:

$\vspace{-5pt}Score(\mathcal{F}_{i})=||\mathcal{F}_{i}||=\sum_{j=1}^{J}|\mathcal{F}_{i,j}|.$
(1)

$\mathcal{L}1$-FP measures the sum of absolute weights for all filters in a
well-trained model and sorts the filters based on the relative importance
score of filters in each layer. The least important filters in each layer are
then pruned to achieve a desired compression ratio. Finally, the pruned model
is trained again on the source domain.

####  3.1.2 Knowledge Distillation

We use vanilla KD [8], where we train the large teacher model on the source
domain and compute the logits for each class using the softmax output layer,
as shown below:

$p_{i}(\tau)=\frac{e^{t_{i}/\tau}}{\sum_{j=1}^{N}e^{t_{j}/\tau}},\quad
q_{i}(\tau)=\frac{e^{s_{i}/\tau}}{\sum_{j=1}^{N}e^{s_{j}/\tau}},$ (2)

where $t_{i}$ and $s_{i}$ are the logits generated by the classification
layers of the teacher and student network, respectively, given a data sample
for the class $i$; $N$ is the number of classes; and $\tau$ is the temperature
parameter, which is set to 1 by default. The higher $\tau$ value softens the
probability distribution output over classes. The “soft targets” of the
teacher and student model are outputs of the softmax layers:
$\mathbf{p}=(p_{1},..,p_{N})$ and $\mathbf{q}=(q_{1},..,q_{N})$ respectively.
We use the class probabilities generated by the teacher model as “soft
targets” for student model training, transferring the generalization ability
of the larger teacher model to the compact student model.

The objective function is calculated as below:

$\vspace{-5pt}\mathcal{L}_{KD}=CrossEntropy(\mathbf{p},\mathbf{q})=-\sum_{i=1}^{N}{p_{i}^{\tau}\log
q_{i}^{\tau}},$ (3)
$\mathcal{L}_{CE}=CrossEntropy(\mathbf{y},\mathbf{q})=-\sum_{i=1}^{N}{y_{i}\log
q_{i}},$ (4)
$\mathcal{L}_{total}=\alpha\tau^{2}\mathcal{L}_{KD}+(1-\alpha)\mathcal{L}_{CE},$
(5)

where $\mathbf{y}$ is the one-hot vector of the ground-truth label, and
$\alpha$ is the weighting hyper-parameter. $\mathcal{L}_{CE}$ measures the
classification loss of the student model, while $\mathcal{L}_{KD}$ encourages
the student model’s predicted probabilities to match the teacher model’s
probabilities.

####  3.1.3 Quantization

We also apply network quantization, which reduces the precision of
computations and weight storage by using lower bit-widths instead of floating-
point precision. We choose post-training static quantization (PTSQ) [48],
which is one of the most common and fastest quantization techniques in
practice. This technique determines the scales and zero points prior to
inference. Specifically, we quantize the 32-bit weights (_e.g_.,
$w\in[\alpha,\beta]$) and activations of the trained baseline models to 8-bit
integer values (_e.g_., $w_{q}\in[\alpha_{q},\beta_{q}]$). The quantization
process is defined as

$w_{q}=\text{round}\left(\frac{1}{s}w+z\right),$ (6)

where $s$ is the scale, and $d$ is the zero-point, defined as

$s=\frac{\beta-\alpha}{\beta_{q}-\alpha_{q}},\quad
z=\text{round}\left(\frac{\beta\alpha_{q}-\alpha\beta_{q}}{\beta-\alpha}\right).$
(7)

###  3.2 Domain Shifts

In this study, we evaluate our approach on the Office-31 dataset [49] (shown
in Fig. 2). This dataset is a widely used benchmark for domain adaptation
research. It consists of 4,110 images of 31 object classes from an office
environment, divided into three image domains: Amazon ($\mathcal{D}_{A}$),
Webcam ($\mathcal{D}_{W}$), and DSLR ($\mathcal{D}_{D}$). The
$\mathcal{D}_{A}$ domain contains 2,871 medium-resolution images with studio
lighting conditions, downloaded from the Amazon website; the $\mathcal{D}_{W}$
domain has 795 low-resolution images with significant noise and white balance
artifacts, captured with a simple webcam; the $\mathcal{D}_{D}$ domain
contains 498 low-noise high-resolution images captured with a digital SLR
camera in realistic environments with natural lighting conditions [49]. Each
domain is split into 90% train data and 10% validation data using a random
seed of 1, and the model is trained on the train data. We report the
validation accuracy on the source domain and the testing accuracy on the
entire target domain. We perform six domain shifts in total, where we train on
one Office-31 domain and test on one of the other two domains.

Figure 2: Sample images of backpack, bike, and bookcase from Amazon, Webcam,
and DSLR domain of the Office-31 [49].

###  3.3 Adversarial Attacks

We evaluate the robustness of models against seven commonly used adversarial
attacks, briefly discussed below.

  * •

Fast Gradient Sign Method (FGSM): This method generates an adversarial sample,
$x_{adv}$, by adding the scaled gradient of the loss function
($\nabla_{x}J(x,y,\theta)$) to the input image ($x$) [29]. Equation 8 shows
how this method generates the adversarial data, where $y$ represents the image
label, $\theta$ represents the model parameters, and $\epsilon$ is the
perturbation strength.

$x_{adv}=x+\epsilon\text{sign}(\nabla_{x}J(x,y,\theta))$ (8)

  * •

Projected Gradient Decent (PGD): This method [30] is an iterative version of
FGSM. The input image $x_{t}$ is updated to $x_{t+1}$ using

$x_{t+1}=\prod_{x+S}(x_{t}+\delta\text{sign}(\nabla_{x}J(x,y,\theta))),$ (9)

where $\delta$ is the step size, and $S$ is the set of possible perturbations.

  * •

DeepFool: This method approximates the model with a linear classifier and
measures the minimum distance required to reach the decision boundary of each
class. Finally, the adversarial sample is obtained by adding the minimum
distance to the input image [33].

  * •

Decoupled Direction and Norm (DDN): This attack is an iterative approach that
refines the noise added to the input image in each iteration to make it
adversarial. At iteration $i$, the adversarial input image, $x_{i}$, is
generated as $x_{i}=x+\eta_{i}$, where $\eta_{i}$ is the noise with a norm of
$\sigma_{i}$. If $x_{i}$ is adversarial, the norm of the next iteration noise
is decreased ($\sigma_{i+1}=\sigma_{i}(1-\epsilon)$). Otherwise, the norm of
the next noise is increased ($\sigma_{i+1}=\sigma_{i}(1+\epsilon)$). This
process repeats until the minimum required perturbation is found [31].

  * •

Carlini and Wagner (C&W): This is considered one of the strongest adversarial
attacks as it aims to find the minimum perturbation needed to change the
predicted class of an input image. C&W reformulates the mentioned optimization
problem to find the adversarial sample by using an optimizer [32] like Adam
[50].

  * •

Elastic-Net Attacks to DNNs (EAD): This technique uses Iterative Shrinkage-
Thresholding Algorithm [51] to solve the same optimization problem that C&W is
trying to solve.

  * •

Salt&Pepper: A non-gradient-based attack that repeatedly adds Salt & Pepper
noise to the input to fool the model.

We evaluate the strength of adversarial attacks by varying the perturbation
strength ($\epsilon$) and assessing both light and heavy attacks based on the
accuracy drop of ResNet-18. All compressed models obtained through pruning and
KD are evaluated against all discussed attacks under both light and heavy
attacks, including gradient-based and non-gradient-based ones. However, the
PTSQ framework does not support backward path calculations, so we only
evaluate the quantized models against EAD, C&W, and Salt&Pepper.

##  4 Experimental Setup

We initialize ResNets [52] with various depths (_i.e_., ResNet-18, -34, -50,
-101, and -152) using pre-trained parameters on ImageNet [18] from PyTorch
[53]. To obtain our baseline models (_i.e_., baseline-A, baseline-W, and
baseline-D), we fine-tuned these models on each source domain (_i.e_.,
$\mathcal{D}_{A}$, $\mathcal{D}_{W}$, $\mathcal{D}_{D}$). The input images are
resized to $224\times 224$ and the same data transforms as the pre-trained
models are used. We use a stochastic gradient descent (SGD) optimizer [54]
with a momentum of 0.9, a fixed learning rate (LR) of 1e-4, a batch size of 8
for {$\mathcal{D}_{A},\mathcal{D}_{W}$} and of 16 for $\mathcal{D}_{D}$, and
trained for 100 epochs. We select the fine-tuned models with the best
validation accuracy as our baseline models. These baseline models are then
evaluated directly on the target domain.

To obtain compact models, we compress the baseline models using
$\mathcal{L}1$-FP pruning [24], KD [8], and PTSQ [48]. First, we apply
$\mathcal{L}1$-FP pruning to the baseline models, using a single sensitivity
hyper-parameter to control the pruning ratios in each ResNet block. The
compressed models are then re-trained from scratch on the source domain for
100 epochs. We use an SGD optimizer with a momentum of 0.9, a weight decay of
1e-4, and a batch size of 8. For ResNet-50 and ResNet-152, we used a fixed LR
of 1e-4, while for ResNet-18, ResNet-34, and ResNet-101, we used an adaptive
LR with a base LR of 1e-3, divided by 10 at epoch 70. For KD, we use the
larger baseline models (_i.e_., already fine-tuned on the source domain) as
our teacher models and the pre-trained smaller ResNet models (provided by
PyTorch) as our student models. We set $\alpha$ as 0.2 and $\tau$ as 10. We
train the student models for 100 epochs using an SGD optimizer with a momentum
of 0.9, a weight decay of 1e-3, and a batch size of 8. We use a step LR
scheduler, which divided the base LR by 5 at epoch 70. For PTSQ, we use the
built-in quantization modules provided by PyTorch, which allows us to fuse
modules, calibrate the model using the training data to determine the
appropriate scale factor, and finally quantize the weights and activation in
the model. We choose hyper-parameters for the pruned and distilled models that
maximize their validation accuracy.

To evaluate the performance of compressed models and baselines against
adversarial attacks, we use the fine-tuned checkpoints on Amazon (_i.e_.,
ResNets baseline-A and its compressed models). Then we use
Foolox111https://github.com/bethgelab/foolbox [55, 56] to generate an
adversarial test dataset for each model, using the original test dataset
obtained by splitting 0.1 of randomly selected data samples from
$\mathcal{D}_{A}$. The batch size for the adversarial experiments is set to 8.
We use default hyper-parameters for most of the attacks, except for EAD, C&W,
and Salt&Pepper, which require 100, 100, and 400 steps, respectively. Also, we
set the step size of C&W to 0.001.

We implement our models and experiments using PyTorch and run them on
V100LGPU. Hyper-parameters are chosen using a basic grid search that tunes
batch size, base LR, step size, and gamma for the step LR scheduler, weight
decay of the SGD optimizer, $\alpha$, and $\tau$ for KD.

##  5 Results and discussion

###  5.1 Domain Shifts

Table 1: The validation accuracies (%) of ResNets models on each source domain of the Office-31 dataset. The baseline (uncompressed) ResNets are obtained after fine-tuning the pre-trained model on the ImageNet dataset. The pruned models are obtained using the $\mathcal{L}1$-FP method with different pruning ratios, and the distilled/student models are obtained using teacher networks of different sizes. Base/Teacher Model | Compression Method | # Params (M) | Compression Rate (%) | Amazon | Webcam | DSLR | Avg Acc
---|---|---|---|---|---|---|---
ResNet-18 | - | 11.19 | - | 89.32 | 89.87 | 93.88 | 91.02
FP | 4.51 | 59.70 | 89.68 | 87.34 | 87.76 | 88.26
ResNet-34 | - | 21.30 | - | 90.75 | 89.87 | 93.88 | 91.50
FP | 11.19 | 47.46 | 85.41 | 89.87 | 93.88 | 89.72
FP | 4.51 | 78.83 | 75.44 | 78.48 | 79.59 | 77.84
KD | ResNet18 (11.19) | 47.46 | 90.04 | 91.14 | 97.96 | 93.05
ResNet-50 | - | 23.57 | - | 90.04 | 89.87 | 97.96 | 92.62
FP | 21.30 | 9.63 | 92.17 | 89.87 | 97.96 | 93.33
FP | 11.19 | 52.52 | 87.90 | 86.08 | 93.88 | 89.29
FP | 4.51 | 80.87 | 75.44 | 83.54 | 81.63 | 80.20
KD | ResNet34 (21.30) | 9.63 | 90.04 | 89.87 | 97.96 | 92.62
KD | ResNet18 (11.19) | 52.52 | 90.39 | 91.14 | 97.96 | 93.16
ResNet-101 | - | 42.56 | - | 91.81 | 91.14 | 97.96 | 93.64
FP | 23.57 | 44.62 | 90.04 | 89.87 | 97.96 | 92.62
FP | 11.19 | 73.71 | 86.12 | 88.61 | 97.96 | 90.90
FP | 4.51 | 89.40 | 72.95 | 84.81 | 77.55 | 78.44
ResNet-152 | - | 58.21 | - | 90.75 | 87.34 | 95.92 | 91.34
FP | 42.56 | 26.89 | 91.46 | 91.14 | 95.92 | 92.84
FP | 23.57 | 59.51 | 89.68 | 88.61 | 97.96 | 92.08
FP | 11.19 | 80.78 | 82.56 | 88.61 | 95.92 | 89.03
FP | 4.51 | 92.25 | 68.68 | 81.01 | 85.71 | 78.47
KD | ResNet18 (11.19) | 80.78 | 90.04 | 91.14 | 95.92 | 92.37
Table 2: The test accuracies (%) of ResNets on target domains of the Office-31 dataset other than what they were trained on. Base/Teacher Model | Compression Method | # Params (M) |  A $\rightarrow$ W |  A $\rightarrow$ D |  W $\rightarrow$ A |  W $\rightarrow$ D |  D $\rightarrow$ A |  D $\rightarrow$ W | Avg Acc
---|---|---|---|---|---|---|---|---|---
ResNet-18 | - | 11.19 | 64.15 | 60.84 | 50.05 | 97.39 | 47.43 | 92.70 | 68.76
FP | 4.51 | 57.36 | 59.64 | 47.39 | 96.99 | 44.73 | 91.32 | 66.24
ResNet-34 | - | 21.30 | 67.80 | 68.27 | 54.67 | 98.59 | 52.01 | 92.70 | 72.34
FP | 11.19 | 48.93 | 51.61 | 46.22 | 95.78 | 43.66 | 89.69 | 62.65
FP | 4.51 | 16.73 | 14.06 | 16.68 | 63.86 | 10.47 | 57.36 | 29.86
KD | 11.19 | 62.52 | 63.25 | 48.49 | 99.00 | 52.79 | 95.47 | 70.25
ResNet-50 | - | 23.57 | 68.93 | 71.89 | 64.25 | 99.00 | 60.95 | 94.84 | 76.64
FP | 21.30 | 69.43 | 72.69 | 61.52 | 98.39 | 59.53 | 94.34 | 75.98
FP | 11.19 | 53.46 | 61.45 | 41.36 | 93.57 | 35.64 | 85.28 | 61.79
FP | 4.51 | 19.37 | 26.10 | 11.89 | 50.00 | 9.23 | 34.72 | 25.22
KD | 21.30 | 57.74 | 59.04 | 56.59 | 98.80 | 58.18 | 97.11 | 71.24
KD | 11.19 | 57.99 | 57.63 | 52.18 | 99.40 | 48.78 | 95.47 | 68.58
ResNet-101 | - | 42.56 | 74.59 | 75.70 | 64.32 | 99.00 | 24.35 | 90.19 | 71.36
FP | 23.57 | 62.89 | 63.45 | 54.10 | 98.59 | 55.80 | 91.19 | 71.00
FP | 11.19 | 43.02 | 40.16 | 28.19 | 90.36 | 19.63 | 84.65 | 51.00
FP | 4.51 | 19.37 | 17.27 | 12.11 | 57.83 | 5.18 | 32.58 | 24.10
ResNet-152 | - | 58.21 | 74.97 | 74.70 | 63.97 | 98.39 | 63.76 | 95.72 | 78.59
FP | 42.56 | 71.19 | 72.69 | 61.95 | 99.00 | 61.59 | 93.46 | 76.65
FP | 23.57 | 65.41 | 66.47 | 55.13 | 99.00 | 43.02 | 86.79 | 69.30
FP | 11.19 | 36.23 | 39.36 | 25.49 | 81.93 | 10.69 | 57.11 | 41.80
FP | 4.51 | 17.86 | 15.26 | 9.83 | 50.00 | 4.40 | 23.02 | 20.06
KD | 11.19 | 58.49 | 62.25 | 48.49 | 98.39 | 50.80 | 95.85 | 69.05
Table 3: Accuracies of ResNet baselines and quantized 8-bit ResNets trained on the Amazon domain and accuracies of quantized ResNets for light and heavy adversarial perturbations. Note that a “-” in the compression method column indicates the baseline model is used. Base Model |  Compression Method |  Memory Size (MB) |  Source (A) |  Target (W) |  Target (D) | EAD |  C&W${}_{L_{2}}$ | Salt&Pepper
---|---|---|---|---|---|---|---|---
$\epsilon=1$ | $\epsilon=10$ | $\epsilon=0.4$ | $\epsilon=4$ | $\epsilon=1$ | $\epsilon=15$
ResNet-18 | - | 42.79 | 88.61 | 63.77 | 66.87 | 83.99 | 50.89 | 14.23 | 0.00 | 85.77 | 31.67
PTSQ | 10.81 | 88.26 | 63.02 | 65.26 | 87.90 | 87.90 | 87.90 | 87.90 | 85.77 | 44.84
ResNet-34 | - | 81.42 | 90.39 | 64.28 | 68.27 | 81.49 | 49.47 | 14.95 | 0.71 | 84.70 | 50.89
PTSQ | 20.55 | 89.32 | 64.28 | 67.47 | 89.32 | 88.97 | 88.97 | 88.97 | 87.54 | 46.98
ResNet-50 | - | 90.26 | 90.04 | 72.45 | 75.30 | 84.34 | 48.04 | 13.17 | 0.00 | 87.19 | 42.35
PTSQ | 23.26 | 87.19 | 71.45 | 72.69 | 87.90 | 88.26 | 87.90 | 87.54 | 86.83 | 58.01
ResNet-101 | - | 163.03 | 90.39 | 75.09 | 76.71 | 83.99 | 57.30 | 12.10 | 00.36 | 87.90 | 64.41
PTSQ | 42.17 | 89.68 | 74.21 | 74.50 | 87.19 | 87.54 | 88.26 | 86.48 | 82.92 | 61.92
ResNet-152 | - | 223.01 | 90.75 | 74.84 | 76.91 | 87.54 | 61.92 | 15.30 | 00.36 | 89.68 | 65.12
PTSQ | 57.81 | 88.26 | 73.46 | 75.90 | 88.97 | 88.97 | 89.32 | 88.61 | 86.12 | 58.72

The ResNet baselines are achieved after fine-tuning ResNets pre-trained models
on the corresponding source domain. Then, we compress the baseline models
using the $\mathcal{L}1$-FP with different pruning rates and distill the
baselines to different sizes of student models using vanilla KD. Table 1
records the validation accuracies of various baselines and compressed ResNets
to the training distribution, which is one of the three domains in the
Office-31 dataset. The results show that the baseline models perform well in
the source domain they were trained on. However, as we compress the model with
the filter pruning technique, the validation accuracy of the compressed model
in the source domain gets worse as the pruning ratio increases. In contrast,
the distilled models (_i.e_., the ResNet-18 student model) generally have
better training performance than the model with the same structure and size
(_i.e_., the ResNet-18 baseline model). This is expected since the larger
teacher model transfers its generalization ability to the student model.

Next, we evaluate the baseline and compressed/distilled models on the target
domains and report their accuracies in Tab. 2. The results show that both
baseline and compressed models of ResNets perform poorly on the target domains
and their performance decreases as the compression rate increases, indicating
a lack of robustness to domain shifts. However, after pruning their ResNet
baseline models to a particular size, the smaller baseline model has a higher
test accuracy on the target domains than the larger baseline networks, which
are less affected by domain shifts. For example, the compressed ResNet with
4.51M parameters pruned from ResNet-18 has an average accuracy of 66.24% on
target domains, while the pruned models with the same size obtained from the
base models ResNet-34, ResNet-50, ResNet-101, and ResNet-152 have average
accuracies of 29.86%, 25.22%, 24.10%, and 20.06% respectively. Similar
patterns are observed for other compressed network sizes. This suggests that
it is more beneficial to prune a smaller base model rather than a larger one
to a particular size since its corresponding compressed model has better
generalization performance on unseen domains.

However, the situation differs for networks compressed through KD. In this
case, the student models obtained using KD can possess similar or even better
performance in target domains compared to the small baseline network of the
same size. For instance, consider the networks with a parameter count of
11.19M. The average accuracy for the unpruned baseline is 68.76%. However, as
shown in Tab. 2, the accuracy of student models with 11.19M is generally
superior to that of the small baseline network, and the decline in performance
when using larger teachers is minimal.

Furthermore, Tab. 3 demonstrates the performance of quantized 8-bit ResNets
trained on the Amazon domain. The quantization process reduces the size of the
model to one-quarter of its original size without any significant loss or
degradation of accuracy in target domains. On average, quantized ResNet
models, compressed to 1/4 of their original size, experience only a slight
decrease in their target-domain accuracy of about 1.2%. In contrast, models
pruned and distilled from baseline-A, even at a relatively modest pruning
ratio of 50% (_i.e_., compressing from ResNet-50 to a model with 11.19M
parameters), experience a much larger decrease in target-domain accuracy of
approximately 13%. This suggests that quantized models exhibit significantly
greater robustness to domain shifts compared to pruned and distilled models.
However, further research should investigate the robustness of quantized
4-bit/2-bit integer neural networks to better understand their potential
benefits.

Taking into account the robustness to domain shift, it can be concluded that
distillation is a better approach to train a network of a specific size than
using pruning to obtain a smaller network from a larger one. Furthermore, the
post-training quantization method is highly recommended over network pruning
and KD techniques if compressing three-quarters of the model size is adequate
for deployment, and no further training is required. This is due to the
superior robustness of quantized models in dealing with domain shifts.

###  5.2 Adversarial Attacks

Table 4: Accuracies of both baseline-A and compressed ResNets under light
adversarial perturbations.

| Base/Teacher
---
Model
| Compression
---
Method
| # Params
---
(M)
| In-Domain
---
$\mathcal{D}_{A}$ Acc ($\%$)
| DeepFool${}_{L_{\infty}}$
---
($\epsilon=0.0003$)
| PGD${}_{L_{\infty}}$
---
($\epsilon=0.0003$)
| FGSM
---
($\epsilon=0.0003$)
| C&W${}_{L_{2}}$
---
($\epsilon=0.03$)
| DDN
---
($\epsilon=0.003$)
| EAD
---
($\epsilon=1$)
| Salt&Pepper
---
($\epsilon=1$)
| Avg Adversarial
---
Acc ($\%$)
ResNet-18 | - | 11.19 | 89.32 | 77.93 | 78.29 | 79.00 | 84.34 | 88.61 | 83.99 | 85.77 | 82.56
FP | 4.51 | 89.68 | 82.21 | 82.56 | 83.63 | 86.48 | 87.54 | 85.77 | 85.77 | 84.85
ResNet-34 | - | 21.30 | 90.75 | 76.87 | 76.51 | 78.29 | 83.63 | 89.32 | 81.49 | 84.70 | 81.54
FP | 11.19 | 85.41 | 82.21 | 82.21 | 82.21 | 83.95 | 85.41 | 82.92 | 81.85 | 82.96
FP | 4.51 | 75.44 | 71.17 | 71.17 | 71.17 | 73.31 | 75.44 | 71.17 | 70.82 | 72.03
KD | 11.19 | 90.04 | 87.90 | 87.90 | 88.26 | 89.32 | 90.39 | 87.90 | 86.83 | 88.36
ResNet-50 | - | 23.57 | 90.04 | 81.85 | 80.78 | 83.27 | 86.83 | 90.03 | 84.34 | 87.19 | 84.90
FP | 21.30 | 92.17 | 85.05 | 84.34 | 85.41 | 87.19 | 91.46 | 86.12 | 89.32 | 86.98
FP | 11.19 | 87.90 | 66.90 | 64.06 | 72.60 | 75.80 | 86.83 | 75.80 | 82.92 | 74.99
FP | 4.51 | 75.44 | 49.11 | 45.20 | 54.80 | 63.34 | 74.02 | 59.07 | 68.68 | 59.17
KD | 21.30 | 90.04 | 87.19 | 87.19 | 87.19 | 88.97 | 90.03 | 87.54 | 88.61 | 88.10
KD | 11.19 | 90.39 | 87.54 | 87.54 | 87.54 | 89.32 | 90.75 | 88.26 | 87.90 | 88.41
ResNet-101 | - | 42.56 | 91.81 | 80.07 | 79.00 | 80.78 | 85.05 | 91.10 | 83.99 | 87.90 | 83.98
FP | 23.57 | 90.04 | 86.83 | 86.48 | 87.19 | 87.90 | 90.03 | 86.48 | 87.90 | 87.54
FP | 11.19 | 86.12 | 81.49 | 81.14 | 83.27 | 84.70 | 86.12 | 82.92 | 83.27 | 83.27
FP | 4.51 | 72.95 | 60.14 | 60.14 | 60.85 | 65.12 | 71.53 | 60.14 | 63.70 | 63.09
ResNet-152 | - | 58.21 | 90.75 | 82.92 | 81.85 | 84.70 | 88.97 | 91.10 | 87.54 | 89.68 | 86.68
FP | 42.56 | 91.46 | 81.85 | 80.43 | 83.98 | 85.76 | 91.46 | 85.41 | 88.61 | 85.36
FP | 23.57 | 89.68 | 77.93 | 75.09 | 79.71 | 82.56 | 89.32 | 82.92 | 86.12 | 81.95
FP | 11.19 | 82.56 | 60.14 | 57.29 | 63.70 | 68.32 | 81.49 | 67.97 | 72.95 | 67.41
KD | 11.19 | 90.04 | 85.76 | 85.76 | 85.76 | 87.90 | 89.68 | 87.19 | 85.77 | 86.83

Table 5: Accuracies of both baseline-A and compressed ResNets under heavy
adversarial perturbations.

| Base/Teacher
---
Model
| Compression
---
Method
| # Params
---
(M)
| In-Domain
---
$\mathcal{D}_{A}$ Acc ($\%$)
| DeepFool${}_{L_{\infty}}$
---
($\epsilon=0.004$)
| PGD${}_{L_{\infty}}$
---
($\epsilon=0.004$)
| FGSM
---
($\epsilon=0.004$)
| C&W${}_{L_{2}}$
---
($\epsilon=0.4$)
| DDN
---
($\epsilon=0.24$)
| EAD
---
($\epsilon=10$)
| Salt&Pepper
---
($\epsilon=15$)
| Avg Adversarial
---
Acc ($\%$)
ResNet-18 | - | 11.19 | 89.32 | 2.13 | 1.78 | 34.52 | 14.23 | 20.64 | 50.89 | 31.67 | 22.27
FP | 4.51 | 89.68 | 20.28 | 11.73 | 49.47 | 38.43 | 48.75 | 62.28 | 16.73 | 35.38
ResNet-34 | - | 21.30 | 90.75 | 3.91 | 0.36 | 47.69 | 14.95 | 18.51 | 49.47 | 50.89 | 26.54
FP | 11.19 | 85.41 | 40.57 | 32.38 | 58.01 | 48.75 | 58.36 | 64.41 | 40.93 | 49.06
FP | 4.51 | 75.44 | 30.60 | 21.71 | 46.95 | 40.21 | 49.82 | 48.04 | 21.71 | 37.01
KD | 11.19 | 90.04 | 66.90 | 61.92 | 75.44 | 70.82 | 76.16 | 75.80 | 61.57 | 69.80
ResNet-50 | - | 23.57 | 90.04 | 0.36 | 0.00 | 45.51 | 13.17 | 6.05 | 48.04 | 42.35 | 22.21
FP | 21.30 | 92.17 | 6.76 | 1.78 | 53.74 | 22.06 | 26.33 | 57.30 | 55.16 | 31.88
FP | 11.19 | 87.90 | 0.71 | 0.35 | 32.38 | 4.63 | 1.78 | 37.37 | 40.57 | 16.83
FP | 4.51 | 75.44 | 0.00 | 0.00 | 9.25 | 0.71 | 0.00 | 17.08 | 12.46 | 5.64
KD | 21.30 | 90.04 | 69.03 | 64.06 | 73.31 | 70.82 | 77.58 | 76.87 | 66.55 | 71.17
KD | 11.19 | 90.39 | 62.63 | 58.72 | 71.89 | 67.97 | 75.80 | 75.44 | 49.11 | 65.94
ResNet-101 | - | 42.56 | 91.81 | 6.05 | 1.78 | 59.07 | 12.10 | 11.39 | 57.30 | 64.41 | 30.30
FP | 23.57 | 90.04 | 41.28 | 27.76 | 66.19 | 53.38 | 62.28 | 70.46 | 56.23 | 53.94
FP | 11.19 | 86.12 | 28.47 | 17.44 | 49.82 | 43.42 | 53.38 | 56.94 | 27.40 | 39.55
FP | 4.51 | 72.95 | 2.13 | 0.71 | 13.52 | 10.68 | 12.81 | 19.93 | 6.05 | 9.40
ResNet-152 | - | 58.21 | 90.75 | 7.12 | 4.98 | 63.34 | 15.30 | 12.46 | 61.92 | 65.12 | 32.89
FP | 42.56 | 91.46 | 5.69 | 1.07 | 61.56 | 9.96 | 7.12 | 56.23 | 52.67 | 27.76
FP | 23.57 | 89.68 | 1.42 | 0.71 | 55.87 | 7.83 | 5.34 | 50.53 | 35.59 | 22.47
FP | 11.19 | 82.56 | 0.35 | 0.00 | 29.54 | 4.27 | 1.07 | 22.42 | 7.12 | 9.25
KD | 11.19 | 90.04 | 69.75 | 65.48 | 75.80 | 72.95 | 78.65 | 77.94 | 53.74 | 70.62

The accuracy of the quantized ResNets and their corresponding baselines
against EAD, C&W, and Salt&Pepper adversarial attacks are demonstrated in Tab.
3. Our results indicate that while the gradient-based attacks, EAD and C&W,
significantly degrade the robustness of baselines, these attacks barely reduce
the accuracy of the quantized models. For example, the accuracy of the
uncompressed ResNet-18 drops significantly from 88.61% on the clean data to
50.89% and 0.00% under heavy EAD and C&W attacks, respectively. However, the
accuracy of the quantized ResNet-18 is reduced from 88.26% on the clean data
to 87.90% under the gradient-based attacks. This observation shows the
comparative robustness of quantized models against gradient-based attacks due
to the phenomenon of gradient masking [44, 45, 24], which hinders the
attackers’ ability to find gradients that can mislead the quantized model.
However, adversarial attacks can be specifically designed to overcome gradient
masking [57] and challenge the quantized models.

While our experiments in Tab. 3 revealed that quantized models are vulnerable
to non-gradient-based attacks such as Salt&Pepper, we also observed that
quantization improves the accuracy of smaller models (_i.e_., ResNet-18 and
ResNet-50) compared to their baselines under heavy Salt&Pepper attacks.
However, deeper models such as ResNet-152 experienced a reduction in accuracy
after quantization.

We also investigate the robustness of models that are compressed using pruning
and KD against light and heavy adversarial attacks, and the results are
presented in Tabs. 4 and 5. In contrast to the quantized models, the pruned
and distilled models are vulnerable to all types of attacks. Comparing the
accuracy of the pruned models with their corresponding baselines, we observe
that, with the exception of ResNet-152, the pruned models with the lowest
pruning rate outperform the baselines under both light and heavy attacks.
However, the larger size of ResNet-152 and, consequently, the larger amount of
pruned parameters, may explain why pruning does not improve the adversarial
robustness of this model. We also notice that the average adversarial
robustness decreases by increasing the pruning rate.

Furthermore, as shown in both Tabs. 4 and 5, the compressed models that use KD
consistently outperform both their baselines/teacher models and pruned models
of the same size with a significant performance gap between 20% and 60% under
heavy attacks. Moreover, the adversarial robustness of a ResNet-18, trained by
distillation from teachers with different sizes (_i.e_., ResNet-34, ResNet-50,
and ResNet-152) remains roughly the same against both light and heavy attacks.
This suggests that, unlike pruning, KD can maintain the model’s adversarial
robustness across different compression rates.

A comparison between the quantized models and pruned/distilled ones is not
quite fair since they have different architectures. However, according to our
results in Sec. 5.1, quantized models performed the best on domain shift
experiments. Also, according to the accuracy of compressed models under
Salt&Pepper, quantized models achieved the best results.

##  6 Conclusion

This paper explores the robustness of compressed networks to various
distribution shifts using the Office-31 dataset. We observe that the
compressed models perform worse in the unseen domain as the compression ratio
increases due to distribution shifts. The results indicate that as the
compression ratio increases, the compressed models perform more poorly in the
unseen domain due to distribution shifts. Furthermore, we discover that
compressed networks originating from smaller models demonstrate better
generalization abilities in the target domain, indicating that they are more
robust to distribution shifts compared to networks that were originally as
large.

In terms of compression techniques for neural networks, the pruning technique
is known for generating highly sensitive compressed networks that are
vulnerable to domain shifts and adversarial perturbations. On the other hand,
compact networks produced through knowledge distillation are less affected by
these issues. It is worth emphasizing that quantized networks, which are
compressed to approximately 25% of their original size, offer significantly
more robustness to distribution shifts, particularly in the case of domain
shifts, than other compressed networks.

## References

  * [1] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in _International Conference on Learning Representations_ , 2015.
  * [2] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 40, no. 04, pp. 834–848, 2018.
  * [3] H. Chen, Y. Wang, C. Xu, B. Shi, C. Xu, Q. Tian _et al._ , “AdderNet: Do we really need multiplications in deep learning?” in _IEEE Conference on Computer Vision and Pattern Recognition_. IEEE Computer Society, 2020, pp. 1465–1474.
  * [4] Y. He, P. Liu, Z. Wang, Z. Hu, and Y. Yang, “Filter pruning via geometric median for deep convolutional neural networks acceleration,” in _IEEE Conference on Computer Vision and Pattern Recognition_. IEEE Computer Society, 2019, pp. 4335–4344.
  * [5] S. Han, J. Pool, J. Tran, and W. J. Dally, “Learning both weights and connections for efficient neural networks,” in _International Conference on Neural Information Processing Systems_. MIT Press, 2015, p. 1135–1143.
  * [6] P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning convolutional neural networks for resource efficient inference,” in _International Conference on Learning Representations_ , 2017.
  * [7] N. Yu, S. Qiu, X. Hu, and J. Li, “Accelerating convolutional neural networks by group-wise 2D-filter pruning,” in _International Joint Conference on Neural Networks_ , 2017, pp. 2502–2509.
  * [8] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” in _Advances in Neural Information Processing Systems Workshop_ , 2014.
  * [9] M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, “XNOR-Net: ImageNet classification using binary convolutional neural networks,” in _Computer Vision – ECCV 2016_. Springer International Publishing, 2016, p. 525–542.
  * [10] H. Qin, R. Gong, X. Liu, X. Bai, J. Song, and N. Sebe, “Binary neural networks: A survey,” _Pattern Recognition_ , vol. 105, p. 107281, 2020.
  * [11] I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Binarized neural networks,” in _Advances in Neural Information Processing Systems_ , vol. 29. Curran Associates, Inc., 2016.
  * [12] M. Courbariaux, Y. Bengio, and J.-P. David, “BinaryConnect: Training deep neural networks with binary weights during propagations,” in _International Conference on Neural Information Processing Systems_. MIT Press, 2015, p. 3123–3131.
  * [13] F. N. Iandola, M. W. Moskewicz, K. Ashraf, S. Han, W. J. Dally, and K. Keutzer, “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size,” _CoRR_ , vol. abs/1602.07360, 2016.
  * [14] S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,” in _International Conference on Learning Representations_ , 2016.
  * [15] E. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, “Exploiting linear structure within convolutional networks for efficient evaluation,” in _International Conference on Neural Information Processing Systems_. MIT Press, 2014, p. 1269–1277.
  * [16] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition_. IEEE Computer Society, 2017, pp. 1800–1807.
  * [17] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand _et al._ , “MobileNets: Efficient convolutional neural networks for mobile vision applications,” _CoRR_ , vol. abs/1704.04861, 2017.
  * [18] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2009, pp. 248–255.
  * [19] M. Wang and W. Deng, “Deep visual domain adaptation: A survey,” _Neurocomputing_ , vol. 312, pp. 135–153, 2018.
  * [20] E. Iofinova, A. Peste, M. Kurtz, and D. Alistarh, “How well do sparse ImageNet models transfer?” in _Conference on Computer Vision and Pattern Recognition_. IEEE Computer Society, Jun. 2022, pp. 12 256–12 266.
  * [21] R. Mishra, H. Prabhat Gupta, and T. Dutta, “A survey on deep neural network compression: Challenges, overview, and solutions,” _arXiv e-prints_ , p. arXiv:2010.03954, 2020.
  * [22] Y. Cheng, D. Wang, P. Zhou, and T. Zhang, “A survey of model compression and acceleration for deep neural networks,” _arXiv e-prints_ , p. arXiv:1710.09282, 2017.
  * [23] F. Meng, H. Cheng, K. Li, H. Luo, X. Guo, G. Lu _et al._ , “Pruning filter in filter,” in _Advances in Neural Information Processing Systems_ , vol. 33. Curran Associates, Inc., 2020, pp. 17 629–17 640.
  * [24] H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,” in _International Conference on Learning Representations_ , 2017.
  * [25] S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz _et al._ , “EIE: Efficient inference engine on compressed deep neural network,” in _International Symposium on Computer Architecture_. IEEE Press, 2016, p. 243–254.
  * [26] X. Suau, u. Zappella, and N. Apostoloff, “Filter distillation for network compression,” in _IEEE Winter Conference on Applications of Computer Vision_ , 2020, pp. 3129–3138.
  * [27] J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” _International Journal of Computer Vision_ , vol. 129, no. 6, p. 1789–1819, 2021.
  * [28] A. Gholami, S. Kim, D. Zhen, Z. Yao, M. Mahoney, and K. Keutzer, _Low-Power Computer Vision_. Chapman and Hall/CRC, Jan. 2022, ch. A Survey of Quantization Methods for Efficient Neural Network Inference, pp. 291–326.
  * [29] I. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in _International Conference on Learning Representations_ , 2015.
  * [30] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in _International Conference on Learning Representations_ , 2018.
  * [31] J. Rony, L. G. Hafemann, L. S. Oliveira, I. B. Ayed, R. Sabourin, and E. Granger, “Decoupling direction and norm for efficient gradient-based l2 adversarial attacks and defenses,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 4322–4330.
  * [32] N. Carlini and D. Wagner, “Adversarial examples are not easily detected: Bypassing ten detection methods,” in _ACM workshop on artificial intelligence and security_ , 2017, pp. 3–14.
  * [33] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “Deepfool: A simple and accurate method to fool deep neural networks,” in _IEEE/CVF conference on computer vision and pattern recognition_ , 2016, pp. 2574–2582.
  * [34] C. Qin, J. Martens, S. Gowal, D. Krishnan, K. Dvijotham, A. Fawzi _et al._ , “Adversarial robustness through local linearization,” _Advances in Neural Information Processing Systems_ , vol. 32, 2019.
  * [35] P. Bashivan, R. Bayat, A. Ibrahim, K. Ahuja, M. Faramarzi, T. Laleh _et al._ , “Adversarial feature desensitization,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 10 665–10 677, 2021.
  * [36] H. Zhang, H. Chen, C. Xiao, S. Gowal, R. Stanforth, B. Li _et al._ , “Towards stable and efficient training of verifiably robust neural networks,” in _International Conference on Learning Representations_ , 2020.
  * [37] M. Goldblum, L. Fowl, S. Feizi, and T. Goldstein, “Adversarially robust distillation,” _AAAI Conference on Artificial Intelligence_ , vol. 34, no. 04, pp. 3996–4003, Apr. 2020.
  * [38] S. Ye, K. Xu, S. Liu, H. Cheng, J.-H. Lambrechts, H. Zhang _et al._ , “Adversarial robustness vs. model compression, or both?” in _IEEE/CVF International Conference on Computer Vision_ , Oct. 2019, pp. 111–120.
  * [39] S. Gui, H. Wang, H. Yang, C. Yu, Z. Wang, and J. Liu, “Model compression with adversarial robustness: A unified optimization framework,” in _Advances in Neural Information Processing Systems_ , vol. 32. Curran Associates, Inc., 2019.
  * [40] A. W. Wijayanto, J. J. Choong, K. Madhawa, and T. Murata, “Towards robust compressed convolutional neural networks,” in _IEEE International Conference on Big Data and Smart Computing_ , 2019, pp. 1–8.
  * [41] Y. Guo, C. Zhang, C. Zhang, and Y. Chen, “Sparse dnns with improved adversarial robustness,” in _Advances in Neural Information Processing Systems_ , vol. 31. Curran Associates, Inc., 2018.
  * [42] A. Jordão and H. Pedrini, “On the effect of pruning on adversarial robustness,” in _IEEE/CVF International Conference on Computer Vision Workshop_ , Oct. 2021, pp. 1–11.
  * [43] I. Shumailov, Y. Zhao, R. Mullins, and R. Anderson, “To compress or not to compress: Understanding the interactions between adversarial attacks and neural network compression,” _Machine Learning and Systems_ , vol. 1, pp. 230–240, 2019.
  * [44] A. Galloway, G. W. Taylor, and M. Moussa, “Attacking binarized neural networks,” in _International Conference on Learning Representations_ , 2018.
  * [45] N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami, “Practical black-box attacks against machine learning,” in _ACM on Asia conference on computer and communications security_ , 2017, pp. 506–519.
  * [46] J. Lin, C. Gan, and S. Han, “Defensive quantization: When efficiency meets robustness,” in _International Conference on Learning Representations_ , 2019.
  * [47] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [48] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 2704–2713.
  * [49] K. Saenko, B. Kulis, M. Fritz, and T. Darrell, “Adapting visual category models to new domains,” in _European Conference on Computer Vision_ , 2010.
  * [50] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in _International Conference on Learning Representations_ , 2015.
  * [51] A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” _SIAM Journal on Imaging Sciences_ , vol. 2, pp. 183–202, 2009.
  * [52] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 770–778.
  * [53] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan _et al._ , “PyTorch: An imperative style, high-performance deep learning library,” in _Advances in Neural Information Processing Systems_ , vol. 32. Curran Associates, Inc., 2019.
  * [54] L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in _Computational Statistics_. Physica-Verlag HD, 2010, pp. 177–186.
  * [55] J. Rauber, W. Brendel, and M. Bethge, “Foolbox: A python toolbox to benchmark the robustness of machine learning models,” in _International Conference on Machine Learning Workshop_ , 2017.
  * [56] J. Rauber, R. Zimmermann, M. Bethge, and W. Brendel, “Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX,” _Journal of Open Source Software_ , vol. 5, no. 53, p. 2607, 2020.
  * [57] A. Athalye, N. Carlini, and D. Wagner, “Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples,” in _International conference on machine learning_. PMLR, 2018, pp. 274–283.


Arbitrary unitary rotation of
three-dimensional pixellated images
Alejandro R. Urzúa and Kurt Bernardo Wolf\star\star$\star$Deceased May 25,
2022

Instituto de Ciencias Físicas

Universidad Nacional Autónoma de México

Av. Universidad s/n, Cuernavaca, Morelos 62251, México

###### Abstract

Using the coefficients introduced by Bargmann and Moshinsky for the reduction
of the su($3$) algebra of Cartesian three-dimensional oscillator multiplet
states into so($3$) angular momentum submultiplets, we implement unitary
rotations of three-dimensional Cartesian arrays that form finite pixellated
“volume images.” Transforming between the Cartesian and spherical bases, the
subgroup of rotations in the latter is converted into rotations of the former,
allowing for proper concatenation and inversion of these unitary
transformations, which entail no loss of information.

##  1 Introduction

Unitary rotations of three-dimensional ($3D$) pixellated ‘images’ in a cube of
$N^{3}$ volumetric data, will be _imported_ from the rotations of the $3D$
quantum harmonic oscillator wavefunctions between Cartesian and spherical
coordinates onto the $3D$ _discrete_ oscillator model [1]. The importation
process will follow closely the process which was applied in the $2D$ case of
$N^{2}$ pixellated arrays or screens [2, 3]. This work relies heavily on the
understanding of group theory, and although it is not a requirement, we do not
rely on conventional notation in signal analysis; nevertheless, the reader can
follow, without problem, the development of the presented methods

We start with the well-known $D$-dimensional Heisenberg-Weyl algebra of
position and momentum operators
$\overline{Q}_{i},\overline{P}_{i},\it{1}\in\hbox{\sf hw}_{i}$, with their
standard commutation relation $[\overline{Q}_{i},\overline{P}_{j}]={\rm
i}\delta_{i,j}\it{1}$. These serve to construct the raising and lowering
operators, $\overline{Q}_{i}\pm{\rm i}\overline{P}_{i}$, out of which
oscillator Hamiltonians for each axis are built as the products
$\overline{H}_{i}:={\textstyle\frac{1}{2}}(\overline{P}{}^{2}_{i}+\overline{Q}{}^{2}_{i})$,
also defining the _number_ operators
$\overline{N}_{i}:=\overline{H}_{i}-{\textstyle\frac{1}{2}}\it{1}$ with
integer eigenvalues $n_{i}\in\\{0,1,2,\ldots\\}$, as well as the quadratic
product operators that shift quanta between the axes, which form the
_symmetry_ Lie algebra su($D$) of the $D$-dim oscillator system. These
operators conserve the total energy
$E_{n}=\hbar\omega(n+{\textstyle\frac{1}{2}}D)$, given by the principal
quantum number $n=n_{1}+n_{2}+\cdots+n_{D}$. This su($D$) Lie algebra contains
$D$-dimensional orthogonal subalgebras so($D$) of _rotations_.

The harmonic oscillator wavefunctions are separable in Cartesian coordinates
and also in polar or, generally, spherical coordinates. In the former, the
quantum numbers $n_{i}$ are provided by the $D$ commuting number operators
$\overline{N}_{i}$, while the latter use quantum numbers from the rotation
subalgebra chain. For $D=2$ dimensions, denoted $x,y$, the oscillator states
are classified by

$\displaystyle\overline{N}_{x}\oplus\overline{N}_{y}\quad\subset\quad\hbox{\sf
su($2$)}$ $\displaystyle=\hbox{\sf so($3$)}\quad\supset\quad\hbox{\sf
so($2$)}$ (1) $\displaystyle n_{x}\hskip 20.00003ptn_{y}\quad\quad n:=n_{x}$
$\displaystyle+n_{y}=:2\ell\qquad\quad m$ $\displaystyle
n_{i}\in\\{0,1,\ldots\\},i\in\\{x,y\\}$ $\displaystyle\qquad
m\in\\{-\ell,-\ell+1,\ldots,\ell-1,\ell\\}.$

and form an inverted infinite triangle with apex at the ground state. At level
$n=2\ell$ there are $n+1=2\ell+1$ states degenerate in energy
$E_{n}=n+{\textstyle\frac{1}{2}}$, that form an angular momentum multiplet of
states classified by $m$. [2, 3].

Correspondingly, for $D=3$ dimensions, we have the subalgebra chains and
quantum numbers given by

$\displaystyle\overline{N}_{x}\oplus\overline{N}_{y}\oplus\overline{N}_{z}\qquad\quad\subset$
$\displaystyle\hbox{\sf su($3$)}\qquad\quad\supset\hbox{\sf
so($3$)}\supset\hbox{\sf so($2$)}$ (2) $\displaystyle n_{x}\hskip
20.00003ptn_{y}\hskip 20.00003ptn_{z}\quad\quad n:=n_{x}$
$\displaystyle+n_{y}+n_{z}\qquad\quad l\qquad m$ $\displaystyle
n_{i}\in\\{0,1,\ldots\\},i\in\\{x,y,z\\}$
$\displaystyle\qquad\genfrac{}{}{0.0pt}{0}{l\in
L(n):=\\{n,n-2,\ldots,0\;\mathrm{or}\;1\\}}{m\in
M(\ell):=\\{-\ell,-\ell+1,\ldots,\ell-1,\ell\\}}.$

In this case, each energy level $n$ contains states that can be arranged in an
inverted infinite triangular prism with apex at the ground state. The
multiplet of states with energy $E_{n}=n+1$ forms triangular numbers
$\tau_{n}:=\sum_{0(+2)}^{n}(2\ell+1)={\textstyle\frac{1}{2}}(n+1)(n+2)$, which
for $n\geq 2$ contain more than one angular momentum submultiplet of states.
The linear combination coefficients that relate the Cartesian states
$|\,n_{x},n_{y},n_{z}\,\rangle$ and the spherical $3D$ quantum oscillator
states $|\,n,\ell,m\,)$ were given by Bargmann and Moshinsky in [4] for the
nuclear shell model, and further investigated in [5, 6]. We use a _round_ ket
because $\ell$ is not the eigenvalue of any one operator in the set, but
$\ell(\ell+1)$, $\ell\geq 0$, is the eigenvalue of _squared_ total angular
momentum.

The _finite model oscillator_ was proposed in [1, 7] to form a system with
oscillator dynamics but with a finite number $N$ of energy states. This was
achieved from the Heisenberg-Weyl algebra by _substituting_ $\overline{Q}_{i}$
and $\overline{P}_{i}$ with the triads of operators
$Q_{i},P_{i},K_{i}\in\hbox{\sf su($2$)}$ with the commutation relations
$[Q_{i},P_{i}]={\rm i}K_{i}$, $[K_{i},Q_{i}]={\rm i}P_{i}$,
$[K_{i},P_{i}]=-{\rm i}Q_{i}$ and, in the irreducible representation
$j\in\\{0,\frac{1}{2},1,\frac{3}{2},\ldots\\}$ of su($2$), the ‘displaced
finite oscillator number operators’ $N_{i}:=K_{i}+j{\it 1}$ whose multiplets
have $N:=2j+1$ states with ‘number’ and ‘energy’ eigenvalues
$n=E_{n}^{{\scriptscriptstyle(N)}}-{\textstyle\frac{1}{2}}\in\\{0,1,2,\ldots,N-1\\}$.

Finally, the _symmetry importation_ from the continuous to the discrete model
is performed by applying the rotation linear combination coefficients of the
former, given by the Wigner Big-$D$ matrices
$D^{\ell}_{m,m^{\prime}}(\alpha,\beta,\gamma)$, onto the latter.

In Sect. 2 we give a short account of the construction of the transformation
coefficients for the continuous system; in Sect. 3 we import this symmetry
onto the discrete cube of data, give the correct implementation of the
explicit expression given by [5]. In Sect. 4 we present some examples of
unitary rotations and their concatenation.

##  2 Transformation coefficients for
the $3D$ quantum oscillator

To conform with the prevailing notations in the field, instead of the position
and momentum operators, $Q_{i}$, $P_{i}$ we use the creation and annihilation
operators\star\star$\star$People used to the standard quantum notation can
make the relations $\eta\equiv\hat{a}$ and $\xi\equiv\hat{a}^{\dagger}$.

$\eta_{i}:=\frac{1}{\surd 2}(Q_{i}+{\rm i}P_{i}),\quad\xi_{i}:=\frac{1}{\surd
2}(Q_{i}-{\rm i}P_{i}),\qquad[\xi_{i},\eta_{j}]=\delta_{i,j}{\it 1},\quad
i,j\in\\{x,y,z\\}$ (3)

The three Cartesian commuting oscillator boson creation operators
$\vec{\eta}=(\eta_{x},\eta_{y},\eta_{z})$, and their three adjoint (hermitian
conjugate) annihilation operators $\vec{\xi}=(\xi_{x},\xi_{y},\xi_{z})$,
$\xi_{i}=\eta_{i}^{\dagger}$, act on the independent oscillator mode states,
indicated by $|\,n_{i}\,\rangle$ in Dirac ket notation,

$\eta_{i}|\,n_{i}\,\rangle=\sqrt{n_{i}{+}1}\,|\,n_{i}{+}1\,\rangle,\quad\xi_{i}|\,n_{i}\,\rangle=\sqrt{n_{i}}\,|\,n_{i}{-}1\,\rangle,\qquad\eta_{i}\xi_{i}|\,n_{i}\,\rangle=n_{i}|\,n_{i}\,\rangle.$
(4)

They serve to create the Cartesian oscillator states out of the ground state
$|\,0\,\rangle$ as

$|\,n_{x},n_{y},n_{z}\,\rangle=\frac{\eta_{x}^{n_{x}}\eta_{y}^{n_{y}}\eta_{z}^{n_{z}}}{\sqrt{n_{x}!\,n_{y}!\,n_{z}!}}|\,0\,\rangle.$
(5)

Equivalently it is useful to define their three _polar_ complex linear
combinations,

$\displaystyle\eta_{+}:=-\frac{1}{\surd 2}(\eta_{x}+{\rm
i}\eta_{y}),\quad\eta_{0}$
$\displaystyle:=\eta_{z},\quad\eta_{-}:=+\frac{1}{\surd 2}(\eta_{x}-{\rm
i}\eta_{y}),$ (6) $\displaystyle\xi_{+}:=-\frac{1}{\surd 2}(\xi_{x}-{\rm
i}\xi_{y}),\quad\xi_{0}$ $\displaystyle:=\xi_{z},\quad\xi_{-}:=+\frac{1}{\surd
2}(\xi_{x}+{\rm i}\xi_{y}),$

where $\xi_{r}=\eta_{r}^{\dagger}$ and $[\xi_{r},\eta_{s}]=\delta_{r,s}{\it
1}$ and $r,s\in\\{+,0,-\\}$. The inverse transformation is

$\displaystyle\eta_{x}:=-\frac{1}{\surd 2}(\eta_{+}-\eta_{-}),\quad\eta_{y}$
$\displaystyle:=+{\rm i}\frac{1}{\surd
2}(\eta_{+}+\eta_{-}),\quad\eta_{z}:=\eta_{0},$ (7)
$\displaystyle\xi_{x}:=-\frac{1}{\surd 2}(\xi_{+}-\xi_{-}),\quad\xi_{y}$
$\displaystyle:=-{\rm i}\frac{1}{\surd
2}(\xi_{+}+\xi_{-}),\quad\xi_{z}:=\xi_{0},$

The polar operators act, as the Cartesian ones in (8), on states
$|\,n_{r}\,\rangle^{\\!\circ}$, $r\in\\{+,0,-\\}$, through

$\eta_{r}|\,n_{r}\,\rangle^{\\!\circ}=\sqrt{n_{r}{+}1}\,|\,n_{r}{+}1\,\rangle^{\\!\circ},\
\xi_{r}|\,n_{r}\,\rangle^{\\!\circ}=\sqrt{n_{r}}\,|\,n_{r}{-}1\,\rangle^{\\!\circ},\
\
\eta_{r}\xi_{r}|\,n_{r}\,\rangle^{\\!\circ}=n_{r}|\,n_{r}\,\rangle^{\\!\circ}.$
(8)

Thus one defines the _polar mode_ states

$|\,n_{+},\,n_{0},\,n_{-}\,\rangle^{\\!\circ}:=\frac{\eta_{+}^{n_{+}}\eta_{0}^{n_{0}}\eta_{-}^{n_{-}}}{\sqrt{n_{+}!\,n_{0}!\,n_{-}!}}{|\,0\,\rangle}^{\\!\circ},$
(9)

with the unique ground state $|\,0\,\rangle^{\\!\circ}\equiv{|\,0\,\rangle}$.

The Cartesian and polar states thus relate through

$\displaystyle|\,n_{x},n_{y},n_{z}\,\rangle$ $\displaystyle=\frac{{\rm
i}^{n_{y}}(\eta_{-}{-}\eta_{+})^{n_{x}}(\eta_{-}{+}\eta_{+})^{n_{y}}\eta_{0}^{n_{z}}}{\sqrt{2^{n_{x}+n_{y}}\,n_{x}!\,n_{y}!\,n_{z}!}}|\,0\,\rangle$
(10) $\displaystyle={\rm
i}^{n_{y}}\sqrt{\frac{n_{x}!\,n_{y}!}{2^{n_{x}+n_{y}}}}\sum_{\lambda=0}^{n_{x}}\sum_{\mu=0}^{n_{y}}(-1)^{\lambda}$
$\displaystyle{}\times\frac{\sqrt{(\lambda{+}\mu)!\,(n_{x}{+}n_{y}{-}\lambda{-}\mu)!}}{\lambda!\,\mu!\,(n_{x}{-}\lambda)!\,(n_{y}{-}\mu)!}{\
}|\,\lambda{+}\mu,\,n_{z},\,n_{x}{+}n_{y}{-}\lambda{-}\mu\,\rangle^{\\!\circ},$

and the inverse relation

$\displaystyle|\,n_{+},n_{0},n_{-}\,\rangle^{\\!\circ}$
$\displaystyle=\frac{(-1)^{n_{+}}(\eta_{x}{+}{\rm
i}\eta_{y})^{n_{+}}\eta_{z}^{n_{0}}(\eta_{x}{-}{\rm
i}\eta_{y})^{n_{-}}}{\sqrt{2^{n_{+}+n_{-}}\,n_{+}!\,n_{0}!\,n_{-}!}}|\,0\,\rangle^{\\!\circ}$
(11) $\displaystyle=(-{\rm
i})^{n_{+}+n_{-}}\sqrt{\frac{n_{+}!\,n_{-}!}{2^{n_{+}+n_{-}}}}\sum_{\rho=0}^{n_{+}}\sum_{\sigma=0}^{n_{-}}{\rm
i}^{\sigma-\rho}$
$\displaystyle{}\times\frac{\sqrt{(n_{+}{+}n_{-})!\,(n_{+}{+}n_{-}{-}\rho{-}\sigma)!}}{\rho!\,\sigma!\,(n_{+}{-}\rho)!\,(n_{-}{-}\sigma)!}{\
}|\,n_{+}{+}n_{-},\,n_{+}{+}n_{-}{-}\rho{-}\sigma,\,n_{0}\,\rangle.$

The corresponding Schrödinger wavefunction
$\langle\,\vec{r}\,|\,n_{x},n_{y},n_{z}\rangle$, $\vec{r}=(x,y,z)$, are the
well-known separated Hermite-Gauss functions, a denumerable and orthonormal
basis for functions in the ${\cal L}^{2}({\sf R}^{3})$ Hilbert space.

In the spherical basis, the states are determined by [5, 8], [9, Sects. I.8-9]

$|\,n,\ell,m\,):=A_{n,\ell}\,(\eta^{2})^{\frac{1}{2}(n-\ell)}\,{\cal
Y}_{\ell,m}(\vec{\,\eta})|\,0\,\rangle,$ (12)

where
$\eta^{2}:=\eta^{2}_{x}{+}\eta^{2}_{y}{+}\eta^{2}_{z}=\eta_{0}^{2}-2\eta_{+}\eta_{-}$
is raised to the power $\nu:={\textstyle\frac{1}{2}}(n-\ell)$; $\nu$ is a non-
negative integer identified as the _radial quantum number_. The ${\cal
Y}_{\ell,m}(\vec{\,\eta})$’s are the _solid spherical harmonics_ in the three
creation operators $\vec{\eta}$; written in polar form (6), these are [10,
Sect. 3.10],

$\displaystyle{\cal Y}_{\ell,m}(\vec{\,\eta})$
$\displaystyle=\sqrt{\frac{(2\ell{+}1)\,(\ell{+}m)!\,(\ell{-}m)!}{4\pi\,2^{m}}}$
(13)
$\displaystyle\qquad{}\times\sum_{\mu}\frac{1}{2^{\mu}}\,\frac{\eta_{+}^{m+\mu}}{(m+\mu)!}\,\frac{\eta_{0}^{\ell-m-2\mu}}{(\ell-m-2\mu)!}\,\frac{\eta_{-}^{\mu}}{\mu!}\,.$

When $m\geq 0$, then $\mu$ ranges from 0 to
$\lfloor\frac{1}{2}(\ell-m)\rfloor$; when $m<0$, we use ${\cal
Y}_{\ell,-m}(\vec{\,\eta})=(-1)^{m}{\cal Y}_{\ell,m}(\vec{\,\eta})^{*}$. This
is a polynomial of homogeneous degree $\ell$ in the creation operators.
Together with the factor $(\eta^{2})^{\nu}$ in (12), we have a polynomial
operator of degree $n=2\nu+\ell$ that creates oscillator states of energy
$E_{n}=\hbar\omega(2\nu{+}\ell{+}\frac{3}{2})$. Finally, the normalization
coefficient in (12) is [4, Eqs. (11)]\star\star$\star$Please note that Chacón
and de Llano use “$\nu$” for the total quantum number $n$, while we use it for
the radial quantum number in expressions where it simplifies notation.

$A_{n,\ell}:=(-1)^{\frac{1}{2}(n-\ell)}\sqrt{\frac{4\pi}{(n-\ell)!!\,(n{+}\ell{+}1)!!}}=(-1)^{\nu}\sqrt{\frac{4\pi}{(2\nu)!!\,(2\nu{+}2\ell{+}1)!!}}.$
(14)

We use the principal quantum number $n$, rather than the radial $\nu$, for our
notation of bra-kets and coefficients (in contradistinction to [4] and [Pei-
Liu-1]) because it simplifies many analytical and numerical consideratios.
With $n=2\nu+\ell$, the spherical kets (12)–(14) can be written in terms of
the polar ones (9) as [5]

$\displaystyle|\,n,\ell,m\,)$
$\displaystyle=\frac{(-1)^{\nu}}{2^{\ell}\,\ell!}\sqrt{\frac{(2\ell{+}1)\,2^{m}\,(\ell{-}m)!}{(n{-}\ell)!!\,(n{+}\ell{+}1)!!}}$
(15)
$\displaystyle\phantom{\frac{(-1)^{\nu}}{2^{\ell}\,\ell!}}{}\times\sum_{s=0}^{\ell}\sum_{r=0}^{\nu+s}(-1)^{r+s}2^{r}\bigg{(}{\ell\atop
s}\bigg{)}\bigg{(}{\nu{+}s\atop
r}\bigg{)}\frac{(2\ell{-}2s)!}{(\ell{-}2s{-}m)}$
$\displaystyle\phantom{\frac{(-1)^{\nu}}{2^{\ell}\,\ell!}\sum_{s=0}\sum_{r=0}(-1)^{r+s}2^{r}}\times|\,r+m,\,n-2r-m,\,r\,\rangle^{\\!\circ},$

for $m\geq 0$, while for $m<0$, the expression for $|\,n,\ell,-m\,)$ in (15)
will exchange
$|\,n_{+},\,n_{0},\,n_{-}\,\rangle^{\\!\circ}\mapsto|\,n_{-},\,n_{0},\,n_{+}\,\rangle^{\\!\circ}$.
The Schrödinger wavefunctions $\langle\,\vec{r}\,|\,n,\ell,m)$, are well known
to be separated into a power and associated Laguerre-Gauss function of the
radius, times a spherical harmonic of the direction of $\vec{r}$; these form
also a denumerable and orthonormal basis for ${\cal L}^{2}({\sf R}^{3})$. We
use round kets for the spherical basis to distinguish them when labeled with
numerical values.

In [5] the overlap between the Cartesian and spherical bases, (10) and (12) is
reported. It is nonzero only when they belong to the same energy level and
have the same parity in the $x$–$y$ plane, i.e.,

$n_{x}+n_{y}+n_{z}=n,\quad\hbox{and}\quad n_{x}+n_{y}\pm m\hbox{ is even}.$
(16)

Their expression, with $\nu={\textstyle\frac{1}{2}}(n-\ell)$, can be written
as

$\displaystyle\langle\,n_{x},n_{y},n_{z}\,|\,n,\ell,m)$ $\displaystyle={\rm
i}^{n_{y}}(-1)^{\nu+\frac{1}{2}(n_{x}-n_{y}\pm
m)}2^{-\ell}\,({\textstyle\frac{1}{2}}[n_{x}{+}n_{y}{+}m])!$ (17)
$\displaystyle{}\times\sqrt{(2\ell{+}1)\frac{(\ell{-}m)!}{(\ell{+}m)!}\frac{n_{x}!\,n_{y}!\,n_{z}!}{(n{-}\ell)!!\,(n{+}\ell{+}1)!!}}$
$\displaystyle\times\sum_{r=0}^{\ell}\sum_{s=0}^{n_{x}}\frac{(-1)^{r+s}\,(\nu+r)!\,(2\ell-2r)!}{{s!\,(n_{x}{-}s)!\,(\frac{1}{2}[n_{x}{+}n_{y}{+}m]-s)!\,(\frac{1}{2}[n_{y}{-}n_{x}{-}m]+s)!\atop
r!\,(\ell{-}r)!\,(\ell{-}2r{-}m)!\,(\frac{1}{2}[n_{z}{-}\ell{+}m]+r)!}},$

for $m\geq 0$, while for $m<0$,

$\langle\,n_{x},n_{y},n_{z}\,|\,n,\ell,m)=(-1)^{n_{x}}\langle\,n_{x},n_{y},n_{z}\,|\,n,\ell,-m).$
(18)

Note that all factorials in (17) are applied to integer numbers due to (16).

The overlap coefficients (17) transform unitarily between the $\tau_{n}$
Cartesian states within each energy level $n$, which form a basis for a
completely symmetric irreducible representation of SU($3$), and the spherical
states which reduce into SO($3$) angular momentum multiplets, and which sum
the same number of states, $\sum_{\ell\in L(n)}(2\ell{+}1)=\tau_{n}$. For each
$n\in\\{0,1,2,\ldots\\}$, the transformation relations between bases are

$\displaystyle|\,n,\ell,m\,)$
$\displaystyle=\\!\\!\\!\sum_{n_{x}+n_{y}+n_{z}=n}\\!\\!\\!|\,n_{x},n_{y},n_{z}\,\rangle\langle\,n_{x},n_{y},n_{z}\,|\,n,\ell,m),$
(19) $\displaystyle|\,n_{x},n_{y},n_{z}\,\rangle$
$\displaystyle=\\!\\!\sum_{\ell\in L(n),\ m\in
M(\ell)}\\!\\!|\,n,\ell,m\,)(\,n,\ell,m\,|\,n_{x},n_{y},n_{z}\rangle.$

Now, in the spherical basis, $3D$ _rotations_ $R\in\hbox{\sf SO(3)}$ are
transformations that only act _within_ the angular momentum multiplets

$\displaystyle R:|\,n,\ell,m\,)$ $\displaystyle=\sum_{m^{\prime}\in
M(\ell)}|\,n,\ell,m^{\prime}\,)(\,n,\ell,m^{\prime}\,|\,R\,|\,n,\ell,m\,)$
(20) $\displaystyle=\sum_{m^{\prime}\in
M(\ell)}|\,n,\ell,m^{\prime}\,)\,D_{m^{\prime},m}^{\ell}(R),$

where $D_{m^{\prime},m}^{\ell}(R)$ are the well known Wigner _big-D_ rotation
matrices, usually expressed in Euler angles $R(\phi,\theta,\psi)$.

##  3 Importation of SU($3$) symmetry on the
finite $3D$ oscillator model

The gist of the $1D$ finite oscillator model is to replace the Heisenberg-Weyl
(HW) Lie algebra of raising ($\eta$) and lowering
($\xi{\,\equiv\,}\eta^{\dagger}$) operators

$\\{\eta,\,\xi,\,{\it 1}\\}\in\hbox{span \hbox{\sf HW}},\qquad[\xi,\eta]={\it
1},$ (21)

with an su($2$) algebra of generators $\\{J_{k}\\}_{k=1}^{3}$ in a fixed
representation $j$, of dimension $N=2j+1$, and classify the basis states with
the unit-spaced eigenvalues of either position or mode, as

$\displaystyle\hbox{position $q$:}\quad J_{1}|j,q\rangle_{1}$
$\displaystyle=q\,|j,q\rangle_{1},q|_{-j}^{j},$ (22) $\displaystyle\hbox{mode
$n$:}\quad J_{3}|j,n\rangle_{3}$
$\displaystyle=(n{-}j)\,|j,n\rangle_{3},n|_{0}^{N-1}.$

There are thus only $N$ mode eigenstates $n\in\\{0,1,\ldots,N{-}1\\}$ in the
$1D$ finite oscillator model [1]. The su($2$) commutation relations are the
usual ones [10],

$[J_{3},J_{1}]={\rm i}J_{2},\quad[J_{3},-J_{2}]={\rm
i}J_{1},\quad[J_{1},J_{2}]={\rm i}J_{3}.$ (23)

The first two are the geometric and dynamic Hamilton equations of the quantum
oscillator, while the third distinguishes between the finite oscillator from
the continuous model. The overlaps between the position and mode bases
constitute the finite oscillator eigenstates found accross the related
literature [1, 11],

$\displaystyle{\\!\\!\\!}\Psi_{n}^{(j)}(q)$ $\displaystyle:={}_{1}\langle
j,q|j,n\rangle_{3}=d^{j}_{n-j,q}({\textstyle\frac{1}{2}}\pi)$ (24)
$\displaystyle=\frac{(-1)^{n}}{2^{j}}\sqrt{\bigg{(}{2j\atop
n}\bigg{)}\bigg{(}{2j\atop
j{+}q}\bigg{)}}\,K_{n}(j{+}q;{\textstyle\frac{1}{2}};2j),$

where $d^{j}_{m,m^{\prime}}({\textstyle\frac{1}{2}}\pi)$ are the Wigner
_little-d_ functions [10] for the ${\textstyle\frac{1}{2}}\pi$ angle between
$J_{1}$ and $J_{3}$, and $K_{n}(j{+}q;{\textstyle\frac{1}{2}};2j)$ are the
symmetric Kravchuk polynomials of degree $n$ in $q|_{-j}^{j}$. When
$j\to\infty$ in an appropriate limit, the Kravchuk functions (24) become the
Hermite-Gauss eigenfunctions of the continuous quantum oscillator.

In the three dimensions that we now study, we have three sets of commuting
$\hbox{\sf su($2$)}_{i}$ generators, $i\in\\{x,y,z\\}$, each giving position
and mode (22) along the three space axes. We are here interested in the case
where $j_{x}=j_{y}=j_{z}=:j$ so that we have a total of $N^{3}=(2j+1)^{3}$
states that we can represent as points in a cube of side $N$, which we picture
in Fig. 1 (above). Modifying slightly the notation in (24) where subscripts
denotes the related basis, we write the discrete and finite position states as
$|\,q_{x},q_{y},q_{z}\,\rangle_{1}^{{\scriptscriptstyle(N)}}$. On the other
hand, the discrete mode states, correspondingly written as
$|\,n_{x},n_{y},n_{z}\,\rangle_{3}^{{\scriptscriptstyle(N)}}$ with
$n_{i}|_{0}^{N-1}$, are pictured in Fig. 1 (below left-right) with the total
mode number $n=n_{x}+n_{y}+n_{z}$ along the vertical axis, which ranges from
$n=0$ at the bottom vertex, up to $n=3(N-1)=6j$ at the top vertex. A basis of
$N^{3}$ discrete Cartesian wavefunctions is built from the $1D$ model in (24)
simply as the direct product of the individual one-dimensional wavefunctions

$\displaystyle\Psi_{n_{x},n_{y},n_{z}}^{(j)}(q_{x},q_{y},q_{z})$
$\displaystyle:={}^{{\scriptscriptstyle(N)}}_{\phantom{N}1}\langle\,q_{x},q_{y},q_{z}\,|\,n_{x},n_{y},n_{z}\rangle_{3}^{{\scriptscriptstyle(N)}}$
(25)
$\displaystyle\phantom{:}=\Psi_{n_{x}}^{(j)}(q_{x})\Psi_{n_{y}}^{(j)}(q_{y})\Psi_{n_{z}}^{(j)}(q_{z}).$

These are orthonormal and complete bases for the $N^{3}$ space of ‘$3D$
images’, ‘voxels’, ‘signals’ or _states_ on the discrete cube of Fig. 1. These
act as unitary transformation matrix elements between the two bases depicted
in that figure, i.e.,

$\displaystyle|\,n_{x},n_{y},n_{z}\,\rangle_{3}^{{\scriptscriptstyle(N)}}$
$\displaystyle=\\!\\!\\!\\!\\!\\!\sum_{q_{x},q_{y},q_{z}=-j}^{j}\\!\\!\\!\\!\\!|\,q_{x},q_{y},q_{z}\,\rangle^{{\scriptscriptstyle(N)}}_{1}\,{}^{{\scriptscriptstyle(N)}}_{\phantom{N}1}\\!\langle\,q_{x},q_{y},q_{z}\,|\,n_{x},n_{y},n_{z}\rangle_{3}^{{\scriptscriptstyle(N)}},$
(26)
$\displaystyle|\,q_{x},q_{y},q_{z}\,\rangle_{1}^{{\scriptscriptstyle(N)}}$
$\displaystyle=\\!\\!\\!\\!\\!\\!\sum_{n_{x},n_{y},n_{z}=0}^{2j}\\!\\!\\!\\!\\!|\,n_{x},n_{y},n_{z}\,\rangle^{{\scriptscriptstyle(N)}}_{3}\,{}^{{\scriptscriptstyle(N)}}_{\phantom{N}3}\\!\langle\,n_{x},n_{y},n_{z}\,|\,q_{x},q_{y},q_{z}\rangle_{1}^{{\scriptscriptstyle(N)}}.$

where we remark that the basis for the modes $n_{i}$ is not so intuitive for
visual representation, despite it has the same exact information that the
basis of position $q_{i}$.

Figure 1: Above: States of the $3D$ Cartesian finite oscillator wavefunctions
$\Psi_{n_{x},n_{y},n_{z}}^{{\scriptscriptstyle(N)}}(q_{x},q_{y},q_{z})$ in
(25), of points $q_{x},q_{y},q_{z}|_{-j}^{j}$, arranged into a cube of side
$N=2j+1$. Below: Eigenstates of mode number, arranged by axes
$(n_{x},n_{y},n_{z})$, with mode numbers $n_{i}|_{0}^{N-1}$. The diagonal axis
crossing the cube distinguishes the total modes $n=\sum_{i}n_{i}$, the right-
down vertex is $n=0$ and the left-top vertex corresponds to $n=3(N{-}1)=6j$.
Below-left. Triangular planes are drawn between the lower (inverted) pyramid
$n|_{0}^{2}j$, and the upper pyramid $n|_{4j}^{6j}$. Below-right. Hexagonal
planes are drawn for the intermediate region $n|_{2j+1}^{4j-1}$.

In the _two_ -dimensional case examined in Refs. [2, 3], the position states
are placed on an $N\times N$ square pixellated screen as shown in Fig. 2
(instead of the cube in Fig. 1), and the mode states are arranged in an
$N\times N$ rhombus whose vertical axis is the total mode number
$n=n_{x}+n_{y}$ and the horizontal axis is the mode difference $n_{x}-n_{y}$.
The horizontal rungs in the rhombus, characterized by $n|_{0}^{2N-2}$, contain
each $n+1$ members in the triangle that is the lower half of the rhombus
$0\leq n\leq N{-}1=2j$, while the upper triangle $N\leq n\leq 2N{-}2=4j$
contains the high-mode components, whose members reflect the lower triangle.
At this point, the _importation_ of the su($2$) symmetry consists in taking
these sets of $n+1$ states to be su($2$) _multiplets_ , $|\,\lambda,\mu\,)$,
of angular momentum $\lambda={\textstyle\frac{1}{2}}n$ and distinguished by
the mode _difference_ $\mu={\textstyle\frac{1}{2}}(n_{x}-n_{y})$,
$\mu|_{-\lambda}^{\lambda}$. This holds for the states in the triangle in the
lower half of the rhombus; the upper triangle is treated in the same way for
$n\mapsto 4j-n$. Thereby, plane rotations by $\alpha$ in the $x$–$y$ plane of
the pixellated screen are obtained from the well-defined rotations of the
$|\,\lambda,\mu\,)$ basis states, which are only multiplied by $\exp({\rm
i}\alpha\mu)$ .

Figure 2: Left: States of a two-dimensional finite oscillator represented as
points (or pixels) on a square $N\times N$ screen ($N=2j{+}1$). Right: Mode
states of the $2D$ finite oscillator, $|\,n_{x},n_{y}\,\rangle_{3}$,
$n_{x},n_{y}|_{0}^{2j}$, and $n_{x}{+}n_{y}=n|_{0}^{4j}$ arranged into a
rhombus. The horizontal lines join the imported su($2$) multiplets
$|\,\lambda,\mu\,)$ of angular momenta
$\lambda={\textstyle\frac{1}{2}}n\in\\{0,{\textstyle\frac{1}{2}},1,\ldots,j\\}$
in the lower (and correspondingly in the upper) half of the rhombus,
distinguished by
$\mu={\textstyle\frac{1}{2}}(n_{x}-n_{y})|_{-\lambda}^{\lambda}$.

Returning to the _three_ -dimensional case examined here, we must import the
continuous oscillator su($3$) symmetry algebra onto the finite oscillator mode
states at each total mode level $n=n_{x}+n_{y}+n_{z}$ of Fig. 1; and then use
its so($3$) subalgebra to define states which transform properly under
rotations as in Eq. (20). In this case we have to slice the rhomboid-cube of
modes into $n={}$ constant planes and see if and how we can accommodate them
into $\hbox{\sf so($3$)}\subset\hbox{\sf su($3$)}$ multiplets. In slicing the
cube in this way, we have now _three_ regions to examine, instead of the two
triangles of the $2D$ rhombus in Fig. 2.

Consider first the mode levels in the range $0\leq n\leq N-1$ where the slices
of the cube yield triangular arrangements of $\tau_{n}$ points of side $n+1$,
such as that shown in Fig. 1. These form _completely symmetric_ (or bosonic)
irreducible representations of su($3$), containing the multiplet of the states
$|\,n_{x},n_{y},n_{z}\,\rangle$ of an ordinary quantum oscillator with $n$
energy quanta. These quanta can be shifted among the three axes by means of
the nine u($3$) generators. These generators, ${\cal C}_{i}^{j}$,
$i,j\in\\{x,y,z\\}$, are built in terms of three commuting sets of boson
creation and annihilation operators, $\eta_{i},\,\xi_{j}$, fulfilling the
commutators

${\cal C}_{i}^{j}:=\eta_{i}\xi_{j},\qquad[{\cal C}_{i}^{j},{\cal
C}_{k}^{l}]=\delta_{j,k}\,{\cal C}_{i}^{l}-\delta_{l,i}\,{\cal
C}_{k}^{j},\qquad({\cal C}_{i}^{j})^{\dagger}={\cal C}_{j}^{i}.$ (27)

In the vector space of these nine generators there is the invariant of total
mode

${\cal C}:=\sum_{i}{\cal
C}^{i}_{i}=\eta_{x}\xi_{x}+\eta_{y}\xi_{y}+\eta_{z}\xi_{z}=\eta_{+}\xi_{+}+\eta_{0}\xi_{0}+\eta_{-}\xi_{-},$
(28)

which is at the center in the decomposition $\hbox{\sf u($3$)}=\hbox{\sf
u($1$)}\oplus\hbox{\sf su($3$)}$ that determines this completely symmetric
(bosonic) representation of su($3$) to be characterized by the single non-
negative integer $n$.

Within the set of su($3$) generators (27) we find the subset that generates
the rotation subalgebra $\hbox{\sf so($3$)}\subset\hbox{\sf su($3$)}$ of self-
adjoint operators

$\displaystyle{\cal L}_{x}$ $\displaystyle:=-{\rm i}({\cal C}_{y}^{z}-{\cal
C}_{z}^{y})=\phantom{-{\rm i}}{\textstyle\frac{1}{\surd
2}}[(\eta_{+}{+}\eta_{-})\xi_{0}+\eta_{0}(\xi_{+}{+}\xi_{-})],$ (29)
$\displaystyle{\cal L}_{y}$ $\displaystyle:=-{\rm i}({\cal C}_{z}^{x}-{\cal
C}_{x}^{z})=-{\rm i}{\textstyle\frac{1}{\surd
2}}[(\eta_{+}{-}\eta_{-})\xi_{0}-\eta_{0}(\xi_{+}{-}\xi_{-})],$
$\displaystyle{\cal L}_{z}$ $\displaystyle:=-{\rm i}({\cal C}_{x}^{y}-{\cal
C}_{y}^{x})=\phantom{-{\rm i}}\eta_{+}\xi_{+}-\eta_{-}\xi_{-},$

whose commutation relations are $[{\cal L}_{i},{\cal L}_{j}]=-{\rm i}\,{\cal
L}_{k}$ (with $i,\,j,\,k$ a cyclic permutation of $x,\,y,\,z$). The usual
raising and lowering operators in so($3$) are

$\displaystyle{\cal L}_{+}$ $\displaystyle:={\cal L}_{x}+{\rm i}{\cal
L}_{y}=\sqrt{2}(\eta_{+}\xi_{0}-\eta_{0}\xi_{-}),$ (30) $\displaystyle{\cal
L}_{-}$ $\displaystyle:={\cal L}_{x}-{\rm i}{\cal
L}_{y}=\sqrt{2}(\eta_{-}\xi_{0}-\eta_{0}\xi_{+}),$

with $({\cal L}_{+})^{\dagger}={\cal L}_{-}$, and commutators $[{\cal
L}_{z},{\cal L}_{\pm}]=\pm{\cal L}_{\pm}$ and $[{\cal L}_{+},{\cal
L}_{-}]=-2{\cal L}_{0}\equiv-2{\cal L}_{z}$. The invariant Casimir operator is

${\cal L}^{2}={\cal L}_{x}^{2}+{\cal L}_{y}^{2}+{\cal L}_{z}^{2}={\cal
L}_{\pm}{\cal L}_{\mp}+{\cal L}_{0}({\cal L}_{0}\pm 1).$ (31)

The three commuting generators

${\cal C}_{x}^{x}=\eta_{x}\xi_{x},\quad{\cal
C}_{y}^{y}=\eta_{y}\xi_{y},\quad{\cal C}_{z}^{z}=\eta_{z}\xi_{z},$ (32)

yield the three labels of the $\tau_{n}$ Cartesian states
$|\,n_{x},n_{y},n_{z}\,\rangle$, while the invariant ${\cal C}$ in (28),
${\cal L}^{2}$ in (31), and ${\cal L}_{z}$ in (29) determine the labels of the
spherical basis $|\,n,\ell,m\,)$ with the ranges specified in (2) and shown in
Fig. 1. The $+$ — $-$ mode _difference_ $m:=n_{+}-n_{-}$, as in the $2D$ case,
will have the role of angular momentum projection along the $z$–axis,
_provided_ that the $\tau_{n}$ points in set can be fitted into _complete_
$\hbox{\sf so($3$)}\supset\hbox{\sf so($2$)}$ multiplets characterized by
ranges $m|_{-\ell}^{\ell}$,i.e., those given by (2).

Figure 1 also provides a geometric proof that shows that there are complete
so($3$) multiplets in any one of the levels $n|_{0}^{N-1}$ in the lower
‘inverted pyramid’ of finite-cube states. The lowest two rungs in the triangle
of the figure contain $n+1$ and $n$ points respectively; when projected, on
the $m={\textstyle\frac{1}{2}}(n_{x}-n_{y})$ axis they sum $2n+1$ points
equidistant by unity, _as if_ they were a multiplet of highest angular
momentum $\ell=n$. Then come the next higher two rungs of $n-1$ and $n-2$
points, that project on $2n-3$ equidistant points, as if they belonged to a
multiplet $\ell=n-2$. In the triangle of Fig. 1 the process continues with
every two rungs yielding $\ell=n-4,\,n-6$, etc., ending with $\ell=1$ if $n$
is odd or, when $n$ is even, the projection of the single apex point, $m=0$ of
$\ell=0$. Of course, only the two extreme pairs of points $\pm\mu=\ell=n$ and
$\pm(m-1)$ are _single_ and belong to that highest $\ell$; all other $m$’s
will be linear combinations of the $\ell$’s that we projected out
geometrically in this figure.

The symmetry importation on $0\leq n\leq N-1=2j$ consists in using the
coefficients that bind the quantum harmonic oscillator Cartesian and spherical
states; namely, the coefficients (17) given explicitly by Chacón and de Llano
[5]. The slices in the upper pyramid of the cube of modes, for $4j=2(N-1)\leq
n\leq 3(N-1)=6j$, yield reflected multiplets of ‘anti’-states, for which we
may expect extra phases.

Figure 3: Mode states of the $3D$ finite oscillator,
$|\,n_{x},n_{y},n_{z}\,\rangle_{3}$, $n_{i}|_{0}^{N-1}$ ($N=2j{+}1$), for
total mode $n=\sum_{i}n_{i}$, form triangles of side $n+1$ and irregular
hexagons that depends on the level $n$ reached. This slices the cube in Fig.
1. a) At the lower levels $n\in\\{0,1,\ldots,2j\\}$ of that figure and, b)
rotating by 180∘ those within the highest range,
$n\in\\{4j,\,4j{+}1,\ldots,6j\\}$. When we _import_ the $\hbox{\sf
su($3$)}\supset\hbox{\sf so($3$)}$ symmetry contained in (27)–(29), we project
out _complete_ multiplets $|\,\ell,m\,)$ of ${\cal L}^{2}$ and ${\cal L}_{z}$,
where $\ell$ and $m$ have the ranges (2).

Figure 4: Hexagonal slices of the mode state space
$|\,n_{x},n_{y},n_{z}\,\rangle_{3}$ of the $3D$ finite oscillator: a cube of
$N=7$ points on the side (corresponding to $j=3$) and having $7^{3}=343$
points. a) For total mode numbers $n=8,\,9$ we show the hexagon $(6,2)$ and b)
$(5,3)$. For $n=11,\,12$ the hexagons $(3,5)$ and $(2,6)$ are 180∘-rotated
versions of these. Notice that $n=10$ has $(4,4)$, because we are dealing with
an odd number of points, there exist a middle symmetrical hexagonal plane

The $3D$ case is more complicated than the $2D$ one, because we also have to
consider the _intermediate_ range of total modes $N=2j+1\leq n\leq 4j-1=2N-3$.
There, the $n$-slices of the mode cube are hexagons of generally unequal
sides. We indicate by $(B,T)$ a hexagon with $B$ points on the base and $T$
points on the top. In particular, the $N$ triangles in the bottom pyramid
($n\in\\{0,1,\ldots,N-1\\}$) are $(n+1,1)$ in Fig. 1, and the $N$ inverted
triangles in the top pyramid $(1,6j-n+1)$. The $N-2$ intermediate hexagons are
thus $(N-1,2)$, $(N-2,3)$, $\ldots$, $(3,N-2)$, $(2,N-1)$. We have thus to ask
whether the sets of points $(n_{x},n_{y},n_{z})$ in this intermediate region
can be linearly combined into complete angular momentum multiplets, as those
in the pyramid of Fig. 1 —or not. In Fig. 4 we show these intermediate slices
for $N=7$ (a cube of $7^{3}=343$ voxels) that form the hexagons $(6,2)$ and
$(5,3)$ for $n=6$ and $7$; the next two hexagons, $(3,5)$ and $(2,6)$ for
$n=8$ and $9$, can be seen rotating the figure by 180∘. We note that we can
indeed project out _complete_ multiplets $|\,\ell,m\,)$ of eigenstates where
$m={\textstyle\frac{1}{2}}(n_{x}-n_{y})|_{-\ell}^{\ell}$, with linear
combinations of $\ell\in\\{5,\,4,\,2\\}$ in the first hexagon and
$\ell\in\\{5,\,4,\,3\\}$ in the second. As long as $N$ is even, we can scale
up the figures to see that beyond the $(N,1)$ triangle at total mode $n=N-1$,
in the first intermediate hexagon $(N-1,\,2)$ for $n=N$, the projected points
will accommodate themselves into angular momentum multiplets
$\ell\in\\{N-1,\,N-2,N-4,\ldots,2\\}$. For mode $n=N+1$, the hexagon
$(N-2,\,3)$ will contain so($3$) multiplets
$\ell\in\\{N-1,\,N-2,\,N-3,\,N-5,\ldots,3\\}$, and generally for $n=N+M$, the
hexagon $(N-M+1,M)$ will contain
$\ell\in\\{N-1,\,N-2,\ldots,\,N-M\\}\cap\\{N-M-2,\,N-M-4,\ldots,M\\}$.

##  4 Analysis of rotations

Figure 5: Rotation of a _double-T_ , that is, a T on top and bottom facing
opposite directions. The three-dimensional space has dimension of $9^{3}$
voxels. The rotation is done around the $q_{1}$ and $q_{2}$ axes
simultaneously, from $0$ to $\pi/2$, using the coefficients (17) and the
rotation kernel (20). Here we see a spiral-like path from a vertical to a
horizontal position of the volumetric figure.

In Figure 5 we show the composition of rotations for a “volumetric pixelated
image”. The example presented is based on the analytic expression for the
coefficients (17) and the rotation kernel (20). There we have a _double-T_ ,
that is, a figure resembling the letter “T” facing opposing directions. From
left to right, we concatenate rotations around axes $q_{1}$ and $q_{2}$,
simultaneously; we choose steps $\Delta=\pi/16$, from $0$ to $\pi/2$. In this
configuration, the Euler angles $\alpha=\beta$ and $\gamma$, define a spiral-
like path around $q_{3}$, transitioning from vertical to horizontal position,
and rotating the _legs_ of the double-T. At $\alpha=\beta=\pi/4$ the rod
points toward the direction vector $n=(1,1,1)$ of the pixelated cube.

The voxels show false color since the amplitude of every one of them has been
scaled to be within the range $[0,1]$, using methods previously reported [12],
where the _principal_ voxels that form the body of the rod are presented in a
bluish tone, and the voxels with orangish tone are pixels appearing due the
“discrete Gibbs-_like_ phenomena” of the sharp edges in the transformation. We
want to remark that, since the transformation is unitary when the rotation is
performed, every one of the voxels acquires some amplitude value; but the
voxels ranging below a threshold far from the body of the rod are treated as
residues, and thus they are not shown in the figure, although they are taken
into account when the transformation is sequentially performed.

##  5 Conclusions

The use of the Cartesian-to-spherical transformation coefficients of Bargmann
and Moshinsky has been applied in optics before, by Pei and Liu [13] for the
purpose of efficiently approximating $3D$ Cartesian data through solid
spherical harmonics under various cutoffs; subsequent work has used them to
estimate rotation angles between Cartesian data [14].

Here we extend the three-dimensional rotations previously reported at [15],
were due to a limitation of the group of rotations importation, there was no
way to freely rotate a set of voxels under concatenation. In this work, this
limitation is overcome by the coupling between Cartesian and spherical basis,
whose conjecture is proved to be accurate, since the composition of the
transformation is truly unitary. Moreover, we don’t need to perform some
previous treatment to the original data, like smoothing or filtering, to
overpass the limitations discussed above. In the sense of unitarity, as the
set of previous works always stated, this method is extremely slow since every
voxel depends on all the others, thus we are facing a computational cost of
$~{}\mathcal{O}(N^{6})$.

Finally, the Fourier group is but a subgroup of the most general group
U($N^{D}$) of unitary transformations among the $N^{D}$ pixel elements of
images $f_{m_{1},\ldots,m_{D}}\in{\cal C}^{N^{D}}$. This is the ‘aberration
group’ described in Ref. [16] for 1D finite signals. At present we see no
compelling application for this group beyond the two-dimensional case. Yet it
would further the understanding of the structure of all transformations that
conserve information in finite discrete systems, from the same viewpoint where
linear and nonlinear canonical transformations conserve the structure of
Hamiltonian geometric optics.

## Funding

Consejo Nacional de Ciencia y Tecnología Postdoctoral Grant 2021.

## Acknowledgements

A.R. Urzúa thanks ICF UNAM for the resources provided during the postdoctoral
stay. A.R. also wants to make the present work a homage and a memorial to
Prof. Bernardo Wolf, who sadly passed away during the preparation of the
manuscript.

##  Appendix A Counting modes in the cube

Here we give the detailed number of modes in each plane of the space
$\\{n_{x}+n_{y}+n_{z}=n$. This show the completeness of the method developed
in this work since at the end, we must have the same number of points $N^{3}$
when we count the coplanar modes.

For the sake of clarity, we present three tables, the first and second are
worked examples for $N=5,7$, and the third is a generalization for
$N=\\#\mathrm{odd}$. The first column is the counting mode $n\in\\{0,3N-3\\}$,
and the second column shows the number of modes in the planes depicted as the
base $B$ and tops $T$ in the pair $(B,T)$, the third column shows the number
of angular momentum $\\{l\\}$ in the plane, and in the final column, we give
the total number of modes that are coplanar at each level $n$.

###### Conjecture 1

The number of angular momenta over the whole cube is

$\\{\ell\\}^{N}\;\\{\ell_{max}-1\\}^{N}\;\cdots\;\\{1\\}^{N},\;\;\mathrm{with}\;\;\ell_{max}=N-1,\;\;\mathrm{and}\;\;\\{\ell\\}^{N}:=N(2\ell+1).$

From this last, we can calculate the total number of points in the cube in
terms of the angular momenta as

$\displaystyle\sum\limits_{l=0}^{N-1}\\{\ell\\}^{N}$
$\displaystyle=N(2N+1)+N(2N-3)+\cdots+3N+N$
$\displaystyle=N\sum\limits_{M=1}^{N}(2N-2M+1)$
$\displaystyle=2N^{3}-2N\sum\limits_{M=1}^{N}(M+N^{2})$
$\displaystyle=2N^{3}-2N\left(\frac{1}{2}N(N+1)\right)+N^{2}$
$\displaystyle=N^{3}.$

$\begin{array}[]{ |c|c|c|c|
}\hline\cr\vrule\lx@intercol\hfil\hfil\lx@intercol\vrule\lx@intercol\\\
\vrule\lx@intercol\hfil N=5\hfil\lx@intercol\vrule\lx@intercol\\\
\vrule\lx@intercol\hfil\hfil\lx@intercol\vrule\lx@intercol\\\
\hline\cr\hphantom{5em}n\hphantom{5em}&\hphantom{5em}(B,T)\hphantom{5em}&\hphantom{5em}\\{\ell\\}\hphantom{5em}&\hphantom{5em}\sum(2\ell+1)\hphantom{5em}\\\
\hline\cr 0&(1,1)&\\{0\\}&1\\\ 1&(2,1)&\\{1\\}&2+1=3\\\
2&(3,1)&\\{2,0\\}&3+3=6\\\ 3&(4,1)&\\{3,1\\}&4+6=10\\\
4&(5,1)&\\{4,2,0\\}&5+10=15\\\ \hline\cr 5&(4,2)&\\{5,3\\}&15+3=18\\\
6&(3,3)&\\{6,4,2\\}&18+1=19\\\ 7&(2,4)&\\{5,3\\}&15+3=18\\\ \hline\cr
8&(1,5)&\\{4,2,0\\}&15\\\ 9&(1,4)&\\{3,1\\}&10\\\ 10&(1,3)&\\{2,0\\}&6\\\
11&(1,2)&\\{1\\}&3\\\ 12&(1,1)&\\{0\\}&1\\\ \hline\cr&&&\\\
&&\textrm{Total}:&125=5^{3}=N^{3}\\\ \hline\cr\end{array}$

Table 1: For $N=5$, we have a total of $N^{3}=125$ points in the cube. For
every coplanar level $n$, we have a subset of points that has symmetrical
reflection around $n=6$, where $(B,T)=(3,3)$. The sum of the total angular
momentum ${l}$ leads to the total number of points $N^{3}$.

$\begin{array}[]{ |c|c|c|c|
}\hline\cr\vrule\lx@intercol\hfil\hfil\lx@intercol\vrule\lx@intercol\\\
\vrule\lx@intercol\hfil N=7\hfil\lx@intercol\vrule\lx@intercol\\\
\vrule\lx@intercol\hfil\hfil\lx@intercol\vrule\lx@intercol\\\
\hline\cr\hphantom{5em}n\hphantom{5em}&\hphantom{5em}(B,T)\hphantom{5em}&\hphantom{5em}\\{\ell\\}\hphantom{5em}&\hphantom{5em}\sum(2\ell+1)\hphantom{5em}\\\
\hline\cr 0&(1,1)&\\{0\\}&1\\\ 1&(2,1)&\\{1\\}&2+1=3\\\
2&(3,1)&\\{2,0\\}&3+3=6\\\ 3&(4,1)&\\{3,1\\}&4+6=10\\\
4&(5,1)&\\{4,2,0\\}&5+10=15\\\ 5&(6,1)&\\{5,3,1\\}&6+15=21\\\
6&(7,1)&\\{6,4,2,0\\}&7+21=28\\\ \hline\cr 7&(2,6)&\\{5,3,1\\}&5+28=33\\\
8&(3,5)&\\{8,6,4,2\\}&3+33=36\\\ 9&(4,4)&\\{9,7,5,3\\}&1+36=37\\\
10&(3,5)&\\{8,6,4,2\\}&3+33=36\\\ 11&(2,6)&\\{5,3,1\\}&5+28=33\\\ \hline\cr
12&(1,7)&\\{6,4,2,0\\}&28\\\ 13&(1,6)&\\{5,3,1\\}&21\\\
14&(1,5)&\\{4,2,0\\}&15\\\ 15&(1,4)&\\{3,1\\}&10\\\ 16&(1,3)&\\{2,0\\}&6\\\
17&(1,2)&\\{1\\}&3\\\ 18&(1,1)&\\{0\\}&1\\\ \hline\cr&&&\\\
&&\textrm{Total}:&343=7^{3}=N^{3}\\\ \hline\cr\end{array}$

Table 2: For $N=57$, we have a total of $N^{3}=343$ points in the cube. For
every coplanar level $n$, we have a subset of points that has symmetrical
reflection around $n=9$, where $(B,T)=(4,4)$. The sum of the total angular
momentum ${l}$ leads to the total number of points $N^{3}$.

$\begin{array}[]{ |c|c|c|c|
}\hline\cr\vrule\lx@intercol\hfil\hfil\lx@intercol\vrule\lx@intercol\\\
\vrule\lx@intercol\hfil N=\mathrm{odd}\hfil\lx@intercol\vrule\lx@intercol\\\
\vrule\lx@intercol\hfil\hfil\lx@intercol\vrule\lx@intercol\\\
\hline\cr\hphantom{5em}n\hphantom{5em}&\hphantom{5em}(B,T)\hphantom{5em}&\hphantom{5em}\\{\ell\\}\hphantom{5em}&\hphantom{5em}\sum(2\ell+1)\hphantom{5em}\\\
\hline\cr 0&(1,1)&\\{0\\}&1\\\ 1&(2,1)&\\{1\\}&2+1=3\\\
\cdots&\cdots&\cdots&\cdots\\\
n&(n+1,1)&\\{n,n-2,...,0\;\textrm{or}\;1\\}&\tau_{n}=\frac{1}{2}(n+1)(n+2)\\\
\cdots&\cdots&\cdots&\cdots\\\ N-2&(N-1,1)&\\{N-2,N-4,0\\}&\tau_{N-2}\\\
N-1&(N,1)&\\{N-1,N-3,1\\}&\tau_{N-1}\\\ \hline\cr
N&(N-1,2)&\\{N-1,N-2,...,2\\}&\frac{1}{2}(N^{2}+3N+2)\\\
N+1&(N-2,3)&\\{N-1,N-2,...,3\\}&\frac{1}{2}(N^{2}+5N-6)\\\
\cdots&\cdots&\cdots&\cdots\\\
\frac{3}{2}(N-1)&(\frac{N+1}{2},\frac{N+1}{2})&\\{N-1,N-2,...,\frac{N-1}{2}\\}&\frac{1}{4}(3N^{2}+1)\\\
\cdots&\cdots&\cdots&\cdots\\\
2N-4&(3,N-2)&\\{N-1,N-2,...,3\\}&\frac{1}{2}(N^{2}+5N-6)\\\
2N-3&(2,N-1)&\\{N-1,N-2,...,2\\}&\frac{1}{2}(N^{2}+3N+2)\\\ \hline\cr
2N-2&(1,7)&\\{N-1,N-3,1\\}&\tau_{N-1}\\\
2N-1&(1,6)&\\{N-2,N-4,0\\}&\tau_{N-2}\\\ \cdots&\cdots&\cdots&\cdots\\\
3N+3-n^{\prime}&(1,n^{\prime}+1)&\\{n^{\prime},n-2,1\;\textrm{or}\;0\\}&\tau_{n^{\prime}}\\\
\cdots&\cdots&\cdots&\cdots\\\ 3N-4&(1,2)&\\{1\\}&3\\\ 3N-3&(1,1)&\\{0\\}&1\\\
\hline\cr&&&\\\ &&\textrm{Total}:&N^{3}\\\ \hline\cr\end{array}$

Table 3: Generalization for the counting modes when we deal with a cube of
$N^{3}$ odd number of points. We see that, in the same fashion as the $2D$
case for integer representation size $j$, we have a central mode at
$\frac{3}{2}(N-1)$. For semi-integer $j$, there exist two middle points at
$\frac{3}{2}(N-2)$ and $\frac{3}{2}(N-1)$.

## References

  * [1] Natig M. Atakishiyev, George S. Pogosyan, and Kurt Bernardo Wolf.  Finite models of the oscillator.  2005\.
  * [2] Natig M Atakishiyev, George S Pogosyan, Luis Edgar Vicent, and Kurt Bernardo Wolf.  Finite two-dimensional oscillator: II. the radial model.  Journal of Physics A: Mathematical and General, 34(44):9399–9415, oct 2001.
  * [3] Luis Edgar Vicent and Kurt Bernardo Wolf.  Unitary transformation between cartesian- and polar-pixellated screens.  Journal of the Optical Society of America A, 25(8):1875, jul 2008\.
  * [4] V. Bargmann and M. Moshinsky.  Group theory of harmonic oscillators.  Nuclear Physics, 18:697–712, August 1960.
  * [5] E. Chacon and M. de Llano.  Transformation brackets between cartesian and angular momentum harmonic oscillator basis functions with and without spin-orbit coupling tables for the $2s$–$1d$ nuclear shell.  Rev. Mex. Fix., 12, 1 1963.
  * [6] R.R. Chasman and S. Wahlborn.  Transformation scheme for harmonic-oscillator wave functions.  Nuclear Physics A, 90(2):401–406, Jan 1967.
  * [7] Natig M Atakishiyev, George S Pogosyan, Luis Edgar Vicent, and Kurt Bernardo Wolf.  Finite two-dimensional oscillator: I. the cartesian model.  Journal of Physics A: Mathematical and General, 34(44):9381–9398, oct 2001.
  * [8] K. T. R. Davies and S. J. Krieger.  Harmonic-oscillator transformation coefficients.  Canadian Journal of Physics, 69(1):62–69, jan 1991.
  * [9] M. Moshinsky and Yu.F. Smirnov.  Harmonic Oscillator.  05 1996.
  * [10] L. C. Biedenharn and James D. Louck.  Angular Momentum in Quantum Physics.  12 1984.
  * [11] Alejandro R. Urzúa and Kurt Bernardo Wolf.  Unitary rotation and gyration of pixelated images on rectangular screens.  Journal of the Optical Society of America A, 33(4):642, mar 2016\.
  * [12] Alejandro Urzua and Kurt Bernardo Wolf.  Unitary rotation of pixellated polychromatic images.  Journal of the Optical Society of America A, jun 2022.
  * [13] Soo-Chang Pei and Chun-Lin Liu.  Discrete spherical harmonic oscillator transforms on the cartesian grids using transformation coefficients.  IEEE Transactions on Signal Processing, 61(5):1149–1164, mar 2013\.
  * [14] Soo-Chang Pei and Chun-Lin Liu.  3d rotation estimation using discrete spherical harmonic oscillator transforms.  In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, may 2014.
  * [15] Guillermo Krötzsch, Kenan Uriostegui, and Kurt Bernardo Wolf.  Unitary rotations in two-, three-, and d-dimensional cartesian data arrays.  Journal of the Optical Society of America A, 31(7):1531, jun 2014\.
  * [16] Kurt Bernardo Wolf.  Linear transformations and aberrations in continuous and finite systems.  Journal of Physics A: Mathematical and Theoretical, 41(30):304026, jul 2008.