# What can we learn from misclassified ImageNet images?

Shixian Wen
Department of Computer Science
University of Southern California
Los Angeles, CA 90089
<EMAIL_ADDRESS>
&Amanda Sofie Rios
Neuroscience Program
University of Southern California
Los Angeles, CA 90089
<EMAIL_ADDRESS>
&Kiran Lekkala
Department of Computer Science
University of Southern California
Los Angeles, CA 90089
<EMAIL_ADDRESS>
&Laurent Itti
Department of Computer Science
University of Southern California
Los Angeles, CA 90089
<EMAIL_ADDRESS>

###### Abstract

Understanding the patterns of misclassified ImageNet images is particularly
important, as it could guide us to design deep neural networks (DNN) that
generalize better. However, the richness of ImageNet imposes difficulties for
researchers to visually find any useful patterns of misclassification. Here,
to help find these patterns, we propose "Superclassing ImageNet dataset". It
is a subset of ImageNet which consists of 10 superclasses, each containing
7-116 related subclasses (e.g., 52 bird types, 116 dog types). By training
neural networks on this dataset, we found that: (i) Misclassifications are
rarely across superclasses, but mainly among subclasses within a superclass.
(ii) Ensemble networks trained each only on subclasses of a given superclass
perform better than the same network trained on all subclasses of all
superclasses. Hence, we propose a two-stage Super-Sub framework, and
demonstrate that: (i) The framework improves overall classification
performance by 3.3%, by first inferring a superclass using a generalist
superclass-level network, and then using a specialized network for final
subclass-level classification. (ii) Although the total parameter storage cost
increases to a factor $N+1$ for $N$ superclasses compared to using a single
network, with finetuning, delta and quantization aware training techniques
this can be reduced to $0.2N+1$. Another advantage of this efficient
implementation is that the memory cost on the GPU during inference is
equivalent to using only one network. The reason is we initiate each subclass-
level network through addition of small parameter variations (deltas) to the
superclass-level network. (iii) Finally, our framework promises to be more
scalable and generalizable than the common alternative of simply scaling up a
vanilla network in size, since very large networks often suffer from
overfitting and gradient vanishing.

##  1 Motivation

DNNs have led to a series of breakthroughs for image classification [1, 2]. To
improve classification accuracy, researchers proposed different kinds of
strategies, including designing better network structures (e.g., ResNet [3]
and DenseNet [4]), designing better optimizers (e.g., Adam optimizer [5] and
RMSprop optimizer [6]), applying better loss formulation (e.g., contrastive
loss [7]), and using large-scale pretraining [8].

Figure 1: Superclassing ImageNet dataset. Superclassing ImageNet dataset is a
subset of the ImageNet dataset. It contains broad classes which each subsume
several of the original ImageNet classes. It is consisted of 10 superclasses —
Bird, Boat, Car, Cat, Dog, Fruit, Fungus, Insect, Monkey and Truck
superclasses. Each superclass contains several subclasses, for example, the
Bird superclass is consisted of Chicken, Ostrich, Black Swan subclasses and
etc.

However, few researchers have explored the patterns of misclassifications by
current DNNs. For instance, Dodge et al. [9] found that DNNs performance is
much lower than human performance on distorted images. Understanding these
patterns is particularly important because it could guide us to design more
robust DNNs that make fewer mistakes and generalize better.

In addition, while in early days the answer to improving performance was often
to build ever larger networks, this approach has more recently been shown to
not scale well because of (i) overfitting [10, 11, 12] and (ii) gradient
vanishing [13] — (i) When there is too much overfitting, this results in poor
generalization in the test set. For example, Laurence et al. ([12], in their
Fig.2) showed that, generally, as the network size increases, training error
decreases but this ultimately leads to overfitting. As a consequence, test
errors exhibit a "U" shaped behavior as a function of network size. (ii) The
gradient vanishing problem limits the number of parameters and layers for
scaling up the size of networks. Residual networks [3] are famous for easing
the training of networks that are substantially deeper. They create a gradient
highway for reducing the effect of gradient vanishing. However, residual
networks still cannot avoid the problem entirely. For example, in Tab. 6 of
the original residual network paper [3], performance of a residual network
with 1202 layers is lower than that of residual networks with 32, 44, 56, 110
layers. We find similar scaling issues as detailed in our Results. This
prompts us to explore alternatives for boosting performance with more
parameters rather than naively expanding the size of a single network.

##  2 Observations and General Framework

Often, in image classification research, datasets with the richness of
ImageNet [14] are desired. However, the richness of the datasets imposes
extraordinary difficulties for researchers to visually find any useful pattern
of errors. For example, there are 1000 classes in ImageNet. It is impractical
to find any useful pattern in a 1000 by 1000 confusion matrix. To smooth the
path towards finding these patterns, without adding complexity to the ImageNet
dataset, we “superclass” ImageNet — _"Superclassing ImageNet dataset"_.
Superclassing ImageNet dataset is a hierarchical subset of ImageNet that
contains broad superclasses (derived from Wordnet, see Sec. 1) which each
subsume several of the original ImageNet classes (Fig. 1). Our dataset
contains 10 superclasses and each superclass contains 7 to 116 subclasses, for
a total of 253 subclasses. Thus, by adopting this hierarchical subdivision, we
initially only navigate through 10 superclasses instead of the 1000 classes in
the original ImageNet dataset. We found useful patterns in a 10 by 10
confusion matrix (Fig. 2) and in the summary of classification accuracies for
all subclasses within each superclass (Fig. 3). We highlight two observations
from training convolutional neural networks on this dataset:

Figure 2: Confusion matrix for inter-superclasses prediction. Correct
classification (green), mild wrong classification (orange), severe wrong
classification (red). The convolutional neural network (ResNet-18) barely make
inter-superclasses misclassifications as the averaged correct prediction
performance is 96.52%.

(i) Misclassifications are seldom across superclasses (Fig. 2), but mainly
intra-superclass.

We trained a ResNet-18 neural network (pretrained on ImageNet) [3] on all
images from all superclasses of the Superclassing ImageNet dataset to make a
10-way superclass prediction. We named it the _Superclass Network_. The high
values on the diagonal of the confusion matrix in Fig. 2 mean that the
Superclass Network barely makes any inter-superclass mistakes, as the averaged
correct prediction performance is 96.52%. For example, the probability of
recognizing a bird image as belonging to the Bird superclass is 96.77%, while
the probability of recognizing a bird image as belonging to the Car superclass
is only 0.08%. In addition, for a network that is still trained on all images
from all superclasses, but for which the goal is to perform a 253-way
classification directly among all subclasses, the averaged correct prediction
performance decreases to 71.18%. This lower performance suggests that most
misclassifications are in intra-superclass predictions.

(ii) Ensemble networks trained each only on subclasses of a given superclass
(upperbound) perform better than the same network trained on all subclasses of
all superclasses (lowerbound; Fig. 3).

In more details:

1\. _Upperbound (with a superclass oracle)_ : We trained a ResNet-18 neural
network (pretrained on ImageNet) on images from a particular superclass in the
superclassing ImageNet dataset. We named it a _Subclass Network_. There are
ten superclasses in the superclassing ImageNet dataset, so we have ten
Subclass Networks. For a test image, with a superclass oracle, we select the
right Subclass Network and only infer the subclasses inside the superclass it
belongs to (upperbound). For example, a Subclass Network for the Dog
superclass is trained only on dog images. It would only predict the 116 dog
subclasses (e.g., Husky, Malamute, Papillon, etc). The correct prediction
performance is 75.07% when averaged across all 10 superclasses.

2._Lowerbound (without a superclass oracle)_ : we trained a ResNet-18 neural
network (pretrained on ImageNet) on all images from all subclasses in the
Superclassing ImageNet dataset. For a test image, the network would infer the
subclass among all 253 subclasses in all superclasses, without any prior
knowledge of which superclass the test image belongs to (lowerbound). For
exposition purposes, in Fig. 3), the results are shown for all subclasses
within each superclass. The averaged performance across all 253 subclasses is
71.18%. The performance gap can be quantified as: upperbound is 5.47% better
than lowerbound.

Figure 3: Performance gap between classifying an image for subclasses inside
the superclass it belongs to (upperbound, green, with a superclass oracle, the
averaged correct prediction performance is 75.07%) and classifying an image
for all subclasses from all superclasses (lowerbound, without a superclass
oracle, red, the averaged correct prediction performance is 71.18%).
Upperbound is 5.47% better than lowerbound.

In this paper, our goal is to design a framework such that its classification
performance can _approach the upperbound performance without requiring a
superclass oracle_. Here, we propose a _two-stage Super-Sub framework_ (Fig.
4) to reduce the performance gap. In details, in the first stage, we use a
Superclass Network to decide the superclass of the test image. In the second
stage, we choose the corresponding Subclass Network according to the
superclass decision made in the first stage. This Subclass Network then
outputs the final fine-grained subclass of the test image. More generally, our
two-stage Super-Sub framework enables a neural network system to
_automatically adjust its behavior according to different inputs_ [15], here
achieved through the aforementioned hierarchical two-step inference. From our
first observation, the Superclass Network barely makes inter-superclass
mistakes, so the role of the _Superclass Network can be approximated as a
superclass oracle_ , helping to choose the correct Subclass Network for
subclass classification. As a result, our two-stage Super-Sub framework is
_3.30% better_ than the lowerbound on the new Superclassing ImageNet dataset.
Nonetheless, our vanilla implementation of the framework incurs a large cost
in both parameter storage and inference runtime memory on the GPU (a factor of
$N+1$ for $N$ superclasses compared to a single network). To improve this, we
propose a method that keeps the same inference runtime memory as only one
network, and that reduces the parameter storage cost to $0.2N+1$ based on
finetuning [16], delta and quantization aware training (QAT) [17] techniques,
all while preserving the same accuracy boost. Most importantly, our framework
promises to be more scalable and generalizable than the common alternative of
simply scaling up a vanilla network in size since very large networks suffer
from overfitting and gradient vanishing. We demonstrate our framwork’s
capacity for scalability (see Results Sec. 5) by increasing the
representational capacity starting from an already very large network, i.e.
Resnet-152.

##  3 Our Approach

###  3.1 The Superclassing ImageNet dataset

Table 1: Superclassing ImageNet dataset: For each row, we present the name of each Superclass and its contained number of subclasses, number of training images and number of test images.  Superclass Name |  # of Subclasses |  # of Training Images |  # of Test Images
---|---|---|---
Bird | 52 | 67,441 | 2,600
Boat | 6 | 7,706 | 300
Car | 10 | 13,000 | 500
Cat | 8 | 10,400 | 400
Dog | 116 | 145,273 | 5,800
Fruit | 7 | 9,100 | 350
Fungus | 7 | 9,100 | 350
Insect | 27 | 35,100 | 1,350
Monkey | 13 | 16,900 | 650
Truck | 7 | 8,959 | 350

_Superclassing ImageNet dataset_ (Fig. 1) is a new dataset that contains broad
classes which each subsume several of the original ImageNet classes. It is a
subset of the ImageNet Dataset that contains 10 superclasses with 253
subclasses. It contains 311,279 training images and 12,650 test images.

Making custom dataset: We leverage the WordNet hierarchy according to which
ImageNet is organized [18] to create the Superclassing ImageNet dataset with
10 superclasses (Tab. 1):

Figure 4: Two-stage Super-Sub framework: On stage 1, a Superclass Network is
trained to decide the superclass of a test image. On stage 2, we choose the
corresponding Subclass Network to decide the subclass inside the decided
superclass in stage 1.

###  3.2 Super-Sub framework

Our goal is to approach the upperbound performance without having a superclass
oracle. We proposed a two-stage Super-Sub framework to achieve this goal (Fig.
4): In the first stage, a Superclass Network is trained to decide the
superclass of a test image. Since the Superclass Network barely makes inter-
superclass mistakes, it can be approximated as a superclass oracle. The
approximated superclass oracle can help us select the right Subclass Network
for the second stage. In the second stage, we use the right Subclass Network
to decide the subclasses inside the superclass decided in the first stage.

####  3.2.1 Vanilla Network Architectures and Vanilla Inference Rules

Superclass Network: The Superclass Network decides the superclass of a test
image (e.g., Bird, Boat, Cat, etc.) We trained a ResNet-18 neural network on
all images from all superclasses of Superclassing ImageNet dataset. Formally,
we trained the Superclass Network by minimizing the loss function as:

$\begin{aligned}
\mathop{\min}_{\,\mathbf{\theta}_{super}}\end{aligned}\,\,L\,(f_{super}\,(\mathbf{x}_{all},\,\mathbf{y}_{super};\,\mathbf{\theta}_{super}))$
(1)

where $f_{super}$ and $L$ are the Superclass Network and cross entropy loss.
$\mathbf{x}_{all}$ and $\mathbf{y}_{super}$ are the image inputs from all
superclasses and its superclass label. $\mathbf{\theta}_{super}$ is the
parameters of the Superclass Network. The learning rate is 0.01. The training
epochs are 200.

Vanilla Subclass Networks: Each Subclass Network decides the subclasses within
the corresponding superclass. Each Vanilla Subclass Network is a ResNet-18. We
trained each Subclass Network on images from each particular superclass of our
Superclassing ImageNet Dataset. Each Subclass Network would only infer in the
range of the subclasses for the superclass it corresponds to. If there are $N$
superclasses, there are $N$ Subclass Networks. For example, a Subclass Network
for Bird superclass is trained on bird images only. It would only predict the
subclasses (e.g., Chicken, Ostrich , Black Swan, etc.) within the Bird
superclass. Formally, we trained the Subclass Networks by minimizing the loss
function as:

$\begin{aligned}
\mathop{\min}_{\,\mathbf{\theta}^{i}_{sub}}\end{aligned}\,\,L\,(f^{i}_{sub}\,(\mathbf{x}^{i}_{sub},\,\mathbf{y}^{i}_{sub};\,\mathbf{\theta}^{i}_{sub})),\,\,\,\,\,\,\,\,\,\,\,\forall\,i\in[Bird,\,Boat,\,Car,\,...\,Truck]$
(2)

where $i$ is an iterator that indicates the current superclass. $f^{i}_{sub}$
is the Subclass Network corresponding to superclass $i$. $L$ is the cross
entropy loss. $\mathbf{x}^{i}_{sub}$ and $\mathbf{y}^{i}_{sub}$ are the input
images from superclass $i$ and its corresponding subclass labels.
$\mathbf{\theta}^{i}_{sub}$ is the parameters of the Subclass Network for
superclass $i$. For our experiments, we use a learning rate of 0.01 with 100
epochs. The settings of all hyper-parameters are the same for all subclass
networks.

Vanilla Inference Rules: Formally, we define the vanilla inference rules as
following:

Algorithm 1 Vanilla Inference rules

Loading the Superclass Network and all Subclass Networks to GPU

For an image input $x$

On the first stage:

Deciding the superclass: $i\leftarrow Superclass\\_Network(x)$

On the second stage:

Choosing the corresponding $Subclass\\_Network^{i}$ according to the $i$ in
the first stage

Deciding the subclass: $y^{i}_{sub}\leftarrow Subclass\\_Network^{i}(x)$

####  3.2.2  Efficient Network Architectures and Efficient Inference Rules

In this subsection, our goal is to implement the Super-sub framework
efficiently with particular focus on deployment of deep learning systems on
edge devices. It is costly to design an edge device that is capable of loading
all networks to its GPU as required by our vanilla implementation. Memory-
wise, it is cheaper to load one network at a time to the GPU. However, on edge
devices switching between different networks incurs expensive (in terms of
time) input/output (I/O) operations between GPU and main memory. Since at
inference time we need to keep switching between the super-class network and
sub-network, the cost of the I/O operations would become expensive. Here, we
propose an efficient implementation (Fig. 5) that trades-off the expensive I/O
operations to likely cheaper add operations.

Figure 5: Efficient implementation of Super-Sub framework: one network at a
time for inference. The compressed deltas (20% of its original size) between
Superclass Network and subclass finetuned networks are stored inside the main
memory. The subclass finetuned networks are reconstructed by adding the
corresponding deltas to the parameters of the Superclass Network. This
approach is less time consuming because the compress deltas incur less I/O
cost.

The problem domains of Subclass Networks are subsets of the problem domain of
the Superclass Network. For example, recognizing an image of a Husky in the
dog superclass is a subproblem of recognizing an image of dog in various
superclasses. Since it is a domain reduction problem [19], Subclass Networks
can be created by finetuning from the Superclass Network efficiently (Subclass
Finetuned Networks). We found that the deltas between the parameters of the
Superclass Network and each Subclass Finetuned Network are small and because
of this, we are able to greatly compress them. Thus, in our efficient
implementation, to recreate a Subclass Network, we load the compressed deltas
(reducing the I/O operations to 20%), and add them in the GPU to the
parameters of the Superclass Network (Fig. 5).

Superclass Network: The network architecture and training of the Superclass
Network in the efficient implementation is the same as the Vanilla Superclass
Network described in Sec. 3.2.1.

Subclass Finetuned Network: Instead of training each Subclass Network from
scratch for each superclass, we created it by finetuning the Superclass
network on the images from each particular superclass of the superclassing
ImageNet dataset, by minimizing the loss function:

$f^{i}_{sub\\_finetuned}\,(\mathbf{x}^{i}_{sub},\mathbf{y}^{i}_{sub};\mathbf{\theta}^{i}_{sub})\leftarrow\begin{aligned}
\mathop{\min}_{\,\mathbf{\theta}_{super}}\end{aligned}\,\,L\,(f_{super}\,(\mathbf{x}^{i}_{sub},\mathbf{y}^{i}_{sub};\mathbf{\theta}_{super})),\,\,\,\,\forall\,i\in[Bird,\,Boat,\,...\,Truck]$

(3)

where, $f^{i}_{sub\\_finetuned}$ is the Subclass Finetuned Network for
superclass $i$, other notations are the same as in Eqn. 1 and Eqn. 2. The
hyper-parameters are the same in the Vanilla Subclass Networks.

Deltas: The deltas for a particular superclass are created with the
subtraction of the parameters of the Superclass Network from that of the
corresponding Subclass Finetuned Network:

$deltas^{i}=\mathbf{\theta}^{i}_{sub}-\mathbf{\theta}_{super}$ (4)

where $deltas^{i}$ corresponds to the deltas for superclass $i$. other
notations are as in Eqn. 1 and Eqn. 2. $deltas^{i}$ is calculated through
mini-batch gradient descent via Eqn. 3. Since these gradients are in half-
precision (16 bits), $deltas^{i}$ is represented in half-precision (50% of the
size to store the Subclass Finetuned network in full-precision 32 bits). Then,
we use 7-Zip to further compress $deltas^{i}$ (reduce to 44% of the size to
store the Subclasses Finetuned network in full-precision) and store them. In
the following, we show how to further reduce the compressed delta size to a
final 20% with Quantization Aware Training techniques. For inference, to
recreate the Subclass Finetuned Network, we can load the corresponding
compressed deltas to GPU and add them to the Superclass Network.

Efficient Inference Rules: Formally, we defined the efficient inference rules
as:

Algorithm 2 Efficient Inference rules

Loading the Superclass Network to GPU

For an image input $x$

On the first stage:

Deciding the superclass: $i\leftarrow Superclass\\_Network(x)$

On the second stage:

Loading the compressed $deltas^{i}$ according to the superclass $i$

Recreating the Subclass Finetuned Network for superclass $i$ through:
$\mathbf{\theta}^{i}_{sub}\leftarrow deltas^{i}+\mathbf{\theta}^{i}_{super}$

Deciding the subclass: $y^{i}_{sub}\leftarrow
Subclass\\_Finetuned\\_Network^{i}(x)$

Quantization Aware Training (QAT): To further reduce the compressed size of
the deltas, we used Quantization aware training (QAT) [17], available in the
Pytorch Framework, to finetune the Superclass Network (See Supplementary).
With QAT, the values are limited to 256 possibilities in 8 bit quantization.
Thus, with less variability, the compressed size of the deltas is further
reduced to 20% compared to storing the original quantized Subclass Network.

In summary, in our efficient implementation there is always only one network
running on the GPU. Furthermore, the extra main memory cost for each Subclass
Network is reduced to around 20% of its original size.

##  4 Experiments

We evaluated our Vanilla and Efficient implementations on the Superclassing
ImageNet Dataset. We report classification accuracy on each superclass and
average accuracy across all superclasses. In addition, we report the
compression ratio for storing the deltas used in the efficient implementation.

To our best knowledge, we are the first to leverage the patterns of
misclassified images to improve the scalability and generalizability of DNNs.
Because our method is in a very novel direction, we did not find directly
applicable benchmarks to compare to. Instead, we propose the lowerbound and
upperbound defined in Sec. 2 to act as natural references to our framework’s
performance.

##  5 Results and Analysis

The Super-Sub framework improved overall classification performance without a
superclass oracle: On average, with a RestNet-18 backbone, our Efficient
(Vanilla) implementation is 3.20% (3.30%) better than the lowerbound (Fig. 6)
and approaches the upperbound. The performances of Efficient and Vanilla
implementations are almost equivalent. With a RestNet-152 backbone, our
Vanilla implementation is 8.00% better than the lowerbound.

Figure 6: The performances of Efficient (rose-red) and Vanilla (yellow)
Implementations of the Super-Sub framework. Both Efficient and Vanilla
Implementations reduced the performance gap between upperbound (green) and
lowerbound (red). The rose-red and yellow eclipses represent the relative
position of efficient and vanilla implementations’ performances to the
upperbound and lowerbound. The definitions of upperbound and lowerbound are
the same as that of Fig. 3.

Deltas are small and can be greatly compressed: We found that the deltas of
parameters between the Superclass Network and Subclass Finetuned Networks are
small (Fig. 7.a,b; black curve fluctuates around 0). The histograms of deltas
for different superclasses further confirmed this (Fig. 7.c). In addition, we
found we can compress the deltas because their distributions are highly
concentrated around 0 (Fig. 7.c). The average compression ratio is 0.44
without using QAT (Fig. 7.d) and is 0.20 with QAT (Fig. 7.e). This means that
the GPU only needs to load 20% of the size of the original quantized Subclass
Finetuned Network to recreate it from an already loaded Superclass Network.

Figure 7: a-b) Visualization of weights of a convolutional layer and a batch
normalization layer in a Superclass Network (red) and Subclass Finetuned
Network (for fruit superclass, blue). The black curve is the difference
between the two networks. c) Various histograms of deltas between Superclass
network and Subclass Finetuned Network for each superclass. For d-e) The dash
line is the averaged compressed ratio across all superclasses. d) Compressed
ratio for Subclass Networks without using QAT. e) Compressed ratio for
Subclass Networks with QAT.

Super-sub framework promises to be more scalable and generalizable: To
demonstrate this, we pick a starting point of a large network (ResNet-152) as
a backbone of our Super-Sub framework. Resnet-152, in terms of memory size,
marks a threshold after which gradient vanishing and overfitting can start
occurring, prohibiting further layer increases within the traditional ResNet
regime. This is often referred as the "U" shaped test error performance as a
function of network size. We show that if we scale up Resnet-152 using our
super-sub framework, which incurs in adding 440 MB of memory, we observe a
huge performance boost from 78.43% to 84.63% in accuracy. In contrast, if
Resnet-152 is scaled up naively just by adding layers, which here would be
approximately equal to expanding to a Resnet-455 (Super-Sub framework with a
Resnet-152 backbone $\approx$ Resnet-455 in terms of memory), we observe a
stark performance degradation, with the accuracy lowering to 68.01%.

##  6 Discussion

In this paper, we studied the patterns of misclassified ImageNet images and
leveraged these patterns to improve the scalability and generalizability of a
DNN. By training a neural network on "Superclassing ImageNet Dataset", we
found that: (i) Misclassifications are rarely inter-superclasses but mainly
intra-superclass. (ii) Ensemble networks trained each only on subclasses of a
given superclass perform better than the same network trained on all
subclasses of all superclasses. Hence, we proposed the Super-Sub framework and
demonstrated that: (i) The Super-Sub framework reduced the performance gap
without having a superclass oracle, (ii) We leverage [17] techniques to create
an efficient implementation with low parameter and memory costs. (iii) We
demonstrated that our Super-Sub framework promises to be more scalable and
generalizable.

To date, scaling up the size and depth of DNN quickly becomes prohibitive due
to the emergence of gradient vanishing [13] and overfitting issues. Our two-
stage super-sub framework provides a new and efficient way of scaling up a DNN
beyond the current limitations. Our framework subdivides the overall
classification problem domain into separate smaller sub-domains that are
semantically grouped. Each image goes first through a high-level
classification by a Superclass Network which then routes it to the appropriate
Subclass Network specialized for the sub-domain fine-grained classification.
In our approach, each sub-domain is only a subset of the original overall
classification problem. Hence, each can be expressed through a smaller model
with less parameters and therefore less overfitting when compared to using a
larger model to tackle directly the original problem. As previously discussed,
naively increasing the size and depth within one DNN is not scalable because
of gradient vanishing. Here, our model addresses this by increasing the number
of parameters "laterally" into several separate network modules (sub-domains)
which naturally confers more robustness to gradient vanishing and allows a
much greater potential for upward scalability.

In sum, our approach is a more scalable and generalizable alternative to just
naively expanding the size of a single network. Nonetheless, despite those
excellent benefits, two-stage Super-Sub framework has some limitations: (i)
Errors in first stage are unrecoverable; We could create better superclasses
to maximize the Superclass Network performance using a data-driven approach.
(ii) We have not yet fully implemented the efficient version on an edge
device. This may require a next generation of devices that could support
uncompress, weight operations, etc, if our approach gains sufficient general
interest.

## References

  * [1] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  Advances in neural information processing systems, 25:1097–1105, 2012.
  * [2] Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel.  Backpropagation applied to handwritten zip code recognition.  Neural computation, 1(4):541–551, 1989.
  * [3] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
  * [4] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
  * [5] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [6] Tijmen Tieleman and GE Hinton.  Neural networks for machine learning.  Coursera (Lecture 65-RMSprop), 2012.
  * [7] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan.  Supervised contrastive learning.  arXiv preprint arXiv:2004.11362, 2020.
  * [8] Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten.  Exploring the limits of weakly supervised pretraining.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 181–196, 2018.
  * [9] Samuel Dodge and Lina Karam.  A study and comparison of human and deep learning recognition performance under visual distortions.  In 2017 26th international conference on computer communication and networks (ICCCN), pages 1–7. IEEE, 2017.
  * [10] Shaeke Salman and Xiuwen Liu.  Overfitting mechanism and avoidance in deep neural networks.  arXiv preprint arXiv:1901.06566, 2019.
  * [11] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  Dropout: a simple way to prevent neural networks from overfitting.  The journal of machine learning research, 15(1):1929–1958, 2014\.
  * [12] Steve Lawrence, C Lee Giles, and Ah Chung Tsoi.  Lessons in neural network training: Overfitting may be harder than expected.  In AAAI/IAAI, pages 540–545. Citeseer, 1997.
  * [13] Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber, et al.  Gradient flow in recurrent nets: the difficulty of learning long-term dependencies, 2001.
  * [14] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * [15] Yizeng Han, Gao Huang, Shiji Song, Le Yang, Honghui Wang, and Yulin Wang.  Dynamic neural networks: A survey.  arXiv preprint arXiv:2102.04906, 2021.
  * [16] Nima Tajbakhsh, Jae Y Shin, Suryakanth R Gurudu, R Todd Hurst, Christopher B Kendall, Michael B Gotway, and Jianming Liang.  Convolutional neural networks for medical image analysis: Full training or fine tuning?  IEEE transactions on medical imaging, 35(5):1299–1312, 2016.
  * [17] Raghuraman Krishnamoorthi.  Quantization aware training.  http://https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html.  Accessed: 2010-09-30.
  * [18] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet website.  https://image-net.org/about#:~:text=What%20is%20ImageNet%3F,them%20are%20nouns%20(80%2C000%2B). Accessed: 2010-09-30.
  * [19] Mohit Tawarmalani and Nikolaos V Sahinidis.  Convexification and global optimization in continuous and mixed-integer nonlinear programming: theory, algorithms, software, and applications, volume 65.  Springer Science & Business Media, 2013.

##  7 Supplementary

###  7.1 Quantization Aware Training (QAT):

"QAT is the quantization method that typically results in the highest accuracy
. With QAT, all weights and activations are ’fake quantized’ during both the
forward and backward passes of training: that is, float values are rounded to
mimic int8 values, but all computations are still done with floating point
numbers. Thus, all the weight adjustments during training are made while
“aware” of the fact that the model will ultimately be quantized; after
quantizing, therefore, this method will usually yield higher accuracy than
either dynamic quantization or post-training static quantization."[17].

## Checklist

  1. 1.

For all authors…

    1. (a)

Do the main claims made in the abstract and introduction accurately reflect
the paper’s contributions and scope?

Yes.

    2. (b)

Did you describe the limitations of your work?

Yes, We descriped them in the discussion section.

    3. (c)

Did you discuss any potential negative societal impacts of your work?

We don’t know any potential negative societal impacts of out work.

    4. (d)

Have you read the ethics review guidelines and ensured that your paper
conforms to them? Yes, it conforms.

  2. 2.

If you ran experiments…

    1. (a)

Did you include the code, data, and instructions needed to reproduce the main
experimental results (either in the supplemental material or as a URL)?

Upon publication, we will release them.

    2. (b)

Did you specify all the training details (e.g., data splits, hyperparameters,
how they were chosen)?

Yes, we include it in the Section 2 and Section 3.

    3. (c)

Did you report error bars (e.g., with respect to the random seed after running
experiments multiple times)?

We ran the experiments multiple times and can reproduce the results.

    4. (d)

Did you include the total amount of compute and the type of resources used
(e.g., type of GPUs, internal cluster, or cloud provider)?

We used Nvidia 1080Ti and Nvidia V100 to train the model.


#  Thermoviscous Hydrodynamics in Non-Degenerate Dipolar Bose Gases

Reuben R.W. Wang<EMAIL_ADDRESS>John L. Bohn  JILA, University of
Colorado, Boulder, Colorado 80309, USA

###### Abstract

We present a hydrodynamic model of ultracold, but not yet quantum condensed,
dipolar Bosonic gases. Such systems present both $s$-wave and dipolar
scattering, the latter of which results in anisotropic transport tensors of
thermal conductivity and viscosity. This work presents an analytic derivation
of the viscosity tensor coefficients, utilizing the methods established in
[Wang et al., arXiv:2205.10465], where the thermal conductivities were
derived. Taken together, these transport tensors then permit a comprehensive
description of hydrodynamics that is now embellished with dipolar anisotropy.
An analysis of attenuation in linear waves illustrates the effect of this
anisotropy in dipolar fluids, where we find a clear dependence on the dipole
orientation relative to the direction of wave propagation.

††preprint: APS/123-QED

##  I  Introduction

The field of ultracold dipolar physics has seen exciting progress in the last
2 decades Bohn  _et al._ (2017); Moses  _et al._ (2017); Liu and Luo (2021),
brought about by technological advances in the cooling and trapping of
magnetic lanthanide atoms Griesmaier  _et al._ (2005); Lu  _et al._ (2011);
Aikawa  _et al._ (2012); Phelps  _et al._ (2020); Chomaz  _et al._ (2022), and
heteronuclear polar molecules Sage  _et al._ (2005); Ni  _et al._ (2008);
Anderegg  _et al._ (2017); Voges  _et al._ (2020); Valtolina  _et al._ (2020).
Of note are recent experiments that have realized evaporative cooling in
3-dimensional polar molecular gases, made possible by electric field Li  _et
al._ (2021) or microwave shielding Anderegg  _et al._ (2021); Schindewolf  _et
al._ (2022). The observed high ratio of elastic to inelastic collision rates
permit long lived samples even at high densities, motivating study of dipolar
induced anisotropic phenomena deep in the hydrodynamic regime. Ref. Wang and
Bohn (2022) takes the first step in formulating a hydrodynamic model of
ultracold dipolar Bose gases, by deriving the transport tensor of thermal
conductivity using the Chapman-Enskog procedure Uehling (1934); Taxman (1958);
Monchick and Mason (1961); Chapman and Cowling (1990). Here we extend this
formulation by deriving the viscosity tensor for gases consisting of dipolar
constituents.

To construct these tensors, we assume molecular scattering cross sections due
to ideal point dipoles aligned in a particular direction in space, as
described in Ref. Bohn and Jin (2014). Fortunate for our analysis, the
effective long-ranged molecular interaction potential between microwave
shielded molecules is equivalent to the classical dipole potential Karman  _et
al._ (2022); Schindewolf  _et al._ (2022), permitting use of the collision
cross sections derived in Ref. Bohn and Jin (2014). The same is apparently not
strictly true for electric field shielded molecules Lassablière and Quéméner
(2022), but use of the same cross section has previously proven adequate in
describing cross-dimensional thermalization experiments Li  _et al._ (2021);
Wang and Bohn (2021).

These transport tensors then permit us to study the propagation of waves
through the dipolar gas via the dispersion relation Sommerfeld and Brillouin
(1960), derived from the equations of conservation and constitution. Although
extensively studied in quantum degenerate dipolar gases Wenzel  _et al._
(2018); Ticknor  _et al._ (2011); Block  _et al._ (2012); Andreev (2013);
Ronen and Bohn (2010), wave phenomena in their still thermal counterparts
remain less treated. A goal of this work is, therefore, also to motivate
deeper investigations of normal phase dipolar gases in the ultracold
community. These systems promise a vast variety of dynamical phenomena still
unexplored, for example anisotropies in flow behavior or in turbulence.

The remainder of this manuscript is organized as follows: In Sec. II, we
follow the procedure outlined in Ref. Wang and Bohn (2022) to derive the
transport tensor of viscosity from a microscopic theory of dipolar collisions.
A linearization of the hydrodynamic equations allow us to extract the
dispersion relation in Sec. III, where it is used to illustrate anisotropic
wave attenuation. Finally, discussions and concluding remarks are drawn in
sec. IV.

##  II  Anisotropic Viscosity

###  II.1 General

Whereas a dilute gas is described in terms of the distribution of molecular
velocities, a fluid in the hydrodynamic limit is described in terms of
macroscopic observables such as the density $\rho$, velocity $\bm{U}$, and
temperature $T$, all of which may vary in time and space. The governing
equations for these quantities, including in ultracold systems Nikuni and
Griffin (1998a), are the equations of conservation Uehling and Uhlenbeck
(1933):

$\displaystyle\frac{\partial\rho}{\partial t}+\partial_{j}\left({\rho
U_{j}}\right)=0,$ (1a) $\displaystyle\frac{\partial}{\partial t}\left(\rho
U_{i}\right)+\partial_{j}\left(\rho
U_{j}U_{i}\right)=\partial_{j}\sigma_{ij},$ (1b)
$\displaystyle\frac{\partial}{\partial t}(\rho T)+\partial_{j}\left(\rho
TU_{j}\right)=\frac{2m}{3k_{B}}\left(\sigma_{ij}\partial_{j}U_{i}-\partial_{j}J_{j}\right),$
(1c)

where $\partial_{i}$ denotes the partial derivative with respect to coordinate
$r_{i}$, and we invoke the summation conventions. The left-hand side of each
of these equations denotes the rate of change of the relevant quantity at a
given location in the fluid, including convective transport due to the fact
that the fluid is in motion. In the first, this derivative of the mass density
$\rho$ is zero; this is the equation of continuity. In the second equation,
the change of the momentum density $\rho\bm{U}$ is given by forces acting in
the fluid, thus giving the Navier-Stokes equation Navier (1823); Stokes
(2007). These forces are in turn given as gradients of the stress tensor
$\sigma$, denoting the force on a surface with normal vector
$\hat{\bm{r}}_{i}$, in direction $\hat{\bm{r}}_{j}$. These forces are
illustrated schematically in Figure 1, where all vectors appear to have the
same length, although this is not generally true in the fluid. In the third
equation of motion, for the mass-weighted temperature distribution, the rate
of change depends on both the stress tensor and on the heat conduction vector
$\bm{J}$.

The equations (1) are general in the absence of long-ranged forces between the
molecules, and are described in Refs. Bond  _et al._ (1965); Chapman and
Cowling (1990); De Groot and Mazur (2013). Additional external forces, such as
the trap confining the atoms, can be added as necessary. To apply the
equations of change to a particular fluid, such as the dipolar gas we have in
mind, requires some constitutive relations describing the fluid. For example,
the stress tensor is written

$\displaystyle\sigma_{ij}=-P\delta_{ij}+\tau_{ij}.$ (2)

Here $P$ is the thermodynamic pressure, related to the density and temperature
by the fluid’s equation of state, which we will take as the ideal gas law in
what follows. The remaining part, $\bm{\tau}$, is the viscous stress tensor,
i.e., the part arising from viscosity. In a Newtonian fluid the viscous shear
is assumed to be a linear function of the velocity gradients, so that
generally,

$\displaystyle\tau_{ij}=\mu_{ijk\ell}\;\partial_{\ell}U_{k},$ (3)

where $\bm{\mu}$ is in general a fourth-rank viscosity tensor and the flow
velocity gradients $\partial_{\ell}U_{k}$, characterize the rate of strain on
differential fluid volumes. We will see, however, that ultracold inelastic
collisions leave only a symmeterized portion of $\partial_{\ell}U_{k}$
relevant. Meanwhile, the heat conduction vector is assumed to be linearly
related to temperature gradients in accordance with Fourier’s law,

$\displaystyle J_{i}=-\kappa_{ij}\partial_{i}T,$ (4)

where $\bm{\kappa}$ is the thermal conductivity tensor,

The hydrodynamic relations of the dilute dipolar gas are therefore specified
once the tensors $\bm{\mu}$ and $\bm{\kappa}$ are determined. The thermal
conductivity $\bm{\kappa}$ was previously derived in Ref. Wang and Bohn
(2022). Here we turn our attention to the viscosity tensor $\bm{\mu}$.

###  II.2 Microscopic Theory

The hydrodynamic variables are given by the velocity-averaged quantities

$\displaystyle\rho(\bm{r},t)=mn(\bm{r},t)=\int d^{3}vf(\bm{r},\bm{v},t)m,$
(5a) $\displaystyle\bm{U}(\bm{r},t)=\frac{1}{n(\bm{r},t)}\int
d^{3}vf(\bm{r},\bm{v},t)\bm{v},$ (5b) $\displaystyle
T(\bm{r},t)=\frac{2}{3n(\bm{r},t)k_{B}}\int
d^{3}vf(\bm{r},\bm{v},t)\frac{1}{2}m\bm{u}^{2}.$ (5c)

where $f(\bm{r},\bm{v},t)$ denotes the phase space distribution of the
molecules. The so-called peculiar velocity
$\bm{u}(\bm{r})=\bm{v}-\bm{U}(\bm{r})$, is the velocity of molecules relative
to the local flow velocity.

Similarly, the stress tensor is defined microscopically by the integral

$\displaystyle\sigma_{ij}$ $\displaystyle=-m\int d^{3}uf(\bm{u})u_{i}u_{j}.$
(6)

This integral computes the mean flux of momentum $mu_{i}$ through a surface
with normal unit vector ${\hat{r}}_{j}$, thus describing a force per area on
that surface.

In the spirit of the perturbative method of Chapman and Enskog, the phase
space density is assumed to differ but little from its equilibrium value,

$\displaystyle f\approx f_{0}\left[1+\Phi\right],$ (7)

where $\Phi\ll 1$ and the equilibrium distribution of peculiar velocities is

$\displaystyle f_{0}(\bm{u},\beta)$
$\displaystyle=n_{0}(\beta)c_{0}(\bm{u},\beta)$
$\displaystyle=n_{0}(\beta)\left(\frac{m\beta}{2\pi}\right)^{3/2}\exp\left(-\frac{m\beta}{2}\bm{u}^{2}\right),$
(8)

$n_{0}$ is the gas equilibrium number density, $\beta=(k_{B}T)^{-1}$ and
$\bm{u}^{2}=u_{k}u_{k}$.

Reference Wang and Bohn (2022) argued, after a lengthy derivation based on
Bond  _et al._ (1965), that a suitable variational ansatz for $\Phi$ is given
in terms of gradients of temperature and velocity as

$\displaystyle\Phi=V_{l}b_{lk}\partial_{k}(\ln T)+2m\beta
W_{ij}a_{ijkl}D_{kl}.$ (9)

This expression is written in terms of the vector

$\displaystyle V_{i}(\bm{u})\equiv\left(\frac{\beta
m\bm{u}^{2}}{2}-\frac{5}{2}\right)u_{i},$ (10)

and symmetrized quantities

$\displaystyle W_{ij}(\bm{u})\equiv
u_{i}u_{j}-\frac{1}{3}\delta_{ij}\bm{u}^{2},$ (11a) $\displaystyle
D_{ij}(\bm{U})\equiv\frac{1}{2}\left(\partial_{j}U_{i}+\partial_{i}U_{j}\right)-\frac{1}{3}\delta_{ij}\partial_{k}U_{k},$
(11b)

where the coefficients $\bm{b}$ and $\bm{a}$ are to be determined
variationally.

The term in $\Phi$ involving $\ln T$ contributes to the thermal conductivity
and was evaluated in Ref. Wang and Bohn (2022). Here we focus on the other
term, in terms of which the stress tensor becomes

$\displaystyle\sigma_{ij}$ $\displaystyle=-m\int
d^{3}uf_{0}(\bm{u})\left[1+\Phi(\bm{u})\right]u_{i}u_{j}$ (12)
$\displaystyle=-\frac{n_{0}}{\beta}\delta_{ij}-2\left(m^{2}\beta\int
d^{3}uf_{0}(\bm{u})u_{i}u_{j}W_{mn}a_{mnk\ell}\right)D_{k\ell}.$

This expression identifies the thermodynamic pressure as $P=n_{0}/\beta$, and
the quantity in parentheses as related to the shear viscosity.

###  II.3 A Note on Symmetry

The viscosity tensor as defined by Eq. (3) gives the stresses as linear
combinations of the unsymmetrized second-rank tensor $\partial_{l}U_{k}$
giving the gradients of the fluid’s velocity components. By contrast, the
microscopic evaluation of stresses (II.2) from the Boltzmann equation, relates
these stresses only to the symmetrized tensor $\bm{D}$. The difference is
telling: generally this tensor can be reduced in the usual way into the
traceless, second-rank $\bm{D}$, along with an antisymmetric tensor $\bm{R}$
and a scalar,

$\displaystyle\partial_{\ell}U_{k}=D_{k\ell}+R_{k\ell}+\frac{1}{3}\delta_{k\ell}\gradient\cdot\bm{U},$
(13)

where

$\displaystyle
R_{k\ell}=\frac{1}{2}\left(\partial_{\ell}U_{k}-\partial_{k}U_{\ell}\right).$
(14)

The absence of the antisymmetric tensor and the scalar from the expression
connotes that there are no rotational viscosities, nor bulk viscosities in a
dilute gas of particles with no internal degrees of freedom Chapman and
Cowling (1990); Gad-el Hak (1995); Graves and Argrow (1999). Without loss of
generality, the viscous stress tensor can now be written in terms of just the
traceless symmetric flow-velocity gradient tensor (a.k.a. strain rate tensor)

$\displaystyle\tau_{ij}=2\mu_{ijk\ell}D_{k\ell}.$ (15)

Note that this conclusion is independent of the form of the collision cross
section of the molecules.

The relation between the two forms of the symmetrized tensors is conveniently
handled via a contraction,

$\displaystyle W_{ij}(\bm{u})={\cal
I}_{ijk\ell}u_{k}u_{\ell}=u_{k}u_{\ell}{\cal I}_{k\ell ij},$ (16a)
$\displaystyle D_{ij}(\bm{U})={\cal
I}_{ijk\ell}\partial_{\ell}U_{k}=\partial_{\ell}U_{k}{\cal I}_{k\ell ij},$
(16b)

with the traceless symmetric tensor

$\displaystyle{\cal
I}_{ijmn}=\frac{\delta_{im}\delta_{jn}+\delta_{in}\delta_{jm}}{2}-\frac{1}{3}\delta_{ij}\delta_{mn}.$
(17)

Written in these terms, the expression for the shear stress tensor is

$\displaystyle\tau_{ij}=-2m^{2}\beta\int d^{3}uf_{0}u_{i}u_{j}u_{m}u_{n}{\cal
I}_{mnop}a_{opk\ell}D_{k\ell}.$ (18)

The integrand now consists of products of components of the peculiar velocity
$\bm{u}$, multiplied by the known equilibrium velocity distribution. All such
integrals are readily evaluated (many leading to Kronecker delta functions),
whereby the viscosity tensor ultimately becomes

$\displaystyle\mu_{ijk\ell}=-\frac{2n_{0}}{\beta}{\cal I}_{ijmn}a_{mnk\ell},$
(19)

in terms of the variational parameters $\bm{a}$. These parameters must be
determined by an approximate solution of the Boltzmann equation.

###  II.4 Approximate Solution to the Boltzmann Equation

To this end, we start with the Boltzmann equation for the local phase space
distribution

$\displaystyle\left(\frac{\partial}{\partial
t}+v_{i}\partial_{i}\right)f(\bm{r},\bm{v},t)={\cal C}[f(\bm{r},\bm{v},t)],$
(20)

where ${\cal C}[f]$ is the two-body collision integral Reif (2009). We ignore
molecular finite-size effects required at higher densities, which would modify
the collision integral above Chapman and Cowling (1990); Hoffman and Curtiss
(1965). At the current experimental regime of interest (detailed in Sec. III),
we find that such effects only contribute $<5\%$ changes to the transport
coefficients.

Assuming the close-to-equilibrium phase space distribution of Eq. (7), we
arrive at the ansatz in Eq. (9) and the approximate Boltzmann equation

$\displaystyle f_{0}\>W_{k\ell}D_{k\ell}\approx
2C[f_{0}W_{ij}]a_{ijk\ell}D_{k\ell}.$ (21)

Note that this refers to the portion of the simplified equation that pertains
to viscosity, i.e., it does not include terms with temperature gradients.
Further details of this approximation are provided in Ref. Wang and Bohn
(2022).

Figure 1:  A visualization of the stresses (black arrows) on a differential
fluid volume element (red cube) due to thermodynamic pressure and gradients in
the velocity field.

To obtain explicit forms for the variational coefficients $a_{ijk\ell}$ and
hence the viscosities by Eq. (19), we rewrite the right-hand side of Eq. (21)
as

$\displaystyle 2C[f_{0}W_{mn}]a_{mnk\ell}D_{k\ell}$
$\displaystyle\quad\quad=2C[f_{0}W_{mn}]\left({\cal
I}_{mnrs}a_{rsk\ell}\right)D_{k\ell}$
$\displaystyle\quad\quad=-\frac{\beta}{n_{0}}C[f_{0}W_{mn}]\mu_{mnk\ell}D_{k\ell}.$
(22)

Multiplying both sides of Eq. (21) by $W_{ij}$ and integrating over $\bm{u}$
then gives

$\displaystyle T_{ijk\ell}=M_{ijmn}\mu_{mnk\ell}.$ (23)

where

$\displaystyle T_{ijk\ell}=\int
d^{3}uf_{0}(\bm{u})W_{ij}W_{k\ell}=\frac{2n_{0}}{(m\beta)^{2}}{\cal
I}_{ijk\ell},$ (24a) $\displaystyle M_{ijmn}=-\frac{\beta}{n_{0}}\int
d^{3}u\>W_{ij}{\cal C}[f_{0}W_{mn}].$ (24b)

The $M_{ijmn}$ integrals are particularly involved due to the highly
anisotropic differential cross section of dipolar Bosons, for which the
appropriately symmeterized scattering amplitude from Ref. Bohn and Jin (2014)
is given as

$\displaystyle
f_{B}\left(\hat{\bm{u}}^{\prime}_{r},\hat{\bm{u}_{r}}\right)=\frac{a_{d}}{\sqrt{2}}\left(\frac{4}{3}-\frac{2a_{s}}{a_{d}}-\frac{2(\hat{\bm{u}_{r}}\cdot\hat{\bm{{\cal
E}}})^{2}+2(\hat{\bm{u}}^{\prime}_{r}\cdot\hat{\bm{{\cal
E}}})^{2}-4(\hat{\bm{u}_{r}}\cdot\hat{\bm{{\cal
E}}})(\hat{\bm{u}}^{\prime}_{r}\cdot\hat{\bm{{\cal
E}}})(\hat{\bm{u}_{r}}\cdot\hat{\bm{u}}^{\prime}_{r})}{1-(\hat{\bm{u}_{r}}\cdot\hat{\bm{u}}^{\prime}_{r})^{2}}\right),$
(25)

where $a_{s}$ is the $s$-wave scattering length,
$\bm{u}_{\mathrm{r}}=\bm{u}-\bm{u}_{1}$ is the relative peculiar velocity
between colliding molecules, primes denote post-collision velocities and
$\hat{\bm{{\cal E}}}$ is the dipole alignment axis. This provides us the
differential cross section via
${d\sigma/d\Omega^{\prime}}=\absolutevalue{f_{B}\left(\hat{\bm{u}}^{\prime}_{r},\hat{\bm{u}_{r}}\right)}^{2}$,
with which to compute $\bm{M}$.

After evaluating the integrals in Eq. (24b), as detailed in App. A, obtaining
the $\bm{\mu}$ tensor now requires us to invert $\bm{M}$, which is most easily
performed by converting $\bm{M}$ into its matrix representation denoted by an
overhead circle, $\mathring{\bm{M}}$. This is done by mapping index pairs to
single indices $(i,j)\rightarrow(i^{\prime})$, via the relation

$\displaystyle i^{\prime}=3(j-1)+i,$ (26)

rendering $M_{ijk\ell}\rightarrow\mathring{M}_{i^{\prime}k^{\prime}}$. In its
$9\times 9$ matrix representation, the inherent symmetries of $\bm{M}$ reduces
its matrix rank from 9 to 5. This prevents us from inverting the matrix in its
current representation, so we are now required to perform a change of basis
transformation which decomposes the $9\times 9$ matrix into a block-diagonal
matrix with a $5\times 5$ irreducible block. The desired change of basis
matrix $\mathring{\bm{C}}$, is obtained by diagonalizing the isotropic tensor
$\bm{{\cal I}}$ in its matrix representation,

$\displaystyle\mathring{\bm{{\cal
I}}}=\mathring{\bm{C}}\left(\mathbb{I}_{5\times 5}\oplus\bm{0}_{4\times
4}\right)\mathring{\bm{C}}^{-1},$ (27)

where $\mathbb{I}$ and $\bm{0}$ are the identity and zero matrices
respectively, with dimensions specified by their subscripts, and $\oplus$
denotes a direct sum. Applying the transformation $\mathring{\bm{C}}$ to Eq.
(23) gives

$\displaystyle\mathring{\bm{C}}^{-1}\mathring{\bm{T}}\mathring{\bm{C}}$
$\displaystyle=\mathring{\bm{C}}^{-1}\left(\mathring{\bm{M}}\mathring{\bm{\mu}}\right)\mathring{\bm{C}}$
$\displaystyle=\left(\mathring{\bm{C}}^{-1}\mathring{\bm{M}}\mathring{\bm{C}}^{-1}\right)\left(\mathring{\bm{C}}\mathring{\bm{\mu}}\mathring{\bm{C}}\right),$
(28)

which leaves both sides of the equation above to only have a $5\times 5$ non-
trivial matrix block. The structure of these matrices are shown more
explicitly by writing

$\displaystyle\left[\mathring{\bm{C}}^{-1}\mathring{\bm{T}}\mathring{\bm{C}}\right]_{5\times
5}\oplus\bm{0}_{4\times 4}=$
$\displaystyle\quad\quad\quad\left[\left(\mathring{\bm{C}}^{-1}\mathring{\bm{M}}\mathring{\bm{C}}\right)\left(\mathring{\bm{C}}^{-1}\mathring{\bm{\mu}}\mathring{\bm{C}}\right)\right]_{5\times
5}\oplus\bm{0}_{4\times 4}.$ (29)

The direct sum with $\bm{0}_{4\times 4}$ is trivial, so we can just consider
the $5\times 5$ irreducible subspace. This now allows us to effectively invert
$\mathring{\bm{M}}$ by

$\displaystyle\left[\mathring{\bm{C}}^{-1}\mathring{\bm{\mu}}\mathring{\bm{C}}\right]_{5\times
5}$
$\displaystyle=\left[\left(\mathring{\bm{C}}^{-1}\mathring{\bm{M}}\mathring{\bm{C}}\right)\right]_{5\times
5}^{-1}\left[\left(\mathring{\bm{C}}^{-1}\mathring{\bm{T}}\mathring{\bm{C}}\right)\right]_{5\times
5},$ (30)

and taking the direct sum of the expression above with $\bm{0}_{4\times 4}$,
to give

$\displaystyle\mathring{\bm{\mu}}$
$\displaystyle=\mathring{\bm{C}}\left(\mathring{\bm{C}}^{-1}\mathring{\bm{M}}^{+}\mathring{\bm{C}}\right)\left(\mathring{\bm{C}}^{-1}\mathring{\bm{T}}\mathring{\bm{C}}\right)\mathring{\bm{C}}^{-1}=\mathring{\bm{M}}^{+}\mathring{\bm{T}},$
(31)

where $\mathring{\bm{M}}^{+}$ is a pseudo-inverse of $\mathring{\bm{M}}$
defined by the procedure above, which satisfies
$\mathring{\bm{M}}^{+}\mathring{\bm{M}}\mathring{\bm{\mu}}=\mathring{\bm{\mu}}$.
Finally, we apply the inverse mapping of Eq. (26) to attain the rank-4 tensor
form of $\bm{\mu}$. The explicit expressions for the viscosity coefficients
are highly complicated, but tabulated in App. A as a function of the dipole
angle $\Theta$, defined to be the angle between $\hat{\bm{{\cal E}}}$, and
$\hat{\bm{z}}$ in the $x,z$-plane.

###  II.5  Viscosities for Upright Dipoles

In the case where the dipole and $\hat{\bm{z}}$ axes coincide (i.e.
$\Theta=0$), we find that the stress tensor simplifies greatly and can be
written in a form more familiar in the usual theory of viscosity. That is, the
stress tensor can be decomposed into a “normal” part which includes a
proportionality with the symmetrized velocity gradients,

$\displaystyle\tau^{0}_{ij}=2\mu^{0}_{ij}\circ D_{ij},$ (32)

and 2 additional “anomalous” stress terms which modify the radial diagonal
stresses 111 Such an anomalous stress also arises in nematic liquid crystals,
where it is phenomenologically grouped together with the thermodynamic
pressure to give an effective pressure De Gennes and Prost (1993).

$\displaystyle\tau^{\prime}_{11}=\tau^{\prime}_{22}=2\mu^{\prime}D_{33}.$ (33)

The symbol $\circ$ denotes the Hadamard product (the element-wise product in
which repeated indices remain unsummed). The total stress is then written as
the sum of the above two parts $\tau_{ij}=\tau^{0}_{ij}+\tau^{\prime}_{ij}$,
with the corresponding viscosity coefficients in this representation given
explicitly as

$\displaystyle\mu^{0}_{13}=\mu^{0}_{23}$
$\displaystyle=\frac{\mu_{0}}{2(\mathscr{A}_{2}-3\mathscr{A}_{1}-{4\tilde{a}_{d}^{2}/63})},$
(34a) $\displaystyle\mu^{0}_{12}=\mu^{0}_{11}=\mu^{0}_{22}$
$\displaystyle=\frac{\mu_{0}}{2\mathscr{A}_{2}},$ (34b)
$\displaystyle\mu^{0}_{33}$
$\displaystyle=\frac{\mu_{0}}{2\mathscr{A}_{2}}+\frac{2\mu_{0}\mathscr{A}_{1}}{\mathscr{A}_{2}(\mathscr{A}_{2}-4\mathscr{A}_{1})},$
(34c) $\displaystyle\mu^{\prime}$
$\displaystyle=\frac{1}{2}\left(\mu^{0}_{11}-\mu^{0}_{33}\right).$ (34d)

Above, the $\bm{\mathscr{A}}$’s are polynomial functions of the reduced dipole
length $\tilde{a}_{d}\equiv a_{d}/a_{s}$, in units of the scattering length

$\displaystyle\mathscr{A}_{1}=\frac{4}{63}\tilde{a}_{d}^{2}-\frac{16}{21}\tilde{a}_{d},$
(35a)
$\displaystyle\mathscr{A}_{2}=\frac{32}{63}\tilde{a}_{d}^{2}-\frac{32}{21}\tilde{a}_{d}+4.$
(35b) The dipole length is defined as
$a_{d}=md^{2}/(8\pi\epsilon_{0}\hbar^{2})$ where $d$ is the electric dipole
moment, $m$ is the molecular mass and $\epsilon_{0}$ is the vacuum
permittivity. The viscosity coefficients are given in units of the Chapman-
Enskog result
$\displaystyle\mu_{0}=\frac{5}{16a_{s}^{2}}\sqrt{\frac{m}{\pi\beta}}.$ (36)

corresponding to the viscosity of a gas of hard spheres with diameter $a_{s}$
Chapman and Cowling (1990).

We remark in passing that the stress tensor in this representation remains
traceless, therefore the gas should not possess a bulk viscosity. This is a
feature expected of monatomic gases in general. It applies here since, at
ultralow temperatures, only the ground state of the molecule is accessible.

In the limit where the scattering length remains finite and the dipole length
goes to zero, we get $\mu^{\prime}=0$ and all the other coefficients reduce to
the same value,

$\displaystyle\mu^{0}_{ij}(a_{d}=0)=\frac{\mu_{0}}{8}=\frac{5}{128a_{s}^{2}}\sqrt{\frac{m}{\pi\beta}},$
(37)

which is 8 times smaller than $\mu_{0}$ since the $s$-wave scattering cross
section in Bose gases is $8\pi a_{s}^{2}$ (instead of $\pi a_{s}^{2}$ for
classical hard spheres), consistent with Ref. Kavoulakis  _et al._ (1998).

To illustrate the deviation from the isotropic result above as $a_{d}$
increases, we plot the the unique viscosities of Eq. (34) normalized by the
isotropic viscosity of Eq. (37):

$\displaystyle\eta^{0}_{ij}(a_{d})\equiv\frac{\mu^{0}_{ij}(a_{d})}{\mu^{0}_{ij}(a_{d}=0)},\quad\eta^{\prime}(a_{d})\equiv\frac{\mu^{\prime}(a_{d})}{\mu^{0}_{ij}(a_{d}=0)},$
(38)

as a function of $\tilde{a}_{d}$ for both positive scattering length (Fig. 2)
and negative scattering length (Fig. 3). To make these plots, we assume
bosonic NaK molecules with a scattering length of magnitude
$\absolutevalue{a_{s}}=500a_{0}$.

Figure 2:  The unique viscosity tensor elements with dipoles aligned along
$\hat{\bm{z}}$, normalized by the finite scattering length isotropic viscosity
of Eq. (37), plotted as a function of $a_{d}/a_{s}$ from 0 to 10. The
scattering length here is assumed positive ($a_{s}>0$).  Figure 3:  The unique
viscosity tensor elements with dipoles aligned along $\hat{\bm{z}}$,
normalized by the finite scattering length isotropic viscosity of Eq. (37),
plotted as a function of $a_{d}/a_{s}$ from 0 to 10. The scattering length
here is assumed negative ($a_{s}<0$).

On the microscopic level, the differential scattering of dipolar molecules
exhibits a large anisotropy due to the dipole interaction itself, compounded
by the interference of this scattering with the $s$-wave part characterized by
a scattering length. The way in which this interference plays out is via a
tremendous variation of the relative viscosity coefficients as $\tilde{a}_{d}$
is varied. Indeed, the ratio of $\mu^{0}_{33}$ to $\mu^{0}_{11}$ can vary from
0.36 to 9.3 based on the value and sign of $a_{s}$. Relative viscosities are
therefore highly tunable in laboratory experiments via the scattering length
with blue-detuned circularly polarized microwaves Lassablière and Quéméner
(2018) (also tunable in Lanthanide atoms via the multitude of Fano-Feshbach
resonances Tang  _et al._ (2015); Patscheider  _et al._ (2022)), leading to
phenomena yet to be explored.

Fig. 3 reveals that the anomalous viscosity is negative when $a_{s}<0$,
implying a negative proportionality between radial viscous stresses and axial
strain rates. Fortunately, this aberration does not imply an unwarranted
dynamical instability since the positive axial viscous stress ensures the bulk
fluid dilation remains identically zero.

##  III  Attenuation in Dipolar Gases

Having both the thermal conductivity and viscosity tensors now allows fluid
dynamical studies in ultracold dipolar systems. As a first application, we
analyze linear wave propagation through an initially uniform density gas. To
do so, the density, flow velocity, and temperature variations are written in
terms of fluctuations about their equilibrium distributions

$\displaystyle\rho(\bm{r},t)=\rho_{0}[1+\chi(\bm{r},t)],$ (39a) $\displaystyle
U_{i}(\bm{r},t)=v_{s}\xi_{i}(\bm{r},t),$ (39b) $\displaystyle
T(\bm{r},t)=T_{0}[1+\epsilon(\bm{r},t)].$ (39c)

where $v_{s}=\sqrt{{5/(3m\beta_{0})}}$ is the ideal gas speed of sound.
Plugging the form of Eq. (39) into Eq. (1) and assuming
$\chi,\>\xi_{i},\>\epsilon\ll 1$, allows a linearization to first-order in the
fluctuations which gives

$\displaystyle\frac{\partial\chi}{\partial t}\approx-
v_{s}{\partial_{j}\xi_{j}},$ (40a)
$\displaystyle\frac{\partial\xi_{i}}{\partial
t}\approx-\frac{3}{5}v_{s}\partial_{i}(\epsilon+\chi)+\frac{3\beta_{0}v_{s}}{5n_{0}}\partial_{j}\tau_{ij},$
(40b) $\displaystyle\frac{\partial\epsilon}{\partial
t}\approx-\frac{2}{3}v_{s}\partial_{j}\xi_{j}+\frac{2}{3n_{0}k_{B}}\kappa_{ij}\partial_{i}\partial_{j}\epsilon,$
(40c)

as detailed in App. B.

Solutions to Eqs. (40) are obtained by utilizing the planewave ansatz for each
dynamical variable, resulting in the system of equations

$\displaystyle\omega\chi\approx v_{s}K_{j}\xi_{j},$ (41a)
$\displaystyle\omega\xi_{i}\approx\frac{3}{5}v_{s}K_{i}(\epsilon+\chi)-\frac{i}{\rho_{0}}\mu_{ijk\ell}K_{j}K_{\ell}\xi_{k},$
(41b)
$\displaystyle\omega\epsilon\approx\frac{2}{3}v_{s}K_{j}\xi_{j}-\frac{2i}{3n_{0}k_{B}}\kappa_{ij}K_{i}K_{j}\epsilon,$
(41c)

Defining thermal conductivity and viscosity associated rates

$\displaystyle\Gamma=-\frac{2\kappa_{ij}}{3n_{0}k_{B}}K_{i}K_{j},$ (42a)
$\displaystyle\Lambda_{ik}=-\frac{\mu_{ijk\ell}}{\rho_{0}}K_{j}K_{\ell},$
(42b)

we get the linear system above written as the eigenvalue matrix equation

$\displaystyle\begin{pmatrix}0&v_{s}\bm{K}^{T}&0\\\
\frac{3}{5}v_{s}\bm{K}&i\bm{\Lambda}&\frac{3}{5}v_{s}\bm{K}\\\
0&\frac{2}{3}v_{s}\bm{K}^{T}&i\Gamma\end{pmatrix}\begin{pmatrix}\chi\\\
\bm{\xi}\\\ \epsilon\end{pmatrix}=\omega\begin{pmatrix}\chi\\\ \bm{\xi}\\\
\epsilon\end{pmatrix}.$ (43)

A dispersion relation is then obtained via the characteristic polynomial of
Eq. (43). Further analytical insight is gained by asserting only long
wavelength ($\lambda$) excitations and large densities, which allow us to
define the small parameter $\varepsilon=K_{0}L$, where $K_{0}=2\pi/\lambda$ is
the sourced-fixed wave-number, $L=(\overline{\sigma}n_{0})^{-1}$ is the
molecular mean-free path and $\overline{\sigma}={32\pi a_{d}^{2}/45}$ is the
angular averaged total cross section Bohn and Jin (2014). The dispersion
relation to first-order in $\varepsilon$ is then given as

$\displaystyle\omega^{3}-i\omega^{2}[\Gamma+\tr(\bm{\Lambda})]-\omega[v_{s}K]^{2}$
(44)
$\displaystyle\quad+\frac{i}{5}v_{s}^{2}\big{[}K_{x}^{2}(3\Gamma+5(\Lambda_{22}+\Lambda_{33}))$
$\displaystyle\quad\quad\quad\>\>\>+K_{y}^{2}(3\Gamma+5(\Lambda_{11}+\Lambda_{33}))$
$\displaystyle\quad\quad\quad\>\>\>+K_{z}^{2}(3\Gamma+5(\Lambda_{11}+\Lambda_{22}))$
$\displaystyle\quad\quad\quad\>\>\>-10(\Lambda_{12}K_{y}+\Lambda_{13}K_{z})K_{x}-10\Lambda_{23}K_{y}K_{z}\big{]}=0,$

where $\tr(\bm{\Lambda})$ denotes the trace of matrix $\bm{\Lambda}$.

To analyze the dispersion relation, we envision an experiment with a uniform
density sample of ultracold 23Na39K polar molecules. A wave generation source
of constant frequency $\nu=\frac{\omega}{2\pi}$, is then applied that
propagates waves along the $z$-direction. We focus our attention to the case
where the scattering length is zero, which emphasizes the universal dipolar
anisotropy while simplifying the viscosity and thermal conductivity
expressions. We then solve the dispersion relation for the wave vector $K$ as
functions of $\nu$ and $\Theta$, to first order in $\varepsilon$. This yields
2 pairs of $K$ solutions:

$\displaystyle
K_{1,\pm}(\omega,\Theta)=\pm\frac{\omega}{v_{s}}\left[1-\frac{i63\omega}{16384v_{s}a_{d}^{2}n_{0}}\sqrt{\frac{5}{3\pi}}\left(3\cos(4\Theta)-21\cos(2\Theta)-94\right)\right],$
(45a) $\displaystyle
K_{2,\pm}(\omega,\Theta)=\pm\sqrt{\frac{i\omega}{v_{s}a_{d}^{2}n_{0}}\sqrt{\frac{5}{3\pi}}}\left[\frac{16384a_{d}^{2}n_{0}\sqrt{\pi}+i(189\cos(4\Theta)-1323\cos(2\Theta)-5922)\omega\sqrt{m\beta_{0}}}{128\sqrt{105\left(6\cos(4\Theta)+3\cos(2\Theta)+7\right)}}\right].$
(45b)

The second solution pair $K_{2,\pm}$, has dominant imaginary terms which scale
as $\sqrt{K_{0}a_{d}^{2}n_{0}}\sim K_{0}/\sqrt{K_{0}L}$, causing corresponding
wave solutions to attenuate within length scales of order $\sqrt{L}$. We
therefore take it that these solutions do not support wave propagation.

As for $K_{1,\pm}$, these do support propagating waves with attenuation length
$r_{a}$, set by

$\displaystyle r_{a}$
$\displaystyle=\absolutevalue{\imaginary[K_{1,\pm}]}^{-1},$ (46)

and phase velocity

$\displaystyle v_{p}=\frac{\omega}{\absolutevalue{\real[K_{1,\pm}]}}.$ (47)

We see that to first-order in $\varepsilon$, the phase velocity is simply the
frequency-independent ideal gas speed of sound $v_{p}=v_{s}$, whereas the
attenuation length still retains a frequency dependence:

$\displaystyle
r_{a}(\omega)=\frac{1}{L}\left(\frac{v_{s}}{\omega}\right)^{2}\frac{512\sqrt{{15/\pi}}}{7(94+21\cos(2\Theta)-3\cos(4\Theta))}.$
(48)

Therefore, we plot the ratio of attenuation length to the source-fixed
wavelength $\lambda=\nu/v_{s}$ in Fig. 4 for $\Theta=0$ to $\pi$ and
$\nu=50$Hz to $250$Hz. This frequency range is chosen to ensure
$\varepsilon\approx 0.1$. The experimental parameters used to generate this
plot are listed in Tab. 1, intended to reflect relevant experiments with NaK
Voges  _et al._ (2020); Schindewolf  _et al._ (2022).

Fig. 4 showcases a clear variation of $r_{a}$ with the dipole angle $\Theta$,
indicating that the attenuation of waves are highly dependent on the direction
of propagation relative to the dipole orientation. In particular, waves that
travel parallel to the dipole orientation attenuate faster in this case than
those perpendicular to it. Moreover, the attenuation length is seen to
decrease at higher frequencies as occurs with acoustic waves in ordinary dry
air. The $\Theta$-dependence of wave attenuation is further made clear in Fig.
5, which plots the percentage extinction of the waveform
$\cos(\absolutevalue{\real[K_{1}]}r)\exp(-r/r_{a})\times 100\%$, as a function
of distance from the wave source $r$, for $\Theta=0$ and $\frac{\pi}{2}$ with
$\nu=150$Hz. Fig. 5 also plots the decay envelop with fainter colors for
clarity.

Table 1:  Table of parameter values utilized to generate the plots for Bosonic 23Na39K dipolar molecules. Da $=1.661\times 10^{-27}$ kg stands for Dalton (atomic mass unit), $a_{0}=5.292\times 10^{-11}$ m is the Bohr radius and D$=3.336\times 10^{-30}$ Cm is a Debye.  Parameter | Symbol | Value | Unit
---|---|---|---
Relative molecular mass, | Mr | 62 | Da
Effective electric dipole moment | $d_{\mathrm{eff}}$ | 0.75 | D
Dipole length, | $a_{d}$ | $4.95\times 10^{4}$ | $a_{0}$
Equilibrium number density, | $n_{0}$ | $5\times 10^{12}$ | cm-3
Equilibrium gas temperature, | $T_{0}$ | 250 | nK
Figure 4:  The attenuation length $r_{a}$, normalized by the sourced-fixed
wavelength $\lambda$, as a function of frequency $\nu$ (in Hertz, Hz) and
dipole alignment angle $\Theta$ (in radians, rad).  Figure 5:  The percentage
extinction as a function of distance from the source $r$, normalized by the
sourced-fixed wavelength $\lambda$, for $\Theta=0$ (dashed blue curve) and
$\Theta=\frac{\pi}{2}$ (solid green curve) with $\nu=150$Hz. The fainter
curves are the decay envelopes associated to each extinction curve.

We have thus far neglected long-range effects, which would modify the Navier-
Stokes equation by adding the gradient of a mean-field potential

$\displaystyle U_{\mathrm{mf}}(\bm{r})=\int
d^{3}r^{\prime}n(\bm{r}^{\prime})U_{\mathrm{dd}}(\bm{r}-\bm{r}^{\prime}),$
(49) where $\displaystyle
U_{\mathrm{dd}}(\bm{r})=\frac{d^{2}}{4\pi\epsilon_{0}}\left(\frac{1-3(\hat{\bm{r}}\cdot\hat{\bm{{\cal
E}}})}{r^{3}}\right)$ (50)

is the potential between 2 point electric dipoles. Close to uniform density
with $\chi\approx 0.1$, the experimental parameters of Tab. 1 yield
$U_{\mathrm{mf}}/k_{B}T\approx 0.01$, validating this approximation.

As an added remark, we find that the viscosities of ultracold NaK molecules
presented here are around $10^{11}$ times less than that of ordinary dry air
at room temperature, $\mu_{\mathrm{air}}(T=300\text{K})\approx
18.5\>\mu$Pa$\cdot$s.

##  IV  Conclusions and Outlooks

Recent experiments have demonstrated the ability to shield ultracold molecules
from inelastic collisional losses, allowing long-lived dense samples of highly
dipolar gases. As these molecular gases enter the hydrodynamic regime, a
continuum model which includes the transport tensors of thermal conductivity
and viscosity is warranted to describe the fluid dynamics. The thermal
conductivities have been derived in Ref. Wang and Bohn (2022), and viscosities
in this work, which now permit comprehensive phenomenological studies of
Bosonic fluid systems that are enriched by dipolar anisotropy.

As a first analysis, we looked at how the dipole orientation dependence of our
derived transport tensors carry over into the attenuation of linear waves
generated by a constant frequency source. We find that attenuation is most
pronounced when the directions of wave propagation and dipole alignment
coincide (i.e. $\Theta=0$), whereas least attenuation occurs in the orthogonal
configuration (i.e. $\Theta=\frac{\pi}{2}$). These results are illustrated
with plots of the attenuation length and percentage extinction, that show a
significant variation of these quantities with $\Theta$. Experiments which
measure the attenuation length as a function of the dipole orientation could
therefore provide an experimental means to extracting the anisotropic
transport tensor coefficients.

In the future, higher density corrections to the derived transport tensors can
be included using the generalized Chapman-Enskog method Chapman and Cowling
(1990); Hoffman and Curtiss (1965). This modification would result in the
emergence of a bulk viscosity and explorations of hydrodynamic phenomena not
present in the current theory. At lower temperatures, the inclusion of quantum
statistical effects Uehling and Uhlenbeck (1933) to our derived transport
tensors could permit a normal-superfluid phase coupled hydrodynamic model for
dipolar systems, extending the formalism established by Zaremba, Griffin and
Nikuni Nikuni and Griffin (1998b); Zaremba  _et al._ (1999); Nikuni  _et al._
(2000). The mechanism to which dipolar gases crossover from the dilute to
hydrodynamic regimes might also be of interest to the ultracold community,
such as in the context of evaporative cooling. Such a theory would interpolate
the formulation presented in this work with that in Refs. Wang  _et al._
(2020); Wang and Bohn (2021). Finally, the results of this work can also be
extended to systems of Fermionic polar molecules and lanthanide atoms, by
utilizing the scattering cross section for identical Fermions found in Ref.
Bohn and Jin (2014).

###### Acknowledgements.

This work is supported by the National Science Foundation under Grant Number
PHY-2110327. The authors thank Eli Halperin for engaging discussions on the
dispersion relations and their physical interpretations.

##  Appendix A  Evaluation of the Viscosity Tensor

To obtain the viscosity tensor, the collision integrals of Eq. (24b) must be
evaluated. These integrals are evaluated with methods identical to those
described in Ref. Wang and Bohn (2022). However, the integrand differs
slightly where instead of

$\displaystyle N_{ik}\equiv\frac{2m\beta}{5n_{0}}\int
d^{3}u\>V_{i}C[f_{0}V_{k}],$ (51)

we are now evaluating

$\displaystyle M_{ijmn}=-\frac{\beta}{2n_{0}}\int d^{3}u\>W_{ij}{\cal
C}[f_{0}W_{mn}].$ (52)

The collision integral is therefore similarly set-up by rewriting it in terms
of center-of-mass (COM) and relative (r) velocity coordinates,
$\bm{u}_{\mathrm{COM}}$ and $\bm{u}_{\mathrm{r}}$ respectively, which renders
the product of equilibrium distributions as

$\displaystyle
f_{0}(\bm{u})f_{0}(\bm{u}_{1})=f_{\mathrm{COM}}(\bm{u}_{\mathrm{COM}})f_{\mathrm{r}}(\bm{u}_{\mathrm{r}}).$
(53)

Expanding the collision integral and writing it in terms of the COM and r
coordinates leaves us with

$\displaystyle M_{ijmn}$ $\displaystyle=-\frac{\beta}{2n_{0}}\int
d^{3}{u}_{\mathrm{COM}}f_{\mathrm{COM}}(\bm{u}_{\mathrm{COM}})$ (54)
$\displaystyle\quad\quad\>\>\>\times\int
d^{3}{u}_{\mathrm{r}}\>{u}_{\mathrm{r}}f_{\mathrm{r}}(\bm{u}_{\mathrm{r}})W_{ij}\int
d\Omega^{\prime}\frac{d\sigma}{d\Omega^{\prime}}\Delta W_{mn}.$

The collision-varied quantity is written as

$\displaystyle\Delta W_{ij}=\Delta(u_{i}u_{j})$
$\displaystyle=\frac{1}{2}\left(u^{\prime}_{r,i}u^{\prime}_{r,j}-u_{r,i}u_{r,j}\right),$
(55)

so integral over post-collision velocities with the differential cross section
for dipolar Bosons given in Ref. Bohn and Jin (2014), becomes

$\displaystyle I_{\Omega^{\prime}}$ $\displaystyle\equiv\frac{1}{2}\int
d\Omega^{\prime}_{r}\frac{d\sigma}{d\Omega^{\prime}_{r}}\left(u^{\prime}_{r,i}u^{\prime}_{r,j}-u_{r,i}u_{r,j}\right).$
(56)

We then utilize Mathematica Wolfram Research, Inc. (2022) to evaluate and plug
the above integral back into Eq. (54) to obtain the elements of $M_{ijmn}$.
Finally, we follow the procedure described in Sec. II to obtain the viscosity
tensor elements. For brevity of presentation, we divide each viscosity by the
isotropic viscosity as derived by Chapman and Enskog Chapman and Cowling
(1990), denoted by a tilde:

$\displaystyle\tilde{\mu}_{ijk\ell}\equiv\mu_{ijk\ell}/\mu_{0}.$ (57)

The 13 unit-free, unique and non-trivial viscosity tensor elements are
tabulated below as a function of scattering length $a_{s}$, dipole length
$a_{d}$ and dipole orientation angle $\Theta$:

$\displaystyle\tilde{\mu}_{1111}=\frac{21}{128\mathscr{P}_{1}\mathscr{P}_{2}\mathscr{P}_{3}}\left[9\mathscr{P}_{1}\mathscr{P}_{2}+11\mathscr{P}_{1}\mathscr{P}_{3}+12\mathscr{P}_{2}\mathscr{P}_{3}-48\mathscr{O}_{1}\mathscr{P}_{1}\cos(2\Theta)+12\mathscr{O}_{3}\cos(4\Theta)\right],$
(58a)
$\displaystyle\tilde{\mu}_{1113}=\frac{63}{32\mathscr{P}_{1}\mathscr{P}_{2}\mathscr{P}_{3}}\left[2\mathscr{O}_{1}\mathscr{P}_{1}\sin(2\Theta)-\mathscr{O}_{3}\sin(4\Theta)\right],$
(58b)
$\displaystyle\tilde{\mu}_{1122}=-\frac{21}{8\mathscr{P}_{2}\mathscr{P}_{3}}\left[1+\mathscr{P}_{1}-3\mathscr{O}_{1}\cos(2\Theta)\right],$
(58c)
$\displaystyle\tilde{\mu}_{1133}=\frac{21}{128\mathscr{P}_{1}\mathscr{P}_{2}\mathscr{P}_{3}}\left[3\mathscr{P}_{1}\mathscr{P}_{2}-7\mathscr{P}_{1}\mathscr{P}_{3}-12\mathscr{P}_{2}\mathscr{P}_{3}-12\mathscr{O}_{3}\cos(4\Theta)\right],$
(58d)
$\displaystyle\tilde{\mu}_{1221}=\frac{63}{32\mathscr{P}_{1}\mathscr{P}_{3}}\left[\mathscr{P}_{1}+\mathscr{P}_{3}-4\mathscr{O}_{2}\cos(2\Theta)\right],$
(58e)
$\displaystyle\tilde{\mu}_{1223}=\frac{63}{8\mathscr{P}_{1}\mathscr{P}_{3}}\mathscr{O}_{2}\sin(2\Theta),$
(58f)
$\displaystyle\tilde{\mu}_{1331}=\frac{63}{128\mathscr{P}_{1}\mathscr{P}_{2}\mathscr{P}_{3}}\left[\mathscr{P}_{1}\mathscr{P}_{2}+3\mathscr{P}_{1}\mathscr{P}_{3}+4\mathscr{P}_{2}\mathscr{P}_{3}-4\mathscr{O}_{3}\cos(4\Theta)\right],$
(58g)
$\displaystyle\tilde{\mu}_{1322}=-\frac{63}{8\mathscr{P}_{2}\mathscr{P}_{3}}\mathscr{O}_{1}\sin(2\Theta),$
(58h)
$\displaystyle\tilde{\mu}_{1333}=\frac{63}{16\mathscr{P}_{1}\mathscr{P}_{2}\mathscr{P}_{3}}\left[\mathscr{O}_{1}\mathscr{P}_{1}+\mathscr{O}_{3}\cos(2\Theta)\right]\sin(2\Theta),$
(58i)
$\displaystyle\tilde{\mu}_{2222}=\frac{21}{16\mathscr{P}_{2}\mathscr{P}_{3}}\left(3\mathscr{P}_{2}+\mathscr{P}_{3}\right),$
(58j)
$\displaystyle\tilde{\mu}_{2233}=-\frac{21}{8\mathscr{P}_{2}\mathscr{P}_{3}}\left[1+\mathscr{P}_{1}+3\mathscr{O}_{1}\cos(2\Theta)\right],$
(58k)
$\displaystyle\tilde{\mu}_{2332}=\frac{63}{32\mathscr{P}_{1}\mathscr{P}_{3}}\left[\mathscr{P}_{1}+\mathscr{P}_{3}+4\mathscr{O}_{2}\cos(2\Theta)\right],$
(58l)
$\displaystyle\tilde{\mu}_{3333}=\frac{21}{128\mathscr{P}_{1}\mathscr{P}_{2}\mathscr{P}_{3}}\left[9\mathscr{P}_{1}\mathscr{P}_{2}+11\mathscr{P}_{1}\mathscr{P}_{3}+12\mathscr{P}_{2}\mathscr{P}_{3}+48\mathscr{O}_{1}\mathscr{P}_{1}\cos(2\Theta)+12\mathscr{O}_{3}\cos(4\Theta)\right],$
(58m)

having defined the adimensonal polynomials of reduced dipole length
$\tilde{a}_{d}=a_{d}/a_{s}$,

$\displaystyle\mathscr{O}_{1}=\tilde{a}_{d}^{2}-12\tilde{a}_{d},$ (59a)
$\displaystyle\mathscr{O}_{2}=\tilde{a}_{d}^{2}-9\tilde{a}_{d},$ (59b)
$\displaystyle\mathscr{O}_{3}=369-60\tilde{a}_{d}-4\tilde{a}_{d}^{2},$ (59c)
$\displaystyle\mathscr{P}_{1}=63+12\tilde{a}_{d}+4\tilde{a}_{d}^{2},$ (59d)
$\displaystyle\mathscr{P}_{2}=63+24\tilde{a}_{d}+4\tilde{a}_{d}^{2},$ (59e)
$\displaystyle\mathscr{P}_{3}=63-24\tilde{a}_{d}+8\tilde{a}_{d}^{2}.$ (59f)

The latter 3 polynomials above that appear as denominators in the viscosities,
$\mathscr{P}_{1},\mathscr{P}_{2}$ and $\mathscr{P}_{3}$, do not have real
roots for any combination of $a_{s},a_{d}>0$, preventing unphysical poles.

Other non-trivial viscosity terms are specified by the tensor symmetry
identities

$\displaystyle\mu_{ijmn}=\mu_{jimn},$ (60a)
$\displaystyle\mu_{ijmn}=\mu_{jinm},$ (60b)
$\displaystyle\mu_{ijmn}=\mu_{mnij},$ (60c)
$\displaystyle\mu_{ijmn}\delta_{ij}=0,$ (60d)
$\displaystyle\mu_{ijmn}\delta_{mn}=0,$ (60e)
$\displaystyle\mu_{ijmn}\delta_{ijmn}=0,$ (60f)

where $\delta_{ijmn}$ is 1 if $i=j=k=\ell$ and $0$ otherwise. Repeated indices
are summed over. All other unspecified tensor elements are zero.

##  Appendix B  Linearizing the Balance Equations

This section of the appendix details the linearization of the balance
equations with the variational forms in Eq. (39). First, we shall take that
the thermodynamic pressure to be given by the ideal gas law

$\displaystyle P=\frac{n}{\beta}.$ (61)

Then starting with the continuity equation, we have

$\displaystyle\frac{\partial\rho}{\partial t}+\partial_{j}\left({\rho
U_{j}}\right)=0,$ $\displaystyle\Rightarrow\quad$
$\displaystyle\rho_{0}\frac{\partial\chi}{\partial
t}+\rho_{0}v_{s}\left({\partial_{j}\xi_{j}}+{\xi_{j}\partial_{j}\chi}\right)=0,$
$\displaystyle\Rightarrow\quad$ $\displaystyle\frac{\partial\chi}{\partial
t}+\sqrt{\frac{5}{3m\beta_{0}}}{\partial_{j}\xi_{j}}\approx 0.$ (62)

The Navier-Stokes equation is then

$\displaystyle\frac{\partial}{\partial t}\left(\rho
U_{i}\right)+\partial_{j}\left(\rho
U_{j}U_{i}\right)=-\partial_{i}P+\partial_{j}\tau_{ij},$
$\displaystyle\Rightarrow\quad$ $\displaystyle
v_{s}\rho_{0}\left((1+\chi)\frac{\partial\xi_{i}}{\partial
t}+\xi_{i}\frac{\partial\chi}{\partial t}\right)$
$\displaystyle\quad+v_{s}^{2}\rho_{0}\Big{(}\partial_{j}(\xi_{j}\xi_{i})+\xi_{j}\xi_{i}\partial_{j}\chi\Big{)}$
$\displaystyle\quad=-\frac{n_{0}}{\beta_{0}}(1+\epsilon)\partial_{i}\chi-\frac{n_{0}}{\beta_{0}}(1+\chi)\partial_{i}\epsilon+\partial_{j}\tau_{ij},$
$\displaystyle\Rightarrow\quad$ $\displaystyle\frac{\partial\xi_{i}}{\partial
t}\approx-\sqrt{\frac{3}{5m\beta_{0}}}\partial_{i}(\epsilon+\chi)+\frac{1}{n_{0}}\sqrt{\frac{3\beta_{0}}{5m}}\partial_{j}\tau_{ij}.r_{i},$
(63)

Finally, we have the energy balance equation as

$\displaystyle\frac{\partial}{\partial t}(\rho T)+\partial_{j}\left(\rho
TU_{j}\right)=\frac{2m}{3k_{B}}(\sigma_{ij}\partial_{j}U_{i}-\partial_{j}J_{j}),$
$\displaystyle\Rightarrow\quad$
$\displaystyle\rho_{0}T_{0}\left((1+\chi)\frac{\partial\epsilon}{\partial
t}+(1+\epsilon)\frac{\partial\chi}{\partial t}\right)$
$\displaystyle\quad+v_{s}\rho_{0}T_{0}\Big{[}(1+\chi)(1+\epsilon)\partial_{j}\xi_{j}$
$\displaystyle\quad\quad\quad+(1+\chi)\xi_{j}\partial_{j}\epsilon+(1+\epsilon)\xi_{j}\partial_{j}\chi\Big{]}$
$\displaystyle\quad=\frac{2m}{3k_{B}}(v_{s}\sigma_{ij}\partial_{j}\xi_{i}-\partial_{j}J_{j}),$
$\displaystyle\Rightarrow\quad$ $\displaystyle\frac{\partial}{\partial
t}\left(\epsilon+\chi\right)+\frac{5}{3}\sqrt{\frac{5}{3m\beta_{0}}}\partial_{j}\xi_{j}\approx-\frac{2}{3}\frac{\beta_{0}}{n_{0}}\partial_{j}J_{j},$
$\displaystyle\Rightarrow\quad$ $\displaystyle\frac{\partial\epsilon}{\partial
t}+\frac{2}{3}\sqrt{\frac{5}{3m\beta_{0}}}\partial_{j}\xi_{j}\approx-\frac{2}{3}\frac{\beta_{0}}{n_{0}}\partial_{j}J_{j}.$
(64)

In summary these grant us the closed set of equations in Eq. (40) of the main
text.

## References

  * Bohn  _et al._ (2017) J. L. Bohn, A. M. Rey, and J. Ye, Science 357, 1002 (2017).
  * Moses  _et al._ (2017) S. A. Moses, J. P. Covey, M. T. Miecnikowski, D. S. Jin, and J. Ye, Nature Physics 13, 13 (2017).
  * Liu and Luo (2021) Y. Liu and L. Luo, Frontiers of Physics 16, 1 (2021).
  * Griesmaier  _et al._ (2005) A. Griesmaier, J. Werner, S. Hensler, J. Stuhler, and T. Pfau, Phys. Rev. Lett. 94, 160401 (2005).
  * Lu  _et al._ (2011) M. Lu, N. Q. Burdick, S. H. Youn, and B. L. Lev, Phys. Rev. Lett. 107, 190401 (2011).
  * Aikawa  _et al._ (2012) K. Aikawa, A. Frisch, M. Mark, S. Baier, A. Rietzler, R. Grimm, and F. Ferlaino, Phys. Rev. Lett. 108, 210401 (2012).
  * Phelps  _et al._ (2020) G. A. Phelps, A. Hébert, A. Krahn, S. Dickerson, F. Öztürk, S. Ebadi, L. Su, and M. Greiner, Sub-second production of a quantum degenerate gas (2020).
  * Chomaz  _et al._ (2022) L. Chomaz, I. Ferrier-Barbut, F. Ferlaino, B. Laburthe-Tolra, B. L. Lev, and T. Pfau, Dipolar physics: A review of experiments with magnetic quantum gases (2022), arXiv:2201.02672 [cond-mat.quant-gas] .
  * Sage  _et al._ (2005) J. M. Sage, S. Sainis, T. Bergeman, and D. DeMille, Phys. Rev. Lett. 94, 203001 (2005).
  * Ni  _et al._ (2008) K.-K. Ni, S. Ospelkaus, M. H. G. de Miranda, A. Pe’er, B. Neyenhuis, J. J. Zirbel, S. Kotochigova, P. S. Julienne, D. S. Jin, and J. Ye, Science 322, 231 (2008).
  * Anderegg  _et al._ (2017) L. Anderegg, B. L. Augenbraun, E. Chae, B. Hemmerling, N. R. Hutzler, A. Ravi, A. Collopy, J. Ye, W. Ketterle, and J. M. Doyle, Phys. Rev. Lett. 119, 103201 (2017).
  * Voges  _et al._ (2020) K. K. Voges, P. Gersema, M. Meyer zum Alten Borgloh, T. A. Schulze, T. Hartmann, A. Zenesini, and S. Ospelkaus, Phys. Rev. Lett. 125, 083401 (2020).
  * Valtolina  _et al._ (2020) G. Valtolina, K. Matsuda, W. G. Tobias, J.-R. Li, L. De Marco, and J. Ye, Nature 588, 239 (2020).
  * Li  _et al._ (2021) J.-R. Li, W. G. Tobias, K. Matsuda, C. Miller, G. Valtolina, L. De Marco, R. R. Wang, L. Lassablière, G. Quéméner, J. L. Bohn, _et al._ , Nature Physics 17, 1144 (2021).
  * Anderegg  _et al._ (2021) L. Anderegg, S. Burchesky, Y. Bao, S. S. Yu, T. Karman, E. Chae, K.-K. Ni, W. Ketterle, and J. M. Doyle, Science 373, 779 (2021).
  * Schindewolf  _et al._ (2022) A. Schindewolf, R. Bause, X.-Y. Chen, M. Duda, T. Karman, I. Bloch, and X.-Y. Luo, Nature 607, 677 (2022).
  * Wang and Bohn (2022) R. R. W. Wang and J. L. Bohn, Thermal conductivity of an ultracold paramagnetic bose gas (2022), arXiv:2205.10465 [cond-mat.quant-gas] .
  * Uehling (1934) E. A. Uehling, Phys. Rev. 46, 917 (1934).
  * Taxman (1958) N. Taxman, Phys. Rev. 110, 1235 (1958).
  * Monchick and Mason (1961) L. Monchick and E. Mason, The Journal of Chemical Physics 35, 1676 (1961).
  * Chapman and Cowling (1990) S. Chapman and T. G. Cowling,  _The mathematical theory of non-uniform gases: an account of the kinetic theory of viscosity, thermal conduction and diffusion in gases_ (Cambridge university press, 1990).
  * Bohn and Jin (2014) J. L. Bohn and D. S. Jin, Phys. Rev. A 89, 022702 (2014).
  * Karman  _et al._ (2022) T. Karman, Z. Z. Yan, and M. Zwierlein, Phys. Rev. A 105, 013321 (2022).
  * Lassablière and Quéméner (2022) L. Lassablière and G. Quéméner, Model for two-body collisions between ultracold dipolar molecules around a förster resonance in an electric field (2022).
  * Wang and Bohn (2021) R. R. W. Wang and J. L. Bohn, Phys. Rev. A 103, 063320 (2021).
  * Sommerfeld and Brillouin (1960) A. Sommerfeld and L. Brillouin,  _Wave propagation and group velocity_ (Academic Press, New York, 1960).
  * Wenzel  _et al._ (2018) M. Wenzel, F. Böttcher, J.-N. Schmidt, M. Eisenmann, T. Langen, T. Pfau, and I. Ferrier-Barbut, Phys. Rev. Lett. 121, 030401 (2018).
  * Ticknor  _et al._ (2011) C. Ticknor, R. M. Wilson, and J. L. Bohn, Phys. Rev. Lett. 106, 065301 (2011).
  * Block  _et al._ (2012) J. K. Block, N. T. Zinner, and G. M. Bruun, New Journal of Physics 14, 105006 (2012).
  * Andreev (2013) P. Andreev, Russian Physics Journal 55, 1190 (2013).
  * Ronen and Bohn (2010) S. Ronen and J. L. Bohn, Phys. Rev. A 81, 033601 (2010).
  * Nikuni and Griffin (1998a) T. Nikuni and A. Griffin, Journal of Low Temperature Physics 111, 793 (1998a).
  * Uehling and Uhlenbeck (1933) E. A. Uehling and G. E. Uhlenbeck, Phys. Rev. 43, 552 (1933).
  * Navier (1823) C. Navier, Mem Acad. Sci. Inst. France 1, 414 (1823).
  * Stokes (2007) G. G. Stokes, in  _Classics of Elastic Wave Theory_ (Society of Exploration Geophysicists, 2007).
  * Bond  _et al._ (1965) J. W. Bond, K. M. Watson, and J. A. Welch,  _Atomic theory of gas dynamics_ , Vol. 633 (Addison-Wesley, 1965).
  * De Groot and Mazur (2013) S. De Groot and P. Mazur,  _Non-Equilibrium Thermodynamics_, Dover Books on Physics (Dover Publications, 2013).
  * Gad-el Hak (1995) M. Gad-el Hak, Journal of Fluids Engineering 117, 3 (1995).
  * Graves and Argrow (1999) R. E. Graves and B. M. Argrow, Journal of Thermophysics and Heat Transfer 13, 337 (1999).
  * Reif (2009) F. Reif,  _Fundamentals of statistical and thermal physics_ (Waveland Press, 2009).
  * Hoffman and Curtiss (1965) D. K. Hoffman and C. Curtiss, The Physics of Fluids 8, 667 (1965).
  * Note (1) Such an anomalous stress also arises in nematic liquid crystals, where it is phenomenologically grouped together with the thermodynamic pressure to give an effective pressure De Gennes and Prost (1993).
  * Kavoulakis  _et al._ (1998) G. M. Kavoulakis, C. J. Pethick, and H. Smith, Phys. Rev. A 57, 2938 (1998).
  * Lassablière and Quéméner (2018) L. Lassablière and G. Quéméner, Phys. Rev. Lett. 121, 163402 (2018).
  * Tang  _et al._ (2015) Y. Tang, A. Sykes, N. Q. Burdick, J. L. Bohn, and B. L. Lev, Phys. Rev. A 92, 022703 (2015).
  * Patscheider  _et al._ (2022) A. Patscheider, L. Chomaz, G. Natale, D. Petter, M. J. Mark, S. Baier, B. Yang, R. R. W. Wang, J. L. Bohn, and F. Ferlaino, Phys. Rev. A 105, 063307 (2022).
  * Nikuni and Griffin (1998b) T. Nikuni and A. Griffin, Journal of Low Temperature Physics 111, 793 (1998b).
  * Zaremba  _et al._ (1999) E. Zaremba, T. Nikuni, and A. Griffin, Journal of Low Temperature Physics 116, 277 (1999).
  * Nikuni  _et al._ (2000) T. Nikuni, A. Griffin, and E. Zaremba, Canadian Journal of Physics 78, 415 (2000), https://doi.org/10.1139/p00-043 .
  * Wang  _et al._ (2020) R. R. W. Wang, A. G. Sykes, and J. L. Bohn, Phys. Rev. A 102, 033336 (2020).
  * Wolfram Research, Inc. (2022) Wolfram Research, Inc.,  _Mathematica 12.0_ (2022).
  * De Gennes and Prost (1993) P.-G. De Gennes and J. Prost,  _The physics of liquid crystals_, 83 (Oxford university press, 1993).