# Dilated convolution with learnable spacings

Ismail Khalfaoui-Hassani
Artificial and Natural Intelligence Toulouse Institute (ANITI)
Université de Toulouse, France
<EMAIL_ADDRESS>
Thomas Pellegrini
IRIT, ANITI, Université de Toulouse,
CNRS, Toulouse INP, UT3, France
<EMAIL_ADDRESS>&Timothée Masquelier
CerCo UMR 5549
CNRS & Université de Toulouse, France
<EMAIL_ADDRESS>
###### Abstract

Recent works indicate that convolutional neural networks (CNN) need large
receptive fields (RF) to compete with visual transformers and their attention
mechanism. In CNNs, RFs can simply be enlarged by increasing the convolution
kernel sizes. Yet the number of trainable parameters, which scales
quadratically with the kernel’s size in the 2D case, rapidly becomes
prohibitive, and the training is notoriously difficult. This paper presents a
new method to increase the RF size without increasing the number of
parameters. The dilated convolution (DC) has already been proposed for the
same purpose. DC can be seen as a convolution with a kernel that contains only
a few non-zero elements placed on a regular grid. Here we present a new
version of the DC in which the spacings between the non-zero elements, or
equivalently their positions, are no longer fixed but learnable via
backpropagation thanks to an interpolation technique. We call this method
“Dilated Convolution with Learnable Spacings” (DCLS) and generalize it to the
n-dimensional convolution case. However, our main focus here will be on the 2D
case for computer vision only. We first tried our approach on ResNet50: we
drop-in replaced the standard convolutions with DCLS ones, which increased the
accuracy of ImageNet1k classification at iso-parameters, but at the expense of
the throughput. Next, we used the recent ConvNeXt state-of-the-art
convolutional architecture and drop-in replaced the depthwise convolutions
with DCLS ones. This not only increased the accuracy of ImageNet1k
classification but also of typical downstream and robustness tasks, again at
iso-parameters but this time with negligible cost on throughput, as ConvNeXt
uses separable convolutions. Conversely, classic DC led to poor performance
with both ResNet50 and ConvNeXt. The code of the method is based on PyTorch
and available.

##  1 Introduction

The receptive field of a deep convolutional network is a crucial element to
consider when dealing with recognition and downstream tasks in computer
vision. For instance, a logarithmic relationship between classification
accuracy and receptive field size was observed in Araujo et al. (2019). This
tells us that large receptive fields are necessary for high-level vision
tasks, but with logarithmically decreasing rewards and thus a higher
computational cost to reach them.

Recent advances in vision transformers (Dosovitskiy et al., 2020) and in CNNs
(Liu et al., 2022b; Ding et al., 2022; Trockman & Kolter, 2022; Liu et al.,
2022a) highlight the beneficial effect that a large convolution kernel can
have, compared to the $3\times 3$ kernels traditionally used in previous
state-of-the-art CNN models (He et al., 2016). However, when naively
increasing the kernel size, the accuracy rapidly plateaus or even decreases.
For example, in ConvNeXt, the best accuracy was achieved by a $7\times 7$
kernel (Liu et al., 2022b; a). Using a structural re-parameterization trick,
Ding et al. (2022) demonstrated the benefit of increasing the kernel size up
to 31 by 31. Thereafter, Liu et al. (2022a) showed that there was still room
for improvement by moving to 51 by 51, using the depthwise implicit matrix
multiplication (gemm) method developed by Ding et al. (2022) and for which the
implementation has been integrated into the open-sourced framework MegEngine
(Megvii, 2020), in addition to a spatial separation of the depthwise kernel
followed by an accumulation of the resulting activations. Yet, all these
improvements have a cost in terms of memory and computation, and it does not
seem possible to increase the size of the kernels indefinitely.

One of the first approaches that allow inflating the receptive field of a
convolutional layer without increasing the number of learnable parameters nor
the computational cost is called dilated convolution (DC). DC or “atrous
convolution” was first described in Holschneider et al. (1990) and Shensa
(1992), under the name “convolution with a dilated filter” before being
referred to as “dilated convolution” in Yu & Koltun (2015). The purpose of
this approach is to inflate the convolutional kernel by regularly inserting
spaces (i.e. zeros) between the kernel elements, as depicted in Figure 2b. The
spacing between elements is thus constant, it is a hyper-parameter usually
referred to as “dilation” or “dilation rate”.

Figure 1: Classification accuracy on ImageNet-1K as a function of latency
(i.e. inverse of the throughput). Dot diameter corresponds to the number of
parameters.

Despite its early successes in classification since Yu et al. (2017), and its
even more convincing results in semantic segmentation Sandler et al. (2018);
Chen et al. (2017; 2018) and object detection Lu et al. (2019), DC has
gradually fallen out of favor and has been confined to downstream tasks such
as those described above. Without much success, Ding et al. (2022) tried to
implement DC in their ReplKNet architecture. Our own investigation on ResNet
and ConvNeXt with standard dilated convolution (Section 4.2) will lead to a
similar conclusion. The failure of this method for classification tasks could
be attributed to the great rigidity imposed by its regular grid as discussed
in Wang & Ji (2018).

In this context, we propose DCLS (Dilated Convolution with Learnable
Spacings), a new convolution method. In DCLS, the positions of the non-zero
elements within the convolutional kernels are learned in a gradient-based
manner. The inherent problem of non-differentiability due to the integer
nature of the positions in the kernel is circumvented by interpolation (Fig.
2c). DCLS is a differentiable method that only constructs the convolutional
kernel. To actually apply the method, we could either use the native
convolution provided by PyTorch or a more advanced one such as the depthwise
implicit gemm convolution method (Ding et al., 2022), using the constructed
kernel. DCLS comes in six sub-versions: 1D, 2D, 3D and what we call N-MD
methods, namely: “2-1D, 3-1D and 3-2D” where a N-dimension kernel is used but
positions are learned only along M dimension(s). The main focus of this paper
will be the 2D version for which we detail mathematical proofs, implementation
specificities and results on image classification, downstream and robustness
tasks.

The principal motivation of DCLS is to explore the possibility of improving
the fixed grid imposed by the standard DC via learning the spacings in an
input-independent way. Instead of having a grid of kernel elements like in
standard and dilated convolutions, DCLS allows an arbitrary number of kernel
elements (Fig. 2d). We refer to this free tunable hyper-parameter as “kernel
count”. In this paper, we set it in order to be at iso or fewer parameters
than the baselines we will compare ourselves to. Conversely, we refer to the
size of the kernel, or rather the maximum size in which the kernel elements
are allowed to move inside the dilated kernel, as the “dilated kernel size”.
It is also a tunable hyper-parameter.

The positions of kernel elements in DCLS are randomly initialized, and are
allowed to move throughout the learning process within the dilated kernel size
limit. We will then show how sharing positions across multiple blocks of a
same convolution stage could further increase accuracy while reducing the
number of learnable parameters. This, together with other learning techniques,
empirically and consistently improve the overall performance of the method.
They are summarized in Section 3.

Figure 2: (a): a standard $3\times 3$ kernel. (b): a dilated $3\times 3$
kernel with dilation rate 4. (c): a 2D-DCLS kernel with 9 kernel elements and
a dilated kernel size of 9. Each weight is spread over up to four adjacent
pixels. (d): a 2D-DCLS kernel with 3 kernel elements and still a dilated
kernel size of 9.

##  2 Kernel construction in DCLS

Our method entails that we learn the float coordinates (${\bm{p}}^{1}$ and
${\bm{p}}^{2}$ in the 2D case for example) for every weight ${\bm{w}}$ of the
dilated kernel (in addition to the actual weights themselves). The positions
or coordinates of the weights within the convolution kernel are conceptually
integer parameters, yet in order to compute their derivatives, we should
consider them as float parameters. This problem of integer positions learning
is smoothed by making use of a bilinear interpolation that will be described
in equations 2, 3 and 4.

###  2.1 Notation and preliminaries

We denote by $\lfloor\ \rfloor$ the floor function and we define its
derivative by the zero function.

$\forall x\in\mathbb{R},\lfloor x\rfloor^{\prime}\overset{\text{def}}{=}0$ (1)

We denote by $m\in\mathbb{N}^{*}$ the number of kernel elements inside the
constructed kernel and we refer to it as the “kernel count”. Moreover, we
denote respectively by $s_{1},s_{2}\in\mathbb{N}^{*}\times\mathbb{N}^{*}$, the
sizes of the constructed kernel along the x-axis and the y-axis. The latter
could be seen as the limits of the dilated kernel, and we refer to them as the
“dilated kernel size”.

The $s_{1}\times s_{2}$ matrix space over $\mathbb{R}$ is defined as the set
of all $s_{1}\times s_{2}$ matrices over $\mathbb{R}$, and is denoted
$\mathcal{M}_{s_{1},s_{2}}(\mathbb{R})$.

The characters $w$, $p^{1}$ and $p^{2}$ respectively stand for the weight, the
position of that weight along the x-axis (width) and its position along the
y-axis (height) in the scalar case while the bold ${\bm{w}}=(w_{i})_{1\leq
i\leq m}$, ${\bm{p}}^{1}=(p^{1}_{i})_{1\leq i\leq m}$ and
${\bm{p}}^{2}=(p^{2}_{i})_{1\leq i\leq m}$ respectively stand for the weight,
the width-position of that weight and its height-position in the vector case.

###  2.2 Mathematical formulation

The mathematical construction of the forward pass as well as the proofs for
the derivations used in the backward pass are in Appendix 7. This construction
relies on bilinear interpolation and could be described by the following
function:

$\displaystyle\begin{split}F\colon\quad\mathbb{R}^{m}\times\mathbb{R}^{m}\times\mathbb{R}^{m}&\to\mathcal{M}_{s_{1},s_{2}}(\mathbb{R})\\\
{\bm{w}},{\bm{p}}^{1},{\bm{p}}^{2}\mapsto&{\bm{K}}=\sum_{k=1}^{m}f(w_{k},p^{1}_{k},p^{2}_{k})\end{split}$
(2)

With $f$ defined as follows:

$\displaystyle\begin{split}f\colon\mathbb{R}\times\mathbb{R}\times\mathbb{R}&\to\mathcal{M}_{s_{1},s_{2}}(\mathbb{R})\\\
w,p^{1},p^{2}&\mapsto\quad{\bm{K}}\end{split}$ (3)

where $\forall i\in\llbracket 1\ ..\ s_{1}\rrbracket$, $\forall j\in\llbracket
1\ ..\ s_{2}\rrbracket\colon$

$\displaystyle{\bm{K}}_{ij}=\left\\{\begin{array}[]{cl}w\ (1-r^{1})\
(1-r^{2})&\text{if }i=\lfloor p^{1}\rfloor,\ j=\lfloor p^{2}\rfloor\\\ w\
r^{1}\ (1-r^{2})&\text{if }i=\lfloor p^{1}\rfloor+1,\ j=\lfloor
p^{2}\rfloor\\\ w\ (1-r^{1})\ r^{2}&\text{if }i=\lfloor p^{1}\rfloor,\
j=\lfloor p^{2}\rfloor+1\\\ w\ r^{1}\ r^{2}&\text{if }i=\lfloor
p^{1}\rfloor{+}1,\ j=\lfloor p^{2}\rfloor{+}1\\\ 0&\text{otherwise
}\end{array}\right.$ (4)

and where the fractional parts are:

$\begin{array}[]{ccc}r^{1}=\\{p^{1}\\}=p^{1}-\lfloor
p^{1}\rfloor&\text{and}&r^{2}=\\{p^{2}\\}=p^{2}-\lfloor
p^{2}\rfloor\end{array}$ (5)

We invite the reader to look at Appendix 7 for more details on the
mathematical proofs and derivatives that permits to calculate the gradients of
a differentiable loss function with respect to the weights and their
positions. Those derivations lead to the DCLS kernel construction algorithm
described for the 2D case in pseudo-code in Appendix 8. Furthermore, the real
code for the kernel construction in 1D, 2D, and 3D cases is included in
Appendix 9. This code is written in native PyTorch language, with classical
modules, and does not require any compilation or adaptation.

##  3 Learning techniques

So far, we have seen how to implement the DCLS method. We now turn to the
techniques that allow us to get the most out of the method. In what follows,
we list the training techniques that we have retained and for which we have
noticed a consistent and empirical advantage on validation accuracy.

  * •

Weight decay: weight decay is a regularization method widely used in a variety
of deep learning models. Though its beneficial effect on generalization, we
noticed that when applied to the kernel positions in DCLS method, weight decay
tends to “artificially” over-concentrate the positions around the center of
the kernel, resulting in poorer accuracy. Therefore, we set this
hyperparameter to 0 for the kernel positions and kept it unchanged for all the
other parameters.

  * •

Positions initialization: the DCLS positions tend to cluster around the center
of the RF throughout the learning process (see Appendix 10). In an attempt to
facilitate learning, we chose an initial distribution close to the one
obtained at the end of training, that is a centered normal law of standard
deviation 0.5. Yet in practice, the uniform law gives a similar performance.

  * •

Positions clamping / overlapping: kernel elements that reach the dilated
kernel size limit are clamped. This is done at the end of every batch step to
force kernel positions to stay within limits. Agglutination around those
limits can sometimes be observed and this indicates that the dilated kernel
size is too small and should be enlarged. Positions of different kernel
weights (or their interpolations) could also overlap. They are added together
in such a case.

  * •

Dilated kernel size tuning: we empirically tuned it using the remark above.
For simplicity, we used the same dilated kernel size in all the model layers
(7 for ResNet-50-dcls and 17 for ConvNeXt-dcls; larger values did not bring
any gain in accuracy). Note that increasing the dilated kernel size has no
cost on the number of trainable parameters, but has a cost on throughput,
especially when using non-separable convolutions. Besides, the convolution
algorithm (which is the most time-consuming part of DCLS) that is used after
the kernel construction (whether it is the native one or the depthwise
implicit gemm one) does not leverage the kernel sparsity. Thus, the kernel
count does not impact the throughput; only the dilated kernel size does. The
development of a convolution method that takes into account sparse kernels,
such as Kundu et al. (2019), could further help to reduce the throughput gap
between DCLS convolution and standard dilated convolution.

  * •

Kernel count tuning: as said before, we have set this hyper-parameter to the
maximal integer value that allows us to be below the baselines to which we
compare ourselves in terms of the number of trainable parameters. Note that
adding one element to the 2D-DCLS kernel leads to having three additional
learnable parameters: the weight, its vertical and its horizontal position.
For simplicity, we used the same kernel count in all the model layers.

  * •

Positions learning rate scaling: we found that kernel positions could benefit
from a special scaling when it comes to the learning rate. As their magnitude
is different from regular weights, we scaled the learning rate of all the
kernel positions by a factor of 5. This is the best factor we found
empirically. Custom-made schedulers for positions have been tested, but
scaling the learning rate while using the same scheduler as for the kernel
weights remained the best choice. Interestingly, we found that throughout
learning, the average speed of the positions follows precisely the shape of
the learning rate scheduler curve (see Appendix 11).

  * •

Synchronizing positions: we shared the kernel positions across convolution
layers with the same number of parameters (typically, those belonging to a
same stage of the ConvNeXt/ResNet model), without sharing the weights.
Positions in this kind of stages were centralized in common parameters that
accumulate the gradients. This constraint has surprisingly enhanced the
accuracy while reducing the number of extra parameters dedicated to positions
(an ablation of this technique led to a 0.13% accuracy drop on ImageNet1k with
ConvNeXt-T-dcls).

  * •

Repulsive loss: following the work of Thomas et al. (2019) on 3D cloud points,
we implemented the repulsive loss for the DCLS kernel positions to discourage
multiple elements from overlapping. Despite a slight advantage with the
ResNet-50-dcls model, this technique did not significantly improve the results
with ConvNeXt-dcls.

  * •

Depthwise implicit gemm: This method has been developed by Ding et al. (2022)
and integrated in Megvii (2020), it has for goal to modify the im2col
algorithm as well as the matrix multiplication that follows, both used in the
native 2D convolution method of PyTorch, by a very efficient algorithm which
does not build explicitly the im2col tensor. The advantage of this method is
that it is much faster than the native one for large kernel sizes, without
affecting the convolution speed for small kernel ones. In our experiments, the
largest dilated kernel size is 17, therefore this method is not absolutely
necessary. However, the DCLS user can choose to use it instead of the native
method, which will improve the throughput.

##  4 Results and discussion

###  4.1 Settings

We started with an exploratory study on ResNet-50, where we drop-in replaced
all the $3\times 3$ convolutions of the model by 2D-DCLS ones. For that, we
used a lightweight procedure named the “A3 configuration”, described in
Wightman et al. (2021). We then moved to the ConvNeXt models, where we limited
our studies to its three first variants namely: the tiny, the small and the
base models with input crops of size $224\times 224$ (Liu et al., 2022b).
Here, we drop-in replaced all the depthwise convolutions of the model by DCLS
ones. We reconducted all the experiments, and evaluated the seed sensitivity
for the ConvNeXt-dcls model by calculating the standard deviation of the top-1
accuracy on three different seeds, for the tiny variant. We found the standard
deviation to be $\pm 0.04$, which is compatible with what was found in Liu et
al. (2022b). Given this reasonably low variability, the remaining experiments
were done on one seed only. Code and scripts to reproduce the training are
available (see reproducibility statement Reproducibility Statement).

###  4.2 Empirical Evaluations on ImageNet1k

In the following, we report the top-1 accuracies found on the ImageNet1k
validation dataset (Deng et al., 2009), using ImageNet1k training dataset
only.

Using ResNet-50. Table 1 presents the results obtained for the ResNet-50
experiment using the A3 configuration. The aim of this study is not to exceed
the state-of-the-art for this particular configuration, but rather to give a
first experimental evidence of the relevance of the DCLS method with non-
separable convolutions and with one of the most popular CNN architectures in
computer vision. We can observe that when using the standard dilated
convolution, the results only get worse as we increase the dilation rate.
Moreover, increasing the kernel size ($3\rightarrow 7$) in the case of
standard convolution increases the accuracy but at the expense of not only the
throughput, which decreases, but also of the number of parameters, which
triples.

With fewer parameters, the ResNet-50-dcls model surpasses the baseline but at
the expense of the throughput. This last disadvantage is due to the fact that
ResNet-50 uses non-separable convolutions. We will see in Table 2 that for the
ConvNeXt model, this extra cost in throughput is minimized thanks to the
depthwise separable convolution.

$\begin{array}[]{lccccccc}\text{ model }&\begin{array}[]{l}\text{ kernel size
}\\\ \text{ / count }\end{array}&\text{ dil }&\text{ \\# param.}&\text{ FLOPs
}&\begin{array}[]{l}\text{ throughput }\\\ \text{ (image / s)
}\end{array}&\begin{array}[]{l}\text{ Top-1 acc. }\\\ \text{ (crop 160)
}\end{array}\\\ \hline\cr\text{ ResNet-50
}&3/9&1&25.6\mathrm{M}&4.1\mathrm{G}&\textbf{1021.9}&75.8\\\ \text{ ResNet-50
}&7/49&1&75.9\mathrm{M}&12.3\mathrm{G}&642.6&\textbf{77.0}\\\ \text{ ResNet-50
}&3/9&2&25.6\mathrm{M}&4.1\mathrm{G}&931.8&71.7\\\ \text{ ResNet-50
}&3/9&3&25.6\mathrm{M}&4.1\mathrm{G}&943.4&70.1\\\ \hline\cr\text{
ResNet50-dcls \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ ResNet50-dcls \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\end{array}$ Table 1: Classification accuracy on ImageNet-1K using
ResNet-50. The throughput was calculated at inference time, on image crops of
size $224\times 224$ using a single V100-32gb gpu. When the model contains
DCLS convolutions, we reported the kernel count and dilated kernel size.
Otherwise, the kernel size is reported and thus the kernel count is in fact
the square of that parameter. $\begin{array}[]{lccccc}\text{ model }&\text{
img size }&\text{ \\# param.}&\text{ FLOPs }&\begin{array}[]{l}\text{
throughput }\\\ \text{ (image / s) }\end{array}&\text{ Top-1 acc. }\\\
\hline\cr\text{ Swin-T }&224^{2}&28\mathrm{M}&4.5\mathrm{G}&757.9&81.3\\\
\text{ ConvNeXt-T \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}
}&224^{2}&29\mathrm{M}&4.5\mathrm{G}&\mathbf{774.7}&82.1\\\ \text{ ConvNeXt-T-
dil2 }&224^{2}&29\mathrm{M}&4.5\mathrm{G}&\mathbf{773.6}&80.8\\\ \text{
ConvNeXt-T-ker17 }&224^{2}&30\mathrm{M}&5\mathrm{G}&560.0&82.0\\\ \text{
SLaK-T \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{1,0.7578125,0.02734375}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@stroke{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}} \leavevmode\hbox
to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0.80078125,0.47265625,0.65625}\definecolor[named]{pgfstrokecolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@stroke{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}&224^{2}&30^{\leavevmode\hbox
to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{1,0.7578125,0.02734375}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@stroke{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}\ /\
38^{\leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0.80078125,0.47265625,0.65625}\definecolor[named]{pgfstrokecolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@stroke{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{1,0.7578125,0.02734375}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@stroke{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}\ /\
9.4^{\leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0.80078125,0.47265625,0.65625}\definecolor[named]{pgfstrokecolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@stroke{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ ConvNeXt-T-dcls \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\text{ Swin-S }&224^{2}&50\mathrm{M}&8.7\mathrm{G}&436.7&83.0\\\
\text{ ConvNeXt-S \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ SLaK-S \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{1,0.7578125,0.02734375}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@stroke{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}} \leavevmode\hbox
to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0.80078125,0.47265625,0.65625}\definecolor[named]{pgfstrokecolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@stroke{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}&224^{2}&55^{\leavevmode\hbox
to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{1,0.7578125,0.02734375}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@stroke{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}\ /\
75^{\leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0.80078125,0.47265625,0.65625}\definecolor[named]{pgfstrokecolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@stroke{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{1,0.7578125,0.02734375}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@stroke{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}\ /\
16.6^{\leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0.80078125,0.47265625,0.65625}\definecolor[named]{pgfstrokecolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@stroke{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ ConvNeXt-S-dcls \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\text{ Swin-B }&224^{2}&88\mathrm{M}&15.4\mathrm{G}&286.6&83.5\\\
\text{ ConvNeXt-B \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ RepLKNet-31B \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0.8359375,0.3671875,0}\definecolor[named]{pgfstrokecolor}{rgb}{0.8359375,0.3671875,0}\pgfsys@color@rgb@stroke{0.8359375}{0.3671875}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0.8359375}{0.3671875}{0}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0.8359375,0.3671875,0}\definecolor[named]{pgffillcolor}{rgb}{0.8359375,0.3671875,0}\pgfsys@color@rgb@fill{0.8359375}{0.3671875}{0}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ SLaK-B \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{1,0.7578125,0.02734375}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@stroke{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}} \leavevmode\hbox
to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0.80078125,0.47265625,0.65625}\definecolor[named]{pgfstrokecolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@stroke{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}&224^{2}&95^{\leavevmode\hbox
to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{1,0.7578125,0.02734375}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@stroke{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}\ /\
122^{\leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0.80078125,0.47265625,0.65625}\definecolor[named]{pgfstrokecolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@stroke{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{1,0.7578125,0.02734375}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@stroke{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}}\ /\
25.9^{\leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0.80078125,0.47265625,0.65625}\definecolor[named]{pgfstrokecolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@stroke{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\definecolor[named]{pgffillcolor}{rgb}{0.80078125,0.47265625,0.65625}\pgfsys@color@rgb@fill{0.80078125}{0.47265625}{0.65625}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ ConvNeXt-B-dcls \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\end{array}$ Table 2: Classification accuracy on ImageNet-1K. The
inference throughput was calculated at inference using a single V100-32gb gpu
and scaled to take into account all the optimizations used in Liu et al.
(2022b). For the SLaK model, we report both the effective number of parameters
and FLOPs returned by PyTorch  and the one reported in Liu et al. (2022a) ,
that takes sparsity into account.

Using ConvNeXt. We present numerically in Table 2 and graphically in Fig. 1,
the results obtained for ConvNeXt using the settings for ImageNet1k training
with input crops of size $224\times 224$, as described in Liu et al. (2022b):
Table 5. Exponential Moving Average (EMA) (Polyak & Juditsky, 1992) was used,
and for all models in Table 2, we report the accuracy found with this
technique. We replaced ConvNeXt’s depthwise separable convolutions (of kernel
size $7\times 7$), by 2D-DCLS ones of dilated kernel size $17\times 17$ and of
kernel count equal to 34 for the tiny variant, and 40 for the small and base
variants. We used all the techniques previously described in Section 3 during
training. From Table 2, we highlight the fact that ConvNeXt with DCLS
convolutions always surpasses the ConvNeXt baseline in accuracy (gains ranging
from 0.3 to 0.6) with the same number of parameters and only a little cost on
throughput. We believe that this gain in accuracy is remarkable, given that we
only replaced the depthwise convolutional layers, which represent just about
1% of the total number of parameters and 2% of the total number of FLOPs in
ConvNeXt models. ConvNeXt model with a standard dilation of rate 2 performed
poorly (see ConvNeXt-T-dil2). SLaK model performs about as well as DCLS but
with a higher cost on throughput and parameters count.

DCLS can be seen as a kernel reparametrization technique that reduces the
number of trainable parameters, and thus regularizes large kernels. For
example, in the case of ConvNeXt-T-dcls, a $17\times 17$ kernel ($289$
parameters) is parameterized by $34$ triplets (x-position, y-position,
weight), i.e. 102 parameters. The kernels that could be represented by the
DCLS reparametrization constitute a subset of all possible dense kernels. In
fact, by learning the suitable weights during training, a dense kernel could
implement any DCLS one. It may therefore be counter-intuitive that DCLS leads
to higher accuracy than a dense $17\times 17$ convolution layer (see ConvNeXt-
T-ker17). The problem with dense convolutional layers having large kernel
sizes is that the number of trainable parameters is huge, which makes learning
impractical. Finally, we observe that after training, the DCLS position
density is higher around the center of the RF (see Appendix 10), suggesting
that the central region is the most important one (yet we experienced that
reducing the dilated kernel size to values $<17$ degrades the accuracy, so the
positions far from the center also matter). Conversely, DC samples the whole
RF uniformly, which is most likely sub-optimal, which could explain its poor
performance (see ConvNeXt-T-dil2).

###  4.3 Empirical Evaluation on Downstream and Robustness Tasks

We now report the results found for semantic segmentation on the ADE20K
dataset (Zhou et al., 2019) and for object detection on the COCO dataset (Lin
et al., 2014) using ConvNeXt-dcls backbones. Note that the depthwise implcit
gemm algorithm was not used for those tasks as it led to throughput issues. In
addition, we present the results found for robustness tasks consisting of
directly testing (without further tuning) the previously obtained ConvNeXt-
dcls backbones on the following robustness benchmarks:
ImageNet-C/$\overline{\text{C}}$/A/R/Sketch (Hendrycks & Dietterich, 2019;
Mintun et al., 2021; Hendrycks et al., 2021b; a; Wang et al., 2019).

Semantic segmentation on ADE20k. The results obtained in semantic segmentation
show an improvement in performance by the ConvNeXt-dcls tiny and base
backbones with equal number of parameters and FLOPs (Table 3). As in Liu et
al. (2021) and Bao et al. (2021), we evaluated the mIoU with single scale
testing and used the exact same configurations as in Liu et al. (2022b).

$\begin{array}[]{lccccc}\text{ backbone }&\text{ input crop. }&\text{ mIoU
(ss) }&\text{ \\# param. }&\text{ FLOPs }&\begin{array}[]{l}\text{ throughput
}\\\ \text{ (image / s) }\end{array}\\\ \hline\cr\text{ ConvNeXt-T
\leavevmode\hbox to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ SLaK-T \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{1,0.7578125,0.02734375}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@stroke{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\definecolor[named]{pgffillcolor}{rgb}{1,0.7578125,0.02734375}\pgfsys@color@rgb@fill{1}{0.7578125}{0.02734375}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ ConvNeXt-T-dcls \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\text{ ConvNeXt-S \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ ConvNeXt-S-dcls \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\text{ ConvNeXt-B \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ ConvNeXt-B-dcls \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\end{array}$ Table 3: ADE20K validation results using UperNet (Xiao
et al., 2018). We report mIoU results with single-scale testing. FLOPs are
based on input sizes of (2048, 512). The inference throughput was calculated
at inference using a single A100-80gb gpu and for input sizes of (3, 512,
512).

Object detection and segmentation on COCO. All ConvNeXt-dcls backbones have
shown a noticeable improvement in average accuracy on both the object
detection and segmentation tasks on the COCO dataset, again at iso-parameters
and iso-FLOPS (Table 4). We only tested with Cascade Mask-RCNN (Cai &
Vasconcelos, 2018) and used the exact same configurations as in Liu et al.
(2022b).

$\begin{array}[]{lccccccccc}\text{ backbone }&\text{ FLOPs }&\text{ TPUT
}&\text{ AP }^{\text{box }}&\text{ AP }_{50}^{\text{box }}&\text{ AP
}_{75}^{\text{box }}&\text{ AP }^{\text{mask }}&\text{ AP}_{50}^{\text{mask
}}&\text{ AP }_{75}^{\text{mask }}\\\ \hline\cr\lx@intercol\hfil\text{ Cascade
Mask-RCNN }3\times\text{ schedule }\hfil\lx@intercol\\\ \text{ ResNet-50
}&739\mathrm{G}&-&46.3&64.3&50.5&40.1&61.7&43.4\\\ \text{ X101-32
}&819\mathrm{G}&-&48.1&66.5&52.4&41.6&63.9&45.2\\\ \text{ X101-64
}&972\mathrm{G}&-&48.3&66.4&52.3&41.7&64.0&45.1\\\ \hline\cr\text{ Swin-T
}&745\mathrm{G}&-&50.4&69.2&54.7&43.7&66.6&47.3\\\ \text{ ConvNeXt-T
\leavevmode\hbox to4.4pt{\vbox to4.4pt{\pgfpicture\makeatletter\hbox{\hskip
2.2pt\lower-2.2pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ CNeXt-dcls-T \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\text{ Swin-S }&838\mathrm{G}&-&51.9&70.7&56.3&45.0&68.2&48.8\\\
\text{ ConvNeXt-S \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ CNeXt-dcls-S \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\text{ Swin-B }&982\mathrm{G}&-&51.9&70.5&56.4&45.0&68.1&48.9\\\
\text{ ConvNeXt-B \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ CNeXt-dcls-B \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\end{array}$ Table 4:  COCO object detection and segmentation results
using Cascade Mask-RCNN. Average Precision of the ResNet-50 and X101 models
are from (Liu et al., 2021). FLOPs are calculated with image size (3, 1280,
800). The inference throughput (”TPUT”) was calculated at inference using a
single A100-80gb gpu and for input sizes of (3, 512, 512).

Robustness Evaluation on ImageNet-C/$\overline{\text{C}}$/A/R/Sketch.
ConvNeXt-dcls backbones show very good performances when it comes to
robustness. This is illustrated by the results obtained for the different
benchmarks we have tried and for which we have reconducted the experiments.
All of them show a gain in classification accuracy with DCLS, except SK with
the S model (Table 5).

$\begin{array}[]{l|lccccccc}\text{ Model }&\text{ FLOPs / Params }&\text{
Clean
}&\mathrm{C}(\downarrow)&\overline{\mathrm{C}}(\downarrow)&\mathrm{A}&\mathrm{R}&\mathrm{SK}\\\
\hline\cr\text{ ResNet-50 }&4.1/25.6&76.1&76.7&57.7&0.0&36.1&24.1\\\
\hline\cr\text{ ConvNeXt-T \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ ConvNeXt-dcls-T \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\text{ ConvNeXt-S \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ ConvNeXt-dcls-S \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\text{ ConvNeXt-B \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.4453125,0.69921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@stroke{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.4453125,0.69921875}\pgfsys@color@rgb@fill{0}{0.4453125}{0.69921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\text{ ConvNeXt-dcls-B \leavevmode\hbox to4.4pt{\vbox
to4.4pt{\pgfpicture\makeatletter\hbox{\hskip 2.2pt\lower-2.2pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{}{{}}{}{{{}} {}{}{}{}{}{}{}{}
}\pgfsys@beginscope\pgfsys@invoke{
}\color[rgb]{0,0.62109375,0.44921875}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@stroke{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\definecolor[named]{pgffillcolor}{rgb}{0,0.62109375,0.44921875}\pgfsys@color@rgb@fill{0}{0.62109375}{0.44921875}\pgfsys@invoke{
}{}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@moveto{2.0pt}{0.0pt}\pgfsys@curveto{2.0pt}{1.10458pt}{1.10458pt}{2.0pt}{0.0pt}{2.0pt}\pgfsys@curveto{-1.10458pt}{2.0pt}{-2.0pt}{1.10458pt}{-2.0pt}{0.0pt}\pgfsys@curveto{-2.0pt}{-1.10458pt}{-1.10458pt}{-2.0pt}{0.0pt}{-2.0pt}\pgfsys@curveto{1.10458pt}{-2.0pt}{2.0pt}{-1.10458pt}{2.0pt}{0.0pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@fillstroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys<EMAIL_ADDRESS>\hline\cr\end{array}$ Table 5: Robustness evaluation of ConvNeXt-dcls. We
reconducted this study for ConvNeXt. For ImageNet-C and ImageNet-Cbar, the
error is reported rather than the accuracy. It was calculated for both
datasets by taking the average error over 5 levels of noise severity and over
all the noise categories available in the datasets.

##  5 Related work

One of the studies that motivated the DCLS method is that of the _effective
receptive field_ (ERF) (Luo et al., 2016), which characterizes how much each
input pixel in a receptive field can impact the output of a unit in the
downstream convolutional layers of a neural network, leading to the notion of
an effective receptive field. Among the findings of this study, the most
relevant ones for ours are that not all pixels in a receptive field contribute
equally to an output response, the kernel center has a much larger impact, and
that the effective receptive field size increases linearly with the square
root of convolutional layers. Given these findings, introducing a new degree
of freedom by learning the positions of non-zero weights in dilated kernels
might increase the expressive power of convolutional neural networks. A visual
comparison between DCLS and non-DCLS ERFs is available in Appendix 12.

The work that relates the most to DCLS is that of deformable convolutions Dai
et al. (2017) where offsets from the regular dilated grid are optimized.
Deformable convolutions are used for several computer vision tasks such as
object detection. Even if both approaches share the use of a rather similar
bilinear interpolation, DCLS remains very different from deformable
convolution in several aspects: firstly, deformable convolutions require the
application of a regular convolution to get the offsets (which are thus input-
dependent) that are then passed to the actual deformable method. Conversely,
in DCLS method, a dilated kernel with learnable positions is constructed and
then passed to the convolution operation, making the positions input-
independent. Secondly, DCLS positions are channel-dependent, unlike deformable
convolution ones. Thirdly, deformable convolutions have been developed for
plain convolutions, not for depthwise separable ones. Finally, the number of
extra learnable parameters in 2D deformable convolutions is the number of
kernel elements in the preliminary convolution, precisely (channels_in //
groups, $2*K_{H}*K_{W},K_{H},K_{W}$) with $K_{H}$ and $K_{W}$ being the kernel
height and width, which establishes a strong dependence between the offsets
and the input feature map. Contrarily, in 2D-DCLS, the number of extra
parameters dedicated to learning positions is simply twice the number of
kernel weights (2, channels_out, channels_in // groups, $K_{H}$, $K_{W}$). We
have not been able to compare our work to the deformable convolution v2 in
ConvNeXt as the training becomes very slow (the throughput is divided by 4 for
the ConvNeXt-tiny model). In addition, we noticed that the training loss grew
at the first epochs, which means that the method is not adapted as a drop-in
replacement of the depthwise separable convolution in the ConvNeXt
architecture. Another input-dependent convolution method which seeks to learn
the dilation rates rather than the kernel positions is the one of ADCNN (Yao
et al., 2022). But in contrast to DCLS, this method uses regular grids with
learnable rates.

DCLS is also similar to other input-independent kernel re-parameterization
techniques, yet different from them. For example, in CKConv (Romero et al.,
2021b) and FlexConv (Romero et al., 2021a), the kernel weights are not learned
directly; what is learned is the continuous function that maps the positions
to the weights. In Jacobsen et al. (2016), the kernel is modeled as a weighted
sum of basis functions, which consist of centered Gaussian filters and their
derivatives. Pintea et al. (2021) extended the approach by also learning the
width of the Gaussians, which is equivalent to learning the optimal
resolution. Shelhamer et al. (2019) proposed to factorize the kernel as the
composition of a standard kernel with a structured Gaussian one. Finally,
other methods such as(Worrall & Welling, 2019; Sosnovik et al., 2019; 2021a;
2021b; Bekkers, 2019; Zhu et al., 2019), where the goal is to build a scale
equivariant neural network, could be considered as similar to the approach.
One limitation of all these studies is that only small datasets were used
(e.g., CIFAR), and whether they scale well to larger and more challenging
datasets like ImageNet1k is unknown. In addition, they cannot be used as a
drop-in replacement for the depthwise separable convolution in ConvNeXt, at
least in their current forms, so we could not benchmark them with DCLS.

##  6 Conclusion

In this work, we proposed DCLS, a new dilated convolution method where the
positions of non-zero kernel elements are made learnable via backpropagation.
The non-differentiability issue of learning discrete positions was
circumvented by interpolation. We demonstrated that DCLS often outperforms the
standard and the dilated convolution. We listed a number of techniques that
improve the learning process of DCLS, in particular sharing the positions
within stages was key. We provided evidence that searching for optimal
positions of weights within a dilated kernel can improve not only the accuracy
in image classification, but also in downstream and robustness tasks, using
existing CNN architectures, without increasing their number of parameters. We
reported a throughput overhead introduced by DCLS, but it remains marginal,
provided that we use CNN architectures involving separable convolutions, which
is the case for most modern CNNs, such as ConvNeXts. Future work: So far we
have shown that DCLS can be used to drop-in replace standard convolution
layers in existing architectures. We now would like to search for an
architecture dedicated to DCLS, that would get the maximum benefit out of the
method. In addition, we will explore alternatives to bilinear interpolation,
e.g. bicubic. Finally, we wish to show that the method is also of interest in
1D and 3D use cases. For 1D cases, one can think of audio waveform generation,
where dilated 1D convolution filters with large receptive fields are key, as,
for example, in the autoregressive WaveNet model (Oord et al., 2016), and in
generative adversarial networks (Yamamoto et al., 2020; Greshler et al.,
2021). For 3D applications, video classification with architectures using 3D
convolutions, e.g., X3D Feichtenhofer (2020), would be a good fit.

## Acknowledgments

This work was performed using HPC resources from GENCI–IDRIS (Grant
2021-[AD011013219]) and from CALMIP (Grant 2021-[P21052]). Support from the
ANR-3IA Artificial and Natural Intelligence Toulouse Institute is gratefully
acknowledged. We would also like to thank the region of Toulouse Occitanie.

## Reproducibility Statement

The code of the method is based on PyTorch and available at
https://github.com/K-H-Ismail/Dilated-Convolution-with-Learnable-Spacings-
PyTorch. Code and scripts to reproduce the training and the experiments are
available at https://github.com/K-H-Ismail/ConvNeXt-dcls.

## References

  * Araujo et al. (2019) André Araujo, Wade Norris, and Jack Sim.  Computing receptive fields of convolutional neural networks.  _Distill_ , 2019.  doi: 10.23915/distill.00021.  https://distill.pub/2019/computing-receptive-fields.
  * Bao et al. (2021) Hangbo Bao, Li Dong, and Furu Wei.  Beit: Bert pre-training of image transformers.  _arXiv preprint arXiv:2106.08254_ , 2021.
  * Bekkers (2019) Erik J Bekkers.  B-spline cnns on lie groups.  _arXiv preprint arXiv:1909.12057_ , 2019.
  * Cai & Vasconcelos (2018) Zhaowei Cai and Nuno Vasconcelos.  Cascade r-cnn: Delving into high quality object detection.  In _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR)_ , pp. 6154–6162, 2018.
  * Chen et al. (2017) Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille.  Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs.  _IEEE Trans. Pattern Anal. Mach. Intell._ , 40(4):834–848, 2017.
  * Chen et al. (2018) Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam.  Encoder-decoder with atrous separable convolution for semantic image segmentation.  In _Proc. Eur. Conf. Comput. Vis. (ECCV)_ , pp. 801–818, 2018.
  * Dai et al. (2017) Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei.  Deformable convolutional networks.  In _Int. Conf. Comput. Vis._ , pp. 764–773, 2017.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR)_ , pp. 248–255. IEEE, 2009.
  * Ding et al. (2022) Xiaohan Ding, Xiangyu Zhang, Jungong Han, and Guiguang Ding.  Scaling up your kernels to 31x31: Revisiting large kernel design in CNNs.  In _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR)_ , pp. 11963–11975, 2022.
  * Dosovitskiy et al. (2020) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ , 2020.
  * Feichtenhofer (2020) Christoph Feichtenhofer.  X3d: Expanding architectures for efficient video recognition.  In _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR)_ , pp. 203–213, 2020.
  * Greshler et al. (2021) Gal Greshler, Tamar Shaham, and Tomer Michaeli.  Catch-a-waveform: Learning to generate audio from a single short example.  _Advances in Neural Information Processing Systems_ , 34:20916–20928, 2021.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR)_ , pp. 770–778, 2016.
  * Hendrycks & Dietterich (2019) Dan Hendrycks and Thomas Dietterich.  Benchmarking neural network robustness to common corruptions and perturbations.  _arXiv preprint arXiv:1903.12261_ , 2019.
  * Hendrycks et al. (2021a) Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al.  The many faces of robustness: A critical analysis of out-of-distribution generalization.  In _Int. Conf. Comput. Vis._ , pp. 8340–8349, 2021a.
  * Hendrycks et al. (2021b) Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song.  Natural adversarial examples.  In _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR)_ , pp. 15262–15271, 2021b.
  * Holschneider et al. (1990) Matthias Holschneider, Richard Kronland-Martinet, Jean Morlet, and Ph Tchamitchian.  A real-time algorithm for signal analysis with the help of the wavelet transform.  In _Wavelets_ , pp. 286–297. Springer, 1990.
  * Jacobsen et al. (2016) Jorn-Henrik Jacobsen, Jan Van Gemert, Zhongyu Lou, and Arnold WM Smeulders.  Structured receptive fields in cnns.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 2610–2619, 2016.
  * Kundu et al. (2019) Souvik Kundu, Saurav Prakash, Haleh Akrami, Peter A Beerel, and Keith M Chugg.  A pre-defined sparse kernel based convolution for deep cnns.  _arXiv preprint arXiv:1910.00724_ , 2019.
  * Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In _Proc. Eur. Conf. Comput. Vis. (ECCV)_ , pp. 740–755. Springer, 2014.
  * Liu et al. (2022a) Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Mykola Pechenizkiy, Decebal Mocanu, and Zhangyang Wang.  More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity.  _arXiv preprint arXiv:2207.03620_ , 2022a.
  * Liu et al. (2021) Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In _Int. Conf. Comput. Vis._ , pp. 10012–10022, 2021.
  * Liu et al. (2022b) Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.  A convnet for the 2020s.  In _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR)_ , pp. 11976–11986, 2022b.
  * Lu et al. (2019) Xin Lu, Buyu Li, Yuxin Yue, Quanquan Li, and Junjie Yan.  Grid R-CNN.  In _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR)_ , pp. 7363–7372, 2019.
  * Luo et al. (2016) Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel.  Understanding the effective receptive field in deep convolutional neural networks.  In _Proc. Adv. Neural Inform. Process. Syst. (NIPS)_ , pp. 4905–4913, 2016.
  * Megvii (2020) Megvii.  Megengine:a fast, scalable and easy-to-use deep learning framework.  https://github.com/MegEngine/MegEngine, 2020.
  * Mintun et al. (2021) Eric Mintun, Alexander Kirillov, and Saining Xie.  On interaction between augmentations and corruptions in natural corruption robustness.  _Proc. Adv. Neural Inform. Process. Syst. (NIPS)_ , 34:3571–3583, 2021.
  * Oord et al. (2016) Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu.  WaveNet: A generative model for raw audio.  _arXiv preprint arXiv:1609.03499_ , 2016.
  * Pintea et al. (2021) Silvia L Pintea, Nergis Tömen, Stanley F Goes, Marco Loog, and Jan C van Gemert.  Resolution learning in deep convolutional networks using scale-space theory.  _IEEE Transactions on Image Processing_ , 30:8342–8353, 2021.
  * Polyak & Juditsky (1992) Boris T Polyak and Anatoli B Juditsky.  Acceleration of stochastic approximation by averaging.  _SIAM journal on control and optimization_ , 30(4):838–855, 1992.
  * Romero et al. (2021a) David W Romero, Robert-Jan Bruintjes, Jakub M Tomczak, Erik J Bekkers, Mark Hoogendoorn, and Jan C van Gemert.  Flexconv: Continuous kernel convolutions with differentiable kernel sizes.  _arXiv preprint arXiv:2110.08059_ , 2021a.
  * Romero et al. (2021b) David W Romero, Anna Kuzina, Erik J Bekkers, Jakub M Tomczak, and Mark Hoogendoorn.  Ckconv: Continuous kernel convolution for sequential data.  _arXiv preprint arXiv:2102.02611_ , 2021b.
  * Sandler et al. (2018) Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR)_ , pp. 4510–4520, 2018.
  * Shelhamer et al. (2019) Evan Shelhamer, Dequan Wang, and Trevor Darrell.  Blurring the line between structure and learning to optimize and adapt receptive fields.  _arXiv preprint arXiv:1904.11487_ , 2019.
  * Shensa (1992) Mark J Shensa.  The discrete wavelet transform: wedding the a trous and mallat algorithms.  _IEEE Trans. Signal Process._ , 40(10):2464–2482, 1992.
  * Sosnovik et al. (2019) Ivan Sosnovik, Michał Szmaja, and Arnold Smeulders.  Scale-equivariant steerable networks.  _arXiv preprint arXiv:1910.11093_ , 2019.
  * Sosnovik et al. (2021a) Ivan Sosnovik, Artem Moskalev, and Arnold Smeulders.  Disco: accurate discrete scale convolutions.  _arXiv preprint arXiv:2106.02733_ , 2021a.
  * Sosnovik et al. (2021b) Ivan Sosnovik, Artem Moskalev, and Arnold Smeulders.  How to transform kernels for scale-convolutions.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 1092–1097, 2021b.
  * Thomas et al. (2019) Hugues Thomas, Charles R. Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J. Guibas.  Kpconv: Flexible and deformable convolution for point clouds.  _Int. Conf. Comput. Vis._ , 2019.
  * Trockman & Kolter (2022) Asher Trockman and J Zico Kolter.  Patches are all you need?  _arXiv preprint arXiv:2201.09792_ , 2022.
  * Wang et al. (2019) Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing.  Learning robust global representations by penalizing local predictive power.  _Proc. Adv. Neural Inform. Process. Syst. (NIPS)_ , 32, 2019.
  * Wang & Ji (2018) Zhengyang Wang and Shuiwang Ji.  Smoothed dilated convolutions for improved dense prediction.  In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pp. 2486–2495, 2018.
  * Wightman et al. (2021) Ross Wightman, Hugo Touvron, and Hervé Jégou.  Resnet strikes back: An improved training procedure in timm.  _arXiv preprint arXiv:2110.00476_ , 2021.
  * Worrall & Welling (2019) Daniel Worrall and Max Welling.  Deep scale-spaces: Equivariance over scale.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Xiao et al. (2018) Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun.  Unified perceptual parsing for scene understanding.  In _Proc. Eur. Conf. Comput. Vis. (ECCV)_ , pp. 418–434, 2018.
  * Yamamoto et al. (2020) Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim.  Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram.  In _Proc. ICASSP_ , pp. 6199–6203. IEEE, 2020.
  * Yao et al. (2022) Jie Yao, Dongdong Wang, Hao Hu, Weiwei Xing, and Liqiang Wang.  Adcnn: Towards learning adaptive dilation for convolutional neural networks.  _Pattern Recognition_ , 123:108369, 2022.
  * Yu & Koltun (2015) Fisher Yu and Vladlen Koltun.  Multi-scale context aggregation by dilated convolutions.  _arXiv preprint arXiv:1511.07122_ , 2015.
  * Yu et al. (2017) Fisher Yu, Vladlen Koltun, and Thomas Funkhouser.  Dilated residual networks.  In _Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR)_ , pp. 472–480, 2017.
  * Zhou et al. (2019) Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba.  Semantic understanding of scenes through the ade20k dataset.  _Int. J. Comput. Vis._ , 127(3):302–321, 2019\.
  * Zhu et al. (2019) Wei Zhu, Qiang Qiu, Robert Calderbank, Guillermo Sapiro, and Xiuyuan Cheng.  Scaling-translation-equivariant networks with decomposed convolutional filters.  _arXiv preprint arXiv:1909.11193_ , 2019.

##  7 Appendix: proofs and derivation for the DCLS method

In the following, we show how to mathematically describe the DCLS kernel
construction and how to explicitly calculate the gradients of the loss
function with respect to weights and positions that are used in the
backpropagation algorithm. These gradients are useful to implement the DCLS
method in a way that is compatible with the automatic differentiation of
PyTorch.

###  7.1 Notation and preliminaries

We denote by $\lfloor\ \rfloor$ the floor function and we define its
derivative by the zero function.

$\forall x\in\mathbb{R},\lfloor x\rfloor^{\prime}\overset{\text{def}}{=}0$ (6)

We denote by $m\in\mathbb{N}^{*}$ the number of kernel elements inside the
constructed kernel and we refer to it as the “kernel count”. Moreover, we
denote respectively by $s_{1},s_{2}\in\mathbb{N}^{*}\times\mathbb{N}^{*}$, the
sizes of the constructed kernel along the x-axis and the y-axis. The latter
could be seen as the limits of the dilated kernel, and we refer to them as the
“dilated kernel size”.

The $n\times p$ matrix space over $\mathbb{R}$ is defined as the set of all
$n\times p$ matrices over $\mathbb{R}$, and is denoted
$\mathcal{M}_{n,p}(\mathbb{R})$.

The Frobenius inner product $\underset{\text{F}}{\times}$ of two matrices
$\displaystyle{\bm{A}}$ and $\displaystyle{\bm{B}}$ of
$\mathcal{M}_{n,p}(\mathbb{R})$ is defined by:

$\displaystyle{\bm{A}}\underset{\text{F}}{\times}{\bm{B}}=\text{tr}({\bm{A}}^{T}{\bm{B}})$

Where “tr” stands for the trace of the square matrix ${\bm{A}}^{T}{\bm{B}}$.

The characters $w$, $p^{1}$ and $p^{2}$ respectively stand for the weight, the
position of that weight along the x-axis (width) and its position along the
y-axis (height) in the scalar case while the bold ${\bm{w}}=(w_{i})_{1\leq
i\leq m}$, ${\bm{p}}^{1}=(p^{1}_{i})_{1\leq i\leq m}$ and
${\bm{p}}^{2}=(p^{2}_{i})_{1\leq i\leq m}$ respectively stand for the weight,
the width-position of that weight and its height-position in the vector case.

The proofs and algorithms that will be shown in the next subsections are made
for the case of tensors with one input channel and one output channel. Without
loss of generality, those proofs and algorithms hold for the general case of
4D tensors and higher by considering and applying them channel-wise.

###  7.2 2D-DCLS, scalar weight case

We begin by the case of a kernel containing only one element.

The function $f$ that defines the kernel construction in the scalar weight
case is as follows:

$\displaystyle\begin{split}f\colon\mathbb{R}\times\mathbb{R}\times\mathbb{R}&\to\mathcal{M}_{s_{1},s_{2}}(\mathbb{R})\\\
w,p^{1},p^{2}&\mapsto\displaystyle{\bm{K}}\end{split}$ (7)

where $\forall i\in\llbracket 1\ ..\ s_{1}\rrbracket$, $\forall j\in\llbracket
1\ ..\ s_{2}\rrbracket\colon$

$\displaystyle{\bm{K}}_{ij}=\left\\{\begin{array}[]{cl}w\ (1-r^{1})\
(1-r^{2})&\text{if }i=\lfloor p^{1}\rfloor,\ j=\lfloor p^{2}\rfloor\\\ w\
r^{1}\ (1-r^{2})&\text{if }i=\lfloor p^{1}\rfloor+1,\ j=\lfloor
p^{2}\rfloor\\\ w\ (1-r^{1})\ r^{2}&\text{if }i=\lfloor p^{1}\rfloor,\
j=\lfloor p^{2}\rfloor+1\\\ w\ r^{1}\ r^{2}&\text{if }i=\lfloor
p^{1}\rfloor{+}1,\ j=\lfloor p^{2}\rfloor{+}1\\\ 0&\text{otherwise
}\end{array}\right.$ (8)

and where the fractional parts are:

$\begin{array}[]{ccc}r^{1}=\\{p^{1}\\}=p^{1}-\lfloor
p^{1}\rfloor&\text{and}&r^{2}=\\{p^{2}\\}=p^{2}-\lfloor
p^{2}\rfloor\end{array}$ (9)

The constructed kernel ${\bm{K}}$ is zero except for at most the 4 adjacent
positions that represent the 2D interpolation of the single weight $w$. Note
that $\sum\limits_{j=1}^{s_{2}}\sum\limits_{i=1}^{s_{1}}{\bm{K}}_{ij}=w$.

We then define the scalar loss function as:

$loss=g(f(w,p^{1},p^{2}))$ (10)

with $g\colon\mathcal{M}_{s_{1},s_{2}}(\mathbb{R})\to\mathbb{R}$ a
differentiable function that models the action of all the layers that will
follow $f$ in the model.

By applying the chain rule we obtain:

$\displaystyle\frac{\partial loss}{\partial
w}=g^{\prime}(f(w,p^{1},p^{2}))\underset{\text{F}}{\times}\frac{\partial
f(w,p^{1},p^{2})}{\partial w}$ (11) $\displaystyle\frac{\partial
loss}{\partial
p^{1}}=g^{\prime}(f(w,p^{1},p^{2}))\underset{\text{F}}{\times}\frac{\partial
f(w,p^{1},p^{2})}{\partial p^{1}}$ (12) $\displaystyle\frac{\partial
loss}{\partial
p^{2}}=g^{\prime}(f(w,p^{1},p^{2}))\underset{\text{F}}{\times}\frac{\partial
f(w,p^{1},p^{2})}{\partial p^{2}}$ (13)

with

$g^{\prime}(f(w,p^{1},p^{2}))=\frac{\partial
loss}{\partial{\bm{K}}}=\frac{\partial loss}{\partial f(w,p^{1},p^{2})}$ (14)

Let us put

$g^{\prime}(f(w,p^{1},p^{2}))=\displaystyle{\bm{G}}=\begin{bmatrix}g_{11}&g_{12}&\cdots&g_{1s_{2}}\\\
g_{21}&\ddots&&g_{2s_{2}}\\\ \vdots&&\ddots&\vdots\\\
g_{s_{1}1}&g_{s_{1}2}&\cdots&g_{s_{1}s_{2}}\\\ \end{bmatrix}$ (15)

and let us consider two column vectors
$\displaystyle{\bm{x}}=\begin{bmatrix}x_{1}&x_{2}&\cdots&x_{s_{1}}\end{bmatrix}^{T}$
of $\mathbb{R}^{s_{1}}$ and
$\displaystyle{\bm{y}}=\begin{bmatrix}y_{1}&y_{2}&\cdots&y_{s_{2}}\end{bmatrix}^{T}$
of $\mathbb{R}^{s_{2}}$. We have:

$\displaystyle{\bm{x}}^{T}f(w,p^{1},p^{2})\displaystyle{\bm{y}}=\sum_{i=1}^{s_{1}}\sum_{j=1}^{s_{2}}\displaystyle{\bm{K}}_{ij}x_{i}y_{j}$
(16)

Since ${\bm{K}}$ is zero except for the 4 aforementioned positions, we have:

$\displaystyle\begin{split}\displaystyle{\bm{x}}^{T}f(w,p^{1},p^{2})\displaystyle{\bm{y}}=&\
{\bm{K}}_{\lfloor p^{1}\rfloor\lfloor p^{2}\rfloor}\ x_{\lfloor p^{1}\rfloor}\
y_{\lfloor p^{2}\rfloor}\\\ &+{\bm{K}}_{\lfloor p^{1}\rfloor+1\lfloor
p^{2}\rfloor}\ x_{\lfloor p^{1}\rfloor+1}\ y_{\lfloor p^{2}\rfloor}\\\
&+{\bm{K}}_{\lfloor p^{1}\rfloor\lfloor p^{2}\rfloor+1}\ x_{\lfloor
p^{1}\rfloor}\ y_{\lfloor p^{2}\rfloor+1}\\\ &+{\bm{K}}_{\lfloor
p^{1}\rfloor+1\lfloor p^{2}\rfloor+1}\ x_{\lfloor p^{1}\rfloor+1}\ y_{\lfloor
p^{2}\rfloor+1}\end{split}$ (17)

By deriving this expression with respect to $w$, $p^{1}$ and $p^{2}$ we
obtain:

$\displaystyle\begin{split}\frac{\partial({\bm{x}}^{T}f(w,p^{1},p^{2}){\bm{y}})}{\partial
w}=\ &\ (1-r^{1})\ (1-r^{2})\ x_{\lfloor p^{1}\rfloor}\ y_{\lfloor
p^{2}\rfloor}\\\ &+r^{1}\ (1-r^{2})\ x_{\lfloor p^{1}\rfloor+1}\ y_{\lfloor
p^{2}\rfloor}\\\ &+(1-r^{1})\ r^{2}\ x_{\lfloor p^{1}\rfloor}\ y_{\lfloor
p^{2}\rfloor+1}\\\ &+r^{1}\ r^{2}\ x_{\lfloor p^{1}\rfloor+1}\ y_{\lfloor
p^{2}\rfloor+1}\end{split}$ (18)
$\displaystyle\begin{split}\frac{\partial({\bm{x}}^{T}f(w,p^{1},p^{2}){\bm{y}})}{\partial
p^{1}}=w\ [&-(1-r^{2})\ x_{\lfloor p^{1}\rfloor}\ y_{\lfloor p^{2}\rfloor}\\\
&+(1-r^{2})\ x_{\lfloor p^{1}\rfloor+1}\ y_{\lfloor p^{2}\rfloor}\\\ &-r^{2}\
x_{\lfloor p^{1}\rfloor}\ y_{\lfloor p^{2}\rfloor+1}\\\ &+r^{2}\ x_{\lfloor
p^{1}\rfloor+1}\ y_{\lfloor p^{2}\rfloor+1}]\end{split}$ (19)
$\displaystyle\begin{split}\frac{\partial({\bm{x}}^{T}f(w,p^{1},p^{2}){\bm{y}})}{\partial
p^{2}}=w\ [&-(1-r^{1})\ x_{\lfloor p^{1}\rfloor}\ y_{\lfloor p^{2}\rfloor}\\\
&-r^{1}\ x_{\lfloor p^{1}\rfloor+1}\ y_{\lfloor p^{2}\rfloor}\\\ &+(1-r^{1})\
x_{\lfloor p^{1}\rfloor}\ y_{\lfloor p^{2}\rfloor+1}\\\ &+r^{1}\ x_{\lfloor
p^{1}\rfloor+1}\ y_{\lfloor p^{2}\rfloor+1}]\end{split}$ (20)

Because of the linearity of the differentiation in the three previous
equations, we could write:

$\displaystyle\frac{\partial({\bm{x}}^{T}f(w,p^{1},p^{2}){\bm{y}})}{\partial
w}={\bm{x}}^{T}\frac{\partial f(w,p^{1},p^{2})}{\partial
w}{\bm{y}}={\bm{x}}^{T}{\bm{G}}_{w}{\bm{y}}$ (21)
$\displaystyle\frac{\partial({\bm{x}}^{T}f(w,p^{1},p^{2}){\bm{y}})}{\partial
p^{1}}={\bm{x}}^{T}\frac{\partial f(w,p^{1},p^{2})}{\partial
p^{1}}{\bm{y}}={\bm{x}}^{T}{\bm{G}}_{p^{1}}{\bm{y}}$ (22)
$\displaystyle\frac{\partial({\bm{x}}^{T}f(w,p^{1},p^{2}){\bm{y}})}{\partial
p^{2}}={\bm{x}}^{T}\frac{\partial f(w,p^{1},p^{2})}{\partial
p^{2}}{\bm{y}}={\bm{x}}^{T}{\bm{G}}_{p^{2}}{\bm{y}}$ (23)

where ${\bm{G}}_{w}$, ${\bm{G}}_{p^{1}}$, ${\bm{G}}_{p^{2}}$, respectively
stand for the $s_{1}$ by $s_{2}$ matrices described below and which have zeros
everywhere except at the four positions of interpolation.

${\bm{G}}_{w}=$

$\blockarray{ccclccl}&\lfloor p^{2}\rfloor\lfloor p^{2}\rfloor{+}1\\\
\block{(cccccc)l}0\cdots 00\cdots 0\\\ \vdots\ddots 00\iddots\vdots\\\
00(1{-}r^{1})(1{-}r^{2})r^{2}(1{-}r^{1})00\lfloor p^{1}\rfloor\\\
00r^{1}(1{-}r^{2})r^{1}r^{2}00\lfloor p^{1}\rfloor{+}1\\\ \vdots\iddots
00\ddots\vdots\\\ 0\cdots 00\cdots 0\\\ $ (24)

${\bm{G}}_{p^{1}}=$

$\blockarray{ccclccl}&\lfloor p^{2}\rfloor\lfloor p^{2}\rfloor{+}1\\\
\block{(cccccc)l}0\cdots 00\cdots 0\\\ \vdots\ddots 00\iddots\vdots\\\
00-w(1{-}r^{2})-wr^{2}00\lfloor p^{1}\rfloor\\\ 00w(1{-}r^{2})wr^{2}00\lfloor
p^{1}\rfloor{+}1\\\ \vdots\iddots 00\ddots\vdots\\\ 0\cdots 00\cdots 0\\\ $
(25)

${\bm{G}}_{p^{2}}=$

$\blockarray{ccclccl}&\lfloor p^{2}\rfloor\lfloor p^{2}\rfloor{+}1\\\
\block{(cccccc)l}0\cdots 00\cdots 0\\\ \vdots\ddots 00\iddots\vdots\\\
00-w(1{-}r^{1})w(1{-}r^{1})00\lfloor p^{1}\rfloor\\\ 00-wr^{1}wr^{1}00\lfloor
p^{1}\rfloor{+}1\\\ \vdots\iddots 00\ddots\vdots\\\ 0\cdots 00\cdots 0\\\ $
(26)

From the three equations (21), (22) and (23), we can identify

$\displaystyle\frac{\partial f(w,p^{1},p^{2})}{\partial w}={\bm{G}}_{w}$ (27)
$\displaystyle\frac{\partial f(w,p^{1},p^{2})}{\partial
p^{1}}={\bm{G}}_{p^{1}}$ (28) $\displaystyle\frac{\partial
f(w,p^{1},p^{2})}{\partial p^{2}}={\bm{G}}_{p^{2}}$ (29)

Finally, we have:

$\displaystyle\begin{split}\frac{\partial loss}{\partial
w}={\bm{G}}\underset{\text{F}}{\times}{\bm{G}}_{w}=\ &(1-r^{1})\ (1-r^{2})\
g_{\lfloor p^{1}\rfloor\lfloor p^{2}\rfloor}\\\ &+r^{1}\ (1-r^{2})\ g_{\lfloor
p^{1}\rfloor+1\lfloor p^{2}\rfloor}\\\ &+(1-r^{1})\ r^{2}\ g_{\lfloor
p^{1}\rfloor\lfloor p^{2}\rfloor+1}\\\ &+r^{1}\ r^{2}\ g_{\lfloor
p^{1}\rfloor+1\lfloor p^{2}\rfloor+1}\end{split}$ (30)
$\displaystyle\begin{split}\frac{\partial loss}{\partial
p^{1}}={\bm{G}}\underset{\text{F}}{\times}{\bm{G}}_{p^{1}}=\ w\ [&-(1-r^{2})\
g_{\lfloor p^{1}\rfloor\lfloor p^{2}\rfloor}\\\ &+(1-r^{2})\ g_{\lfloor
p^{1}\rfloor+1\lfloor p^{2}\rfloor}\\\ &-r^{2}\ g_{\lfloor p^{1}\rfloor\lfloor
p^{2}\rfloor+1}\\\ &+r^{2}\ g_{\lfloor p^{1}\rfloor+1\lfloor
p^{2}\rfloor+1}]\end{split}$ (31) $\displaystyle\begin{split}\frac{\partial
loss}{\partial p^{2}}={\bm{G}}\underset{\text{F}}{\times}{\bm{G}}_{p^{2}}=\ w\
[&-(1-r^{1})\ g_{\lfloor p^{1}\rfloor\lfloor p^{2}\rfloor}\\\ &-r^{1}\
g_{\lfloor p^{1}\rfloor+1\lfloor p^{2}\rfloor}\\\ &+(1-r^{1})\ g_{\lfloor
p^{1}\rfloor\lfloor p^{2}\rfloor+1}\\\ &+r^{1}\ g_{\lfloor
p^{1}\rfloor+1\lfloor p^{2}\rfloor+1}]\end{split}$ (32)

In the next subsection, we will see how this result can be generalized to the
vector case.

###  7.3 2D-DCLS, general case

The general case is the one where the weights
${\bm{w}}=\begin{bmatrix}w_{1}&w_{2}&\cdots&w_{m}\end{bmatrix}^{T}$ and the
positions
${\bm{p}}^{1}=\begin{bmatrix}p^{1}_{1}&p^{1}_{2}&\cdots&p^{1}_{m}\end{bmatrix}^{T}$
,
${\bm{p}}^{2}=\begin{bmatrix}p^{2}_{1}&p^{2}_{2}&\cdots&p^{2}_{m}\end{bmatrix}^{T}$
are stored in vectors, with the fractional parts
${\bm{r}}^{1}=\\{{\bm{p}}^{1}\\}={\bm{p}}^{1}-\lfloor{\bm{p}}^{1}\rfloor=\begin{bmatrix}r^{1}_{1}&r^{1}_{2}&\cdots&r^{1}_{m}\end{bmatrix}^{T}$
and
${\bm{r}}^{2}=\\{{\bm{p}}^{2}\\}={\bm{p}}^{2}-\lfloor{\bm{p}}^{2}\rfloor=\begin{bmatrix}r^{2}_{1}&r^{2}_{2}&\cdots&r^{2}_{m}\end{bmatrix}^{T}$
extended as well.

The function $f$ defined in equation (7) is then extended to the function $F$
defined as follows:

$\displaystyle\begin{split}F\colon\quad\mathbb{R}^{m}\times\mathbb{R}^{m}\times\mathbb{R}^{m}&\to\mathcal{M}_{s_{1},s_{2}}(\mathbb{R})\\\
{\bm{w}},{\bm{p}}^{1},{\bm{p}}^{2}\mapsto&{\bm{K}}=\sum_{i=1}^{m}f(w_{i},p^{1}_{i},p^{2}_{i})\end{split}$
(33)

The constructed kernel ${\bm{K}}$ here is the result of a summation of the
function $f$ defined in (7) over the elements of weight and position vectors.
We then define the scalar loss function as in (10).

$loss=g(F({\bm{w}},{\bm{p}}^{1},{\bm{p}}^{2}))$ (34)

with $g\colon\mathcal{M}_{s_{1},s_{2}}(\mathbb{R})\to\mathbb{R}$ a
differentiable function that models the action of all the layers that will
follow $F$ in the model.

Let us put