# Partial Large Kernel CNNs for Efficient Super-Resolution

Dongheon Lee Seokju Yun Youngmin Ro
Machine Intelligence Laboratory, University of Seoul, Korea
Code: https://github.com/dslisleedh/PLKSR Corresponding author (E-mail:
<EMAIL_ADDRESS>
###### Abstract

Recently, in the super-resolution (SR) domain, transformers have outperformed
CNNs with fewer FLOPs and fewer parameters since they can deal with long-range
dependency and adaptively adjust weights based on instance. In this paper, we
demonstrate that CNNs, although less focused on in the current SR domain,
surpass Transformers in direct efficiency measures. By incorporating the
advantages of Transformers into CNNs, we aim to achieve both computational
efficiency and enhanced performance. However, using a large kernel in the SR
domain, which mainly processes large images, incurs a large computational
overhead. To overcome this, we propose novel approaches to employing the large
kernel, which can reduce latency by 86% compared to the naive large kernel,
and leverage an Element-wise Attention module to imitate instance-dependent
weights. As a result, we introduce Partial Large Kernel CNNs for Efficient
Super-Resolution (PLKSR), which achieves state-of-the-art performance on four
datasets at a scale of $\times$4, with reductions of 68.1% in latency and
80.2% in maximum GPU memory occupancy compared to SRFormer-light.

##  1 Introduction

Figure 1: Comparison on performance, latency, and Maximum GPU memory Occupancy
(MGO) at BSD100$\times$2 dataset. Our PLKSR performs the best when compared to
the SOTA SR methods, with 68.1 and 80.2 less latency and MGO compared to
SRFormer-light, respectively. All metrics are measured by restoring an HD
(1280×720) image using RTX4090 GPU at FP16 precision.

Image Super-Resolution (SR) aims to reconstruct High-Resolution (HR) images
from Low-Resolution (LR) observations. A fundamental challenge in SR is the
ill-posed nature of the task, where a single LR image may correspond to
multiple valid HR reconstructions. The advent of Convolutional Neural Networks
(CNNs) [1, 2, 3, 4, 5, 6, 7, 8] has significantly advanced this task, thanks
to their ability to process local features efficiently. The recent rise in the
use of streaming media has increased the demands to restore or enhance high-
resolution images on resource-constrained high-definition devices such as
smartphones, driving the need for lightweight SR models. Due to these demands,
Transformers [9, 10, 11, 12, 13, 11, 14, 15, 16, 17, 18, 19, 20], which
outperform CNNs with relatively fewer FLoating point OPerations (FLOPs) and
parameter counts using Multi-Head Self-Attention (MHSA), are emerging as a
promising alternative.

However, we observe that FLOPs and parameter counts do not consistently align
with more direct efficiency measures, such as latency or Maximum GPU memory
Occupancy (MGO). In Table 1, we compare the FLOPs, parameter counts, latency,
and MGO required by representative CNNs [2, 3] and Transformers [12, 16] to
process a single image. Surprisingly, despite having on average 8.1$\times$
and 9.5$\times$ as many FLOPs and parameter counts, respectively, CNNs
actually outperform Transformers in efficiency, exhibiting on average
2.4$\times$ lower latency and 4.9$\times$ lower maximum GPU memory occupancy.
Therefore, this study aims to enhance a CNN-based model by incorporating the
factors that contribute to Transformers’ superior performance while
maintaining computational efficiency.

Transformers are known to outperform CNNs because they can handle long-range
dependencies and generate instance-dependent weights[12]. Through rigorous
exploration and experiments, we develop an efficient CNN-based model that
successfully integrates the Transformer’s capabilities for handling long-range
dependencies and providing instance-dependent weighting. Firstly, to improve
efficiency, our approach focuses on processing only a specified chunk of input
features. While previous methods [21, 22] lacked clear criteria for dividing
the chunk, our rigorous experiments suggest optimal criteria for this division
in terms of latency. Although it is common to enlarge the receptive field of
CNNs by stacking multiple small convolutional kernels [23], our investigation
demonstrates that using a single large kernel offers faster latency with
similar MGO levels compared to stacking small kernels multiple times.
Furthermore, our visual analysis (see in Figure 4 and Figure 5) indicates that
employing a single large kernel aligns more closely with the Transformer’s
ability to capture low-frequency features, such as edges and structures,
compared to the use of multiple small kernels. Finally, while dynamic
convolution offers a straightforward way of generating instance-dependent
weights, our experimental findings indicate that it adversely affects latency.
Thus, we leverage Element-wise Attention (EA) into our model, which assigns
individual attention weights to each element within the input feature tensor,
optimizing performance.

Drawing on these insights, we introduce Partial Large Kernel CNNs for
Efficient Super-Resolution (PLKSR). Our PLKSR utilizes advantages from both
CNNs and Transformers, achieving state-of-the-art performance on four datasets
at scale $\times$4 while achieving 42.3% lower latency and 45.6% lower MGO
than ELAN-light [13]. The tiny variants of PLKSR maintain high performance
even when scaled down, demonstrating superior efficiency compared to other
approaches that utilize large receptive fields. The ablation study confirms
that each proposal in our model significantly contributes to performance
improvements. Compared to tiny variants of PLKSR (PLKSR-tiny) and other
approaches with large receptive fields on an edge device (iPhone 12), Our
PLKSR-tiny demonstrates the lowest latency, highlighting its superior
efficiency. Additionally, by visualizing the pixels used by the model to
reconstruct images, we demonstrate that compared to other methods, PLKSR
effectively utilizes the long-range dependencies captured by its large
kernels.

Our contributions can be summarized as follows:

  * •

We develop a CNN-based model incorporating the Transformer’s capabilities for
handling long-range dependencies and instance-dependent weighting, optimizing
processing and division of input features for enhanced efficiency.

  * •

Our proposed model achieves state-of-the-art performance on four datasets with
significant reductions in latency and MGO.

  * •

We demonstrate through visual analysis and empirical testing that PLKSR
effectively captures essential low-frequency features for super-resolution,
similar to transformers.

##  2 Related Work

###  2.1 Super-Resolution Models

Since SRCNN [1] has shown better performance than existing super-resolution
methods, various researches [2, 3, 24, 4, 8, 5] have proposed super-resolution
models using CNNs. Recently, Transformers [12, 11] outperformed CNNs with
lower FLOPs and parameter counts due to their ability to handle long-range
dependencies. Many researchers have enhanced Transformer by widening the
receptive field [16, 14], extracting various features [15, 18, 17], or
speeding up inference speed by removing layer normalization and proposing
simplified MHSA [13, 20].

###  2.2 Large kernel CNNs

Since VGGNet [23], most CNNs have used 3$\times$3 kernel convolution, but
recent studies reported comparable performance to Transformer by using large
kernel convolution in CNNs. For example, ConvNeXt [25] performed comparably to
Swin Transformer [10] by leveraging a 7$\times$7 depth-wise convolution (DWC),
and DWNet [26] introduced dynamic convolution focusing on similarity of
attention and DWC. RepLKNet [27] and SLaK [28] employed inverse implicit
GEneralized Matrix Multiplication (iGEMM) DWC for their efficiency and scaled
up the kernel size up to 51$\times$51\. Also, in the Super-Resolution task,
there have been efforts to mimic MHSA operations with DWC [6], max pooling
[7], and dynamic convolution [19].

###  2.3 Partial Channel Design

FasterNet [21] proposed Partial Convolution (PConv), which performs
convolution operations on a subset of channels. SHViT [22] reduced multi-head
redundancy by computing single-head self-attention on the subset of channels.
PCEVA [8] extracted multi-scale features using PConvs sequentially with
decreasing partial convolution channels.

##  3 Analysis

###  3.1 Discripency Btw. Direct and Indirect Metrics

Table 1:  Discrepancies between FLOPs/parameter counts and latency/MGO. All
metrics are measured by restoring an HD (1280$\times$720) image at scale
$\times$2 using RTX4090 GPU at FP16 precision.

Arch. | Methods |  | #FLOPs
---
(G)
| #Params
---
(K)
| Latency
---
(ms)
| MGO
---
(mb)
CNNs | EDSR-baseline [2] | 316 | 1370 | 9.9 | 320.2
RCAN [3] | 3530 | 15444 | 101.8 | 401.2
Transformers | SwinIR-light [12] | 243.7 | 910 | 124.9 | 1764.3
SRFormer-light [16] | 229.4 | 853 | 152.7 | 1744.4

Previous research has focused on Transformers for SR tasks, considering them
more efficient than CNNs because they require fewer parameters and FLOPs.
However, recent studies [29, 30, 21] have argued that parameter counts and
FLOPs may not accurately indicate a model’s efficiency. To verify this in SR
tasks, we measure the parameter counts and FLOPs of representative CNN-based
[2, 3] and Transformer-based [12, 16] models, along with more direct
indicators such as latency and Maximum GPU memory Occupancy (MGO). In Table 1,
EDSR-baseline has 1.4$\times$ and 1.6$\times$ more FLOPs and Parameters than
SRFormer-light, which seems inefficient, but it is 15.5$\times$ faster in
terms of latency and has 5.4$\times$ less MGO. Furthermore, RCAN has
15.4$\times$ and 18.1$\times$ more FLOPs and parameter counts than SRFormer-
light, respectively, but is 1.5$\times$ faster in latency and has 4$\times$
fewer on MGO. This can be explained by the fact that the convolution module
has a more efficient memory access pattern than the Multi-Head Self-Attension
(MHSA) module [31] and rarely uses operations that increase latency while
adding little FLOPs, such as reshaping or layer normalization [29].

###  3.2 Importance of Large Kernel

The ability to deal with long-range dependency distinguishes Transformers from
traditional CNNs as a key advantage. However, in the SR task, which primarily
processes large images, simply imitating this advantage with convolution
results in significant computational overhead. Inspired by previous studies
[21, 22], we address long-range dependencies in only a subset of channels. As
shown in Figure 2 (a), we observed that latency increases are minimal up to 16
channels. Therefore, in our study, we handle long-range dependencies at 16
channels, and the careful selection of channels differs from previous studies.
In CNNs, implementing a large receptive field is typically achieved by
utilizing either a single large kernel or stacked 3$\times$3 kernels. By
measuring direct metrics for both approaches, we confirm that the single large
kernel has lower latency and slightly higher memory usage, as shown in Figure
2 (b). Furthermore, we compare the performance of the single large kernel and
the stacked 3$\times$3 kernel in our small variant model. As shown in Table 2,
the single large kernel has lower latency and better performance than the
stacked 3$\times$3 kernel.

Figure 2:  Analysis of convolution operation. (a) demonstrates the change in latency of convolution operation as the channel and kernel size change, and (b) demonstrates the latency, MGO, and parameters of using a large kernel directly and using successive small kernels (w/ and w/o GELU) with the same receptive field as the large kernel when computing convolution on 16 channels. All metrics are measured by processing a feature map with a size of 640$\times$360 using RTX4090 GPU at FP16 precision.  Table 2: Comparison of different approaches to enlarge receptive field. All metrics are measured by restoring an HD (1280$\times$720) image using RTX4090 GPU at FP16 precision. The best results are bolded. Methods |  | MGO
---
(mb)
| Latency
---
(ms)
| Performance
---
(PSNR, Urban100)
PConv3×3 ($\times 6$) | 226.5 | 17.2 | 32.35
PConv13×13 | 228.5 | 16.5 | 32.58

##  4 Proposed Methods

Figure 3:  Overview of our architecture. The PLK (Partial Large Kernel) Block,
the main block of PLKSR, consists of three modules: DCCM (Double Conv Channel
Mixer), PLKC (Partial Large Kernel Conv), and EA (Element-wise Attention).

In this section, we demonstrate our proposed model for SR, as shown in Figure
3 Initially, the low-resolution image $I^{LR}$ is processed through a
3$\times$3 convolution, followed by the N numbers of PLK Blocks sequentially,
and concluded with another 3$\times$3 convolution to generate $F_{h}$. This is
formulated as Equation 1.

$\begin{gathered}F^{0}_{in}=\mathrm{Conv_{3\times 3}}(I^{LR})\\\
F^{N}_{out}=\mathrm{PLKBlock}^{N}(~{}\dotsi~{}\mathrm{PLKBlock}^{0}(F^{0}_{in})~{}\dotsi~{})\\\
F_{h}=\mathrm{Conv_{3\times 3}}(F^{N}_{out})\vspace{-0.2cm}\end{gathered}$ (1)

In a parallel path, as suggested in previous work [4], the $I^{LR}$’s RGBs are
repeated $\mathrm{r^{2}}$ times to produce $F_{l}$ as depicted in Equation 2,
where $r$ denotes the upscaling factor. Following the $\mathrm{PixelShuffle}$
operation [32], $F_{l}$ becomes the nearest interpolated image.

$\begin{gathered}F_{l}=\mathrm{Repeat}_{r^{2}}(I^{LR}),\quad
F_{l}\in\mathbb{R}^{3\cdot r^{2}\times h\times w}\end{gathered}$ (2)

Ultimately, the final feature map $F$ is created by adding $F_{h}$ and $F_{l}$
from each respective path, and the upscaled image $I^{SR}$ is reconstructed by
applying the $\mathrm{PixelShuffle}$ operation which moves channel dimension
into spatial dimension. This is equivalent to Equation 3, where $h$ and $w$
denote the feature map’s height and width, respectively.

$\begin{gathered}F=F_{h}+F_{l}\\\ I^{SR}=\mathrm{PixelShuffle}(F),\quad
I^{SR}\in\mathbb{R}^{3\times h\cdot r\times w\cdot r}\end{gathered}$ (3)

###  4.1 Partial Large Kernel Block

The PLK Block consists of three primary modules: the Doubled Convolutional
Channel Mixer (DCCM) for local feature extraction, the Partial Large Kernel
Convolution (PLKC) to deal with long-range dependencies, and the Element-wise
Attention (EA) module for instance-dependent modulation. The incoming feature
$F_{in}$ to the PLK Block is processed sequentially through three main
modules, followed by a final 1$\times$1 convolution that is subsequently added
back to $F_{in}$, as illustrated in Equation 4.

$\begin{gathered}F_{local}=\mathrm{DCCM}(F_{in})\\\
F_{cat}=\mathrm{PLKC}(F_{local})\\\ F_{att}=\mathrm{EA}(F_{cat})\\\
F_{out}=F_{in}+\mathrm{Conv_{1\times 1}}(F_{att})\end{gathered}$ (4)

####  4.1.1 Double Convolutional Channel Mixer.

In Transformers, Feedforward Neural Networks (FFN) are often utilized to deal
with channel information. With $X$ and $Y$, each denoting the input and output
respectively, FFN can be expressed by Equation 5. The $\mathrm{MLP^{proj}}$
expands the channel’s dimension($\mathbb{R}^{c\times h\times
w}\rightarrow\mathbb{R}^{2\cdot c\times h\times w}$), and the
$\mathrm{MLP^{agg}}$ aggregates channel information by reducing the channel to
its original dimension($\mathbb{R}^{2\cdot c\times h\times
w}\rightarrow\mathbb{R}^{c\times h\times w}$) where $c$ denotes feature map’s
channel size.

$\displaystyle\begin{split}Y=\mathrm{MLP^{agg}}(\mathrm{GELU}(\mathrm{MLP^{proj}}(X)))\end{split}$
(5)

Recently, several variants [7, 19, 16, 17] have been introduced, incorporating
a convolution layer into FFNs to enhance high-frequency details. Disregarding
residual skips or gating mechanisms, these approaches can be fundamentally
categorized into two types.

$\displaystyle Y=\mathrm{MLP^{agg}}(\mathrm{GELU}(\mathrm{Conv^{proj}_{3\times
3}}(X)))$ (6) $\displaystyle Y=\mathrm{Conv^{agg}_{3\times
3}}(\mathrm{GELU}(\mathrm{MLP^{proj}}(X)))$ (7)

Equation 6 corresponds to the Convolutional Channel Mixer (CCM) [7], while
Equation 7 introduces the concept of the Inverted Convolutional Channel Mixer
(ICCM). Additionally, we introduce the Double Convolutional Channel Mixer
(DCCM), which substitutes all MLPs with the 3$\times$3 convolutions,
contrasting with existing methods that leverage the 3$\times$3 convolution
only once.

$\displaystyle\begin{split}Y=\mathrm{Conv^{agg}_{3\times
3}}(\mathrm{GELU}(\mathrm{Conv^{proj}_{3\times 3}}(X))\end{split}$ (8)

We replace all MLPs with 3x3 convolution to improve the ability to extract
local features and experimentally confirm that this performs better than other
channel mixing methods. Within the PLK Block, the input $X$ corresponds to
$F_{in}$, and the output $Y$ corresponds to $F_{local}$.

####  4.1.2 Partial Large Kernel convolution Module.

Upon entering the Partial Large Kernel Convolution (PLKC) module, the input
feature map $F_{local}$ is divided into two feature maps based on the C
channels: $F_{conv}$ for convolution and $F_{id}$ for identity. Subsequently,
the module applies a K$\times$K large kernel convolution exclusively on
$F_{conv}$, generating a large kernel-filtered feature map, denoted as
$F_{global}$. After then, $F_{global}$ is concatenated with $F_{id}$ channel-
wise, resulting in $F_{cat}$. This is formulated as Equation 9.

$\displaystyle\begin{split}F_{conv},F_{id}=\mathrm{Split_{channel}}([F_{local}],C)\\\
F_{global}=\mathrm{Conv_{K\times K}}(F_{conv})\\\
F_{cat}=\mathrm{Concat_{channel}}([F_{global},F_{id}])\end{split}$ (9)

Table 3: Comparisons of the other SR methods trained on DIV2K Datasets. All
metrics are measured by restoring an HD (1280$\times$720) image using RTX4090
GPU at FP16 precision. MGO means maximum GPU memory occupancy. Performances
are measured using (PSNR/SSIM) † denotes that since their codes were
unavailable, we re-implemented them. See supplement for implementation
details. The best and second-best results are bolded and underlined,
respectively.

Methods | Scale |  | Latency
---
(ms)
| MGO
---
(mb)
Set5 | Set14 | BSD100 | Urban100 | Manga109
EDSR[2] | $\times$2 | 196.6 | 1488.4 | 38.11/0.9602 | 33.92/0.9195 | 32.32/0.9013 | 32.93/0.9351 | 39.10/0.9773
SwinIR-light[12] | 124.9 | 1764.3 | 38.14/0.9611 | 33.86/0.9206 | 32.31/0.9012 | 32.76/0.9340 | 39.12/0.9783
ELAN-light[13] | 50.2 | 687.0 | 38.17/0.9611 | 33.94/0.9207 | 32.30/0.9012 | 32.76/0.9340 | 39.11/0.9782
SwinIR-NG[33] | 143.9 | 1712.4 | 38.17/0.9612 | 33.94/0.9205 | 32.31/0.9013 | 32.78/0.9340 | 39.20/0.9781
OmniSR[15] | 76.6 | 892.2 | 38.22/0.9613 |  33.98/0.9210 |  32.36/0.9020 |  33.05/0.9363 | 39.28/0.9784
CRAFT[18] | 107.2 | 632.8 |  38.23/0.9615 | 33.92/0.9211 |  32.33/0.9016 | 32.86/0.9343 |  39.39/0.9786
SRFormer-light[16] | 152.7 | 1744.4 |  38.23/0.9613 | 33.94/0.9209 |  32.36/0.9019 | 32.91/0.9353 | 39.28/0.9785
DLGSANet-light[19] | 272.9 | 562.2 | 38.20/0.9612 | 33.89/0.9203 | 32.30/0.9012 | 32.94/0.9355 | 39.29/0.9780
DITN†[20] | 62.8 | 679.1 | 38.17/0.9611 | 33.79/0.9199 | 32.32/0.9014 | 32.78/0.9343 | 39.21/0.9781
PLKSR (Ours) | 49.6 | 241.9 |  38.25/0.9613 |  34.03/0.9214 |  32.36/0.9020 |  32.99/0.9365 |  39.31/0.9781
EDSR[2] | $\times$3 | 85.9 | 1318.9 | 34.65/0.9280 | 30.52/0.8462 | 29.25/0.8093 | 28.80/0.8653 | 34.17/0.9476
SwinIR-light[12] | 50.6 | 810.4 | 34.62/0.9289 | 30.54/0.8463 | 29.20/0.8082 | 28.66/0.8624 | 33.98/0.9478
ELAN-light[13] | 21.1 | 284.9 | 34.61/0.9288 | 30.55/0.8463 | 29.21/0.8081 | 28.69/0.8624 | 34.00/0.9478
SwinIR-NG[33] | 60.1 | 788.2 | 34.64/0.9293 | 30.58/0.8471 | 29.24/0.8090 | 28.75/0.8639 |  34.22/0.9488
OmniSR[15] | 27.6 | 409.3 |  34.70/0.9294 | 30.57/0.8469 |  29.28/0.8094 |  28.84/0.8656 |  34.22/0.9487
CRAFT[18] | 43.5 | 284.7 |  34.71/0.9295 |  30.61/0.8469 | 29.24/0.8093 | 28.77/0.8635 |  34.29/0.9491
SRFormer-light[16] | 58.9 | 783.0 | 34.67/0.9296 | 30.57/0.8469 | 29.26/0.8099 | 28.81/0.8655 | 34.19/0.9489
DLGSANet-light[19] | 101.8 | 251.7 |  34.70/0.9295 | 30.58/0.8465 | 29.24/0.8089 | 28.83/0.8653 | 34.16/0.9483
DITN†[20] | 21.8 | 336.3 | 34.63/0.9290 | 30.55/0.8467 | 29.23/0.8088 | 28.68/0.8630 | 34.15/0.9482
PLKSR (Ours) | 18.5 | 131.9 |  34.70/0.9292 |  30.60/0.8473 |  29.27/0.8096 |  28.86/0.8666 | 34.13/0.9485
EDSR[2] | $\times$4 | 51.8 | 1445.5 | 32.46/0.8968 | 28.80/0.7876 | 27.71/0.7420 | 26.64/0.8033 | 31.02/0.9148
SwinIR-light[12] | 28.7 | 462.7 | 32.44/0.8976 | 28.77/0.7858 | 27.69/0.7406 | 26.47/0.7980 | 30.92/0.9151
ELAN-light[13] | 18.9 | 173.5 | 32.43/0.8975 | 28.78/0.7858 | 27.69/0.7406 | 26.54/0.7982 | 30.92/0.9150
SwinIR-NG[33] | 48.5 | 450.9 | 32.44/0.8980 | 28.83/0.7870 | 27.73/0.7418 | 26.61/0.8010 | 31.09/0.9161
OmniSR[15] | 24.0 | 238.3 | 32.49/0.8988 | 28.78/0.7859 | 27.71/0.7415 | 26.64/0.8018 | 31.02/0.9151
CRAFT[18] | 43.6 | 212.3 |  32.52/0.8989 |  28.85/0.7872 | 27.72/0.7418 | 26.56/0.7995 |  31.18/0.9168
SRFormer-light[16] | 34.2 | 477.2 | 32.51/0.8988 | 28.82/0.7872 |  27.73/0.7422 |  26.67/0.8032 |  31.17/0.9165
DLGSANet-light[19] | 70.3 | 143.7 |  32.54/0.8993 | 28.84/0.7871 |  27.73/0.7415 | 26.66/0.8033 | 31.13/0.9161
DITN†[20] | 18.5 | 253.6 |  32.54/0.8988 |  28.86/0.7874 | 27.72/0.7420 | 26.54/0.8001 | 30.98/0.9153
PLKSR (Ours) | 10.9 | 94.4 |  32.54/0.8996 | 28.84/0.7880 |  27.74/0.7424 |  26.69/0.8054 | 31.10/0.9164

####  4.1.3 Element-wise Attention Module.

Element-wise Attention (EA) is an attention mechanism [24] that maintains the
spatial and channel dimension size intact, distinguishing it from other
attention mechanisms, such as spatial/channel attention, that typically
involve dimensionality reduction [34, 35, 36]. We leverage this module for
instance-dependent modulation similar to Transformers, as detailed in Equation
10.

$\begin{gathered}F_{att}=F_{cat}\odot\mathrm{Sigmoid}(\mathrm{Conv_{3\times
3}}(F_{cat}))\end{gathered}$ (10)

Empirically, we find that leveraging EA only improves performance for deeper
models, so we exclude it in tiny variants.

##  5 Experiments

###  5.1 Implementation Details

We implement our models using Torch 2.1.0, CUDA 12.1, and BasicSR. The PLKSR
model consists of 28 PLK blocks and 64 channels, employing a PLKC
configuration of 16 channels (C) and a kernel size (K) of 17. The PLKSR-tiny
model scales down to 12 PLK blocks and 64 channels, with a PLKC of 16C and a
13K, without the PA module to leverage more blocks.

###  5.2 Training Details

PLKSR is trained on DIV2K [2] datasets, and two versions of PLKSR-tiny train
on DIV2K and DF2K(DIV2K + Flicker2K [37]), respectively. We use L1 loss to
train the model using 64 patches with 96$\times$96 size, randomly flipping
patches horizontally and rotating for data augmentation. We use the Adam [38]
optimizer with $\beta 1$ = 0.9 and $\beta 2$ = 0.99 to train the model for
450k iterations. The initial learning rate is set as 2e-4 and subsequently
halved at the [100k, 200k, 300k, 400k, 425k]-th iterations. PLKSR$\times$3 and
PLKSR$\times$4 are fine-tuned using pre-trained PLKSR$\times$2 following
previous research [2, 12, 20], while PLKSR-tiny is trained from scratch for
all scales. For fine-tuning, we use the Adam optimizer with $\beta 1$ = 0.9
and $\beta 2$ = 0.99 to train the model for 50k iterations. The initial
learning rate for fine-tuning is set as 2e-4. FP16 precision is used for all
training to accelerate them.

Table 4:  Comparisons of ESR methods leveraging large kernel convolution or
MHSA(-like) operation. All metrics are measured by restoring an HD
(1280$\times$720) image using RTX4090 GPU at FP16 precision. MGO means maximum
GPU memory occupancy. Performances are measured using (PSNR/SSIM) † denotes
that since their codes were unavailable, we re-implemented them. The best and
second-best results are bolded and underlined, respectively.

Methods | Scale |  | Latency
---
(ms)
| MGO
---
(mb)
Dataset | Set5 | Set14 | BSD100 | Urban100 | Manga109
DITN-tiny† [20] | $\times$2 | 20.2 | 569.8 | DIV2K | 38.00/0.9604 | 33.70/0.9192 | 32.16/0.8995 | 32.08/0.9282 | 38.59/0.9769
DITN-real [20] | 18.8 | 543.2 | 38.01/0.9605 | 33.65/0.9184 | 32.16/0.8995 | 31.96/0.9273 | 38.49/0.9767
PLKSR-tiny (Ours) | 16.6 | 228.5 |  38.11/0.9608 |  33.73/0.9193 |  32.25/0.9008 |  32.43/0.9314 |  38.84/0.9775
ShuffleMixer [6] | 28.4 | 356.5 | DF2K | 38.01/0.9606 | 33.63/0.9180 | 32.17/0.8995 | 31.89/0.9257 | 38.83/0.9774
SAFMN [7] | 12.5 | 213.6 | 38.00/0.9605 | 33.54/0.9177 | 32.16/0.8995 | 31.84/0.9256 | 38.71/0.9771
PLKSR-tiny (Ours) | 16.6 | 228.5 |  38.14/0.9610 |  33.81/0.9199 |  32.29/0.9011 |  32.58/0.9328 |  39.18/0.9782
DITN-tiny† [20] | $\times$3 | 7.4 | 284.6 | DIV2K | 34.38/0.9271 | 30.37/0.8435 | 29.10/0.8057 | 28.14/0.8529 | 33.56/0.9447
DITN-real [20] | 7.1 | 318.1 | 34.33/0.9266 | 30.33/0.8424 | 29.08/0.8051 | 28.06/0.8512 | 33.44/0.9441
PLKSR-tiny (Ours) | 5.9 | 103.2 |  34.50/0.9279 |  30.45/0.8447 |  29.15/0.8070 |  28.35/0.8571 |  33.71/0.9460
ShuffleMixer [6] | 10.5 | 253.2 | DF2K | 34.40/0.9272 | 30.37/0.8423 | 29.12/0.8051 | 28.08/0.8498 | 33.69/0.9448
SAFMN [7] | 7.1 | 95.73 | 34.34/0.9267 | 30.33/0.8418 | 29.08/0.8048 | 27.95/0.8474 | 33.52/0.9437
PLKSR-tiny (Ours) | 5.9 | 103.2 |  34.54/0.9282 |  30.48/0.8455 |  29.20/0.8079 |  28.51/0.8599 |  34.05/0.9473
DITN-tiny† [20] | $\times$4 | 6.5 | 248.2 | DIV2K | 32.16/0.8943 | 28.62/0.7825 | 27.59/0.7370 | 26.04/0.7853 | 30.40/0.9076
DITN-real [20] | 5.0 | 240.3 | 32.10/0.8940 | 28.59/0.7813 | 27.57/0.7363 | 25.99/0.7837 | 30.29/0.9068
PLKSR-tiny (Ours) | 3.6 | 65.3 | 32.18/0.8956 |  28.67/0.7838 |  27.61/0.7380 |  26.12/0.7888 | 30.52/0.9087
ShuffleMixer [6] | 10.1 | 242.1 | DF2K |  32.21/0.8953 | 28.66/0.7827 |  27.61/0.7366 | 26.08/0.7835 |  30.65/0.9093
SAFMN [7] | 7.0 | 54.7 | 32.18/0.8948 | 28.60/0.7813 | 27.58/0.7359 | 25.97/0.7809 | 30.43/0.9063
PLKSR-tiny (Ours) | 3.6 | 65.3 |  32.33/0.8970 |  28.76/0.7857 |  27.68/0.7398 |  26.34/0.7942 |  30.83/0.9119

###  5.3 Quantitative Results

To assess the efficiency of our SR model, we utilize latency and MGO as the
main metrics. These metrics are measured while restoring HD (1280$\times$720)
image using an RTX4090 GPU at FP16 precision. MGO is measured by
torch.cuda.max_memory_allocated function provided by PyTorch. For performance
evaluation, we utilize the Peak Signal-to-Noise Ratio (PSNR) and the
Structural Similarity Index Measure (SSIM), calculated on the Y channel in the
YCbCr space after cropping the image boundary by a factor equivalent to the
scaling factor. We assess performance across five widely used datasets: Set5
[39], Set14 [40], BSD100 [41], Urban100 [42], and Manga109 [43].

In our comparative analysis detailed in Table 3, PLKSR is evaluated with SOTA
SR methods, including EDSR [2], SwinIR-light [12], ELAN-light [13], SwinIR-NG
[33], OmniSR [15], SRFormer-light [16], DLGSANet-light [19], and DITN [20]. At
a scaling factor of $\times$2, PLKSR shows similar latency ELAN-light while
achieving the best PSNR on the Set5, Set14, and BSD100 datasets. Remarkably,
against SRFormer-light on the BSD100$\times$2, PLKSR achieves the same
performance with significantly reduced latency and MGO(up to 68.5%, 86.1%
lower, respectively). At a scale factor of $\times$4, PLKSR outperforms ELAN-
light with a 43.3% reduction in latency, simultaneously recording the highest
PSNR values on Set5, BSD100, and Urban100 datasets. Although some models do
not use pre-training strategies, PLKSR outperforms SOTA models in both
performance and efficiency, confirming PLKSR’s outstanding efficiency-
performance trade-off.

To highlight PLKSR’s efficiency and scalability, we evaluate PLKSR-tiny with
SOTA lightweight SR models employing large kernel convolution [6] or
MHSA(-like) mechanisms [7, 20]. As detailed in Table 4, PLKSR-tiny outperforms
its competitors by achieving a PSNR above 39 on the Manga109 dataset at a
scaling factor of $\times$2, while also showcasing the second lowest latency
and MGO. Impressively, at a scaling factor of $\times$4, PLKSR-tiny exhibits
28% lower latency compared to DITN-real and achieves the top performance
across all evaluated datasets. This suggests that PLKC is scalable and is the
most suitable implementation for dealing with long-range dependencies in
lightweight models.

Table 5:  Ablation study. The KS, CM, and EA are Kernel Size, Channel Mixer,
and Element-wise Attention, respectively. All metrics are measured by
restoring an HD (1280$\times$720) image using RTX4090 GPU at FP16 precision.
Performances are measured using (PSNR/SSIM). The best results are bolded.

Blocks | KS | CM | EA |  | Latency
---
(ms)
| MGO
---
(mb)
Set5 | B100
40 | 5 | CCM | $\times$ | 36.7 | 264.5 | 38.15/0.9611 | 32.31/0.9015
9 | 39.0 | 267.8 | 38.18/0.9609 | 32.32/0.9017
13 | 43.5 | 272.9 | 38.18/0.9610 | 32.33/0.9017
17 | 48.2 | 280.0 | 38.19/0.9610 | 32.33/0.9017
ICCM | 47.7 | 251.8 | 38.20/0.9612 | 32.32/0.9016
32 | DCCM | 48.1 | 241.1 | 38.21/0.9612 | 32.34/0.9018
28 | ✔ | 49.6 | 241.9 | 38.25/0.9613 | 32.36/0.9020

###  5.4 Ablation Study

Our ablation study evaluates the impact of kernel size, channel mixer choices,
and the integration of EA. As shown in Table 5, the study reveals a positive
correlation between increased kernel size and improved performance,
highlighting the benefits of larger convolutional kernels. Among various
channel mixers assessed under similar latency, the DCCM emerges as the
superior option, delivering the highest performance. Moreover, adapting the EA
module achieves the most favorable balance of performance and latency,
showcasing the effectiveness of our design choices in optimizing SR model
efficiency.

Table 6:  Comparison of mobile latency on ESR models leveraging large kernel
convolution or MHSA(-like) operation. All latencies are measured at two input
sizes using the CoreML library on iPhone 12. † means that the code was not
available, so we reimplemented it. § means that the code contains an operation
that cannot be converted using CoreML, so we reimplemented it. The best and
second-best results are bolded and underlined, respectively.

Methods | Scale | Latency (ms)
---|---|---
$X\in\mathbb{R}^{3\times 96\times 96}$ | $X\in\mathbb{R}^{3\times 360\times 640}$
ShuffleMixer§ [6] | $\times$2 | 8.6 | OOM
SAFMN [7] | 36.4 | 224.3
DITN-tiny† [20] | 35.2 | OOM
DITN-real [20] | 26.2 | 1460.6
PLKSR-tiny (Ours) | 6.0 | 181.0

###  5.5 Comparison on Mobile device

To validate the practicality and wide applicability of the architecture in
real-world scenarios, we evaluate the latency of PLKSR-tiny on a mobile device
(iPhone 12). We compare PLKSR-tiny with other ESR models that employ either
large kernel convolution [6] or MHSA(-like) mechanisms [7, 20] on two image
sizes. As shown in Table 6, PLKSR-tiny achieves the lowest latency for both
image sizes among its competitors. These results demonstrate that the PLKC
module is the most efficient implementation for dealing with long-range
dependencies on real-world edge devices.

###  5.6 Large Kernel Analysis

MHSA effectively captures low-frequency features, such as shapes and edges, in
contrast to convolution, which captures high-frequency textures [44]. To
validate this, we visualize the relative log amplitude after Fourier-
transforming the MHSA feature maps of SRFormer-light$\times$2 and the
large/small kernel feature maps of PLKSR$\times$2\. In Figure 4, It can be
shown that large kernels lean towards low-frequency features similar to MHSA,
while small kernels capture high-frequency features in contrast to the MHSA
and large kernel.

Further visualization in Figure 5 highlights the large kernel’s ability to
capture structural features, such as facial contours, and the small kernel’s
ability to capture textural details found in objects, such as hair or hat.
This means that PLKC can easily capture features that are difficult to capture
with the small kernel, similar to MHSA. This explains PLKSR’s outstanding
performance for the complementary use of two features.

Figure 4:  $\Delta$Log amplitude of Fourier-transformed MHSA feature maps of
SRFormer-light$\times$2 and large/small kernel feature maps of
PLKSR$\times$2\. We visualize diagonal values after the center of each
Fourier-transformed feature map following previous research [44].  Figure 5:
Feature map activation visualization. The large/small kernel feature maps of
the last PLK block from PLKSR$\times$2 are averaged channel-wise and
normalized for visualization.  Figure 6:  LAM [45] results of SRFormer-light
[16] and PLKSR on challenging examples at Urban100$\times$4\. LAM attribution
represents the range of pixels used to restore the red bounding box patch, and
the area of contribution represents the density of the LAM.

###  5.7 Comparison on LAM

To demonstrate that PLKSR practically utilizes long-range dependencies
captured by PLKC to reconstruct images, we introduce LAM [45], a tool to
represent the range of pixels that the model utilizes to reconstruct a
specific region and compare it to SRFormer-light [16]. Our model uses a
17$\times$17 large kernel, while SRFormer-light uses a 16$\times$16 window
MHSA, which is a similar receptive field for comparison. As shown in Figure 6,
PLKSR utilizes a wider range of pixels to reconstruct the red bounding box
than SRFormer-light. Surprisingly, PLKSR utilizes almost all pixels in regions
adjacent to the patch while it captures mostly edges in regions far away from
the patch, which is consistent with previous experiments that demonstrated
PLKC’s structural bias.

Figure 7:  Visual comparisons with the other SR models on the Urban100
dataset. Visualize the result of upscaling the red bounding box and include
PSNR. The best and second-best results are bolded and underlined,
respectively.

###  5.8 Visual Results

To illustrate the improved visual quality achieved by PLKSR, we visually
compare the upscaled results with the other SOTA SR models, SwinIR-light [12],
OmniSR [15], CRAFT [18], and SRFormer-light [16] on Urban100 dataset. In
Figure 7, a detailed zoom on the areas within the red bounding boxes showcases
the precision with which each model performs upscaling alongside the PSNR for
these reconstructions. Dissimilar to competitors, which often yield over-
smoothed textures or fail to capture details, PLKSR successfully reconstructs
the image and captures the edge. This strongly supports the fact that PLKSR
produces more visually pleasing results than other SR models.

##  6 Conclusion

This study demonstrated that contrary to the common belief that CNNs are
inefficient compared to transformers, they are actually much more efficient
based on direct metrics. We reduced the computational overhead by
incorporating the advantages of transformers into CNNs, utilizing 17$\times$17
convolution for large receptive fields and element-wise attention for context-
dependent weights. PLKSR achieved state-of-the-art performance at scale
$\times$4 on four datasets, outperforming ELAN-light with a 42% reduction in
latency. Our experiments also showed that PLKC performed similarly to MHSA on
modern GPUs and edge devices, but was significantly more efficient. We believe
PLKC will spark a comeback for CNNs in SR tasks, which currently receive less
attention than transformers.

## References

  * [1] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang.  Image super-resolution using deep convolutional networks.  IEEE transactions on pattern analysis and machine intelligence, 38(2):295–307, 2015.
  * [2] Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee.  Enhanced deep residual networks for single image super-resolution.  In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 136–144, 2017.
  * [3] Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu.  Image super-resolution using very deep residual channel attention networks.  In Proceedings of the European conference on computer vision (ECCV), pages 286–301, 2018.
  * [4] Zongcai Du, Jie Liu, Jie Tang, and Gangshan Wu.  Anchor-based plain net for mobile image super-resolution.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2494–2502, 2021.
  * [5] Bin Sun, Yulun Zhang, Songyao Jiang, and Yun Fu.  Hybrid pixel-unshuffled network for lightweight image super-resolution.  In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 2375–2383, 2023.
  * [6] Long Sun, Jinshan Pan, and Jinhui Tang.  Shufflemixer: An efficient convnet for image super-resolution.  Advances in Neural Information Processing Systems, 35:17314–17326, 2022.
  * [7] Long Sun, Jiangxin Dong, Jinhui Tang, and Jinshan Pan.  Spatially-adaptive feature modulation for efficient image super-resolution.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13190–13199, 2023.
  * [8] Zhou Zhou, Jiahao Chao, Jiali Gong, Hongfan Gao, Zhenbing Zeng, and Zhengfeng Yang.  Enhancing real-time super resolution with partial convolution and efficient variance attention.  In Proceedings of the 31st ACM International Conference on Multimedia, pages 5348–5357, 2023.
  * [9] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [10] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021.
  * [11] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang.  Restormer: Efficient transformer for high-resolution image restoration.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5728–5739, June 2022.
  * [12] Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte.  Swinir: Image restoration using swin transformer.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 1833–1844, 2021.
  * [13] Xindong Zhang, Hui Zeng, Shi Guo, and Lei Zhang.  Efficient long-range attention network for image super-resolution.  In European conference on computer vision, pages 649–667. Springer, 2022.
  * [14] Xiangyu Chen, Xintao Wang, Jiantao Zhou, Yu Qiao, and Chao Dong.  Activating more pixels in image super-resolution transformer.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22367–22377, 2023.
  * [15] Hang Wang, Xuanhong Chen, Bingbing Ni, Yutian Liu, and Jinfan Liu.  Omni aggregation networks for lightweight image super-resolution.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22378–22387, 2023.
  * [16] Yupeng Zhou, Zhen Li, Chun-Le Guo, Song Bai, Ming-Ming Cheng, and Qibin Hou.  Srformer: Permuted self-attention for single image super-resolution.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12780–12791, 2023.
  * [17] Zheng Chen, Yulun Zhang, Jinjin Gu, Linghe Kong, Xiaokang Yang, and Fisher Yu.  Dual aggregation transformer for image super-resolution.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 12312–12321, 2023.
  * [18] Ao Li, Le Zhang, Yun Liu, and Ce Zhu.  Feature modulation transformer: Cross-refinement of global representation via high-frequency prior for image super-resolution.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12514–12524, 2023.
  * [19] Xiang Li, Jiangxin Dong, Jinhui Tang, and Jinshan Pan.  Dlgsanet: lightweight dynamic local and global self-attention networks for image super-resolution.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12792–12801, 2023.
  * [20] Yong Liu, Hang Dong, Boyang Liang, Songwei Liu, Qingji Dong, Kai Chen, Fangmin Chen, Lean Fu, and Fei Wang.  Unfolding once is enough: A deployment-friendly transformer unit for super-resolution.  In Proceedings of the 31st ACM International Conference on Multimedia, pages 7952–7960, 2023.
  * [21] Jierun Chen, Shiu-hong Kao, Hao He, Weipeng Zhuo, Song Wen, Chul-Ho Lee, and S-H Gary Chan.  Run, don’t walk: Chasing higher flops for faster neural networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12021–12031, 2023.
  * [22] Seokju Yun and Youngmin Ro.  Shvit: Single-head vision transformer with memory efficient macro design.  arXiv preprint arXiv:2401.16456, 2024.
  * [23] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [24] Hengyuan Zhao, Xiangtao Kong, Jingwen He, Yu Qiao, and Chao Dong.  Efficient image super-resolution using pixel attention.  In Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pages 56–72. Springer, 2020.
  * [25] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.  A convnet for the 2020s.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11976–11986, 2022.
  * [26] Qi Han, Zejia Fan, Qi Dai, Lei Sun, Ming-Ming Cheng, Jiaying Liu, and Jingdong Wang.  On the connection between local attention and dynamic depth-wise convolution, 2022.
  * [27] Xiaohan Ding, Xiangyu Zhang, Jungong Han, and Guiguang Ding.  Scaling up your kernels to 31x31: Revisiting large kernel design in cnns.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11963–11975, 2022.
  * [28] Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Tommi Kärkkäinen, Mykola Pechenizkiy, Decebal Mocanu, and Zhangyang Wang.  More convnets in the 2020s: Scaling up kernels beyond 51x51 using sparsity.  arXiv preprint arXiv:2207.03620, 2022.
  * [29] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun.  Shufflenet v2: Practical guidelines for efficient cnn architecture design.  In Proceedings of the European conference on computer vision (ECCV), pages 116–131, 2018.
  * [30] Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun.  Repvgg: Making vgg-style convnets great again.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13733–13742, 2021.
  * [31] Tri Dao.  Flashattention-2: Faster attention with better parallelism and work partitioning.  arXiv preprint arXiv:2307.08691, 2023.
  * [32] Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang.  Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1874–1883, 2016.
  * [33] Haram Choi, Jeongmin Lee, and Jihoon Yang.  N-gram in swin transformers for efficient lightweight image super-resolution.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2071–2081, 2023.
  * [34] Jie Hu, Li Shen, and Gang Sun.  Squeeze-and-excitation networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141, 2018.
  * [35] Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, and Tat-Seng Chua.  Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5659–5667, 2017.
  * [36] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon.  Cbam: Convolutional block attention module.  In Proceedings of the European conference on computer vision (ECCV), pages 3–19, 2018.
  * [37] Radu Timofte, Eirikur Agustsson, Luc Van Gool, Ming-Hsuan Yang, and Lei Zhang.  Ntire 2017 challenge on single image super-resolution: Methods and results.  In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 114–125, 2017.
  * [38] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [39] Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie Line Alberi-Morel.  Low-complexity single-image super-resolution based on nonnegative neighbor embedding.  2012\.
  * [40] Roman Zeyde, Michael Elad, and Matan Protter.  On single image scale-up using sparse-representations.  In Curves and Surfaces: 7th International Conference, Avignon, France, June 24-30, 2010, Revised Selected Papers 7, pages 711–730. Springer, 2012.
  * [41] David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics.  In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, volume 2, pages 416–423. IEEE, 2001.
  * [42] Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja.  Single image super-resolution from transformed self-exemplars.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5197–5206, 2015.
  * [43] Yusuke Matsui, Kota Ito, Yuji Aramaki, Azuma Fujimoto, Toru Ogawa, Toshihiko Yamasaki, and Kiyoharu Aizawa.  Sketch-based manga retrieval using manga109 dataset.  Multimedia tools and applications, 76:21811–21838, 2017.
  * [44] Namuk Park and Songkuk Kim.  How do vision transformers work?  In International Conference on Learning Representations, 2021.
  * [45] Jinjin Gu and Chao Dong.  Interpreting super-resolution networks with local attribution maps.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9199–9208, 2021.
  * [46] Xiaojie Chu, Liangyu Chen, Chengpeng Chen, and Xin Lu.  Improving image restoration by revisiting global information aggregation.  In European Conference on Computer Vision, pages 53–71. Springer, 2022.
  * [47] Xiaohan Ding, Yiyuan Zhang, Yixiao Ge, Sijie Zhao, Lin Song, Xiangyu Yue, and Ying Shan.  Unireplknet: A universal perception large-kernel convnet for audio, video, point cloud, time-series and image recognition.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024.

Partial Large Kernel CNNs for Efficient Super-Resolution

—— Supplementary Material ——

This supplementary material presents implementation details, results of
structural re-parameterization, comparisons of GPU-optimized implementations,
and additional comparisons not included in the manuscript.

##  Appendix A Implementation Details

This section describes the implementation details.

####  A.0.1 DITN

As DITN [20] only released the code for DITN-real, we re-implement DITN-tiny
and DITN based on the descriptions in the paper and the available code. We re-
implement DITN-tiny, replacing DITN-real’s Tanh and Conv1$\times$1 with Layer
Normalization according to Equation (7) in the paper. We also re-implement
DITN by tripling the number of UPONE unit in DITN-tiny.

####  A.0.2 DLGSANet

DLGSANet’s [19] Dynamic Convolution [26] is excluded from Automatic Mixed
Precision due to its lack of support for FP16 precision. Consequently, we have
doubled the number of CUDA threads. As the paper reports performance using the
Test-time Local Converter (TLC) [46], we also report metrics using TLC.

####  A.0.3 Mobile Conversion

Since PyTorch’s Tensor.var function cannot be converted by CoreML, we
implemente it from scratch. Additionally, since CoreML does not convert the
dynamic value assignment and torch.repeat_interleave functions used by PLKSR,
we re-implement these using split/concatenate and reshape/permute.

##  Appendix B Structural Re-parameterizations

To explore performance improvements, we consider structural re-
parameterization. We compare the performances of various structural re-
parameterizations [27, 28, 47] utilized with large kernels. However, as shown
in Table 7, the optimal performance occurs without structural re-
parameterization. As these methods usally integrate with DWC in high-level
vision tasks, we attribute this result to influences from different domains
and large kernel approaches.

##  Appendix C GPU-optimized implementations

In this section, we compare recently proposed GPU-optimized implementations
designed to address long-range with the Partial Large Kernel Convolution
(PLKC) utilized by PLKSR.

Table 7:  Comparisons on Structural Re-parameterization Methods. After
training, kernels are merged into a single kernel, and its performance metrics
(PSNR/SSIM) are measured.

Kernel sizes | Strides | Set5 | Urban100
---|---|---|---
[17$\times$17, 5$\times$5] | [1, 1] | 38.25/0.9613 | 32.96/0.9363
[17$\times$5, 5$\times$17, 5$\times$5] | [1, 1, 1] | 38.21/0.9612 | 32.97/0.9363
[17$\times$17, 5$\times$5, 9$\times$9, 5$\times$5, 5$\times$5] | [1, 1, 2, 3, 4] | 38.25/0.9614 | 32.95/0.9361
17$\times$17 | 1 | 38.25/0.9613 | 32.99/0.9365

###  C.1 Depth-Wise Convolution

In recent high-level vision tasks, several studies have employed large kernels
based on inverse implicit Generalized Matrix Multiplication (iGEMM) for Depth-
Wise Convolution (DWC) [27, 28, 47], as also noted in a previous Super-
Resolution (SR) study [6]. While exploring various implementations for large
kernels, we discover that although they prove efficient for general high-level
vision tasks, they do not yield the same efficiency for SR tasks. Table 8
shows that iGEMM DWC is significantly slower than Pytorch’s standard DWC. This
discrepancy is due to iGEMM DWC being optimized for high throughput at large
batch sizes, whereas Pytorch’s DWC is better suited for SR tasks processing
feature maps with small batch sizes and large spatial sizes. Consistently, the
iGEMM DWC shows a minimal increase in latency as the batch size increases,
while the Pytorch DWC shows a significant increase in latency. Notably, PLKC
is the most efficient, offering the lowest latency and Maximum GPU-memory
Occupancy(MGO) across all batch sizes.

Table 8:  Comparisons on Depth-wise Convolutions and PConv. Metrics are calculated using a 17$\times$17 kernel on a feature map $F\in\mathbb{R}^{64\times 640\times 360}$ via RTX4090 GPU at FP16 precision. PLKC only processes the first 16 channels of the feature map.  Batch Size | Methods | Latency (ms) | MGO (mb)
---|---|---|---
1 | Pytorch DWC | 0.6 | 35.3
iGEMM DWC | 318.4 | 35.3
PLKC (16C) | 0.2 | 44.5
64 | Pytorch DWC | 37.5 | 2250.1
iGEMM DWC | 324.7 | 2250.1
PLKC (16C) | 7.4 | 2812.9
Table 9:  Comparisons on Various Large Kernel Approaches. All metrics are
measured by restoring an HD (1280$\times$720) image using RTX4090 GPU at FP16
precision. MGO means maximum GPU memory occupancy. The best results are
bolded.

Methods | Scale |  | Latency
---
(ms)
| MGO
---
(mb)
Set5 | Set14 | BSD100 | Urban100 | Manga109
Dynamic Conv [19] | $\times$2 | 55.4 | 2216.2 | 37.67/0.9592 | 33.20/0.9145 | 31.94/0.8968 | 31.10/0.9177 | 37.61/0.9748
Depth-wise Conv | 51.1 | 216.4 | 38.18/0.9611 | 33.81/0.9198 | 32.31/0.9014 | 32.61/0.9335 | 39.00/0.9777
PLKC | 49.6 | 241.9 |  38.25/0.9613 |  34.03/0.9214 |  32.36/0.9020 |  32.99/0.9365 |  39.31/0.9781

Table 10:  Comparisons of Partial Channel Designs. All metrics are measured by
restoring an HD (1280$\times$720) image using RTX4090 GPU at FP16 precision.
MGO means maximum GPU memory occupancy. The best results are bolded.

Methods | Scale |  | Latency
---
(ms)
| MGO
---
(mb)
Set5 | Set14 | BSD100 | Urban100 | Manga109
Partial ASA [13] | $\times$2 | 50.7 | 568.5 |  38.23/0.9613 | 33.98/0.9211 | 32.34/0.9018 | 32.87/0.9355 | 39.16/0.9780
PLKC | 49.6 | 241.9 |  38.25/0.9613 |  34.03/0.9214 |  32.36/0.9020 |  32.99/0.9365 |  39.31/0.9781

###  C.2 Flash Attention

Flash Attention has recently been introduced to enhance the memory access
pattern of Multi-Head Self-Attention (MHSA), thus accelerating inference speed
and reducing MGO on GPU devices. To demonstrate the superiority of PLKC over
the most optimized MHSA implementations, we assess the latency and MGO of
various SR Transformers [12, 16] accelerated by Flash Attention-2 [31]. We
exclude attention masks and relative positional bias for shifted windows due
to their unavailability. Table 11 illustrates that, at a scaling factor of
$\times$4, SwinIRF-light, when accelerated by Flash Attention-2, shows a 49%
reduction in latency compared to the original SwinIR-light. Notably, this
result underscores the exceptional efficiency of PLKC since PLKSR remains 25%
faster than SwinIRF-light, despite this comparison being the highly favorable
setting for Transformers.

Table 11:  Comparison of Transformers accelerated by Flash Attention. F
denotes each model is converted to use Flash Attention-2 [31]. All metrics are
measured by restoring an HD (1280$\times$720) image using RTX4090 GPU at FP16
precision. The best results are bolded.

Methods | Latency (ms) / MGO (mb)
---|---
$\times$2 |  $\times$3 |  $\times$4
SwinIRF-light | 68.0 / 1235.9 | 24.9 / 573.4 | 14.6 / 328.6
SRFormerF-light | 89.7 / 1281.7 | 31.1 / 581.1 | 19.6 / 356.6
PLKSR (Ours) |  49.6 / 241.9 |  18.5 / 131.9 |  10.9 / 94.4

##  Appendix D Comparison on other LK approaches

Recent studies have explored using large kernels in SR tasks, leveraging
approaches like Depth-wise Convolution [6] and Dynamic Convolution [19]. To
demonstrate PLKC’s superiority, we substitute PLKC with other large kernel
approaches and compare their performance. We adjust the number of main blocks
in each model to equalize latency, and we omit Element-wise Attention (EA)
since Dynamic Convolution has instance-dependent weights. As indicated in
Table 9, PLKC not only achieves the lowest latency but also delivers the best
results, thereby confirming its remarkable performance.

##  Appendix E Comparison on Self-Attention

SHViT [22] has achieved state-of-the-art performance and latency by computing
single-head self-attention on a subset of channels. Given the similarity of
this approach to ours in dealing with long-range dependencies on partial
channels, we substitute PLKC with ELAN’s Accelerated Self-Attention (ASA) [13]
with partial channel design for performance comparison. We adjust the number
of main blocks to equalize latency and omit Element-wise Attention (EA) since
ASA has instance-dependent weights. As demonstrated in Table 10, PLKSR
outperforms ASA-variants across all datasets, thereby underscoring PLKC’s
outstanding efficiency and performance.

Figure 8:  More LAM results of SRFormer-light [16] and PLKSR on challenging
examples at Urban100×4. LAM attribution represents the range of pixels used to
restore the red bounding box patch, and the area of contribution represents
the density of the LAM.

##  Appendix F More Comparison on LAM

In this section, we present additional Local Attention Map (LAM) [45] results
with our PLKSR and SRFormer-light [16]. The figure 8 shows that PLKSR utilizes
a wider range of pixels compared to SRFormer-light in various examples,
demonstrating that PLKSR effectively utilizes long-range dependent features
captured by PLKC.

##  Appendix G More Visual Results

This section provides additional visual evaluations of our PLKSR alongside
other state-of-the-art methods. As demonstrated in Figure 9, PLKSR not only
achieves a higher PSNR but also delivers superior visual results, surpassing
other models.

Figure 9:  More visual results on Set14 and Urban100 datasets. The best and
second-best results are bolded and underlined, respectively.


d^{2}\bm{p}_{\bot 2}\Big{[}8\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{4}-8\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{2}+1\Big{]}\;\Delta D^{h_{1}}\,\Delta
D^{h_{2}}$ $\displaystyle\quad+\int d^{2}\bm{p}_{\bot
2}\Big{[}-\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{4}+\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{2}\Big{]}\;\Delta D^{h_{1}}\,\Delta
D^{h_{2}}$ (237) $\displaystyle\int d^{2}\bm{p}_{\bot
2}\,\cos\varphi_{2}\sin^{3}\varphi_{2}\;\Delta D^{h_{1}}\,\Delta D^{h_{2}}$
$\displaystyle=\cos\phi_{1}\sin^{3}\phi_{1}\int d^{2}\bm{p}_{\bot
2}\Big{[}8\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{4}-8\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{2}+1\Big{]}\;\Delta D^{h_{1}}\,\Delta
D^{h_{2}}$ $\displaystyle\quad+\cos\phi_{1}\sin\phi_{1}\int d^{2}\bm{p}_{\bot
2}\Big{[}-4\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{4}+5\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{2}-1\Big{]}\;\Delta D^{h_{1}}\,\Delta
D^{h_{2}}$ $\displaystyle\int d^{2}\bm{p}_{\bot
2}\,\cos^{3}\varphi_{2}\sin\varphi_{2}\;\Delta D^{h_{1}}\,\Delta D^{h_{2}}$
$\displaystyle=\cos^{3}\phi_{1}\sin\phi_{1}\int d^{2}\bm{p}_{\bot
2}\Big{[}8\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{4}-8\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{2}+1\Big{]}\;\Delta D^{h_{1}}\,\Delta
D^{h_{2}}$ $\displaystyle\quad+\cos\phi_{1}\sin\phi_{1}\int d^{2}\bm{p}_{\bot
2}\Big{[}-4\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{4}+5\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{2}-1\Big{]}\;\Delta D^{h_{1}}\,\Delta
D^{h_{2}}\,.$

We can then reconstruct directly the following quantities appearing in Section
5:

$\displaystyle\int d^{2}\bm{p}_{\perp 2}\;\cos(2\varphi_{2})\,\Delta
D^{h_{1}}\,\Delta D^{h_{2}}=\cos(2\phi_{1})\,\int d^{2}\bm{p}_{\perp
2}\,[2(\hat{\bm{p}}_{\perp 2}\cdot\hat{\bm{P}}_{1T})^{2}-1]\;\Delta
D^{h_{1}}\,\Delta D^{h_{2}}$ (240) $\displaystyle\int d^{2}\bm{p}_{\perp
2}\;\sin(2\varphi_{2})\,\Delta D^{h_{1}}\,\Delta
D^{h_{2}}=\sin(2\phi_{1})\,\int d^{2}\bm{p}_{\perp 2}\,[2(\hat{\bm{p}}_{\perp
2}\cdot\hat{\bm{P}}_{1T})^{2}-1]\;\Delta D^{h_{1}}\,\Delta D^{h_{2}}$ (241)
$\displaystyle\int d^{2}\bm{p}_{\perp 2}\;\cos(3\varphi_{2})\,\Delta
D^{h_{1}}\,\Delta D^{h_{2}}$ $\displaystyle=\cos(3\phi_{1})\int
d^{2}\bm{p}_{\perp 2}\,[4(\hat{\bm{p}}_{\perp
2}\cdot\hat{\bm{P}}_{1T})^{3}-3(\hat{\bm{p}}_{\perp
2}\cdot\hat{\bm{P}}_{1T})]\;\Delta D^{h_{1}}\,\Delta D^{h_{2}}$ (242)
$\displaystyle\int d^{2}\bm{p}_{\perp 2}\;\sin(3\varphi_{2})\,\Delta
D^{h_{1}}\,\Delta D^{h_{2}}$ $\displaystyle=\sin(3\phi_{1})\int
d^{2}\bm{p}_{\perp 2}\,[4(\hat{\bm{p}}_{\perp
2}\cdot\hat{\bm{P}}_{1T})^{3}-3(\hat{\bm{p}}_{\perp
2}\cdot\hat{\bm{P}}_{1T})]\;\Delta D^{h_{1}}\,\Delta D^{h_{2}}$ (243)
$\displaystyle\int d^{2}\bm{p}_{\bot 2}\,\cos(4\varphi_{2})\;\Delta
D^{h_{1}}\,\Delta D^{h_{2}}$ $\displaystyle=\cos(4\phi_{1})\int
d^{2}\bm{p}_{\bot 2}\Big{[}8\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{4}-8\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{2}+1\Big{]}\;\Delta D^{h_{1}}\,\Delta
D^{h_{2}}$ (244) $\displaystyle\int d^{2}\bm{p}_{\bot
2}\,\sin(4\varphi_{2})\;\Delta D^{h_{1}}\,\Delta D^{h_{2}}$
$\displaystyle=\sin(4\phi_{1})\int d^{2}\bm{p}_{\bot
2}\Big{[}8\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{4}-8\big{(}\hat{\bm{p}}_{\bot
2}\cdot\hat{\bm{P}}_{1T}\big{)}^{2}+1\Big{]}\;\Delta D^{h_{1}}\,\Delta
D^{h_{2}}\,.$ (245)

##  Appendix E Helicity frames

Our physical observables are computed in two kinematical configurations, the
thrust and the hadron frames, with axes denoted by $\hat{\bm{x}}_{L}$,
$\hat{\bm{y}}_{L}$, and $\hat{\bm{z}}_{L}$, where we use $L$ to indicate a
generic laboratory (LAB) frame. The helicity frame of a particle with momentum
$\bm{p}$ along the direction
$\hat{\bm{p}}=(\sin\theta\cos\varphi,\,\sin\theta\sin\varphi,\,\cos\theta)$ –
as defined in the laboratory frame – can be reached by performing the
rotations Leader:2001gr

$R(\varphi,\theta,0)=R_{y^{\prime}}(\theta)\,R_{z_{L}}(\varphi)\>.$ (246)

The first is a rotation by an angle $\varphi$ around the
$\hat{\bm{z}}_{L}$-axis and the second is a rotation by an angle $\theta$
around the new (that is, obtained after the first rotation)
$\hat{\bm{y}}^{\prime}$-axis. This means

$a)\;\hat{\bm{z}}=\hat{\bm{p}}\quad\quad\quad
b)\;\hat{\bm{y}}=\frac{\hat{\bm{z}}_{L}\times\hat{\bm{p}}}{|\hat{\bm{z}}_{L}\times\hat{\bm{p}}|}=\hat{\bm{z}}_{L}\times\hat{\bm{p}}_{\perp}\quad\quad\quad
c)\;\hat{\bm{x}}=\hat{\bm{y}}\times\hat{\bm{z}}\,.$ (247)

In the present study we are interested in the helicity frames of the final
quark/antiquark as well as those of the two hadrons coming from their
fragmentation. Let us describe them, starting from the relations between the
parent quark and the corresponding hadron helicity frames and then focusing
separately on the two configurations for a process
$e^{+}(k^{+})e^{-}(k^{-})\to c(q_{1})\,d(q_{2})\to
h_{1}(P_{h_{1}})h_{2}(P_{h_{2}})\,X$.

The helicity frame of a _non-collinear_ hadron with momentum

$\bm{P}_{h}=P_{h}(\sin\theta_{h}\cos\phi_{h},\sin\theta_{h}\sin\phi_{h},\cos\theta_{h})\,,$

as reached from the helicity frame of its parent quark, following the above
procedure $a)$-$c)$ in Eq. (247), is simply given as:

$\displaystyle\hat{\bm{Z}}_{h}$ $\displaystyle=$
$\displaystyle\sin\theta_{h}\cos\phi_{h}\,\hat{\bm{x}}+\sin\theta_{h}\sin\phi_{h}\,\hat{\bm{y}}+\cos\theta_{h}\,\hat{\bm{z}}\equiv\hat{\bm{P}}_{h}$
(248) $\displaystyle\hat{\bm{Y}}_{h}$ $\displaystyle=$
$\displaystyle-\sin\phi_{h}\,\hat{\bm{x}}+\cos\phi_{h}\,\hat{\bm{y}}$ (249)
$\displaystyle\hat{\bm{X}}_{h}$ $\displaystyle=$
$\displaystyle\cos\theta_{h}\cos\phi_{h}\,\hat{\bm{x}}+\cos\theta_{h}\sin\phi_{h}\,\hat{\bm{y}}-\sin\theta_{h}\,\hat{\bm{z}}\,.$
(250)

###  E.1 The thrust frame

In this case the particles $c$ and $d$ move along the $\hat{\bm{z}}_{L}$ and
$-\hat{\bm{z}}_{L}$ direction respectively. This results in the helicity
frames with axes along the following directions in the laboratory frame:

$\hat{\bm{x}}_{c}=\hat{\bm{x}}_{L}\quad\quad\quad\hat{\bm{y}}_{c}=\hat{\bm{y}}_{L}\quad\quad\quad\hat{\bm{z}}_{c}=\hat{\bm{z}}_{L}$
(251)

for a quark/antiquark $c$ moving along +$\hat{\bm{z}}_{L}$, and

$\hat{\bm{x}}_{d}=\hat{\bm{x}}_{L}\quad\quad\quad\hat{\bm{y}}_{d}=-\hat{\bm{y}}_{L}\quad\quad\quad\hat{\bm{z}}_{d}=-\hat{\bm{z}}_{L}$
(252)

for a quark/antiquark $d$ moving along $-\hat{\bm{z}}_{L}$. From these
relations we find that the helicity axes of the hadrons $h_{1}$ in the LAB
frame as reached from the quark helicity frames are [using Eqs. (248)-(250)
with $h=h_{1}$ and Eq. (251)]

$\displaystyle\hat{\bm{Z}}_{h_{1}}$ $\displaystyle=$
$\displaystyle\sin\theta_{h_{1}}\cos\phi_{h_{1}}\,\hat{\bm{x}}_{L}+\sin\theta_{h_{1}}\sin\phi_{h_{1}}\,\hat{\bm{y}}_{L}+\cos\theta_{h_{1}}\,\hat{\bm{z}}_{L}$
(253) $\displaystyle\hat{\bm{Y}}_{h_{1}}$ $\displaystyle=$
$\displaystyle-\sin\phi_{h_{1}}\,\hat{\bm{x}}_{L}+\cos\phi_{h_{1}}\,\hat{\bm{y}}_{L}$
(254) $\displaystyle\hat{\bm{X}}_{h_{2}}$ $\displaystyle=$
$\displaystyle\cos\theta_{h_{1}}\cos\phi_{h_{1}}\,\hat{\bm{x}}_{L}+\cos\theta_{h_{1}}\sin\phi_{h_{1}}\,\hat{\bm{y}}_{L}-\sin\theta_{h_{1}}\,\hat{\bm{z}}_{L}$
(255)

and the ones for the hadron $h_{2}$ [using Eqs. (248)-(250) with $h=h_{2}$ and
Eq. (252)]

$\displaystyle\hat{\bm{Z}}_{h_{2}}$ $\displaystyle=$
$\displaystyle\sin\theta_{h_{2}}\cos\phi_{h_{2}}\,\hat{\bm{x}}_{L}-\sin\theta_{h_{2}}\sin\phi_{h_{2}}\,\hat{\bm{y}}_{L}-\cos\theta_{h_{2}}\,\hat{\bm{z}}_{L}$
(256) $\displaystyle\hat{\bm{Y}}_{h_{2}}$ $\displaystyle=$
$\displaystyle-\sin\phi_{h_{2}}\,\hat{\bm{x}}_{L}-\cos\phi_{h_{2}}\,\hat{\bm{y}}_{L}$
(257) $\displaystyle\hat{\bm{X}}_{h_{2}}$ $\displaystyle=$
$\displaystyle\cos\theta_{h_{2}}\cos\phi_{h_{2}}\,\hat{\bm{x}}_{L}-\cos\theta_{h_{2}}\sin\phi_{h_{2}}\,\hat{\bm{y}}_{L}+\sin\theta_{h_{2}}\,\hat{\bm{z}}_{L}\,.$
(258)

On the other hand, the hadron unit three-momenta as defined in the LAB frame,
see Eqs. (15) and (16), are

$\displaystyle\hat{\bm{P}}_{h_{1}}$ $\displaystyle=$
$\displaystyle\hat{\bm{Z}}_{h_{1}}=\frac{2\eta_{\bot
1}}{z_{p_{1}}}\cos\varphi_{1}\,\hat{\bm{x}}_{L}+\frac{2\eta_{\bot
1}}{z_{p_{1}}}\sin\varphi_{1}\,\hat{\bm{y}}_{L}+\beta_{1}\,\hat{\bm{z}}_{L}$
(259) $\displaystyle\hat{\bm{P}}_{h_{2}}$ $\displaystyle=$
$\displaystyle\hat{\bm{Z}}_{h_{2}}=\frac{2\eta_{\bot
2}}{z_{p_{2}}}\cos\varphi_{2}\,\hat{\bm{x}}_{L}+\frac{2\eta_{\bot
2}}{z_{p_{2}}}\sin\varphi_{2}\,\hat{\bm{y}}_{L}-\beta_{2}\,\hat{\bm{z}}_{L}$
(260)

By a direct comparison between Eqs. (253) and (259) and between Eqs. (256) and
(260) we then get

$\phi_{h_{1}}=\varphi_{1}\hskip 56.9055pt\phi_{h_{2}}=2\pi-\varphi_{2}\,.$
(261)

###  E.2 The hadron frame

In the hadron frame the unit vectors specifying the helicity frames of the two
particles $c$ and $d$, as reached from the laboratory frame are:

$\displaystyle\hat{\bm{z}}_{c}$ $\displaystyle=$
$\displaystyle\frac{2\eta_{\bot
2}}{z_{p_{2}}}\cos\varphi_{2}\,\hat{\bm{x}}_{L}+\frac{2\eta_{\bot
2}}{z_{p_{2}}}\sin\varphi_{2}\,\hat{\bm{y}}_{L}+\beta_{2}\,\hat{\bm{z}}_{L}$
(262) $\displaystyle\hat{\bm{y}}_{c}$ $\displaystyle=$
$\displaystyle-\sin\varphi_{2}\,\hat{\bm{x}}_{L}+\cos\varphi_{2}\,\hat{\bm{y}}_{L}$
(263) $\displaystyle\hat{\bm{x}}_{c}$ $\displaystyle=$
$\displaystyle\beta_{2}\cos\varphi_{2}\,\hat{\bm{x}}_{L}+\beta_{2}\sin\varphi_{2}\,\hat{\bm{y}}_{L}-\frac{2\eta_{\bot
2}}{z_{p_{2}}}\,\hat{\bm{z}}_{L}$ (264) $\displaystyle\hat{\bm{z}}_{d}$
$\displaystyle=$ $\displaystyle-\frac{2\eta_{\bot
2}}{z_{p_{2}}}\cos\varphi_{2}\,\hat{\bm{x}}_{L}-\frac{2\eta_{\bot
2}}{z_{p_{2}}}\sin\varphi_{2}\,\hat{\bm{y}}_{L}-\beta_{2}\,\hat{\bm{z}}_{L}$
(265) $\displaystyle\hat{\bm{y}}_{d}$ $\displaystyle=$
$\displaystyle\sin\varphi_{2}\,\hat{\bm{x}}_{L}-\cos\varphi_{2}\,\hat{\bm{y}}_{L}$
(266) $\displaystyle\hat{\bm{x}}_{d}$ $\displaystyle=$
$\displaystyle\beta_{2}\cos\varphi_{2}\,\hat{\bm{x}}_{L}+\beta_{2}\sin\varphi_{2}\,\hat{\bm{y}}_{L}-\frac{2\eta_{\bot
2}}{z_{p_{2}}}\,\hat{\bm{z}}_{L}\,$ (267)

where we started from the quark-antiquark directions in the LAB frame as
defined in Eqs. (LABEL:q2) and (19) and, then applied the standard procedure
to identify the other two axes (see Eq. (247)).

From the above relations one can get the expressions of the hadron helicity
axes as reached from the parton helicity ones in the LAB frame. In particular,
for the $Z$ axes we find

$\displaystyle\hat{\bm{Z}}_{h_{1}}$ $\displaystyle=$
$\displaystyle\Big{(}\sin\theta_{h_{1}}(\beta_{2}\cos\phi_{h_{1}}\cos\varphi_{2}-\sin\phi_{h_{1}}\sin\varphi_{2})+\frac{2\eta_{\bot
2}}{z_{p_{2}}}\cos\theta_{h_{1}}\cos\varphi_{2}\Big{)}\,\hat{\bm{x}}_{L}$
(268) $\displaystyle+$
$\displaystyle\Big{(}\sin\theta_{h_{1}}(\beta_{2}\cos\phi_{h_{1}}\sin\varphi_{2}+\sin\phi_{h_{1}}\cos\varphi_{2})+\frac{2\eta_{\bot
2}}{z_{p_{2}}}\cos\theta_{h_{1}}\sin\varphi_{2}\Big{)}\,\hat{\bm{y}}_{L}$
$\displaystyle-$ $\displaystyle\Big{(}\frac{2\eta_{\bot
2}}{z_{p_{2}}}\sin\theta_{h_{1}}\cos\phi_{h_{1}}-\beta_{2}\cos\theta_{h_{1}}\Big{)}\,\hat{\bm{z}}_{L}$
$\displaystyle\hat{\bm{Z}}_{h_{2}}$ $\displaystyle=$
$\displaystyle\Big{(}\sin\theta_{h_{2}}(\beta_{2}\cos\phi_{h_{2}}\cos\varphi_{2}+\sin\phi_{h_{2}}\sin\varphi_{2})-\frac{2\eta_{\bot
2}}{z_{p_{2}}}\cos\theta_{h_{2}}\cos\varphi_{2}\Big{)}\,\hat{\bm{x}}_{L}$
(269) $\displaystyle+$
$\displaystyle\Big{(}\sin\theta_{h_{2}}(\beta_{2}\cos\phi_{h_{2}}\sin\varphi_{2}-\sin\phi_{h_{2}}\cos\varphi_{2})-\frac{2\eta_{\bot
2}}{z_{p_{2}}}\cos\theta_{h_{2}}\sin\varphi_{2}\Big{)}\,\hat{\bm{y}}_{L}$
$\displaystyle-$ $\displaystyle\Big{(}\frac{2\eta_{\bot
2}}{z_{p_{2}}}\sin\theta_{h_{2}}\cos\phi_{h_{2}}+\beta_{2}\cos\theta_{h_{2}}\Big{)}\,\hat{\bm{z}}_{L}\,,$

that have to be compared with the corresponding expressions in the LAB frame,
coming from Eqs. (LABEL:Ph1had) and (LABEL:Ph2had),

$\displaystyle\hat{\bm{Z}}_{h_{1}}$ $\displaystyle=$
$\displaystyle\frac{2\eta_{1T}}{z_{p_{1}}}\cos\phi_{1}\,\hat{\bm{x}}_{L}+\frac{2\eta_{1T}}{z_{p_{1}}}\sin\phi_{1}\,\hat{\bm{y}}_{L}+\sqrt{1-\frac{4\eta^{2}_{1T}}{z^{2}_{p_{1}}}}\,\hat{\bm{z}}_{L}$
(270) $\displaystyle\hat{\bm{Z}}_{h_{2}}$ $\displaystyle=$
$\displaystyle-\hat{\bm{z}}_{L}\,.$ (271)

By using

$\displaystyle\sin\theta_{h_{1,2}}$ $\displaystyle=$
$\displaystyle\frac{2\eta_{\perp 1,2}}{z_{p_{1,2}}}\,,$ (272)

after some algebra we get

$\displaystyle\cos\phi_{h_{1}}$ $\displaystyle=$
$\displaystyle\frac{\eta_{1T}}{\eta_{\bot
1}}\beta_{2}\cos(\phi_{1}-\varphi_{2})-\frac{\eta_{\bot 2}}{\eta_{\bot
1}}\frac{z_{p_{1}}}{z_{p_{2}}}\sqrt{1-\frac{4\eta^{2}_{1T}}{z^{2}_{p_{1}}}}\simeq\frac{P_{1T}}{p_{\bot
1}}\cos(\phi_{1}-\varphi_{2})-\frac{p_{\bot 2}}{p_{\bot
1}}\frac{z_{p_{1}}}{z_{p_{2}}}$ $\displaystyle\sin\phi_{h_{1}}$
$\displaystyle=$ $\displaystyle\frac{\eta_{1T}}{\eta_{\bot
1}}\sin(\phi_{1}-\varphi_{2})=\frac{P_{1T}}{p_{\bot
1}}\sin(\phi_{1}-\varphi_{2})\,,$ (274)

together with $\phi_{h_{2}}=0$.

For their role we give here the explicit expressions of the helicity axes of
the hadrons $h_{1}$ and $h_{2}$ in terms of the variables in the LAB frame

$\displaystyle\hat{\bm{Z}}_{h_{1}}$ $\displaystyle=$
$\displaystyle\frac{2\eta_{1T}}{z_{p_{1}}}\cos\phi_{1}\,\hat{\bm{x}}_{L}+\frac{2\eta_{1T}}{z_{p_{1}}}\sin\phi_{1}\,\hat{\bm{y}}_{L}+\sqrt{1-\frac{4\eta^{2}_{1T}}{z^{2}_{p_{1}}}}\,\hat{\bm{z}}_{L}$
(275) $\displaystyle\simeq$
$\displaystyle\frac{2\eta_{1T}}{z_{p_{1}}}\cos\phi_{1}\,\hat{\bm{x}}_{L}+\frac{2\eta_{1T}}{z_{p_{1}}}\sin\phi_{1}\,\hat{\bm{y}}_{L}+\hat{\bm{z}}_{L}\simeq\hat{\bm{z}}_{L}$
$\displaystyle\hat{\bm{Y}}_{h_{1}}$ $\displaystyle\simeq$
$\displaystyle\Big{(}\frac{z_{p_{1}}}{z_{p_{2}}}\frac{p_{\perp 2}}{p_{\perp
1}}\sin{\varphi_{2}}-\frac{P_{1T}}{p_{\perp
1}}\sin{\phi_{1}}\Big{)}\,\hat{\bm{x}}_{L}+\Big{(}\frac{P_{1T}}{p_{\perp
1}}\cos{\phi_{1}}-\frac{z_{p_{1}}}{z_{p_{2}}}\frac{p_{\perp 2}}{p_{\perp
1}}\cos{\varphi_{2}}\Big{)}\,\hat{\bm{y}}_{L}$ (276)
$\displaystyle\hat{\bm{X}}_{h_{1}}$ $\displaystyle\simeq$
$\displaystyle\Big{(}\frac{P_{1T}}{p_{\perp
1}}\cos{\phi_{1}}-\frac{z_{p_{1}}}{z_{p_{2}}}\frac{p_{\perp 2}}{p_{\perp
1}}\cos{\varphi_{2}}\Big{)}\,\hat{\bm{x}}_{L}+\Big{(}\frac{P_{1T}}{p_{\perp
1}}\sin{\phi_{1}}-\frac{z_{p_{1}}}{z_{p_{2}}}\frac{p_{\perp 2}}{p_{\perp
1}}\sin{\varphi_{2}}\Big{)}\,\hat{\bm{y}}_{L}$ (277)
$\displaystyle\hat{\bm{Z}}_{h_{2}}$ $\displaystyle=$
$\displaystyle-\hat{\bm{z}}_{L}$ (278) $\displaystyle\hat{\bm{Y}}_{h_{2}}$
$\displaystyle=$
$\displaystyle\sin\varphi_{2}\,\hat{\bm{x}}_{L}-\cos\varphi_{2}\,\hat{\bm{y}}_{L}$
(279) $\displaystyle\hat{\bm{X}}_{h_{2}}$ $\displaystyle=$
$\displaystyle\cos\varphi_{2}\,\hat{\bm{x}}_{L}+\sin\varphi_{2}\,\hat{\bm{y}}_{L}\,.$
(280)

Notice that, while the $\hat{\bm{Z}}_{h_{1}}$ axis does not involve any
intrinsic transverse momentum dependence, special care, when carrying out the
integration over the unobserved variables, is due for the transverse axes.
Moreover, at the lowest order in $\eta_{1T}$ the $\hat{\bm{Z}}_{h_{1}}$ axis
almost coincides with the $\hat{\bm{z}}_{L}$ one.

Before concluding this section we warn the reader that the hadron helicity
frames obtained directly from the LAB frame, without passing through the
parton helicity frame, give a different result where, while the $\hat{\bm{Z}}$
axes coincide, the $\hat{\bm{X}}$ and $\hat{\bm{Y}}$ axes are rotated:

$\displaystyle\hat{\bm{Z}}^{\rm{LAB}}_{h_{1}}$ $\displaystyle=$
$\displaystyle\frac{2\eta_{1T}}{z_{p_{1}}}\cos\phi_{1}\,\hat{\bm{x}}_{L}+\frac{2\eta_{1T}}{z_{p_{1}}}\sin\phi_{1}\,\hat{\bm{y}}_{L}+\sqrt{1-\frac{4\eta_{1T}^{2}}{z^{2}_{p_{1}}}}\,\hat{\bm{z}}_{L}$
(281) $\displaystyle\hat{\bm{Y}}^{\rm{LAB}}_{h_{1}}$ $\displaystyle=$
$\displaystyle-\sin\phi_{1}\,\hat{\bm{x}}_{L}+\cos\phi_{1}\,\hat{\bm{y}}_{L}=\hat{\bm{n}}$
(282) $\displaystyle\hat{\bm{X}}^{\rm{LAB}}_{h_{1}}$ $\displaystyle=$
$\displaystyle\sqrt{1-\frac{4\eta_{1T}^{2}}{z^{2}_{p_{1}}}}\cos\phi_{1}\,\hat{\bm{x}}_{L}+\sqrt{1-\frac{4\eta_{1T}^{2}}{z^{2}_{p_{1}}}}\cos\phi_{1}\,\hat{\bm{y}}_{L}-\frac{2\eta_{1T}}{z_{p_{1}}}\,\hat{\bm{z}}_{L}$
(283) $\displaystyle\hat{\bm{Z}}^{\rm{LAB}}_{h_{2}}$ $\displaystyle=$
$\displaystyle-\hat{\bm{z}}_{L}$ (284)
$\displaystyle\hat{\bm{Y}}^{\rm{LAB}}_{h_{2}}$ $\displaystyle=$
$\displaystyle-\hat{\bm{y}}_{L}$ (285)
$\displaystyle\hat{\bm{X}}^{\rm{LAB}}_{h_{2}}$ $\displaystyle=$
$\displaystyle\hat{\bm{x}}_{L}\,.$ (286)

## References

  * (1) U. D’Alesio and F. Murgia, _Azimuthal and Single Spin Asymmetries in Hard Scattering Processes_ , _Prog. Part. Nucl. Phys._ 61 (2008) 394 [0712.4328].
  * (2) V. Barone, F. Bradamante and A. Martin, _Transverse-spin and transverse-momentum effects in high-energy processes_ , _Prog. Part. Nucl. Phys._ 65 (2010) 267 [1011.0909].
  * (3) C.A. Aidala, S.D. Bass, D. Hasch and G.K. Mallot, _The Spin Structure of the Nucleon_ , _Rev. Mod. Phys._ 85 (2013) 655 [1209.2803].
  * (4) M. Anselmino, A. Mukherjee and A. Vossen, _Transverse spin effects in hard semi-inclusive collisions_ , _Prog. Part. Nucl. Phys._ 114 (2020) 103806 [2001.05415].
  * (5) D.W. Sivers, _Single Spin Production Asymmetries from the Hard Scattering of Point-Like Constituents_ , _Phys. Rev. D_ 41 (1990) 83.
  * (6) D.W. Sivers, _Hard scattering scaling laws for single spin production asymmetries_ , _Phys. Rev. D_ 43 (1991) 261.
  * (7) J.C. Collins, _Fragmentation of transversely polarized quarks probed in transverse momentum distributions_ , _Nucl. Phys. B_ 396 (1993) 161 [hep-ph/9208213].
  * (8) X.-d. Ji, J.-P. Ma and F. Yuan, _QCD factorization for spin-dependent cross sections in DIS and Drell-Yan processes at low transverse momentum_ , _Phys. Lett. B_ 597 (2004) 299 [hep-ph/0405085].
  * (9) X.-d. Ji, J.-p. Ma and F. Yuan, _QCD factorization for semi-inclusive deep-inelastic scattering at low transverse momentum_ , _Phys. Rev. D_ 71 (2005) 034005 [hep-ph/0404183].
  * (10) J. Collins, _Foundations of perturbative QCD_ , vol. 32, Cambridge University Press (11, 2013).
  * (11) M.G. Echevarria, A. Idilbi and I. Scimemi, _Factorization Theorem For Drell-Yan At Low $q_{T}$ And Transverse Momentum Distributions On-The-Light-Cone_, _JHEP_ 07 (2012) 002 [1111.4996].
  * (12) P.J. Mulders and R.D. Tangerman, _The complete tree-level result up to order $1/{Q}$ for polarized deep-inelastic leptoproduction_, _Nucl. Phys._ B461 (1996) 197.
  * (13) A. Bacchetta, M. Diehl, K. Goeke, A. Metz, P.J. Mulders and M. Schlegel, _Semi-inclusive deep inelastic scattering at small transverse momentum_ , _JHEP_ 02 (2007) 093 [hep-ph/0611265].
  * (14) M. Anselmino, M. Boglione, U. D’Alesio, S. Melis, F. Murgia, E.R. Nocera and A. Prokudin, _General Helicity Formalism for Polarized Semi-Inclusive Deep Inelastic Scattering_ , _Phys. Rev._ D83 (2011) 114019 [1101.1011].
  * (15) S. Arnold, A. Metz and M. Schlegel, _Dilepton production from polarized hadron hadron collisions_ , _Phys. Rev. D_ 79 (2009) 034005 [0809.2262].
  * (16) D. Boer, R. Jakob and P.J. Mulders, _Asymmetries in polarized hadron production in $e^{+}e^{-}$ annihilation up to order $1/{Q}$_, _Nucl. Phys._ B504 (1997) 345.
  * (17) M. Anselmino, M. Boglione, U. D’Alesio, E. Leader, S. Melis and F. Murgia, _The general partonic structure for hadronic spin asymmetries_ , _Phys. Rev. D_ 73 (2006) 014020 [hep-ph/0509035].
  * (18) M. Anselmino, M. Boglione, U. D’Alesio, F. Murgia and A. Prokudin, _Role of transverse momentum dependence of unpolarized parton distribution and fragmentation functions in the analysis of azimuthal spin asymmetries_ , _Phys. Rev. D_ 98 (2018) 094023 [1809.09500].
  * (19) L. Gamberg, Z.-B. Kang, D. Pitonyak, M. Schlegel and S. Yoshida, _Polarized hyperon production in single-inclusive electron-positron annihilation at next-to-leading order_ , _JHEP_ 01 (2019) 111 [1810.08645].
  * (20) Z.-B. Kang, D.Y. Shao and F. Zhao, _QCD resummation on single hadron transverse momentum distribution with the thrust axis_ , _JHEP_ 12 (2020) 127 [2007.14425].
  * (21) M. Boglione and A. Simonelli, _Factorization of $e^{+}e^{-}\to H\,X$ cross section, differential in $z_{h}$, $P_{T}$ and thrust, in the $2$-jet limit_, _JHEP_ 02 (2021) 076 [2011.07366].
  * (22) M. Boglione and A. Simonelli, _Universality-breaking effects in $e^{+}e^{-}$ hadronic production processes_, _Eur. Phys. J. C_ 81 (2021) 96 [2007.13674].
  * (23) Belle collaboration, _Observation of Transverse $\Lambda/\bar{\Lambda}$ Hyperon Polarization in $e^{+}e^{-}$ Annihilation at Belle_, _Phys. Rev. Lett._ 122 (2019) 042001 [1808.05000].
  * (24) U. D’Alesio, F. Murgia and M. Zaccheddu, _First extraction of the $\Lambda$ polarizing fragmentation function from Belle $e^{+}e^{-}$ data_, _Phys. Rev. D_ 102 (2020) 054001 [2003.01128].
  * (25) D. Callos, Z.-B. Kang and J. Terry, _Extracting the transverse momentum dependent polarizing fragmentation functions_ , _Phys. Rev. D_ 102 (2020) 096007 [2003.04828].
  * (26) M. Anselmino, M. Boglione, U. D’Alesio, A. Kotzinian, F. Murgia, A. Prokudin and C. Türk, _Transversity and Collins functions from SIDIS and $e^{+}e^{-}$ data_, _Phys. Rev._ D75 (2007) 054032 [hep-ph/0701006].
  * (27) A. Bacchetta, U. D’Alesio, M. Diehl and C.A. Miller, _Single-spin asymmetries: The Trento conventions_ , _Phys. Rev._ D70 (2004) 117504 [hep-ph/0410050].
  * (28) D. Boer and P. Mulders, _Time reversal odd distribution functions in leptoproduction_ , _Phys. Rev. D_ 57 (1998) 5780 [hep-ph/9711485].
  * (29) M. Anselmino, D. Boer, U. D’Alesio and F. Murgia, _$\Lambda$ polarization from unpolarized quark fragmentation_, _Phys. Rev. D_ 63 (2001) 054029 [hep-ph/0008186].
  * (30) M. Anselmino, D. Boer, U. D’Alesio and F. Murgia, _Transverse $\Lambda$ polarization in semiinclusive DIS_, _Phys. Rev. D_ 65 (2002) 114014 [hep-ph/0109186].
  * (31) E. Leader, _Spin in particle physics_ , _Camb. Monogr. Part. Phys. Nucl. Phys. Cosmol._ 15 (2001) 1.