11institutetext: 1 Key Laboratory of Perception (MoE), School of AI, Peking
University
2 Microsoft Research Asia 3 Baidu

# Conditional DETR V2: Efficient Detection Transformer with Box Queries

Xiaokang Chen 11 Fangyun Wei 22 Gang Zeng 11 Jingdong Wang 33

###### Abstract

In this paper, we are interested in Detection Transformer (DETR), an end-to-
end object detection approach based on a transformer encoder-decoder
architecture without hand-crafted postprocessing, such as NMS. Inspired by
Conditional DETR, an improved DETR with fast training convergence, that
presented box queries (originally called spatial queries) for internal decoder
layers, we reformulate the object query into the format of the box query that
is a composition of the embeddings of the reference point and the
transformation of the box with respect to the reference point. This
reformulation indicates the connection between the object query in DETR and
the anchor box that is widely studied in Faster R-CNN. Furthermore, we learn
the box queries from the image content, further improving the detection
quality of Conditional DETR still with fast training convergence. In addition,
we adopt the idea of axial self-attention to save the memory cost and
accelerate the encoder. The resulting detector, called Conditional DETR V2,
achieves better results than Conditional DETR, saves the memory cost and runs
more efficiently. For example, for the DC$5$-ResNet-$50$ backbone, our
approach achieves $44.8$ AP with $16.4$ FPS on the COCO $val$ set and compared
to Conditional DETR, it runs $1.6\times$ faster, saves $74$% of the overall
memory cost, and improves $1.0$ AP score.

##  1 Introduction

Object detection is a problem of localizing the objects and predicting their
categories. It is a fundamental and challenging problem in computer vision [5,
35, 36, 8, 7, 6] and has many practical applications, such as robot navigation
and autonomous driving. Deep learning has been the dominant solution to object
detection. The early framework regresses the object box by starting from a
default box (or anchor box) and optionally iteratively refines the box
regression, e.g., Faster R-CNN [30]. Recently, detection transformer (DETR)
exploits the transformer decoder for object detection, with introducing the
object queries (learnable positional embeddings). It is an end-to-end solution
without the post-processing, such as non-maximum suppression (NMS).

DETR typically needs $500$ epochs to achieve satisfactory performance. Several
follow-up works, [34, 13, 42, 25, 11, 48] develop DETR variants to accelerate
the training process. Conditional DETR [25] and Anchor DETR [42] identify that
the decoder cross-attention has limited capability to exploit the spatial
attention and designs the new query format to boost the capability.

Figure 1: The overall architecture. (1) We use a conventional CNN backbone
($e.g.$, ResNet-$50$) to extract the feature of an input image. (2) The
learned embedding is fed into the transformer encoder layers to model the
global dependencies between the inputs. (3) Then, we predict candidate boxes
from the encoder embedding and select reference points according to the
classification score of these boxes. The input of the decoder is the
concatenation of the box query and the content query. The box query consists
of the embedding of the selected reference points and the transformation
predicted from the corresponding encoder embedding of these reference points.
The content query is initialized from the selected candidate boxes. (4) We
pass each output embedding of the decoder to a shared feed-forward network
(FFN) that predicts either a detection (class and bounding box) or a ”no
object” class.

Inspired by the spatial queries (we call box queries) that Conditional DETR
introduces for internal decoder layers, we propose to study the object query
(positional encodings), the input of the first decoder layer, and reformulate
it as the form of the box query in the embedding space. The box query is a
composition of the embeddings of the reference point and the transformation of
the box with respect to the reference point. The box query form in some sense
builds the connection between DETR and Faster R-CNN that both use the boxes as
one input. There are several differences. The box query is in the embedding
space and the anchor box is in a form of $2$D coordinates. The box query is
explored through the attention mechanism to find the extremities for box
regression and the regions inside the object for classification, while the
anchor box is used as the initial guess.

We predict the box queries from the image content for each image, instead of
learning them as model parameters.

Specifically, we predict the probability of belonging to an object for the
points in the encoder embedding, and select the top-scored points as the
reference points. The transformation that contains the scale information of
the object is predicted from the corresponding encoder embedding. The image-
dependent box query helps locate the object and improve the performance.

In addition, we adopt the idea of axial self-attention [16] and propose the
Horizontal-Vertical Attention. For each query embedding, we perform attention
in the horizontal and vertical directions in parallel. Experiments show that
the Horizontal-Vertical Attention could save the memory cost and accelerate
the encoder, especially for high-resolution backbones, such as
DC$5$-ResNet-$50$ and DC$5$-ResNet-$101$.

##  2 Related Work

Anchor-based object detectors. Anchor plays an important role in object
detection. Faster R-CNN [30] first introduces the anchor boxes to make dense
predictions. A series of works such as SSD [22], RetinaNet [20], YOLO [28, 29,
2] and Cascade R-CNN [3] are proposed to make object detection more efficient
and accurate. To avoid hand-crafted anchor design, some recent methods, e.g.
FCOS [37], CenterNet [46], CornerNet [19] and ExtremeNet [47], use anchor
points to represent objects, these methods are also known as anchor-free
detectors.

In this paper, we reformulate the object query in DETR into the format of the
box query that consists of the embeddings of the reference point and the
transformation of the box with respect to the reference point. This
reformulation indicates the connection between the object query in DETR and
the anchor box in Faster R-CNN.

DETR and its variants. DETR successfully applies transformers to object
detection and removes the need for non-maximum suppression. The object queries
(or called position embeddings) in DETR are responsible for the category
prediction and box regression of each object. Some works [48, 41] aim to solve
the high computation complexity issue caused by the global encoder self-
attention in DETR. Deformable DETR [48] designs the sparse attention that each
query only attends to keys in a local region. PNP-DETR [41] proposes the poll-
and-pool strategy to reduce the spatial redundancy in the input, resulting in
a smaller input size for the encoder.

DETR also suffers from another issue, the slow training convergence, which
attracts a lot of attention. TSP [34] eliminates the cross-attention modules
and combines the FCOS and R-CNN-like detection heads. Deformable DETR [48]
proposes deformable attention and uses multiple level features to accelerate
the converge. SMCA[13] modulates the DETR multi-head global cross-attentions
with gaussian maps around a few (shifted) centers to focus more on the
restricted local regions.

Anchor DETR [42] argues that object query in DETR could not focus on a
specific region and replace object queries with anchor points. Conditional
DETR [25] proposes to search the extremity and distinct regions of the object
through learning the conditional spatial queries from the decoder embeddings.
However, the spatial query of the first layer could not find such regions well
due to the lack of the image content in the query. To solve this issue, we
propose the box query and initialize it from the image content, which could
find extremity regions since the first layer.

Efficient attention mechanism. The global self-attention [39] could model the
global dependencies between the inputs, while the computation complexity is
quadratic to the input size. Many efficient attention mechanisms have been
designed in the NLP field to improve the memory and computation efficiency of
self-attention, such as Sparse Transformer [9], Linformer [40], Performer
[10], RFA [26], Nyströmformer [43], Reformer [18], Routing Transformer [33],
BigBird [45], Longformer [1], Transformer-XL [12]. In the vision field, some
methods [27, 38] restrict the attention region of the key for each query
instead of the whole region. For example, Swin Transformer [23] and HRFormer
[44] adopt the local self-attention mechanism and add interaction across
different local windows. Axial self-attention [16] applies the local window
along the horizontal or vertical axis sequentially to achieve global
attention. Following axial self-attention, we propose the Horizontal-Vertical
Attention, where each query pixel attends to pixels that belong to the same
row or column as it in parallel.

##  3 Approach

###  3.1 Architecture

We follow detection transformer (DETR) to use the transformer encoder and
decoder structure for object detection. The architecture consists of a CNN
backbone, a transformer encoder, a transformer decoder, and predictors for
classification and box regression. The transformer encoder aims to improve the
content embeddings output from the CNN backbone. It is a stack of multiple
encoder layers, where each layer mainly consists of a self-attention layer and
a feed-forward layer.

The transformer decoder is a stack of decoder layers. Each decoder layer is
composed of three main layers: (1) a self-attention layer with the embeddings,
outputted from the previous decoder layer as the input, (2) a cross-attention
layer, with the output of the self-attention layer as the queries and the
output of the encoder forming the keys and the values, and (3) a feed-forward
layer predicting the box and the category score.

A candidate box is predicted from each decoder embedding as follows,

$\displaystyle\hat{\mathbf{b}}=\operatorname{sigmoid}(\operatorname{FFN}(\mathbf{f})+[{\mathbf{s}}^{\top}~{}0~{}0]^{\top}).$
(1)

Here, $\mathbf{f}$ is the decoder embedding. $\hat{\mathbf{b}}$ is a four-
dimensional vector $[b_{cx}~{}b_{cy}~{}b_{w}~{}b_{h}]^{\top}$, consisting of
the box center, the box width and the box height. ${\mathbf{s}}$ is the
unnormalized $2$D coordinate of the reference point, and is $(0,0)$ in the
original DETR. The classification score for each candidate box is also
predicted from the decoder embedding through another FNN,
$\mathbf{e}=\operatorname{FFN}(\mathbf{f})$.

###  3.2 Box Query

Object query. The original DETR learns  _object queries_ ,

$\displaystyle\\{\mathbf{o}_{1},\dots,\mathbf{o}_{N}\\},$ (2)

where each query is a high-dimensional embedding vector. These object queries
are model parameters and are the same for all the images. In DETR [4], they
are also called positional encodings, and initially used to encode the
information of the positions at which the objects might appear. The object
queries are combined into the decoder embeddings, forming the queries of the
self-attention and cross-attention layers in the decoder.

Box query. Conditional DETR learns spatial queries, which we call box queries
in this paper, from the decoder output embeddings and use them to replace the
object queries for the decoders except for the first decoder layer. The box
queries are combined with the content queries through concatenation instead of
addition in the original DETR. Accordingly, the keys are formed by
concatenating the positional embeddings and the encoder output embeddings.

The box queries $\mathbf{p}_{q}$ are predicted by transforming the 2D
reference point in the embedding space with the transformation of the box with
respect to the reference point:

$\displaystyle\mathbf{p}_{q}=\boldsymbol{\uplambda}_{q}\odot{\mathbf{p}}_{s}.$
(3)

Here, $\mathbf{p}_{s}$ is the projected positional embedding from the
reference point and $\boldsymbol{\uplambda}_{q}$ is a transformation in the
embedding space. $\odot$ is the element-wise multiplication operator.

Learning initial box queries as model parameters. We use the box queries to
replace the object queries that Conditional DETR still uses in the first
decoder layer. We learn $2$D references points,
$\\{(c_{x1},c_{y1}),\dots,(c_{xK},c_{yK})$}, and use the sinusoidal positional
embedding to form the corresponding embeddings
$\\{\mathbf{p}_{s1},\dots,\mathbf{p}_{sK}\\}$. $\boldsymbol{\uplambda}_{q}$ is
set as the model parameter that is randomly initialized.

Predicting initial box queries from image content. The pipeline is shown in
Figure 2. We propose to select several positions as reference points,
$\\{(c_{x1},c_{y1}),\\\ \dots,(c_{xK},c_{yK})$} from the image content. We
then build the reference embeddings from these points and the transformation
for each reference point according to the corresponding encoder embedding
$\mathbf{\hat{X}}$ as the following,

$\displaystyle\boldsymbol{\uplambda}_{q}=\operatorname{FFN}(\mathbf{\hat{x}}{(c_{x},c_{y})}),$
(4)

where $\mathbf{\hat{x}}{(c_{x},c_{y})}$ is the encoder embedding at the
position $(c_{x},c_{y})$. $(c_{x},c_{y})$ is the normalized position which
ranges from $0$ to $1$. We select the reference positions by classifying each
position. The classification score is also predicted from the encoder
embedding through an FFN,

$\displaystyle\mathbf{s}=\operatorname{FFN}(\mathbf{\hat{x}}{(c_{x},c_{y})}).$
(5)

We perform binary classification here and $\mathbf{s}$ is a $2$-d vector,
indicating the possibility of being an object ($1$) or non-object ($0$). We
rank all the positions according to their probability of being an object and
the top-scored predictions are selected.

The purpose of using image content is different from the two-stage strategy in
Deformable DETR. They generate the initial positions and box sizes from the
image content, then sample the key set around those positions. The final box
size is predicted based on the initial box size, which is a form of residual
prediction and easier to optimize. By contrast, we project the position and
box information to high-dimensional space and use them to form the box query.
The box query could scale the range that we search for the object according to
its size.

Figure 2: Pipeline of box query initialization. The box query is a composition
of the embeddings of the box center (reference point) and the box size. The
attention map is a toy example, not the actual output from the model.

Initial content queries from image. In the internal decoder layers, the
content query, output from the previous decoder layer, is used to predict the
box, indicating that the content query also contains the box information. In
light of this, we initialize the content query for the first decoder layer by
incorporating the box information.

We consider two ways. One is to initialize the content query from the encoder
embeddings:

$\displaystyle\mathbf{c}_{q}=\operatorname{FFN}(\mathbf{\hat{x}}{(c_{x},c_{y})}).$
(6)

The other one uses the box estimated from the position:

$\displaystyle\mathbf{c}_{q}=\operatorname{FFN}(\operatorname{PE}(\mathbf{b}(c_{x},c_{y})).$
(7)

Here, $\mathbf{b}(c_{x},c_{y})$ is the box estimated from the position
$(c_{x},c_{y})$: $\operatorname{sigmoid}(\operatorname{FFN}(\\\
\hat{\mathbf{x}}(c_{x},c_{y}))+\sigma^{-1}([c_{x}~{}c_{y}~{}c_{w}~{}c_{h}]^{\top}))$,
and $\operatorname{PE}()$ is the positional embedding. $\sigma^{-1}$ means the
inverse sigmoid function. $c_{w}$ and $c_{h}$ are hyper-parameters and we will
study them in the experiment section. Our results empirically indicate that
the two ways perform similarly and the latter way performs slightly better.

###  3.3 Horizontal-Vertical Attention

The encoder attention is formulated as follows,

$\displaystyle\operatorname{Atten}(\mathbf{x}_{uv},\mathbf{X},\mathbf{X})$
$\displaystyle=\sum_{i=1}^{H}\sum_{j=1}^{W}\mathbf{x}_{ij}\alpha(\mathbf{x}_{uv},\mathbf{x}_{ij}).$
(8) $\displaystyle\alpha(\mathbf{x}_{uv},\mathbf{x}_{ij})$
$\displaystyle=\frac{e^{\frac{1}{\sqrt{d}}\mathbf{x}_{uv}^{\top}\mathbf{x}_{ij}}}{\sum_{i=1}^{H}\sum_{j=1}^{W}e^{\frac{1}{\sqrt{d}}\mathbf{x}_{uv}^{\top}\mathbf{x}_{ij}}.}$
(9)

Here $\mathbf{X}$ is the embedding output from the backbone, $e.g.,$
ResNet-$50$. We use one query $\mathbf{x}_{uv}$ at the position $(u,v)$ as an
example. $d$ is the feature dimension. The memory and computation complexities
for all the $HW$ queries are quadratic with respect to the map size $HW$.

Figure 3: The proposed Horizontal-Vertical Attention. Each pixel will
aggregate embeddings of the pixels in the same row or column.
$\mathbf{Q}^{row}$ ($\mathbf{Q}^{col}$) is generated by the addition of the
query embedding and the $1$D row (col) position encoding. $\mathbf{F}^{row}$
($\mathbf{F}^{col}$) is generated by the addition of the pooled row (col)
embedding and the $1$D row (col) position encoding. $\mathbf{V}^{row}$
($\mathbf{V}^{col}$) is from the pixels that lie in the same row (col) as the
query pixel.

We propose to decrease the number of keys for complexity reduction, as shown
in Figure 3. The idea is inspired by CCNet [17] and axial self-attention [16].
We pool the embedding $\mathbf{X}$ along the vertical and horizontal
directions respectively,
$\bar{\mathbf{x}}_{:j}=\frac{1}{H}\sum_{i=1}^{H}\mathbf{x}_{ij}$, and
$\bar{\mathbf{x}}_{i:}=\frac{1}{W}\sum_{j=1}^{W}\mathbf{x}_{ij}$, and form
$(H+W)$ keys:

$\displaystyle\mathbf{K}=[\bar{\mathbf{x}}_{1:}~{}\dots~{}\bar{\mathbf{x}}_{H:}~{}\bar{\mathbf{x}}_{:1}~{}\dots~{}\bar{\mathbf{x}}_{:W}].$
(10)

The HV-attention process is as follows,

$\displaystyle\operatorname{HV-
Atten}(\mathbf{x}_{uv},\mathbf{K},\mathbf{X})=~{}$
$\displaystyle\operatorname{Concat}(\sum_{i=1}^{H}\mathbf{x}_{iv}\alpha(\mathbf{x}_{uv},\bar{\mathbf{x}}_{i:}),$
$\displaystyle\sum_{j=1}^{W}\mathbf{x}_{uj}\alpha(\mathbf{x}_{uv},\bar{\mathbf{x}}_{:j})).$
(11)

Complexity analysis. The computation complexity of HV-attention is reduced
from quadratic to linear with respect to $HW$. The computational complexity of
global self-attention and our Horizontal-Vertical Attention based on the
feature map $\mathbf{F}\in\mathbb{R}^{d\times H\times W}$ are:

$\displaystyle\Omega(\operatorname{Global-Atten})=4HWd^{2}+2(HW)^{2}d,$ (12)
$\displaystyle\Omega(\operatorname{HV-Atten})=7HWd^{2}+2HWd(H+W),$ (13)

where the coefficient of the first item increases from $4$ to $7$ because we
apply two linear projections on the query, key and value. Eq. (12) is
quadratic to $HW$ and is unaffordable when $HW$ is large.

Comparison with CC Attention [17]. The process of CC Attention is as follows,

$\displaystyle\mathbf{K}_{uv}=[\mathbf{x}_{1v},~{}\dots~{}\mathbf{x}_{Hv},$
$\displaystyle\mathbf{x}_{u1},~{}\dots~{}\mathbf{x}_{uW}].$ (14)
$\displaystyle\operatorname{CC-
Atten}(\mathbf{x}_{uv},\mathbf{K}_{uv},\mathbf{X})=~{}$
$\displaystyle\sum_{i=1}^{H}\mathbf{x}_{iv}\alpha(\mathbf{x}_{uv},\mathbf{x}_{iv})+$
$\displaystyle\sum_{j=1}^{W}\mathbf{x}_{uj}\alpha(\mathbf{x}_{uv},\mathbf{x}_{uj}).$
(15)

Here $\mathbf{X}$ is the embedding output from the backbone, $e.g.,$
ResNet-$50$. We use one query $\mathbf{x}_{uv}$ at the position $(u,v)$ as an
example. $\mathbf{K}_{uv}$ is the key set for the query position $(u,v)$.
There are two differences between our method and CC Attention. (1) We use a
shared key set for all the queries, further saving the memory cost of the
attention module. For example, $13$% memory cost is saved for the $256\times
100\times 150$ ($d\times H\times W$) input. (2) We use concatenation instead
of addition when aggregating the horizontal and vertical features, which
brings $0.5$ AP improvement ($44.3\rightarrow 44.8$) on the DC$5$-ResNet-$50$
backbone.

Comparison with RCDA [42]. We also compare our Horizontal-Vertical Attention
with the Row-Column Decoupled Attention (RCDA). RCDA decouples the $2$D key
feature to the row feature and the column feature by $1$D global average
pooling, and performs the row attention and the column attention successively.
Our proposed method is different from them in that: (1) We conduct row
attention and column attention in a parallel way. (2) The value set we use is
the feature of pixels that belong to the same row or column as the query
pixel, while RCDA views all the pixels as the value set, bringing more
computational cost.

Figure 4:  Illustrating the spatial attention weight maps from the first
decoder layer. For each scene, the upper row is from the model with random
initialized box query and the lower row is from the model with image-dependent
box query. The attention weight maps are from $5$ heads out of the $8$ heads
and the green box in each image is the ground-truth box. Best viewed in color.

###  3.4 Visualization

Figure 4 visualizes the spatial attention weight maps of the first decoder
layer. The upper row of each scene adopts the randomly initialized box query,
and the lower row is our image-dependent box query. The maps are soft-max
normalized over the dot products between the box query and the position
embedding of the key. We show $5$ out of the $8$ maps, and the other three are
duplicates. The duplicates might be different for models trained several
times, but the detection performance is almost the same.

In the visualizations of our method, the spatial attention map in the first
four columns highlights an extremity region near a certain edge of the
bounding box, $e.g.$, the head of the cat in the second row. The map in the
last column highlights a small region inside the box, whose representations
might already encode enough information for object classification. We also
find that the image-dependent box query is able to scale the spatial spread
for the extremity it highlights. For a larger object, such as the bus in the
last row, our method highlights a larger spread. For a smaller object, such as
the cat in the second row, our method highlights a smaller spread.

By contrast, the spatial attention map of the randomly initialized box query
does not show this property. Each head only highlights a small region near the
object center. Although a certain head may try to find an extremity region,
such as the fourth column, it could not scale the spatial spread of the
highlight regions according to the object size.

###  3.5 Discussion: Box Query vs Anchor Box

The box query builds the connection between DETR and Faster R-CNN. They both
use the form of boxes as one input and extract features according to the box.
However, there are some differences. (1) The box query is in the embedding
space, which consists of the embedding of the reference point and the
transformation. The transformation could transform the embedding of the
reference point to the extremity regions of the object, which is like a box.
The anchor box is in the form of $2$D coordinates, consisting of box center
and box height/width. (2) The box query is explored through the attention
mechanism to find the extremities of the object for box regression and the
regions inside the object for classification. While the anchor box is used as
the initial guess and all areas in the box are treated as the region of
interest (ROI), which is used for classification and box regression.

###  3.6 Loss Function

The box query and the content query are initialized from the top-scored
candidate boxes. We supervise the box predictions and view it as a bipartite
matching problem between the predicted boxes and the ground-truth boxes.

The matching cost is defined as:

$\displaystyle\mathcal{C}_{\operatorname{match}}=\sum\nolimits_{i=1}^{G}[\mu_{\operatorname{cls}}\ell_{\operatorname{FL}}(\mathbf{p}_{\xi(i)},\bar{c}_{i})+\ell_{\operatorname{box}}(\mathbf{b}_{\xi(i)},\bar{\mathbf{b}}_{i})].$
(16)

Here $G$ is the number of ground-truth objects. $\ell_{\operatorname{FL}}$ is
the focal loss and $\mu_{\operatorname{cls}}=2$ is the trade-off coefficient.
$\bar{c}_{i}$ is the class of the $i$th ground-truth box and
$\bar{c}_{i}=\operatorname{null}$ if $i>G$. We binarize the ground-truth label
and $\bar{c}_{i}=1$ if it is an object. $\bar{\mathbf{b}}_{i}$ is the $i$th
ground-truth box and $\ell_{\operatorname{box}}$ is a combination of
$\ell_{1}$ loss and GIoU loss [32] and the loss weights are $5$ and $2$,
respectively.

$\xi(\cdot)$ is a permutation of N predictions and we could find an optimal
solution $\hat{\xi}(\cdot)$ through minimizing the matching cost. Then the
loss function to supervise the initialization of the box query and the content
query is computed as:

$\displaystyle\mathcal{L}_{init}=\sum_{i=1}^{N}\lambda_{\operatorname{cls}}\ell_{\operatorname{FL}}(\mathbf{p}_{\hat{\xi}(i)},\bar{c}_{i})+\delta_{[i\leqslant
G]}\ell_{\operatorname{box}}(\mathbf{b}_{\hat{\xi}(i)},\bar{\mathbf{b}}_{i}),$
(17)

where $\lambda_{\operatorname{cls}}=2$ is the trade-off coefficient. $\delta$
is the indicator function. We use the same loss function as Conditional DETR
[25] to supervise the output of the decoder, which is noted as
$\mathcal{L}_{dec}$. Then the overall loss function is defined as:

$\displaystyle\mathcal{L}=\mathcal{L}_{init}+\mathcal{L}_{dec}.$ (18)

##  4 Experiments

###  4.1 Setting

Dataset. We perform the experiments on the COCO $2017$ [21] detection dataset,
which contains about $118$K training (train) images and $5$K validation (val)
images.

Table 1: Comparison of Conditional DETR V2 with other detection models on COCO
2017 val. ∗ means these methods use multi-scale feature. + means we use $900$
box queries.

Model | #epochs | GFLOPs | #params | AP |  AP50 |  AP75 |  APS |  APM |  APL
---|---|---|---|---|---|---|---|---|---
DETR-R$50$ [4] | $500$ | $86$ |  $41$M | $42.0$ | $62.4$ | $44.2$ | $20.5$ | $45.8$ | $61.1$
DETR-R$50$ [4] | $50$ | $86$ |  $41$M | $34.9$ | $55.5$ | $36.0$ | $14.4$ | $37.2$ | $54.5$
PNP-DETR-R$50$ [41] | $500$ | $82$ |  $41$M | $41.8$ | $62.1$ | $44.4$ | $21.2$ | $45.3$ | $60.8$
Deformable DETR-R50 [48] | $50$ | $78$ |  $34$M | $39.4$ | $59.6$ | $42.3$ | $20.6$ | $42.9$ | $55.5$
UP-DETR-R$50$ [11] | $150$ | $86$ |  $41$M | $40.5$ | $60.8$ | $42.6$ | $19.0$ | $44.4$ | $60.0$
Anchor DETR-R50 [42] | $50$ | $94$ |  $37$M | $42.1$ | $63.1$ | $44.9$ | $22.3$ | $46.2$ | $60.0$
Conditional DETR-R50 [25] | $50$ | $90$ |  $44$M | $40.9$ | $61.8$ | $43.3$ | $20.8$ | $44.6$ | $59.2$
Conditional DETR V2-R50 | $50$ | $89$ |  $46$M | 42.5 | $63.4$ | $44.9$ | $22.5$ | $45.9$ | $61.4$
DETR-DC5-R$50$ [4] | $500$ | $187$ |  $41$M | $43.3$ | $63.1$ | $45.9$ | $22.5$ | $47.3$ | $61.1$
DETR-DC5-R$50$ [4] | $50$ | $187$ |  $41$M | $36.7$ | $57.6$ | $38.2$ | $15.4$ | $39.8$ | $56.3$
Faster RCNN-FPN-R$50$∗ [31] | $108$ | $180$ |  $42$M | $42.0$ | $62.1$ | $45.5$ | $26.6$ | $45.5$ | $53.4$
TSP-RCNN-R$50$∗ [34] | $36$ | $188$ | $-$ | $43.8$ | $63.3$ | $48.3$ | $28.6$ | $46.9$ | $55.7$
SMCA-R$50$∗ [13] | $50$ | $152$ |  $40$M | $43.7$ | $63.6$ | $47.2$ | $24.2$ | $47.0$ | $60.4$
PNP-DETR-DC5-R$50$ [41] | $500$ | $144$ |  $41$M | $43.1$ | $63.4$ | $45.3$ | $22.7$ | $46.5$ | $61.1$
Deformable DETR-DC5-R50 [48] | 50 | $128$ |  $34$M | $41.5$ | $61.8$ | $44.9$ | $24.1$ | $45.3$ | $56.0$
Two-Stage Deformable DETR-DC5-R50 [48] | 50 | $128$ |  $34$M | $43.6$ | $63.6$ | $47.0$ | $25.8$ | $46.6$ | $58.1$
Anchor DETR-DC5-R50 [42] | $50$ | $173$ |  $37$M | $44.2$ | $64.7$ | $47.5$ | $24.7$ | $48.2$ | $60.6$
Conditional DETR-DC5-R50[25] | $50$ | $195$ |  $44$M | $43.8$ | $64.4$ | $46.7$ | $24.0$ | $47.6$ | $60.7$
Conditional DETR V2-DC5-R50 | $50$ | $161$ |  $46$M | $44.8$ | $65.3$ | $48.2$ | $25.5$ | $48.6$ | $62.0$
Conditional DETR V2-DC5-R50+ | $50$ | $181$ |  $46$M | 45.2 | $66.0$ | $48.4$ | $26.5$ | $49.0$ | $62.1$
DETR-R$101$ [4] | $500$ | $152$ |  $60$M | $43.5$ | $63.8$ | $46.4$ | $21.9$ | $48.0$ | $61.8$
DETR-R$101$ [4] | $50$ | $152$ |  $60$M | $36.9$ | $57.8$ | $38.6$ | $15.5$ | $40.6$ | $55.6$
Anchor DETR-R101 [42] | $50$ | $160$ |  $56$M | $43.5$ | $64.3$ | $46.6$ | $23.2$ | $47.7$ | $61.4$
Conditional DETR-R101 [25] | $50$ | $156$ |  $63$M | $42.8$ | $63.7$ | $46.0$ | $21.7$ | $46.6$ | $60.9$
Conditional DETR V2-R101 | $50$ | $155$ |  $65$M | 43.9 | $65.3$ | $46.7$ | $25.2$ | $48.0$ | $62.6$
DETR-DC5-R$101$ [4] | $500$ | $253$ |  $60$M | ${44.9}$ | ${64.7}$ | $47.7$ | $23.7$ | ${49.5}$ | ${62.3}$
DETR-DC5-R$101$ [4] | 50 | $253$ |  $60$M | $38.6$ | $59.7$ | $40.7$ | $17.2$ | $42.2$ | $57.4$
Faster RCNN-FPN-R$101$∗ [31] | $108$ | $246$ |  $60$M | $44.0$ | $63.9$ | $47.8$ | $27.2$ | $48.1$ | $56.0$
TSP-RCNN-R$101$∗ [34] | $36$ | $254$ | $-$ | $44.8$ | $63.8$ | $49.2$ | $29.0$ | $47.9$ | $57.1$
SMCA-R$101$∗ [13] | $50$ | $218$ |  $58$M | $44.4$ | $65.2$ | $48.0$ | $24.3$ | $48.5$ | $61.0$
Anchor DETR-DC5-R101 [42] | $50$ | $239$ |  $56$M | $45.1$ | $65.7$ | $48.8$ | $25.8$ | $49.4$ | $61.6$
Conditional DETR-DC5-R101[25] | $50$ | $262$ |  $63$M | $45.0$ | $65.5$ | $48.4$ | $26.1$ | $48.9$ | $62.8$
Conditional DETR V2-DC5-R101 | $50$ | $228$ |  $65$M | $45.5$ | $66.0$ | $48.7$ | $26.3$ | $49.0$ | $62.9$
Conditional DETR V2-DC5-R101+ | $50$ | $247$ |  $65$M | 45.9 | $66.6$ | $49.6$ | $27.4$ | $49.9$ | $62.6$
Conditional DETR-HR48 [25] | $50$ | $1090$ |  $87$M | $48.2$ | $68.2$ | $52.4$ | $30.6$ | $52.3$ | $64.3$
Conditional DETR V2-HR48 | $50$ | $521$ |  $90$M | 49.8 | $70.2$ | $54.2$ | $32.1$ | $53.3$ | $65.9$

Training. We follow the DETR training protocol [4]. The backbone is the
ImageNet-pretrained model from TORCHVISION with batchnorm layers fixed, and
the transformer parameters are initialized using the Xavier initialization
scheme [14]. We train the model on the COCO training set for $50$ epochs, with
the AdamW [24] optimizer. The weight decay is set to be $10^{-4}$. The
learning rates for the backbone and the transformer are initialized as
$10^{-5}$ and $10^{-4}$, respectively. The learning rate is dropped by a
factor of $10$ after $40$ epochs. The dropout rate in the transformer is
$0.1$. The head for the attention is $8$, the attention feature channel is
$256$ and the hidden dimension of the feed-forward network is $2048$. The
number of box queries is set as $300$ by default.

We use the augmentation scheme same as DETR [4]: resize the input image such
that the short side is at least $480$ and at most $800$ pixels and the long
side is at most $1333$ pixels; randomly crop the image such that a training
image is cropped with a probability $0.5$ to a random rectangular patch.

Evaluation. We use the standard COCO evaluation. We report the average
precision (AP), and the AP scores at $0.50$, $0.75$ and for the small, medium,
and large objects.

###  4.2 Results

Table 1 shows comparison of the proposed method with other detection methods
on COCO 2017 val. We first report the results of single scale methods: DETR
[4], PNP-DETR [41], deformable DETR [48], UP-DETR [11], Anchor DETR [42] and
Conditional DETR [25]. We follow [4] and report the results over four
backbones: ResNet-$50$ [15], ResNet-$101$, and their $16\times$-resolution
extensions DC$5$-ResNet-$50$ and DC$5$-ResNet-$101$. The $16\times$-resolution
extensions are obtained by adding a dilation convolution in the last stage and
removing a stride from the first convolution of the last stage. Our
Conditional DETR V2 with $50$ training epochs for R$50$ and R$101$ as the
backbones not only outperforms Conditional DETR, but also outperforms DETR
with $500$ training epochs. When it comes to a higher-resolution backbone such
as DC$5$-ResNet-$50$, our Conditional DETR V2 saves $17$% GFLOPs and improves
AP score by $1.0$ compared to Conditional DETR.

In addition, we list some methods that use the multi-scale feature. Although
our main purpose is not to study how to make better use of the multi-scale
feature, our single-scale models still achieve similar performance to most of
theirs. In the meanwhile, our computational cost ($e.g.,$ GFLOPs) is lower
than most of these methods and comparable to SMCA, which illustrates the
effectiveness of the proposed method.

We also adopt HRNet-W$48$ as the backbone to verify the generalization ability
of the proposed method. We down-sample the feature map from $4\times$ (the
initial output of HRNet) to $8\times$ to save the computation cost. Results in
Table 1 show that the proposed method outperforms Conditional DETR by $1.6$ AP
while saving $52$% GFLOPs. This illustrates that the proposed method could
generalize well to the high-resolution backbone.

###  4.3 Ablations

The effect of box query initialization. We empirically study how the model
benefits from the image-dependent box query. We compare three ways to
initialize the box query: (i) RB - we use the box queries to replace the
object queries that Conditional DETR uses in the first decoder layer. The
reference point and the transformation ($\mathbf{\lambda}_{q}$) are randomly
initialized. (ii) IP - the reference point is initialized from the image
content and the $\mathbf{\lambda}_{q}$ is randomly initialized. This is
similar to the two-stage strategy in Deformable DETR. (iii) IP+IT - both the
reference point and the $\mathbf{\lambda}_{q}$ are initialized from the image
content.

Table 2: The empirical results about the box query. RB = random box query. IP = image-dependent reference point. IT = image-dependent transformation. The backbone ResNet-50 is adopted. The Horizontal-Vertical Attention is not used here. Method | GFLOPs | AP
---|---|---
Conditional DETR | $89.5$ | $40.9$
Conditional DETR + RB | $89.5$ | $40.9$
Conditional DETR + IP | $90.0$ | $41.9$
Conditional DETR + IP + IT | $90.1$ | $42.9$
Table 3: The empirical results about the choice of $[c_{w}~{}c_{h}]$. The backbone ResNet-50 is adopted. The Horizontal-Vertical Attention is not used here. $[0.1,0.1]$ | $[0.2,0.2]$ | $[0.4,0.4]$ | GFLOPs | AP
---|---|---|---|---
✓ |  |  | $89.8$ | $42.2$
| ✓ |  | $89.8$ | $42.2$
|  | ✓ | $89.8$ | $42.3$
✓ | ✓ |  | $90.0$ | $42.6$
✓ | ✓ | ✓ | $90.1$ | $42.9$
Table 4: The empirical results about the effect of $[c_{x}~{}c_{y}]$ and $[c_{w}~{}c_{h}]$ that used for content query initialization. The backbone ResNet-50 is adopted. The Horizontal-Vertical Attention is not used here. $[c_{x}~{}c_{y}]$ | $[c_{w}~{}c_{h}]$ | GFLOPs | AP
---|---|---|---
✓ |  | $89.8$ | $42.0$
| ✓ | $89.8$ | $42.4$
✓ | ✓ | $90.1$ | $42.9$

Results are reported in Table 2. We find the randomly initialized box query
does not improve the performance, since the query has no image-dependent
information. If we initialize the reference point with positions selected from
the image content, the AP improves $1.0$. This is reasonable because the
initial reference points are likely in the object regions that we want to pay
attention to. If we further predict the transformation ($\lambda_{q}$) of the
box query from the image content as well, which contains the scale information
of the object, the performance improves to $42.9$ AP. This indicates that both
the initializations of the reference point and the transformation in the box
query are important. With an appropriate transformation that contains scale
information, we could dynamically adjust the spread of the attended regions
for different objects.

The effect of content query initialization. We initialize the content query
with the estimated box, as illustrated in Eq. (7). We first conduct
experiments to verify how different choices of $[c_{w}~{}c_{h}]$ affect the
performance. In implementation, we estimate three candidate boxes with
different scales at each position and the $[c_{w}~{}c_{h}]$ are [$0.1,0.1$],
[$0.2,0.2$] and [$0.4,0.4$], respectively. Please note that $0.1$ is the
normalized length, which is the ratio of box length to image length. In Table
3, we try different choices, $e.g.,$ estimate one or two boxes at one
position. Results show that with only one candidate box per position, the
performance has already improved to $42.2$ AP, which is $1.3$ higher than
Conditional DETR. When we estimate candidate boxes of two scales, the
performance further improves. Three scales performs the best. This is because
the model’s ability to detect objects of different scales is stronger.

When $[c_{w},c_{h}]$ is set as $[0.1,0.1],[0.2,0.2],[0.4,0.4]$, the AP scores
for large objects are $61.3,61.3,62.0$, and the AP scores for small objects
are $22.3,22.0,21.9$. With a larger initial scale, large objects will be
easier to detect. The performance of small objects drops a little because it
is harder to search for small objects with a too large initial scale.

Then, we verify the effect of $[c_{x}~{}c_{y}]$ and $[c_{w}~{}c_{h}]$ for
content query initialization in Table 4. $[c_{x}~{}c_{y}]$ helps search the
distinct region of the object for classification and $[c_{w}~{}c_{h}]$ helps
search the region with similar scale for box regression. Using the two items
together achieves the overall best result.

Table 5: The analysis of the computational cost and the speed. HVA = Horizontal-Vertical Attention. The backbone ResNet-$50$ and DC$5$-ResNet-$50$ are adopted. We follow DETR to test over the first 100 images of COCO 2017 val. The experiments are conducted on a single Tesla V100 GPU. Method | Backbone | GFLOPs | Memory (M) | FPS | AP
---|---|---|---|---|---
Conditional DETR |  ResNet-$50$ | $90$ | $2887$ | $22$ | $40.9$
Conditional DETR v2 w/o HVA |  ResNet-$50$ | $90$ | $2913$ | $20$ | $42.9$
Conditional DETR v2 | ResNet-$50$ | $89$ | $2093$ | $20$ | $42.5$
Conditional DETR |  DC$5$-ResNet-$50$ | $195$ | $18631$ | $10$ | $43.8$
Conditional DETR v2 w/o HVA |  DC$5$-ResNet-$50$ | $197$ | $18748$ | $10$ | $44.8$
Conditional DETR v2 | DC$5$-ResNet-$50$ | $161$ | $4844$ | $16$ | $44.8$
Table 6: Inference speed and AP performance on COCO 2017 val. TorchScript is not used here. All models are trained with $50$ epochs except DETR that is trained with 500 epochs. Our model achieves the best speed/accuracy trade-off. Method | Inference Speed (FPS) | AP
---|---|---
DETR | $10$ | $43.3$
SMCA | $10$ | $43.7$
Conditional DETR | $10$ | $43.8$
Anchor DETR | $16$ | $44.2$
Conditional DETR v2 | $16$ | $44.8$

Analysis of the computational cost and the speed. We compare the computational
cost and speed of the proposed method to Conditional DETR in Table 5. For the
ResNet-$50$ backbone, the proposed Horizontal-Vertical Attention reduces the
memory cost significantly, but only 1G flops are saved. This is because the
overall computational cost of the transformer head is relatively small for the
$32\times$ feature. For the DC$5$-ResNet-$50$ backbone, the global self-
attention consumes huge resources, $e.g.,18748$ M memory. The speed is much
slower due to the high computational complexity. With the help of the proposed
Horizontal-Vertical Attention, the memory cost is reduced by $74$% and the
speed is increased by $60$%. We also compare the efficiency to other DETR
variants in Table 6, where we could find that our method achieves the best
speed/accuracy trade-off.

##  5 Conclusion

We reformulate the object query in DETR to the format of the box query, which
is a composition of the embedding of the reference point and the
transformation of the box with respect to the reference point. We further
learn the box query from the image content instead of setting them as the
model parameters, which improves the detection performance. To save the memory
cost of the transformer encoder, we learn from axial self-attention and
propose to perform attention in horizontal and vertical directions in
parallel. Experiments verify the efficiency of the proposed method.

##  Appendix 0.A Visualization of the Reference Points

We select reference points from the image content, as illustrated in Section
$3.2$ in the main paper. We visualize the reference points in Figure 0.A and
find that the reference points mainly exist in the object regions. For
example, in the last image of the second row, almost all the reference points
are distributed on the three cows.

Figure 5:  Visualization of the reference points. We use $300$ reference
points in the model, which are marked with red in the images. Best viewed in
color.

##  Appendix 0.B Approach Details

Decoder cross attention weight. For the internal decoder layers, we calculate
the decoder cross attention weight in the same way as Conditional DETR [25]:

$\displaystyle\mathbf{c}_{q}^{\top}\mathbf{c}_{k}+\mathbf{p}_{q}^{\top}\mathbf{p}_{k}$
$\displaystyle=~{}$
$\displaystyle\mathbf{c}_{q}^{\top}\mathbf{c}_{k}+({\uplambda}_{q}\odot{\mathbf{p}}_{s})^{\top}\mathbf{p}_{k},$
(19)

where $\mathbf{c}_{q},\mathbf{c}_{k},\mathbf{p}_{q},\mathbf{p}_{k}$ are the
content query, content key, spatial query and spatial key, respectively. The
content query consists of the embedding $(\mathbf{p}_{s})$ of the reference
point, and the image-dependent transformation $({\uplambda}_{q})$.

For the first decoder layer, the cross attention weight of Conditional DETR
[25] is calculated by:

$\displaystyle(\mathbf{c}_{q}+\mathbf{o}_{q})^{\top}(\mathbf{c}_{k}+\mathbf{p}_{k})+\mathbf{o}_{q}^{\top}\mathbf{p}_{k},$
(20)

where $\mathbf{o}_{q}$ is the object query. This is different from Eq. (0.B)
since there is no image-dependent information as input in the first decoder
layer of Conditional DETR [25].

By contrast, we adopt the same formulation for the first decoder layer as Eq.
(0.B). The reference point is initialized from the image content and is likely
to lie in the object regions. The sinusoidal embedding is applied to the
reference point to obtain $\mathbf{p}_{s}$. The transformation
$({\uplambda}_{q})$ that contains the scale information of the object is
predicted from the corresponding encoder embedding of the reference point.

## References

  * [1] Beltagy, I., Peters, M.E., Cohan, A.: Longformer: The long-document transformer. CoRR abs/2004.05150 (2020)
  * [2] Bochkovskiy, A., Wang, C., Liao, H.M.: Yolov4: Optimal speed and accuracy of object detection. CoRR abs/2004.10934 (2020)
  * [3] Cai, Z., Vasconcelos, N.: Cascade R-CNN: delving into high quality object detection. In: CVPR (2018)
  * [4] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: ECCV (2020)
  * [5] Chen, X., Ding, M., Wang, X., Xin, Y., Mo, S., Wang, Y., Han, S., Luo, P., Zeng, G., Wang, J.: Context autoencoder for self-supervised representation learning. arXiv preprint arXiv:2202.03026 (2022)
  * [6] Chen, X., Lin, K.Y., Qian, C., Zeng, G., Li, H.: 3d sketch-aware semantic scene completion via semi-supervised structure prior. In: CVPR. pp. 4193–4202 (2020)
  * [7] Chen, X., Lin, K.Y., Wang, J., Wu, W., Qian, C., Li, H., Zeng, G.: Bi-directional cross-modality feature propagation with separation-and-aggregation gate for rgb-d semantic segmentation. In: ECCV. pp. 561–577. Springer (2020)
  * [8] Chen, X., Yuan, Y., Zeng, G., Wang, J.: Semi-supervised semantic segmentation with cross pseudo supervision. In: CVPR. pp. 2613–2622 (2021)
  * [9] Child, R., Gray, S., Radford, A., Sutskever, I.: Generating long sequences with sparse transformers. CoRR abs/1904.10509 (2019), http://arxiv.org/abs/1904.10509
  * [10] Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlós, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., Belanger, D., Colwell, L., Weller, A.: Rethinking attention with performers. CoRR abs/2009.14794 (2020), https://arxiv.org/abs/2009.14794
  * [11] Dai, Z., Cai, B., Lin, Y., Chen, J.: UP-DETR: unsupervised pre-training for object detection with transformers. CoRR abs/2011.09094 (2020), https://arxiv.org/abs/2011.09094
  * [12] Dai, Z., Yang, Z., Yang, Y., Carbonell, J.G., Le, Q.V., Salakhutdinov, R.: Transformer-xl: Attentive language models beyond a fixed-length context. In: ACL (2019)
  * [13] Gao, P., Zheng, M., Wang, X., Dai, J., Li, H.: Fast convergence of DETR with spatially modulated co-attention. CoRR abs/2101.07448 (2021), https://arxiv.org/abs/2101.07448
  * [14] Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: AISTATS (2010)
  * [15] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
  * [16] Ho, J., Kalchbrenner, N., Weissenborn, D., Salimans, T.: Axial attention in multidimensional transformers. CoRR abs/1912.12180 (2019), http://arxiv.org/abs/1912.12180
  * [17] Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., Liu, W.: Ccnet: Criss-cross attention for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 603–612 (2019)
  * [18] Kitaev, N., Kaiser, L., Levskaya, A.: Reformer: The efficient transformer. In: ICLR (2020), https://openreview.net/forum?id=rkgNKkHtvB
  * [19] Law, H., Deng, J.: Cornernet: Detecting objects as paired keypoints. In: Proceedings of the European conference on computer vision (ECCV). pp. 734–750 (2018)
  * [20] Lin, T., Goyal, P., Girshick, R.B., He, K., Dollár, P.: Focal loss for dense object detection. TPAMI (2020)
  * [21] Lin, T., Maire, M., Belongie, S.J., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. In: ECCV (2014)
  * [22] Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S.E., Fu, C., Berg, A.C.: SSD: single shot multibox detector. In: ECCV (2016)
  * [23] Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. arXiv preprint arXiv:2103.14030 (2021)
  * [24] Loshchilov, I., Hutter, F.: Fixing weight decay regularization in adam. In: ICLR (2017)
  * [25] Meng, D., Chen, X., Fan, Z., Zeng, G., Li, H., Yuan, Y., Sun, L., Wang, J.: Conditional detr for fast training convergence. In: ICCV. pp. 3651–3660 (2021)
  * [26] Peng, H., Pappas, N., Yogatama, D., Schwartz, R., Smith, N.A., Kong, L.: Random feature attention. CoRR abs/2103.02143 (2021), https://arxiv.org/abs/2103.02143
  * [27] Ramachandran, P., Parmar, N., Vaswani, A., Bello, I., Levskaya, A., Shlens, J.: Stand-alone self-attention in vision models. arXiv preprint arXiv:1906.05909 (2019)
  * [28] Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: CVPR (2017)
  * [29] Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. CoRR abs/1804.02767 (2018)
  * [30] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28, 91–99 (2015)
  * [31] Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. TPAMI (2017)
  * [32] Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I.D., Savarese, S.: Generalized intersection over union: A metric and a loss for bounding box regression. In: CVPR (2019)
  * [33] Roy, A., Saffar, M., Vaswani, A., Grangier, D.: Efficient content-based sparse attention with routing transformers. ACL (2021), https://transacl.org/ojs/index.php/tacl/article/view/2405
  * [34] Sun, Z., Cao, S., Yang, Y., Kitani, K.: Rethinking transformer-based set prediction for object detection. CoRR abs/2011.10881 (2020), https://arxiv.org/abs/2011.10881
  * [35] Tang, J., Chen, X., Wang, J., Zeng, G.: Not all voxels are equal: Semantic scene completion from the point-voxel perspective. In: AAAI. vol. 36, pp. 2352–2360 (2022)
  * [36] Tang, J., Chen, X., Wang, J., Zeng, G.: Point scene understanding via disentangled instance mesh reconstruction. In: ECCV (2022)
  * [37] Tian, Z., Shen, C., Chen, H., He, T.: FCOS: fully convolutional one-stage object detection. In: ICCV (2019)
  * [38] Vaswani, A., Ramachandran, P., Srinivas, A., Parmar, N., Hechtman, B., Shlens, J.: Scaling local self-attention for parameter efficient visual backbones. In: CVPR. pp. 12894–12904 (2021)
  * [39] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017)
  * [40] Wang, S., Li, B.Z., Khabsa, M., Fang, H., Ma, H.: Linformer: Self-attention with linear complexity. CoRR abs/2006.04768 (2020), https://arxiv.org/abs/2006.04768
  * [41] Wang, T., Yuan, L., Chen, Y., Feng, J., Yan, S.: Pnp-detr: Towards efficient visual analysis with transformers. In: ICCV. pp. 4661–4670 (2021)
  * [42] Wang, Y., Zhang, X., Yang, T., Sun, J.: Anchor detr: Query design for transformer-based detector. arXiv preprint arXiv:2109.07107 (2021)
  * [43] Xiong, Y., Zeng, Z., Chakraborty, R., Tan, M., Fung, G., Li, Y., Singh, V.: Nyströmformer: A nyström-based algorithm for approximating self-attention. CoRR abs/2102.03902 (2021), https://arxiv.org/abs/2102.03902
  * [44] Yuan, Y., Fu, R., Huang, L., Lin, W., Zhang, C., Chen, X., Wang, J.: Hrformer: High-resolution transformer for dense prediction. arXiv preprint arXiv:2110.09408 (2021)
  * [45] Zaheer, M., Guruganesh, G., Dubey, K.A., Ainslie, J., Alberti, C., Ontañón, S., Pham, P., Ravula, A., Wang, Q., Yang, L., Ahmed, A.: Big bird: Transformers for longer sequences. In: NeurIPS (2020), https://proceedings.neurips.cc/paper/2020/hash/c8512d142a2d849725f31a9a7a361ab9-Abstract.html
  * [46] Zhou, X., Wang, D., Krähenbühl, P.: Objects as points. CoRR abs/1904.07850 (2019), http://arxiv.org/abs/1904.07850
  * [47] Zhou, X., Zhuo, J., Krahenbuhl, P.: Bottom-up object detection by grouping extreme and center points. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 850–859 (2019)
  * [48] Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable DETR: deformable transformers for end-to-end object detection. CoRR abs/2010.04159 (2020), https://arxiv.org/abs/2010.04159


# Estimates for integrals of derivatives
of rational functions in multiply
connected domains in the plane

A. D. Baranov  Saint Petersburg State University<EMAIL_ADDRESS>and  I. R. Kayumov  Kazan Federal University<EMAIL_ADDRESS>
###### Abstract.

We obtain estimates for integrals of derivatives of rational functions in
multiply connected domains in the plane. A sharp order of the growth is found
for the integral of the modulus of the derivative of a finite Blaschke product
in the unit disk. We also extend the results of E. P. Dolzhenko about the
integrals of the derivatives of rational functions to a wider class of
domains, namely, to domains bounded by rectifiable curves without zero
interior angles, and show the sharpness of the obtained results.

###### Key words and phrases:

Rational function, conformal map, Blaschke product, Hardy space, John domain

A. D. Baranov was supported by Ministry of Science and Higher Education of the
Russian Federation, agreement No 075-15-2021-602. I. R. Kayumov was supported
by Ministry of Science and Higher Education of the Russian Federation,
agreement No 075-15-2019-1619.

##  1\. Introduction

70 years ago S. N. Mergelyan [1] showed that there exists a bounded analytic
function $f$ in the disk $\mathbb{D}=\\{|z|<1\\}$ such that

$I(f):=\int_{\mathbb{D}}|f^{\prime}(z)|\,dA(z)=\infty,$

where $dA(z)=\frac{1}{\pi}dxdy$, $z=x+iy$.

This problem was further investigated by W. Rudin [2] who constructed an
infinite Blaschke product

$B(z)=\prod_{k=1}^{\infty}\frac{|z_{k}|}{z_{k}}\frac{z_{k}-z}{1-\overline{z_{k}}z}$

such that $I(B)=\infty$ and, moreover,
$\int_{0}^{1}|B^{\prime}(re^{i\theta})|dr=\infty$ for a.e.
$\theta\in[0,2\pi]$. A similar, but more explicit example, was given by G.
Piranian [3].

It is then natural to ask what happens if $B$ is a finite Blaschke product of
degree $n$? It is obvious, that, for any fixed $n$, the quantity $I(B)$ is
bounded, but it cannot be uniformly bounded with respect to $n$, since any
bounded function in $\mathbb{D}$ is a locally uniform limit of finite Blaschke
products. We find the sharp order of growth for such integrals. Namely, we
have
Theorem 1. Let $B$ be a finite Blaschke product of degree $n$. Then

(1.1) $I(B)\leq\pi(1+\sqrt{\log n}).$

On the other hand, there exists an absolute constant $c>0$ such that for any
$n\in\mathbb{N}$ there exists a finite Blaschke product of degree $n$
satisfying $I(B)\geq c(1+\sqrt{\log n})$.

The proof of sharpness of this inequality is based on subtle results of N. G.
Makarov [4] and R. Bañuelos and C. N. Moore [5] on boundary behaviour of
functions from the Bloch space.

It should be noted that there exists a vast literature dealing with the
membership of the derivatives of the Blaschke products to various functional
spaces, e.g., Bergman-type spaces (see [6, 7, 8] and the references therein).
However, most of these results concern infinite products and the conditions
are formulated in terms of their zeros.

Since a Blaschke product is a bounded rational function in the unit disk, the
problem about the estimates of the derivatives of Blaschke products is related
to a more general question about the integrals of bounded rational functions
studied for the first time by E. P. Dolzhenko [9] for sufficiently smooth
domains. We will say that a curve belongs to the class K if it is a closed
Jordan curve with continuous curvature $k(s)$ satisfying a Hölder condition as
the function of the arc length $s$. Let $G$ be a finitely connected domain
whose boundary curves belong to the class K. Assume that $1\leq p\leq 2$ and
let $R$ be a rational function of degree at most $n$ with the poles outside
$\overline{G}$. Dolzhenko [9, Theorem 2.2] showed that there exists a constant
$C$ depending only on the domain $G$ and on $p$ such that

(1.2) $\int_{G}|R^{\prime}(w)|^{p}\,dA(w)\leq
Cn^{p-1}\|R\|_{H^{\infty}(G)}^{p},\qquad p\in(1,2],$ (1.3)
$\int_{G}|R^{\prime}(w)|\,dA(w)\leq C\ln(n+1)\|R\|_{H^{\infty}(G)}.$

Here we denote by $H^{\infty}(G)$ the space of all bounded analytic functions
in $G$, and $\|f\|_{H^{\infty}(G)}=\sup_{w\in G}|f(w)|$.

Later, inequalities for the derivatives of rational functions (mainly in the
disk) were studied in the papers by V. V. Peller [10], S. Semmes [11], A. A.
Pekarskii [12, 13], V. I. Danchenko [14, 15] and by many other authors (see,
e.g., [16, 17, 18, 19, 20]). A short proof of the Dolzhenko inequalities for
the case of the disk when the $H^{\infty}$-norm is replaced by the weaker
$BMOA$-norm can be found in [19].

In the present article the inequalities (1.2) and (1.3) are proved under
substantially weaker restrictions on the domain, namely, under the condition
that the domain has no zero interior angles (more precisely, for the John
class domains – see the definition in §3).
Theorem 2. Let $G$ be a finitely connected John domain with the rectifiable
boundary and let $1\leq p\leq 2$. Then there exists a constant $C>0$,
depending on the domain $G$ and on $p$, such that for any rational function
$R$ of degree at most $n$ the inequalities (1.2) and (1.3) hold.

The sharpness of (1.2) is seen already on the simplest example of the function
$R(z)=z^{n}$ in the disk (obviously, we can consider polynomials as a special
case of rational functions with the pole at infinity). The question about
sharpness of the estimate (1.3) in the conditions of Theorem 2 remains open.
However, it turns out that under some additional regularity of the domain $G$
inequality (1.3) can be improved.
Theorem 3. Let $G$ be a simply connected domain such that $\varphi^{\prime}\in
H^{2}$, where $\varphi$ is the conformal map of the disk $\mathbb{D}$ onto
$G$. Then there exists a constant $C>0$ depending on the domain $G$ such that
for any rational function $R$ of degree at most $n$ one has

(1.4) $\int_{G}|R^{\prime}(w)|\,dA(w)\leq
C\sqrt{\ln(n+1)}\|R\|_{H^{\infty}(G)}.$

As follows from Theorem 1, the dependence on $n$ in this inequality is sharp.

Finally, we give a statement for the case $p>2$. Here $q$ is the conjugate
exponent, i.e., $1/p+1/q=1$.
Theorem 4.  Let $G$ be a bounded simply connected domain and put
$G_{\rho}=\\{z\in G:{\rm dist}\,(z,\partial G)>\rho\\}$. Then for any rational
function $R$ of degree at most $n$ and $p>2$ one has

(1.5)
$\|R^{\prime}\|_{A^{p}(G_{\rho})}=\bigg{(}\int_{G_{\rho}}|R^{\prime}(w)|^{p}\,dA(w)\bigg{)}^{1/p}\leq
n^{1/p}\rho^{1/p-1/q}\|R\|_{H^{\infty}(G)}.$

In [9] inequality (1.5) was established for domains of the class K, but, as
our result shows, no restrictions on the regularity of the domain are
required.

In [20] another generalization of the Dolzhenko inequality for the case $p>2$
was obtained for the rational functions in the disk. Let $R$ be a rational
function of degree at most $n$ whose poles lie in the complement of the disk
$\\{|z|<1+\rho\\}$. The following inequality follows directly from [19,
Theorem 8.2]:

$\|R^{\prime}\|_{A^{p}(\mathbb{D})}\leq
C(p)n^{1/q}\rho^{1/p-1/q}\|R\|_{BMOA};$

here $BMOA$ denotes the analytic space of functions of bounded mean
oscillation in the disk. It is interesting to note that the dependence on $n$
in Theorem 4 is substantially weaker (since in this case the function $R$ is
assumed to be bounded on a larger set).

In §5 more general inequalities are obtained for the weighted norms of the
derivatives of rational functions, where the weight is given as some power of
the distance to the boundary of the domain.

A suitable toolbox for the study of such inequalities is provided by the
theory of the Hardy spaces. For $p>0$ the Hardy space $H^{p}$ is the set of
all analytic functions in $\mathbb{D}$ satisfying $\|f\|_{H^{p}}<\infty$,
where

$\|f\|^{p}_{H^{p}}:=\sup_{0<r<1}\frac{1}{2\pi}\int_{0}^{2\pi}|f(re^{it})|^{p}dt.$

Note that for $p\geq 1$ the last quantity defines a norm with respect to which
$H^{p}$ is a Banach space.

##  2\. Estimate for the integral of the modulus of the derivative of a finite
Blaschke product

In the proof of Theorem 1 we will use the following simple lemma.
Lemma 1. Assume that the function $g(z)=\sum_{k=0}^{\infty}b_{k}z^{k}$ is
analytic in the disk $\mathbb{D}$. If $\|g\|_{\infty}\leq 1$ and
$p(z)=\sum_{k=0}^{n}b_{k}z^{k}$, $n\geq 2$, then there exists an absolute
constant $C_{0}$ such that

$|p(z)|\leq C_{0},\qquad|z|\leq 1-2\log n/n,$

and

$|g^{\prime}(z)-p^{\prime}(z)|\leq C_{0},\qquad|z|\leq 1-2\log n/n.$

Proof. Since $|b_{k}|\leq 1$ and $|z|^{k}\leq 1/n^{2}$ for $|z|\leq 1-2\log
n/n$ and $k\geq n$, the function $|g(z)-p(z)|$ admits a uniform estimate for
$|z|\leq 1-2\log n/n$, and the first estimate follows.

Clearly,

$\sum_{k=n}^{\infty}(k+1)|b_{k+1}z^{k}|\leq\frac{|z|^{n}}{(1-|z|)^{2}}+\frac{n|z|^{n}}{1-|z|}\leq
const,\qquad|z|\leq 1-2\log n/n,$

and the second inequality is proved.

Proof of Theorem. We use the following well-known facts:

$\int_{0}^{2\pi}|B^{\prime}(re^{it})|dt\leq 2\pi n,\qquad r\in[0,1],$

for any finite Blaschke product of degree at most $n$ and

(2.1)
$\int_{\mathbb{D}}|f^{\prime}(z)|^{2}(1-|z|^{2})\,dA(z)=\sum_{n=1}^{\infty}\frac{n}{n+1}|a_{n}|^{2}\leq\|f\|^{2}_{H^{2}}$

for any function $f(z)=\sum_{n\geq 0}a_{n}z^{n}$ in the Hardy space $H^{2}$.

Let $s\in[0,1]$. We have

$\int_{\\{s<|z|<1\\}}|B^{\prime}(z)|\,dA(z)=\int_{0}^{2\pi}\int_{s}^{1}|B^{\prime}(re^{it})|rdrdt\leq
2\pi n\int_{s}^{1}rdr=\pi n(1-s^{2}).$

In the remaining part we apply the Cauchy–Schwarz inequality:

$\int_{\\{0<|z|\leq s\\}}|B^{\prime}(z)|\,dA(z)$
$\leq\bigg{(}\int_{\\{0<|z|\leq
s\\}}|B^{\prime}(z)|^{2}(1-|z|^{2})\,dA(z)\bigg{)}^{1/2}\bigg{(}\int_{\\{0<|z|\leq
s\\}}\frac{dA(z)}{1-|z|^{2}}\bigg{)}^{1/2}$
$\leq\sqrt{\pi}\sqrt{2\pi\int_{0}^{s}\frac{rdr}{1-r^{2}}}=\pi\sqrt{\log\frac{1}{1-s^{2}}}.$

Thus,

$I(B)\leq\pi n(1-s^{2})+\pi\sqrt{\log\frac{1}{1-s^{2}}}.$

Taking $s^{2}=1-1/n$ we obtain (1.1).

The estimate from below can be obtained by the methods based on the Makarov
law of the iterated logarithm [4]. Recall that the Bloch class $\mathcal{B}$
consists of functions analytic in $\mathbb{D}$ with finite seminorm
$\|f\|_{\mathcal{B}}=\sup_{z\in\mathbb{D}}(1-|z|^{2})|f^{\prime}(z)|$. In [5]
R. Bañuelos and C. N. Moore, answering a question of N. G. Makarov and F.
Przytycki, constructed a function $f(z)=\sum_{k=1}^{\infty}a_{k}z^{k}$ in the
Bloch class such that its asymptotic entropy admits the lower bound

$\liminf_{r\to
1-}\frac{\sum_{k=1}^{\infty}|a_{k}|^{2}r^{2k}}{\log\frac{1}{1-r}}>0,$

whereas for all $\zeta$ with $|\zeta|=1$ one has

$\limsup_{r\to
1-}\frac{f(r\zeta)}{\sqrt{\log\frac{1}{1-r}\log\log\log\frac{1}{1-r}}}=0.$

Moreover, in [5, p. 852–853] a sequence of polynomials

$p_{n}(z)=\sum_{k=4}^{4^{n+1}-1}a_{k}z^{k}=\sum_{j=1}^{n}b_{j}(z),\qquad\text{where}\qquad
b_{j}(z)=\sum_{k=4^{j}}^{4^{j+1}-1}a_{k}z^{k},$

is constructed such that $\|b_{j}\|_{\infty}\leq 1$,

$\sum_{k=1}^{4^{n+1}-1}|a_{k}|^{2}\geq c\log
m,\qquad\|p_{n}\|_{H^{\infty}}\leq C\sqrt{\log m},$

where $m={\rm deg}\,p_{n}=4^{n+1}-1$ and $C,c>0$ are some absolute positive
constants.

It is not difficult to deduce from $\|b_{j}\|_{\infty}\leq 1$ that
$\sup_{n}\|p_{n}\|_{\mathcal{B}}<\infty$. Indeed, by the Schwarz lemma
$|b_{j}(rz)|\leq r^{4^{j}}$, whereas, by the classical Bernstein inequality,
$|b_{j}^{\prime}(rz)|\leq 4^{j}r^{4^{j}}$. It is well known (and easy to show)
that $\sum_{j\geq 1}4^{j}r^{4^{j}}\leq C_{1}/(1-r^{2})$ for some constant
$C_{1}>0$ and, thus, $\sup_{n}\|p_{n}\|_{\mathcal{B}}<\infty$. Without loss of
generality we may assume that $\|p_{n}\|_{\mathcal{B}}\leq 1$.

Let $r=1-1/m$. Then, for some absolute constants $C^{\prime},c^{\prime}>0$,

$c^{\prime}\log\frac{1}{1-r}\leq\sum_{k=1}^{m}|a_{k}|^{2}r^{2k}\leq
2\int_{|z|<r}|p_{n}^{\prime}(z)|^{2}(1-|z|^{2})\,dA(z)\leq
C^{\prime}\int_{|z|<r}|p_{n}^{\prime}(z)|\,dA(z).$

Now put $q_{n}=p_{n}/(C\sqrt{\log m})$. Then $\|q_{n}\|_{\infty}\leq 1$ and

$\int_{|z|<1-1/m}|q_{n}^{\prime}(z)|\,dA(z)\geq c_{1}\sqrt{\log m}$

for some absolute constant $c_{1}>0$. Since

$\int_{1-2\log m/m<|z|<1-1/m}|q_{n}^{\prime}(z)|\,dA(z)\leq\int_{1-2\log
m/m<|z|<1-1/m}\frac{dA(z)}{1-|z|^{2}}=O(\log\log m),$

we have, for some $c_{2}>0$,

(2.2) $\int_{|z|<1-2\log m/m}|q_{n}^{\prime}(z)|\,dA(z)\geq c_{2}\sqrt{\log
m}.$

Let $B$ be a Blaschke product of degree at most $m+1$ such that its first $m$
Taylor coefficients coincide with respective coefficients of the polynomial
$q_{n}$. By Lemma 1,

$|q_{n}^{\prime}(z)-B^{\prime}(z)|\leq 2C_{0},\qquad|z|\leq 1-2\log m/m.$

Hence, it follows from (2.2) that

$\int_{\mathbb{D}}|B^{\prime}(z)|\,dA(z)\geq c_{3}\sqrt{\log m}$

for some absolute constant $c_{3}$. Theorem 1 is proved.

##  3\. Estimates for integrals of rational functions

Recall that a finitely connected domain $\Omega$ is said to be a John domain
if there exists a constant $C>0$ such that any points $a,b\in\Omega$ can be
connected by a curve $\gamma$ in $\Omega$ with the following property: for any
$x\in\gamma$,

$\min\big{(}{\rm diam}\,\gamma(a,x),{\rm diam}\,\gamma(x,b)\big{)}\leq C{\rm
dist}\,(x,\partial\Omega).$

Here $\gamma(a,x)$ and $\gamma(x,b)$ denote the corresponding subarcs of
$\gamma$. For equivalent definitions and properties of John domains see, e.g.,
[21, 22]. Essentially, this definition means that the domain has no zero
interior angles. In particular, a domain is a John domain if it satisfies the
cone conditon: one can touch each boundary point from inside of the domain by
some sufficiently small triangle with fixed angles.

In what follows we will essentially use the following property of simply
connected John domains: if $\varphi$ is the conformal map of $\mathbb{D}$ onto
a simply connected John domain, then

(3.1) $|\varphi^{\prime}(z)|\leq\frac{C}{(1-|z|)^{\alpha}}$

for some $\alpha\in(0,1)$ and $C>0$ (see [22, pp. 96–100]).

In the proof of Theorem 2 we will use the following simple lemma:
Lemma 2. Let $g$ be a bounded and at most $n$-valent function in $\mathbb{D}$.
Then

$\frac{1}{2\pi}\int_{0}^{2\pi}|g^{\prime}(re^{it})|^{2}dt\leq\frac{n}{1-r}\|g\|^{2}_{H^{\infty}(\mathbb{D})}.$

Proof. Let $g(z)=\sum_{k=0}^{\infty}a_{k}z^{k}$. Then

$\frac{1}{2\pi}\int_{0}^{2\pi}|g^{\prime}(re^{it})|^{2}dt=\sum_{k=1}^{\infty}k^{2}|a_{k}|^{2}r^{2k}\leq\frac{1}{1-r}\sum_{k=1}^{\infty}k|a_{k}|^{2}.$

Here we used the elementary inequality $kr^{k-1}(1-r)\leq 1$, $k\geq 1$,
$r\in[0,1)$. Since $g$ is at most $n$-valent in $\mathbb{D}$, we have, in
virtue of the classical area theorem,

$\sum_{k=1}^{\infty}k|a_{k}|^{2}\leq n\|g\|^{2}_{H^{\infty}(\mathbb{D})}.$

Proof of Theorem 2. Assume first that the domain $G$ is bounded. Without loss
of generality we may assume that $G$ is a simply connected domain with a
rectifiable boundary. Indeed, making smooth cuts (and controlling the angles)
one can easily represent our domain as a finite union of simply connected John
domains.

Let $w=\varphi(z)$ be a conformal map of $\mathbb{D}$ onto $G$. Since the
boundary of $G$ is rectifiable, we have $\varphi^{\prime}\in H^{1}$. Also,
$\varphi^{\prime}$ satisfies inequality (3.1).

By the change of the variable,

$\int_{G}|R^{\prime}(w)|^{p}\,dA(w)=\int_{\mathbb{D}}|R^{\prime}(\varphi(z))|^{p}|\varphi^{\prime}(z)|^{2}\,dA(z)=\int_{\mathbb{D}}|(R\circ\varphi)^{\prime}(z)|^{p}|\varphi^{\prime}(z)|^{2-p}dA(z).$

It is obvious, that for $p=2$ the last integral does not exceed
$n\|R\|_{H^{\infty}(G)}^{2}$, since the function $R\circ\varphi$ is at most
$n$-valent in the disk $\mathbb{D}$.

Let us split the last integral into the integrals over the set $\\{|z|\leq
r_{n}\\}$ and over the set $\\{r_{n}<|z|<1\\}$, where
$r_{n}=1-\frac{1}{(n+1)^{K}}$ and the number $K>0$ is to be chosen later.
Estimate of the integral over the set $\\{|z|\leq r_{n}\\}$. Let
$M=\|R\|_{H^{\infty}(G)}$. Set

$J:=\int_{\\{|z|\leq
r_{n}\\}}|(R\circ\varphi)^{\prime}(z)|^{p}|\varphi^{\prime}(z)|^{2-p}dA(z).$

For $p=1$ we use the estimate $(1-|z|^{2})|(R\circ\varphi)^{\prime}(z)|\leq
M$. We have

$\displaystyle J\leq\frac{M}{\pi}\int_{0}^{r_{n}}$
$\displaystyle\frac{1}{1-r}\int_{0}^{2\pi}|\varphi^{\prime}(re^{it})|dtdr$
$\displaystyle\leq
2\|\varphi^{\prime}\|_{H^{1}}M\int_{0}^{r_{n}}\frac{dr}{1-r}=2K\|\varphi^{\prime}\|_{H^{1}}\log(n+1)M.$

In the case $1<p<2$ consider separately the integrals over the sets
$\\{|z|\leq 1-\frac{1}{n+1}\\}$ and ${\\{1-\frac{1}{n+1}<|z|\leq r_{n}\\}}$.
Since $\varphi^{\prime}\in H^{1}$, we have $\varphi^{\prime}\in H^{2-p}$ and
$\|\varphi^{\prime}\|_{H^{2-p}}\leq\|\varphi^{\prime}\|_{H^{1}}$. Hence,

$\displaystyle\int_{\big{\\{}|z|\leq
1-\frac{1}{n+1}\big{\\}}}|(R\circ\varphi)^{\prime}(z)|^{p}|\varphi^{\prime}(z)|^{2-p}dA(z)$
$\displaystyle\leq
2\|\varphi^{\prime}\|_{H^{1}}^{2-p}\int_{0}^{1-\frac{1}{n+1}}\frac{M^{p}}{(1-r)^{p}}dr$
$\displaystyle=2\|\varphi^{\prime}\|_{H^{1}}^{2-p}(p-1)^{-1}(n+1)^{p-1}M^{p}.$

To estimate the integral over the set $\\{1-\frac{1}{n+1}<|z|\leq r_{n}\\}$ we
use the Hölder inequality with exponents $(2-p)^{-1}$ and $(p-1)^{-1}$:

$J\leq
2\int_{1-\frac{1}{n+1}}^{r_{n}}\bigg{(}\frac{1}{2\pi}\int_{0}^{2\pi}|(R\circ\varphi)^{\prime}(re^{it})|^{\frac{p}{p-1}}dt\bigg{)}^{p-1}\bigg{(}\frac{1}{2\pi}\int_{0}^{2\pi}|\varphi^{\prime}(re^{it})|dt\bigg{)}^{2-p}dr.$

Using successively the inequality
$(1-|z|^{2})|(R\circ\varphi)^{\prime}(z)|\leq M$ and Lemma 2 we get

$\displaystyle J$ $\displaystyle\leq
2\|\varphi^{\prime}\|_{H^{1}}^{2-p}M^{p-2(p-1)}\int_{1-\frac{1}{n+1}}^{r_{n}}\frac{1}{(1-r)^{p-2(p-1)}}\bigg{(}\frac{1}{2\pi}\int_{0}^{2\pi}|(R\circ\varphi)^{\prime}(re^{it})|^{2}dt\bigg{)}^{p-1}dr$
$\displaystyle\leq
2\|\varphi^{\prime}\|_{H^{1}}^{2-p}M^{p}n^{p-1}\int_{1-\frac{1}{n+1}}^{r_{n}}\frac{dr}{(1-r)^{2-p+p-1}}$
$\displaystyle=2\|\varphi^{\prime}\|_{H^{1}}^{2-p}M^{p}n^{p-1}\int_{1-\frac{1}{n+1}}^{1-\frac{1}{(n+1)^{K}}}\frac{dr}{1-r}=2\|\varphi^{\prime}\|_{H^{1}}^{2-p}Kn^{p-1}M^{p}.$

Estimate of the integral over the set $\\{r_{n}<|z|<1\\}$. In this case the
argument applies to all $p\in[1,2)$. Choose $\delta$ such that
$0<\delta<1-p/2$. Then $2-p-\delta\in(0,1)$. Let $\beta$ be the exponent
conjugate to $(2-p-\delta)^{-1}$. One has, by (3.1),

$I:=\int_{r_{n}<|z|<1}|(R\circ\varphi)^{\prime}(z)|^{p}|\varphi^{\prime}(z)|^{2-p}dA(z)$
$\leq
C^{\delta}\int_{r_{n}<|z|<1}\frac{|(R\circ\varphi)^{\prime}(z)|^{p}|\varphi^{\prime}(z)|^{2-p-\delta}}{(1-|z|)^{\alpha\delta}}dA(z)$
$\leq
2C^{\delta}\int_{r_{n}}^{1}\frac{1}{(1-r)^{\alpha\delta}}\bigg{(}\frac{1}{2\pi}\int_{0}^{2\pi}|(R\circ\varphi)^{\prime}(re^{it})|^{p\beta}dt\bigg{)}^{1/\beta}\bigg{(}\frac{1}{2\pi}\int_{0}^{2\pi}|\varphi^{\prime}(re^{it})|dt\bigg{)}^{2-p-\delta}dr.$

Note that it follows from $\delta<1-p/2$ that $p\beta>2$. Applying the
estimate $(1-|z|^{2})|(R\circ\varphi)^{\prime}(z)|\leq M$ and Lemma 2, we get

$\displaystyle I$ $\displaystyle\leq
2C^{\delta}\|\varphi^{\prime}\|_{H^{1}}^{2-p-\delta}M^{\frac{p\beta-2}{\beta}}\int_{r_{n}}^{1}\frac{1}{(1-r)^{\alpha\delta+\frac{p\beta-2}{\beta}}}\bigg{(}\frac{1}{2\pi}\int_{0}^{2\pi}|(R\circ\varphi)^{\prime}(re^{it})|^{2}dt\bigg{)}^{1/\beta}dr$
$\displaystyle\leq
2C^{\delta}\|\varphi^{\prime}\|_{H^{1}}^{2-p-\delta}M^{p}n^{1/\beta}\int_{r_{n}}^{1}\frac{dr}{(1-r)^{\alpha\delta+\frac{p\beta-2}{\beta}+\frac{1}{\beta}}}$
$\displaystyle=2C^{\delta}\|\varphi^{\prime}\|_{H^{1}}^{2-p-\delta}M^{p}n^{1/\beta}\int_{r_{n}}^{1}\frac{dr}{(1-r)^{\alpha\delta+p-\frac{1}{\beta}}}.$

It remains to notice that
$\alpha\delta+p-\frac{1}{\beta}=\alpha\delta+p-(1-(2-p-\delta))=1-(1-\alpha)\delta$,
whence

(3.2) $I\leq
2(1-\alpha)^{-1}\delta^{-1}C^{\delta}\|\varphi^{\prime}\|_{H^{1}}^{2-p-\delta}M^{p}n^{1/\beta}(1-r_{n})^{(1-\alpha)\delta}.$

If we fix $\delta\in(0,1-p/2)$ and choose a sufficiently large $K$ in
$r_{n}=1-\frac{1}{(n+1)^{K}}$, we conclude that $I\leq
C^{\delta}\|\varphi^{\prime}\|_{H^{1}}^{2-p-\delta}M^{p}$ (and even $o(1)$ as
$n\to\infty$).

We now consider the case when $\infty\in G$. It is clear that such domain can
be represented as a union (possibly with an intersection) of the complement of
a disk of sufficiently large radius with a simply connected bounded John
domain. The statement is already proved for bounded simply connected domains,
while for the complement of a disk it follows from the results of Dolzhenko
cited above. Theorem 2 is proved.

##  4\. Proofs of Theorems 3 and 4

Proof of Theorem 3. As in the proof of Theorem we set
$r_{n}=1-\frac{1}{(n+1)^{K}}$, where $K>0$. Since $\varphi^{\prime}\in
H^{2}\subset H^{1}$ and the condition (3.1) is satisfied with $\alpha=1/2$,
one can use the estimate (3.2) for the integral over the set
$\\{r_{n}<|z|<1\\}$ established in the proof of Theorem 2. For a sufficiently
large $K$ this integral is uniformly bounded over $n$ (and even tends to zero
as $n\to\infty$).

Thus, it suffices to estimate

$J:=\int_{0<|z|\leq
r_{n}}|(R\circ\varphi)^{\prime}(z)||\varphi^{\prime}(z)|dA(z).$

By the Cauchy–Schwarz inequality,

$\displaystyle J$ $\displaystyle\leq\bigg{(}\int_{0<|z|\leq
r_{n}}(1-|z|)|(R\circ\varphi)^{\prime}(z)|^{2}dA(z)\bigg{)}^{1/2}\bigg{(}\int_{0<|z|\leq
r_{n}}\frac{|\varphi^{\prime}(z)|^{2}}{1-|z|}dA(z)\bigg{)}^{1/2}$
$\displaystyle\leq M\bigg{(}\int_{0<|z|\leq
r_{n}}\frac{|\varphi^{\prime}(z)|^{2}}{1-|z|}dA(z)\bigg{)}^{1/2}\leq\sqrt{2}M\|\varphi^{\prime}\|_{H^{2}}\bigg{(}\int_{0}^{r_{n}}\frac{dr}{1-r}\bigg{)}^{1/2}$
$\displaystyle=\sqrt{2}M\|\varphi^{\prime}\|_{H^{2}}\sqrt{K\ln(n+1)}.$

Theorem 3 is proved.

Proof of Theorem 4. Let $\varphi$ be a conformal map of $\mathbb{D}$ onto $G$
and let $D_{\rho}=\varphi^{-1}(G_{\rho})$. Since
$\rho\leq(1-|z|^{2})|\varphi^{\prime}(z)|$, $z\in D_{\rho}$, we have

$\displaystyle\int_{G_{\rho}}|R^{\prime}(\zeta)|^{p}dA(\zeta)$
$\displaystyle=\int_{D_{\rho}}|(R\circ\varphi)^{\prime}(z)|^{p}|\varphi^{\prime}(z)|^{2-p}dA(z)$
$\displaystyle\leq\rho^{2-p}\int_{D_{\rho}}|(R\circ\varphi)^{\prime}(z)|^{p}(1-|z|^{2})^{p-2}dA(z)$
$\displaystyle\leq\rho^{2-p}M^{p-2}\int_{D_{\rho}}|(R\circ\varphi)^{\prime}(z)|^{2}dA(z)\leq\rho^{2-p}nM^{p}.$

At the last step we used the fact that $R\circ\varphi$ covers each point of
the disk of radius $M$ with multiplicity at most $n$. Theorem 4 is proved.

##  5\. Weighted inequalities of Dolzhenko and Peller type

As a natural generalization of the Dolzhenko inequalities one can consider
weighted integrals of derivatives of rational functions. Similar inequalities
were studied extensively in the setting of the Bergman (or Besov) spaces.
E.g., a well-known inequality by V.V. Peller [10] states that for a rational
function $R$ of degree $n$ with poles outside $\overline{\mathbb{D}}$ one has

$\|R\|_{B_{p}^{1/p}}\leq Cn^{1/p}\|R\|_{BMOA},$

where $B_{p}^{1/p}$ is the Besov space, $p>0$, $C=C(p)$. In particular, for
$1<p<\infty$,

$\int_{\mathbb{D}}|R^{\prime}(z)|^{p}(1-|z|)^{p-2}dA(z)\leq
Cn\|R\|^{p}_{H^{\infty}}.$

Various proofs and generalizations of this inequality can be found in [11, 12,
13, 19].

Using the methods of §3 one can obtain more general weighted estimates where
the weight equals to some power of the distance to the boundary. To formulate
the corresponding result, we set, for a bounded domain $G\subset\mathbb{C}$
and $z\in G$,

$d_{G}(z):={\rm dist}\,(z,\partial G).$

For $p\geq 1$, $\beta\in\mathbb{R}$ and a function $f$ analytic in $G$ put

$I_{p,\beta}(f):=\int_{G}|f^{\prime}(\zeta)|^{p}\,d_{G}^{\beta}(\zeta)\,dA(\zeta)$

(in general, the quantity $I_{p,\beta}(f)$ can be infinite). We are interested
in the estimates of the form

$I_{p,\beta}(R)\leq C\Psi(n)\|R\|^{p}_{H^{\infty}(G)},$

which hold for all rational functions $R$ of degree at most $n$ with poles
outside $\overline{G}$ and with a constant $C$, depending on $G$, $p$ and
$\beta$, but not on $n$ and $R$. Here $\Psi$ is some function depending on $n$
only. It is easy to see that such estimates are possible only for $\beta\geq
p-2$; it is seen already from the example $G=\mathbb{D}$ and rational
fractions $R(\zeta)=\frac{1}{\zeta-\lambda}$ that for $\beta<p-2$ the integral
$I_{p,\beta}(f)$ does not admit the estimate depending only on $n$, one has
also to take into account the distance from the poles of $R$ to $\partial G$.

To simplify the notations, in what follows we write $X(R,n)\lesssim Y(R,n)$,
if $X(R,n)\leq CY(R,n)$ with a constant $C$, depending only from $G$, $p$ and
$\beta$, but not on $n$ and $R$.

Theorem 5. Let $G$ be a simply connected bounded domain, $\varphi$ is the
conformal map of $\mathbb{D}$ onto $G$, $p\geq 1$, $\beta\geq p-2$. The
following estimates hold true.

1\. If $\beta>p-1$ and $\varphi^{\prime}\in H^{\gamma}$ for some $\gamma>1$,
then $I_{p,\beta}(R)\lesssim\|R\|^{p}_{H^{\infty}(G)}$, i.e., the dependence
on $n$ disappears.

2\. If $\beta=p-1$, $1\leq p<2$ and $\varphi^{\prime}\in H^{\frac{2}{2-p}}$,
then

$I_{p,\beta}(R)\lesssim\big{(}\log
n\big{)}^{1-\frac{p}{2}}\|R\|^{p}_{H^{\infty}(G)}.$

If $\beta=p-1$, $p\geq 2$ and $\varphi^{\prime}\in H^{\infty}$, then
$I_{p,\beta}(R)\lesssim\|R\|^{p}_{H^{\infty}(G)}$.

3\. If $p-2\leq\beta<p-1$, $p\geq 2$, $\varphi^{\prime}\in H^{1}$ and $G$ is a
John domain, then

$I_{p,\beta}(R)\lesssim n^{p-1-\beta}\|R\|^{p}_{H^{\infty}(G)}.$

The dependence on $n$ in the inequalities in Theorem 5 is sharp already for
the case of the unit disk. In statement 3 the optimal growth is attained on
$R(z)=z^{n}$, while the sharpness of the inequality in statement 2 can be
shown by considering the Bañuelos–Moore construction (as $R$ one can take a
polynomial or a Blaschke product).

Note that statement 3 of the theorem does not cover the case
$p-2\leq\beta<p-1$ and $1<p<2$. In this case it would be sufficient to prove
the following analogue of Peller’s inequality:

$\int_{\mathbb{D}}|(R\circ\varphi)^{\prime}(z)|^{p}(1-|z|)^{p-2}dA(z)\lesssim
n\|R\|_{H^{\infty}(G)}^{p},$

where $G=\varphi(\mathbb{D})$ is a John domain and $R$ is a rational function
of degree at most $n$ with the poles outside $\overline{G}$. However, we do
not know whether this inequality holds true.
Proof. Put $M=\|R\|_{H^{\infty}(G)}$. Let us make the change of the variable
$\zeta=\varphi(z)$. Taking into account that
$d_{G}(\zeta)\leq|\varphi^{\prime}(z)|(1-|z|^{2})$, we obtain

$I_{p,\beta}(R)\lesssim\int_{\mathbb{D}}|(R\circ\varphi)^{\prime}(z)|^{p}|\varphi^{\prime}(z)|^{2-p+\beta}(1-|z|)^{\beta}\,dA(z).$

Statement 3 follows from Theorem 2. Indeed, $1<p-\beta\leq 2$ and, using the
inequality $|(R\circ\varphi)^{\prime}(z)|(1-|z|)\leq M$, we obtain

$I_{p,\beta}(R)\leq
M^{\beta}\int_{\mathbb{D}}|(R\circ\varphi)^{\prime}(z)|^{p-\beta}|\varphi^{\prime}(z)|^{2-(p-\beta)}\,dA(z)\lesssim
n^{p-\beta-1}M^{p}.$

Let us prove statement 1. The quantity $d_{G}(z)$ is bounded, so it suffices
to prove the statement for $\beta\in(p-1,p-2+\gamma]$. For such $\beta$ it
follows from the inequality $p-\beta<1$ and the inclusion $\varphi^{\prime}\in
H^{\gamma}\subset H^{2-p+\beta}$ that

$I_{p,\beta}(R)\lesssim
M^{p}\int_{0}^{1}\bigg{(}\int_{0}^{2\pi}|\varphi^{\prime}(re^{it})|^{2-p+\beta}dt\bigg{)}\frac{rdr}{(1-r)^{p-\beta}}\lesssim
M^{p}.$

At the first step we used the inequality
$|(R\circ\varphi)^{\prime}(z)|(1-|z|)\leq M$.

Consider the most interesting statement 2: $\beta=p-1$. Let $1\leq p<2$. Put
$s=1-\frac{1}{n}$. Applying the Hölder inequality with exponents $\frac{2}{p}$
and $\frac{2}{2-p}$ and inequality (2.1), we get

$\displaystyle\int_{\\{|z|\leq s\\}}$
$\displaystyle|(R\circ\varphi)^{\prime}(z)|^{p}|\varphi^{\prime}(z)|(1-|z|)^{p-1}\,dA(z)$
$\displaystyle\leq\bigg{(}\int_{\\{|z|\leq
s\\}}|(R\circ\varphi)^{\prime}(z)|^{2}(1-|z|)\,dA(z)\bigg{)}^{\frac{p}{2}}\bigg{(}\int_{\\{|z|\leq
s\\}}\frac{|\varphi^{\prime}(z)|^{\frac{2}{2-p}}}{1-|z|}\,dA(z)\bigg{)}^{1-\frac{p}{2}}$
$\displaystyle\lesssim
M^{p}\bigg{(}\int_{0}^{s}\bigg{(}\int_{0}^{2\pi}|\varphi^{\prime}(re^{it})|^{\frac{2}{2-p}}dt\bigg{)}\frac{rdr}{1-r}\bigg{)}^{1-\frac{p}{2}}\lesssim\big{(}\log
n\big{)}^{1-\frac{p}{2}}M^{p}.$

It remains to estimate the integral over the set $\\{s<|z|<1\\}$:

$\int_{\\{s<|z|<1\\}}|(R\circ\varphi)^{\prime}(z)|^{p}|\varphi^{\prime}(z)|(1-|z|)^{p-1}\,dA(z)$
$\leq
M^{p-1}\int_{\\{s<|z|<1\\}}|(R\circ\varphi)^{\prime}(z)|\cdot|\varphi^{\prime}(z)|\,dA(z)$
$\lesssim
M^{p-1}\bigg{(}\int_{\\{s<|z|<1\\}}|(R\circ\varphi)^{\prime}(z)|^{2}dA(z)\bigg{)}^{1/2}\bigg{(}\int_{\\{s<|z|<1\\}}|\varphi^{\prime}(z)|^{2}dA(z)\bigg{)}^{1/2}\lesssim
M^{p}.$

In the last inequality we used the fact that

$\int_{\\{s<|z|<1\\}}|(R\circ\varphi)^{\prime}(z)|^{2}dA(z)\lesssim nM^{2},$

since $R\circ\varphi$ covers the disk of radius $M$ with multiplicity at most
$n$, as well as the inclusion $\varphi^{\prime}\in H^{2}$.

The case $p\geq 2$ is trivial:

$I_{p,p-1}(R)\lesssim
M^{p-2}\int_{\mathbb{D}}|(R\circ\varphi)^{\prime}(z)|^{2}(1-|z|)\,dA(z)\lesssim
M^{p},$

i.e., the quantity $I_{p,p-1}(R)$ is uniformly bounded over $R$ and $n$.

Theorem 5 is proved.

## References

  * [1] S. N. Mergelyan, “On an integral connected with analytic functions”, Izvestiya Akad. Nauk SSSR. Ser. Mat., 15 (1951), no. 5, 395–400 (Russian).
  * [2] W. Rudin, “The radial variation of analytic functions”, Duke Math. J., 22 (1955), no. 2, 235–242.
  * [3] G. Piranian, “Bounded functions with large circular variation”, Proc. Amer. Math. Soc., 19 (1968), no. 6, 1255–1257.
  * [4] N. G. Makarov, “Probability methods in the theory of conformal mappings”, Algebra i Analiz, 1 (1989), no. 1, 3–59; English transl.: Leningrad Math. J., 1 (1990), no. 1, 1–56.
  * [5] R. Bañuelos, C. N. Moore, “Mean growth of Bloch functions and Makarov’s law of the iterated logarithm”, Proc. Amer. Math. Soc., 112 (1991), 851–854.
  * [6] A. Aleman, D. Vukotić, “On Blaschke products with derivatives in Bergman spaces with normal weights”, J. Math. Anal. Appl., 361 (2010), no. 2, 492–505.
  * [7] D. Protas, “Blaschke products with derivative in function spaces”, Kodai Math. J., 34 (2011), no. 1, 124–131.
  * [8] D. Protas, “Derivatives of Blaschke products and model space functions”, Canad. Math. Bull., 63 (2020), no. 4, 716–725.
  * [9] E. P. Dolzhenko, “Rational approximations and boundary properties of analytic functions”, Mat. Sb. (N.S.), 69(111) (1966), no. 4, 497–524 (Russian).
  * [10] V. V. Peller, “Hankel operators of class $\mathfrak{S}_{p}$ and their applications (rational approximation, Gaussian processes, the problem of majorizing operators)”, Mat. Sb. (N.S.), 113(155) (1980), no. 4(12), 538–581; English transl.: Math. USSR-Sb., 41 (1982), no. 4, 443–479.
  * [11] S. Semmes, “Trace ideal criteria for Hankel operators, and applications to Besov spaces”, Integr. Equat. Oper. Theory, 7 (1984), no. 2, 241–281.
  * [12] A. A. Pekarskii, “Inequalities of Bernstein type for derivatives of rational functions, and inverse theorems of rational approximation”, Mat. Sb., 24(166):4(8) (1984), 571–588; English transl.: Math. USSR-Sb., 52 (1985), no. 2, 557–574.
  * [13] A. A. Pekarskii, “New proof of the Semmes inequality for the derivative of the rational function”, Mat. Zametki, 72 (2002), no. 2, 258–264; English transl.: Math. Notes, 72 (2002), no. 2, 230–236.
  * [14] V. I. Danchenko, “An integral estimate for the derivative of a rational function”, Izvestiya Russ. Akad. Nauk SSSR. Ser. Mat., 43 (1979), no. 2, 277–293; English transl.: Math. USSR-Izv., 14 (1980), no. 2, 257–273.
  * [15] V. I. Danchenko, “Several integral estimates of the derivatives of rational functions on sets of finite density”, Mat. Sb., 187 (1996), no. 10, 33–52; English transl.: Sb. Math., 187 (1996), no. 10, 1443–1463.
  * [16] E. M. Dyn’kin, “Inequalities for rational functions”, J. Approx. Theory, 91 (1997), 349–367.
  * [17] E. M. Dyn’kin, “Rational functions in Bergman spaces”. In: V. P. Havin, N. K. Nikolski (eds.) Complex Analysis, Operators, and Related Topics, Operator Theory: Advances and Applications, vol. 113, pp. 77–94. Birkhäuser, Basel, 2000.
  * [18] A. Baranov, R. Zarouf, “A Bernstein-type inequality for rational functions in weighted Bergman spaces”, Bull. Sci. Math., 137 (2013), no. 4, 541–556.
  * [19] A. Baranov, R. Zarouf, “The differentiation operator from model spaces to Bergman spaces and Peller type inequalities”, J. Anal. Math., 137 (2019), no. 1, 189–209.
  * [20] A. Baranov, R. Zarouf, “$H^{\infty}$ interpolation and embedding theorems for rational functions”, Integr. Equat. Oper. Theory, 91 (2019), article number 18.
  * [21] O. Martio, J. Sarvas, “Injectivity theorems in plane and space”, Ann. Acad. Sci. Fenn. Ser. A I Math., 4 (1979), no. 2, 383–401.
  * [22] Ch. Pommerenke, Boundary Behaviour of Conformal Maps, Springer-Verlag New-York, 1992.