# TransCrowd: Weakly-Supervised Crowd Counting with Transformer

Dingkang Liang1, Xiwu Chen1, Wei Xu2, Yu Zhou1, Xiang Bai1
1Huazhong University of Science and Technology
2Beijing University of Posts and Telecommunications

###### Abstract

The mainstream crowd counting methods usually utilize the convolution neural
network (CNN) to regress a density map, requiring point-level annotations.
However, annotating each person with a point is an expensive and laborious
process. During the testing phase, the point-level annotations are not
considered to evaluate the counting accuracy, which means the point-level
annotations are redundant. Hence, it is desirable to develop weakly-supervised
counting methods that just rely on count-level annotations, a more economical
way of labeling. Current weakly-supervised counting methods adopt the CNN to
regress a total count of the crowd by an image-to-count paradigm. However,
having limited receptive ﬁelds for context modeling is an intrinsic limitation
of these weakly-supervised CNN-based methods. These methods thus can not
achieve satisfactory performance, with limited applications in the real-word.
The Transformer is a popular sequence-to-sequence prediction model in NLP,
which contains a global receptive field. In this paper, we propose TransCrowd,
which reformulates the weakly-supervised crowd counting problem from the
perspective of sequence-to-count based on Transformer. We observe that the
proposed TransCrowd can effectively extract the semantic crowd information by
using the self-attention mechanism of Transformer. To the best of our
knowledge, this is the first work to adopt a pure Transformer for crowd
counting research. Experiments on five benchmark datasets demonstrate that the
proposed TransCrowd achieves superior performance compared with all the
weakly-supervised CNN-based counting methods and gains highly competitive
counting performance compared with some popular fully-supervised counting
methods. An implementation of our method is available at
https://github.com/dk-liang/TransCrowd.

##  1 Introduction

Crowd counting is a hot topic in the computer vision community, which plays an
essential role in video surveillance, public safety, and crowd analysis.
Typical crowd counting methods [66, 21, 58, 33, 33] usually utilize the
convolution neural network (CNN) to regress a density map, which has achieved
significant progress recently. A standard regressor consists of an encoder and
decoder: the encoder extracts the high-level feature information, and the
decoder is designed for pixel-level regression based on the extracted feature.

Figure 1: (a) Traditional fully-supervised CNN-based method. All the training
images are labeled with point-level annotations. (b) Weakly-supervised CNN-
based method from image-to-count perspective, only relying on the annotated
total count of the crowd. (c) The proposed TransCrowd, a weakly-supervised
method, reformulates the counting problem from the sequence-to-count
perspective.

However, these density-map regression-based methods [66, 21, 58, 2] still have
some drawbacks. 1) They apply the point-level annotations to generate ground
truth density maps, which are usually expensive cost. Actually, some methods
[62, 11, 40, 20] discover that we can collect a new crowd dataset by using a
more economical strategy, such as mobile crowd-sensing [11] technology or GPS-
less [40] energy-efficient sensing scheduling. For a given crowd scene with
different viewpoints and the total count keeps the same (such as auditoria,
classroom), if we know the total count of one viewpoint, then the total count
of other viewpoints is known. Besides, we can obtain the crowd number at a
glance for some sparse crowd scenes. (2) The annotated point label will not be
taken to evaluating the counting performance, meaning the point label is
redundant to some extent. Thus, point-level annotations are not absolutely
necessary for the crowd counting task.

Based on the above observations, it is desirable to develop the count-level
crowd counting method. Following previous works [20, 62], we call the methods
which rely on the point-level annotations are fully-supervised paradigm, and
the methods which only rely on count-level are weakly-supervised paradigm. The
fully-supervised methods first utilize the point annotation to generate the
ground truth density map and then elaborately design a regressor to generate a
prediction density map and finally apply the $L2$ loss to measure the
difference between the prediction and the ground truth, as shown in Fig. 1(a).
The existing weakly-supervised methods usually regress the total count of
crowd image directly, which is from the image-to-count perspective, as shown
in Fig. 1(b).

Recently, Transformer [48], a popular language model proposed by Google, has
been explored in many vision tasks, such as classification [19], detection [4,
68], and segmentation [67]. Unlike the CNN, which utilizes a limited receptive
field, the Transformer [48] provides the global receptive field, showing
excellent advantages over pure CNN architectures. In this paper, we propose
TransCrowd, which is the first to explore the Transformer into the weakly-
supervised crowd counting task, establishing the perspective of sequence-to-
count prediction, as illustrated in Fig. 1(c).

Only a few methods are proposed with the considerations of reducing the
annotations burden (e.g., semi, weakly-supervised). L2R [30] facilitates the
counting task by ranking the image patch. Wang et al. [56] introduce a
synthetic crowd dataset named GCC, and the model is pre-trained on the GCC
dataset and then fine-tuned on real data. One of the most relevant works for
our method is [62], which proposes a soft-label network to facilitate the
counting task, directly regressing the crowd number without the supervision of
location labels. However, [62] is a CNN-based method, which has a limited
respective field. The Transformer has a global receptive field, which
effectively solves the limited respective field problem of CNN-based methods
once and for all. It means that the Transformer architecture is more suitable
for the weakly-supervised counting task since the task aims to directly
predict a total count from the whole image and rely on the global perspective.

In this paper, we introduce two types of TransCrowd, named TransCrowd-Token
and TransCrowd-GAP, respectively. TransCrowd-Token utilizes an extra learnable
token to represent the count. TransCrowd-GAP adopts the global average pooling
(GAP) over all items in the output sequence of Transformer-encoder to obtain
the pooled visual tokens. The regression tokens or pooled visual tokens are
then fed into the regression head to generate the prediction count. We
empirically find that the TransCrowd-GAP can obtain more reasonable attention
weight, achieve higher count accuracy, and present fast-converging compared
with TransCrowd-Token.

In summary, this work contributes to the following:

  1. 1.

TransCrowd is the first pure transformer-based crowd counting framework. We
reformulate the counting problem from a sequence-to-count perspective and
propose a weakly-supervised counting method, which only utilizes the count-
level annotations without the point-level information in the training phase.

  2. 2.

We provide two different types of TransCrowd, named TransCrowd-Token and
TransCrowd-GAP, respectively. We observe that the TransCrowd-GAP can generate
a more reasonable attention weight and reports faster converging and higher
counting performance than TransCrowd-Token.

  3. 3.

Extensive experiments demonstrate that the proposed method achieves state-of-
the-art counting performance compared with the weakly-supervised methods.
Additionally, our method has a highly competitive counting performance
compared with the fully-supervised counting methods.

##  2 Related Works

###  2.1 CNN-based crowd counting.

The CNN-based crowd counting methods can be categorized into localization-
based methods and regression-based methods. The localization-based methods
[38, 28] usually learn to predict bounding boxes for each human, relying on
box-level annotations. Recently, some methods [1, 32, 22, 57, 7] try to
utilize the pseudo bounding boxes based on point-level annotations or design a
suitable map to realize counting and localization tasks. However, these
localization-based methods usually report unsatisfactory counting performance.
The mainstream of crowd counting is the density map CNN-based crowd counting
methods [66, 21, 63, 8, 34, 16, 59], whose integral of the density map gives
the total count of a crowd image. Due to the commonly heavy occlusion that
existed in crowd images, multi-scale architecture is developed. Specifically,
MCNN [66] utilizes multi-size filters to extract different scale feature
information. Sam et al. [46] capture the multi-scale information by the
proposed contextual pyramid CNN. TEDNet [15] assembles multiple encoding-
decoding paths hierarchically to generate a high-quality density map for
accurate crowd counting. Method in [35] proposes a scale-aware probabilistic
model to handle large scale variations through DPN, and each level of DPN
copes with a given scale range. Using the perspective information to diminish
the scale variations is effective [41, 10, 61]. PACNN [41] proposes a novel
generating ground truth perspective maps strategy and predicts both the
perspective maps and density maps at the testing phase. Yang et al. [61]
propose a reverse perspective network to estimate the perspective factor of
the input image and then warp the image. Appropriate measure matching can help
to improve the counting performance. S3 [23] propose a novel measure matching
based on Sinkhorn divergence, avoiding generating the density maps. UOT [36]
use unbalanced optimal transport (UOT) distance to quantify the discrepancy
between two measures, outputting sharper density maps.

The Attention-based mechanism is another useful technique adopted by many
methods [27, 16, 63]. ADCrowdNet [27] generates an attention map for the crowd
images via a network called Attention Map Generate (AMG). Jiang et al. [16]
propose a density attention network to generate attention masks concerning
regions of different density levels. Zhang et al. [63] propose a Relation
Attention Network (RANet) that utilizes local self-attention and global self-
attention to capture long-range dependencies. It is noteworthy that RANet [63]
is actually a non-local/self-attention mechanism based on CNN instead of pure
Transformers, and we utilize a pure Transformer without convolution layers.

Figure 2: (a) The pipeline of TransCrowd. The input image is split into fixed-
size patches, each of which is linearly embedded with position embeddings.
Then, the feature embedding sequence is fed into the Transformer-encoder,
followed by a regression head to generate the prediction count. (b) We utilize
an extra token to represent the crowd count, similar to the class token in
Bert [17] and ViT [19]. (c) A global average pooling is adopted to pool the
output visual tokens of the Transformer-encoder.

###  2.2 Weakly-supervised crowd counting.

Only a few methods focus on counting with a lack of labeled data. L2R [30]
proposes a learning-to-rank framework based on an extra collected crowd
dataset. Wang et al. [56] introduce a synthetic crowd scene for the pre-
trained model. However, these two methods still rely on point-level
annotations, which are fully-supervised instead of weakly-supervised
paradigms.

The traditional method [5] relies on hand-crafted features, such as GLCM and
edge orientations, which are turned to be sub-optimal for this weakly-
supervised counting task. MATT [20] learns a model from a small amount of
point-level annotations (fully-supervised) and a large amount of count-level
annotations (weakly-supervised). The method in [49] proposes a weakly-
supervised solution based on the Gaussian process for crowd density
estimation. Shang et al. simultaneous predicted the global count and local
count. Wang et al. directly regress the global count, and some negative
samples are fed into the network to boost the robustness. Similarly, Yang et
al. [62] also directly maps the images to the crowd numbers without the
location supervision based on the proposed soft-label sorting network.

However, the counting performance of these count-level weakly-supervised
counting methods still does not achieve comparable results to the fully-
supervised counting methods, existing massive degradation, limiting the
application of weakly-supervised methods in real-world. Different from the
previous works, the proposed TransCrowd utilizes the Transformer architecture
to directly regress the crowd number, which formulates the counting problem as
the sequence-to-count paradigm and achieves comparable counting performance
compared with the popular fully-supervised methods.

###  2.3 Visual Transformer.

The Transformers [48], dominating the natural language modeling [17, 31],
utilize the self-attention mechanism to capture the global dependencies
between input and output. Recently, many works [4, 19, 67, 54, 6] attempt to
apply the Transformer into the vision task. Specifically, DETR [4] firstly
utilizes a CNN backbone to extract the visual features, followed by the
Transformer blocks for the box regression and classification. ViT [19] is the
first which directly applies Transformer-encoder [48] to sequences of images
patch to realize classification task. SETR [67] regards semantic segmentation
from a sequence-to-sequence perspective with Transformers. IPT [6] develops a
pre-trained model for image processing (low-level task) using the transformer
architecture.

To the best of our knowledge, we are the first to explore the pure Transformer
[48] to the counting task.

##  3 Our Method

The overview of our method consists of the sequence (tokens) of the image, a
Transformer-encoder, and a naive regression head, as shown in Fig. 2(a).
Specifically, the input image is first transformed into fixed-size patches and
then flatten to a sequence of vectors. The sequence is feed into the
Transformer-encoder, followed by a naive regression head to generate the
prediction count.

###  3.1 Image to sequence

In general, the Transformer adopts a 1$D$ sequence of feature embeddings
$Z\in\mathbb{R}^{N\times D}$ as input, where $N$ is the length of the sequence
and the $D$ means the input channel size. Thus, the first step of TransCrowd
is to transform the input image $I$ into a sequence of 2D flattened patches.
Specifically, given an RGB image 111$H$, $W$, 3 indicate the spatial height,
width and channel number, respectively. $I\in\mathbb{R}^{H\times W\times 3}$ ,
we reshape the $I$ into a grid of $N$ patches, resulting in
{$x^{i}_{p}\in\mathbb{R}^{K^{2}\cdot 3}|i=1,...,N$}, where
$N=\frac{H}{K}\times\frac{W}{K}$ and $K$ is the pre-defined patch size.

###  3.2 Patch Embedding

Next, we need to map the $x$ into a latent $D$-dimensional embedding feature
by a learnable projection, since the transformer uses constant latent vector
size D through all of its layers, defined as

$e=[e_{1};e_{2}\cdots;e_{N}]=[x^{1}_{p}E;x^{2}_{p}E;\cdots;x^{N}_{p}E],\qquad
E\in\mathbb{R}^{(K^{2}\cdot 3)\times D},$ (1)

where $E$ is a learnable matrix, and $e\in\mathbb{R}^{N\times D}$ is the
mapped features. Thus, we add a specific position embedding
{$p_{i}\in\mathbb{R}^{D}|i=1,...,N$} into the $e$, maintaining position
information, defined as:

$Z_{0}=[Z^{1}_{0};Z^{2}_{0};\cdots;Z^{N}_{0}]=[e_{1}+p_{1};e_{2}+p_{2};\cdots;e_{N}+p_{N}],$
(2)

where $Z_{0}$ is the input of the first transformer layer.

Figure 3: A standard Transformer layer consists of Multi-Head Attention and
MLP blocks. Meanwhile, the layer normalization (LN) and residual connections
are employed.

###  3.3 Transformer-encoder

We only adopt the Transformer encoder [48], without the decoder, similar to
ViT [19]. Specifically, the encoder contains $L$ layers of Multi-head self-
attention ($MSA$) and Multilayer Perceptron ($MLP$) blocks. For each layer
$l$, layer normalization (LN) and residual connections are employed. A stand
transformer layer is shown in Fig. 3, and the output can be written as
follows:

$Z^{\prime}_{l}=MSA(LN(Z_{l-1}))+Z_{l-1},$ (3)

$Z_{l}=MLP(LN(Z^{\prime}_{l}))+Z^{\prime}_{l},$ (4)

where $Z_{l}$ is the output of layer $l$. Here, the $MLP$ contains two linear
layers with a GELU [12] activation function. In particular, the first linear
layer of MLP expands the feature embedding’ dimension from $D$ to 4$D$, while
the second layer shrinks the dimension from 4$D$ to $D$.

$MSA$ is an extension with $m$ independent self-attention ($SA$) modules:
$MSA(Z_{l-1})=[SA_{1}(Z_{l-1});$
$SA_{2}(Z_{l-1});\cdots;SA_{m}(Z_{l-1})]W_{O}$, where
$W_{O}\in\mathbb{R}^{D\times D}$ is a re-projection matrix. At each
independent $SA$, the input consists of query (Q), key (K), and value (V),
which are computed from $Z^{l-1}$:

$Q=Z_{l-1}W_{Q},\quad K=Z_{l-1}W_{K},\quad V=Z_{l-1}W_{V},$ (5)

$SA(Z_{l})=softmax(\frac{QK^{T}}{\sqrt{D}})V$ (6)

where $W_{Q}$/$W_{K}$/$W_{V}$ $\in\mathbb{R}^{D\times\frac{D}{m}}$ are three
learnable matrices. The $softmax$ function is applied over each row of the
input matrix and $\sqrt{D}$ provides appropriate normalization.

###  3.4 The input of regression head

We introduce two different inputs for the regression heads to evaluate the
effectiveness of TransCrowd. The goal of the regression head is to generate
the prediction count instead of the density map. We briefly describe the two
types of input.

(1) Regression Token. Similar to the class token in Bert [17] and ViT [19], we
prepend a learnable embedding named regression token to the input sequence
$Z_{0}$, as shown in Fig. 2(b). This architecture forces the self-attention to
spread information between the patch tokens and the regression token, making
the regression token contain overall semantic crowd information. The
regression head is implemented by MLP containing two linear layers. We refer
to the TransCrowd with the extra regression token as TransCrowd-Token.

(2) Global average pooling. We apply the global average pooling (GAP) to
shrink the sequence length, as shown in Fig. 2(c). Similar to TransCrowd-
Token, two linear layers are used for the regression head. We refer to the
TransCrowd with global average pooling as TransCrowd-GAP. The global average
pooling can effectively maintain the useful semantic crowd information in
patch tokens. We find that using pooled visual tokens will generate richer
discriminative semantic crowd patterns and achieve better counting performance
than using the extra regression token, the detailed discussion listed in Sec.
6.

We utilize $L_{1}$ loss to measure the difference between prediction and
ground truth:

$L_{1}=\frac{1}{M}\sum_{i=1}^{M}\left|P_{i}-G_{i}\right|,$ (7)

where $P_{i}$ and $G_{i}$ are the prediction crowd number and the
corresponding ground truth of $i$-th image, respectively. $M$ is the batch
size of training images.

##  4 Experiments

###  4.1 Implementation details

The Transformer-encoder is similar to ViT [19], which contains 12 transformer
layers, and each $MSA$ consists of 12 $SA$. We utilize the fixed $H$ and $W$,
both of which are set as 384. We set $K$ as 16, which means $N$ is equal to
576. We use Adam [18] to optimize our model, in which the learning rate and
weight decay are set to 1e-5 and 1e-4, respectively. The weights pre-trained
on ImageNet are used to initialize the Transformer-encoder. During training,
the widely adopted data augmentation strategies are utilized, including random
horizontal flipping and grayscaling. Due to some datasets having various
resolution images, we resize all the images into the size of 1152 $\times$
768\. Each resized image can be regarded as six independent sub-image, and the
resolution of each sub-image is 384 $\times$ 384\. We set the batch size as 24
and use a V100 GPU for the experiments. Code and models will be released for
the community to follow.

###  4.2 Dataset

NWPU-Crowd [55], a large-scale and challenging dataset, consists of 5,109
images, 2,133,375 instances annotated elaborately. To be specific, the images
are randomly split into three parts, including training, validation, and
testing sets, which contain 3,109, 500, and 1,500 images, respectively.

JHU-CROWD++ [44] contains 2,722 training images, 500 validation images, and
1,600 testing images, collected from diverse scenarios. The total number of
people in each image ranges from 0 to 25,791.

UCF-QNRF [14] contains 1,535 images captured from unconstrained crowd scenes
with about one million annotations. It has a count range of 49 to 12,865, with
an average count of 815.4. Specifically, the training set consists of 1,201
images, and the testing set consists of 334 images.

Method | Year | Training label | UCF-QNRF | Part A | Part B
---|---|---|---|---|---
Location | Crowd number | MAE | MSE | MAE | MSE | MAE | MSE
MCNN [66] | CVPR 16 | $\surd$ | $\surd$ | 277.0 | 426.0 | 110.2 | 173.2 | 26.4 | 41.3
CL [14] | ECCV 18 | $\surd$ | $\surd$ | 132.0 | 191.0 | - | - | - | -
CSRNet [21] | CVPR 18 | $\surd$ | $\surd$ | - | - | 68.2 | 115.0 | 10.6 | 16.0
L2R [30] | TPAMI 19 | $\surd$ | $\surd$ | 124.0 | 196.0 | 73.6 | 112.0 | 13.7 | 21.4
CFF [42] | ICCV 19 | $\surd$ | $\surd$ | - | - | 65.2 | 109.4 | 7.2 | 12.2
PGCNet [60] | ICCV19 | $\surd$ | $\surd$ | - | - | 57.0 | 86.0 | 8.8 | 13.7
TEDnet [15] | CVPR 19 | $\surd$ | $\surd$ | 113.0 | 188.0 | 64.2 | 109.1 | 8.2 | 12.8
BL [34] | ICCV 19 | $\surd$ | $\surd$ | 88.7 | 154.8 | 62.8 | 101.8 | 7.7 | 12.7
ASNet [16] | CVPR 20 | $\surd$ | $\surd$ | 91.5 | 159.7 | 57.7 | 90.1 | - | -
LibraNet [25] | ECCV20 | $\surd$ | $\surd$ | 88.1 | 143.7 | 55.9 | 97.1 | 7.3 | 11.3
NoisyCC [50] | NeurIPS 20 | $\surd$ | $\surd$ | 85.8 | 150.6 | 61.9 | 99.6 | 7.4 | 11.3
DM-Count [50] | NeurIPS 20 | $\surd$ | $\surd$ | 85.6 | 148.3 | 59.7 | 95.7 | 7.4 | 11.8
Method in [35] | MM 20 | $\surd$ | $\surd$ | 84.7 | 147.2 | 58.1 | 91.7 | 6.5 | 10.1
S3 [23] | IJCAI 21 | $\surd$ | $\surd$ | 80.6 | 139.8 | 57.0 | 96.0 | 6.3 | 10.6
UOT [36] | AAAI 21 | $\surd$ | $\surd$ | 83.3 | 142.3 | 58.1 | 95.9 | 6.5 | 10.2
Method in [62]* | ECCV 20 | $-$ | $\surd$ | - | - | 104.6 | 145.2 | 12.3 | 21.2
MATT [20]* | PR 21 | $-$ | $\surd$ | - | - | 80.1 | 129.4 | 11.7 | 17.5
TransCrowd-Token (ours)* | - | $-$ | $\surd$ | 98.9 | 176.1 | 69.0 | 116.5 | 10.6 | 19.7
TransCrowd-GAP (ours)* | - | $-$ | $\surd$ | 97.2 | 168.5 | 66.1 | 105.1 | 9.3 | 16.1

Table 1: Quantitative comparison (in terms of MAE and MSE) of the proposed
method and some popular methods on three widely adopted benchmark datasets. *
represents the weakly-supervised method.

Method | Year | Training label | Val set
---|---|---|---
Low | Medium | High | Overall
Location | Crowd number | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE
MCNN [66] | CVPR16 | $\surd$ | $\surd$ | 90.6 | 202.9 | 125.3 | 259.5 | 494.9 | 856.0 | 160.6 | 377.7
CMTL [45] | AVSS17 | $\surd$ | $\surd$ | 50.2 | 129.2 | 88.1 | 170.7 | 583.1 | 986.5 | 138.1 | 379.5
DSSI-Net [26] | ICCV19 | $\surd$ | $\surd$ | 50.3 | 85.9 | 82.4 | 164.5 | 436.6 | 814.0 | 116.6 | 317.4
CAN [29] | CVPR19 | $\surd$ | $\surd$ | 34.2 | 69.5 | 65.6 | 115.3 | 336.4 | 619.7 | 89.5 | 239.3
SANet [3] | ECCV18 | $\surd$ | $\surd$ | 13.6 | 26.8 | 50.4 | 78.0 | 397.8 | 749.2 | 82.1 | 272.6
CSRNet [21] | CVPR18 | $\surd$ | $\surd$ | 22.2 | 40.0 | 49.0 | 99.5 | 302.5 | 669.5 | 72.2 | 249.9
CG-DRCN [44] | PAMI20 | $\surd$ | $\surd$ | 17.1 | 44.7 | 40.8 | 71.2 | 317.4 | 719.8 | 67.9 | 262.1
MBTTBF [47] | ICCV19 | $\surd$ | $\surd$ | 23.3 | 48.5 | 53.2 | 119.9 | 294.5 | 674.5 | 73.8 | 256.8
SFCN [56] | CVPR19 | $\surd$ | $\surd$ | 11.8 | 19.8 | 39.3 | 73.4 | 297.3 | 679.4 | 62.9 | 247.5
BL [34] | ICCV19 | $\surd$ | $\surd$ | 6.9 | 10.3 | 39.7 | 85.2 | 279.8 | 620.4 | 59.3 | 229.2
TransCrowd-Token (ours)* | - | - | $\surd$ | 7.1 | 10.7 | 33.3 | 54.6 | 302.5 | 557.4 | 58.4 | 201.1
TransCrowd-GAP (ours)* | - | - | $\surd$ | 6.7 | 9.5 | 34.5 | 55.8 | 285.9 | 532.8 | 56.8 | 193.6

Table 2: Quantitative results on the JHU-Crowd++ (val set) dataset. ”Low”,
”Medium” and ”High” respectively indicates three categories based on different
ranges:[0,50], (50,500], and >500\. * represents the weakly-supervised crowd
counting methods.

ShanghaiTech [66] contains 1,198 crowd images with 330,165 annotations. The
images of the dataset are divided into two parts: Part A and Part B. In
particular, Part A contains 300 training images and 182 testing images, and
Part B consists of 400 training images and 316 testing images.

UCF_CC_50 [13] is a small dataset for dense crowd counting, which just
contains 50 images with an average of 1,280 individuals per image. The images
are captured in a diverse set of events, and the pedestrians count of each
image range between 94 and 4,543.

WorldExpo’10 [64] contains 1,132 surveillance videos from 108 cameras. The
training set consists of 3,380 images captured from 103 different scenes, and
the testing set contains 600 images from 5 scenes. There are 199,923
annotations labelled in the whole 3,980 images.

###  4.3 Evaluation Metrics

We choose Mean Absolute Error (MAE) and Mean Squared Error (MSE) to evaluate
the counting performance:

$MAE=\frac{1}{N}\sum_{i=1}^{N}\left|P_{i}-G_{i}\right|,MSE=\sqrt{\frac{1}{N}\sum_{i=1}^{N}\left|P_{i}-G_{i}\right|^{2}},$
(8)

where $N$ is the number of testing images, $P_{i}$ and $G_{i}$ are the
predicted and ground truth count of the $i$-th image, respectively.

Method | Year | Training label | Testing set
---|---|---|---
Low | Medium | High | Overall
Location | Crowd number | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE
MCNN [66] | CVPR16 | $\surd$ | $\surd$ | 97.1 | 192.3 | 121.4 | 191.3 | 618.6 | 1,166.7 | 188.9 | 483.4
CMTL [45] | AVSS17 | $\surd$ | $\surd$ | 58.5 | 136.4 | 81.7 | 144.7 | 635.3 | 1,225.3 | 157.8 | 490.4
DSSI-Net [26] | ICCV19 | $\surd$ | $\surd$ | 53.6 | 112.8 | 70.3 | 108.6 | 525.5 | 1,047.4 | 133.5 | 416.5
CAN [29] | CVPR19 | $\surd$ | $\surd$ | 37.6 | 78.8 | 56.4 | 86.2 | 384.2 | 789.0 | 100.1 | 314.0
SANet [3] | ECCV18 | $\surd$ | $\surd$ | 17.3 | 37.9 | 46.8 | 69.1 | 397.9 | 817.7 | 91.1 | 320.4
CSRNet [21] | CVPR18 | $\surd$ | $\surd$ | 27.1 | 64.9 | 43.9 | 71.2 | 356.2 | 784.4 | 85.9 | 309.2
CG-DRCN [44] | PAMI20 | $\surd$ | $\surd$ | 19.5 | 58.7 | 38.4 | 62.7 | 367.3 | 837.5 | 82.3 | 328.0
MBTTBF [47] | ICCV19 | $\surd$ | $\surd$ | 19.2 | 58.8 | 41.6 | 66.0 | 352.2 | 760.4 | 81.8 | 299.1
SFCN [56] | CVPR19 | $\surd$ | $\surd$ | 16.5 | 55.7 | 38.1 | 59.8 | 341.8 | 758.8 | 77.5 | 297.6
BL [34] | ICCV19 | $\surd$ | $\surd$ | 10.1 | 32.7 | 34.2 | 54.5 | 352.0 | 768.7 | 75.0 | 299.9
UOT [36] | AAAI21 | $\surd$ | $\surd$ | 11.2 | 26.2 | 28.7 | 45.3 | 274.1 | 648.2 | 60.5 | 252.7
S3 [23] | IJCAI21 | $\surd$ | $\surd$ | - | - | - | - | - | - | 59.4 | 244.0
TransCrowd-Token (ours)* | - | - | $\surd$ | 8.5 | 23.2 | 33.3 | 71.5 | 368.3 | 816.4 | 76.4 | 319.8
TransCrowd-GAP (ours)* | - | - | $\surd$ | 7.6 | 16.7 | 34.8 | 73.6 | 354.8 | 752.8 | 74.9 | 295.6

Table 3: Quantitative results on the JHU-Crowd++ (testing set) dataset. ”Low”,
”Medium” and ”High” respectively indicates three categories based on different
ranges:[0,50], (50,500], and >500\. * represents the weakly-supervised crowd
counting methods.

Method | Year | Training label | Val set | Testing set
---|---|---|---|---
Overall | Overall | Scene Level (only MAE)
Location | Crowd number | MAE | MSE | MAE | MSE | Avg. | $S0\sim S4$
C3F-VGG [9] | Tech19 | $\surd$ | $\surd$ | 105.79 | 504.39 | 127.0 | 439.6 | 666.9 | 140.9/26.5/58.0/307.1/2801.8
CSRNet [21] | CVPR18 | $\surd$ | $\surd$ | 104.89 | 433.48 | 121.3 | 387.8 | 522.7 | 176.0/35.8/59.8/285.8/2055.8
PCC-Net-VGG [10] | CVPR19 | $\surd$ | $\surd$ | 100.77 | 573.19 | 112.3 | 457.0 | 777.6 | 103.9/13.7/42.0/259.5/3469.1
CAN [29] | CVPR19 | $\surd$ | $\surd$ | 93.58 | 489.90 | 106.3 | 386.5 | 612.2 | 82.6/14.7/46.6/269.7/2647.0
SFCN† [56] | CVPR19 | $\surd$ | $\surd$ | 95.46 | 608.32 | 105.7 | 424.1 | 712.7 | 54.2/14.8/44.4/249.6/3200.5
BL [34] | ICCV19 | $\surd$ | $\surd$ | 93.64 | 470.38 | 105.4 | 454.2 | 750.5 | 66.5/8.7/41.2/249.9/3386.4
KDMG [52] | PAMI20 | $\surd$ | $\surd$ | - | - | 100.5 | 415.5 | 632.7 | 77.3/10.3/38.5/259.4/2777.9
NoisyCC [50] | NeurIPS20 | $\surd$ | $\surd$ | - | - | 96.9 | 534.2 | 608.1 | 218.7/10.7/35.2/203.2/2572.8
DM-Count [53] | NeurIPS20 | $\surd$ | $\surd$ | 70.5 | 357.6 | 88.4 | 388.6 | 498.0 | 146.6/7.6/31.2/228.7/2075.8
S3 [23] | IJCAI21 | $\surd$ | $\surd$ | - | - | 87.8 | 387.5 | 566.5 | 80.7 / 7.9 / 36.3 / 212.0 / 2495.4
UOT [36] | AAAI21 | $\surd$ | $\surd$ | - | - | 83.5 | 346.9 | - | -
TransCrowd-Token (ours)* | - | $-$ | $\surd$ | 88.2 | 446.9 | 119.6 | 463.9 | 736.0 | 88.0/12.7/47.2/311.2/3216.1
TransCrowd-GAP (ours)* | - | $-$ | $\surd$ | 88.4 | 400.5 | 117.7 | 451.0 | 737.8 | 69.3/12.8/46.0/309.0/3252.2

Table 4: Comparison of the counting performance on the NWPU-Crowd. $S0\\!\sim\\!S4$ respectively indicate five categories according to the different number range: $0$, $(0,100]$, $(100,500]$, $(500,5000]$, $\textgreater 5000$. * represents the weakly-supervised crowd counting methods. Method | Training label | MAE | MSE
---|---|---|---
Location | Crowd number
MCNN [65] | $\surd$ | $\surd$ | 377.6 | 509.1
CSRNet [21] | $\surd$ | $\surd$ | 266.1 | 397.5
ADCrowdNet [27] | $\surd$ | $\surd$ | 257.1 | 363.5
MATT [20]* | $-$ | $\surd$ | 355.0 | 550.2
TransCrowd-Token (ours)* | $-$ | $\surd$ | 288.9 | 407.6
TransCrowd-GAP (ours)* | $-$ | $\surd$ | 272.2 | 395.3
Table 5: The performance comparison on the UCF_CC_50 dataset. * represents the weakly-supervised crowd counting methods. Method | Training label | MAE
---|---|---
Location | Crowd number | S1 | S2 | S3 | S4 | S5 | Ave
M-CNN [66] | $\surd$ | $\surd$ | 3.4 | 20.6 | 12.9 | 13.0 | 8.1 | 11.6
CP-CNN[46] | $\surd$ | $\surd$ | 2.9 | 14.7 | 10.5 | 10.4 | 5.8 | 8.8
Liu et al.[24] | $\surd$ | $\surd$ | 2.0 | 13.1 | 8.9 | 17.4 | 4.8 | 9.2
IC-CNN[37] | $\surd$ | $\surd$ | 17.0 | 12.3 | 9.2 | 8.1 | 4.7 | 10.3
CSR-Net[21] | $\surd$ | $\surd$ | 2.9 | 11.5 | 8.6 | 16.6 | 3.4 | 8.6
SA-Net[3] | $\surd$ | $\surd$ | 2.6 | 13.2 | 9.0 | 13.3 | 3.0 | 8.2
LSC-CNN[39] | $\surd$ | $\surd$ | 2.9 | 11.3 | 9.4 | 12.3 | 4.3 | 8.0
MATT[20]* | $-$ | $\surd$ | 3.8 | 13.1 | 10.4 | 15.9 | 5.3 | 9.7
TransCrowd-Token (ours)* | $-$ | $\surd$ | 2.3 | 14.2 | 9.9 | 14.0 | 4.3 | 8.9
TransCrowd-GAP (ours)* | $-$ | $\surd$ | 2.1 | 13.3 | 8.9 | 13.8 | 4.4 | 8.5
Table 6: Comparison results of different methods on 5 scenes in the
WorldExpo’10 dataset. * represents the weakly-supervised crowd counting
methods.

Method | Training label | MAE | MSE
---|---|---|---
Location | Crowd number
FCN-HA [65] | $\surd$ | $\surd$ | 4.21 | -
CSRNet [21] | $\surd$ | $\surd$ | 3.56 | -
ADCrowdNet [27] | $\surd$ | $\surd$ | 2.44 | -
TransCrowd-Token (ours)* | $-$ | $\surd$ | 3.28 | 4.80
TransCrowd-GAP (ours)* | $-$ | $\surd$ | 3.23 | 4.66

Table 7: The performance comparison on the Trancos dataset. * represents the
weakly-supervised crowd counting methods.

Figure 4: Examples of attention maps from TransCrowd-Token and TransCrowd-GAP.
TransCrowd-GAP generates more reasonable attention weights compared with
TransCrowd-Token.

##  5 Results

We conduct extensive experiments to demonstrate the effectiveness of the
proposed weakly-supervised crowd counting method on five popular benchmarks.
For each dataset, we divide the existing methods into fully-supervised methods
(based on point-level annotations) and weakly-supervised methods (based on
count-level annotations).

Compared with the weakly-supervised counting methods. Our method achieves
state-of-the-art counting performance on all the conducted datasets, as listed
in Tab. 1 \- Tab. 7. Specifically, on ShanghaiTech part A, our TransCrowd-GAP
improves 17.5% in MAE and 18.8% in MSE compared with MATT [20], improves 36.8%
in MAE and 27.6% in MSE compared with [62]. On ShanghaiTech part B, TransCrod-
GAP improves 20.5% in MAE and 8.0% in MSE compared with MATT [20], improves
24.4% in MAE and 24.1% in MSE compared with [62]. Besides, the proposed
TransCrowd-Token also achieves significant improvement compared with MATT [20]
and [62] in terms of MAE and MSE, and only the proposed methods report
counting performance close to the fully-supervised methods. Note that MATT
[20] still applies a small number of images, which contain point-level
annotations for training.

Compared with the fully-supervised counting methods. Although it is unfair to
compare the fully-supervised and weakly-supervised crowd counting methods, our
method still achieves highly competitive performance on the five counting
datasets, as shown in Tab. 1 \- Tab. 7. An impressive phenomenon is that the
proposed method even surpasses some popular fully-supervised methods. For
example, as shown in Tab. 3, our TransCrowd-GAP brings 11.0 MAE and 13.6 MSE
improvement compared with CSRNet [21] on the JHU-Crowd++ (testing set)
dataset. BL [34], a recent strong counting method, achieves 75.0 in MAE and
299.9 in MSE, one of the state-of-the-art methods on the JHU-Crowd++ (testing
set) dataset, while our TransCrowd-GAP improves 0.1 MAE and 4.3 MSE,
respectively. Besides, from the results on UCF-QNRF, ShanghaiTech, and NWPU-
Crowd datasets, we can also observe that our method achieve significant
improvement compared to some popular fully-supervised methods (e.g., MCNN
[66], CSRNet [21], L2R [30]). We think the reasons why the proposed method
outperforms some fully supervised methods in the NWPU-Crowd and JHU-Crowd++
may be two-fold. First, the Transformer is beneficial to capture the long-
range dependence, and these two datasets contain many large-scale persons. The
proposed TransCrowd can effectively learn the global crowd semantic feature
representation. However, some state-of-the-art methods (e.g., BL [34]) utilize
a fixed Gaussian kernel for these datasets, and the fixed Gaussian kernel can
not effectively cover the large scale variations. Second, Dosovitskiy et al.
[19] prove that the CNNs outperform Transformer on small datasets (despite
regularization optimization), but with the larger datasets, Transformer
overtakes. For instance, the NWPU-Crowd is a large dataset containing 5,190
images, which may help the Transformer better fit the dataset. These
impressive results further demonstrate the effectiveness of the proposed
method and indicate point-level annotations are not entirely necessary for the
counting task.

Method | Resolution | Parameters | Backbone | FPS
---|---|---|---|---
CSRNET [21] |  384 $\times$ 384 | 16.2 M | VGG16 | 21.67
BL [34] |  384 $\times$ 384 | 21.6 M | VGG19 | 45.66
TransCrowd-Token |  384 $\times$ 384 | 86.8 M | Transformer | 46.41
TransCrowd-GAP |  384 $\times$ 384 | 90.4 M | Transformer | 46.73

Table 8: Comparison with BL [34] and CSRNET [21] using the same input image
resolution on a Titan XP.

Method | Year | Training label | None | Pre-ImgNet | Pre-GCC
---|---|---|---|---|---
Location | Crowd number | MAE | MSE | MAE | MSE | MAE | MSE
CSRNet [21] | CVPR18 | $\surd$ | $\surd$ | 120.0 | 179.4 | 68.2 | 115.0 | 67.4 | 112.3
TransCrowd-Token (ours)* | - | $-$ | $\surd$ | 142.0 | 212.5 | 69.0 | 116.5 | 67.2 | 111.9
TransCrowd-GAP (ours)* | - | $-$ | $\surd$ | 139.9 | 231.0 | 66.1 | 105.1 | 63.8 | 102.3

Table 9: The fine-tuning CSRNet’s and TransCrowd-GAP’s results on ShanghaiTech
part A dataset by using three different pre-trained strategies.  Figure 5:
Convergence curves of CSRNet, TransCrowd-Token, and TransCrowd-GAP on
ShanghaiTech part A dataset. The proposed TransCrowd-GAP achieves the best
counting performance and is fast-converging.

Method | Year | Training label | Part B$\rightarrow$Part A | Part A$\rightarrow$Part B | QNRF$\rightarrow$Part A | QNRF$\rightarrow$Part B
---|---|---|---|---|---|---
Location | Crowd number | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE
MCNN [66] | CVPR16 | $\surd$ | $\surd$ | 221.4 | 357.8 | 85.2 | 142.3 | - | - | - | -
D-ConvNet [43] | ECCV18 | $\surd$ | $\surd$ | 140.4 | 226.1 | 49.1 | 99.2 | - | - | - | -
RRSP[51] | CVPR19 | $\surd$ | $\surd$ | - | - | 40.0 | 68.5 | - | - | - | -
BL [34] | ICCV19 | $\surd$ | $\surd$ | - | - | - | - | 69.8 | 123.8 | 15.3 | 26.5
TransCrowd-GAP (ours)* | - | $-$ | $\surd$ | 141.3 | 258.9 | 18.9 | 31.1 | 78.7 | 122.5 | 13.5 | 21.9

Table 10: Experimental results on the transferability of different methods
under cross-dataset evaluation.

##  6 Analysis

###  6.1 The input of regression head

We introduce two different inputs for the regression head. Specifically,
TransCrowd-Token utilizes an extra learnable regression token to perform
counting, similar to the class token in Bert [17] and ViT [19]. TransCrowd-GAP
utilizes global average pooling to obtain the pooled visual tokens for count
prediction. The result of TransCrowd-Token and TransCrowd-GAP are listed in
Tab. 1 \- Tab. 7. We find that the results of TransCrowd-GAP are better
TransCrowd-Token in all conducted datasets. For example, TransCrowd-GAP
outperforms TransCrowd-Token by 2.8 MAE and 11.4 MSE on the ShanghaiTech Part
A dataset, a significant improvement. TransCrowd-GAP also has steady
improvement on ShanghaiTech part B, a sparse crowd dataset. Based on the
superior performance, we hope the researchers can design a more reasonable
regression head based on the Transformer-encoder in the future.

###  6.2 Visualizations

To further investigate the proposed TransCrowd, we provide qualitative
comparison results in Fig. 4 to understand what the Transformer attends to. We
observe that both TransCrowd-Token and TransCrowd-GAP can successfully focus
on the crowd region, which demonstrates the effectiveness of both methods.
Moreover, the TransCrowd-GAP generates a more reasonable attention map
compared with the TransCrowd-Token. Specifically, the TransCrowd-Token may pay
more attention to the background, leading to amplifying the counting error.
This observation explains why the result of TransCrowd-GAP is better than
TransCrowd-Token.

###  6.3 Convergence curves

We further compare the convergence curves between the popular fully-supervised
method (CSRNet [21]) and the proposed TransCrowd. Detailed convergence curves
are shown in Fig. 5. Based on the convergence curves, we can observe the
following phenomena: (1) Compared with CSRNet, TransCrowd-GAP achieves better
performance with 1.9 $\times$ fewer training epochs. (2) Using global average
pooled visual tokens can converge faster and achieve better count accuracy
than using the extra regression token. (3) Both TransCrowd-Token and
TransCrowd-GAP present a smooth curve and fast converging, while the curve of
CSRNet is oscillating. These observations show the potential value of the
Transformer in the counting task.

###  6.4 Comparison of run-time.

As shown in Tab. 8, we compare with two popular fully supervised counting
methods, including BL [34] and CSRNET [21]. The experiment is conducted on a
Titan XP GPU. Even though both the proposed TransCrowd-Token and TransCrowd-
GAP contain more parameters than other methods, they still achieve outstanding
run-time. This is because the fully supervised methods need to maintain high-
resolution features to generate high-quality density maps (e.g., 1/8 size of
the input in CSRNET [21] and 1/16 size of the input in BL [34]). Additionally,
we can observe that the FPS of VGG19-based BL [34] outperforms the VGG16-based
CSRNet [21], mainly because the BL generates a small-resolution density map
(1/16 of the input image). This phenomenon further demonstrates the influence
of feature resolution on run-time.

###  6.5 Comparison of Different Pre-trained strategies.

In this section, we study the impact of the pre-trained model in TransCrowd.
We choose the popular CNN-based method CSRNet [21] as a comparison, and the
results are listed in Tab. 9. Specifically, there are three strategies: (1)
None: The models are directly trained on ShanghaiTech part A. (2) Pre-ImgNet:
The models are pre-trained on the ImageNet and fine-tune on ShanghaiTech part
A. (3) Pre-GCC: The models are pre-trained on GCC [56], a synthetic dataset,
and are fine-tuned on ShangahiTech part A dataset.

From the Tab. 9, there are some interesting findings: 1) Without any pre-
trained dataset, the CNN-based method outperforms the Transformer-based
method. 2) Using the extra pre-trained data can effectively prompt the
performance, and the proposed TransCrowd-GAP achieves better counting
performance than CSRNet. 3) Besides, when the model pre-trained on the GCC
dataset, the proposed method can even outperform several recent fully-
supervised methods (e.g., CFF [42], TEDNet [15]). Note that the GCC dataset is
a synthetic crowd dataset, without any annotation cost, which means the
TransCrowd-GAP can achieve similar counting performance to the fully-
supervised methods by using small count-level labeled real-data and extensive
free synthetic data, promoting the practical applications. It is noteworthy
that the proposed method only uses count-level annotations of the GCC dataset,
different from the previous fully-supervised work.

###  6.6 Cross-dataset evaluation.

Finally, we conduct cross-dataset experiments on the UCF-QNRF, ShanghaiTech
Part A and Part B datasets to explore the transferability of the proposed
TransCrowd-GAP. In the Cross-dataset evaluation, models are trained on the
source dataset and tested on the target dataset without further ﬁne-tuning.
Quantitative results are shown in Tab. 10. Although our method is a weakly-
supervised paradigm, we still achieve highly competitive performance, which
shows remarkable transferability.

##  7 Conclusion

In this work, we present an alternative perspective for weakly-supervised
crowd counting in images by introducing a sequence-to-count prediction
framework based on Transformer-encoder, named TransCrowd. To the best of our
knowledge, we are the first to solve the counting problem based on the
Transformer. We analyze and show that the attention mechanism is very
promising to capture the semantic crowd information. Extensive experiments on
five challenging datasets demonstrate that TransCrowd achieves superior
counting performance compared with the state-of-the-art weakly-supervised
methods and achieves competitive performance compared with some popular fully-
supervised methods. In the future, we plan to make fully-supervised counting
using Transformer architecture and extend it to video-based counting task.

## References

  * [1] Shahira Abousamra, Minh Hoai, Dimitris Samaras, and Chao Chen.  Localization in the crowd with topological constraints.  In Proc. of the AAAI Conf. on Artificial Intelligence, 2021.
  * [2] Shuai Bai, Zhiqun He, Yu Qiao, Hanzhe Hu, Wei Wu, and Junjie Yan.  Adaptive dilated network with self-correction supervision for counting.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2020.
  * [3] Xinkun Cao, Zhipeng Wang, Yanyun Zhao, and Fei Su.  Scale aggregation network for accurate and efficient crowd counting.  In Proc. of European Conference on Computer Vision, 2018.
  * [4] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.  End-to-end object detection with transformers.  In Proc. of European Conference on Computer Vision, 2020.
  * [5] Antoni B Chan, Zhang-Sheng John Liang, and Nuno Vasconcelos.  Privacy preserving crowd monitoring: Counting people without people models or tracking.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2008.
  * [6] Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao.  Pre-trained image processing transformer.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 12299–12310, 2021.
  * [7] Yajie Chen, Dingkang Liang, Xiang Bai, Yongchao Xu, and Xin Yang.  Cell localization and counting using direction field map.  IEEE Journal of Biomedical and Health Informatics, 2021.
  * [8] Dawei Du, Longyin Wen, Pengfei Zhu, Heng Fan, Qinghua Hu, Haibin Ling, Mubarak Shah, Junwen Pan, Ali Al-Ali, Amr Mohamed, et al.  Visdrone-cc2020: The vision meets drone crowd counting challenge results.  In Proc. of European Conference on Computer Vision, pages 675–691. Springer, 2020.
  * [9] Junyu Gao, Wei Lin, Bin Zhao, Dong Wang, Chenyu Gao, and Jun Wen.  C^ 3 framework: An open-source pytorch code for crowd counting.  arXiv preprint arXiv:1907.02724, 2019.
  * [10] Junyu Gao, Qi Wang, and Xuelong Li.  Pcc net: Perspective crowd counting via spatial convolutional network.  IEEE Transactions on Circuits and Systems for Video Technology, 2019\.
  * [11] Bin Guo, Zhu Wang, Zhiwen Yu, Yu Wang, Neil Y Yen, Runhe Huang, and Xingshe Zhou.  Mobile crowd sensing and computing: The review of an emerging human-powered sensing paradigm.  ACM computing surveys (CSUR), 48(1):1–31, 2015.
  * [12] Dan Hendrycks and Kevin Gimpel.  Bridging nonlinearities and stochastic regularizers with gaussian error linear units.  2016\.
  * [13] Haroon Idrees, Imran Saleemi, Cody Seibert, and Mubarak Shah.  Multi-source multi-scale counting in extremely dense crowd images.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2013.
  * [14] Haroon Idrees, Muhmmad Tayyab, Kishan Athrey, Dong Zhang, Somaya Al-Maadeed, Nasir Rajpoot, and Mubarak Shah.  Composition loss for counting, density map estimation and localization in dense crowds.  In Proc. of European Conference on Computer Vision, 2018.
  * [15] Xiaolong Jiang, Zehao Xiao, Baochang Zhang, Xiantong Zhen, Xianbin Cao, David Doermann, and Ling Shao.  Crowd counting and density estimation by trellis encoder-decoder networks.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2019.
  * [16] Xiaoheng Jiang, Li Zhang, Mingliang Xu, Tianzhu Zhang, Pei Lv, Bing Zhou, Xin Yang, and Yanwei Pang.  Attention scaling for crowd counting.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2020.
  * [17] Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of NAACL-HLT, pages 4171–4186, 2019.
  * [18] DP Kingma and JL Ba.  Adam: A method for stochastic optimization 3rd international conference on learning representations.  In Proc. of International Conference on Learning Representations, 2015.
  * [19] Alexander Kolesnikov, Alexey Dosovitskiy, Dirk Weissenborn, Georg Heigold, Jakob Uszkoreit, Lucas Beyer, Matthias Minderer, Mostafa Dehghani, Neil Houlsby, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  2021\.
  * [20] Yinjie Lei, Yan Liu, Pingping Zhang, and Lingqiao Liu.  Towards using count-level weak supervision for crowd counting.  Pattern Recognition, 109:107616, 2021.
  * [21] Yuhong Li, Xiaofan Zhang, and Deming Chen.  CSRNet: Dilated convolutional neural networks for understanding the highly congested scenes.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2018.
  * [22] Dingkang Liang, Wei Xu, Yingying Zhu, and Yu Zhou.  Focal inverse distance transform maps for crowd localization and counting in dense crowd.  arXiv preprint arXiv:2102.07925, 2021.
  * [23] Hui Lin, Xiaopeng Hong, Zhiheng Ma, Xing Wei, Yunfeng Qiu, Yaowei Wang, and Yihong Gong.  Direct measure matching for crowd counting.  In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2021.
  * [24] Jiang Liu, Chenqiang Gao, Deyu Meng, and Alexander G Hauptmann.  Decidenet: counting varying density crowds through attention guided detection and density estimation.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2018.
  * [25] Liang Liu, Hao Lu, Hongwei Zou, Haipeng Xiong, Zhiguo Cao, and Chunhua Shen.  Weighing counts: Sequential crowd counting by reinforcement learning.  In Proc. of European Conference on Computer Vision, pages 164–181. Springer, 2020.
  * [26] Lingbo Liu, Zhilin Qiu, Guanbin Li, Shufan Liu, Wanli Ouyang, and Liang Lin.  Crowd counting with deep structured scale integration network.  In Porc. of IEEE Intl. Conf. on Computer Vision, 2019.
  * [27] Ning Liu, Yongchao Long, Changqing Zou, Qun Niu, Li Pan, and Hefeng Wu.  Adcrowdnet: An attention-injective deformable convolutional network for crowd understanding.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2019.
  * [28] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg.  Ssd: Single shot multibox detector.  In Proc. of European Conference on Computer Vision, 2016.
  * [29] Weizhe Liu, Mathieu Salzmann, and Pascal Fua.  Context-aware crowd counting.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2019.
  * [30] Xialei Liu, Joost Van De Weijer, and Andrew D Bagdanov.  Exploiting unlabeled data in cnns by self-supervised learning to rank.  IEEE transactions on pattern analysis and machine intelligence, 2019\.
  * [31] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  arXiv preprint arXiv:1907.11692, 2019.
  * [32] Yuting Liu, Miaojing Shi, Qijun Zhao, and Xiaofang Wang.  Point in, box out: Beyond counting persons in crowds.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2019.
  * [33] Zhihao Liu, Zhijian He, Lujia Wang, Wenguan Wang, Yixuan Yuan, Dingwen Zhang, Jinglin Zhang, Pengfei Zhu, Luc Van Gool, Junwei Han, et al.  Visdrone-cc2021: The vision meets drone crowd counting challenge results.  In Porc. of IEEE Intl. Conf. on Computer Vision, pages 2830–2838, 2021.
  * [34] Zhiheng Ma, Xing Wei, Xiaopeng Hong, and Yihong Gong.  Bayesian loss for crowd count estimation with point supervision.  In Porc. of IEEE Intl. Conf. on Computer Vision, 2019.
  * [35] Zhiheng Ma, Xing Wei, Xiaopeng Hong, and Yihong Gong.  Learning scales from points: A scale-aware probabilistic model for crowd counting.  In Proc. of ACM Multimedia, pages 220–228, 2020.
  * [36] Zhiheng Ma, Xing Wei, Xiaopeng Hong, Hui Lin, Yunfeng Qiu, and Yihong Gong.  Learning to count via unbalanced optimal transport.  In Proc. of the AAAI Conf. on Artificial Intelligence, volume 35, pages 2319–2327, 2021.
  * [37] Viresh Ranjan, Hieu Le, and Minh Hoai.  Iterative crowd counting.  In Proc. of European Conference on Computer Vision, 2018.
  * [38] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In Proc. of Advances in Neural Information Processing Systems, 2015\.
  * [39] Deepak Babu Sam, Skand Vishwanath Peri, Mukuntha Narayanan Sundararaman, Amogh Kamath, and Venkatesh Babu Radhakrishnan.  Locate, size and count: Accurately resolving people in dense crowds via detection.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020\.
  * [40] Xiang Sheng, Jian Tang, Xuejie Xiao, and Guoliang Xue.  Leveraging gps-less sensing scheduling for green mobile crowd sensing.  IEEE Internet of Things Journal, 1(4):328–336, 2014.
  * [41] Miaojing Shi, Zhaohui Yang, Chao Xu, and Qijun Chen.  Revisiting perspective information for efficient crowd counting.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2019.
  * [42] Zenglin Shi, Pascal Mettes, and Cees GM Snoek.  Counting with focus for free.  In Porc. of IEEE Intl. Conf. on Computer Vision, pages 4200–4209, 2019.
  * [43] Zenglin Shi, Le Zhang, Yun Liu, Xiaofeng Cao, Yangdong Ye, Ming-Ming Cheng, and Guoyan Zheng.  Crowd counting with deep negative correlation learning.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2018.
  * [44] Vishwanath Sindagi, Rajeev Yasarla, and Vishal MM Patel.  Jhu-crowd++: Large-scale crowd counting dataset and a benchmark method.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020\.
  * [45] Vishwanath A Sindagi and Vishal M Patel.  Cnn-based cascaded multi-task learning of high-level prior and density estimation for crowd counting.  In Proc. of IEEE Intl. Conf. on Advanced Video and Signal Based Surveillance, 2017.
  * [46] Vishwanath A Sindagi and Vishal M Patel.  Generating high-quality crowd density maps using contextual pyramid cnns.  In Porc. of IEEE Intl. Conf. on Computer Vision, 2017.
  * [47] Vishwanath A Sindagi and Vishal M Patel.  Multi-level bottom-top and top-bottom feature fusion for crowd counting.  In Porc. of IEEE Intl. Conf. on Computer Vision, 2019.
  * [48] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Proc. of Advances in Neural Information Processing Systems, 2017\.
  * [49] Matthias von Borstel, Melih Kandemir, Philip Schmidt, Madhavi K Rao, Kumar Rajamani, and Fred A Hamprecht.  Gaussian process density counting from weak supervision.  In Proc. of European Conference on Computer Vision, pages 365–380. Springer, 2016.
  * [50] Jia Wan and Antoni Chan.  Modeling noisy annotations for crowd counting.  Advances in Neural Information Processing Systems, 2020.
  * [51] Jia Wan, Wenhan Luo, Baoyuan Wu, Antoni B Chan, and Wei Liu.  Residual regression with semantic prior for crowd counting.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2019.
  * [52] Jia Wan, Qingzhong Wang, and Antoni B Chan.  Kernel-based density map generation for dense object counting.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020\.
  * [53] Boyu Wang, Huidong Liu, Dimitris Samaras, and Minh Hoai.  Distribution matching for crowd counting.  In Proc. of Advances in Neural Information Processing Systems, 2020\.
  * [54] Ning Wang, Wengang Zhou, Jie Wang, and Houqiang Li.  Transformer meets tracker: Exploiting temporal context for robust visual tracking.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 1571–1580, 2021.
  * [55] Qi Wang, Junyu Gao, Wei Lin, and Xuelong Li.  Nwpu-crowd: A large-scale benchmark for crowd counting and localization.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020\.
  * [56] Qi Wang, Junyu Gao, Wei Lin, and Yuan Yuan.  Learning from synthetic data for crowd counting in the wild.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2019.
  * [57] Chenfeng Xu, Dingkang Liang, Yongchao Xu, Song Bai, Wei Zhan, Xiang Bai, and Masayoshi Tomizuka.  Autoscale: Learning to scale for crowd counting.  International Journal of Computer Vision, pages 1–30, 2022.
  * [58] Chenfeng Xu, Kai Qiu, Jianlong Fu, Song Bai, Yongchao Xu, and Xiang Bai.  Learn to scale: Generating multipolar normalized density map for crowd counting.  In Porc. of IEEE Intl. Conf. on Computer Vision, 2019.
  * [59] Wei Xu, Dingkang Liang, Yixiao Zheng, Jiahao Xie, and Zhanyu Ma.  Dilated-scale-aware category-attention convnet for multi-class object counting.  IEEE Signal Processing Letters, 28:1570–1574, 2021.
  * [60] Zhaoyi Yan, Yuchen Yuan, Wangmeng Zuo, Xiao Tan, Yezhen Wang, Shilei Wen, and Errui Ding.  Perspective-guided convolution networks for crowd counting.  In Porc. of IEEE Intl. Conf. on Computer Vision, 2019.
  * [61] Yifan Yang, Guorong Li, Zhe Wu, Li Su, Qingming Huang, and Nicu Sebe.  Reverse perspective network for perspective-aware object counting.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2020.
  * [62] Yifan Yang, Guorong Li, Zhe Wu, Li Su, Qingming Huang, and Nicu Sebe.  Weakly-supervised crowd counting learns from sorting rather than locations.  In Proc. of European Conference on Computer Vision, 2020.
  * [63] Anran Zhang, Lei Yue, Jiayi Shen, Fan Zhu, Xiantong Zhen, Xianbin Cao, and Ling Shao.  Attentional neural fields for crowd counting.  In Porc. of IEEE Intl. Conf. on Computer Vision, 2019.
  * [64] Cong Zhang, Hongsheng Li, Xiaogang Wang, and Xiaokang Yang.  Cross-scene crowd counting via deep convolutional neural networks.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 833–841, 2015.
  * [65] Shanghang Zhang, Guanhang Wu, Joao P Costeira, and José MF Moura.  Fcn-rlstm: Deep spatio-temporal neural networks for vehicle counting in city cameras.  In Porc. of IEEE Intl. Conf. on Computer Vision, 2017.
  * [66] Yingying Zhang, Desen Zhou, Siqin Chen, Shenghua Gao, and Yi Ma.  Single-image crowd counting via multi-column convolutional neural network.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2016.
  * [67] Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip HS Torr, et al.  Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers.  In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, pages 6881–6890, 2021.
  * [68] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai.  Deformable detr: Deformable transformers for end-to-end object detection.  In International Conference on Learning Representations, 2020.