# DenoiseReID: Denoising Model for Representation Learning of Person Re-
Identification

Zhengrui Xu$\phantom{}{}^{1}$
<EMAIL_ADDRESS>&Guan’an Wang $\phantom{}{}^{\dagger}$
<EMAIL_ADDRESS>
&Xiaowen Huang $\phantom{}{}^{1,2,3}$
<EMAIL_ADDRESS>
&Jitao Sang $\phantom{}{}^{1,2,3}$
<EMAIL_ADDRESS>&$\phantom{}{}^{1}$School of Computer Science and
Technology, Beijing Jiaotong University Equal Contribution.Project
Lead.Corresponding Author. $\phantom{}{}^{2}$Beijing Key Lab of Traffic Data
Analysis and Mining, Beijing Jiaotong University  $\phantom{}{}^{3}$Key
Laboratory of Big Data & Artificial Intelligence
in Transportation(Beijing Jiaotong University), Ministry of Education

###### Abstract

The denoising model has been proven a powerful generative model but has little
exploration of discriminative tasks. Representation learning is important in
discriminative tasks, which is defined as "learning representations (or
features) of the data that make it easier to extract useful information when
building classifiers or other predictors" in [4]. In this paper, we propose a
novel Denoising Model for Representation Learning and take Person Re-
Identification (ReID) as a benchmark task, named DenoiseReID, to improve
feature discriminative with joint feature extraction and denoising. We first
view each embedding layer in a backbone as a denoising layer, processing the
cascaded embedding layers as if we are recursively denoise features step-by-
step. This unifies the frameworks of feature extraction and feature denoising,
where the former progressively embeds features from low-level to high-level,
and the latter recursively denoises features step-by-step. Then we design a
novel Feature Extraction and Feature Denoising Fusion Algorithm (FEFDFA) and
theoretically demonstrate its equivalence before and after fusion. FEFDFA
merges parameters of the denoising layers into existing embedding layers, thus
making feature denoising computation-free. This is a label-free algorithm to
incrementally improve feature also complementary to the label if available.
Experimental results on 4 ReID datasets and various of backbones show the
stability and impressive improvements. We also extend the proposed method to
large-scale (ImageNet) and fine-grained (CUB200, Oxford-Pet, Flowers)
classification tasks, similar improvements are observed. The code will be
released on github.

##  1 Introduction

Denoising Diffusion Probabilistic Models (DDPM) [20] or Diffusion Model for
short have been proven to be a powerful generative model [5]. Generative
models can generate vivid samples (such as images, audio and video) by
modelling the joint distribution of the data $P(X,Y)$, where $X$ is the sample
and $Y$ is the condition. Diffusion models achieve this goal by adding
Gaussian noise to the data and training a denoising model of inversion to
predict the noise. Diffusion models can generate multi-formity and rich
samples, such as Stable diffusion [45], DALL [43] series and Midjourney, these
powerful image generation models, which are essentially diffusion models.

However, its application to discriminative models has not been extensively
explored. Different from generative models, discriminative models predict data
labels by modelling the marginal distribution of the data $P(Y|X)$. $Y$ can be
various labels, such as image tags for classification, object boxes for
detection, and pixel tags for segmentation. Currently, there are several
methods based on diffusion models implemented in specific fields. For example,
DiffusionDet [6] is a new object detection framework that models object
detection as a denoising diffusion process from noise boxes to object boxes.
It describes object detection as a generative denoising process and performs
well compared to previous mature object detectors. DiffSeg [47] for image
segmentation, which is a method of unsupervised zero-shot sample segmentation
using pre-trained models (stable diffusion). It introduces a simple and
effective iterative merging process to measure the attention maps between KL
divergence and merge them into an effective segmentation mask. The proposed
method does not require any training or language dependency to extract the
quality segmentation of any image.

Figure 1:  A brief description of our idea. (a) A typical denoising model for
generative tasks recursively applies a denoising layer. (b) A naive idea that
applies a denoising strategy to a discriminative model is applying a recursive
denoise layer on the feature of a backbone and taking extra inference latency.
(c,d) Our DenoiseReID first unifies the frameworks of feature extraction and
denoising in a backbone pipeline, then merges parameters of the denoising
layers into embedding layers, making the feature more discriminative without
extra latency cost.

The methods above are carefully designed for specific tasks and require a
particular data structure. For example, DiffusionDet [6] uses noise boxes and
DiffSeg [47] uses noise segmentation. In this paper, we explore a more general
conception of how the denoising model can improve representation learning,
i.e. "learning representations (or features) of the data that make it easier
to extract useful information when building classifiers or other predictors"
[4], and contribute to discriminative models. We take person Re-Identification
(ReID) [60, 3] as a benchmark task. ReID aims to match images of a pedestrian
under disjoint cameras, and is suffered by pose, lighting, occlusion and so
on, thus requiring more identity-discriminative feature.

A straightforward approach is applying the denoising process to a backbone’s
final feature [24, 14], reducing noise in the final output and making the
feature more discriminative, as Fig. 1(b) shows. However, this way can be
computationally intensive. Because the denoising layer needs to be proceeded
on the output of the previous one in a recursive and step-by-step manner.
Considering that a backbone typically consists of cascaded embedding layers
(e.g., convolution layer, multi-head attention layer), we propose a novel
perspective: treating each embedding layer as a denoising layer. As shown in
Fig. 1(c), it allows us to process the cascaded layers as if we are
recursively proceeding through the denoising layer step-by-step. This method
transforms the backbone into a series of denoising layers, each working on a
different feature extraction level. While this idea is intuitive and simple,
its practical implementation presents a significant challenge. The main issue
arises from the requirement of the denoising layer for the input and output
features to exist in the same feature space. However, in a typical backbone
(e.g. ResNet [24], ViT [14])), the layers progressively map features from a
low level to a high level. It means that the feature space changes from layer
to layer, which contradicts the requirement of the denoising layer.

To resolve all the difficulties above and efficiently apply the denoising
process to improve discriminative tasks, our proposed Denoising Model for
Representation Learning of Person Re-Identification (DenoiseReID) is as below:
Firstly, we utilize a well-trained backbone and keep it fixed throughout all
subsequent procedures. This step is a free launch as we can easily use any
publicly available backbone without requiring additional training time. This
approach allows us to preserve the backbone’s inherent characteristics of
semantic feature extraction. Given the backbone and an image, we can get a
list of features. Next, we train denoising layers on those features. The
weights of denoising layers are randomly initialized and their weights are not
shared. The training process is the same as that in DDPM [20], where the only
difference is that the denoising layer in DDPM takes a dynamic $t\in[1,T]$,
and our denoising layers take fixed $n\in[1,N]$, where $n$ is the layer index,
$T$ is denoise times and $N$ is backbone layer number as shown in Fig. 1(c).
Finally, considering that the $N$ denoising layers consume additional
execution latency, we propose a novel Feature Extraction and Feature Denoising
Fusion Algorithm (FEFDFA). As shown in Fig. 1(d), FEFDFA merges parameters of
extra denoising layers into weights of the existing embedding layers, thus
enabling joint feature extraction and denoising without any extra computation
cost. We also theoretically demonstrate the total equivalence before and after
parameter fusion. Please see Section 3.3 and Eq (7) for more details.

Our contributions can be summarized as follows:

(1) We propose a novel Denoising Model for Representation Learning of Person
Re-Identification (DenoiseReID), which innovatively integrates the denoising
process, originating from generative tasks, into the discriminative tasks. It
treats $N$ cascaded embedding layers of a backbone as $T$ times recursively
proceeded denoising layers. This idea enables joint feature extraction and
denoising is a backbone, thus making features more discriminative.

(2) We propose a novel Feature Extraction and Feature Denoising Fusion
Algorithm (FEFDFA), which fuses the parameters of the denoising layers into
the parameters of the corresponding embedding layers and theoretically
demonstrates their equivalence. This contributes to a computation-efficient
algorithm, which takes no extra latency.

(3) Extensive experiments on 4 ReID datasets verified that our proposed
DenoiseReID can effectively improve feature performance in a label-free manner
and performs better in the case of label-argumented supervised training or
introduction of additional training data. We also extend our proposed method
to large-scale (ImageNet) and fine-grained (e.g. CUB200, Oxford-Pet, etc.)
image classifications, showing its effectiveness.

##  2 Related Work

Generative models learn the distribution of inputs before estimating class
probabilities. A generative model is a model that learns the data generation
process by learning the probability distribution of the input data and
generating new data samples. The generative models first estimate the
conditional density of categories $P(x|y=k)$ and prior category probabilities
$P(y=k)$ from the training data. The $P(x)$ is obtained by the full
probability formula. So as to model the probability distribution of each type
of data. Generative models can generate new samples by modelling data
distribution. For example, Generative Adversarial Networks (GANs) [17, 39, 21]
and Variational Autoencoders (VAEs) [22, 48, 62, 58] are both classic
generative models that generate real samples by learning potential
representations of data distributions, demonstrating excellent performance in
data distribution modeling. Recent research has focused on using diffusion
models for generative tasks. The diffusion model was first proposed by the
article [46] in 2015, with the aim of eliminating Gaussian noise from
continuous application to training images. The DDPM [20] proposed in 2020 have
made the use of diffusion models for image generation mainstream. In addition
to its powerful generation ability, the diffusion model also has good
denoising ability through noise sampling, which can denoise noisy data and
restore its original data distribution.

Discriminative models learn condition datatribution, e.g. $P(y|x)$, where $x$
is input data and $y$ is task-relative features, classiification task [1, 2,
12] map data into its tags, retrieval task [33, 57] map data into a feature
space where similar data should be near otherwise farawary, detection task
[44, 25] map data into is space position and size. Person Re-Identification
(ReID) is a fine-grained retrieval task that solves the challenging task of
identifying individuals in different camera views. Due to its fine-grained
nature, it has a more discriminative feature, so we chose it as our benchmark
task. Existing ReID methods can be grouped into hand-crafted descriptors [32,
38, 59] incorporated with metric learning [23, 31, 64] and deep learning
algorithms [53, 52, 51, 40, 16, 9]. State-of-the-art ReID models often
leverage convolutional neural networks (CNNs) [26] to capture intricate
spatial relationships and hierarchical features within person images.
Attention mechanisms [49, 14], spatial-temporal modeling [29, 28], and domain
adaptation techniques [8] have further enhanced the adaptability of ReID
models to diverse and challenging real-world scenarios.

Reparameterization of a model adopts an approximate equivalent new method to
replace an existing modeling method. Reparameterization can reduce the number
of parameters, reduce model computational complexity, and improve design
flexibility. As RepVGG [13] proposed, a multi-branch structure is set up in
the VGG network, which only has linear operations inside. During the inference
stage, the parameters of multiple branches are merged into parameters of a
single branch through reparameterization, without the need for retraining,
thereby reducing the number of parameters and the time complexity of
inference.

##  3 Denoising Representation for Person Re-Identification

###  3.1 Review Representation Learning of Person Re-Identification

The standard ReID pipeline consists of a backbone to learn representation
followed by a classification loss and triplet loss for learning an embedding
space where images of the same identity have high similarity. Common backbones
include CNN series [18] and Transformer series [14].

In this work, we chose TransReID-SSL [37] as the baseline, which uses ViT as
the backbone for feature extraction. As shown in the structural diagram on
Fig. 2, the input image is first passed through PatchEmbed, which splits the
image into chunks, and these chunks are fed to the transformer encoder. After
that, it goes through $N$ Blocks for feature extraction and outputs a feature
vector to represent this image.

###  3.2 Feature Extraction and Feature Denoising Unified Framework

In representing learning, feature representation is the most important factor
that affects the person re-identification (ReID) accuracy, so we would like to
learn features with better representation to improve the retrieval
performance. We suppose that in the inference stage, the features obtained by
backbone extraction are noisy, which may come from the input image or may be
generated during the feature extraction process.

To solve the problem above, we propose a novel Feature Extraction and Feature
Denoising Unified Framework (FEFDUF), which unifies feature extraction and
feature denoising in a single backbone to obtain semantic and discriminative
features. We refer to the diffusion modeling approach to denoise the noisy
features through T-steps to obtain clean features. At the beginning, we use
the features output from the backbone network as data samples for diffusion
training, and get the noisy samples by continuously adding noise and learning
through the network in order to simulate the data distribution of its
features.

$\displaystyle q(\mathbf{x}_{1:T}|\mathbf{x}_{0})$
$\displaystyle\coloneqq\prod_{t=1}^{T}q(\mathbf{x}_{t}|\mathbf{x}_{t-1})\qquad$
(1) $\displaystyle q(\mathbf{x}_{t}|\mathbf{x}_{t-1})$
$\displaystyle\coloneqq\mathcal{N}(\mathbf{x}_{t};\sqrt{1-\beta_{t}}\mathbf{x}_{t-1},\beta_{t}\mathbf{I})$
(2)

where $X_{0}$ represents the feature vector output by the backbone, $t$
represents the diffusion step size, $\beta_{t}$ is a set of pre-set
parameters, and $X_{t}$ represents the noise sample obtained through diffusion
process.

In the inference stage, as shown in Fig. 1(b), we perform T-step denoising on
the output features, to obtain cleaner features and improve the expressiveness
of the features.

$\displaystyle p_{\theta}(\mathbf{x}_{0:T})$ $\displaystyle\coloneqq
p(\mathbf{x}_{T})\prod_{t=1}^{T}p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})$
(3) $\displaystyle p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})$
$\displaystyle\coloneqq\mathcal{N}(\mathbf{x}_{t-1};\mu_{\theta}(\mathbf{x}_{t},t),\Sigma_{\theta}(\mathbf{x}_{t},t))$
(4)

where $X_{t}$ represents the feature vector output by the backbone in the
inference stage, which we suppose contains some noise. $T$ is the denoising
step size, representing the magnitude of the noise. We adjust $t$
appropriately based on different datasets and backbones to obtain the optimal
denoising amplitude. In ViT-base and ViT-small backbone, $t$ is set to 10 and
15, respectively. According to $p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})$
denoise it step by step, and finally obtains $X_{0}$, which represents the
clean feature after denoising.

Figure 2: Pipeline of our proposed method. ViT consists of $N$ cascaded
transformer encoder layers. During the training phase (see the right side
“Train Only” process), we freeze the backbone parameters and only train the
extra denoising layers. In the inference stage (see the left side “Infer Only”
process), we merge the parameters of denoising layers to corresponding encoder
layers with the proposed Feature Extraction and Feature Denoising Fusion
Algorithm (FEFDFA). So there is no extra inference latency cost. Please find
definitions of $W$, $b$, $W_{D}$, $W^{\prime}$, $i$ and $b^{\prime}$ in
Algorithm 2.

###  3.3 Feature Extraction and Feature Denoising Fusion Algorithm

As described in section 3.2, the proposed Feature Extraction and Feature
Denoising Unified Framework (FEFDUF) could effectively polish noisy features
but extra inference latency is introduced caused by recursive calling of the
denoising layers.

To solve the problem above, we proposed a novel Feature Extraction and Feature
Denoising Fusion Algorithm (FEFDFA) to fuse parameters of the extra denoising
layers into parameters of existing backbone layers and get a computation-free
method in the final. FEFDFA expands the linear layer in the MLP of each block
into two branches, one for its original $FC$ module and the other for FEFDFA
proposed in this paper. As shown on Fig. 2, during the training phase, we
freeze the original parameters and only trained the FEFDFA. The training
method is consistent with section 3.2, and the features are diffused and fed
into the predictive noise network of FEFDFA for prediction. For details,
please refer to Algorithm. 1. In the inference stage, we use the
reparameterization method to combine the pre-trained $W$ parameters with the
denoising module parameters, merging the two branches into a single branch
without additional inference time. The derivation of parameter merging is as
follows:

$X_{t-1}=\frac{1}{\sqrt{a_{t}}}(X_{t}-\frac{1-a_{t}}{\sqrt{1-\bar{a_{t}}}}D_{\theta}(X_{t},t))+\sigma_{t}z$
(5)

where $a_{t}=1-\beta_{t}$, $D_{\theta}$ are the parameters of the prediction
noise network.

$\displaystyle Y$ $\displaystyle=WX+b$ (6)
$\displaystyle\frac{1}{\sqrt{a_{t}}}X_{t}-X_{t-1}$
$\displaystyle=\frac{1-a_{t}}{\sqrt{a_{t}}\sqrt{1-\bar{a_{t}}}}D_{\theta}X_{t}-\sigma_{t}z$
$\displaystyle\frac{1}{\sqrt{a_{t}}}Y_{t}-Y_{t-1}$
$\displaystyle=\frac{1-a_{t}}{\sqrt{a_{t}}\sqrt{1-\bar{a_{t}}}}D_{\theta}Y_{t}-\sigma_{t}z$

We make a simple transformation of Eq. (5) and multiply both sides
simultaneously by $W$. The simplified equation can be obtained by bringing
$Y_{t}$ in terms of $WX_{t}+b$:

$\displaystyle Y_{t-1}=[W-C_{1}(t)$ $\displaystyle
WW_{D}]X_{t}+WC_{2}(t)C_{3}+b$ (7) $\displaystyle
C_{1}(t)=\frac{1-a_{t}}{\sqrt{a_{t}}\sqrt{1-\bar{a_{t}}}}\qquad C_{2}(t)$
$\displaystyle=\frac{1-\bar{a_{t-1}}}{1-\bar{a_{t}}}\beta_{t}\qquad
C_{3}=Z\sim N(0,I)$

where $W_{D}$ denotes the parameters of $D_{\theta}(X_{t},t)$, $X_{t}$ denotes
the input of this linear layer, $Y_{t}$ denotes the output of this linear
layer, and $Y_{t-1}$ denotes the result after denoising in one step of
$Y_{t}$. Due to the cascading relationship of blocks, as detailed in
Algorithm. 2, different $t$ values are set according to the order between
levels, and the one-step denoising of one layer is combined to achieve the
denoising process of $Y_{t}\rightarrow Y_{0}$, ensuring the continuity of
denoising and ultimately obtaining clean features. We split the original
single branch into a dual branch structure. During the training phase, the
backbone maintains its original parameters and needs to train the denoising
module parameters. In the inference stage, as shown on the left side of Fig.
2, we use the method of reparameterization, to replace the original $W$
parameter with $W^{\prime}$, where $W^{\prime}=[W-C_{1}(t)WW_{D}]$ in Eq. (7),
which has the same number of parameters as $W$, thus achieving the combination
of $FC$ operation and denoising without additional time cost. It is a
Computation-free method.

Algorithm 1 Denoising Model Training

1:The number of feature layers in the backbone N, Features extracted from each
layer $\\{F_{i}\\}_{i=1}^{N}$, The denoising module that needs to be trained
$\\{D_{i}(\cdot)\\}_{i=1}^{N}$

2:repeat

3: for each $i\in[N,1]$ do

4: $t=i$: Specify the diffusion step t for the current layer based on the
order of layers.

5: $\epsilon\sim N(0,I)$: Randomly sample a Gaussian noise

6: $X_{t}=\sqrt{\bar{a_{t}}}F_{i}+\sqrt{1-\bar{a_{t}}}\epsilon$: Forward
diffusion process in Eq.(2)

7: Take gradient descent step on $\nabla_{\theta}\left\|\epsilon-
D_{i}(X_{t},t)\right\|^{2}$

8: end for

9:until converged

In Eq. (7), we achieve one-step denoising $Y_{t}\rightarrow Y_{t-1}$. If we
need to increase the denoising amplitude, we can extend it to two-step or
multi-step denoising. The following is the derivation formula for two-step
denoising:

$\frac{1}{\sqrt{a_{t}}}Y_{t}-Y_{t-1}=C_{1}(t)D_{\theta}Y_{t}-\sigma_{t}z$ (8)
$\frac{1}{\sqrt{a_{t-1}}}Y_{t-1}-Y_{t-2}=C_{1}(t-1)D_{\theta}Y_{t-1}-\sigma_{t-1}z$
(9)

We can obtain this by eliminating $Y_{t-1}$ from Eq.(8) and Eq.(9) and
replacing $Y_{t}$ with $WX_{t}+b$:

$\displaystyle Y_{t-2}=W^{\prime\prime}X_{t}+C^{\prime\prime}$ (10)
$\displaystyle
W^{\prime\prime}=\frac{1}{\sqrt{a_{t}-1}}\\{\frac{W}{\sqrt{a_{t}}}-[C_{1}(t)$
$\displaystyle+C_{1}(t-1)]WW_{D}+\sqrt{a_{t}}C_{1}(t-1)C_{1}(t)WW_{D}W_{D}\\}$
$\displaystyle C^{\prime\prime}=\frac{1}{\sqrt{a_{t}-1}}[WC_{2}(t)+$
$\displaystyle\sqrt{a_{t}}WC_{2}(t-1)-\sqrt{a_{t}}C1_{(}t-1)C_{2}(t)WW_{D}]Z+b$

Note that a single module completes two steps of denoising. To ensure the
continuity of denoising, the $t$ value should be sequentially reduced by 2.

The FEFDFA we propose is based on feature level denoising and can be migrated
to various downstream tasks. It denoises the features on each layer for better
removal of noise at each stage, as the noise in the inference stage comes from
multiple sources, which could be noise in the input image or noise generated
while passing through the network. Denoising each layer avoids noise
accumulation and gives better quality output. And according to the noise
challenges brought by data in different scenarios, the denoising intensity can
be adjusted by controlling $t$, $\beta_{t}$, and the number of denoising
times, which has good generalization ability.

Algorithm 2 Denoising Model Inference

1:The number of feature layers in the backbone N, Features extracted from each
layer $\\{F_{i}\\}_{i=1}^{N}$, Trained denoising module parameters
$\\{W_{D_{i}}\\}_{i=1}^{N}$ in Algorithm(1), After obtaining the initial
feature $F^{N}$ through patch_embed, it is necessary to remove N-step noise
from it, The pre-trained parameters $\\{W_{i}\\}_{i=1}^{N}$ and
$\\{b_{i}\\}_{i=1}^{N}$ for the backbone.

2:Feature $F^{0}$ after denoising.

3:for each $i\in[N,1]$ do

4: $t=i$: Set the denoising amplitude based on the depth of the current layer.

5: $W^{\prime}=[W_{i}-C_{1}(t)W_{i}W_{D_{i}}]$,
$b^{\prime}=W_{i}C_{2}(t)C_{3}+b_{i}$: Parameter fusion according to Eq.(7).

6: $F^{t-1}=W^{\prime}F^{t}+b^{\prime}$: Feature Extraction and Feature
Denoising Fusion.

7:end for

8:return $F^{0}$

###  3.4 Unsupervised Learning Manner

Our proposed FEFDFA is suitable for label-free unsupervised training because
its essence is a generative model that models data by learning its
distribution. Thus the training Loss contains only the $Loss_{p}$ of
prediction noise module:

$Loss_{p}=\left\|\epsilon-D_{\theta}(X_{t},t)\right\|$ (11)

where $\epsilon$ denotes the sampled noise, $X_{t}$ denotes the noise sample,
$t$ denotes the diffusion step, and $D_{\theta}(X_{t},t)$ denotes the noise
predicted by the model.

However, it is worth noting that our method is complementary to label if the
label is available. $Loss_{ReID}$ is the original search loss for baseline,
$\lambda$ is the trade-off parameter between two loss functions. The label-
argumented learning is defined as:

$Loss=(1-\lambda)Loss_{ReID}+\lambda Loss_{p}$ (12)

Results in experiments section 4.1 shows the improved performance by
introducing label information for supervised training.

##  4 Experiments

###  4.1 FEFDFA Performance Analysis

Table 1: FEFDFA performance analysis using mAP metric. The baseline method TransReID-SSL is based on ViT-small backbone. "label-arg" means label-argumented and "merged ds" means merged datasets. | Method
---
| DukeMTMC(%)
---
| MSMT17(%)
---
| Market1501(%)
---
| CUHK-03(%)
---
TransReID-SSL | 80.40 | 66.30 | 91.20 | 83.50
+FEFDFA (lable-free) | 80.92 (↑ 0.52) | 66.87 (↑ 0.57) | 91.82 (↑ 0.62) | 83.72 (↑ 0.22)
+FEFDFA (label-arg) | 81.22 (↑ 0.82) | 67.33 (↑ 1.03) | 92.05 (↑ 0.85) | 84.11 (↑ 0.61)
+FEFDFA (merged ds) | 80.98 (↑ 0.58) | 66.99 (↑ 0.69) | 91.80 (↑ 0.60) | 83.86 (↑ 0.36)

As mentioned in Section 3.3, Feature Extraction and Feature Denoising Fusion
Algorithm (FEFDFA) is an unsupervised denoising module, and its training does
not require the assistance of label information. We conducte the following
experiments to identify three key issues.

(1) Is this label-free and unsupervised training denoising plugin effective?
As shown in Table 1 (line2), compared with baseline method (line1), the
baseline method performs better after adding our label-free plugin, which
shows that our FEFDFA does have denoising capability for features.

(2) Could introducing label information for supervised training further
improve performance? Introducing label information is actually adding
$Loss_{ReID}$ as mentioned in Section 3.4 as a supervised signal. As shown in
Table 1 (line3), baseline method with label-argumented FEFDFA achieve
improvements of 0.32% - 0.70% on the mAP metric, indicating that our denoising
plugin has label compatibility, in other words, the plug-in is effective for
feature denoising regardless of label-argumented supervised or lable-free
unsupervised training.

(3) Since our plugin can perform unsupervised denoising of features, it is
natural to think about whether adding more data for training the plugin could
further improve its performance? We merge four datasets for training and then
test on each dataset using mAP to evaluate. Comparing the results of training
on sigle dataset (line2) with on merged datasets (line4), we found that
adopting other datasets for unsupervised learning can further improve the
performance of FEFDFA, which also proves that FEFDFA has good generalization
ability.

To demonstrate that our method can perform unsupervised learning and has good
generalization, we merged four datasets and rearranged the sequence IDs to
ensure the reliability of the experiment. The model is tested on the entire
dataset. During the training process, we freeze the baseline parameters and
only train the DenoiseReID module, without the need for labels, for
unsupervised learning. Then test on a single dataset and compare the results
of training on a single dataset. As shown in Table 1, it can be observed that
adding unlabeled training data from different datasets can improve the model’s
performance on a single dataset, proving that this module has a certain degree
of generalization.

###  4.2 Comparison with State-of-the-Art ReID Methods

We compare several state-of-the-art ReID methods on four datasets. One of the
best performing comparison methods is TransReID-SSL, which is a series of ReID
methods based on the ViT backbones. Other methods are based on structures such
as CNNs. We add the proposed FEFDFA to TransReID-SSL series and observe their
performance. As shown in Table 2, we have the following findings:

Table 2: Comparison with State-of-the-Art ReID methods using mAP and Rank-1
metrics.

Method | Backbone | MSMT17 | Market1501 | DukeMTMC | CUHK03-L
---|---|---|---|---|---
mAP | R1 | mAP | R1 | mAP | R1 | mAP | R1
MGN [54] | ResNet-50 | – | – | 86.90 | 95.70 | 78.40 | 88.70 | 67.40 | 68.00
OSNet [66] | OSNet | 52.90 | 78.70 | 84.90 | 94.80 | 73.50 | 88.60 | – | –
BAT-net [15] | GoogLeNet | 56.80 | 79.50 | 87.40 | 95.10 | 77.30 | 87.70 | 76.10 | 78.60
ABD-Net [7] | ResNet-50 | 60.80 | 82.30 | 88.30 | 95.60 | 78.60 | 89.00 | – | –
RGA-SC [61] | ResNet-50 | 57.50 | 80.30 | 88.40 | 96.10 | – | – | 77.40 | 81.10
ISP [67] | HRNet-W32 | – | – | 88.60 | 95.30 | 80.00 | 89.60 | 74.10 | 76.50
CDNet [27] | CDNet | 54.70 | 78.90 | 86.00 | 95.10 | 76.80 | 88.60 | – | –
Nformer [55] | ResNet-50 | 59.80 | 77.30 | 91.10 | 94.70 | 83.50 | 89.40 | 78.00 | 77.20
TransReID [19] | ViT-base-ics | 67.70 | 85.30 | 89.00 | 95.10 | 82.20 | 90.70 | 84.10 | 86.40
TransReID | ViT-base | 61.80 | 81.80 | 87.10 | 94.60 | 79.60 | 89.00 | 82.30 | 84.60
TransReID-SSL [37] | ViT-small | 66.30 | 84.80 | 91.20 | 95.80 | 80.40 | 87.80 | 83.50 | 85.90
TransReID-SSL | ViT-base | 75.00 | 89.50 | 93.10 | 96.52 | 84.10 | 92.60 | 87.80 | 89.20
TransReID+FEFDFA | ViT-base-ics | 68.10 | 85.72 | 89.56 | 95.50 | 82.35 | 90.87 | 84.15 | 86.39
TransReID+FEFDFA | ViT-base | 62.23 | 82.02 | 87.25 | 94.63 | 80.12 | 89.33 | 82.44 | 84.61
TransReID-SSL+FEFDFA | ViT-small | 67.33 | 85.50 | 92.05 | 96.68 | 81.22 | 88.72 | 84.11 | 86.47
TransReID-SSL+FEFDFA | ViT-base | 75.35 | 89.62 | 93.26 | 96.55 | 84.31 | 92.90 | 88.08 | 89.29

(1) our method stands out on four datasets on ViT-base backbone with a large
number of parameters, achieving almost the best performance on two evaluation
metrics.

(2) The methods using our plugin outperforms the original methods with the
same backbone on all datasets. In addition, the performance improvement of
small-scale backbones with the addition of FEFDFA is more significant than the
large-scale backbones approach due to the fact that FEFDFA is essentially a
denoising module that removes the noise contained in the features during the
inference stage. For large-scale backbones, the extracted features already
have good performance, so the denoising amplitude is limited. It has already
fitted the dataset well. For small-scale backbones with poor performance, due
to their limited fitting ability, there is a certain amount of noise in the
extracted features during the inference stage. Denoising them can obtain
better feedback.

(3) In fact, our FEFDFA can be applied to any other backbone, just add it to
each layer. In particular, the performance improvement of adding the denoising
plugin to a poorly performing backbone might be more significant. This needs
to be further verified in subsequent work. However, it is undeniable that we
have verified the denoising ability of the FEFDFA in the currently optimal
ReID method.

In this section, a comparative analysis was conducted on four datasets to
assess various existing ReID methods. These methods represent current
mainstream ReID approaches, employing ResNet101, ViT-S, ViT-B, and ResNet50 as
backbone architectures for feature extraction, respectively. Experimental
results indicate that our proposed method outperforms other approaches in
terms of both mAP and Rank-1 metrics.

###  4.3 Parameter Fusion Performance Analysis

Our Feature Extraction and Feature Denoising Fusion Algorithm (FEFDFA) is a
computation-free way, In section 3.3, we proved by theoretical derivation that
inserting our plugin into each layer and fusion it does not introduce
additional computation. In this section, we also conduct related validation
experiments, the results of which are shown in Table 3.

Table 3: Parameter Fusion Performance Analysis. The Feature Extraction and Feature Denoising Unified Framework (FEFDUF) denoises based on the features of the final layer, while the Feature Extraction and Feature Denoising Fusion Algorithm (FEFDFA) denoises based on the features of each layer. The baseline method TransReID-SSL is based on ViT-small backbone. | Method
---
| DukeMTMC
---
| MSMT17
---
| Market1501
---
| CUHK-03
---
| Inference Time
---
TransReID-SSL | 80.40% | 66.30% | 91.20% | 83.50% | 0.34s
+FEFDUF | 80.76% | 66.81% | 91.07% | 83.59% | 0.39s (+15%)
+FEFDFA | 81.22% | 67.33% | 92.05% | 84.11% |  0.34s (+0%)

Compare to the baseline method TransReID-SSL, adding FEFDUF is able to improve
the the performance, proving that feature based denoising is effective.
However, it also brings extra inference latency (about 15%) because it is
adding an extra parameter-independent denoising module at the end of the
model.

Adopting FEFDFA achieves a greater increase, it denoise the features on each
layer, which can better remove noise at each stage because the noise in the
inference stage comes from multiple aspects, which may be the noise in the
input image or generated when passing through the network. Denoising each
layer avoids noise accumulation and obtains a better quality output. Most
importantly, since the operation of fusion can merge the parameters of the
denoising module with the original parameters, the adoption of FEFDFA does not
take extra inference latency cost, which is a computation-free efficient
approach.

###  4.4 Experiments on Classification Tasks

Table 4: The effectiveness of our method in image classification tasks was validated on three fine-grained classification datasets (CUB200, Oxford-Pet, Flowers) and ImageNet-1k. | Method
---
| Datasets
---
| param
---
| acc@1
---
| acc@5
---
| acc@1(ours)
---
| acc@5(ours)
---
SwinV2-T [36] | ImageNet-1k | 28M | 81.82% | 95.88% | 82.13% | 96.06%
Vmanba-T [35] | ImageNet-1k | 30M | 82.38% | 95.80% | 82.51% | 95.89%
ResNet50 [18] | ImageNet-1k | 26M | 76.13% | 92.86% | 76.28% | 92.95%
ViT-B [14] | CUB200 | 87M | 91.78% | – | 91.99% | –
ViT-B | Oxford-Pet | 87M | 94.37% | – | 94.58% | –
ViT-B | Flowers | 87M | 99.12% | – | 99.30% | –

The Feature Extraction and Feature Denoising Fusion Algorithm (FEFDFA) is
based on denoising at the feature level and has good generalization. To
demonstrate its good generalization ability, we conduct experiments on other
vision tasks to test the effectiveness of the FEFDFA. We validated the
generalization ability of FEFDFA in image classification tasks on ImageNet-1k
[11] datasets and three fine-grained image classification datasets (CUB200
[50], Oxford-Pet [42], and Flowers [41]). The accuracy index is selected as
the evaluation indicator to evaluate the performance of the model. As shown in
Table 4, We compared multiple classic backbones for representation learning on
ImageNet-1k, and after adding the FEFDFA module, the accuracy of both top-1
and top-5 metrics improved without adding model parameters. As shown in Table
4, our method shows significant improvement in accuracy metrics compared to
baseline on three fine-grained classification datasets. Prove that the FEFDFA
module can improve the model’s ability in image classification for different
classification tasks.It has been proven that our method can enhance the
model’s representation learning ability and obtain more effective features
through denoising without adding additional time cost. More experimental
analysis can be found in Table 6 in Section C of the appendix.

##  5 Conclusion

In this work, we demonstrated that the diffusion model paradigm is effective
for feature level denoising in discriminative model, and proposed computation-
free and label-free algorithm: Feature Extraction and Feature Denoising Fusion
Algorithm (FEFDFA). It utilizes the denoising ability of diffusion models to
denoise the features in the feature extraction layer, and fuses the parameters
of the denoising layer with those of the feature extraction layer through
parameter fusion, further improving retrieval accuracy without incurring
additional computational costs. We validated the effectiveness of the FEFDFA
method on multiple common image discrimination task datasets.

## References

  * Agarap [2017] Abien Fred Agarap.  An architecture combining convolutional neural network (cnn) and support vector machine (svm) for image classification.  _arXiv preprint arXiv:1712.03541_ , 2017.
  * Albawi et al. [2017] Saad Albawi, Tareq Abed Mohammed, and Saad Al-Zawi.  Understanding of a convolutional neural network.  In _2017 international conference on engineering and technology (ICET)_ , pages 1–6. Ieee, 2017.
  * Bedagkar-Gala and Shah [2014] Apurva Bedagkar-Gala and Shishir K Shah.  A survey of approaches and trends in person re-identification.  _Image and vision computing_ , 32(4):270–286, 2014.
  * Bengio et al. [2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent.  Representation learning: A review and new perspectives.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 35(8):1798–1828, 2013.  doi: 10.1109/TPAMI.2013.50.
  * Cao et al. [2024] Hanqun Cao, Cheng Tan, Zhangyang Gao, Yilun Xu, Guangyong Chen, Pheng-Ann Heng, and Stan Z. Li.  A survey on generative diffusion models.  _IEEE Transactions on Knowledge and Data Engineering_ , pages 1–20, 2024.  doi: 10.1109/TKDE.2024.3361474.
  * Chen et al. [2023] Shoufa Chen, Peize Sun, Yibing Song, and Ping Luo.  Diffusiondet: Diffusion model for object detection.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 19830–19843, 2023.
  * Chen et al. [2019] Tianlong Chen, Shaojin Ding, Jingyi Xie, Ye Yuan, Wuyang Chen, Yang Yang, Zhou Ren, and Zhangyang Wang.  Abd-net: Attentive but diverse person re-identification.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 8351–8361, 2019.
  * Chen et al. [2018] Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Domain adaptive faster r-cnn for object detection in the wild.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3339–3348, 2018.
  * Cho et al. [2022] Yoonki Cho, Woo Jae Kim, Seunghoon Hong, and Sung-Eui Yoon.  Part-based pseudo label refinement for unsupervised person re-identification.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 7308–7318, 2022.
  * Chu et al. [2019] Ruihang Chu, Yifan Sun, Yadong Li, Zheng Liu, Chi Zhang, and Yichen Wei.  Vehicle re-identification with viewpoint-aware metric learning.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 8282–8291, 2019.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _Computer Vision and Pattern Recognition_ , pages 248–255, 2009.
  * Deng [2012] Li Deng.  The mnist database of handwritten digit images for machine learning research [best of the web].  _IEEE signal processing magazine_ , 29(6):141–142, 2012.
  * Ding et al. [2021] Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun.  Repvgg: Making vgg-style convnets great again.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 13733–13742, 2021.
  * Dosovitskiy et al. [2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ , 2020.
  * Fang et al. [2019] Pengfei Fang, Jieming Zhou, Soumava Kumar Roy, Lars Petersson, and Mehrtash Harandi.  Bilinear attention networks for person retrieval.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 8030–8039, 2019.
  * Fu et al. [2021] Dengpan Fu, Dongdong Chen, Jianmin Bao, Hao Yang, Lu Yuan, Lei Zhang, Houqiang Li, and Dong Chen.  Unsupervised pre-training for person re-identification.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 14750–14759, 2021.
  * Goodfellow et al. [2014] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  _Advances in neural information processing systems_ , 27, 2014.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * He et al. [2021] Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang.  Transreid: Transformer-based object re-identification.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 15013–15022, October 2021.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  _Advances in neural information processing systems_ , 33:6840–6851, 2020.
  * Karras et al. [2019] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 4401–4410, 2019.
  * Kingma and Welling [2013] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_ , 2013.
  * Koestinger et al. [2012] Martin Koestinger, Martin Hirzer, Paul Wohlhart, Peter M Roth, and Horst Bischof.  Large scale metric learning from equivalence constraints.  In _Computer Vision and Pattern Recognition_ , pages 2288–2295, 2012.
  * Krizhevsky et al. [2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  _Advances in neural information processing systems_ , 25, 2012.
  * Law and Deng [2018] Hei Law and Jia Deng.  Cornernet: Detecting objects as paired keypoints.  In _Proceedings of the European conference on computer vision (ECCV)_ , pages 734–750, 2018.
  * LeCun et al. [1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.
  * Li et al. [2021] Hanjun Li, Gaojie Wu, and Wei-Shi Zheng.  Combined depth space based architecture search for person re-identification.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 6729–6738, 2021.
  * Li et al. [2022] Jianxin Li, Shuai Zhang, Hui Xiong, and Haoyi Zhou.  Autost: Towards the universal modeling of spatio-temporal sequences.  _Advances in Neural Information Processing Systems_ , 35:20498–20510, 2022.
  * Li et al. [2020] Jing Li, Yu Liu, and Lei Zou.  Dyngcn: A dynamic graph convolutional network based on spatial-temporal modeling.  In _Web Information Systems Engineering–WISE 2020: 21st International Conference, Amsterdam, The Netherlands, October 20–24, 2020, Proceedings, Part I 21_ , pages 83–95. Springer, 2020.
  * Li et al. [2014] Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang.  Deepreid: Deep filter pairing neural network for person re-identification.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 152–159, 2014.
  * Liao and Li [2015] Shengcai Liao and Stan Z Li.  Efficient psd constrained asymmetric metric learning for person re-identification.  In _International Conference on Computer Vision_ , pages 3685–3693, 2015.
  * Liao et al. [2015] Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z Li.  Person re-identification by local maximal occurrence representation and metric learning.  In _Computer Vision and Pattern Recognition_ , pages 2197–2206, 2015.
  * Lin et al. [2015] Kevin Lin, Huei-Fang Yang, Jen-Hao Hsiao, and Chu-Song Chen.  Deep learning of binary hash codes for fast image retrieval.  In _Proceedings of the IEEE conference on computer vision and pattern recognition workshops_ , pages 27–35, 2015.
  * Liu et al. [2016] Hongye Liu, Yonghong Tian, Yaowei Yang, Lu Pang, and Tiejun Huang.  Deep relative distance learning: Tell the difference between similar vehicles.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2167–2175, 2016.
  * Liu et al. [2024] Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu.  Vmamba: Visual state space model.  _arXiv preprint arXiv:2401.10166_ , 2024.
  * Liu et al. [2021] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 10012–10022, 2021.
  * Luo et al. [2021] Hao Luo, Pichao Wang, Yi Xu, Feng Ding, Yanxin Zhou, Fan Wang, Hao Li, and Rong Jin.  Self-supervised pre-training for transformer-based person re-identification.  _arXiv preprint arXiv:2111.12084_ , 2021.
  * Ma et al. [2014] Bingpeng Ma, Yu Su, and Frederic Jurie.  Covariance descriptor based on bio-inspired features for person re-identification and face verification.  _Image and Vision Computing_ , 32(6-7):379–390, 2014.
  * Mirza and Osindero [2014] Mehdi Mirza and Simon Osindero.  Conditional generative adversarial nets.  _arXiv preprint arXiv:1411.1784_ , 2014.
  * Ni et al. [2023] Hao Ni, Yuke Li, Lianli Gao, Heng Tao Shen, and Jingkuan Song.  Part-aware transformer for generalizable person re-identification.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 11280–11289, 2023.
  * Nilsback and Zisserman [2008] Maria-Elena Nilsback and Andrew Zisserman.  Automated flower classification over a large number of classes.  In _2008 Sixth Indian conference on computer vision, graphics & image processing_, pages 722–729. IEEE, 2008.
  * Parkhi et al. [2012] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar.  Cats and dogs.  In _2012 IEEE conference on computer vision and pattern recognition_ , pages 3498–3505. IEEE, 2012.
  * Ramesh et al. [2021] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever.  Zero-shot text-to-image generation.  In _International Conference on Machine Learning_ , pages 8821–8831. PMLR, 2021.
  * Ren et al. [2015] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  _Advances in neural information processing systems_ , 28, 2015.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 10684–10695, 2022.
  * Sohl-Dickstein et al. [2015] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli.  Deep unsupervised learning using nonequilibrium thermodynamics.  In _International conference on machine learning_ , pages 2256–2265. PMLR, 2015.
  * Tian et al. [2023] Junjiao Tian, Lavisha Aggarwal, Andrea Colaco, Zsolt Kira, and Mar Gonzalez-Franco.  Diffuse, attend, and segment: Unsupervised zero-shot segmentation using stable diffusion.  _arXiv preprint arXiv:2308.12469_ , 2023.
  * Van Den Oord et al. [2017] Aaron Van Den Oord, Oriol Vinyals, et al.  Neural discrete representation learning.  _Advances in neural information processing systems_ , 30, 2017.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _Advances in neural information processing systems_ , 30, 2017.
  * Wah et al. [2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie.  The caltech-ucsd birds-200-2011 dataset.  2011\.
  * Wang et al. [2023] Benzhi Wang, Yang Yang, Jinlin Wu, Guo-jun Qi, and Zhen Lei.  Self-similarity driven scale-invariant learning for weakly supervised person search.  _arXiv preprint arXiv:2302.12986_ , 2023.
  * Wang et al. [2019] Guan’an Wang, Yang Yang, Jian Cheng, Jinqiao Wang, and Zengguang Hou.  Color-sensitive person re-identification.  In _International Joint Conference on Artificial Intelligence_ , pages 933–939, 2019.
  * Wang et al. [2020] Guan’an Wang, Shuo Yang, Huanyu Liu, Zhicheng Wang, Yang Yang, Shuliang Wang, Gang Yu, Erjin Zhou, and Jian Sun.  High-order information matters: Learning relation and topology for occluded person re-identification.  2020\.
  * Wang et al. [2018] Guanshuo Wang, Yufeng Yuan, Xiong Chen, Jiwei Li, and Xi Zhou.  Learning discriminative features with multiple granularities for person re-identification.  In _Proceedings of the 26th ACM international conference on Multimedia_ , pages 274–282, 2018.
  * Wang et al. [2022] Haochen Wang, Jiayi Shen, Yongtuo Liu, Yan Gao, and Efstratios Gavves.  Nformer: Robust person re-identification with neighbor transformer.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 7297–7307, 2022.
  * Wei et al. [2018] Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian.  Person transfer gan to bridge domain gap for person re-identification.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 79–88, 2018.
  * Weyand et al. [2020] Tobias Weyand, Andre Araujo, Bingyi Cao, and Jack Sim.  Google landmarks dataset v2-a large-scale benchmark for instance-level recognition and retrieval.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 2575–2584, 2020.
  * Xu et al. [2022] Pei Xu, Jean-Bernard Hayet, and Ioannis Karamouzas.  Socialvae: Human trajectory prediction using timewise latents.  In _European Conference on Computer Vision_ , pages 511–528. Springer, 2022.
  * Yang et al. [2014] Yang Yang, Jimei Yang, Junjie Yan, Shengcai Liao, Dong Yi, and Stan Z Li.  Salient color names for person re-identification.  In _European conference on computer vision_ , pages 536–551, 2014.
  * Ye et al. [2021] Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi.  Deep learning for person re-identification: A survey and outlook.  _IEEE transactions on pattern analysis and machine intelligence_ , 44(6):2872–2893, 2021.
  * Zhang et al. [2020] Zhizheng Zhang, Cuiling Lan, Wenjun Zeng, Xin Jin, and Zhibo Chen.  Relation-aware global attention for person re-identification.  In _Proceedings of the ieee/cvf conference on computer vision and pattern recognition_ , pages 3186–3195, 2020.
  * Zhao et al. [2017] Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi.  Learning discourse-level diversity for neural dialog models using conditional variational autoencoders.  _arXiv preprint arXiv:1703.10960_ , 2017.
  * Zheng et al. [2015] Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian.  Scalable person re-identification: A benchmark.  In _International Conference on Computer Vision_ , pages 1116–1124, 2015.
  * Zheng et al. [2013] Wei-Shi Zheng, Shaogang Gong, and Tao Xiang.  Reidentification by relative distance comparison.  _IEEE transactions on pattern analysis and machine intelligence_ , 35(3):653–668, 2013.
  * Zheng et al. [2017] Zhedong Zheng, Liang Zheng, and Yi Yang.  Unlabeled samples generated by gan improve the person re-identification baseline in vitro.  _arXiv preprint arXiv:1701.07717_ , 2017.
  * Zhou et al. [2019] Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang.  Omni-scale feature learning for person re-identification.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 3702–3712, 2019.
  * Zhu et al. [2020] Kuan Zhu, Haiyun Guo, Zhiwei Liu, Ming Tang, and Jinqiao Wang.  Identity-guided human semantic parsing for person re-identification.  In _European Conference on Computer Vision_ , pages 346–363. Springer, 2020.

##  Appendix A Experimental settings

Datasets and Evaluation metrics. We conducted training and evaluation on four
datasets: DukeMTMC-reID [65], Market-1501 [63], MSMT17 [56], and CUHK-03 [30].
These datasets encompass a wide range of scenarios for person re-
identification. For accuracy, we use standard metrics including Rank-1 curves
(The probability that the image with the highest confidence in the search
results is the correct result.) and mean average precision (MAP). All the
results are from a single query setting.

Implementation Details. We implemented our method using Python on a server
equipped with a 2.10GHz Intel Core Xeon (R) Gold 5218R processor and two
NVIDIA RTX 3090 GPUs. The epochs we trained are set to 120, the learning rate
is set to 0.0004, the batch size during training is 64, the inference stage is
256, and the diffusion step size $t$ is set to 1000.

Training and evaluation. To better constrain the performance of the denoised
features of the FEFDFA for downstream tasks, we used alternate fine-tune
methods. The parameters of the FEFDFA and baseline were trained alternately,
and when training a part of the parameters, the rest of the parameters were
frozen and fine-tuned for 10 epochs at a time, with a total number of epochs
of 120. When evaluating, we averaged the results of the experiments under the
same settings for 5 times, thus ensuring the reliability of the data.

##  Appendix B Experiment on Vehicle Identification

In the image retrieval task, we also conducted experiments to verify the
effectiveness of our method on the vehicle recognition task. Vehicle
recognition in practical scenarios often results in images containing a large
amount of noise due to environmental factors such as lighting or occlusion,
which increases the difficulty of detection. Our method is based on denoising
to obtain features with better representation ability. Therefore, we want to
experimentally verify whether the FEFDFA module plays a role in vehicle
recognition tasks with higher noise levels. We selected vehicleID [34] as the
dataset, vehicle-ReID [10] as the baseline, and ResNet50 as the feature
extractor for the experiment.

Table 5: The performance of the FEFDFA module in vehicle recognition tasks. | Method
---
| Backbone
---
| Datasets
---
| mAP
---
| Rank-1
---
vehicle-ReID | ResNet50 | vehicleID | 76.4% | 69.1%
vehicle-ReID(FEFDFA) | ResNet50 | vehicleID | 77.3% | 70.2%

From the results in Table 5, it can be seen that FEFDFA has excellent
performance in vehicle detection tasks. Compared to the baseline, adding the
FEFDFA module significantly improves both mAP and Rank-1 metrics without
incurring additional detection time costs. Verified the denoising ability of
the FEFDFA module in noisy environments.

##  Appendix C Experiment on Large Scale Image Classification Tasks

In this section, we aim to test the generalization ability of FEFDFA in other
tasks. We conducted experiments on two image classification datasets,
ImageNet-1k and Cifar-10. These two datasets are both classic image
classification datasets, rich in common images in daily life, and belong to
large-scale image databases. ImageNet-1k is a subset of the ImageNet dataset,
containing images from 1000 categories. Each category typically has hundreds
to thousands of images, totaling over one million images. The Cfiar-10
contains 60000 32x32 pixel color images, divided into 10 categories. Each
category contains 6000 images. To evaluate the effectiveness of our method, we
use standard metrics, including Top-1 accuracy and Top-5 accuracy, which are
commonly used to evaluate model performance in image classification tasks and
are widely used on various image datasets, and we conducted detailed
comparative experiments on multiple backbones and models with different
parameter versions to verify the reliability of our method.

Table 6: The effectiveness of our method in image classification tasks was validated on Cifar-10 and ImageNet-1k. | Method
---
| Datasets
---
| param
---
| acc@1
---
| acc@5
---
| acc@1(ours)
---
| acc@5(ours)
---
SwinV2-T [36] | ImageNet-1k | 28M | 81.82% | 95.88% | 82.13% | 96.06%
SwinV2-S [36] | ImageNet-1k | 50M | 83.73% | 96.62% | 83.97% | 96.86%
SwinV2-B [36] | ImageNet-1k | 88M | 84.20% | 96.93% | 84.31% | 97.06%
Vmanba-T [35] | ImageNet-1k | 30M | 82.38% | 95.80% | 82.51% | 95.89%
Vmanba-S [35] | ImageNet-1k | 50M | 83.12% | 96.04% | 83.27% | 96.22%
Vmanba-B [35] | ImageNet-1k | 89M | 83.83% | 96.55% | 83.91% | 96.70%
ResNet50 [18] | ImageNet-1k | 26M | 76.13% | 92.86% | 76.28% | 92.95%
ViT-S [14] | Cifar-10 | 22M | 96.13% | – | 96.20% | –
ViT-B [14] | Cifar-10 | 87M | 98.02% | – | 98.31% | –

As shown in Table 6, We compared multiple classic backbones for representation
learning on two datasets, and after adding the FEFDFA module, the accuracy
metrics improved without adding model parameters. It has been proven that our
method can enhance the model’s representation learning ability and obtain more
effective features through denoising without increasing additional time costs.
Moreover, this method can be well generalized to other image tasks.

##  Appendix D Limitations

Our proposed method FEFDFA improves the accuracy of current mainstream
backbones while ensuring label-free and no additional computational costs, and
it has been experimentally verified to be generalizable in multiple image
tasks. However, from the experimental results, it can be found that our method
has limited improvement in model accuracy when generalized to general tasks,
and in order to fuse the parameters of the denoising layer and the feature
extraction layer, only one or two steps of denoise for each denoising layer,
and the number of denoising layers is not more than that of the feature
extraction layer, which limits the denoising intensity. We will continue to
explore how to further improve the accuracy of the model without adding
additional inference time costs or only adding a small number of additional
parameters.