# AggMatch: Aggregating Pseudo Labels for Semi-Supervised Learning

Jiwon Kim∗, Kwangrok Ryoo∗, Gyuseong Lee, Seokju Cho, Junyoung Seo,
Daehwan Kim, Hansang Cho, and Seungryong Kim† ∗Equal
contribution†Corresponding authorJ. Kim, K. Ryoo, G. Lee, J. Seo, and S. Kim
are with the Department of Computer Science and Engineering, Korea University,
Seoul 02841, Korea. (E-mail: {naancoco, kwangrok21, jpl358, se780,
seungryong_kim}@korea.ac.kr).S. Cho is with the Department of Computer Science
and Engineering, Yonsei University, Seoul 03722, Korea. (E-mail:
hamacojr@yonsei.ac.kr).D. Kim and H. Cho are with Samsung Electro-Mechanics,
Suwon 16674, Korea. (E-mail<EMAIL_ADDRESS>hansang.cho@samsung.com).

###### Abstract

Semi-supervised learning (SSL) has recently proven to be an effective paradigm
for leveraging a huge amount of unlabeled data while mitigating the reliance
on large labeled data. Conventional methods focused on extracting a pseudo
label from individual unlabeled data sample and thus they mostly struggled to
handle inaccurate or noisy pseudo labels, which degenerate performance.

In this paper, we address this limitation with a novel SSL framework for
aggregating pseudo labels, called AggMatch, which refines initial pseudo
labels by using different confident instances. Specifically, we introduce an
aggregation module for consistency regularization framework that aggregates
the initial pseudo labels based on the similarity between the instances. To
enlarge the aggregation candidates beyond the mini-batch, we present a class-
balanced confidence-aware queue built with the momentum model, encouraging to
provide more stable and consistent aggregation. We also propose a novel
uncertainty-based confidence measure for the pseudo label by considering the
consensus among multiple hypotheses with different subsets of the queue. We
conduct experiments to demonstrate the effectiveness of AggMatch over the
latest methods on standard benchmarks and provide extensive analyses.

###### Index Terms:

Semi-supervised learning, image classification, uncertainty estimation

##  I Introduction

Recently, semi-supervised learning (SSL), an approach for learning a model on
a large amount of unlabeled data with few labeled data, has become a promising
solution to mitigate the reliance on large labeled dataset in numerous image
processing and computer vision tasks [1, 2, 3, 4, 5]. Unlike the supervised
learning paradigm [6, 7, 8, 9] that relies on ground-truth labels, SSL poses
additional challenges in that good pseudo labels should be extracted from
unlabeled data and used to learn the model.

Figure 1: Intuition of our AggMatch in comparison to FixMatch [4]. By
aggregating initial pseudo labels, AggMatch generates more confident pseudo
labels, which boosts the classification performance.

Most recent SSL approaches mainly followed two trends; pseudo-labeling which
encourages the model to follow the pseudo label from model’s prediction itself
[10, 11, 12, 13, 14, 15], which is closely related to entropy minimization
[16, 17] and consistency regularization which enables the model to produce the
same prediction when perturbations are applied to the input [18, 19, 20, 21,
22, 23, 12, 24, 3] or the model [25, 19, 20, 23, 22, 24, 3, 26]. Very
recently, advanced consistency regularization methods, called holistic
approach, have been presented by integrating the two above paradigms in a
unified framework. For instance, FixMatch [4] first generates a pseudo label
using the model’s prediction on weakly-augmented unlabeled data and then
encourages the prediction from strongly-augmented unlabeled data to follow the
pseudo label with a confidence threshold. Although FixMatch [4] serves as one
of the state-of-the-art frameworks [27, 28, 4, 29, 30], it inherits the
limitations of pseudo-labeling, i.e., confirmation bias and noise sensitivity.
In addition, confidence, measured by only considering the prediction itself as
in FixMatch [4], has limited discriminative power and often cannot catch false
positives at the early stage of training, which may be confident pseudo label
candidates, thus hindering performance boosting.

TABLE I: Summary of our key results on standard CIFAR-10 [31] with 40 labels
and on asymmetric label noise setting.

Datasets |  | CIFAR-10 [31]
---
(40 labels)
| CIFAR-10 [31]
---
(noise 50% + 250 labels)
| FixMatch [4]
---
(previous SOTA)
86.19$\pm$3.37 | 79.68$\pm$4.84
| AggMatch
---
(Ours)
92.54$\pm$0.76 |  85.41$\pm$7.07
|  |

Leveraging the relationships between different instances would be an
alternative solution to address the aforementioned limitations of recent SSL
methods [27, 28, 4, 29, 30], by refining the pseudo label and measuring its
confidence. Interestingly, traditional SSL approaches such as label
propagation [32, 33, 34] that propagate the class distribution of labeled data
to unlabeled data by considering the manifold structure also leverage the
relationships between different instances, but their scalability is limited as
the graph should be built for all the data, and they are not differentiable to
be learned. Very recently, PAWS [35] attempted to propagate the class
distribution of labeled data to unlabeled data within the current mini-batch,
similar to label propagation [36, 37], but it highly relies on a large amount
of labeled data, thereby limiting the performance on label-scarce scenarios or
with noisy labeled data.

In this paper, we present a novel SSL framework, called AggMatch, that uses
the relationships between different instances to achieve more confident pseudo
label as shown in Fig. 1. Inspired by cost aggregation in stereo matching
literature [38, 39] and self-attention module in Transformers [40], we present
an aggregation module that aggregates initial pseudo labels from labeled and
unlabeled data by considering the affinity between the instances, which is
measured by using both the feature and class distribution itself. To enlarge
the aggregation candidates beyond the current mini-batch, we utilize a queue
that memorizes information of previous batch samples during training, allowing
for improved scalability, as in [41, 42, 9]. In addition, we present a class-
balanced confidence-aware queue, built by considering the confidence and class
distribution and updated with a momentum, which encourages better aggregation.
Finally, we also present a novel confidence measure for the pseudo label by
measuring the consensus among possible pseudo labels with multiple disjoint
subsets of the queue, which helps to improve the robustness to noise.
Experiments on standard SSL benchmarks for image classification [31, 43] and
with noisy settings [44, 45] prove the effectiveness of our approach over the
latest methods. As summarized in Table I, we show the new state-of-the-art
performance on CIFAR-10 [31] with 40 labels and with 50% noise on 250 labels,
which is a large margin of improvement compared to FixMath [4], one of the
state-of-the-arts.

The rest of this paper is organized as follows. We discuss related works in
Sec. II and introduce the background in Sec. III. Sec. IV presents the details
of our algorithm. We conduct experiments on extensive SSL benchmarks and
provide an ablation study to validate and analyze components of our approach
in Sec. V. In Sec. VI, we conclude this paper.

##  II Related Work

In this section, we first review existing methods for SSL, which are
categorized as pseudo-labeling, consistency regularization, and label
propagation methods. We then review uncertainty estimation for SSL.

###  II-A Pseudo-Labeling

Pseudo-labeling, also known as self-training [14], is based on the idea of
utilizing the model’s prediction itself to explicitly generate an artificial
label for unlabeled data as shown in Fig. 2(a). The pseudo labels from the
model predictions can be defined as soft (continuous distribution) or hard
(one-hot distribution) pseudo labels [10, 11, 12, 36, 13, 14, 15]. More
specifically, a soft and hard version of pseudo-labeling can be designed by
sharpening the predicted distribution and introducing argmax of the
prediction, respectively. It is closely related to entropy minimization [16,
17], where the model’s predictions are encouraged to be low-entropy (i.e.,
high-confidence) on unlabeled data. Specifically, they [10, 11, 36, 13] and
earlier related methods [46, 47, 48] first trained a model on labeled samples,
and then fined-tuned the pre-trained model using the additional pseudo labels.
It is also extended to an explicit teacher-student configuration [3],
formulated by a pair of networks. In general, a teacher network generates
pseudo labels for unlabeled samples, which are used for training a student
network. Although pseudo-labeling is simple and intuitive, the initialized
model is vulnerable to bias from the limited amount of labeled data. In
addition, it heavily relies on the quality of the model’s prediction, thus
suffering from confirmation bias [13], where the prediction errors would
accumulate and worsen the model performance.

(a) Pseudo-Labeling (b) Consistency Regularization

(c) Label Propagation (d) Ours

Figure 2: Motivation of our approach: (a) pseudo-labeling methods [10, 20, 3,
15], (b) consistency regularization methods [18, 19], (c) label propagation
methods [36, 37, 35], and (d) ours.

###  II-B Consistency Regularization

Consistency regularization is based on the manifold assumption or the
smoothness assumption [49] and consequently, it can be regarded to find a
smooth manifold given few labeled and a large amount of unlabeled data. As
exemplified in Fig. 2(b), it constraints consistency between model predictions
after injecting different noise to the input, hidden states or model
parameters, assuming to keep the class semantics unaffected. This idea was
first proposed in [50] and popularized by [18, 19]. Specifically, methods
using stochastic data augmentation as perturbation can be defined as follows;
domain-specific data augmentation [19, 51, 27, 28], stochastic regularization
[18, 19], random max pooling [18], or adversarial transformation [21]. On the
other hand, other methods apply the perturbation on the model, such as Dropout
[25, 19, 20, 23, 22, 3], running average or time ensemble of past model
predictions at different time step (EMA or SWA) [19, 20, 24, 52], stochastic
depth [52] or an adversarial perturbation on model’s parameter (DropConnect)
[26]. Both L2 norm [18, 19, 20] and cross entropy [21, 12, 28, 4] are
available as unsupervised loss function for consistency regularization. A
further extension of the consistency regularization methods, also called a
holistic approach, combines pseudo-labeling and consistency regularization in
a unified loss function [27, 28, 4, 29]. However, the limitation of those
consistency regularization methods is that they primarily focus on the
relationship between differently augmented samples for one instance itself
where once wrong pseudo labeling occurs, model can be overfitted and
contaminated.

###  II-C Label Propagation

The key assumption in label propagation is that data points in the same
manifold can be considered to share the same semantic label distribution [32].
This can be closely related to the manifold assumption of graph-based learning
algorithms in which nodes consist of the labeled and unlabeled samples, and
edges are weighted by the similarity between those nodes. The label
propagation aggregates pairwise similarity between features and class
probability to infer pseudo labels, as illustrated in Fig. 2(c).
Traditionally, it was performed in a transductive setting [53, 34],
propagating labels within a given set of unlabeled examples or in an inductive
setting [54, 20], generalizing inference to new unseen data, while the
original training data are discarded.

Recently, some methods [36, 37] integrated label propagation in an iterative
way with pseudo-labeling. They alternate between training the network with
pseudo labels and constructing the affinity graph of nearest neighbors based
on the similarity between the feature representations. Recent methods in
consistency regularization [35, 30] combine the aggregation technique based on
similarity of a mini-batch of labeled or other unlabeled samples to define the
aggregated pseudo labels. However, their similarity measurements are
implemented between in-batch samples, and then can limits performance
depending on the size of the mini-batches.

###  II-D Uncertainty Estimation

Estimating the uncertainty over the predictions of neural networks [55, 56,
57] often aims to regress a distribution of output [58, 59, 60, 61].
Uncertainty estimation can be divided into two approaches; empirical and
predictive approach. In empirical methods, it can be approximated by sampling
a finite number of weight configurations for a given network and computing the
mean and variance of the predictions [62, 63, 64]. On the other hand, output
distributions can also be trained by predictive methods, i.e., a Laplacian or
Gaussian distribution [58].

For classification in SSL, the idea of confidence can be represented as a
class probability of pseudo-labeling in general. A confidence-based strategy
was previously used in [65, 51] along with pseudo labeling, encouraging the
model to use samples with high confidence (i.e., low entropy). Recently, such
confidence strategy is implemented by thresholding by leaving samples above a
pre-defined threshold and filtering out noisy unlabeled data as proposed in
UDA [12] and FixMatch [4].

However, the pre-defined constant threshold fails to consider different
learning status per training iteration and learning difficulty level per
sample, leading to slow convergence speed. To solve the trade-off between the
quality and the quantity of pseudo labels, we combine uncertainty-based
confidence on the unsupervised loss to manipulate the loss magnitude
dynamically depending on the uncertainty estimation.

Figure 3: Network configuration: For differently perturbed images by weak and
strong augmentations, we regularize the model to generate consistent
predictions between them. Class distributions of the weakly augmented images
are refined by our aggregation module with the help of a large and consistent
queue. Our confidence estimator estimates the unreliable samples voted to be
uncertain from the hypotheses generated from the randomly and evenly
partitioned queue.

##  III Preliminaries

Let us define a batch of labeled instances as
$\mathcal{X}=\\{(x_{b},y_{b}):b\in(1,...,B)\\}$, where $x_{b}$ is an instance
and $y_{b}$ is a label representing one of $Y$ labels. In addition, let us
define a batch of unlabeled instances as $\mathcal{U}=\\{u_{b}:b\in(1,...,\mu
B)\\}$, where $\mu$ is a hyper-parameter that determines the size of
$\mathcal{U}$ relative to $\mathcal{X}$. The objective of SSL is to
simultaneously use $\mathcal{X}$ and $\mathcal{U}$ to train a model
$p_{\mathrm{model}}(y|r;\theta)$ taking an instance $r$ as input and
outputting a distribution over class labels $y$ with parameters $\theta$,
where the learned model is expected to yield better performance than solely
using $\mathcal{X}$. In general, $p_{\mathrm{model}}$ consists of two parts;
feature extractor, which extracts a feature $\mathbf{v}$ from an instance $r$,
and classifier, which measures a class probability $\mathbf{p}$ from
$\mathbf{v}$.

For SSL, consistency regularization methods [50, 18, 19] rely on the
assumption that the model should generate similar predictions when perturbed
versions of the same instance are fed. FixMatch [4] extends this by performing
two types of augmentations such as weak and strong, denoted by $\alpha(\cdot)$
and $\mathcal{A}(\cdot)$, and enforcing the consistency between them with
pseudo-labeling [10]. The pseudo label is always obtained from _weakly_
-augmented version of an unlabeled instance, which enforces the consistency to
the model’s output for _strongly_ -augmented version of the same instance. The
loss function is defined as

$\frac{1}{\mu B}\sum_{b=1}^{\mu
B}{c_{b}\mathcal{D}(\mathbf{q}_{b},p_{\mathrm{model}}(y|\mathcal{A}(u_{b});\theta))},$
(1)

where a cross-entropy is used for $\mathcal{D}$, and
$\mathbf{p}_{b}=p_{\mathrm{model}}(y|\alpha(u_{b});\theta)$ denotes a class
probability, ${c}_{b}$ denotes a confidence of $\mathbf{p}_{b}$ such that
${c}_{b}=\mathbbm{1}(\mathrm{max}(\mathbf{p}_{b})\geq\tau)$. More
specifically, the pseudo label $\mathbf{q}_{b}$ for $\mathbf{p}_{b}$ can be
defined as continuous distribution (soft) based on sharpening or one-hot
distribution (hard) based on argmax operation, respectively.

One of main limitations of existing SSL methods, including consistency
regularization [18, 19, 20], pseudo-labeling [46, 47, 48, 10] and holistic
approaches [27, 28], is that they can only generate confident pseudo labels
when a model is sufficiently trained. In this case, the immature model at the
early stage of the training may be significantly degraded depending on the
quality of pseudo-labeling. For instance, if the labeled data $\mathcal{X}$
has different distribution with unlabeled data $\mathcal{U}$, the size of
$\mathcal{X}$ is not enough, or $\mathcal{X}$ and $\mathcal{U}$ are
contaminated by noise and bias, the pseudo-labeling overfits to incorrect
pseudo labels, which leads to optimizing the model’s parameters to undesirable
directions, also known as the confirmation bias [13]. To prevent the
aforementioned problem, the confidence-based strategy [65, 51] is combined
with pseudo-labeling to focus on pseudo labels with high confidence by setting
a predefined confidence threshold [12, 4]. However, estimating the confidence
of a pseudo label with a simple threshold can be too strong constraint in
early iterations of training. A huge amount of true positive samples may be
discarded as false positive without considering the inherent learning
difficulties of different samples and learning status during the training.

##  IV Our Algorithm: AggMatch

###  IV-A Motivation and Overview

In this section, we address the limitations aforementioned of conventional SSL
methods [10, 18, 19, 20, 27, 28, 12, 4] by refining an initial class
distribution individually extracted from a weakly-augmented version,
$p_{\mathrm{model}}(y|\alpha(u);\theta)$. Based on the observation that
similar instance should have similar class distribution [34], we present a
novel semi-supervised learning framework that aggregates initial class
distributions both with labeled data $\mathcal{X}$ and unlabeled data
$\mathcal{U}$ to provide better pseudo labels for a _strongly_ -augmented
version, $p_{\mathrm{model}}(y|\mathcal{A}(u_{b});\theta)$.

In this paper, we mainly study how to accurately measure the similarity
between instances and how to effectively aggregate the class distributions
from the candidates by considering the similarities. Inspired by cost
aggregation in stereo matching literature [38, 39] and self-attention module
in Transformers [40], we present to measure the similarity of feature
embedding by means of query, key and value. A query can be a feature
$\mathbf{v}_{b}$ of an instance and keys can be other features
$\mathbf{v}_{l}$ in $\mathcal{X}$ and $\mathcal{U}$, as well as class
distribution $\mathbf{p}_{b}$ and $\mathbf{p}_{l}$, respectively. The
corresponding initial class distributions $\mathbf{p}_{l}$ are then considered
as values. A naïve framework [35, 30] is to let this procedure perform within
a current batch. The quality of aggregated distribution highly depends on the
size of the batch, but a large batch size cannot be guaranteed by limited GPU
memories. To resolve this issue, we utilize a queue that memorizes information
of previous batch samples during training. By allowing that, the confident
samples are selectively enqueued with a momentum model, and the queue slowly
and stably changes, enabling consistent propagation. We also measure the
confidence of pseudo label by measuring the consensus among its multiple
hypotheses with multiple subsets of the queue. Our overall network
architecture is illustrated in Fig. 3.

###  IV-B Class Distribution Aggregation

A class distribution independently predicted from an individual instance often
contains ambiguous values and often generates erroneous pseudo labels. To
remedy this, we propose an aggregation module that aims to refine the
ambiguous or noisy class distributions utilizing the relationship between
other confidence-aware samples. Specifically, a class distribution
$\mathbf{p}_{b}$ for $u_{b}$ is aggregated by $\mathbf{p}_{l}$ of other
instances $r_{l}$ from the set of $\mathcal{X}$ and $\mathcal{U}$, by
considering the similarity between them as

$\overline{\mathbf{p}}_{b}=\sum_{l}\left(\frac{\exp(\mathcal{S}(u_{b},r_{l})/\tau_{\mathrm{sim}})}{{\sum_{j}}\exp(\mathcal{S}(u_{b},{r_{j}})/\tau_{\mathrm{sim}})}\right)\mathbf{p}_{l},$
(2)

where $l$ and $j$ are indexes for all the samples in the set of $\mathcal{X}$
and $\mathcal{U}$, and $\tau_{\mathrm{sim}}$ is a temperature. Especially, the
similarity function $\mathcal{S}$ measures attention weights between the
instances. Intuitively, one straightforward way is to only consider class
distribution $\mathbf{p}$ based on a class similarity term, but it disregards
the noise involved in the class distribution. We present to additionally use a
feature similarity term for feature affinity between $u_{b}$ and $r_{l}$
defined such that

$\mathcal{S}(u_{b},r_{l})=(\mathbf{v}_{b}\cdot\mathbf{v}_{l})/\|\mathbf{v}_{b}\|\|\mathbf{v}_{l}\|+\lambda_{\mathrm{sim}}\mathrm{JS}\left(\mathbf{p}_{b}\|\mathbf{p}_{l}\right),$
(3)

where the first term denotes a feature similarity and the second term denotes
a class similarity, and $\lambda_{\mathrm{sim}}$ is a weight parameter. We
measure similarity between features by cosine similarity [29, 35]. We use
Jensen-Shannon distance [66] $\mathrm{JS}(\cdot\|\cdot)$ for measuring the
similarity between distribution $\mathbf{p}_{b}$ and $\mathbf{p}_{l}$. Thanks
to this attention, unlike conventional consistency regularization methods [18,
19, 20, 12], ignoring relationships between different instances, our
aggregation explicitly exploits such relationships to generate more confident
pseudo labels and thus enforces more stable convergence.

Figure 4: Illustration of confidence estimation for pseudo label: We define
the confidence of pseudo label as consensus among multiple hypotheses of
pseudo label. In specific, the queue is evenly and randomly partitioned as $M$
disjoint subsets to generate the hypotheses. We build aggregated pseudo labels
using multiple disjoint subsets of the queue. We get a one-hot hard pseudo
label and then apply the voting process for all $M$ predictions to achieve
empirical probability, where entropy is used as the confidence of pseudo
label.

###  IV-C Class-Balanced Confidence-Aware Queue

An aggregation among current batches $\mathcal{X}$ and $\mathcal{U}$ may be
sensitive to the quality of sampled instances in the batch, which prohibits
stable convergence. To alleviate this, we exploit a queue that memorizes
information of previous batch samples during training, which helps enlarge the
aggregation candidates. In specific, at each training step, current batches
are enqueued and the oldest are dequeued, enabling the queue to impose
ensembles of knowledge among all the samples implicitly. To prevent the
propagation of noisy samples, we leave out low confident predictions below a
threshold $\tau$. In other fields such as depth completion [38, 39], it has
also been proven that propagating only confident samples yields better
performance. The queue is formally defined as
$\mathcal{Q}=\\{(\mathbf{v}_{l},\mathbf{p}_{l})\\}$. Note that PAWS [35] also
tried to exploit the similarity with labeled samples, but the propagation only
relies on very sparse labeled samples. Unlike this, thanks to the proposed
confident-aware queue, we can easily enlarge the candidate samples by using
the unlabeled samples.

####  IV-C1 Class Balancing

The use of samples in a class-imbalanced queue for aggregation may suffer from
a biased prediction problem. We design a class-balanced queue to overcome
this, where each sample is enqueued according to its pseudo label prediction.
The queue for class $y$ is defined as
$\mathcal{Q}_{y}=\\{(\mathbf{v}_{y,l},\mathbf{p}_{y,l}):l\in(1,...,L)\\}$ with
the number of samples per class is $L$, which is same for all classes.

####  IV-C2 Momentum Update

Using the queue can help enlarge the aggregation candidates, but directly
updating the queue may cause consistency problems because measuring attentions
between inconsistent features from rapidly updated feature extractor can be
poorly operated as in [41, 42, 9]. To address this issue, we propose a
momentum technique for queue update, which mitigates the problem of
inconsistency between features from the queue and current batch. The momentum
model with parameter $\theta_{m}$ is manipulated by a momentum coefficient
$\lambda_{m}$ and model parameter $\theta$ such that

$\theta_{m}\leftarrow\lambda_{m}\theta_{m}+(1-\lambda_{m})\theta.$ (4)

It should be noted that Mean Teacher [20] also uses an exponential moving
average of model parameters, which was empirically found to improve results.
Compared to this, we differ in that we only use this for updating the queue.

###  IV-D Confidence Estimation for Pseudo Label

SSL generally does not take the model uncertainty into account. FixMatch [4]
uses a simple thresholding technique as confidence measure for pseudo label,
but it requires a hand-tuned threshold parameter and also imposes overly
strong constraint in early iterations so that most possible confident pseudo
labels would be rejected due to their class probability scores below the
threshold.

To tackle the aforementioned issues, we present to measure a confidence of
aggregated soft pseudo label $\overline{\mathbf{p}}_{b}$, based on the
consensus among multiple hypotheses of pseudo labels. To generate those
hypotheses, the queue $\mathcal{Q}$ is evenly and randomly partitioned as $M$
disjoint subsets at each class, i.e., $\mathcal{Q}^{m}$ for $m=1,...,M$. Given
the subsets $\mathcal{Q}^{m}$, similar to Eq. (2), we aggregate to achieve
$\overline{\mathbf{p}}^{m}_{b}$. Aggregated class probabilities then undergo a
voting process. In specific, we sum up one-hot representation of each
prediction such that
$\mathbf{a}_{b}=\frac{1}{M}\sum\nolimits_{m}{\textbf{e}^{m}_{b}}$, where
$\textbf{e}^{m}_{b}$ denotes a one-hot vector by operating argmax on
$\overline{\mathbf{p}}^{m}_{b}$. From such empirical probability
$\mathbf{a}_{b}$, which represents the occurrence of each class in the
hypotheses, we measure the confidence $\overline{c}_{b}$ of the pseudo label
by an entropy [67] such that
$\overline{c}_{b}=\mathrm{exp}\left(\sum\mathbf{a}_{b}\log{\mathbf{a}_{b}}\right)$.
The final pseudo label $\overline{\mathbf{p}}_{b}$ can be simply achieved by
averaging $\overline{\mathbf{p}}^{m}_{b}$ for $m$. By estimating the accurate
confidence of pseudo label, our unsupervised loss function enables boosting
the performance for SSL even in early stages of training. Our overall
confidence estimation process is described in Fig. 4.

###  IV-E Loss Functions

Our loss function for unlabeled batch is defined with the aggregated pseudo
label $\overline{\mathbf{q}}_{b}$ and confidence prediction $\overline{c}_{b}$
as

$\mathcal{L}_{\mathcal{U}}=\frac{1}{\mu B}\sum_{b=1}^{\mu
B}{\overline{c}_{b}\mathcal{D}(\overline{\mathbf{q}}_{b},p_{\mathrm{model}}(y|\mathcal{A}(u_{b});\theta))},$
(5)

where the pseudo label $\overline{\mathbf{q}}_{b}$ is generated by sharpening
$\overline{\mathbf{p}}_{b}$ with temperature scaling $T$ like [12, 28]. A
supervised loss for labeled batch is also defined such that

$\mathcal{L}_{\mathcal{S}}=\frac{1}{B}\sum_{b=1}^{B}\mathcal{D}(y_{b},p_{\mathrm{model}}(y|\alpha(x_{b});\theta)).$
(6)

where $y_{b}$ is the ground-truth label. Our final loss is as follows:
$\mathcal{L}=\mathcal{L}_{\mathcal{S}}+\lambda\mathcal{L}_{\mathcal{U}}$ with
unsupervised weight parameter $\lambda$. Algorithm 1 presents the pseudo-code
of AggMatch.

###  IV-F Relationship with Cognitive Development

Effectiveness and robustness of AggMatch can be explained through the theory
of cognitive development. Vygotsky classified the developmental zone into the
zone of actual development and the out of reach zone, where learners can solve
the task on their own and with the help of the more knowledgeable others [68],
respectively. Vygotsky also defined the gap between them as the zone of
proximal development. The zone of actual development is learned up to the out
of reach zone through elaborate scaffolding. When good learning takes place,
that is, when the current out of reach zone is changed to the actual zone of
development, new out of reach zone and new zone of proximal development are
formed.

Similarly, it can be explained that the label propagation-based methods, such
as PAWS [35], start from the labeled samples and gradually learn the unlabeled
data points adjacent to the labeled data points in the feature space
corresponding to the zone of proximal development. However, according to
Vygotsky’s theory of cognitive development, expanding the zone of actual
development to the out of reach zone is only possible with the precise
guidance of the teacher. For SSL with few labeled data, scaffolding is not
sufficient. To solve this problem, many methods form a teacher-student
relationship through an asymmetric structure. However, if the teacher is not
competent enough, it may not work properly. AggMatch uses a contemplative
encoder as a teacher and an impulse encoder as a student according to Kagan’s
cognitive theory [69] for more sophisticated scaffolding. The contemplation
type encoder is set as a momentum encoder to produce more stable results, and
at the same time, the results are thoroughly reviewed through the aggregation
module. Therefore, AggMatch can expand the actual zone of development well
through more sophisticated scaffold settings.

Algorithm 1 AggMatch

Notation: confidence threshold $\tau$, weak augmentation $\alpha$,
strong augmentation $\mathcal{A}$, queue $\mathcal{Q}$, model
$p_{\mathrm{model}}$,
model parameter $\theta$, momentum model parameter $\theta_{m}$

Input:labeled batch $\mathcal{X}=\\{(x_{b},y_{b})\\}_{b=1}^{B},$ unlabeled
batch
$\mathcal{U}=\\{u_{b}\\}_{b=1}^{\mu{B}}$

1: for $b=1$ to $\mu B$ do

2: $\mathbf{v}_{b,w},\mathbf{p}_{b,w}\leftarrow
p_{\mathrm{model}}(\alpha(\mathit{u}_{b});\theta)$

3: $\mathbf{v}_{b,s},\mathbf{p}_{b,s}\leftarrow
p_{\mathrm{model}}(\mathcal{A}(\mathit{u}_{b});\theta)$

4: $\mathbf{v}_{b,w}^{\mathit{m}},\mathbf{p}_{b,w}^{\mathit{m}}\leftarrow
p_{\mathrm{model}}(\alpha(\mathit{u}_{b});\theta_{m})$

5: if $\max{\mathbf{p}_{b,w}^{m}}\geq\tau$ then

6:
$\text{Enqueue}(\mathcal{Q},(\mathbf{v}_{b,w}^{\mathit{m}},\mathbf{p}_{b,w}^{\mathit{m}}))$

7: end if

8: end for

9:

10: for $b=1$ to $B$ do

11: $\mathbf{v}_{b,w}^{m,l},\mathbf{p}_{b,w}^{m,l}\leftarrow
p_{\mathrm{model}}(\alpha(\mathit{x}_{b});\theta_{m})$

12: $\mathbf{v}_{b,w}^{l},\mathbf{p}_{b,w}^{l}\leftarrow
p_{\mathrm{model}}(\alpha(\mathit{x}_{b});\theta)$

13: $(\mathcal{Q},(\mathbf{v}_{b,w}^{m,l},\mathbf{p}_{b,w}^{m,l}))$

14: end for

15:

16:
$\mathcal{Q}^{1},\mathcal{Q}^{2},...,\mathcal{Q}^{M}\leftarrow\text{RandomSplit}(\mathcal{Q})$

17: for $b=1$ to $\mu B$ do

18: for $m=1$ to $M$ do

19: Calculate $\overline{\mathbf{p}}_{b}^{m}$ using Eq. (2)

20: end for

21:
$\overline{c}_{b}\leftarrow\text{ConfidenceEstimator}(\\{\overline{\mathbf{p}}_{b}^{m}\mid
m=1,...,M\\})$

22:
$\overline{\mathbf{p}}_{b}\leftarrow\frac{1}{M}\sum^{M}_{m=1}{\overline{\mathbf{p}}_{b}^{m}}$

23:
$\overline{\mathbf{q}}_{b}\leftarrow\text{Sharpening}(\overline{\mathbf{p}}_{b})$

24: end for

25: Calculate the loss $\mathcal{L}$ via Eq. (5) and Eq. (6)

26: Update $\theta$ with SGD to minimize $\mathcal{L}$

27: Update $\theta_{m}$ with momentum updates using Eq. (4)

28: return $\theta$

##  V Experimental Results

###  V-A Experimental Settings

In this section, we conduct extensive evaluations to justify why our AggMatch
can solve the limitations of existing pseudo-labeling, consistency
regularization, and label propagation methods; Pseudo-Label [10], pi-Model
[70], Mean Teacher [20], MixMatch [27], UDA [12], FixMatch [4], FeatMatch
[29], and PAWS [35]. We also conduct the experiments with noisy label setting
for SSL dataset. To prove the robustness and effectiveness of our methods, we
compare our performance to one of the state-of-the-art SSL methods, FixMatch
[4]. We also provide an extensive analysis and ablation studies of key
components in our framework.

####  V-A1 Datasets

We use standard benchmarks for SSL, including CIFAR-10/100 [31] and SVHN [43].
Specifically, CIFAR-10 [31] has 50,000 training images of resolution 32x32
with 10 classes and 10,000 test images. Like CIFAR-10 [31], CIFAR-100 [31]
also has the same number of training/test images and image size but with 100
classes. SVHN [43] contains close-up of house number images of 32x32,
belonging to 10 different classes of digits. The training set has 73,257
images, and the test set contains 26,032 images. In many cases, we perform
experiments with fewer labels than previously considered since AggMatch shows
competitiveness in extremely label-scarce settings, which is the most
difficult setting to generate proper pseudo labels from the biased
configuration of few labeled data.

####  V-A2 Evaluation Metric

We compute the mean and standard deviation of the accuracy of the model,
trained on three different folds of labeled data, following the standard
evaluation protocol of labeling a portion of the training data and leaving the
rest unlabeled [4].

###  V-B Implementation Details

Following [27, 4], we use the same backbone architecture as WRN-28-2 [71] for
CIFAR-10 [31] and SVHN [43] with a batch size of 64, Wide ResNet-28-8 [71] for
CIFAR-100 [31] with a batch size of 48. A hyper-parameter for a unlabeled
batch size $\mu$ is set by 7 for both cases. For aggregation, we set
temperature by $\tau_{sim}=0.05$, weight for similarity function by
$\lambda_{sim}=0.5$, confidence threshold for queue by $\tau=0.95$, and the
momentum coefficient for the momentum encoder by $\lambda_{m}=0.999$, as
mentioned by Mean Teacher [20]. Queue size per class $L$ is determined by
2,048 for CIFAR-10 [31] and SVHN [43], and 256 for CIFAR-100 [31]. The number
of disjoint subset $M$ for estimating uncertainty is determined by 64. Hyper-
parameters for unsupervised loss function, unsupervised loss weight $\lambda$
and sharpening temperature for pseudo-labeling $T$ are set by 1 and 0.5,
respectively. To construct the different views, we use two different
magnitudes of data augmentation; weak augmentation $\alpha$, consisting of
only flip-and-shift augmentation, and strong augmentation $\mathcal{A}$,
consisting of randomly selected transformations in RandAugment [72] as
following [4].

TABLE II: Quantitative evaluation on CIFAR-10 [31] and SVHN [43]. The best
results are in bold, and the second best results are underlined.

Methods | CIFAR-10 [31] | CIFAR-100 [31] | SVHN [43]
---|---|---|---
40 | 80 | 250 | 400 | 40 | 250
Pseudo-Label [10] | - | - | 51.22±0.43 | - | - | 79.79±1.09
pi-Model [19] | - | - | 45.74±3.97 | - | - | 81.04±1.92
Mean Teacher [20] | - | - | 67.68±2.30 | - | - | 96.43±0.11
MixMatch [27] | 51.90±11.76 | - | 88.97±0.85 | 32.39±1.32 | 57.45±14.53 | 96.02±0.26
UDA [12] | 71.95±9.64 | - | 91.18±1.08 | 40.72±0.88 | 47.37±20.51 | 94.31±2.76
FixMatch (RA) [4] |  86.19±3.37 |  92.73±0.59 |  94.93±0.65 | 51.15±1.75 |  96.04±2.17 |  97.52±0.38
FeatMatch [29] | 58.68±8.88 | 83.95±2.65 | 92.50±0.64 | - | - |  96.66±0.19
PAWS [35] | 85.74±2.98 | 88.72±0.84 | 91.18±0.62 | - | - | -
AggMatch |  92.54±0.76 |  93.09±0.85 |  94.26±0.23 |  55.58±1.57 |  95.72±0.48 | 95.48±0.66
|  |  |  |  |  |

TABLE III: Per-class quantitative evaluation on CIFAR-10 [31]. The best results are in bold, and the lower scores between semantic labels (airplane $\leftrightarrow$ bird, dog $\leftrightarrow$ cat) are underlined respectively. Methods | air. | auto. | bird | cat | deer | dog | frog | horse | ship | truck | total
---|---|---|---|---|---|---|---|---|---|---|---
FixMatch [4] | 94.3 | 97.6 | 72.6 | 28.1 | 96.6 | 94.1 | 98.1 | 95.6 | 97.5 | 96.6 | 87.11
AggMatch | 92.4 | 96.2 | 84.3 | 82.7 | 96.2 | 88.3 | 98.6 | 95.8 | 97.7 | 97.9 | 93.01
|  |  |  |  |  |  |  |  |  |  |
TABLE IV: Evaluation of robustness of AggMatch on CIFAR-10 [31] and SVHN [43]
with noisy settings under various label fractions, 40 and 250 labels.

Methods |  CIFAR-10 [31] |  SVHN [43]
---|---|---
| 40 labels
---
+Noise 25%
| 250 labels
---
+Noise 50%
| 40 labels
---
+Noise 25%
| 250 labels
---
+Noise 50%
FixMatch [4] |  72.18$\pm$10.95 |  79.68$\pm$4.84 | 81.39$\pm$3.35 |  75.12$\pm$2.04
AggMatch | 77.63$\pm$12.29 | 85.41$\pm$7.07 |  78.66$\pm$5.82 | 89.47$\pm$5.54
|  |  |  |

###  V-C Experimental Results

####  V-C1 Results on Standard SSL Benchmarks.

As shown in Table II, we conduct experiments on CIFAR-10/100 [31], and SVHN
[43] focusing on the label-scarce settings. We use the same backbone
architecture for a fair comparison to other methods. We show the state-of-the-
art accuracy of 92.54%, outperforming FixMatch [4] by 6.35% on CIFAR-10 [31]
with four labeled samples per class and also record 93.09% on CIFAR-10 [31]
with eight labeled samples per class. We obtain 55.58% on CIFAR-100 [31] with
four labeled per class setting, which is 4.43% higher than a previous state-
of-the-art method. We also show comparable results on SVHN [43] dataset by
recording 95.72%, the second-best result on SVHN [43] with four labeled per
class.

Compared to the results of FeatMatch [51] and PAWS [35] closely related to
ours, which utilize the additional aggregation module, we can prove the
robustness and competitiveness of our method. Rather, both of them show lower
performance than FixMatch [4] on CIFAR-10 [31] with 40/80/250 settings and
their accuracy rates declines dramatically as the number of labeled data
become scarce, specifically from 250 to 40. It can be interpreted that their
attention modules significantly rely on the quality and quantity of labeled
data. This is because PAWS leverages only a mini-batch of labeled samples for
measuring similarity between unlabeled samples, which will lead to bias from
the configuration of selected labeled data. FeatMatch [29] utilizes
information from both within-class and across-class prototypical
representations through clustering in the memory bank for feature-based
refinement and augmentation. But the result mentioned above proves that
incorrect prototypes extracted from the memory bank, including samples without
considering confidence, deteriorate the quality of features. Unlike them,
AggMatch shows the state-of-the-art performance in label-scare settings such
as CIFAR-10 [31] with 40 labels and 80 labels. We also outperform PAWS and
FeatMatch by 6.8% and 33.9% at 40 labels setting, respectively. Based on the
above results, our method shows powerful competitiveness in label-scarce
settings by using aggregation with lots of consistent and confident samples in
the queue.

More specifically, We analyze our per-class accuracy on CIFAR-10 [31] with 40
labels, compared to FixMatch [4] in Table III. total accuracy of FixMatch [4]
is 87.11%, but it can be seen that there is a significant difference in
accuracy between pairs of semantic labels, which can share the abstract
feature space due to the similar appearance between the two classes. Fixmatch
[4] shows that the accuracy between pairs of semantic labels are largely
biased on one side of label. For example, the cat label and the bird label
have lower accuracy by 66.0% and 21.7% compared to their counterparts, the dog
label and the airplane label, respectively. On the contrary, Aggmatch shows
that the accuracy between pairs of semantic labels is quite equivalent to the
other. From this observation, we prove that samples, having ambiguous features
and class probabilities, can be corrected by aggregating numerous
discriminative within-class and across-class samples in the confident-aware
queue. And it also proves that the confidence estimation can appropriately
measure the confidence of the pseudo-labeling using multiple hypotheses with
subsets of the queue.

####  V-C2 Results with Noisy Labels

In this section, we conduct experiments on CIFAR-10 [31] and SVHN [43] with
noisy label settings to prove that AggMatch performs well in another
confirmation bias setting caused by the quality of the label. Although
training with noisy labels in supervised learning has been actively discussed
[73], SSL with noisy labels has not been explored yet. Noisy label settings
hinder the model from learning proper representation, closely related to In-
the-wild datasets, having poor annotation qualities or ambiguous class labels
without refining significant noise. We conduct the experiment on SSL
benchmarks, CIFAR-10 [31] and SVHN [43], with asymmetric label noise to
explore the robustness of our approach. The asymmetric label noise mimics
practical annotation error in semantically-similar classes, i.e., cats and
dogs. Following previous work [45], we propose asymmetric mapping of the noisy
labels for CIFAR-10 [31]: TRUCK $\to$ AUTOMOBILE, BIRD $\to$ AIRPLANE, DEER
$\to$ HORSE, CAT $\leftrightarrow$ DOG and for MNIST [74]: $2\to 7$, $3\to 8$,
$5\leftrightarrow 6$, $7\to 1$. Based on the similarity of the numeric
representation, we apply the same asymmetric noise mapping as MNIST [74] to
SVHN [43] for generating noisy labels.

As shown in Table IV, AggMatch outperforms FixMatch [4] on CIFAR-10 [31] and
SVHN [43] by a large margin. More specifically, AggMatch shows the most
significant improvement on CIFAR-10 [31] with 50% noise on 250 labels setting
compared to FixMatch [4] by 5.73%. Also, in SVHN [43], consisting of more
difficult real-world images, AggMatch outperforms FixMatch [4] by 14.35%. Even
in a more challenging setting in which one of the four labels per class is
noise, AggMatch shows 5.5% performance improvement compared to FixMatch [4].
Even if there are continuous noise labels, we can build a robust model by
maintaining the high quality of pseudo labels through aggregation.

(a) Precision (b) Precision on noise setting

(c) Recall on noise setting (d) Ours

Figure 5: Plots of precision and recall: (a) and (c) are conducted on CIFAR-10
[31] with 40 labels. (b) and (d) are conducted on CIFAR-10 [31] with 25% noise
on 40 labels.

####  V-C3 Evaluating Confidence Estimation

Fig. 5 shows the recall and precision evaluation on AggMatch and FixMatch [4]
across the training iterations on CIFAR-10 [31] with 40 labels and also
CIFAR-10 [31] with 25% noise on 40 labels. In SSL, it is important to generate
accurate pseudo labels because confirmation bias from incorrect pseudo-
labeling is the main reason for performance limitation. For this, FixMatch [4]
uses a fixed threshold strategy, using only unlabeled data whose confidence is
above the threshold, but this naïve thresholding strategy only involves very
high-quality unlabeled data and cannot consider the learning status of the
network and the difficulties of different samples. On the other hand, we
dynamically adjust the magnitude of unsupervised loss function based on
measuring uncertainty by changing the confidence-based strategy from
thresholding to weighting.

The results of precision, defined as the number of the correct pseudo labels
over the pseudo labels of the samples in SSL classification, are shown in Fig.
5(a). Fig. 5(a) and Fig. 5(b) show the significant margin between FixMatch [4]
and AggMatch on CIFAR-10 [31] with 40 labels as gradually formulating queue
with the confident-aware samples. Also, recall, defined as the number of the
correct pseudo labels over the ground-truth labels of the samples in SSL
classification, is a trade-off with precision, but AggMatch shows similar or
higher recall despite the overwhelming precision as shown in Fig. 5(c) and
Fig. 5(d). It can be interpreted that the aggregation is effective for
refining label refinement and measuring uncertainty and that weighting
strategy can help accelerate training convergence by securing lots of
confident pseudo labels.

###  V-D Ablation Study

We also conduct ablation study to evaluate the performance gap of components
our overall framework, including more extensive analyses on similarity
function and uncertainty estimation. We also conduct experiments on queue size
and momentum encoder for validating our method. The dataset used for the
experiments is CIFAR-10 [31], and the number of labeled data for training is
limited to four samples per class.

####  V-D1 Effectiveness of Key Components

We analyze the key components of the pseudo label aggregation module in
AggMatch, one is an pseudo label refinement and the other is a confidence
estimation in Table V. We define the baseline as the model without two
modules. The two modules were added alternately on the baseline to analyze the
effectiveness of each module. The improvement of the aggregation module and
confidence estimation module is achieved by 2.92% / 2.62% over the baseline,
respectively. It proves that each module shows adequate improvement and
equally contributes to the performance. The best performance is recorded as
92.54% with both modules, outperforming the previous state-of-the-art,
FixMatch [4], by 6.35%.

TABLE V: Ablation study on key components in AggMatch such as pseudo labeling refinement and confidence estimation module. Components | Accuracy
---|---
Pseudo label refinement |  Confidence estimation |
✗ | ✗ | 87.64±2.19
✓ | ✗ | 90.56±2.63
✗ | ✓ | 90.26±1.93
✓ | ✓ |  92.54±0.76
|  |

####  V-D2 Analysis on Each Term of Similarity Function

In this section, we analyze the similarity function in the aggregation module
$\mathcal{S}$. Unlike previous methods [66, 29, 35], we define it as the
summation of feature similarity term and class similarity term based on cosine
similarity and Jenson-Shannon divergence on feature vector $\mathbf{v}$ and
class distribution $\mathbf{p}$, respectively. Through the ablation study for
similarity function in Table VI, there is about 3% difference in accuracy
depending on whether class similarity term is included in the similarity
function.

We also conduct a detailed analysis by visualizing the attention weights
depending on the similarity function formulation of aggregation. Note that the
principal diagonal of attention weights means the similarity magnitude of
within-class feature vectors, and other regions mean those of across-class
feature vectors. Each row in Table VII shows the transitions of the attention
weights between learned feature vectors based on feature similarity term,
class similarity term and combination of feature similarity term and class
similarity term, respectively as the training of AggMatch progresses. As the
training progresses, regardless of the composition of the similarity function,
we can confirm that the gap between within-class similarity and across-class
similarity is bigger. This means our aggregation is effective for generating
discriminative feature vectors and class distributions by optimizing the
model, consisting of feature extractor and classifier. When comparing the
attention weights in the best model parameters, the diagonal components of
attention weight based on the summation of both terms show high contrast with
other regions. By integrating the results from Table VI and Table VII, it can
be interpreted that aggregating pseudo labels of ambiguous samples, which
cannot be refined by feature similarity term alone, is complemented by class
similarity term.

####  V-D3 Comparison of Thresholding and Weighting for Confidence Estimation

Unlike standard confidence-based algorithms [12, 4], we measure confidence as
uncertainty-based weighting. More specifically, we can scale the loss
magnitude by weighting uncertainty estimation based on the consensus between
hypotheses from different subsets of the queue. To demonstrate the
effectiveness of weighting, we compare thresholding and weighting on
uncertainty approach. We can define uncertainty-based thresholding such as
$\frac{1}{\mu B}\sum_{b=1}^{\mu
B}{\mathbbm{1}(\overline{i}_{b}\leq\tau_{u})\mathcal{D}(\overline{\mathbf{q}}_{b},p_{\mathrm{model}}(y|\mathcal{A}(u_{b});\theta))},$
where $\overline{i}_{b}=-\sum\mathbf{a}_{b}\log{\mathbf{a}_{b}}$. The pre-
defined threshold value $\tau_{u}$ is set in a range from 0.1 to 1.5 where the
lower threshold value uses relatively higher entropy samples for training
compared to the higher threshold.

The result of weighting method outperforms thresholding method by
5.68%/1.72%/2.65%/3.72%, respectively as shown in Table VIII. This is because
it is difficult to manipulate the quantity and quality of the samples by using
only one pre-defined threshold. Compared to $\tau_{u}=1.0$, which is an
optimal threshold value in the experiment, there is 2.54% performance decline
in $\tau_{u}=1.5$ due to the quality problem, generated by incorrect pseudo
labels from low confident samples. On the other hand, we can also prove that
quantity problem in $\tau_{u}=0.1$, caused by the few number of high confident
samples through 4.26% performance gap.

TABLE VI: Ablation on each term of similarity function of the aggregation module. Similarity function | Accuracy
---|---
Feature similarity term |  Class similarity term |
✓ | ✗ | 89.58±5.02
✓ | ✓ |  92.54±0.76
|  |
TABLE VII: Visualization on the transitions of attention weights as training
iteration progress depending on the similarity function configuration in
aggregation on CIFAR-10 [31].

Similarity function | 1000 iter | 10000 iter | Best
---|---|---|---
Feature similarity term |  |  |
Class similarity term |  |  |
| Feature similarity term
---
+
Class similarity term
|  |
|  |  |

TABLE VIII: Comparison of training objective configuration by hand-tuned uncertainty-based thresholding and weighting. Methods | $\tau_{u}$ | Accuracy
---|---|---
Uncertainty-based thresholding | 1.5 | 86.86
1.0 | 90.82
0.5 | 89.89
0.1 | 88.28
| Uncertainty-based weighting
---
- | 92.54
|  |

####  V-D4 Effects of Queue Size

To prove the superiority of AggMatch, we conduct an ablation study on the
queue size. We reduce the queue size to $L=512$ which is 1/4 times smaller
than the original queue size and then record 92.09% accuracy on the CIFAR-10
[31] with 40 labels. There is only 0.45% performance decline compared to the
original queue size ($L=2,048$). It shows that our competitiveness lies in
pseudo-labeling refinement and uncertainty estimation through aggregation with
consistent and confident samples in the queue, not mainly depending on the
queue size. However, when the queue size is set to $L=2,048$, the queue
occupies only 10MB in the GPU memory space during the training. It is an
extremely small in volume compared to 16GB of the backbone network, i.e.,
WideResNet 28-2, having 512 mini-batch sizes. Following [41], adopting queue
size as 65,356 for the dictionary look-up, we also increase the queue size to
maximize our effectiveness.

####  V-D5 Effectiveness of Momentum Encoder

We conduct an ablation study on the effectiveness of the momentum encoder for
aggregation. The samples composing the large queue should be extracted by a
slow progressing encoder structure because they come from mini-batches in
different time-steps. Therefore, in order to maintain consistency between
samples in the queue, we update the encoder for the queue with a momentum-
based moving average as in [41].

The experiment is conducted at a small queue size, e.g., $L=512$, to minimize
the effectiveness of the momentum encoder because the queue can be filled with
the same mini-batch samples in the same time-steps with a high probability. We
confirm that the momentum encoder is significantly important for maintaining
the consistency of the queue through 9.5% drop in the performance compared to
when using the momentum encoder on the CIFAR-10 [31] with 40 labels in Table
II.

TABLE IX: Visualization on the transitions of multi-class feature distribution
(denoted as dots in colors) of training samples from CIFAR-10 [31] in the
feature space at different training iterations. Data projection in 2-D space
is generated by t-SNE [75] based on the feature representation.

Methods | 4,000 iter | 20,000 iter | 30,000 iter | 40,000 iter | Best
---|---|---|---|---|---
FixMatch [4] |  |  |  |  |
AggMatch |  |  |  |  |
|  |  |  |  |

(a) FixMatch [4] (b) AggMatch

Figure 6: Visualization of feature attention map of FixMatch [4] and AggMatch.
TABLE X: Examples of pseudo labels by FixMatch [4] and AggMatch.

Img |  |  |  |  |
---|---|---|---|---|---
GT label |  cat |  dog |  bird |  horse |  airplane
FixMatch [4] |  dog |  horse |  dog |  deer |  cat
|  (1.00) |  (0.97) |  (1.00) |  (0.96) |  (0.99)
AggMatch |  cat |  dog |  bird |  horse |  airplane
|  (1.00) |  (0.99) |  (0.98) |  (0.99) |  (1.00)
|  |  |  |  |

TABLE XI:  Visualization on the predictions of selected unlabeled samples from
CIFAR-10 [31]. The blue bar corresponds to the misclassified class. The yellow
bar corresponds to the ground-truth class and the value of the ground-truth
class is represented below the bar graph.

Img |  |  |  |
---|---|---|---|---
GT label | frog | cat | frog | dog
before |  |  |  |
| 0.27 | 0.34 | 0.43 | 0.44
after |  |  |  |
| 0.56 | 0.55 | 0.79 | 0.81
|  |  |  |

###  V-E Analysis

In this section, we conduct intensive experiments to analyze the effectiveness
of our key contributions, a similarity function and a confidence estimation,
respectively. We provide extensive visualization using the model weights from
specific training iterations in CIFAR-10 [31] test dataset. We also use the
same experiment settings as Sec. V-D.

####  V-E1 Visualization of Feature Distribution

We compare the feature distribution between AggMatch and FixMatch [4] as shown
in Table IX. For visualizing high-dimensional representations of model
predictions, the complex feature matrices are then transformed into two-
dimensional points by using t-Distributed Stochastic Neighbor Embedding
(t-SNE) [75] to effectively display the feature manifolds. As training
iteration progresses, the t-SNE of AggMatch shows that the structure of data
points is increasingly well-organized and the misclassified regions where two
representations from different classes coexist are also significantly
decreased. This indicates that our aggregation module effectively refines
pseudo labels based on well-organized representations that express distinctive
characteristics of each class.

Fig. 6 visualizes attention distribution through the learned attention weights
from in-batch of images to quantify the relationship between within-class and
across-class relationships. We can confirm that the weight of the diagonal
position from AggMatch, equivalent to the inter-class similarity, is
significantly higher than other regions. Compared to FixMatch [4], it can be
seen that the difference between within-class and across-class attention
weights is much larger, which means that AggMatch has distinctive feature
representations for each class.

####  V-E2 Visualization of Pseudo-labeling Refinement

Wrong pseudo-labeling of unlabeled samples with high confidence is considered
the main factor of the confirmation bias. As shown in Table X, FixMatch [4]
has the pseudo-label misclassification problem, even using high confident
pseudo labels, since it relies solely on its own model output of the sample
without considering the relationship between other samples. On the other hand,
AggMatch aggregates pseudo labels by considering the relationship between
confident-aware samples in the queue and finally generates more accurate
pseudo labels. Table XI visualizes the pseudo-labeling refinement of AggMatch
by comparing the class distribution before and after aggregation. It can be
seen that the class probability, which would has the wrong one-hot pseudo
label by the argmax operation is refined to the appropriate pseudo labels
after aggregation.

##  VI Conclusion

In this paper, we have proposed a novel semi-supervised learning (SSL)
framework, dubbed AggMatch, that aggregates initial pseudo labels by
considering the similarity between the different instances to achieve more
confident pseudo labels. We have introduced a class-balanced confidence-ware
queue with the momentum model for more stable and consistent aggregation. We
have also proposed a novel confidence measure for the pseudo label by
considering the consensus among multiple pseudo labels with different subsets
of the queue. We have shown that our method surpasses the current state-of-
the-art in several benchmarks even with noise settings. A natural next step,
which we leave for future work, is to examine how AggMatch could be used to
learn a model in a self-supervised setting without any labeled data.

## References

  * [1] I. Radosavovic, P. Dollár, R. Girshick, G. Gkioxari, and K. He, “Data distillation: Towards omni-supervised learning,” in _CVPR_ , 2018.
  * [2] I. Z. Yalniz, H. Jégou, K. Chen, M. Paluri, and D. Mahajan, “Billion-scale semi-supervised learning for image classification,” _arXiv:1905.00546_ , 2019\.
  * [3] Q. Xie, M.-T. Luong, E. Hovy, and Q. V. Le, “Self-training with noisy student improves imagenet classification,” in _CVPR_ , 2020.
  * [4] K. Sohn, D. Berthelot, N. Carlini, Z. Zhang, H. Zhang, C. A. Raffel, E. D. Cubuk, A. Kurakin, and C.-L. Li, “Fixmatch: Simplifying semi-supervised learning with consistency and confidence,” in _NeurIPS_ , 2020.
  * [5] X. Chen, Y. Yuan, G. Zeng, and J. Wang, “Semi-supervised semantic segmentation with cross pseudo supervision,” in _CVPR_ , 2021.
  * [6] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly _et al._ , “An image is worth 16x16 words: Transformers for image recognition at scale,” in _ICLR_ , 2020.
  * [7] Y. Chen, X. Dai, M. Liu, D. Chen, L. Yuan, and Z. Liu, “Dynamic convolution: Attention over convolution kernels,” in _CVPR_ , 2020.
  * [8] I. Tolstikhin, N. Houlsby, A. Kolesnikov, L. Beyer, X. Zhai, T. Unterthiner, J. Yung, A. P. Steiner, D. Keysers, J. Uszkoreit _et al._ , “Mlp-mixer: An all-mlp architecture for vision,” in _NeurIPS_ , 2021.
  * [9] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” _arXiv:2103.14030_ , 2021.
  * [10] D.-H. Lee _et al._ , “Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,” in _ICML Workshop_ , 2013.
  * [11] W. Shi, Y. Gong, C. Ding, Z. M. Tao, and N. Zheng, “Transductive semi-supervised deep learning using min-max features,” in _ECCV_ , 2018.
  * [12] Q. Xie, Z. Dai, E. Hovy, T. Luong, and Q. Le, “Unsupervised data augmentation for consistency training,” in _NeurIPS_ , 2020.
  * [13] E. Arazo, D. Ortego, P. Albert, N. E. O’Connor, and K. McGuinness, “Pseudo-labeling and confirmation bias in deep semi-supervised learning,” in _IJCNN_. IEEE, 2020, pp. 1–8.
  * [14] B. Zoph, G. Ghiasi, T.-Y. Lin, Y. Cui, H. Liu, E. D. Cubuk, and Q. Le, “Rethinking pre-training and self-training,” in _NeurIPS_ , 2020.
  * [15] H. Pham, Z. Dai, Q. Xie, and Q. V. Le, “Meta pseudo labels,” in _CVPR_ , 2021\.
  * [16] Y. Grandvalet, Y. Bengio _et al._ , “Semi-supervised learning by entropy minimization.” _CAP_ , vol. 367, pp. 281–296, 2005.
  * [17] M. Sajjadi, M. Javanmardi, and T. Tasdizen, “Mutual exclusivity loss for semi-supervised deep learning,” in _ICIP_. IEEE, 2016, pp. 1908–1912.
  * [18] ——, “Regularization with stochastic transformations and perturbations for deep semi-supervised learning,” in _NeurIPS_ , 2016.
  * [19] S. Laine and T. Aila, “Temporal ensembling for semi-supervised learning,” in _ICLR_ , 2017.
  * [20] A. Tarvainen and H. Valpola, “Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,” in _NeurIPS_ , 2017.
  * [21] T. Miyato, S.-i. Maeda, M. Koyama, and S. Ishii, “Virtual adversarial training: a regularization method for supervised and semi-supervised learning,” _TPAMI_ , vol. 41, no. 8, pp. 1979–1993, 2018.
  * [22] S. Park, J. Park, S.-J. Shin, and I.-C. Moon, “Adversarial dropout for supervised and semi-supervised learning,” in _AAAI_ , 2018.
  * [23] Z. Ke, D. Wang, Q. Yan, J. Ren, and R. W. Lau, “Dual student: Breaking the limits of the teacher in semi-supervised learning,” in _ICCV_ , 2019.
  * [24] V. Verma, A. Lamb, J. Kannala, Y. Bengio, and D. Lopez-Paz, “Interpolation consistency training for semi-supervised learning,” in _IJCAI_ , 2019, pp. 3635–3641.
  * [25] M. Belkin, I. Matveeva, and P. Niyogi, “Regularization and semi-supervised learning on large graphs,” in _COLT_. Springer, 2004, pp. 624–638.
  * [26] L. Zhang and G.-J. Qi, “Wcp: Worst-case perturbations for semi-supervised deep learning,” in _CVPR_ , 2020.
  * [27] D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, and C. A. Raffel, “Mixmatch: A holistic approach to semi-supervised learning,” in _NeurIPS_ , 2019.
  * [28] D. Berthelot, N. Carlini, E. D. Cubuk, A. Kurakin, K. Sohn, H. Zhang, and C. Raffel, “Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring,” _arXiv:1911.09785_ , 2019.
  * [29] C.-W. Kuo, C.-Y. Ma, J.-B. Huang, and Z. Kira, “Featmatch: Feature-based augmentation for semi-supervised learning,” in _ECCV_ , 2020.
  * [30] Z. Hu, Z. Yang, X. Hu, and R. Nevatia, “Simple: Similar pseudo label exploitation for semi-supervised classification,” in _CVPR_ , 2021.
  * [31] A. Krizhevsky, G. Hinton _et al._ , “Learning multiple layers of features from tiny images,” 2009.
  * [32] X. Zhuǵ and Z. Ghahramaniǵn, “Learning from labeled and unlabeled data with label propagation,” 2002.
  * [33] M. S. T. Jaakkola and M. Szummer, “Partially labeled classification with markov random walks,” in _NeurIPS_ , 2002.
  * [34] D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Schölkopf, “Learning with local and global consistency,” in _NeurIPS_ , 2004.
  * [35] M. Assran, M. Caron, I. Misra, P. Bojanowski, A. Joulin, N. Ballas, and M. Rabbat, “Semi-supervised learning of visual features by non-parametrically predicting view assignments with support samples,” _arXiv:2104.13963_ , 2021.
  * [36] A. Iscen, G. Tolias, Y. Avrithis, and O. Chum, “Label propagation for deep semi-supervised learning,” in _CVPR_ , 2019.
  * [37] S. Li, B. Liu, D. Chen, Q. Chu, L. Yuan, and N. Yu, “Density-aware graph for deep semi-supervised visual recognition,” in _CVPR_ , 2020.
  * [38] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _CVPR_ , 2016.
  * [39] D. Sun, X. Yang, M.-Y. Liu, and J. Kautz, “Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume,” in _CVPR_ , 2018.
  * [40] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in _NeurIPS_ , 2017.
  * [41] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in _CVPR_ , 2020.
  * [42] P. Jain, A. Jain, T. Zhang, P. Abbeel, J. E. Gonzalez, and I. Stoica, “Contrastive code representation learning,” _arXiv:2007.04973_ , 2020.
  * [43] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” 2011.
  * [44] S. Sukhbaatar, J. Bruna, M. Paluri, L. Bourdev, and R. Fergus, “Training convolutional networks with noisy labels,” in _ICLR_ , 2014.
  * [45] G. Patrini, A. Rozza, A. Krishna Menon, R. Nock, and L. Qu, “Making deep neural networks robust to label noise: A loss correction approach,” in _CVPR_ , 2017.
  * [46] H. Scudder, “Probability of error of some adaptive pattern-recognition machines,” _IEEE Trans. Inf. Theory_ , vol. 11, no. 3, pp. 363–371, 1965\.
  * [47] D. Yarowsky, “Unsupervised word sense disambiguation rivaling supervised methods,” in _ACL_ , 1995, pp. 189–196.
  * [48] E. Riloff, “Automatically generating extraction patterns from untagged text,” in _AAAI_ , 1996.
  * [49] O. Chapelle, B. Scholkopf, and A. Zien, “Semi-supervised learning. 2006,” _Cambridge, Massachusettes: The MIT Press View Article_ , 2006.
  * [50] P. Bachman, O. Alsharif, and D. Precup, “Learning with pseudo-ensembles,” in _NeurIPS_ , 2014.
  * [51] G. French, M. Mackiewicz, and M. Fisher, “Self-ensembling for visual domain adaptation,” _arXiv:1706.05208_ , 2017.
  * [52] B. Athiwaratkun, M. Finzi, P. Izmailov, and A. G. Wilson, “There are many consistent explanations of unlabeled data: Why you should average,” _arXiv:1806.05594_ , 2018.
  * [53] X. Zhu, J. Lafferty, and Z. Ghahramani, _Semi-supervised learning: From Gaussian fields to Gaussian processes_. School of Computer Science, Carnegie Mellon University, 2003.
  * [54] J. Weston, F. Ratle, H. Mobahi, and R. Collobert, “Deep learning via semi-supervised embedding,” in _Neural networks: Tricks of the trade_. Springer, 2012, pp. 639–655.
  * [55] D. J. MacKay, “A practical bayesian framework for backpropagation networks,” _Neural computation_ , vol. 4, no. 3, pp. 448–472, 1992.
  * [56] T. Chen, E. Fox, and C. Guestrin, “Stochastic gradient hamiltonian monte carlo,” in _ICML_ , 2014.
  * [57] M. Welling and Y. W. Teh, “Bayesian learning via stochastic gradient langevin dynamics,” in _ICML_ , 2011.
  * [58] A. Kendall and Y. Gal, “What uncertainties do we need in bayesian deep learning for computer vision?” _arXiv:1703.04977_ , 2017.
  * [59] E. Ilg, O. Cicek, S. Galesso, A. Klein, O. Makansi, F. Hutter, and T. Brox, “Uncertainty estimates and multi-hypotheses networks for optical flow,” in _ECCV_ , 2018.
  * [60] A. Kendall, Y. Gal, and R. Cipolla, “Multi-task learning using uncertainty to weigh losses for scene geometry and semantics,” in _CVPR_ , 2018.
  * [61] M. Poggi, F. Aleotti, F. Tosi, and S. Mattoccia, “On the uncertainty of self-supervised monocular depth estimation,” in _CVPR_ , 2020.
  * [62] A. Graves, “Practical variational inference for neural networks,” in _NeurIPS_ , 2011.
  * [63] C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra, “Weight uncertainty in neural network,” in _ICML_ , 2015.
  * [64] B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” _arXiv:1612.01474_ , 2016.
  * [65] C. Rosenberg, M. Hebert, and H. Schneiderman, “Semi-supervised self-training of object detection models,” in _WACV/MOTION_ , 2005.
  * [66] Y. Cao, J. Xu, S. Lin, F. Wei, and H. Hu, “Gcnet: Non-local networks meet squeeze-excitation networks and beyond,” in _ICCV Workshops_ , 2019\.
  * [67] X. Zhang, F. Chen, C.-T. Lu, and N. Ramakrishnan, “Mitigating uncertainty in document classification,” _arXiv:1907.07590_ , 2019.
  * [68] L. S. Vygotsky, _Mind in society: The development of higher psychological processes_. Harvard university press, 1980\.
  * [69] J. Kagan and N. Kogan, “Individual variation in cognitive processes,” _Carmichael’s manual of child psychology_ , vol. 1, pp. 1273–1365, 1970.
  * [70] A. Rasmus, M. Berglund, M. Honkala, H. Valpola, and T. Raiko, “Semi-supervised learning with ladder networks,” in _NeurIPS_ , 2015.
  * [71] S. Zagoruyko and N. Komodakis, “Wide residual networks,” _arXiv:1605.07146_ , 2016.
  * [72] E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le, “Randaugment: Practical automated data augmentation with a reduced search space,” in _CVPR Workshop_ , 2020.
  * [73] K. Gu, X. Masotto, V. Bachani, B. Lakshminarayanan, J. Nikodem, and D. Yin, “A realistic simulation framework for learning with label noise,” 2021.
  * [74] L. Deng, “The mnist database of handwritten digit images for machine learning research,” _Signal Processing Magazine_ , vol. 29, no. 6, pp. 141–142, 2012\.
  * [75] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in Python,” _JMLR_ , vol. 12, pp. 2825–2830, 2011.