figurec

# Deep Deconfounded Content-based Tag Recommendation for UGC with Causal
Intervention

Yaochen Zhu*, Xubin Ren*, Jing Yi, Zhenzhong Chen†
{yaochenzhu  rxubin  yijing-v<EMAIL_ADDRESS>
Wuhan University

###### Abstract

Traditional content-based tag recommender systems directly learn the
association between user-generated content (UGC) and tags based on collected
UGC-tag pairs. However, since a UGC uploader simultaneously creates the UGC
and selects the corresponding tags, her personal preference inevitably biases
the tag selections, which prevents these recommenders from learning the causal
influence of UGCs’ content features on tags. In this paper, we propose a deep
deconfounded content-based tag recommender system, namely, DecTag, to address
the above issues. We first establish a causal graph to represent the relations
among uploader, UGC, and tag, where the uploaders are identified as
confounders that spuriously correlate UGC and tag selections. Specifically, to
eliminate the confounding bias, causal intervention is conducted on the UGC
node in the graph via backdoor adjustment, where uploaders’ influence on tags
leaked through backdoor paths can be eliminated for causal effect estimation.
Observing that adjusting the causal graph with do-calculus requires
integrating the entire uploader space, which is infeasible, we design a novel
Monte Carlo (MC)-based estimator with bootstrap, which can achieve asymptotic
unbiasedness provided that uploaders for the collected UGCs are i.i.d. samples
from the population. In addition, the MC estimator has the intuition of
substituting the biased uploaders with a hypothetical random uploader from the
population in the training phase, where deconfounding can be achieved in an
interpretable manner. Finally, we establish a YT-8M-Causal dataset based on
the widely used YouTube-8M dataset with causal intervention and propose an
evaluation strategy accordingly to unbiasedly evaluate causal tag
recommenders. Extensive experiments show that DecTag is more robust to
confounding bias than state-of-the-art causal recommenders. Codes and datasets
are available at https://github.com/Re-bin/DecTag.

##  1 Introduction

Recent years have witnessed the profound transformation of user-generated
content (UGC) to people’s information-sharing strategy and entertainment life
[1, 2]. Close on the heels of UGCs is the trend of attaching tags to uploaded
UGCs, which are short terms that concisely summarize the information-rich UGC
features [3, 4]. Tags serve both as short indices of UGC and eye-catchers,
which help both uploaders to gain more exposure to the target audience and
users to more conveniently locate interesting content from numerous
candidates. Since tags can be words, word combinations, or even Emojis, the
candidate pool for tags is so large that it becomes difficult to select
suitable tags for the content. Consequently, tag recommender systems, which
aim to automatically recommend suitable tags to users when they upload UGCs in
the platform, have become an important topic that has attracted lots of
attention among researchers [5]. ††*Yaochen Zhu and Xubin Ren are co-first
authors
$\dagger$ Corresponding author: Zhenzhong Chen

Due to the large candidate tag spaces and the scarce tags associated with
UGCs, collaborative filtering (CF), which recommends suitable tags for one UGC
with pre-existing tags based on other UGCs with similar tags, is restricted in
applications due to lack of sufficient collaborative information. Content-
based filtering, which assigns tags for UGCs based on their content features,
has become the mainstream strategy for tag recommendations [5]. Traditional
content-based tag recommender systems consist of two steps [6, 7, 8]. First,
existing UGC-tag pairs are collected from the platform to form a training
database. Then, a naive matching model is designed to learn the mapping from
UGC features to associated tags based on the collected UGC-tag pairs. If we
define the most suitable tags based solely on the content of a UGC as the
causal effects of the UGC on tags, the goal of the tag recommender should be
to capture such causal effects based on collected UGC-tag pairs. However, the
mapping de facto captured by naive models is unavoidably the association
between existing UGC and tags. Since uploaders simultaneously assume the role
of UGC-creator and tag-selector, they serve as confounders whose personal
preference for tag selections, i.e., the selection bias, is mistakenly
captured as the causal effects. This is problematic, because the bias may
cause naive models to miss important, relevant tags for UGCs possibly
neglected by uploaders, which can cost them to lose potential audiences and
associated economic benefits. Therefore, it is imperative to introduce causal
inference [9], which aims to eliminate confounding bias and faithfully recover
causal relations among variables, into UGC tag recommender systems.

Causality has been well-explored in item recommender systems where the targets
are items such as movies, merchandise, and articles [10, 11, 12]. However,
causal inference for tag recommendations, although crucial, remains under-
explored among researchers. In comparison, causal inference for tag
recommendations faces three main challenges. (1) To eliminate uploader-induced
bias, the causal intervention should be conducted upon UGCs such that only the
influence of UGC content on tags, i.e., the causal effect, is captured by the
model. However, such causal effects are per se difficult to learn since most
UGCs have no tags when recommendations are required, which renders
collaborative filtering, the most effective method in item recommendations,
incompetent. (2) The recommendation subjects and targets, i.e., UGCs and tags,
contain rich multimodal semantic information. Therefore, complex content-
matching relations exist between UGC and tags, which renders the causal
effects per se very difficult to model, even without the influence of
unobserved confounders. (3) Most importantly, confounders in tag
recommendations are UGC uploaders, whose preferences and self-selection bias
are complicated and influence the generated UGC and selected tags in a highly
non-linear manner. In comparison, confounders modeled by most existing item
recommender systems are item popularity (scalar) [13], item attributes
(categorical) [14, 15], user preference (categorical) [16], or historical user
interactions (integral vectors) [17], and some of these methods can only
guarantee unbiasedness provided that the influence of confounders is not
highly non-linear [16]. Finally, a common problem for all causal recommenders
is the inaccessibility of real-world datasets due to lack of bias-free
groundtruth [18]. This is especially evident in tag recommendations because
acquiring tags devoid of uploaders’ bias is anything but realistic. Therefore,
it seems impossible to evaluate the effectiveness of causal tag recommenders
on large-scale real-world datasets. Consequently, tackling both the model and
data side challenges for causal tag recommenders to eliminate uploaders’
confounding bias becomes the main focus of this paper.

To address the above challenges, we propose a deep deconfounded content-based
UGC tag recommender system, namely DecTag, based on causal intervention. We
first establish a causal graph representing the relations among uploaders,
UGC, and tags, where uploaders are clearly identified as confounders.
Specifically, to eliminate the confounding bias, causal intervention is
conducted upon the UGC node from the causal graph via backdoor adjustment with
do-calculus. Previous do-calculus solvers can only guarantee unbiasedness if
tags are linearly dependent upon uploaders. However, complex relations exist
among UGCs, uploaders, and tags, resulting in a large extra bias in tag
predictions for these methods. Therefore, we propose a novel Monte Carlo
(MC)-based do-calculus approximator with bootstrap, which can achieve
asymptotic unbiasedness provided that the uploaders in the collected dataset
are i.i.d. samples from the population. In addition, the proposed MC-based
solver has the intuition of replacing the biased uploader with a hypothetical
random uploader in the model training phase, which allows the deconfounding
uploader’s influence to be achieved in an explainable manner. Finally, the
DecTag algorithm is model-agnostic and can be plugged into any base content-
based tag recommender system, which shows its broad potential applications.
Another main contribution of this paper is that we establish a large-scale
real-world tag recommendation dataset, i.e., YT-8M-Causal, based on the widely
used YouTube-8M dataset to evaluate causal tag recommenders. Specifically, we
conduct causal interventions on the test set based on substitute confounders,
where we demonstrate that if the attributes that we intervened upon have
equivalent influences to the confounders, it can indirectly demonstrate
whether confounding bias is eliminated without the requirement of bias-free
tags. The main contribution of this paper can be summarized as follows:

  * •

We introduce causality to address uploader-induced confounding bias in tag
recommendations. Different from traditional methods, we establish a structural
causal graph to depict the relations among uploaders, UGCs, and tags, where
uploaders can be identified as confounders that spuriously correlate UGCs and
tags. To the best of our knowledge, this is the first work to investigate tag
recommendations from a causal perspective.

  * •

Confounding bias induced by UGC uploaders is tackled by conducting causal
intervention on UGC in the causal graph with backdoor adjustment. The
intractable integration over the entire infinite uploader space with do-
calculus is addressed by a novel Monte-Carlo estimator with bootstrap, which
is guaranteed to be asymptotically consistent and unbiased provided that
uploaders are random i.i.d. samples from the population.

  * •

The proposed MC-based do-calculus approximator in DecTag has an intuitive
explanation that uses an unbiased hypothetical random uploader to replace the
true uploader with her own selection bias when training the base tag
recommender. Therefore, this allows a transparent deconfounding procedure with
good interpretability.

  * •

A tag recommendation dataset with causally intervened test sets, namely
YT-8M-Causal, is established based on the YouTube-8M dataset, with evaluation
procedures introduced accordingly to evaluate causal tag recommender systems
without the requirement of unbiased tags. Experiments demonstrate that DecTag
is more robust to confounding bias than state-of-the-art deconfounding
strategies.

##  2 Related Work

Although the proposed DecTag is the first causality-based algorithm for
content-based UGC tag recommendation to our knowledge that addresses the
uploader-induced confounding bias, it draws inspiration from previous tag
recommender systems and the current trends applying causal inference in item
recommendations, which we will brief survey as follows.

###  2.1 Tag Recommender Systems.

UGC tag recommendation aims to automatically recommend tags to users when they
upload generated content into online platforms [19, 5]. Recently, due to the
popularization of UGC sharing platform such as YouTube, TikTok, etc., and the
handy role tags plays for attracting audiences and indexing numerous UGCs
based on its content, UGC tag recommender system has become a hot research
area that attracts lots of attention from researchers. Existing tag
recommenders can be classified into collaborative-filtering (CF)-based methods
[20] and content-based methods [21]. CF-based methods [20] recommend new tags
to a UGC with pre-existing tags based on UGCs with similar tags. Since
collaborative information is scarce, content-based tag recommendation is more
common where tags are suggested by UGC content [21]. However, both strategies
merely learn the association between existing UGC-tag pairs. Since the
uploader simultaneously creates the UGC content and selects its tags, she is a
confounder that creates a spurious correlation between UGC and the self-
selected tags. This may lead to systematic bias in traditional tag recommender
systems, which can be addressed by the causal inference techniques introduced
in this paper.

It can be noticed that another trend in current tag recommender system
research, which is quite the opposite of introducing causal inference
techniques to remove the influence of uploader’s selection bias in this paper,
is toward personalization [22, 23, 24]. Personalized tag recommender systems
estimate uploaders’ preferences toward tags based on their historical tag
selection behaviors and explicitly utilize the estimated preferences to
recommend tags for their generated content. This can be achieved by either
combining the learned UGC-tag matching module with an uploader-tag
collaborative filtering system (i.e., the late fusion strategy) or explicitly
utilizing user attributes as additional covariates in the vanilla content-
based tag recommender systems (i.e., the early fusion strategy). Although
introducing personalization in the vanilla content-based tag recommender
systems may elicit familiar feelings from the uploaders and therefore increase
the user retention rate due to the explicit consideration of their personal
preferences and habits, the recommended tags may end up catering to the
uploaders’ personal bias instead of determining the most suitable tags for the
content of the UGC. This is undesirable, since it may exacerbate the bias in
tag recommendation results, which causes uploaders to lose potential exposures
to certain audiences and the associated economic benefits and reduce the
accuracy for users to index interesting UGC with tags.

###  2.2 Recommendations with Causality

Researchers have long been aware that introducing causality to recommender
system is conducive to reduce the pervasive confounding bias in collected
historical interaction data. Specifically, Pearl’s causal model is a widely
used causal paradigm for item recommendations [9]. In Pearl’s framework,
variables of interest and their relations are represented by nodes and links
of a probabilistic graphical model, namely a causal graph, which contains the
researcher’s substantive knowledge and assumptions of the problem. Then the
causal graph is analysed to identify factors that affect both the treatment
node (usually item exposures) and the effect node (usually user ratings),
i.e., confounders, where deconfounding techniques such as backdoor adjustment,
etc., are conducted on the treatment node such that spurious correlation
between treatment and effect induced by confounders can be eliminated to
unbiasedly estimate the causal effect of the treatment. Confounders considered
in existing methods usually include item attributes [25, 26], user preferences
[16] and historical interactions [11, 17]. Existing solutions to backdoor
adjustment can be categorized into two classes: inverse propensity-score
reweighting (IPS)-based methods [27, 18, 28] and approximation-based methods
[16]. IPS-based methods reweight the samples by their observation
probabilities such that they can be viewed as uniformly selected from the
population, irrespective of confounders’ influences. However, the unbiasedness
of IPS requires a correctly specified score estimation model. Moreover, it
leads to unstable training dynamics if scores vary drastically among samples.
In contrast, approximation-based methods substantially reduce the training
variance, but most existing strategies will introduce extra bias if non-linear
relations exist between confounders and ratings [16].

In addition, compared with item recommendation tasks, causal inference in tag
recommendation tasks faces new challenges, such as complex treatment
distributions (where treatments are continuous UGC content instead of binary
item exposures), complex confounders (where confounders are UGC uploaders
instead of categorical item attributes or user preferences), and complex
relations among treatments, confounders, and outcomes (i.e., UGC content,
uploader, and tags). Therefore, the above challenges clearly motivate us to
design the proposed DecTag for UGC tag recommendations, where the
deconfounding of uploaders’ influence can be achieved with superior
performance, asymptotic consistency and unbiasedness, and good
interpretability.

##  3 Causal View of Tag Recommenders

Figure 1: Comparisons between causal graphs assumed by traditional tag
recommenders, by the proposed DecTag, and the causal graph of DecTag after
backdoor adjustment.

In this section, we formally define the task of tag recommendation from a
causal perspective. Specifically, a causal graph is established where
uploaders are clearly identified as confounders that spuriously correlate UGC
features and tags. In addition, we introduce backdoor adjustment to block
uploaders’ influence such that the causal effects of UGC content on tags can
be properly estimated.

###  3.1 Problem Formulation

In this paper, we mainly investigate uploader-UGC-tag triplets in the form of
$\mathcal{D}=\\{(\mathbf{u},\mathbf{c},\mathbf{t})_{i},i\in\mathrm{range}(N)\\}$222In
this paper, italic bold symbols $\mathbf{u},\mathbf{c},\mathbf{t}$ represent
vectors, calligraphic symbols $\mathcal{U},\mathcal{C},\mathcal{T}$ are used
to represent sets, and capital non-boldface symbols $U$, $C$, $T$ are used to
represent the corresponding random vectors, with exceptions such as scalars
$N$ and $N^{s}$ easily identifiable from the context.. The triplets are
collected from online UGC sharing platforms, where $\mathbf{u}\in\mathcal{U}$
is the representation of a selected uploader from the population, $\mathbf{c}$
is the feature of a content generated by uploader $\mathbf{u}$, and
$\mathbf{t}$ is one of the tags that uploader $\mathbf{u}$ selects for the
content. Since uploader $\mathbf{u}$ simultaneously creates the UGC content
and selects its tags, her personal bias unavoidably exists in the collected
triplets, which should be eliminated when estimating UGC content’s influence
on tags for unbiased content-based tag recommendations, i.e., its causal
effects. To achieve such a purpose, $\mathcal{D}$ needs to include multiple
UGCs from each uploader such that her bias can be properly modeled and
eliminated. Mathematically, $\mathbf{D}$ ought to have the property that in
$(\mathbf{u},\mathbf{c},\mathbf{t})_{i}$ there exists
$\mathbf{u}_{j}=\mathbf{u}_{k}$ such that $\mathbf{c}_{j}\neq\mathbf{c}_{k},\
\ \forall\mathbf{u}_{j}\in\mathcal{U}$. The goal of this paper is to recommend
suitable tags for newly uploaded UGCs based solely on their content features,
irrespective of any specific uploader’s personal preference, such that
confounding bias in tags that distracts audiences from locating suitable UGCs
can be eliminated for unbiased tag recommendations.

###  3.2 Causal Graph Modeling

To gain more intuition of the problem, we represent the causal relations among
$U$, $C$, $T$ as a graph in Fig. 1b, where links demonstrate the direction of
causal influence between a node pair. In Fig. 1b, link $U\rightarrow C$
depicts the influence of the uploader’s specialized area, personal background,
etc., on the content of the uploaded UGC, link $U\rightarrow T$ depicts the
influences of the uploader’s personal background, social status,
characteristics, and preference, etc., on her selected tags, and link
$C\rightarrow T$ depicts the influence of UGC’s content features to tag
selections. In this paper, the influence represented by link $C\rightarrow T$
is defined as the causal influence of UGC’s content features on tags, of which
we aim to recover from the collected U-C-T triplets, such that bias-free tag
can be recommended for newly uploaded UGCs based solely on their content.

Traditional tag recommender systems assume the simple relationship between
$C$, $T$ represented by Fig. 1a, which neglects the influence of UGC uploaders
as confounders. Consequently, when such models are fit on a UGC-tag dataset
actually generated according to Fig. 1b, the uploader’s selection bias is
mistakenly captured as the causal effect of $C$ on $T$. Consider the following
toy example of uploader-induced confounding bias. Suppose that Alice is an
outdoor streamer who constantly uploads videos about her outdoor activities.
As a naturalist, she prefers to select tags that describe the beautiful
natural view of the places where she shoots the videos instead of tags that
describe the sports she chooses to stream about. If a tag recommendation model
trained on Alice’s data unintentionally captures the spurious association
between outdoor videos and tags regarding natural views as the causal effect
of the video content on the tags, when Bob, who is also an outdoor streamer,
uploads a video regarding outdoor activities (such as parkour or skateboard)
on a busy street at downtown, Alice’s bias will lead the system to recommend
nature-associated tags for Bob’s content, which will cause Bob to lose
potential audiences who use sport-related tags to index the outdoor streaming
content that suits their interests. Theoretically, such an example vividly
illustrates the confounding bias associated with an open backdoor path where
the influence of C can be leaked through $C\leftarrow U\rightarrow T$ on $T$.
Unblocked backdoor paths can lead to systematic bias in estimated causal
effects for traditional non-causality-based tag recommendation models.

##  4 Methodology

We have clearly demonstrated how the uploader’s personal bias can create a
spurious correlation between UGC content and tags via backdoor path with
causal graph in the previous section. In this section, we introduce the
proposed deep deconfounded tag recommender, namely DecTag, for unbiased
content-based tag recommendations for UGCs.

###  4.1 Deconfound via Causal Intervention

To estimate the causal effects of UGC’s content $C$ on tag $T$, Pearl’s
causality theory [9] requires us to control the confounders such that backdoor
paths can be blocked. This is achieved through backdoor adjustment on UGC node
$C$ of the causal graph in Fig. 1b. According to the theory of causal
inference [9], the adjustment is formulated via do-operator as $P(T\mid
do(C=\mathbf{c}))$, which can be intuitively understood as cutting off the
edge $U\rightarrow C$ in the causal graph such that the effects of $U$ on $T$
leaked through path $C\leftarrow U\rightarrow T$ can be blocked for causal
effect estimation (see Fig. 1c). The specific form of backdoor adjustment is
formulated as:

$\displaystyle P(T\mid do(C=\mathbf{c}))$
$\displaystyle=\int_{\mathcal{U}}P(\mathbf{u}\mid do(C=\mathbf{c}))P(T\mid
do(C=\mathbf{c}),\mathbf{u})d\mathbf{u}$ (1a)
$\displaystyle=\int_{\mathcal{U}}P(\mathbf{u})P(T\mid\mathbf{c},\mathbf{u})d\mathbf{u},$
(1b)

where Eq. (1a) is based on the law of total probability and the Bayesian rule,
and Eq. (1b) follows the definition of do-operator [9] and conditional
independence relationships induced by the causal graph. In contrast,
traditional content-based tag recommender systems fits models according to
conditional distribution of $P(T\mid C=\mathbf{c})$ (i.e., causal graph Fig
1a) on data actually generated according to Fig 1b, which can be formulated as

$\displaystyle P(Y\mid
C=\mathbf{c},T=\mathbf{t})=\int_{\mathcal{U}}P(\mathbf{u}\mid\mathbf{c})P(T\mid\mathbf{c},\mathbf{u})d\mathbf{u}.$
(2a)

By comparing Eq. (1) and Eq.(2) we can find that through Eq. (1), DecTag
considers the tag $\mathbf{t}$ for content $\mathbf{c}$ the with every
possible uploader $u$ instead of the true uploader draw from
$P(\mathbf{u}\mid\mathbf{c})$. Therefore, the bias incurred by one specific
uploader can be eliminated.

###  4.2 Monte Carlo Sampler with Bootstrap

Despite the guarantee of DecTag to eliminate the uploader-induced confounding
bias, solving Eq. (1) is inherently intractable because of the integration
over the infinite uploader space, which precludes us from calculating an
analytical solution. Therefore, an approximation strategy should be introduced
to form estimators for the do-calculus integral.

####  4.2.1 Approximation with Hypothetical Averaged Uploader

If we define the expectation of uploader representation $\mathbf{u}$, i.e.,
$\int_{\mathcal{U}}P(\mathbf{u})d\mathbf{u}$ as $\hat{\mathbf{u}}$, the
estimator proposed by DecRS [16] can be formulated as

$\int_{\mathcal{U}}P(\mathbf{u})P(T\mid\mathbf{c},\mathbf{u})d\mathbf{u}\approx
P(T\mid\mathbf{c},\hat{\mathbf{u}}).$ (3)

Theoretical details of the approximation as well as its properties can be
referred to [16]. Here, we provide an intuitive explanation for the
deconfounding mechanism of DecRS. We observe that substituting $\mathbf{u}$
with $\hat{\mathbf{u}}$ in conditional distribution $P(T\mid C,U)$ can be
viewed as using a hypothetical averaged uploader $\hat{\mathbf{u}}$ as the
surrogate for the true uploader $\mathbf{u}$ with selection bias when training
the model. Therefore, if $\hat{\mathbf{u}}$ indeed possesses less bias than an
individual uploader $\mathbf{u}$, the adverse influence of uploaders as
confounders can be diminished by the substitution.

However, one problem of this approximation strategy is that it is
theoretically unclear whether the hypothetical averaged uploader
$\hat{\mathbf{u}}$ is indeed unbiased. Consider the toy example that we have
discussed in Section 3.2. Assume that one group of outdoor streamers is biased
towards selecting nature-related tags for their videos, whereas another group
of streamers is biased towards using sports-related tags. If the size of the
first group is larger than that of the second group, it can be expected that
the hypothetical averaged uploader $\hat{\mathbf{u}}$ is also biased towards
using nature-related tags. We call this bias the residual bias in the averaged
uploader. This bias cannot be addressed by [16]. Furthermore, if the
conditional distribution of tags $P(T\mid C,U)$ is highly non-linear for the
uploader $U$, which is common in tag recommendation due to the complex
relations among uploaders, UGCs, and tags, the approximation could be biased
even if the biases of all uploaders can be negated by averaging. In summary,
we can determine that when the conditional distribution $P(T\mid C,U)$ is non-
linear in $U$, based on the above analysis we can find that the estimator of
DecRS is neither asymptotically consistent nor unbiased for estimating the do-
calculus in Eq. (1).

####  4.2.2 Approximation with Hypothetical Random Uploader

To address the problems, we design a Monte-Carlo (MC)-based approximator with
bootstrap for DecTag in this section. We note that the integral in Eq. (1b)
can be rewritten into an expectation form as follows,

$\int_{\mathcal{U}}P(\mathbf{u})P(T\mid\mathbf{c},\mathbf{u})d\mathbf{u}=\mathbb{E}_{P(U)}[P(T\mid\mathbf{c},U)].$
(4)

With $P(T\mid\mathbf{c},\mathbf{u})$ explicitly formulated in an expectation
form, Monte Carlo method can be introduced to form unbiased estimator for the
LHS of Eq. (4). Specifically, we first draw $N^{s}$ uploaders from $P(U)$ as
$\mathbf{u}_{i}\sim P(U),i\in\mathrm{range}(N^{s})$, where $N^{s}$ is the
sample size. Then, the expectation in Eq. (4) can be unbiasedly estimated by

$\mathbb{E}_{P(U)}[P(T\mid\mathbf{c},U)]\approx\frac{1}{N^{s}}\sum_{i=1}^{N^{s}}P(T\mid\mathbf{c},\mathbf{u}_{i}),$
(5)

where the increase of $N^{s}$ linearly reduces the estimation variance while
increasing the computational complexity simultaneously. The deconfounding
mechanism of the proposed DecTag has an intuition of using a hypothetical
random uploader with probability distribution P(U) to substitute the biased
real uploader $\mathbf{u}$, where the expectation of tags provided by the
random uploader are calculated as the predictions. Compared to DecRS, the
randomness in DecTag could guarantee the elimination of the uploader-induced
confounding bias even if $P(T\mid C,U)$ is highly non-linear for uploader $U$.

Figure 2: Comparisons between hypothetical averaged uploader used in DecRS and
hypothetical random uploader proposed by DecTag for deconfounding uploaders’
bias.

Since the uploader distribution $P(U)$ is assumed to be complex, it cannot be
confidently estimated from the dataset. Therefore, another problem remaining
to be solved in DecTag is how to draw multiple samples from the true uploader
distribution $P(U)$. In this paper, we resort to the bootstrap strategy [29].
Specifically, we draw surrogate samples from the empirical uploader
distribution, which is defined as
$\hat{P}(U=\mathbf{u})=\sum_{i=1}^{N}\mathbb{I}(\mathbf{u}_{i}=\mathbf{u})/N$,
with replacement. $\mathbb{I}$ is the identity function. The MC-estimator with
bootstrap can achieve asymptotic consistency and unbiasedness provided that
the uploaders in the collected datasets are i.i.d. samples from the population
[29]. This can be easily guaranteed in the data collection procedure, where we
first randomly sample uploaders as seed uploaders and from each seed uploader
randomly selected uploaded UGCs from their collections. The comparison between
the deconfounding strategy of DecRS and the proposed DecTag is intuitively
illustrated in Fig. 2 for reference.

####  4.2.3 Bias-Variance Trade-off between DecRS and DecTag

Please note that the asymptotically consistency and unbiasedness properties of
DecTag are never a free lunch. The estimator of DecRS has a low variance by
adopting a deterministic hypothetical averaged uploader $\hat{\mathbf{u}}$ as
the surrogate for the true uploader. This may have an advantage if the
datasets are small-scale or extremely sparse. In contrast, the stochastic MC-
based sampling strategy introduced in DecTag requires a larger sample size
$N^{s}$ to obtain equally low variance in estimation. Therefore, a higher
training and testing complexity is generally expected for DecTag compared with
DecRS. Previous work shows that it is possible to set $N^{s}=1$ to achieve a
low variance provided that the batch size of uploaders is large [30]. This
strategy is adopted for training DecTag due to its simplicity.

####  4.2.4 Specific Form of $\mathbf{u}$ as Confounders

Since only unconfounded conditional distributions (for example, causal Markov
kernels) can be faithfully estimated from confounded datasets [9], we first
fit distribution $P(T\mid C,U)$ on dataset $\mathcal{D}$ for $N_{warm}$ epochs
to warm up the model (see Algorithm 1 for details). After the burn-in phase
accomplishes, the hypothetical random uploader of the MC-based estimator is
then used as the surrogate for true biased uploaders in $P(T\mid C,U)$ as Eq.
(4) to eliminate the uploader-induced confounding bias in tag selections.

Input: Uploader-UGC-tag triplets $\mathcal{D}$; Uploader set $\mathcal{U}$;
The number of warm-up epochs $N_{warm}$; The sample size of Monte-Carlo
(MC)-based approximator $N^{s}$;

Randomly initialize the trainable parameters $\theta$.

Randomly sample $N_{b}$ mini-batches of triplets
$\\{D_{1},D_{2},...,D_{N_{b}}\\}$.

// Burn-in phase

for  _$i=1$ to $N_{warm}$_ do

for  _$j=1$ to $N_{b}$_ do

Initialize the loss $l$ as $0$.

forall  _$(\mathbf{u},\mathbf{c},\mathbf{t})\in\mathcal{D}_{j}$_ do

Predict the recommendation probability $P(\mathbf{t}|\mathbf{u},\mathbf{c})$.

Calculate the predict loss $\hat{l}$ based on ground truth.

Add $l$ with $\hat{l}$.

end forall

Compute the gradient of the loss $l$.

Update $\theta$.

end for

end for

return _Trained Parameter $\theta$_

Algorithm 1 Burn-in phase of DecTag.

In this paper, we assume that the collected UGCs are i.i.d. samples from the
collection of each uploader. This assumption implies that the sampling UGC
distribution can faithfully represent the true UGC distribution of the
uploaders. Based on this, we calculate the statistics of the collected UGCs of
an uploader as her representation $\mathbf{u}$. Specifically, we first obtain
the topics of each UGC included in the dataset (This can be easily acquired
since most platforms, such as YouTube and Instagram, etc., have categorized
contents based on topics). We then randomly initialize trainable UGC topic
embeddings $\mathbf{W}_{t}\in\mathbb{R}^{K\times N_{tp}}$ and calculate the
ratio of each topic in uploaders’ collections, where $N_{tp}$ is the number of
UGC topic categories and $K$ is the dimension of topic embeddings. If we
define $Tp\in\mathrm{range}(1,N^{t})$ as the $Tp$th topic category,
$\mathcal{C}_{\mathbf{u}}$ as the set of collected UGCs for uploader
$\mathbf{u}$, and $Cat(\mathbf{c})$ as the function that returns the topic
category of UGC $\mathbf{c}$, the $Tp$th dimension of $\mathbf{u}$’s UGC topic
distribution $\hat{\mathbf{u}}$ can be calculated by
$\hat{\mathbf{u}}_{Tp}=\sum_{\mathbf{c}\in\mathcal{C}_{u}}\mathbb{I}(Cat(\mathbf{c})=Tp)\
/\ |\mathcal{C}_{u}|$. The uploader representation is then calculated as
$\mathbf{u}=\mathbf{W}_{t}\cdot\hat{\mathbf{u}}$, which is the weighted sum of
UGC topic embeddings. Deep latent variable model can be further introduced by
viewing $\mathbf{u}$ as noisy observations of latent confounders
$\mathbf{z}_{u}$ when $\mathbf{u}$ is a sparse vector lies in a high-
dimensional space [31]. However, this is beyond the scope of this paper and
can be explored as future research.

###  4.3 Model-Agnostic Training Strategy

The derivation of the deconfounding ability of DecTag in Eq. (4) is based
solely on the abstract conditional distribution $P(T\mid C,U)$, which shows
that the deconfounding strategy of the proposed DecTag is model-agnostic and
can be built upon arbitrary content-based tag recommender systems provided
that the uploader for each UGC is known to the system. For DecTag to
materialize, the functional form that parameterizes $P(T\mid C,U)$ remains to
be specified. Theoretically, any function
$\mathrm{f}_{\boldsymbol{\theta}}:\mathcal{C}\times\mathcal{U}\rightarrow\mathcal{T}$
with signature of $U$, $C$ as inputs, probabilities for predicted tags $T$ as
outputs, and $\boldsymbol{\theta}$ as trainable parameters can be used to
parameterize the distribution. In this paper, we assume that $P(T\mid C,U)$
can be decomposed into two compact factors as

$\frac{1}{E}\hat{P}(T\mid C)\cdot\hat{P}(T\mid U),$ (6)

such that it can explicitly disentangle uploader’s bias toward tags and the
influence of UGC’s content to tags [32, 33]. $\hat{P}$ represents unnormalized
probability distribution function, and $E$ is the normalization constant. We
also note that Eq. (6) shows the form of a product-of-expert system, which can
be mathematically derived with further assumptions according to [34]. The
backdoor adjustment-based training procedure of DecTag is summarized in
Algorithm 2 for reference.

Input: Uploader-UGC-tag triplets $\mathcal{D}$; Uploader set $\mathcal{U}$;
The number of warm-up epochs $N_{warm}$; The sample size of Monte-Carlo
(MC)-based approximator $N^{s}$;

Trainable parameters $\theta$ obtained from the burn-in phase.

Randomly sample $N_{b}$ mini-batches of triplets
$\\{D_{1},D_{2},...,D_{N_{b}}\\}$.

// Training with backdoor adjustment

while  _not converged_ do

for  _$i=1$ to $N_{warm}$_ do

for  _$j=1$ to $N_{b}$_ do

Initialize the loss $l$ as $0$.

forall  _$(\mathbf{u},\mathbf{c},\mathbf{t})\in\mathcal{D}_{j}$_ do

// Monte-Carlo sampling

Initialize the recommendation probability $\mathbf{p}$ as $0$.

for  _$k=1$ to $N^{s}$_ do

Randomly sample one use $\mathbf{u}_{k}$ from $\mathcal{U}$.

Predict $P(\mathbf{t}|\mathbf{u}_{k},\mathbf{c})$.

Add $\mathbf{p}$ with $P(\mathbf{t}|\mathbf{u}_{k},\mathbf{c})$.

end for

Divide $\mathbf{p}$ by $N^{s}$.

Calculate the predict loss $\hat{l}$ based on ground truth.

Add $l$ with $\hat{l}$.

end forall

Compute the gradient of the loss $l$.

Update $\theta$.

end for

end for

end while

return _Adjusted Parameter $\theta$_

Algorithm 2 Backdoor adjustment phase of DecTag.

###  4.4 Base Models for Implementations

Specifically, we adopt two widely-used strategies, i.e., neural factorization
machine (NFM) [35] and LightGCN [36], to implement the two decomposed factors
of $P(T\mid C,U)$, i.e., $\hat{P}(T\mid C)$ and $\hat{P}(T\mid U)$. Although
both methods are CF-based originally, they can be easily adapted for content-
based recommendations by replacing the UGC ID embeddings with their content
features. To avoid redundant discussions, we define $X\in\\{C,U\\}$. In our
implementation of NFM, the two unnormalized probabilities are calculated as:

$\hat{P}(\mathbf{t}\mid\mathbf{x})=f_{out}\Big{(}M\Big{(}M(\mathbf{t})\odot
M(\mathbf{x})\ ||\ M\big{(}M(\mathbf{t})\ ||\
M(\mathbf{x})\big{)}\Big{)}\Big{)},$ (7)

where $f_{out}$ is the identity function
$f_{out}(\mathbf{x_{in}})=\mathbf{x_{in}}$ if $X=C$ and is the sigmoid
function $f_{out}(\mathbf{x_{in}})=1/(1+e^{-\mathbf{x_{in}}})$ if $X=U$, $M$
represents the multi-layer perceptron, $\mathbf{t}$ is the one-hot
representation of the target tag, $\odot$ is element-wise product, and $||$
represents the concatenation operator. Another backbone model, i.e., LightGCN,
is a state-of-the-art graph neural network introduced in [36]. In LightGCN, a
bipartite graph is first established between $T$, $C$ based on the collected
U-C-T triplets. For the graph, messages from UGC content features $C$ to tags
$T$ are then propagated as follows:

$\mathbf{t}_{i}^{(k+1)}=\operatorname{AGG}\left(\mathbf{c}_{j}^{(k)},\left\\{\mathbf{c}_{j}^{(k)}:\mathbf{c}_{j}\in\mathcal{N}_{\mathbf{t}_{i}}\right\\}\right),$
(8)

where $\operatorname{AGG}$ is the LightGCN aggregation function defined in
[36], $i$, $j$, $k$ are used to distinguish different tags, UGC content, and
representations from different GCN layers, respectively.
$\mathcal{N}_{\mathbf{t}_{i}}$ denotes the set of neighborhood for tag
$\mathbf{t}_{i}$. Similar message passing mechanism exists from $T$ to $C$.
Finally, inner product between aggregated tag representations and UGC content
representations is calculated as unnormalized probabilities for $\hat{P}(T\mid
C)$. The uploader-conditional tag distribution $\hat{P}(T\mid U)$ is
calculated by taking the inner product between uploader representation
$\mathbf{u}$ and tag embeddings from the first GCN layer learned on the $T-C$
graph established based on the collected data.

###  4.5 Tag Prediction for Newly Uploaded UGCs

After training, the learned $P(T\mid C,U)$ can be used for deconfounded tag
recommendations. Specifically, when uploader $\mathbf{u}$ uploads a new UGC
$\mathbf{c}_{new}$ to the platform, Eq. (4) can be used to form unbiased tag
predictions for $\mathbf{c}_{new}$. This can be intuitively understood as
randomly selecting $N^{s}$ uploaders from the population and calculating
averaged tag predictions. The averaged scores are then ranked where the
top-$K$ tags are selected for recommendations.

##  5 YT-8M-Causal Dataset

It is notoriously difficult to evaluate deep causal recommenders on real-world
datasets due to the lack of groundtruth. The problem is especially evident for
the tag recommendation task considered in this paper because the bias-free
tags for each UGC, (i.e., the tags that best fit the UGC based solely on its
content) is inherently inaccessible. Despite the difficulties, we established
a large-scale real-world tag recommendation dataset, namely YT-8M-causal,
based on the widely adopted YouTube-8M dataset. With a dataset intervention
strategy introduced accordingly, the deconfounding ability of different
strategies can be indirectly evaluated and compared.

###  5.1 Data Collection Before Intervention

Table 1: Statistics of the established YT-8M-Causal dataset. Please note that #Tag Usage in the table shows the sum of tag used in all the videos in the sub-datasets. Topics | #Users | #Videos | #Tags | #Tag Usage
---|---|---|---|---
P.A. & H.L. | 17,125 | 38,442 | 6,308 | 242,471
A.V. & B.I. | 21,686 | 53,399 | 7,203 | 337,751
C.I. & I.T. | 8,140 | 26,437 | 4,755 | 190,860

YouTube-8M [37] is one of the largest multi-label video classification
datasets collected from YouTube333https://www.youtube.com/, which is a world-
famous video-sharing platform with billions of active users. The dataset
consists of 8 million videos from 24 categories of topics, where for each
video, a 1,024-dimensional Inception-net-based feature is extracted as the
video content representation $\mathbf{c}$. Other features, such as context
features, can also be included as [7]. However, we mainly focus on UGC
contents such that the impact of uploaders’ confounding bias on content-based
tag selections can be better investigated in this paper. To establish the
YT-8M-Causal dataset. We first select two groups of videos in four first
categories, i.e., Pets & Animals (P.A.) and Hobbies & Leisure (H.L.) for the
first group, Autos & Vehicles (A.V.) and Business & Industrial (B.I.) for the
second group, and Computers & Electronics (C.E.) and Internet & Telecom (I.T.)
for the third group to form three sub-sections of the dataset. We then collect
the uploader-selected tags of each video via the YouTube Data API (v3).
Filtering is applied on tags in each sub-dataset to ensure the quality of
recommended tags, where tags with less than two letters or used by less than
20 videos are deemed as low-quality and are excluded from the dataset.
Afterward, videos with no tags are further removed according to the tag
filtering results. Finally, we remove uploaders with less than five uploaded
videos to ensure the quality of uploader representations. The dataset
statistics are summarized in Table 1.

###  5.2 Analysis of the Pre-processed Dataset

In the analysis, we first illustrate the distribution of tags for UGCs from
two topics in both subsections of the dataset. From Fig. 3 we can find that
the two UGC topics in each sub-dataset are closely related as most tags are
used in UGCs from both topics. This is desired to study the confounding bias
[38], because when topic $A$ and $B$ are highly related, uploaded videos may
contain both $A$ and $B$ topics, where some uploaders may be more willing to
assign $A$-topic-related tags while others might prefer $B$-topic-related
tags. Under such cases, a deconfounded tag recommender can help them to select
tags based solely on the UGC content irrespective of uploaders’ personal bias.
Then, we analyze uploaders and tags from the perspective of the distribution
of (a) the topics of videos in each uploader’s collections and (b) the topics
of videos with which each tag has been associated, respectively. The two
distributions are illustrated in Fig. 4. From the top part of Fig. 4 we can
determine that most uploaders tend to publish their UGC from both topic
categories in both subsections of the YT-8M-Causal dataset. Since the
uploaders induce the tag-selection bias which we would like to eliminate in
DecTag, and an uploader’s difference in tendency to select tags from one
familiar category to describe her uploaded UGCs with cross-domain
characteristics is one of the main sources of confounding bias in tag
selections, this further demonstrates the value of the established
YT-8M-Causal dataset to facilitate research that addresses confounding bias in
tag recommendations. Similar analysis can be applied to tags illustrated at
the bottom of Fig. 4, which we omitted to reduce redundancy.

###  5.3 Causal Intervention on Test Sets

Figure 3: Tag co-occurrences in two selected topics of the three subsection of
the established YT-8M-Causal dataset. The $ii$th element of the matrices shows
the percentage of tags exclusive for topic $i$, whereas the $ij$th element
shows the percentage of mutual tags for topics $i,j$.

After the preparation of all the data required by DecTag, our discussions
shift to the intervention strategy applied to the test set such that the
deconfounding ability of causal tag recommenders can be evaluated without the
requirement for bias-free tags.

####  5.3.1 Intervening on Test Set by Substitute Confounders

As demonstrated in [39], the deconfounding ability of causal recommenders can
be indirectly assessed by intervening in the distribution of treatment (UGC
content in the case of tag recommendations) on the test set. The intervention
is achieved by manually controlling the difference of treatments’ distribution
induced by confounders between the training set and test set. The rationale
behind such an evaluation strategy is that, since the influence of uploaders
is expected to be eliminated by an ideal deconfounded tag recommender system,
the system selects tags based solely on the content of UGCs; therefore,
recommendation accuracy should not degenerate on the intervened test set.
Otherwise, the spurious correlation induced by uploaders is mistakenly
captured by the system as the causal influence from the UGC content, where the
recommendation performance will unavoidably decrease when the correlation
changes in the test set. In addition, since in most cases, intervention cannot
be achieved by directly changing the value of confounders, we intervene upon
substitute confounders that have similar effects to the true confounders,
i.e., the uploaders. Through this mechanism, the dataset intervention strategy
avoids the intractable requirement of obtaining the bias-free tags for each
uploaded UGC, which is a tractable means to evaluate causal tag recommender
systems.

Specifically, we choose the UGCs’ topic as the substitute confounder upon
which we conduct intervention because it simultaneously influences the content
of UGC the uploaders tend to create and the tags selected for the content.
Consequently, we assume that UGC topics could exert influences on tag
selections similar to uploaders as true confounders, and controlling the UGC
topic distribution as surrogate confounder provides us with a viable strategy
to intervene the dataset according to the influence of true confounders. In
our implementation, we split both subsections of the YT-8M-Causal dataset into
training, validation, and test sets such that the ratio of two selected topics
is $X:(10-X)$ for the training set and $(10-X):X$ for the validation and test
sets. $X\in\operatorname{range}(1,9)$ represents the ratio of UGCs from the
first topic category. In particular, $X$ controls the strength of intervention
conducted upon the test set, where $X$ with a value close to extremity 1 or 9
demonstrates a stronger intervention imposed upon the test set.

Figure 4: Topic distribution for videos (1) from each uploader (Top) and (2)
with which each tag has been associated (Bottom). $y$-axis is clipped to 80 /
1,000 for better illustration.

####  5.3.2 Further Discussions

It is fully possible to select more than two UGC topics to establish the
YT-8M-Causal dataset. We select only two categories of topic with intimate
cross-topic relations because in such cases, more severe confounding bias
exists in collected UGC-tag pairs, where deconfounded tag recommenders can be
of great benefit as they increase UGCs’ exposure to the right target audience
interested in both topics by eliminating uploaders’ personal bias in tag
selections. In contrast, if more topics are included, some of the topics may
not be related to each other, where deconfounding brings about fewer
advantages in comparison.

##  6 Experiments

Table 2: Comparisons between DecTag with other debias strategies on two base
recommendation methods, NFM and LightGCN, on the established YT-8M-Causal
datasets. Method with the highest performance is highlighted in Bold. We also
conduct single-sided t-test on the results and report the p-values with base
model to demonstrate the significance of the improvement.

| Dataset: YT-8M-Causal (P.A. & H.L.)
---|---
methods | Basemodel: NFM | Basemodel: LightGCN
R@10 | R@20 | N@10 | N@20 | R@10 | R@20 | N@10 | N@20
None | 0.2189 | 0.3069 | 0.2002 | 0.2289 | 0.2320 | 0.3079 | 0.2244 | 0.2450
Unawareness | 0.2217 | 0.3125 | 0.2112 | 0.2327 | 0.2376 | 0.3124 | 0.2308 | 0.2509
FairCo | 0.2166 | 0.3055 | 0.1996 | 0.2286 | 0.2308 | 0.3127 | 0.2230 | 0.2459
Calibration | 0.2147 | 0.3054 | 0.1945 | 0.2192 | 0.2318 | 0.3073 | 0.2237 | 0.2442
IPS | 0.2210 | 0.3123 | 0.2050 | 0.2348 | 0.2385 | 0.3145 | 0.2283 | 0.2496
CR | 0.2226 | 0.3133 | 0.2078 | 0.2362 | 0.2414 | 0.3178 | 0.2331 | 0.2540
DecRS | 0.2255 | 0.3150 | 0.2098 | 0.2382 | 0.2434 | 0.3213 | 0.2377 | 0.2578
DecTag | 0.2314 | 0.3200 | 0.2233 | 0.2505 | 0.2492 | 0.3291 | 0.2431 | 0.2640
p-value | 5.71E-2 | 6.35E-2 | 1.03E-2 | 7.58E-3 | 2.12E-4 | 2.27E-4 | 9.15E-4 | 4.51E-4

| YT-8M-Causal (A.V. & B.I.)
---|---
methods | Basemodel: NFM | Basemodel: LightGCN
R@10 | R@20 | N@10 | N@20 | R@10 | R@20 | N@10 | N@20
None | 0.1900 | 0.2673 | 0.1680 | 0.1928 | 0.1920 | 0.2643 | 0.1786 | 0.2001
Unawareness | 0.1923 | 0.2713 | 0.1804 | 0.2023 | 0.2028 | 0.2812 | 0.1888 | 0.2135
FairCo | 0.1877 | 0.2623 | 0.1700 | 0.1935 | 0.1907 | 0.2629 | 0.1798 | 0.2017
Calibration | 0.1879 | 0.2662 | 0.1612 | 0.1856 | 0.1916 | 0.2634 | 0.1779 | 0.1992
IPS | 0.1919 | 0.2703 | 0.1717 | 0.1968 | 0.1997 | 0.2717 | 0.1843 | 0.2065
CR | 0.1931 | 0.2717 | 0.1745 | 0.2004 | 0.2046 | 0.2742 | 0.1939 | 0.2151
DecRS | 0.1960 | 0.2737 | 0.1802 | 0.2046 | 0.2125 | 0.2815 | 0.1988 | 0.2224
DecTag | 0.1974 | 0.2750 | 0.1838 | 0.2082 | 0.2168 | 0.2937 | 0.2020 | 0.2250
p-value | 1.11E-1 | 3.75E-2 | 1.12E-2 | 6.93E-3 | 8.64E-6 | 5.07E-5 | 4.66E-5 | 1.01E-4

Dataset | YT-8M-Causal (C.E. & I.T.)
---|---
methods | Basemodel: NFM | Basemodel: LightGCN
R@10 | R@20 | N@10 | N@20 | R@10 | R@20 | N@10 | N@20
None | 0.2627 | 0.3539 | 0.2780 | 0.3068 | 0.2650 | 0.3555 | 0.2867 | 0.3101
Unawareness | 0.2735 | 0.3632 | 0.2958 | 0.3224 | 0.2773 | 0.3605 | 0.3026 | 0.3209
FairCo | 0.2601 | 0.3527 | 0.2800 | 0.3084 | 0.2635 | 0.3544 | 0.2864 | 0.3092
Calibration | 0.2584 | 0.3508 | 0.2743 | 0.3043 | 0.2638 | 0.3521 | 0.2835 | 0.3050
IPS | 0.2650 | 0.3557 | 0.2804 | 0.3090 | 0.2701 | 0.3572 | 0.2949 | 0.3190
CR | 0.2774 | 0.3664 | 0.2998 | 0.3258 | 0.2782 | 0.3620 | 0.3031 | 0.3228
DecRS | 0.2766 | 0.3669 | 0.3010 | 0.3278 | 0.2834 | 0.3661 | 0.3055 | 0.3240
DecTag | 0.2798 | 0.3688 | 0.3029 | 0.3293 | 0.2885 | 0.3683 | 0.3072 | 0.3299
p-value | 2.81E-5 | 6.22E-5 | 2.35E-6 | 4.59E-6 | 7.50E-7 | 4.64E-4 | 7.59E-7 | 3.38E-6

In this section, we report the extensive experiments conducted on the
established YT-8M-Causal dataset to demonstrate the effectiveness of the
proposed DecTag. Specifically, we focus on investigating the three important
research questions as follows:

  * •

RQ1: How does the proposed DecTag perform on various base models compared with
other deconfounding strategies in terms of recommendation accuracy and
robustness to confounding bias on three sub-sections of the established
YT-8M-Causal datasets?

  * •

RQ2: How does the performance of different deconfounding strategies change
with the increase of intervention strength (i.e., the difference of UGC topic
distribution in the training and test sets) on the YT-8M-Causal test sets?

  * •

RQ3: Could a large batch size indeed help DecTag reduce the estimation
variance of the proposed MC-based do-calculus estimator with bootstrap when
sample size $N^{s}$ is small (e.g., one)?

###  6.1 Experimental Setups

####  6.1.1 Training Strategy

As we have discussed in Section 4.2.3, a large batch size is equivalent to a
large sample size for the MC-based do-calculus to reduce the estimation
variance. Therefore, batch size is set to 1,024 for NFM backbone and 4,096 for
LightGCN backbone to stabilize the training dynamics, which is larger than
batch size normally used in tag recommendations. $N^{s}$ is set to 1. We use
Adam optimizer [40] with learning rate 1e-3. Five epochs are used to learn the
conditional distribution to warm up the model (Section 4.2.4), and the
deconfounded training stops at the 200th and 400th epoch for NFM and LightGCN
backbone, respectively. The parameter that controls dataset intervention
strength, $X$, is set to 3 for comparisons with other deconfounding/debiasing
strategies in Section 6.2.2, and its effects are thoroughly discussed in
Section 6.3.

####  6.1.2 Evaluation Metrics

In this paper, the performance of different tag recommendation systems is
measured by two commonly-used ranking-based metrics, i.e., Recall@K (R@K) and
NDCG@K (N@K). As we have demonstrated in Section 4.2.3, a higher accuracy on
the intervened test set where the UGC topic distribution differs from that of
the training set can also demonstrate the deconfounding ability of different
strategies under certain assumptions. Therefore, these two metrics can also be
used to compare the deconfounding ability of different strategies.

Specifically, we first obtain the rank of the candidate tags by sorting their
match score calculated by the DecTag model. If we denote the tag at rank $r$
by $t(r)$ and the set of the remaining tags for UGC $c$ by $\mathcal{T}_{c}$,
the Recall@$K$ metric can be calculated as:

$\operatorname{Recall}@K(c)=\frac{\sum_{r=1}^{K}\mathbb{I}\left[t(r)\in\mathcal{T}_{c}\right]}{\min\left(K,\left|\mathcal{T}_{c}\right|\right)},$
(9)

where $\mathbb{I}$ in the numerator is the indicator function, and the
denominator is the minimum of $M$ and the number of candidate tags. Recall@$K$
has a maximum of 1, which is achieved when all suitable tags are ranked among
the top $K$ positions. Truncated discounted cumulative gain (DCG@$K$) is
computed as

$\operatorname{DCG}@K(i)=\sum_{r=1}^{K}\frac{2^{\mathbb{I}\left[j(r)\in\mathcal{J}_{i}\right]}-1}{\log(r+1)},$
(10)

which, instead of uniformly weighting all positions, introduces a logarithm
discount function over the ranks where larger weights are applied to
recommended tags that appear at higher ranks. NDCG@$K$ is calculated by
normalizing the DCG@$K$ metric to [0, 1] by the ideal DCG@$K$ where all
relevant tags are ranked at the top positions.

Figure 5: Comparisons between DecTag and different deconfounding/debiasing
strategies on the established YT-8M-Causal dataset with varied test set
intervention strength. Error bar shows the standard deviation on five splits
of the dataset.

###  6.2 Comparisons with Baselines

####  6.2.1 Baseline Descriptions

To answer RQ1, we compare the proposed DecTag model with state-of-the-art
debiasing and deconfounding strategies on two widely-adopted baseline
recommendation models, i.e., NFM [35] and LightGCN [36], as follows:

  * •

None. This is the naive strategy where no debiasing or deconfounding
strategies are applied to the baseline models. Specifically, the strategy
directly fits the conditional distribution $P(T\mid U,C)$ on the uploader-
confounded dataset to predict tags for new UGCs.

  * •

Unawareness [41]. A tag recommender system with an Unawareness strategy is
achieved by explicitly excluding the user-related features from the model when
predicting tags for UGCs. Therefore, it fits the conditional distribution
$P(T\mid C)$ on the confounded dataset where only content features of UGCs are
considered for tag predictions.

  * •

FairCo [42]. This is the dynamic Learning-to-Rank algorithm which introduces
an error term to control exposure-based fairness among different groups. In
our implementation, we divide tags into different category groups based on
historical data and calculate the error term based on the ranking list sorted
by relevance.

  * •

Calibration [43]. This method alleviates bias amplification by introducing and
controlling a calibration metric $C_{KL}$ that measures the imbalance between
interaction history and recommendation list. n our implementation, we
calculate $C_{KL}$ based on the historical tags of uploaders and minimize it
by re-ranking.

  * •

IPS [44]. IPS aims to eliminate confounding bias by reweighting the samples by
the inverse of the chance of their observation (i.e., the propensity score).
The propensity score of each UGC is estimated from its uploader’s
representation as the covariate. In addition, a clipping technique [44] is
introduced to reduce the estimation variance.

  * •

CR [45]. CR was originally designed to address the clickbait issue of online
e-commerce platform by conducting counterfactual inference on causal graphs.
In this work, similar procedures can be applied to estimate the matching
likelihood of a UGC-tag pair in a counterfactual world such that the effect of
uploaders’ bias can be reduced.

  * •

DecRS [16]. DecRS is a deconfounded recommender system where the deconfounding
of uploader’s adverse influence (in the case of tag recommendation) is
achieved by substituting a hypothetical averaged uploader for the true
uploader when training the model, such that the bias of a specific uploader
can be reduced by averaging.

For a fair comparison, the base model structures and the hyperparameters of
both the base models and the applied deconfounding/debiasing strategies, such
as the embedding size of uploader, UGC, and tags, the batch size, $\lambda$ in
the ranking target for FairCo and Calibration, and the clipping threshold $t$
for IPS, are systematically determined through grid research.

####  6.2.2 Comparison Results and Analysis

The comparison results are summarized in Table 2 for reference. Table 2 first
reveals that the Unawareness strategy can improve over the None strategy where
no deconfounding/debiasing is applied to the base model. Recall that both
strategies fit the biased models (i.e., naive conditional distributions) on
the confounded dataset with the discrepancy that Unawareness completely
discards the uploader information and uses only UGC content as the condition.
This may suggest that uploaders indeed exert heavy influence on tag
selections, where uploader-induced confounding bias is exacerbated. However,
the Unawareness strategy can only address explicit uploader-related
confounding bias, because UGC content might be further disentangled into
useful information and implicit confounders such that spurious association
endures even with the removal of uploaders. Consider again the toy examples
introduced in Section 3.2. Suppose that video uploaders who bias to nature-
related tags also tend to include green colors (from forests, prairie, or
meadows, etc.) in their videos. Under such cases, even if Unawareness ignores
uploaders in its conditional modeling, part of the UGC content, i.e., the
green color in the example, can still serve as a surrogate for uploaders that
spuriously correlates UGC content and tags. Moreover, we also find that
Calibration and FairCo may perform worse than the backbone models, which
suggests that simple re-ranking or group fairness constraint without causality
may degenerate tag recommendation models’ performance and robustness to
confounding bias [46, 16].

With causal inference techniques explicitly introduced to address uploaders’
confounding bias, the performance of two base models improves for CR and DecRS
compared to Unawareness and Calibration. Although causality has been employed
in IPS, which deconfounds by reweighting the UGC-tag pairs based on the
inverse of propensity scores estimated from uploader representations, it
achieves the worst performance among all causality-based methods. The reason
could be that the unbiasedness of IPS requires a correctly specified score
estimation model, which relies heavily on the expertise of the researchers.
Moreover, even with clipping strategies, IPS may lead to unstable training
dynamics if the estimated propensity scores vary drastically for different
samples. Conducting counterfactual inference on the causal graph to estimate,
CR improves over IPS, due to the comparatively low variance of counterfactual
reasoning to estimate the likelihood of UGC-tag pairs than IPS-based methods.
Utilizing a hypothetical averaged uploader to replace the true biased uploader
for backdoor adjustment, DecRS performs the best among all included
deconfounding strategies. However, the unbiasedness of DecRS requires two
strong assumptions, i.e., no residual bias and a linear relation between tag
selections and uploader representations, both of which, as we have analyzed in
Section 4.2.3, may not hold in the case of tag recommendations.

The proposed DecTag achieves better performance than DecRS on both datasets
and improves significantly over the base models (see the p-values at the
bottom of each sub-table). This demonstrates the superiority of MC-based do-
calculus approximator with bootstrap to solve the backdoor adjustment on
causal graphs for tag recommendations, where unbiasedness can be achieved with
weaker assumptions compared with DecRS, i.e., the uploaders are i.i.d. samples
from the population. The training time complexity, when using a larger batch
size instead of a larger sample size for the MC estimator, increases only
slightly compared with DecRS and can be effectively vectorized by a single
GTX1080 GPU with 8G memory. In addition, since the MC-based approximator can
be viewed as replacing the biased uploader with an unbiased hypothetical
random uploader when making recommendations, the deconfounding of uploaders’
influences in DecTag can be achieved in an interpretable manner. Therefore,
these clearly demonstrate the advantage of DecTag compared with other state-
of-the-art deconfounding/debiasing strategies for UGC tag recommendations.

Figure 6: The sensitivity analysis of the proposed DecTag’s performance to the
sample size $N^{s}$ of the proposed MC-based do-calculus approximator when the
batch size is large (i.e., 1,024 for NFM base model and 4,096 for LightGCN
base model).

###  6.3 Sensitivity to Dataset Intervention

To answer RQ2, we vary the strength of intervention imposed on the
YT-8M-Causal test sets, i.e., the ratio of the two topics that we intervene
upon and evaluate different deconfounding/debiasing strategies. Specifically,
we vary $X\in\\{\operatorname{range}(1,5,2)\\}$ and conducted similar
experiments as Section 6.2.2. The comparison results are illustrated in Fig.
5. From Fig. 5 we can find that with the increase of intervention strength
conducted upon the test set, the performance decreases for all
deconfounding/debiasing strategies for both the NFM and LightGCN backbones.
This could either indicate that there exists a systematic difference between
the prediction accuracy of tags recommended for UGCs from the two topics in
each subsection of the dataset, or residual confounding bias exists that fails
to be completely eliminated by the deconfounding strategies. However, the
proposed DecTag achieves the best performance compared with the state-of-the-
art deconfounding/debiasing strategies for tag recommendations in all the
considered intervention strength scenarios, which more convincingly
demonstrates the superior deconfounding ability of the proposed DecTag.

###  6.4 Sensitivity to Sample Size of MC-Estimator

In RQ3, the sample size $N^{s}$ for the MC-based do-calculus approximator is a
very important hyperparameter for the proposed DecTag that controls the trade-
off between estimator variance and computational overhead. In Section 4.2.3,
we claim that a large batch size can reduce the variance similar to a large
$N^{s}$, based on previous literature [30]. However, with a large batch size,
it still remains unclear how does the sample size $N^{s}$ influences DecTag’s
performance. Therefore, in this section, we conduct experiments to investigate
whether the DecTag is robust to a small $N^{s}$ when the batch size is large.
Specifically, we keep the large batch size for both base models (i.e., 1,024
for NFM and 4,096 for LightGCN), vary $N_{s}\in\\{2,5,10,25,50,100\\}$, and
report the performance in Fig. 6. From Fig. 6 we can find that the performance
is indeed stable with varied $N^{s}$. Such results indicate that a large batch
size can reduce the variance and stabilize training equivalent to a large
sample size. In addition, setting a large batch size increases only slightly
the training complexity compared with DecRS, which more convincingly shows the
superiority of DecTag.

##  7 Conclusions

In this paper, we have proposed a deconfounded tag recommender system, i.e.,
DecTag, based on causal intervention. Specifically, we derive a Monte Carlo-
based backdoor adjustment strategy, where uploader-induced confounding bias
can be interpretably addressed by substituting a hypothetical random uploader
for the true uploader with personal bias. In addition, to evaluate the
deconfounding ability of different strategies, a large-scale tag
recommendation dataset, i.e., YT-8M-Causal, is established where an evaluation
strategy is introduced accordingly. Extensive experiments demonstrate that
DecTag is more robust to uploaders’ confounding bias than state-of-the-art
deconfounded tag recommendation strategies.

## References

  * Cha et al. [2007] Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn, and Sue Moon.  I tube, you tube, everybody tubes: analyzing the world’s largest user generated content video system.  In _Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement_ , pages 1–14, 2007.
  * Xie et al. [2020] Jiayi Xie, Yaochen Zhu, Zhibin Zhang, Jian Peng, Jing Yi, Yaosi Hu, Hongyi Liu, and Zhenzhong Chen.  A multimodal variational encoder-decoder framework for micro-video popularity prediction.  In _Proceedings of The Web Conference 2020_ , pages 2542–2548, 2020\.
  * Sigurbjörnsson and Van Zwol [2008] Börkur Sigurbjörnsson and Roelof Van Zwol.  Flickr tag recommendation based on collective knowledge.  In _Proceedings of the 17th International Conference on World Wide Web_ , pages 327–336, 2008.
  * Kim et al. [2011] Heung-Nam Kim, Abdulmajeed Alkhaldi, Abdulmotaleb El Saddik, and Geun-Sik Jo.  Collaborative user modeling with user-generated tags for social recommender systems.  _Expert Systems with Applications_ , 38(7):8488–8496, 2011.
  * Belém et al. [2017] Fabiano M Belém, Jussara M Almeida, and Marcos A Gonçalves.  A survey on tag recommendation methods.  _Journal of the Association for Information Science and Technology_ , 68(4):830–844, 2017.
  * Wu et al. [2016] Yong Wu, Yuan Yao, Feng Xu, Hanghang Tong, and Jian Lu.  Tag2word: Using tags to generate words for content based tag recommendation.  In _Proceedings of the 25th ACM International on Conference on Information and Knowledge Management_ , pages 2287–2292, 2016.
  * Tang et al. [2019] Shijie Tang, Yuan Yao, Suwei Zhang, Feng Xu, Tianxiao Gu, Hanghang Tong, Xiaohui Yan, and Jian Lu.  An integral tag recommendation model for textual content.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 5109–5116, 2019.
  * Maity et al. [2019] Suman Kalyan Maity, Abhishek Panigrahi, Sayan Ghosh, Arundhati Banerjee, Pawan Goyal, and Animesh Mukherjee.  Deeptagrec: A content-cum-user based tag recommendation framework for stack overflow.  In _European Conference on Information Retrieval_ , pages 125–131. Springer, 2019.
  * Pearl et al. [2016] Judea Pearl, Madelyn Glymour, and Nicholas P Jewell.  _Causal inference in statistics: A primer_.  John Wiley & Sons, 2016.
  * Zhu and Chen [2022a] Yaochen Zhu and Zhenzhong Chen.  Mutually-regularized dual collaborative variational auto-encoder for recommendation systems.  In _Proceedings of the ACM Web Conference 2022_ , pages 2379–2387, 2022a.
  * Bonner and Vasile [2018] Stephen Bonner and Flavian Vasile.  Causal embeddings for recommendation.  In _Proceedings of the 12th ACM Conference on Recommender Systems_ , pages 104–112, 2018.
  * Zhu and Chen [2022b] Yaochen Zhu and Zhenzhong Chen.  Variational bandwidth auto-encoder for hybrid recommender systems.  _IEEE Transactions on Knowledge and Data Engineering_ , 2022b.
  * Zhang et al. [2021] Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang.  Causal intervention for leveraging popularity bias in recommendation.  In _The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 11–20, 2021.
  * Ma et al. [2021] Jing Ma, Ruocheng Guo, Aidong Zhang, and Jundong Li.  Multi-cause effect estimation with disentangled confounder representation.  In _Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence_ , 2021.
  * Zhu et al. [2022] Yaochen Zhu, Jing Yi, Jiayi Xie, and Zhenzhong Chen.  Deep causal reasoning for recommendations.  _arXiv preprint arXiv:2201.02088_ , 2022.
  * Wang et al. [2021a] Wenjie Wang, Fuli Feng, Xiangnan He, Xiang Wang, and Tat-Seng Chua.  Deconfounded recommendation for alleviating bias amplification.  2021a.
  * Xu et al. [2021] Shuyuan Xu, Yingqiang Ge, Yunqi Li, Zuohui Fu, Xu Chen, and Yongfeng Zhang.  Causal collaborative filtering.  _arXiv preprint arXiv:2102.01868_ , 2021.
  * Zou et al. [2020] Hao Zou, Peng Cui, Bo Li, Zheyan Shen, Jianxin Ma, Hongxia Yang, and Yue He.  Counterfactual prediction for bundle treatment.  _Advances in Neural Information Processing Systems_ , 33, 2020.
  * Song et al. [2011] Yang Song, Lu Zhang, and C Lee Giles.  Automatic tag recommendation algorithms for social recommender systems.  _ACM Transactions on the Web (TWEB)_ , 5(1):1–31, 2011.
  * Marinho and Schmidt-Thieme [2008] Leandro Balby Marinho and Lars Schmidt-Thieme.  Collaborative tag recommendations.  In _Data Analysis, Machine Learning and Applications_ , pages 533–540. Springer, 2008.
  * Lu et al. [2009] Yu-Ta Lu, Shoou-I Yu, Tsung-Chieh Chang, and Jane Yung-jen Hsu.  A content-based method to enhance tag recommendation.  In _Twenty-first International Joint Conference on Artificial Intelligence_ , 2009.
  * Rendle and Schmidt-Thieme [2010] Steffen Rendle and Lars Schmidt-Thieme.  Pairwise interaction tensor factorization for personalized tag recommendation.  In _Proceedings of the third ACM International Conference on Web Search and Data Mining_ , pages 81–90, 2010.
  * Guan et al. [2009] Ziyu Guan, Jiajun Bu, Qiaozhu Mei, Chun Chen, and Can Wang.  Personalized tag recommendation using graph-based ranking on multi-type interrelated objects.  In _Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 540–547, 2009.
  * Zhao et al. [2021] Weibin Zhao, Lin Shang, Yonghong Yu, Li Zhang, Can Wang, and Jiajun Chen.  Personalized tag recommendation via denoising auto-encoder.  _World Wide Web_ , pages 1–20, 2021.
  * Wang et al. [2020] Yixin Wang, Dawen Liang, Laurent Charlin, and David M Blei.  Causal inference for recommender systems.  In _Fourteenth ACM Conference on Recommender Systems_ , pages 426–431, 2020.
  * Feng et al. [2021] Fuli Feng, Weiran Huang, Xiangnan He, Xin Xin, Qifan Wang, and Tat-Seng Chua.  Should graph convolution trust neighbors? a simple causal inference method.  In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 1208–1218, 2021\.
  * Schnabel et al. [2016] Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims.  Recommendations as treatments: Debiasing learning and evaluation.  In _International Conference on Machine Learning_ , pages 1670–1679. PMLR, 2016.
  * Lee et al. [2021] Jae-woong Lee, Seongmin Park, and Jongwuk Lee.  Dual unbiased recommender learning for implicit feedback.  In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 1647–1651, 2021\.
  * Rubin [1981] Donald B Rubin.  The bayesian bootstrap.  _The Annals of Statistics_ , pages 130–134, 1981.
  * Kingma and Welling [2013] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_ , 2013.
  * Louizos et al. [2017] Christos Louizos, Uri Shalit, Joris Mooij, David Sontag, Richard Zemel, and Max Welling.  Causal effect inference with deep latent-variable models.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , pages 6449–6459, 2017.
  * Cadene et al. [2019] Remi Cadene, Corentin Dancette, Matthieu Cord, Devi Parikh, et al.  Rubi: Reducing unimodal biases for visual question answering.  In _Advances in Neural Information Processing Systems_ , pages 841–852, 2019.
  * Niu et al. [2021] Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen.  Counterfactual vqa: A cause-effect look at language bias.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12700–12710, 2021.
  * Wu and Goodman [2018] Mike Wu and Noah Goodman.  Multimodal generative models for scalable weakly-supervised learning.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * He and Chua [2017] Xiangnan He and Tat-Seng Chua.  Neural factorization machines for sparse predictive analytics.  In _Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 355–364, 2017.
  * He et al. [2020] Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang.  Lightgcn: Simplifying and powering graph convolution network for recommendation.  In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 639–648, 2020.
  * Abu-El-Haija et al. [2016] Sami Abu-El-Haija, Nisarg Kothari, Joonseok Lee, Paul Natsev, George Toderici, Balakrishnan Varadarajan, and Sudheendra Vijayanarasimhan.  Youtube-8m: A large-scale video classification benchmark.  _arXiv preprint arXiv:1609.08675_ , 2016.
  * Gupta et al. [2021] Shantanu Gupta, Hao Wang, Zachary C Lipton, and Yuyang Wang.  Correcting exposure bias for link recommendation.  _Proceedings of International Conference on Machine Learning_ , 2021\.
  * Zheng et al. [2021] Yu Zheng, Chen Gao, Xiang Li, Xiangnan He, Yong Li, and Depeng Jin.  Disentangling user interest and conformity for recommendation with causal embedding.  In _Proceedings of the Web Conference 2021_ , pages 2980–2991, 2021\.
  * Kingma and Ba [2015] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In _Proceedings of International Conference on Learning Representations_ , 2015.
  * Grgic-Hlaca et al. [2016] Nina Grgic-Hlaca, Muhammad Bilal Zafar, Krishna P Gummadi, and Adrian Weller.  The case for process fairness in learning: Feature selection for fair decision making.  In _NIPS Symposium on Machine Learning and the Law_ , volume 1, page 2, 2016.
  * Morik et al. [2020] Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims.  Controlling fairness and bias in dynamic learning-to-rank.  In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 429–438, 2020.
  * Steck [2018] Harald Steck.  Calibrated recommendations.  In _Proceedings of the 12th ACM Conference on Recommender Systems_ , pages 154–162, 2018.
  * Saito et al. [2020] Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata.  Unbiased recommender learning from missing-not-at-random implicit feedback.  In _Proceedings of the 13th International Conference on Web Search and Data Mining_ , pages 501–509, 2020.
  * Wang et al. [2021b] Wenjie Wang, Fuli Feng, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua.  Clicks can be cheating: Counterfactual recommendation for mitigating clickbait issue.  In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 1288–1297, 2021b.
  * Ziegler et al. [2005] Cai-Nicolas Ziegler, Sean M McNee, Joseph A Konstan, and Georg Lausen.  Improving recommendation lists through topic diversification.  In _Proceedings of the 14th International Conference on World Wide Web_ , pages 22–32, 2005.