[1]Jie Yin

1]Discipline of Business Analytics, The University of Sydney, Sydney, NSW,
Australia

2]Department of Data Science & AI, Monash University, Melbourne, VIC,
Australia

# Combating Confirmation Bias: A Unified Pseudo-Labeling Framework for Entity
Alignment

Qijie Ding<EMAIL_ADDRESS><EMAIL_ADDRESS>Daokun Zhang
<EMAIL_ADDRESS>Junbin Gao<EMAIL_ADDRESS>[  [

###### Abstract

Entity alignment (EA) aims at identifying equivalent entity pairs across
different knowledge graphs (KGs) that refer to the same real-world identity.
It has been a compelling but challenging task that requires the integration of
heterogeneous information from different KGs to expand the knowledge coverage
and enhance inference abilities. To circumvent the shortage of prior alignment
seeds provided for training, recent EA models utilize pseudo-labeling
strategies to iteratively add unaligned entity pairs predicted with high
confidence to the training data for model retraining. However, the adverse
impact of confirmation bias has been largely overlooked during pseudo-
labeling, thus hindering entity alignment performance. To systematically
combat confirmation bias, we propose a Unified Pseudo-Labeling framework for
Entity Alignment (UPL-EA) that explicitly eliminates pseudo-labeling errors to
boost the accuracy of entity alignment. UPL-EA consists of two complementary
components: (1) The Optimal Transport (OT)-based pseudo-labeling uses discrete
OT modeling as an effective means to enable more accurate determination of
entity correspondences across two KGs and to mitigate the adverse impact of
erroneous matches. A simple but highly effective criterion is further devised
to derive pseudo-labeled entity pairs that satisfy one-to-one correspondences
at each iteration. (2) The cross-iteration pseudo-label calibration operates
across multiple consecutive iterations to further improve the pseudo-labeling
precision rate by reducing the local pseudo-label selection variability with a
theoretical guarantee. The two components are respectively designed to
eliminate Type I and Type II pseudo-labeling errors identified through our
analyse. The calibrated pseudo-labels are thereafter used to augment prior
alignment seeds to reinforce subsequent model training for alignment
inference. The effectiveness of UPL-EA in eliminating pseudo-labeling errors
is both theoretically supported and experimentally validated. The experimental
results show that our approach achieves competitive performance with limited
prior alignment seeds.

###### keywords:

Entity Alignment, Pseudo-labeling, Optimal Transport, Knowledge Graphs

##  1 Introduction

Knowledge Graphs (KGs) are large-scale structured knowledge bases that
represent real-world entities (or concepts) and their relationships as a
collection of factual triplets. Recent years have witnessed the release of
various open-source KGs (e.g., Freebase [2], YAGO [30] and Wikidata [39]) from
general to specific domains and their proliferation to empower many artificial
intelligence (AI) applications, such as recommender systems [11], question
answering [49] and information retrieval [21]. Nevertheless, it has become a
well-known fact that real-world KGs suffer from incompleteness arising from
their complex, semi-automatic construction process. This has led to an
increasing number of research efforts on knowledge graph completion, such as
TransE [3] and TransH [41], which aim to add missing facts to individual KGs.
Unfortunately, due to its limited coverage and incompleteness, a single KG
cannot fulfill the requirements for complex AI applications that build upon
heterogeneous knowledge sources. This necessitates the integration of
heterogeneous information from multiple individual KGs to enrich knowledge
representation. Entity alignment (EA) is a crucial task towards this
objective, which aims to establish the correspondence between equivalent
entity pairs across different KGs that refer to the same real-world identity.

Over the last decade, there has been a surge of research efforts dedicated to
entity alignment across KGs. Most mainstream entity alignment models are based
on knowledge graph embedding, which embeds different KGs into a common latent
space so that similarities between entities can be directly measured via their
embeddings for alignment purposes. To learn better KG embeddings, methods like
GCN-Align [42] leverage graph convolutional networks (GCNs) [12] to capture
structural and neighboring entity information for alignment inference. Along
this line of research, recent studies [44, 43, 54] utilize a highway strategy
[29] to alleviate over-smoothing during GCN propagation, or jointly learn
entity and relation embeddings for improving the accuracy of entity alignment.
These models, however, require an abundant amount of pre-aligned entity pairs
(known as prior alignment seeds) during training, which are labor-intensive
and costly to acquire in real-world KGs. To tackle the shortage of prior
alignment seeds provided for training, recently proposed models, such as
BootEA [33], IPTransE [53], MRAEA [17], and RNM [54], adopt a bootstrapping
strategy that iteratively selects unaligned entity pairs predicted with high
confidence as pseudo-labels and adds them to prior alignment seeds for either
model re-training or posterior embedding distance rectification. The
bootstrapping strategy originates from the field of statistics and is also
referred to as pseudo-labeling – a predominant learning paradigm proposed to
tackle the label scarcity problem in semi-supervised learning.

In general semi-supervised learning, pseudo-labeling approaches inherently
suffer from confirmation bias [1, 35]. The bias stems from using incorrectly
predicted labels generated on unlabeled data for subsequent training, thereby
misleadingly increasing confidence in incorrect predictions and generating a
biased model with degraded performance. Unfortunately, there is a lack of
understanding of the fundamental factors that give rise to confirmation bias
for pseudo-labeling-based entity alignment. Our analysis (see Section 2.2)
advocates that the confirmation bias is exacerbated during pseudo-labeling for
entity alignment. Due to the lack of sufficient prior alignment seeds at the
early stages of training, the existing models are inclined to learn
uninformative entity embeddings and make incorrect predictions on unaligned
entity pairs. As a consequence, the pseudo-labels generated based on the
model’s unreliable high-confidence predictions are error-prone. The resultant
errors can be characterized into two types, as depicted in Fig. 1. Type I
pseudo-labeling errors refer to conflicted misalignments, where a single
entity in one KG is simultaneously aligned with multiple entities in another
KG with erroneous matches. Type II pseudo-labeling errors refer to one-to-one
misalignments, where an entity in one KG is incorrectly matched with an entity
in another KG. These pseudo-labeling errors, if not properly mitigated, would
propagate into subsequent model training, thereby jeopardizing the efficacy of
pseudo-labeling-based entity alignment. However, current pseudo-labeling-based
EA models have made only limited attempts to alleviate alignment conflicts
using simple heuristics [54, 17] or imposing constraints to enforce hard
alignments [33, 9], while the confirmation bias has been left under-explored.

Figure 1: The composition of pseudo-labels generated based on the model’s high
confidence predictions. The resultant pseudo-labeling errors comprise Type I
errors (conflicted misalignments) and Type II errors (one-to-one
misalignments).

To fill in the research gap, in this work, we propose a Unified Pseudo-
Labeling framework for Entity Alignment (UPL-EA) to systematically combat
confirmation bias. The scope of our work focuses primarily on a semi-
supervised learning paradigm, where we aim to eliminate Type I and Type II
pseudo-labeling errors to boost the precision of entity alignment. Towards
this goal, UPL-EA comprises two complementary components – within-iteration
Optimal Transport (OT)-based pseudo-labeling and cross-iteration pseudo-label
calibration. Specifically, OT-based pseudo-labeling uses discrete OT modeling
as an effective means to enable more accurate determination of entity
correspondences across KGs and to mitigate the adverse impact of erroneous
matches. The discrete OT models the alignment as a probabilistic matching
process between the entity sets in two KGs with the minimal overall transport
cost. The transport cost is calculated according to the rectified distance
between entity embeddings obtained through graph convolution augmented with
global-level semantics. Based on the estimated OT solution, a simple but
highly effective selection criterion is further devised to infer pseudo-
labeled entity pairs that satisfy one-to-one correspondence, thus eliminating
conflicted misalignments (Type I pseudo-labeling errors) at each iteration.
The pseudo-label calibration operates across multiple consecutive iterations
to reduce the local pseudo-label selection variability with theoretical
guarantee, thereby eliminating one-to-one misalignments (Type II pseudo-
labeling errors). The calibrated pseudo-labels are then used to augment prior
alignment seeds to reinforce subsequent model training for alignment
inference.

The contribution of this paper can be summarized as follows:

  * •

To our best knowledge, we are the first to investigate the confirmation bias
problem for pseudo-labeling-based entity alignment. We conceptually and
empirically identify two types of pseudo-labeling errors that give rise to
confirmation bias.

  * •

Motivated by our analysis, we propose a unified pseudo-labeling framework
(UPL-EA) that systematically alleviates the problem of confirmation bias for
pseudo-labeling-based entity alignment. UPL-EA is carefully designed to
explicitly eliminate Type I and Type II pseudo-labeling errors, leading to
significant performance gains.

  * •

Extensive experiments on benchmark KG datasets demonstrate that UPL-EA is able
to significantly eliminate pseudo-labeling errors and yield competitive
performance with respect to limited amounts of prior alignment seeds,
outperforming state-of-the-art supervised and semi-supervised baselines.

The remainder of this paper is organized as follows. Section 2 provides a
problem formulation of pseudo-labeling-based entity alignment and presents an
empirical analysis of confirmation bias that motivates this work. Section 3
presents the proposed framework, followed by extensive experimental evaluation
reported in Section 4. Related works are discussed in Section 5, and we
conclude the paper in Section 6.

##  2 Preliminaries

In this section, we first provide a problem formulation of pseudo-labeling-
based entity alignment. Then, we perform a thorough analysis of confirmation
bias during pseudo-labeling, which motivates the design of our proposed
framework.

###  2.1 Problem Formulation

A knowledge graph (KG) can be defined as
$\mathcal{G}=\\{\mathcal{E},\mathcal{R},\mathcal{T}\\}$ with the entity set
$\mathcal{E}$, relation set $\mathcal{R}$, and triplet set $\mathcal{T}$. We
use $e\in\mathcal{E}$, $r\in\mathcal{R}$, $(e_{i},r,e_{j})\in\mathcal{T}$ to
represent an entity, a relation, and a triplet, respectively. Each entity $e$
is characterized by an attribute vector $\bm{x}_{e}$, which can be obtained
from entity names with semantic meanings or entity textual descriptions.
Formally, two heterogeneous KGs are given for the task of entity alignment,
i.e., $\mathcal{G}_{1}=\\{\mathcal{E}_{1},\mathcal{R}_{1},\mathcal{T}_{1}\\}$
and $\mathcal{G}_{2}=\\{\mathcal{E}_{2},\mathcal{R}_{2},\mathcal{T}_{2}\\}$.
An entity $e_{i}\in\mathcal{E}_{1}$ in $\mathcal{G}_{1}$ is likely to
correspond to the same concept with another entity $e_{j}\in\mathcal{E}_{2}$
in $\mathcal{G}_{2}$, denoted as $e_{i}\Leftrightarrow e_{j}$, and vice versa.

To provide supervision for entity alignment, a small number of pre-aligned
entity pairs between $\mathcal{G}_{1}$ and $\mathcal{G}_{2}$ are sometimes
provided as prior alignment seeds in the form of
$\mathcal{L}^{0}_{e}=\\{(e_{i},e_{j})|\;e_{i}\in\mathcal{E}_{1},e_{j}\in\mathcal{E}_{2},e_{i}\Leftrightarrow
e_{j}\\}$. Apart from prior alignment seeds, there are two sets of unaligned
entities $\mathcal{E}^{\prime}_{1}\subseteq\mathcal{E}_{1}$ and
$\mathcal{E}^{\prime}_{2}\subseteq\mathcal{E}_{2}$ respectively from two KGs.

Given two KGs, $\mathcal{G}_{1}$ and $\mathcal{G}_{2}$, the task of pseudo-
labeling-based entity alignment iterates over the following two components:
(1) Training an entity alignment (EA) model that learns entity embeddings
$f_{\theta}(\mathcal{L}_{e}):\bm{h}_{e}\rightarrow\mathbb{R}^{d}$ using the
available alignment seeds $\mathcal{L}_{e}$, where $\mathcal{L}_{e}$ is set to
the prior alignment seeds $\mathcal{L}_{e}^{0}$ at the first iteration,
$e\in\mathcal{E}_{1}\cup\mathcal{E}_{2}$, and $d$ is the embedding dimension;
(2) Designing a pseudo-label selection component that selects unaligned entity
pairs
$g(\bm{h}_{e_{i}},\bm{h}_{e_{j}}):\mathcal{E}^{\prime}_{1}\times\mathcal{E}^{\prime}_{2}\rightarrow\\{0,1\\}$
using learned entity embeddings $\bm{h}_{e}$, then augments alignment seeds
with pseudo-labels, i.e.,
$\mathcal{L}_{e}\leftarrow\mathcal{L}_{e}\cup\\{(e_{i},e_{j})|\;e_{i}\in\mathcal{E}^{\prime}_{1},e_{j}\in\mathcal{E}^{\prime}_{2},g(\bm{h}_{e_{i}},\bm{h}_{e_{j}})=1\\}$.

To achieve accurate entity alignment, we need to ensure that: (1) the EA model
$f_{\theta}$ has the capacity to learn informative, high-quality entity
embeddings; (2) the pseudo-label selection component $g$ can select precise
pseudo-labeled entity pairs for augmenting alignment seeds, providing high-
quality supervision for boosting the EA model. The two components reinforce
each other alternately to make accurate inferences for entity alignment.

###  2.2 Analysis of Confirmation Bias

To investigate the impact of confirmation bias on pseudo-labeling-based entity
alignment, we perform an error analysis of a naive pseudo-labeling strategy,
where unaligned entity pairs with an embedding distance smaller than a pre-
specified threshold are selected as pseudo-labels. To gain insights into the
underlying causes of confirmation bias, we explicitly analyze the variations
in Type I and Type II pseudo-labeling errors throughout the training process,
when the naive pseudo-labeling strategy is used. This analysis is conducted on
a widely used cross-lingual benchmark dataset DBP15KZH_EN [32] (Details of the
dataset are presented in Section 4.1). We follow the conventional 30%-70%
ratio to randomly split the training and test data on the 15,000 ground truth
entity alignment pairs provided. The proportions of two types of errors and
entity alignment accuracy are calculated at different iterations on the test
data. As shown in Fig. 2(a), Type I and Type II pseudo-labeling errors could
propagate and increase proportionally via the naive pseudo-labeling strategy
as training proceeds. The accumulated errors give rise to confirmation bias,
thereby severely hindering the capability of pseudo-labeling for performance
improvements.

(a)

(b)

Figure 2: Error analysis of confirmation bias. (a) Proportion of pseudo-labels
selected by a naive pseudo-labeling strategy over training iterations. (b)
Entity alignment accuracy of the naive pseudo-labeling strategy and the
proposed unified pseudo-labeling strategy over training iterations.

Our analysis affirms that the confirmation bias essentially stems from Type I
and Type II pseudo-labeling errors. These errors, if not adequately addressed,
would propagate into subsequent model training, thus diminishing the
effectiveness of pseudo-labeling for entity alignment. Our work is thus
motivated to explicitly eliminate Type I and Type II pseudo-labeling errors,
leading to significant performance improvements for entity alignment
inference. As shown in Fig. 2(b), our proposed method outperforms the naive
pseudo-labeling strategy, yielding substantial performance gains.

##  3 The Proposed UPL-EA Framework

With the insights derived from our analysis in Section 2.2, the proposed UPL-
EA framework is designed to systematically address the confirmation bias issue
for pseudo-labeling based entity alignment. At the core of UPL-EA are two key
complementary modules that greatly enhance the entity alignment (EA) model:

  * •

Module 1 (Within-Iteration OT-Based Pseudo-Labeling) utilizes Optimal
Transport (OT) modeling to more precisely determine the correspondences
between entities across KGs so as to eliminate conflicted misalignments at
each iteration. (Addressing Type I pseudo-labeling errors)

  * •

Module 2 (Cross-Iteration Pseudo-Label Calibration) further calibrates pseudo-
labels generated from Module 1 across iterations to reduce local variability
of pseudo-label selection so as to minimize one-to-one misalignments across
iterations. (Addressing Type II pseudo-labeling errors)

Fig. 3 illustrates an overview of the proposed UPL-EA framework. At each
iteration, the EA model learns informative entity embeddings via a global-
local aggregation scheme based on the available alignment seeds. Then, Module
1 takes entity embeddings as input to pseudo-label conflict-free entity
alignment pairs (eliminating Type I pseudo-labeling errors), which are further
used by the EA model in the subsequent iteration. After every $m$ iterations,
pseudo-labels across iterations are passed on to Module 2 for further
calibration, preventing one-to-one misalignments (Type II pseudo-labeling
errors) from accumulating in subsequent model training. The calibrated pseudo-
labels are then used to augment the prior alignment seeds. The augmented
alignment seeds are used by the next $m$ iterations to perform pseudo-
labeling-based entity alignment. The EA model and the pseudo-label selection
component empowered by Module 1 and Module 2 mutually reinforce each other to
learn informative entity embeddings. Lastly, the learned entity embeddings are
used for entity alignment inference.

Figure 3: The overall framework of the proposed UPL-EA model.

###  3.1 Entity Alignment Model

The entity alignment (EA) model aims to learn informative entity embeddings
and perform model training for entity alignment inference.

####  3.1.1 Global-Local Aggregation for Entity Embedding Learning

To enable the EA model to learn informative entity embeddings, it is essential
to capture the relational information conveyed in KGs. Towards this end, we
employ a global-local aggregation scheme [9] to perform two levels of
neighborhood aggregation for each entity in $\mathcal{G}_{1}$ and
$\mathcal{G}_{2}$.

Global-Level Relation Aggregation. To capture the global statistics of a given
relation $r_{i}$, we construct its feature vector $\bm{x}_{r_{i}}$ by taking
into account all triplets associated with $r_{i}$ in two KGs. To be specific,
we compute the feature vector $\bm{x}_{r_{i}}$ for each relation
$r_{i}\in\mathcal{R}_{1}\cup\mathcal{R}_{2}$, by taking an average of the
concatenated feature vectors of its associated head and tail entities:

$\bm{x}_{r_{i}}=\frac{1}{|\\{(e_{h},r_{i},e_{t})\in\mathcal{T}_{1}\cup\mathcal{T}_{2}\\}|}\sum_{(e_{h},r_{i},e_{t})\in\mathcal{T}_{1}\cup\mathcal{T}_{2}}[\bm{x}_{e_{h}}\|\bm{x}_{e_{t}}],$
(1)

where $[\cdot\|\cdot]$ denotes the concatenation operation,
$\\{(e_{h},r_{i},e_{t})\in\mathcal{T}_{1}\cup\mathcal{T}_{2}\\}$ is the set of
all triplets associated with relation $r_{i}$. $\bm{x}_{e_{h}}$ and
$\bm{x}_{e_{t}}\in\mathbb{R}^{d}$ are the feature vectors of entity $e_{h}$
and $e_{t}$, respectively. As such, the obtained relation feature vector
$\bm{x}_{r_{i}}\in\mathbb{R}^{2d}$ can capture the intrinsic relational
semantics of relation $r_{i}$. Then, for each entity
$e_{i}\in\mathcal{E}_{1}\cup\mathcal{E}_{2}$, we construct its averaged
neighboring relation feature as follows:

$\bm{x}_{e_{i}\\_rels}=\frac{1}{|\mathcal{N}_{r}(e_{i})|}\sum_{r_{j}\in\mathcal{N}_{r}(e_{i})}\mathbb{I}_{e_{i}}(r_{j})\cdot\bm{x}_{r_{j}},$
(2)

where $\mathcal{N}_{r}(e_{i})$ is the set of one-hop neighboring relations of
entity $e_{i}$. $\mathbb{I}_{e_{i}}(r_{j})$ indicates the direction of
relation $r_{j}$ with respect to $e_{i}$, where $-1$ indicates the successor
of $e_{i}$ and $+1$ indicates the predecessor of $e_{i}$. The incorporation of
directional information enables to capture richer relational information in
the neighborhood, benefiting the learning of entity embeddings.

To perform global-level relation aggregation, we concatenate each entity’s
averaged neighboring relation feature
$\bm{x}_{e_{i}\\_rels}\in\mathbb{R}^{2d}$ with its original feature
$\bm{x}_{e_{i}}\in\mathbb{R}^{d}$, followed by a non-linear transformation to
capture the interplay between different dimensions:

$\tilde{\bm{h}}_{e_{i}}^{(1)}=\mathrm{ReLU}(W_{1}[\bm{x}_{e_{i}}\|\bm{x}_{e_{i}\\_rels}]+b_{1}),$
(3)

where $\mathrm{ReLU}(\cdot)$ is the ReLU activation function,
$W_{1}\in\mathbb{R}^{d\times 3d}$ and $b_{1}\in\mathbb{R}^{d}$ are the weight
matrix and the bias vector, respectively. To avoid over-smoothing and deliver
a more effective representation scheme, we augment the embedding vector
$\tilde{\bm{h}}_{e_{i}}^{(1)}$ with the original entity feature vector
$\bm{x}_{e_{i}}$:

$\bm{h}_{e_{i}}^{(1)}=\tilde{\bm{h}}_{e_{i}}^{(1)}+\bm{x}_{e_{i}}.$ (4)

Local-Level Entity Aggregation. After obtaining entity embeddings via relation
aggregation at the global level, we further perform local-level entity
aggregation to capture neighboring entity structures.

For this purpose, we take advantage of a two-layer GCN [12] along with a
highway gate strategy [29] to avoid over-smoothing. Formally, we first stack
relation aggregated entity embeddings $\bm{h}_{e_{i}}^{(1)}$ for each entity
$e_{i}\in\mathcal{E}_{1}\cup\mathcal{E}_{2}$ into an embedding matrix
$H^{(1)}_{e}\in\mathbb{R}^{(|\mathcal{E}_{1}|+|\mathcal{E}_{2}|)\times d}$,
where each row indicates the embedding vector of the corresponding entity.
Then, the entity embedding matrix $H^{(1)}_{e}$ is updated as follows from
$l=1$:

$\left\\{\begin{aligned}
&\tilde{H}_{e}^{(l+1)}=\mathrm{ReLU}(\tilde{D}^{-\frac{1}{2}}\tilde{A}\tilde{D}^{-\frac{1}{2}}H_{e}^{(l)}W_{l+1}),\\\
&H_{e}^{(l+1)}=T(H_{e}^{(l)})\odot\tilde{H}^{(l+1)}_{e}+(1-T(H_{e}^{(l)}))\odot
H^{(l)}_{e}.\end{aligned}\right.$ (5)

Above, $\tilde{A}=A+I_{|\mathcal{E}_{1}|+|\mathcal{E}_{2}|}$ denotes the
undirected adjacency matrix of the combined graph
$\mathcal{G}_{1}\cup\mathcal{G}_{2}$ with self-connections that are
represented by the
$(|\mathcal{E}_{1}|+|\mathcal{E}_{2}|)\times(|\mathcal{E}_{1}|+|\mathcal{E}_{2}|)$
identity matrix $I_{|\mathcal{E}_{1}|+|\mathcal{E}_{2}|}$.
$\tilde{D}_{ii}=\sum_{j}\tilde{A}_{ij}$ is the degree matrix,
$W_{l+1}\in\mathbb{R}^{d\times d}$ is the weight matrix at layer $l$, and
$\odot$ is the Hadamard product (or element-wise multiplication). In addition,
$T(H_{e}^{(l)})=\sigma(H_{e}^{(l)}W_{T}^{(l)}+b_{T}^{(l)})\in\mathbb{R}^{(|\mathcal{E}_{1}|+|\mathcal{E}_{2}|)\times
d}$ is the transformation gate obtained from $H_{e}^{(l)}$, where
$\sigma(\cdot)$ is the Sigmoid activation function,
$W_{T}^{(l)}\in\mathbb{R}^{d\times d}$ and $b_{T}^{(l)}\in\mathbb{R}^{d}$ are
the gate-specific weight matrix and bias vector, respectively. The
transformation gate helps filter out the over-smoothed feature dimensions.

Finally, the entity embedding matrix is obtained as $H_{e}^{(3)}$.
Accordingly, the final embedding for entity $e_{i}$ is
$\bm{h}_{e_{i}}=\bm{h}^{(3)}_{e_{i}}\in\mathbb{R}^{d}$, where
$\bm{h}^{(3)}_{e_{i}}\in\mathbb{R}^{d}$ is the transpose of the row vector of
$H_{e}^{(3)}$ indexed by entity $e_{i}$.

####  3.1.2 Model Training for Entity Alignment

Given a set of alignment seeds $\mathcal{L}_{e}$ together the constructed
entity embeddings, the loss function for entity alignment can be defined as:

$L=\sum_{(e_{i},e_{j})\in\mathcal{L}_{e}}\sum_{(e_{i^{\prime}},e_{j^{\prime}})\in\mathcal{L}^{\prime}_{e}}\hat{R}(e_{i},e_{j})\cdot\max\bigl{(}0,d(e_{i},e_{j})-d(e_{i^{\prime}},e_{j^{\prime}})+\gamma\bigl{)},$
(6)

where $\mathcal{L}_{e}^{\prime}$ is the set of sampled negative alignment
pairs that are not included in $\mathcal{L}_{e}$. $\gamma$ is a hyper-
parameter that determines the margin to separate positive pairs from negative
pairs. $d(e_{i},e_{j})$ indicates the embedding distance between entity pair
$(e_{i},e_{j})$ across two KGs, defined as:

$d(e_{i},e_{j})=\|\bm{h}_{e_{i}}-\bm{h}_{e_{j}}\|_{1}.$ (7)

Above, $\hat{R}(e_{i},e_{j})$ indicates a weighting factor that measures
distinct contribution of each alignment pair $(e_{i},e_{j})$ to the overall
loss. For any pair from the prior alignment seeds, $\hat{R}(e_{i},e_{j})=1$,
otherwise $\hat{R}(e_{i},e_{j})=1/(1+e^{\tilde{d}(e_{i},e_{j})})\in(0,1)$,
where $\tilde{d}(e_{i},e_{j})$ denotes the rectified embedding distance
defined later using Eq.(8) in Section 3.2.

For model training, we adopt an adaptive negative sampling strategy to obtain
a negative alignment set $\mathcal{L}^{\prime}_{e}$. Specifically, for each
positive pair $(e_{i},e_{j})$ in the alignment set $\mathcal{L}_{e}$, we
select $K$ nearest entities of $e_{i}$ (or $e_{j}$) measured by the embedding
distance in Eq. (7) to replace $e_{j}$ (or $e_{i}$) and form $K$ negative
counterparts $(e_{i},e_{j^{\prime}})$ (or $(e_{i^{\prime}},e_{j})$). This
adaptive strategy helps produce “hard” negative alignment pairs and push their
associated entities to be apart from each other in the embedding space.

After minimizing the loss on the set of alignment seeds, the learned entity
embeddings $\bm{h}_{e}$ can be used to infer unaligned entity pairs as pseudo-
labels, which are used to augment alignment seeds for subsequent training.
However, as we have discussed earlier in Section 2.2, using a naive pseudo-
label selection strategy would inevitably result in a large portion of
erroneous pseudo-labels in the form of Type I and Type II pseudo-labeling
errors, leading to confirmation bias.

Below, we detail the two key modules that are explicitly proposed to mitigate
Type I and Type II pseudo-labeling errors, respectively.

###  3.2 Within-Iteration OT-Based Pseudo-Labeling

To effectively mitigate Type I pseudo-labeling errors, we propose to use
Optimal Transport (OT) as an effective means to more precisely determine one-
to-one correspondence between entities across KGs. This ensures that correct
entity alignment pairs can be sufficiently pseudo-labeled at each iteration to
boost entity alignment performance significantly.

The within-iteration OT-based pseudo-labeling module uses the distances
between the learned entity embeddings to define the transport cost for OT-
based modeling. However, directly employing the original embedding distances
would be error-prone, especially at early training stages where the learned
entity embeddings are uninformative. Thus, we resort to rectifying the
embedding distance defined in Eq. (7) using relational neighborhood matching
[54].

The principle of distance rectification is to leverage the relational context
within local neighborhoods to aid in determining the extent to which two
entities should be aligned. Intuitively, if two entities
$e_{i}\in\mathcal{E}_{1}$ and $e_{j}\in\mathcal{E}_{2}$ share more aligned
neighboring relations/entities, the distance between their embeddings should
be smaller, and thus they are more likely to be aligned with each other. As
such, the embedding distance used for pseudo-labeling is rectified as

$\tilde{d}(e_{i},e_{j})=d(e_{i},e_{j})-\lambda s(e_{i},e_{j}),$ (8)

where $\lambda$ is a trade-off parameter, $s(e_{i},e_{j})$ is a score function
indicating the degree to which relational contexts of two entities match. It
is calculated by determining whether neighboring entities/relations between
$(e_{i},e_{j})$ are aligned or not:

$\displaystyle s(e_{i},e_{j})$ (9) $\displaystyle=$
$\displaystyle\Bigl{(}\sum_{\mathcal{M}_{e_{i},e_{j}}}\frac{1}{|\\{e|\;(e,r,e_{k})\in\mathcal{T}_{1}\\}|\cdot|\\{e|\;(e,r^{\prime},e_{l})\in\mathcal{T}_{2}\\}|}\Bigl{)}\bigg{/}\Bigl{(}|\mathcal{N}_{e}(e_{i})|+|\mathcal{N}_{e}({e}_{j})|\Bigl{)},$

where $\mathcal{M}_{e_{i},e_{j}}$ is the set of matched neighboring relation-
entity tuples:

$\displaystyle\mathcal{M}_{e_{i},e_{j}}=\\{(e_{k},e_{l},r,r^{\prime})|$
$\displaystyle
e_{k}\in\mathcal{N}_{e}(e_{i}),e_{l}\in\mathcal{N}_{e}(e_{j}),(e_{i},r,e_{k})\in\mathcal{T}_{1},(e_{j},r^{\prime},e_{l})\in\mathcal{T}_{2},$
(10)
$\displaystyle(e_{k},e_{l})\in\mathcal{L}_{e},(r,r^{\prime})\in\mathcal{L}_{r}\\},$

$\mathcal{N}_{e}(e)$ is the set of neighboring entities of entity $e$.
$\mathcal{L}_{r}$ is the set of aligned relations obtained via pseudo-labeling
based on the embedding distance between relations.

Moreover, due to the smoothing effect of feature aggregation, entities in the
neighborhood tend to have indistinguishable representations. As a result,
simply using a pre-specified threshold to pseudo-label entity pairs would
inevitably result in conflicted misalignments. To circumvent this issue, we
propose an OT-based strategy to discover a globally optimal alignment
configuration with minimal overall inconsistency.

To warrant one-to-one alignments, we model the alignment between two unaligned
sets $\mathcal{E}^{\prime}_{1}$ and $\mathcal{E}^{\prime}_{2}$ as an OT
process, i.e., transporting each entity $e_{i}\in\mathcal{E}^{\prime}_{1}$ to
a unique entity $e_{j}\in\mathcal{E}^{\prime}_{2}$, with minimal overall
transport cost. Naturally, the rectified distance can be used to define the
transport cost across two KGs:

$C_{e_{i},e_{j}}=\tilde{d}(e_{i},e_{j}),\;e_{i}\in\mathcal{E}^{\prime}_{1},e_{j}\in\mathcal{E}^{\prime}_{2},$
(11)

where
$C\in\mathbb{R}^{|\mathcal{E}^{\prime}_{1}|\times|\mathcal{E}^{\prime}_{2}|}$
is the transport cost matrix. Without loss of generality, we assume
$|\mathcal{E}^{\prime}_{1}|<|\mathcal{E}^{\prime}_{2}|$. The transport plan is
a mapping function $T:e_{i}\rightarrow T(e_{j})$, where
$e_{i}\in\mathcal{E}^{\prime}_{1}$, $T(e_{i})\in\mathcal{E}^{\prime}_{2}$. The
objective is thus to find the optimal transport plan $T^{*}$ that minimizes
the overall transport cost:

$T^{*}=\mathop{\arg\min}\limits_{T}\sum_{e_{i}\in\mathcal{E}^{\prime}_{1}}C_{e_{i},T(e_{i})}.$
(12)

Eq. (12) models a hard assignment optimization problem that is not scaled up.
To enable more efficient optimization and to allow for a more flexible
alignment configuration, we reformulate this objective as a discrete OT
problem, where the optimal transport plan is considered as a coupling matrix
$P^{*}\in\mathbb{R}_{+}^{|\mathcal{E}^{\prime}_{1}|\times|\mathcal{E}^{\prime}_{2}|}$
between two discrete distributions. We denote $\bm{\mu}$ and $\bm{\nu}$ as two
discrete probability distributions over all entities
$\\{e_{i}|e_{i}\in\mathcal{E}^{\prime}_{1}\\}$ and
$\\{e_{j}|e_{j}\in\mathcal{E}^{\prime}_{2}\\}$, respectively. Without any
entity alignment preference, the two discrete distributions $\bm{\mu}$ and
$\bm{\nu}$ are assumed to follow a uniform distribution such that
$\bm{\mu}=\frac{1}{|\mathcal{E}^{\prime}_{1}|}\sum_{e_{i}\in\mathcal{E}^{\prime}_{1}}\delta_{e_{i}}$
and
$\bm{\nu}=\frac{1}{|\mathcal{E}^{\prime}_{2}|}\sum_{e_{j}\in\mathcal{E}^{\prime}_{2}}\delta_{e_{j}}$,
where $\delta_{e_{i}}$ and $\delta_{e_{j}}$ are the Dirac function centered on
$e_{i}$ and $e_{j}$, respectively. Both $\bm{\mu}$ and $\bm{\nu}$ are bounded
to sum up to one:
$\sum_{e_{i}\in\mathcal{E}^{\prime}_{1}}\bm{\mu}(e_{i})=\sum_{e_{i}\in\mathcal{E}^{\prime}_{1}}\frac{1}{|\mathcal{E}^{\prime}_{1}|}=1$
and
$\sum_{e_{j}\in\mathcal{E}^{\prime}_{2}}\bm{\mu}(e_{j})=\sum_{e_{j}\in\mathcal{E}^{\prime}_{2}}\frac{1}{|\mathcal{E}^{\prime}_{2}|}=1$.
Accordingly, the OT objective is formulated as finding the optimal coupling
matrix $P^{*}$ between $\bm{\mu}$ and $\bm{\nu}$:

$\displaystyle P^{*}=$
$\displaystyle\mathop{\arg\min}\limits_{P\in\Pi(\bm{\mu},\bm{\nu})}\sum_{e_{i}\in\mathcal{E}^{\prime}_{1}}\sum_{e_{j}\in\mathcal{E}^{\prime}_{2}}P_{e_{i},e_{j}}\cdot
C_{e_{i},e_{j}},$ (13) subject to:
$\displaystyle\sum_{e_{j}\in\mathcal{E}^{\prime}_{2}}P_{e_{i},e_{j}}=\bm{\mu}(e_{i})=\frac{1}{|\mathcal{E}^{\prime}_{1}|},$
$\displaystyle\sum_{e_{i}\in\mathcal{E}^{\prime}_{1}}P_{e_{i},e_{j}}=\bm{\nu}(e_{j})=\frac{1}{|\mathcal{E}^{\prime}_{2}|},$
$\displaystyle P_{e_{i},e_{j}}\geq 0,\forall
e_{i}\in\mathcal{E}^{\prime}_{1},\forall e_{j}\in\mathcal{E}^{\prime}_{2},$

where
$\Pi(\bm{\mu},\bm{\nu})=\\{P\in\mathbb{R}_{+}^{|\mathcal{E}^{\prime}_{1}|\times|\mathcal{E}^{\prime}_{2}|}|\;P\mathbf{1}_{|\mathcal{E}^{\prime}_{2}|}=\bm{\mu},\;P^{\top}\mathbf{1}_{|\mathcal{E}^{\prime}_{1}|}=\bm{\nu}\\}$
is the set of all joint probability distributions with marginal probabilities
$\bm{\mu}$ and $\bm{\nu}$, $\mathbf{1}_{n}$ denotes an $n$-dimensional vector
of ones. $P$ is a coupling matrix signifying probabilistic alignments between
two unaligned entity sets $\mathcal{E}^{\prime}_{1}$ and
$\mathcal{E}^{\prime}_{2}$. Therefore, $P_{e_{i},e_{j}}$ indicates the amount
of probability mass transported from $\bm{\mu}(e_{i})$ to $\bm{\nu}(e_{j})$. A
larger value of $P_{e_{i},e_{j}}$ indicates a higher likelihood of $e_{i}$ and
$e_{j}$ being aligned to each other.

To solve this discrete OT problem, some exact algorithms have been proposed,
such as interior point methods [40] and network simplex [20]. The exact
algorithms guarantee finding a closed-form optimal transport plan but with
considerably high computational cost, making them intractable to work for
iterative pseudo-labeling. Therefore, instead of Eq. (13), we propose to use
the entropy regularized OT problem, as defined in Eq. (14) below, which can be
solved by the efficient Sinkhorn algorithm [7].

$\displaystyle
P^{*}=\mathop{\arg\min}\limits_{P\in\Pi(\bm{\mu},\bm{\nu})}\sum_{e_{i}\in\mathcal{E}^{\prime}_{1}}\sum_{e_{j}\in\mathcal{E}^{\prime}_{2}}P_{e_{i},e_{j}}\cdot
C_{e_{i},e_{j}}+\beta\sum_{e_{i}\in\mathcal{E}^{\prime}_{1}}\sum_{e_{j}\in\mathcal{E}^{\prime}_{2}}P_{e_{i},e_{j}}\text{log}P_{e_{i},e_{j}},$
(14)

where $\beta$ is a hyper-parameter that controls the extent of regularization.
Solving the above entropy regularized OT problem can be easily implemented
using popular deep-learning frameworks like PyTorch and TensorFlow.

Once the optimal coupling matrix $P^{*}$ is obtained, entity alignments can be
inferred accordingly. Since one-to-one correspondences are crucial for
eliminating conflicted misalignments (Type I pseudo-labeling errors), we
propose a simple yet highly effective decision threshold to select entity
pairs as pseudo-labels such that:

$\hat{\mathcal{L}}_{e}=\\{(e_{i},e_{j})|\;P^{*}_{e_{i},e_{j}}>\frac{1}{2\cdot\text{min}(|\mathcal{E}^{\prime}_{1}|,|\mathcal{E}^{\prime}_{2}|)},e_{i}\in\mathcal{E}^{\prime}_{1},e_{j}\in\mathcal{E}^{\prime}_{2}\\}.$
(15)

This criterion ensures the pseudo-labels satisfy one-to-one correspondence
with theoretical guarantee.

######  Theorem 1.

Any pseudo-labeled entity pair
$(e_{i},e_{j}),e_{i}\in\mathcal{E}^{\prime}_{1},e_{j}\in\mathcal{E}^{\prime}_{2}$
that satisfies the condition
$P^{*}_{e_{i},e_{j}}>\frac{1}{2\cdot\text{min}(|\mathcal{E}^{\prime}_{1}|,|\mathcal{E}^{\prime}_{2}|)}$
warrants one-to-one correspondence such that no conflicted pairs
$\\{(e_{i},e_{k})|\;e_{k}\in\mathcal{E}^{\prime}_{2}\setminus\\{e_{j}\\}\\}$
and
$\\{(e_{l},e_{j})|\;e_{l}\in\mathcal{E}^{\prime}_{1}\setminus\\{e_{i}\\}\\}$
associated with $e_{i}$ and $e_{j}$, respectively, are selected as pseudo-
labels.

###### Proof.

Given a coupling matrix
$P^{*}\in\mathbb{R}_{+}^{|\mathcal{E}^{\prime}_{1}|\times|\mathcal{E}^{\prime}_{2}|}$
with the constraints of
$\sum_{e_{j}\in\mathcal{E}^{\prime}_{2}}P^{*}_{e_{i},e_{j}}=\frac{1}{|\mathcal{E}^{\prime}_{1}|}$
for all rows ($\forall e_{i}\in\mathcal{E}^{\prime}_{1}$) and
$\sum_{e_{i}\in\mathcal{E}^{\prime}_{1}}P^{*}_{e_{i},e_{j}}=\frac{1}{|\mathcal{E}^{\prime}_{2}|}$
for all columns ($\forall e_{j}\in\mathcal{E}^{\prime}_{2}$). Assume
$|\mathcal{E}^{\prime}_{1}|<|\mathcal{E}^{\prime}_{2}|$, then the decision
threshold is
$\frac{1}{2\cdot\text{min}(|\mathcal{E}^{\prime}_{1}|,|\mathcal{E}^{\prime}_{2}|)}=\frac{1}{2|\mathcal{E}^{\prime}_{1}|}$.
Entity pairs
$\\{(e_{i},e_{j})|\;P^{*}_{e_{i},e_{j}}>\frac{1}{2|\mathcal{E}^{\prime}_{1}|},e_{i}\in\mathcal{E}^{\prime}_{1},e_{j}\in\mathcal{E}^{\prime}_{2}\\}$
are selected as pseudo-labels.

For each pseudo-labeled entity pair $(e_{i},e_{j})$ with a probability value
$P^{*}_{e_{i},e_{j}}>\frac{1}{2|\mathcal{E}^{\prime}_{1}|}$, we can prove that
no conflicted pairs
$\\{(e_{i},e_{k})|\;e_{k}\in\mathcal{E}^{\prime}_{2}\setminus\\{e_{j}\\}\\}$
associated with $e_{i}$ are selected as pseudo-labels:

$\displaystyle P^{*}_{e_{i},e_{j}}$
$\displaystyle>\frac{1}{2|\mathcal{E}^{\prime}_{1}|},$
$\displaystyle\sum_{e_{j}\in\mathcal{E}^{\prime}_{2}}P^{*}_{e_{i},e_{j}}-P^{*}_{e_{i},e_{j}}$
$\displaystyle<\sum_{e_{j}\in\mathcal{E}^{\prime}_{2}}P^{*}_{e_{i},e_{j}}-\frac{1}{2|\mathcal{E}^{\prime}_{1}|},$
$\displaystyle\sum_{e_{k}\in\mathcal{E}^{\prime}_{2}\setminus\\{e_{j}\\}}P^{*}_{e_{i},e_{k}}+P^{*}_{e_{i},e_{j}}-P^{*}_{e_{i},e_{j}}$
$\displaystyle<\frac{1}{|\mathcal{E}^{\prime}_{1}|}-\frac{1}{2|\mathcal{E}^{\prime}_{1}|},$
$\displaystyle\sum_{e_{k}\in\mathcal{E}^{\prime}_{2}\setminus\\{e_{j}\\}}P^{*}_{e_{i},e_{k}}$
$\displaystyle<\frac{1}{2|\mathcal{E}^{\prime}_{1}|}.$ (16)

Since the coupling matrix
$P^{*}\in\mathbb{R}_{+}^{|\mathcal{E}^{\prime}_{1}|\times|\mathcal{E}^{\prime}_{2}|}$
has non-negative entries, the summation
$\sum_{e_{k}\in\mathcal{E}^{\prime}_{2}\setminus\\{e_{j}\\}}P^{*}_{e_{i},e_{k}}$
from Eq. (3.2) must be no smaller than any component of it, i.e.,
$P^{*}_{e_{i},e_{k}}\leq\sum_{e_{k}\in\mathcal{E}^{\prime}_{2}\setminus\\{e_{j}\\}}P^{*}_{e_{i},e_{k}}$,
$\forall e_{k}\in\mathcal{E}^{\prime}_{2}\setminus\\{e_{j}\\}$. Together with
Eq. (3.2), we can further derive that any component in the summation is
smaller than the decision threshold, i.e.,
$P^{*}_{e_{i},e_{k}}<\frac{1}{2|\mathcal{E}^{\prime}_{1}|}$, $\forall
e_{k}\in\mathcal{E}^{\prime}_{2}\setminus\\{e_{j}\\}$. In other words, all
other probability values in the same row of $P^{*}_{e_{i},e_{j}}$ are smaller
than the decision threshold. Thus, no conflicted pairs
$\\{(e_{i},e_{k})|\;e_{k}\in\mathcal{E}^{\prime}_{2}\setminus\\{e_{j}\\}\\}$
associated with $e_{i}$ are selected.

Similarly, for each pseudo-labeled entity pair $(e_{i},e_{j})$ with a
probability value $P^{*}_{e_{i},e_{j}}>\frac{1}{2|\mathcal{E}^{\prime}_{1}|}$,
we can prove that no conflicted pairs
$\\{(e_{l},e_{j})|\;e_{l}\in\mathcal{E}^{\prime}_{1}\setminus\\{e_{i}\\}\\}$
associated with entity $e_{j}$ are selected as pseudo-labels. Similar to Eq.
(3.2), we can also obtain
$\sum_{e_{l}\in\mathcal{E}^{\prime}_{1}\setminus\\{e_{i}\\}}P^{*}_{e_{l},e_{j}}<\frac{1}{2|\mathcal{E}^{\prime}_{2}|}$
and
$P^{*}_{e_{l},e_{j}}\leq\sum_{e_{l}\in\mathcal{E}^{\prime}_{1}\setminus\\{e_{i}\\}}P^{*}_{e_{l},e_{j}},\forall
e_{l}\in\mathcal{E}^{\prime}_{1}\setminus\\{e_{i}\\}$. Together with the
assumption of $|\mathcal{E}^{\prime}_{1}|<|\mathcal{E}^{\prime}_{2}|$, we can
further derive that
$P^{*}_{e_{l},e_{j}}\leq\sum_{e_{l}\in\mathcal{E}^{\prime}_{1}\setminus\\{e_{i}\\}}P^{*}_{e_{l},e_{j}}<\frac{1}{2|\mathcal{E}^{\prime}_{2}|}<\frac{1}{2|\mathcal{E}^{\prime}_{1}|}$.
Therefore, all other probability values in the same column of
$P^{*}_{e_{i},e_{j}}$ are smaller than the decision threshold, i.e.,
$P^{*}_{e_{l},e_{j}}<\frac{1}{2|\mathcal{E}^{\prime}_{1}|}$, $\forall
e_{l}\in\mathcal{E}^{\prime}_{1}\setminus\\{e_{i}\\}$. Hence, no conflicted
pairs
$\\{(e_{l},e_{j})|\;e_{l}\in\mathcal{E}^{\prime}_{1}\setminus\\{e_{i}\\}\\}$
associated with entity $e_{j}$ are selected.

In summary, we conclude that the selected pseudo-labels
$\\{(e_{i},e_{j})|\;P^{*}_{e_{i},e_{j}}>\frac{1}{2\cdot\text{min}(|\mathcal{E}^{\prime}_{1}|,|\mathcal{E}^{\prime}_{2}|)},e_{i}\in\mathcal{E}^{\prime}_{1},e_{j}\in\mathcal{E}^{\prime}_{2}\\}$
are guaranteed to be one-to-one alignments when
$|\mathcal{E}^{\prime}_{1}|<|\mathcal{E}^{\prime}_{2}|$, and the same
conclusion also holds when
$|\mathcal{E}^{\prime}_{1}|\geq|\mathcal{E}^{\prime}_{2}|$. ∎

The overall process of the OT-based pseudo-labeling algorithm is provided in
Algorithm 1. In Steps 1-7, the Sinkhorn algorithm takes the transport cost
matrix $C$ as input to estimate the optimal transport plan $P^{*}$ via $k$
iterations of row normalization and column normalization. In Step 8, entity
pairs with values in $P^{*}$ larger than the decision threshold are selected
as pseudo-labels. Finally, in Step 9, the model returns the set of pseudo-
labeled entity pairs. The overall time complexity of Algorithm 1 is
$O(k\cdot|\mathcal{E}^{\prime}_{1}|\cdot|\mathcal{E}^{\prime}_{2}|)$.

Input: Two unaligned entity sets
$\mathcal{E}^{\prime}_{1}\subseteq\mathcal{E}_{1}$ and
$\mathcal{E}^{\prime}_{2}\subseteq\mathcal{E}_{2}$. Transport cost matrix $C$
and regularization hyper-parameter $\beta$.

Output: Pseudo-labeled entity pair set $\hat{\mathcal{L}}_{e}$.

1 Initialize
$P=\mathbf{1}_{|\mathcal{E}^{\prime}_{1}|}\mathbf{1}_{|\mathcal{E}^{\prime}_{2}|}^{\top}$;

2
$\bm{a}=\frac{1}{|\mathcal{E}^{\prime}_{1}|}\mathbf{1}_{|\mathcal{E}^{\prime}_{1}|}$,
$Z=\text{e}^{-\frac{1}{\beta}C}$;

3 for  _iteration in 1…k_ do

4 $Q=Z\odot P$;

5 $\bm{b}=\frac{1}{|\mathcal{E}^{\prime}_{1}|Q^{\top}\bm{a}}$,
$\bm{a}=\frac{1}{|\mathcal{E}^{\prime}_{2}|Q\bm{b}}$;

6 $P=\bm{a}\bm{b}^{\top}\odot Q$;

7

8Obtain the optimal transport plan $P^{*}=P$;

9 Obtain conflict-free pseudo-labels
$\hat{\mathcal{L}}_{e}=\\{(e_{i},e_{j})|\;P^{*}_{e_{i},e_{j}}>\frac{1}{2\cdot\text{min}(|\mathcal{E}^{\prime}_{1}|,|\mathcal{E}^{\prime}_{2}|)},e_{i}\in\mathcal{E}^{\prime}_{1},e_{j}\in\mathcal{E}^{\prime}_{2}\\}$;

return _pseudo-labeled entity pair set $\hat{\mathcal{L}}_{e}$._

Algorithm 1 OT-based Pseudo-labeling with Sinkhorn Algorithm

###  3.3 Cross-Iteration Pseudo-Label Calibration

Through OT-based pseudo-labeling, a set of conflict-free entity alignment
pairs are obtained as pseudo-labels $\hat{\mathcal{L}}^{(t)}_{e}$ at each
iteration $t$. However, these pseudo-labels are still susceptible to one-to-
one misalignments (Type II pseudo-labeling errors), especially at early
training stages. Directly augmenting the prior alignment seeds with the
pseudo-label set $\hat{\mathcal{L}}^{(t)}_{e}$ would make the model overfit
these errors, thereby deteriorating the confirmation bias. Therefore, we
propose to further calibrate pseudo-labels through reducing the variability of
pseudo-label selection across consecutive iterations. As such, the resultant
pseudo-label set could have a higher precision and thus be more reliable.

Specifically, we obtain $m$ sets of pseudo-labels
$\\{\hat{\mathcal{L}}^{(t)}_{e}|t=1,...,m\\}$ respectively from every $m$
consecutive iterations of pseudo-labeling. To prevent pseudo-labeling errors
of previous iterations from adversely impacting the subsequent embedding
learning, we randomly reinitialize model parameters during each of these $m$
consecutive iterations, following the strategy proposed in [4]. This helps
decorrelate the dependency among $m$ consecutive iterations. Furthermore,
inspired by ensemble learning [48], we propose to calibrate pseudo-labels by
taking the common pseudo-labels among
$\\{\hat{\mathcal{L}}^{(t)}_{e}|t=1,...,m\\}$:

$\cap_{t=1}^{m}\hat{\mathcal{L}}^{(t)}_{e}=\bigl{\\{}(e_{i},e_{j})|\sum_{t=1}^{m}\mathbbm{1}\bigl{(}(e_{i},e_{j})\in\hat{\mathcal{L}}^{(t)}_{e}\bigl{)}=m,e_{i}\in\mathcal{E}^{\prime}_{1},e_{j}\in\mathcal{E}^{\prime}_{2}\bigl{\\}},$
(17)

where $\mathbbm{1}(\cdot)$ is a binary indicator function. By excluding
pseudo-labels with high selection variability, we can effectively reduce the
overall variability of pseudo-label selection to obtain calibrated pseudo-
labels $\cap_{t=1}^{m}\hat{\mathcal{L}}^{(t)}_{e}$. Mathematically, we can
prove that calibrated pseudo-labels have a higher precision than non-
calibrated ones.

######  Theorem 2.

The alignment pseudo-labels generated via pseudo-label calibration over $m$
consecutive iterations have a higher precision $\mathbb{P}(y=1|\hat{y}=1)$
than those generated at a single iteration, under the mild condition that
$\mathbb{P}(\hat{y}=1|(e_{i},e_{j}),y=1)$ is a constant $p$ for each
$(e_{i},e_{j})$ at each iteration.

###### Proof.

At the $t$-th iteration, for an entity pair $(e_{i},e_{j})$ with a ground-
truth alignment label $y=0$, the probability of aligning them is
$\mathbb{P}_{t}(\hat{y}=1|(e_{i},e_{j}),y=0)$. With the pseudo-label
calibration on $m$ consecutive iterations, the alignment probability becomes
$\prod_{l=0}^{m-1}\mathbb{P}_{t+l}(\hat{y}=1|(e_{i},e_{j}),y=0)$.
Similarly, at $t$-th iteration, for an entity pair $(e_{i},e_{j})$ with
ground-truth alignment label $y=1$, the alignment probability is
$\mathbb{P}_{t}(\hat{y}=1|(e_{i},e_{j}),y=1)=p_{t}$. With calibration, the
alignment probability is
$\prod_{l=0}^{m-1}\mathbb{P}_{t+l}(\hat{y}=1|(e_{i},e_{j}),y=1)=\prod_{l=0}^{m-1}p_{t+l}$.
With the increase of iterations, we assume the alignment model is increasingly
more accurate, that is, we have

$\displaystyle\prod_{l=0}^{m-1}\mathbb{P}_{t+l}(\hat{y}=1|(e_{i},e_{j}),y=0)\leq\mathbb{P}^{m}_{t}(\hat{y}=1|(e_{i},e_{j}),y=0),$
(18)
$\displaystyle\prod_{l=0}^{m-1}\mathbb{P}_{t+l}(\hat{y}=1|(e_{i},e_{j}),y=1)\geq\mathbb{P}_{t}^{m}(\hat{y}=1|(e_{i},e_{j}),y=1)=p_{t}^{m}.$

In the case with calibration, consider the probabilities,

$\displaystyle\mathbb{P}_{c}(\hat{y}=1,y=0)$
$\displaystyle=\sum_{(e_{i},e_{j})}\prod_{l=0}^{m-1}\mathbb{P}_{t+l}(\hat{y}=1|(e_{i},e_{j}),y=0)\mathbb{P}((e_{i},e_{j}),y=0)$
(19)
$\displaystyle\leq\sum_{(e_{i},e_{j})}\mathbb{P}_{t}^{m}(\hat{y}=1|(e_{i},e_{j}),y=0)\mathbb{P}((e_{i},e_{j}),y=0),$
$\displaystyle\mathbb{P}_{c}(\hat{y}=1,y=1)$
$\displaystyle=\sum_{(e_{i},e_{j})}\prod_{l=0}^{m-1}\mathbb{P}_{t+l}(\hat{y}=1|(e_{i},e_{j}),y=1)\mathbb{P}((e_{i},e_{j}),y=1)$
$\displaystyle\geq\sum_{(e_{i},e_{j})}\mathbb{P}_{t}^{m}(\hat{y}=1|(e_{i},e_{j}),y=1)\mathbb{P}((e_{i},e_{j}),y=1),$
$\displaystyle\mathbb{P}_{c}(\hat{y}=1,y=1)$
$\displaystyle\geq\sum_{(e_{i},e_{j})}p_{t}^{m}\mathbb{P}((e_{i},e_{j}),y=1).$

Thus, we have

$\frac{\mathbb{P}_{c}(\hat{y}=1,y=0)}{\mathbb{P}_{c}(\hat{y}=1,y=1)}\leq\frac{\sum_{(e_{i},e_{j})}\mathbb{P}_{t}^{m}(\hat{y}=1|(e_{i},e_{j}),y=0)\mathbb{P}((e_{i},e_{j}),y=0)}{\sum_{(e_{i},e_{j})}p_{t}^{m}\mathbb{P}((e_{i},e_{j}),y=1)}.$
(20)

In the case without calibration, at the $l$-th iteration, we have

$\frac{\mathbb{P}_{t}(\hat{y}=1,y=0)}{\mathbb{P}_{t}(\hat{y}=1,y=1)}=\frac{\sum_{(e_{i},e_{j})}\mathbb{P}_{t}(\hat{y}=1|(e_{i},e_{j}),y=0)\mathbb{P}((e_{i},e_{j}),y=0)}{\sum_{(e_{i},e_{j})}p_{t}\mathbb{P}((e_{i},e_{j}),y=1)}.$
(21)

By comparing the two ratios,

$\displaystyle\frac{\frac{\mathbb{P}_{c}(\hat{y}=1,y=0)}{\mathbb{P}_{c}(\hat{y}=1,y=1)}}{\frac{\mathbb{P}_{t}(\hat{y}=1,y=0)}{\mathbb{P}_{t}(\hat{y}=1,y=1)}}\leq\frac{\frac{\sum_{(e_{i},e_{j})}\mathbb{P}_{t}^{m}(\hat{y}=1|(e_{i},e_{j}),y=0)\mathbb{P}((e_{i},e_{j}),y=0)}{\sum_{(e_{i},e_{j})}p_{t}^{m}\mathbb{P}((e_{i},e_{j}),y=1)}}{\frac{\sum_{(e_{i},e_{j})}\mathbb{P}_{t}(\hat{y}=1|(e_{i},e_{j}),y=0)\mathbb{P}((e_{i},e_{j}),y=0)}{\sum_{(e_{i},e_{j})}p_{t}\mathbb{P}((e_{i},e_{j}),y=1)}},$
(22)
$\displaystyle\frac{\frac{\mathbb{P}_{c}(\hat{y}=1,y=0)}{\mathbb{P}_{c}(\hat{y}=1,y=1)}}{\frac{\mathbb{P}_{t}(\hat{y}=1,y=0)}{\mathbb{P}_{t}(\hat{y}=1,y=1)}}\leq\frac{\sum_{(e_{i},e_{j})}\mathbb{P}_{t}^{m}(\hat{y}=1|(e_{i},e_{j}),y=0)\mathbb{P}((e_{i},e_{j}),y=0)}{\sum_{(e_{i},e_{j})}p_{t}^{m-1}\mathbb{P}_{t}(\hat{y}=1|(e_{i},e_{j}),y=0)\mathbb{P}((e_{i},e_{j}),y=0)}.$

######  Lemma 1.

If all pseudo-label sets $\\{\hat{\mathcal{L}}^{(t)}_{e}|t=1,...,m\\}$ have
one-to-one correspondence and each pseudo-label set
$\hat{\mathcal{L}}^{(t)}_{e}$ has at least two correct alignments. Then the
condition
$\mathbb{P}(\hat{y}=1|(e_{i},e_{j}),y=1)=p>\mathbb{P}(\hat{y}=1|(e_{i},e_{j}),y=0)$
holds.

The detailed proof of Lemma 1 is provided in Appendix A. As
$\mathbb{P}_{t}(\hat{y}=1|(e_{i},e_{j}),y=0)<p_{t}=\mathbb{P}_{t}(\hat{y}=1|(e_{i},e_{j}),y=1)$,
then
$\mathbb{P}^{m}_{t}(\hat{y}=1|(e_{i},e_{j}),y=0)<p_{t}^{m-1}\mathbb{P}_{t}(\hat{y}=1|(e_{i},e_{j}),y=0)$
for each $(e_{i},e_{j})$. Therefore, we have

$\frac{\frac{\mathbb{P}_{c}(\hat{y}=1,y=0)}{\mathbb{P}_{c}(\hat{y}=1,y=1)}}{\frac{\mathbb{P}_{t}(\hat{y}=1,y=0)}{\mathbb{P}_{t}(\hat{y}=1,y=1)}}<1,\
\ \ \
\mathrm{i.e.,}\;\;\frac{\mathbb{P}_{c}(\hat{y}=1,y=0)}{\mathbb{P}_{c}(\hat{y}=1,y=1)}<\frac{\mathbb{P}_{t}(\hat{y}=1,y=0)}{\mathbb{P}_{t}(\hat{y}=1,y=1)}.$
(23)

Finally, the precision of pseudo-labels can be derived as:

$\displaystyle\mathbb{P}(y=1|\hat{y}=1)$
$\displaystyle=\frac{\mathbb{P}(\hat{y}=1,y=1)}{\mathbb{P}(\hat{y}=1)}=\frac{\mathbb{P}(\hat{y}=1,y=1)}{\mathbb{P}(\hat{y}=1,y=0)+\mathbb{P}(\hat{y}=1,y=1)}$
(24)
$\displaystyle=\frac{1}{\frac{\mathbb{P}(\hat{y}=1,y=0)}{\mathbb{P}(\hat{y}=1,y=1)}+1},$

whose value decreases with the increase of
$\frac{\mathbb{P}(\hat{y}=1,y=0)}{\mathbb{P}(\hat{y}=1,y=1)}$. According to
Eq. (23), we have

$\mathbb{P}_{c}(y=1|\hat{y}=1)>\mathbb{P}_{t}(y=1|\hat{y}=1).$ (25)

∎

Theorem 2 implies that the precision of pseudo-labels increases after cross-
iteration calibration, such that
$\mathbb{P}_{c}(y=1|\hat{y}=1)>\mathbb{P}_{t}(y=1|\hat{y}=1)$, where
$\mathbb{P}_{c}$ and $\mathbb{P}_{t}$ represent the probability of calibrated
set, and the probability of non-calibrated set from iteration $t$,
respectively.

Finally, the calibrated pseudo-labels are used to augment the prior alignment
seeds:

$\hat{\mathcal{L}}^{0}_{e}\leftarrow\mathcal{L}^{0}_{e}\cup\bigl{(}\cap_{t=1}^{m}\hat{\mathcal{L}}^{(t)}_{e}\bigl{)}.$
(26)

The augmented prior alignment seeds $\hat{\mathcal{L}}^{0}_{e}$ contain a
considerable number of reliably calibrated pseudo-labels with high precision
for subsequent model training. This enables to effectively eliminate Type II
pseudo-labeling errors and alleviate the problem of confirmation bias, thus
boosting the performance of entity alignment in turn.

###  3.4 Overall Workflow

The overall procedure of the proposed UPL-EA model is described in Algorithm
2. In Step 1, the algorithm is initialized with the prior alignment seeds if
provided. In Steps 3-8, the EA model and OT-based pseudo-labeling alternately
reinforce each other towards learning more informative entity embeddings.
During this process, negative alignment pairs are sampled in an adaptive
manner for model training. In Steps 2-9, the pseudo-label calibration is
performed every $m$ iterations to augment the prior alignment seeds with
reliably selected pseudo-labels for subsequent training. Finally, the learned
entity embeddings are used to infer newly aligned entity pairs.

Input: Two knowledge graphs
$\mathcal{G}_{1}=\\{\mathcal{E}_{1},\mathcal{R}_{1},\mathcal{T}_{1}\\}$,
$\mathcal{G}_{2}=\\{\mathcal{E}_{2},\mathcal{R}_{2},\mathcal{T}_{2}\\}$, prior
alignment seeds $\mathcal{L}^{0}_{e}$.

Output: Inferred entity alignment pairs.

1 Initialize $\hat{\mathcal{L}}^{0}_{e}=\mathcal{L}^{0}_{e}$;

2 repeat

3 for  _t in 1… $m$_ do

4 Obtain alignment pseudo-labels $\hat{\mathcal{L}}_{e}^{(t)}$ with Algorithm
1;

5 Augment alignment seeds:
$\mathcal{L}_{e}\leftarrow\hat{\mathcal{L}}^{0}_{e}\cup\hat{\mathcal{L}}_{e}^{(t)}$;

6 Sample negative pairs $\mathcal{L}^{\prime}_{e}$ according to
$\mathcal{L}_{e}$;

7 Learn entity embeddings with Eq. (6):
$\bm{h}_{e}\leftarrow\text{argmin}_{\bm{h}_{e}}L$;

8

9 Augment prior alignment seeds with the calibrated pseudo-labels:
$\hat{\mathcal{L}}^{0}_{e}\leftarrow\mathcal{L}^{0}_{e}\cup\bigl{(}\cap_{t=1}^{m}\hat{\mathcal{L}}^{(t)}_{e}\bigl{)}$;

10until  _convergence or reaching a fixed number of epochs_ ;

11Use the final learned entity embeddings to infer new aligned entity pairs;

return _inferred entity alignment pairs._

Algorithm 2 Entity Alignment with UPL-EA Model

##  4 Experiments

In this section, we validate the efficacy of our proposed method through
extensive experiments and ablation analyses on benchmark datasets.

###  4.1 Datasets and Baselines

We evaluate the performance of our UPL-EA111The source code of UPL-EA will be
released publicly upon paper acceptance. method on two benchmark datasets:
DBP15K [32] and SRPRS [10]. DBP15K is a widely used benchmark dataset for
entity alignment [9, 16, 43, 54]. DBP15K contains three cross-lingual
datasets, each of which contains two KGs built upon English and another
different language (Chinese, Japanese or French). Each cross-lingual dataset
has 15,000 aligned entity pairs. SRPRS is a more recently established
benchmark dataset with sparser connections [10]. SRPRS consists of two cross-
lingual datasets, each having two KGs built upon English and French/German as
well as 15,000 aligned entity pairs. The statistics of DBP15K and SRPRS are
summarized in Table 1.

Table 1: Statistics of DBP15K and SRPRS datasets Datasets | Entities | Relations | Rel.triplets
---|---|---|---
DBP15KZH_EN | Chinese | 66,469 | 2,830 | 153,929
English | 98,125 | 2,317 | 237,674
DBP15KJA_EN | Japanese | 65,744 | 2,043 | 164,373
English | 95,680 | 2,096 | 233,319
DBP15KFR_EN | French | 66,858 | 1,379 | 192,191
English | 105,889 | 2,209 | 278,590
SRPRSEN_FR | English | 15,000 | 221 | 36,508
French | 15,000 | 177 | 33,532
SRPRSEN_DE | English | 15,000 | 222 | 38,363
German | 15,000 | 120 | 37,377

For evaluation, we compare UPL-EA with 12 state-of-the-art entity alignment
models, which are categorized into two groups:

  * •

Supervised models, including MTransE [6], JAPE [32], JAPE in its structure-
only variant denoted as JAPE-Stru, GCN-Align [42], GCN-Align in its structure-
only variant denoted as GCN-Stru, RDGCN [43], HGCN [44], HMAN [47], and CEA
[50];

  * •

Pseudo-labeling based models, including IPTransE [53], BootEA [33], MRAEA
[17], RNM [54], and CPL-OT [9].

We utilize Hit@k ($\text{k}=1,10$) and Mean Reciprocal Rank (MRR) as the
evaluation metrics. Hit@k measures the percentage of correctly aligned
entities ranked in the top k candidate list. MRR measures the average of the
reciprocal ranks for the correctly aligned entities. Higher Hit@k and MRR
scores indicate better entity alignment performance.

###  4.2 Experimental Setup

For fair comparisons, we follow the conventional 30%-70% ratio to randomly
partition training and test data on DBP15K and SRPRS. We use semantic meanings
of entity names to construct entity features. On DBP15K with big linguistic
barriers, we first use Google Translate to translate non-English entity names
into English, then look up 768-dimensional word embeddings pre-trained by BERT
[8] with English entity names to form entity features. On SRPRS with small
linguistic barriers, we directly look up word embeddings without translation.
As each entity name comprises one or multiple words, we further use TF-IDF to
measure the contribution of each word towards entity name representation.
Finally, we aggregate TF-IDF-weighted word embeddings for each entity to form
its entity feature vector.

The parameter settings of UPL-EA is specified as follows: $K=125$,
$\beta=0.5$, $m=3$, $\gamma=1$ and $\lambda=10$. The embedding dimension $d$
is set to 300. For BERT pre-trained word embeddings, we use a PCA-based
technique [25] to reduce feature dimension from 768 to 300 with minimal
information loss. The batch size is set to 256 and the number of training
epochs is set to 100. We implement our model in PyTorch. The Adam optimizer is
used with a learning rate of 0.001 and 0.00025 on DBP15K and SRPRS,
respectively. All experiments are run on a computer with an Intel(R) Core(TM)
i9-13900KF CPU @ 3.00GHz and an NVIDIA Geforce RTX 4090 (24GB memory) GPU.

The results of MRAEA and CPL-OT on both benchmark datasets, and RNM on DBP15K
are obtained from their original papers. Results of other baselines are
obtained from [52]. For UPL-EA, we report the average results over five runs.

###  4.3 Comparison with State-of-the-Art Baselines

We compare the performance of the proposed UPL-EA with 12 state-of-the-art
entity alignment baselines. Table 2 and Table 3 report performance comparisons
on DBP15K and SRPRS, respectively. This set of results is reported with 30%
prior alignment seeds used for training. The mark “*” indicates that the
semantic information of entity names is used to construct entity features. The
best and second best results are highlighted in boldface and underlined,
respectively.

Table 2: Performance comparison on DBP15K Models |  DBP15KZH_EN |  DBP15KJA_EN |  DBP15KFR_EN
---|---|---|---
Hit@1 | Hit@10 | MRR | Hit@1 | Hit@10 | MRR | Hit@1 | Hit@10 | MRR
MtransE | 20.9 | 51.2 | 0.31 | 25.0 | 57.2 | 0.36 | 24.7 | 57.7 | 0.36
JAPE-Stru | 37.2 | 68.9 | 0.48 | 32.9 | 63.8 | 0.43 | 29.3 | 61.7 | 0.40
GCN-Stru | 39.8 | 72.0 | 0.51 | 40.0 | 72.9 | 0.51 | 38.9 | 74.9 | 0.51
JAPE* | 41.4 | 74.1 | 0.53 | 36.5 | 69.5 | 0.48 | 31.8 | 66.8 | 0.44
GCN-Align* | 43.4 | 76.2 | 0.55 | 42.7 | 76.2 | 0.54 | 41.1 | 77.2 | 0.53
HMAN* | 56.1 | 85.9 | 0.67 | 55.7 | 86.0 | 0.67 | 55.0 | 87.6 | 0.66
RDGCN* | 69.7 | 84.2 | 0.75 | 76.3 | 89.7 | 0.81 | 87.3 | 95.0 | 0.90
HGCN* | 70.8 | 84.0 | 0.76 | 75.8 | 88.9 | 0.81 | 88.8 | 95.9 | 0.91
CEA* | 78.7 | - | - | 86.3 | - | - | 97.2 | - | -
IPTransE | 33.2 | 64.5 | 0.43 | 29.0 | 59.5 | 0.39 | 24.5 | 56.8 | 0.35
BootEA | 61.4 | 84.1 | 0.69 | 57.3 | 82.9 | 0.66 | 58.5 | 84.5 | 0.68
MRAEA | 75.7 | 93.0 | 0.83 | 75.8 | 93.4 | 0.83 | 78.0 | 94.8 | 0.85
RNM* | 84.0 | 91.9 | 0.87 | 87.2 | 94.4 | 0.90 | 93.8 | 98.1 | 0.95
CPL-OT* | 92.7 | 96.4 | 0.94 | 95.6 | 98.3 | 0.97 | 99.0 | 99.4 | 0.99
UPL-EA | 94.9 | 97.4 | 0.96 | 97.0 | 98.8 | 0.98 | 99.5 | 99.7 | 1.00
Table 3: Performance comparison on SRPRS Models |  SRPRSEN_FR |  SRPRSEN_DE
---|---|---
Hit@1 | Hit@10 | MRR | Hit@1 | Hit@10 | MRR
MtransE | 21.3 | 44.7 | 0.29 | 10.7 | 24.8 | 0.16
JAPE-Stru | 24.1 | 53.3 | 0.34 | 30.2 | 57.8 | 0.40
GCN-Stru | 24.3 | 52.2 | 0.34 | 38.5 | 60.0 | 0.46
JAPE* | 24.1 | 54.4 | 0.34 | 26.8 | 54.7 | 0.36
GCN-Align* | 29.6 | 59.2 | 0.40 | 42.8 | 66.2 | 0.51
HMAN* | 40.0 | 70.5 | 0.50 | 52.8 | 77.8 | 0.62
RDGCN* | 67.2 | 76.7 | 0.71 | 77.9 | 88.6 | 0.82
HGCN* | 67.0 | 77.0 | 0.71 | 76.3 | 86.3 | 0.80
CEA* | 96.2 | - | - | 97.1 | - | -
IPTransE | 12.4 | 30.1 | 0.18 | 13.5 | 31.6 | 0.20
BootEA | 36.5 | 64.9 | 0.46 | 50.3 | 73.2 | 0.58
MRAEA | 46.0 | 76.8 | 0.56 | 59.4 | 81.5 | 0.66
RNM* | 92.5 | 96.2 | 0.94 | 94.4 | 96.7 | 0.95
CPL-OT* | 97.4 | 98.8 | 0.98 | 97.4 | 98.9 | 0.98
UPL-EA | 98.2 | 99.3 | 0.99 | 98.4 | 99.5 | 0.99

Overall, UPL-EA significantly outperforms most of the existing EA models on
five cross-lingual datasets. In particular, on DBP15KZH_EN, UPL-EA outperforms
the second and third performers, CPL-OT and RNM, by over 2% and 10%,
respectively, in terms of Hit@1. This affirms the necessity of systematically
combating confirmation bias for pseudo-labeling-based entity alignment. It is
worth noting that disparities in overall performance can be observed among the
five cross-lingual datasets, where the lowest accuracy is achieved on
DBP15KZH_EN due to its large linguistic barriers. Nevertheless, for the most
challenging EA task on DBP15KZH_EN, UPL-EA yields strong performance gains
over other baselines.

###  4.4 Comparison w.r.t. Different Rates of Prior Alignment Seeds

Table 4 further compares UPL-EA with four representative baselines (BootEA,
RDGCN, RNM, and CPL-OT) with respect to different rates of prior alignment
seeds varying from 10% to 40% on DBP15K. As expected, UPL-EA consistently
outperforms four competitors on all three cross-lingual datasets at all rates.
This is attributed to UPL-EA’s ability to augment the training set with
reliable pseudo-labeled entity pairs by effectively alleviating confirmation
bias. As the rate of prior alignment seeds decreases, the performance of
BootEA significantly degrades due to its limited ability to prevent the
accumulation of pseudo-labeling errors. RNM yields performance gains over
RDGCN owing to its posterior embedding distance editing using iteratively
pseudo-labeled entity pairs; however, the lack of iterative model re-training
largely hinders its performance. CPL-OT demonstrates more stable performance
with different rates of prior alignment seeds because it selects pseudo-labels
via the conflict-aware OT modeling and then uses them to train entity
alignment model in turn; nevertheless, the neglect of one-to-one misalignments
(Type II pseudo-labeling errors) limits the potential of CPL-OT. In contrast,
the performance of UPL-EA remains competitively stable when the rate of prior
alignment seeds decreases, even on the most challenging DBP15KZH_EN dataset.

Table 4: Performance comparison w.r.t different rates of prior alignment seeds Models |  DBP15KZH_EN |  DBP15KJA_EN |  DBP15KFR_EN
---|---|---|---
Hit@1
10% | 20% | 30% | 40% | 10% | 20% | 30% | 40% | 10% | 20% | 30% | 40%
BootEA | 45.7 | 57.3 | 62.9 | 67.9 | 42.9 | 53.5 | 62.2 | 66.0 | 47.3 | 59.8 | 65.3 | 68.6
RDGCN | 66.6 | 68.9 | 70.8 | 72.6 | 72.4 | 74.5 | 76.7 | 79.0 | 86.3 | 87.6 | 88.6 | 89.7
RNM | 79.3 | 81.7 | 84.0 | 85.4 | 83.4 | 85.9 | 87.2 | 88.8 | 92.3 | 93.0 | 93.8 | 94.5
CPL-OT | 91.8 | 92.2 | 92.7 | 93.0 | 94.7 | 95.1 | 95.6 | 96.1 | 98.7 | 98.9 | 99.1 | 99.2
UPL-EA | 93.6 | 94.1 | 94.9 | 95.2 | 96.4 | 96.8 | 97.0 | 97.4 | 99.2 | 99.4 | 99.5 | 99.5

###  4.5 Ablation Study

To testify the importance of different key components of the proposed UPL-EA
model, we conduct a series of ablation studies on five cross-lingual datasets
from DBP15K and SRPRS. To provide more insights, apart from the conventional
setting using 30% prior alignment seeds as training data, we also conduct our
analysis on the setting with no prior alignment seeds. The full UPL-EA model
is compared with its ablated variants, with the best performance highlighted
by boldface. From Table 5 and Table 6, we can see that the full UPL-EA model
performs best in all cases.

Table 5: Ablation study on DBP15K Models |  DBP15KZH_EN |  DBP15KJA_EN |  DBP15KFR_EN
---|---|---|---
Hit@1 | Hit@10 | MRR | Hit@1 | Hit@10 | MRR | Hit@1 | Hit@10 | MRR
30% Prior Alignment Seeds
Full Model | 94.9 | 97.4 | 0.96 | 97.0 | 98.8 | 0.98 | 99.5 | 99.7 | 1.00
w.o. Global-lev. Rel. Aggr. | 92.6 | 96.4 | 0.94 | 96.4 | 98.5 | 0.97 | 99.3 | 99.7 | 1.00
w.o. Within-iter. OT | 80.1 | 90.4 | 0.84 | 87.6 | 94.8 | 0.90 | 94.4 | 97.4 | 0.96
w.o. Cross-iter. Calibration | 84.9 | 90.6 | 0.87 | 91.1 | 95.5 | 0.93 | 98.1 | 99.0 | 0.98
w.o. Within & Cross-iter. | 74.8 | 87.3 | 0.80 | 83.4 | 93.6 | 0.87 | 93.1 | 97.5 | 0.95
No Prior Alignment Seeds
Full Model | 93.1 | 96.2 | 0.94 | 95.7 | 98.3 | 0.97 | 99.2 | 99.5 | 0.99
w.o. Global-lev. Rel. Aggr. | 91.4 | 95.3 | 0.93 | 95.3 | 98.1 | 0.96 | 99.1 | 99.7 | 0.99
w.o. Within-iter. OT | 66.9 | 75.5 | 0.70 | 76.9 | 85.2 | 0.80 | 91.9 | 95.2 | 0.93
w.o. Cross-iter. Calibration | 83.0 | 88.7 | 0.85 | 90.1 | 94.6 | 0.92 | 97.8 | 98.8 | 0.98
w.o. Within & Cross-iter. | 67.1 | 76.8 | 0.71 | 77.1 | 86.2 | 0.81 | 91.4 | 95.9 | 0.93
Table 6: Ablation study on SRPRS Models |  SRPRSEN_FR |  SRPRSEN_DE
---|---|---
Hit@1 | Hit@10 | MRR | Hit@1 | Hit@10 | MRR
30% Prior Alignment Seeds
Full Model | 98.2 | 99.3 | 0.99 | 98.4 | 99.5 | 0.99
w.o. Global-lev. Rel. Aggr. | 97.5 | 98.9 | 0.98 | 97.8 | 99.2 | 0.98
w.o. Within-iter. OT | 93.9 | 96.6 | 0.95 | 94.2 | 97.5 | 0.95
w.o. Cross-iter. Calibration | 94.6 | 97.4 | 0.96 | 94.8 | 98.1 | 0.96
w.o. Within & Cross-iter. | 92.7 | 96.5 | 0.94 | 93.6 | 97.4 | 0.95
No Prior Alignment Seeds
Full Model | 97.9 | 99.2 | 0.98 | 97.7 | 99.2 | 0.98
w.o. Global-lev. Rel. Aggr. | 97.1 | 98.6 | 0.98 | 97.4 | 98.7 | 0.98
w.o. Within-iter. OT | 89.8 | 93.0 | 0.91 | 91.4 | 95.2 | 0.93
w.o. Cross-iter. Calibration | 94.2 | 97.0 | 0.95 | 94.8 | 97.7 | 0.96
w.o. Within & Cross-iter. | 89.7 | 93.1 | 0.91 | 91.1 | 94.9 | 0.93

  * •

w.o. Global-lev. Rel. Aggr.: We compare the full UPL-EA model against the
variant without using global-level relation aggregation for entity embedding
learning. This ablation leads to degraded performance on both settings
primarily due to the increase in conflicted misalignments caused by feature
over-smoothing.

  * •

w.o. Within-iter. OT: To study the effectiveness of within-iteration OT
modeling for pseudo-labeling, we ablate it from the full UPL-EA model. As OT
modeling can eliminate a considerable number of conflicted misalignments (Type
I pseudo-labeling errors) to ensure one-to-one alignments at each iteration,
this ablation results in a significant performance drop across all datasets in
both settings.

  * •

w.o. Cross-iter. Calibration: To study the efficacy of cross-iteration pseudo-
label calibration, we remove it from UPL-EA. This ablation has a profound
negative effect, leading to substantial performance declines in both settings,
especially on DBP15KZH_EN and DBP15KJA_EN with large linguistic barriers,
whereas the performance drops are smaller on DBP15KFR_EN with relatively small
linguistic barriers. This is because larger linguistic barriers tend to incur
more one-to-one misalignments (Type II pseudo-labeling errors). Our findings
confirm that pseudo-label calibration is crucial for UPL-EA to achieve its
full potential, especially when there exists a large number of pseudo-labeling
errors at the beginning of model training.

  * •

w.o. Within & Cross-iter.: We also analyze the overall effect of ablating both
within-iteration OT modeling and cross-iteration pseudo-label calibration from
the full model. This ablation has a substantial adverse impact, leading to a
dramatic performance drop in all cases. The results demonstrate the
effectiveness of our unified UPL-EA framework in systematically combating
confirmation bias for pseudo-labeling-based entity alignment.

Note that under the setting with no prior alignment seeds, the variant without
OT-based pseudo-labeling (w.o. Within-iter. OT) has similar performance as
compared to the variant completely ignoring confirmation bias (w.o. Within &
Cross-iter.). In particular, on DBP15KZH_EN and DBP15KJA_EN, the former
variant even performs slightly worse. This is because under the challenging
case where there are no prior alignment seeds, ablating OT-based pseudo-
labeling might incur considerably more conflicted misalignments (Type I
pseudo-labeling errors), violating Lemma 1 and thus making Theorem 2 invalid
because the number of misalignments (false positives) is not bounded. As a
result, it becomes ineffective to calibrate erroneous pseudo-labels.

###  4.6 Impact of Pre-trained Word Embeddings

To analyze the impact of using different pre-trained word embeddings, we
report the results of UPL-EA that form entity features with Glove embedding
[23], which is widely used in the existing EA models. We conduct this analysis
on the setting with 30% prior alignment seeds. The results on DBP15K are
reported in Table 7 as a case study. We can observe that UPL-EA with Glove
embedding still achieves competitive results, significantly outperforming all
other baselines. This confirms that the efficacy of UPL-EA is not highly
dependent on embedding initialization methods used. When switching from Glove
embedding to BERT pre-trained embeddings, performance gains can be observed,
especially on DBP15KJA_EN. This indicates the usefulness of pre-trained word
embeddings with high quality for entity alignment.

Table 7: Impact of pre-trained word embeddings Models |  DBP15KZH_EN |  DBP15KJA_EN |  DBP15KFR_EN
---|---|---|---
Hit@1 | Hit@10 | MRR | Hit@1 | Hit@10 | MRR | Hit@1 | Hit@10 | MRR
Glove | 94.0 | 97.6 | 0.95 | 95.9 | 98.9 | 0.97 | 99.1 | 99.8 | 0.99
BERT | 94.9 | 97.4 | 0.96 | 97.0 | 98.8 | 0.98 | 99.5 | 99.7 | 1.00

(a) Embedding dimension $d$

(b) Number of iterations for calibration $m$

(c) Regularization hyper-parameter $\beta$

(d) Margin hyper-parameter $\gamma$

Figure 4: Hyper-parameter sensitivity study on DBP15KZH_EN

###  4.7 Parameter Sensitivity Study

We further study the sensitivity of our UPL-EA model with regards to four
hyper-parameters: embedding dimension $d$, number of iterations $m$ for cross-
iteration pseudo-label calibration, regularization hyper-parameter $\beta$ for
Sinkhorn algorithm, and margin parameter $\gamma$ for defining the alignment
ranking loss. This sensitivity analysis is conducted on DBP15KZH_EN with 30%
prior alignment seeds as a case study. The respective results in terms of
Hit@1 and Hit@10 are plotted in Fig. 4.

As we can see from Fig. 4(a), the performance of UPL-EA improves considerably
as the embedding dimension $d$ increases from 100 to 300 and then retains at a
relatively stable level. For the number of iterations $m$ used for pseudo-
label calibration, having cross-iteration calibration ($m>1$) leads to
considerably better performance as opposed to not having calibration ($m=1$).
This proves the effectiveness of our proposed calibration mechanism, requiring
only a very small number of iterations used for calibration (i.e., $m=3$) to
achieve competitive performance (see Fig. 4(b)). In addition, as observed in
Fig. 4(c), the performance of UPL-EA is insensitive to different values of
$\beta$ associated with OT-based pseudo-labeling. As for the margin parameter
$\gamma$, the performance of UPL-EA starts to drop gradually after $\gamma$
exceeds 1, as shown in Fig. 4(d). This observation is reasonable, as a larger
margin parameter would give more tolerance to alignment errors, thus degrading
model performance.

###  4.8 Empirical Evidence on the Effectiveness of Pseudo-label Calibration

Lastly, to validate the effectiveness of cross-iteration pseudo-label
calibration, we perform an empirical analysis to compare the precision of
pseudo-labeling with and without calibration. This empirical analysis is
carried out on DBP15K as a case study and under the setting with 30% prior
alignment seeds. In Fig. 5, we directly compare the precision between pseudo-
labeling with calibration (indicated by the blue curve) and without
calibration (indicated by the orange curve) over eight pseudo-labeling
iterations. Notably, pseudo-labeling with calibration consistently achieves a
higher precision compared to pseudo-labeling without calibration throughout
the training process. This discrepancy is particularly profound in the case of
DBP15KZH_EN and DBP15KJA_EN, where significant linguistic barriers exist.
Specifically, without calibration, the precision of pseudo-labeling
deteriorates gradually as the training progresses due to confirmation bias
arising from the accumulation of one-to-one misalignments (Type II pseudo-
labeling errors). In contrast, when employing pseudo-label calibration, the
precision of pseudo-labeling approaches to almost 100% after only three
consecutive iterations across all datasets. These findings affirm the
necessity and efficacy of calibrating pseudo-labels across iterations to
mitigate the adverse impact of confirmation bias, particularly on challenging
datasets like DBP15KZH_EN.

(a) DBP15KZH_EN

(b) DBP15KJA_EN

(c) DBP15KFR_EN

Figure 5: Precision comparison between pseudo-labeling with calibration and
without calibration

##  5 Related Work

In this section, we review three streams of related literature, including
entity alignment in knowledge graphs, pseudo-labeling in semi-supervised
learning, and optimal transport on graphs.

###  5.1 Entity Alignment in Knowledge Graphs

Most entity alignment models are embedding-based approaches, which exploit
distances between entity embeddings in latent spaces to measure inherent
semantic correspondences between entities. Inspired by TransE [3], MTransE [6]
embeds two KGs into two respective embedding spaces, where a transformation
matrix is learned using prior alignment seeds. To reduce the number of
parameters involved, most subsequent models [32, 33, 53] embed KGs into a
common latent space by imposing the embeddings of pre-aligned entities to be
as close as possible. This ensures that alignment similarities between
entities can be directly measured via their embeddings.

To leverage KG structural information, methods like GCN-Align [42] utilize
GCNs to obtain better embeddings for entity alignment. However, GCNs and their
variants are inclined to result in alignment conflicts, as their feature
smoothness scheme tends to make entities have similar embeddings among local
neighborhoods. To reduce the over-smoothing effect, more recent works [44, 43,
54] adopt a highway strategy [29] on GCN layers, which “mixes” the learned
entity embeddings with the original features. Another line of research efforts
is devoted to improving GCN-based approaches through considering heterogeneous
relations in KGs. HGCN [44] jointly learns the embeddings of entities and
relations, without considering the directions of relations. RDGCN [43]
performs embedding learning on a dual relation graph, but fails to incorporate
statistical information of neighboring relations of an entity. RNM [54] uses
iterative relational neighborhood matching to refine finalized entity
embedding distances. This matching mechanism proves to be empirically
effective, but it is used only after the completion of model training and
fails to reinforce embedding learning in turn. All the aforementioned models,
however, require an abundance of prior alignment seeds provided for training
purposes, which are labor-intensive and costly to acquire in real-world KGs.

To tackle the shortage of prior alignment seeds, semi-supervised EA models
have been proposed in recent years. As a prominent learning paradigm among
such, pseudo-labeling-based methods, e.g., BootEA [33], IPTransE [53], RNM
[54], MRAEA [17], and CPL-OT [9], propose to iteratively pseudo-label
unaligned entity pairs and add them to prior alignment seeds for subsequent
model retraining. For RNM, there is a slight difference that it augments prior
alignment seeds to rectify embedding distance after the completion of model
training. Although these methods have achieved promising performance gains,
the confirmation bias associated with iterative pseudo-labeling has been
largely under-explored. Only limited attempts have been recently made to
alleviate Type I pseudo-labeling errors (conflicted misalignments) while Type
II errors (one-to-one misalignments) have been completely overlooked. To
eliminate Type I pseudo-labeling errors, RNM [54] and MRAEA [17] use simple
heuristics to preserve only the most convincing alignment pairs, for example,
those with the smallest distance, at the presence of conflicts. BootEA [33]
and CPL-OT [9], on the other hand, model the inference of pseudo-labeled
entity pairs as an assignment problem, where the most likely aligned pairs are
selected at each pseudo-labeling iteration. Unlike BootEA that selects a small
set of pseudo-labels using a pre-specified threshold, CPL-OT imposes a full
match between two unaligned entity sets to maximize the number of pseudo-
labels at each iteration. Nevertheless, both methods impose constraints to
enforce hard alignments for the purpose of alleviating alignment conflicts,
but this might potentially lead to an increase in more one-to-one
misalignments (Type II errors).

To fill in the research gap, our work explicitly tackles both Type I and Type
II pseudo-labeling errors to combat confirmation bias in a principled way. The
proposed UPL-EA effectively eliminates Type I pseudo-labeling errors by
casting entity alignment inference into a discrete OT problem; this
formulation enables more accurate determination of entity correspondences
across KGs. In combination with a carefully designed selection criterion, UPL-
EA is guaranteed to generate one-to-one alignments as pseudo-labels at each
iteration. Furthermore, UPL-EA calibrates pseudo-labels by reducing local
pseudo-label selection variability across iterations to alleviate Type II
pseudo-labeling errors, preventing erroneous pseudo-labels from propagating
and jeopardizing subsequent model training.

###  5.2 Pseudo-Labeling

Pseudo-labeling has emerged as an effective semi-supervised approach in
addressing the challenge of label scarcity. It refers to a self-training
paradigm where the model is iteratively bootstrapped with additional labeled
data based on its own predictions. The pseudo-labels generated from model
predictions can be defined as hard (one-hot distribution) or soft (continuous
distribution) labels [13, 28, 1]. More specifically, pseudo-labeling
strategies are designed to select high-confidence unlabeled data by either
directly taking the model’s predictions, or sharpening the predicted
probability distribution. It is closely related to entropy regularization
[27], where the model’s predictions are encouraged to have low entropy (i.e.,
high-confidence) on unlabeled data. The selected pseudo-labels are then used
to augment the training set and to fine-tune the model initially trained on
the given labels. This training regime is also extended to an explicit
teacher-student configuration [24], where a teacher network generates pseudo-
labels from unlabeled data, which are used to train a student network.

Despite its promising results, pseudo-labeling is inevitably susceptible to
erroneous pseudo-labels, thus suffering from confirmation bias [1, 26], where
the prediction errors would accumulate and degrade model performance. The
confirmation bias has been recently studied in the field of computer vision.
In works like [1, 26], confirmation bias is considered as a problem of poor
network calibration, where the network is overfitted towards erroneous pseudo-
labels. To alleviate confirmation bias, pseudo-labeling approaches have
adopted strategies such as mixup augmentation [1] and uncertainty weighting
[26]. Subsequent works like [4, 51] address confirmation bias by applying
curriculum learning principles, where the decision threshold is adaptively
adjusted during the training process and model parameters are re-initialized
after each iteration.

Recently, pseudo-labeling has also been studied on graphs for the task of
semi-supervised node classification [14, 31, 15]. Li et al [14] propose a
self-trained GCN that enlarges the training set by assigning a pseudo-label to
high-confidence unlabeled nodes, and then re-trains the model using both
genuine labels and pseudo-labels. The pseudo-labels are generated via a random
walk model in a co-training manner. Sun et al [31] show that a shallow GCN is
ineffective in propagating label information under few-label settings, and
employ a multi-stage self-training approach that relies on a deep clustering
model to assign pseudo-labels. Li et al [15] propose to incorporate the node
informativeness scores for the selection of pseudo-labels and adopt distinct
loss functions for genuine labels and pseudo-labels during model training.
Despite these research efforts, the problem of confirmation bias remains
under-explored in graph domains. This work systematically analyzes the cause
of confirmation bias and proposes a principled approach to conquer
confirmation bias for pseudo-labeling-based entity alignment across KGs.

###  5.3 Optimal Transport on Graphs

Optimal Transport (OT) is the general problem of finding an optimal plan to
move one distribution of mass to another with the minimal cost [38]. As an
effective metric to define the distance between probability distributions, OT
has been applied in computer vision and natural language processing over a
range of tasks including machine translation, text summarization, and image
captioning [37, 5]. In recent years, OT has also been studied on graphs to
match graphs with similar structures or align nodes/entities across graphs.
For graph partitioning and matching, the transport on the edges across graphs
has been used to define the Gromov-Wasserstein (GW) discrepancy [36] that
measures how the edges in a graph compare to those in another graph [46, 19,
45]. For entity alignment across graphs, Pei et al [22] incorporate an OT
objective into the overall loss to enhance the learning of entity embeddings.
Tang et al [34] propose to jointly perform structure learning and OT alignment
through minimizing the multi-view GW distance matrices between two attributed
graphs. These methods have primarily used OT to define a learning objective,
which involves bi-level optimization for model training. To further enhance
the scalability of OT modeling for entity alignment, Mao et al [18] propose to
make the similarity matrix sparse by dropping its entries close to zero.
However, this sparse OT modeling potentially violates the constraints of the
OT objective, thereby failing to guarantee one-to-one correspondences across
two KGs. In our work, we focus on tackling the scarcity of prior alignment
seeds via iterative pseudo-labeling; we seek to find more accurate
correspondences between entities via OT modeling. This enables us to derive a
one-to-one alignment configuration more precisely to eliminate conflicted
misalignments (Type I pseudo-labeling errors) at each iteration, mitigating
the negative impact of confirmation bias.

##  6 Conclusion

We proposed a novel unified pseudo-labeling framework (UPL-EA) that addresses
the problem of confirmation bias for pseudo-labeling-based entity alignment
across KGs. UPL-EA employs an entity alignment model based on the global-local
aggregation architecture to generate informative entity embeddings. In
addition, UPL-EA includes two modules to combat confirmation bias: within-
iteration Optimal Transport (OT)-based pseudo-labeling and cross-iteration
pseudo-label calibration. The OT-based pseudo-labeling module utilizes OT
modeling to eliminate conflicted misalignments (Type I pseudo-labeling errors)
within each iteration. The pseudo-label calibration module employs pseudo-
labels from multiple consecutive iterations to reduce pseudo-label selection
variability, thus preventing the accumulation and propagation of inevitable
one-to-one misalignments (Type II pseudo-labeling errors) across iterations.
Our extensive experiments on benchmark datasets show that UPL-EA outperforms
state-of-the-art baselines with limited amounts of prior alignment seeds. The
competitive performance of UPL-EA demonstrates its superiority in addressing
confirmation bias and its potential for pseudo-labeling-based entity alignment
across KGs.

##  Appendix A Proof of Lemma 1

###### Proof.

Given two unaligned entity sets $\mathcal{E}^{\prime}_{1}\in\mathcal{G}_{1}$
and $\mathcal{E}^{\prime}_{2}\in\mathcal{G}_{2}$, there is an entity alignment
candidate space
$\mathcal{E}^{\prime}_{1}\times\mathcal{E}^{\prime}_{2}\in\\{y=0,y=1\\}^{|\mathcal{E}^{\prime}_{1}|\times|\mathcal{E}^{\prime}_{2}|}$.
The pseudo-label selection process can be considered as a classification
problem, where each entity pair
$(e_{i},e_{j}),\;e_{i}\in\mathcal{E}^{\prime}_{1},\;e_{j}\in\mathcal{E}^{\prime}_{2}$
in the candidate space is classified as either being aligned ($\hat{y}=1$) or
not aligned ($\hat{y}=0$). Let $|\mathcal{E}^{\prime}_{1}|=m$ and
$|\mathcal{E}^{\prime}_{2}|=n$, we can thus obtain a confusion matrix based on
the pseudo-label selection with one-to-one correspondence as given in Table 8.

Table 8: Confusion matrix for entity alignment classification via pseudo-label selection with one-to-one correspondence | $y=1$ | $y=0$ | Sum
---|---|---|---
$\hat{y}=1$ | $TP$ | $FP$ | $0\leq TP+FP\leq\text{min}(m,n)$
$\hat{y}=0$ | $FN$ | $TN$ | $mn\geq FN+TN\geq mn-\text{min}(m,n)$
Sum | $=\text{min}(m,n)$ | $=mn-\text{min}(m,n)$ | TP+FP+FN+TN $=mn$

$TP$, $FP$, $FN$ and $TN$ represent “True Positive”, “False Positive”, “False
Negative” and “True Negative”, respectively. The number of ground truth entity
alignment pairs with $y=1$ is $TP+FN=\text{min}(m,n)$. Correspondingly, the
number of ground truth entity pairs that are not aligned ($y=0$) is
$FP+TN=mn-\text{min}(m,n)$. Since the pseudo-labels obtained using our
proposed OT-based modeling satisfy one-to-one correspondence, the number of
pseudo-labeled entity pairs with $\hat{y}=1$ is bounded, i.e., $0\leq
TP+FP\leq\text{min}(m,n)$, thus the number of misalignments is bounded such
that $FP\leq\text{min}(m,n)-TP$.

Denote $p=\mathbb{P}(\hat{y}=1|(e_{i},e_{j}),y=1)$ and
$p^{\prime}=\mathbb{P}(\hat{y}=1|(e_{i},e_{j}),y=0)$. Let us consider two
cases below.

  * •

When $m<n$, we have $\text{min}(m,n)=m$, $FP\leq m-TP$, $0<\frac{m}{n}<1$.
Thus,

$\displaystyle
p=\frac{TP}{TP+FN}=\frac{TP}{m},\;p^{\prime}=\frac{FP}{FP+TN}=\frac{FP}{mn-m}.$
(27)

We can calculate the ratio

$\displaystyle\frac{p}{p^{\prime}}=\frac{TP}{m}\cdot\frac{mn-m}{FP}=\frac{TP}{FP}\cdot(n-1).$
(28)

For the condition $p>p^{\prime}$ to hold, we have

$\displaystyle\frac{p}{p^{\prime}}=\frac{TP}{FP}(n-1)$ $\displaystyle>1,$ (29)
$\displaystyle TP(n-1)$ $\displaystyle>FP,$

Since $FP\leq m-TP$, we ensure

$\displaystyle TP(n-1)$ $\displaystyle>m-TP,$ (30) $\displaystyle nTP-TP$
$\displaystyle>m-TP,$ $\displaystyle TP$ $\displaystyle>\frac{m}{n},$

Since $0<\frac{m}{n}<1$, we ensure $TP\geq 1$. Therefore, as long as there is
at least one correct alignment ($TP\geq 1$) in the pseudo-label set when
$m<n$, the condition $p>p^{\prime}$ holds.

  * •

Similarly, when $m\geq n$, $\text{min}(m,n)=n$, we have $FP\leq n-TP$,
$0<\frac{n}{m}\leq 1$. Thus,

$\displaystyle
p=\frac{TP}{n},\;p^{\prime}=\frac{FP}{mn-n},\;\frac{p}{p^{\prime}}=\frac{TP}{FP}\cdot(m-1).$
(31)

For the condition $p>p^{\prime}$ to hold, we have

$\displaystyle\frac{p}{p^{\prime}}=\frac{TP}{FP}(m-1)$ $\displaystyle>1,$ (32)
$\displaystyle TP(m-1)$ $\displaystyle>FP,$

Since $FP\leq n-TP$, we ensure

$\displaystyle mTP-TP$ $\displaystyle>n-TP,$ (33) $\displaystyle TP$
$\displaystyle>\frac{n}{m},$

Since $0<\frac{n}{m}\leq 1$, we ensure $TP>1$. Therefore, as long as there are
at least two correct alignments ($TP\geq 2>1$) in the pseudo-label set when
$m\geq n$, the condition $p>p^{\prime}$ holds.

Putting together, as long as there are at least two correct alignments
($TP\geq 2$) in the pseudo-label set with one-to-one correspondence, the
condition
$\mathbb{P}(\hat{y}=1|(e_{i},e_{j}),y=1)=p>\mathbb{P}(\hat{y}=1|(e_{i},e_{j}),y=0)$
holds. ∎

## References

  *   * Arazo et al [2020] Arazo E, Ortego D, Albert P, et al (2020) Pseudo-labeling and confirmation bias in deep semi-supervised learning. In: IJCNN, IEEE, pp 1–8
  * Bollacker et al [2008] Bollacker K, Evans C, Paritosh P, et al (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: SIGMOD, pp 1247–1250
  * Bordes et al [2013] Bordes A, Usunier N, Garcia-Durán A, et al (2013) Translating embeddings for modeling multi-relational data. In: NeurIPS, pp 2787–2795
  * Cascante-Bonilla et al [2021] Cascante-Bonilla P, Tan F, Qi Y, et al (2021) Curriculum labeling: Revisiting pseudo-labeling for semi-supervised learning. In: AAAI, pp 6912–6920
  * Chen et al [2020] Chen L, Gan Z, Cheng Y, et al (2020) Graph optimal transport for cross-domain alignment. In: ICML, pp 1542–1553
  * Chen et al [2017] Chen M, Tian Y, Yang M, et al (2017) Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In: IJCAI, pp 1511–1517
  * Cuturi [2013] Cuturi M (2013) Sinkhorn distances: Lightspeed computation of optimal transport. In: NeurIPS, pp 2292–2300
  * Devlin et al [2019] Devlin J, Chang M, Lee K, et al (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT, ACL, pp 4171–4186
  * Ding et al [2022] Ding Q, Zhang D, Yin J (2022) Conflict-aware pseudo labeling via optimal transport for entity alignment. In: ICDM, IEEE, pp 915–920
  * Guo et al [2019] Guo L, Sun Z, Hu W (2019) Learning to exploit long-term relational dependencies in knowledge graphs. In: ICML, pp 2505–2514
  * Guo et al [2022] Guo Q, Zhuang F, Qin C, et al (2022) A survey on knowledge graph-based recommender systems. IEEE Transactions on Knowledge and Data Engineering 34(8):3549–3568
  * Kipf and Welling [2017] Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: ICLR
  * Lee [2013] Lee DH (2013) Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: ICML Workshop: Challenges in Representation Learning, p 896
  * Li et al [2018] Li Q, Han Z, Wu X (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: AAAI, pp 3538–3545
  * Li et al [2023] Li Y, Yin J, Chen L (2023) Informative pseudo-labeling for graph neural networks with few labels. Data Mining and Knowledge Discovery 37:228––254
  * Liu et al [2022] Liu X, Hong H, Wang X, et al (2022) Selfkg: Self-supervised entity alignment in knowledge graphs. In: WWW, pp 860–870
  * Mao et al [2020] Mao X, Wang W, Xu H, et al (2020) Mraea: an efficient and robust entity alignment approach for cross-lingual knowledge graph. In: WSDM, pp 420–428
  * Mao et al [2022] Mao X, Wang W, Wu Y, et al (2022) Lightea: A scalable, robust, and interpretable entity alignment framework via three-view label propagation. In: EMNLP, pp 825–838
  * Maretic et al [2019] Maretic HP, Gheche ME, Chierchia G, et al (2019) GOT: an optimal transport framework for graph comparison. In: NeurIPS, pp 13876–13887
  * Orlin [1997] Orlin JB (1997) A polynomial time primal network simplex algorithm for minimum cost flows. Mathematical Programming 78:109–129
  * Paulheim [2017] Paulheim H (2017) Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic Web 8(3):489–508
  * Pei et al [2019] Pei S, Yu L, Zhang X (2019) Improving cross-lingual entity alignment via optimal transport. IJCAI, pp 3231–3237
  * Pennington et al [2014] Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: EMNLP, pp 1532–1543
  * Pham et al [2021] Pham H, Xie Q, Dai Z, et al (2021) Meta pseudo labels. In: CVPR, pp 11557–11568
  * Raunak et al [2019] Raunak V, Gupta V, Metze F (2019) Effective dimensionality reduction for word embeddings. In: The 4th Workshop on Representation Learning for NLP, pp 235–243
  * Rizve et al [2021] Rizve MN, Duarte K, Rawat YS, et al (2021) In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. In: ICLR
  * Sajjadi et al [2016] Sajjadi M, Javanmardi M, Tasdizen T (2016) Mutual exclusivity loss for semi-supervised deep learning. In: ICIP, pp 1908–1912
  * Shi et al [2018] Shi W, Gong Y, Ding C, et al (2018) Transductive semi-supervised deep learning using min-max features. In: ECCV 2018, pp 311–327
  * Srivastava et al [2015] Srivastava RK, Greff K, Schmidhuber J (2015) Highway networks. In: ICML Workshop: Deep Learning
  * Suchanek et al [2007] Suchanek FM, Kasneci G, Weikum G (2007) Yago: a core of semantic knowledge. In: WWW, pp 697–706
  * Sun et al [2020] Sun K, Zhu Z, Lin Z (2020) Multi-stage self-supervised learning for graph convolutional networks. In: AAAI, pp 5892–5899
  * Sun et al [2017] Sun Z, Hu W, Li C (2017) Cross-lingual entity alignment via joint attribute-preserving embedding. In: ISWC, pp 628–644
  * Sun et al [2018] Sun Z, Hu W, Zhang Q, et al (2018) Bootstrapping entity alignment with knowledge graph embedding. In: IJCAI, pp 4396–4402
  * Tang et al [2023] Tang J, Zhang W, Li J, et al (2023) Robust attributed graph alignment via joint structure learning and optimal transport. arXiv preprint arXiv:230112721
  * Tarvainen and Valpola [2017] Tarvainen A, Valpola H (2017) Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In: NeurIPS, pp 1195–1204
  * Titouan et al [2019] Titouan V, Courty N, Tavenard R, et al (2019) Optimal transport for structured data with application on graphs. In: ICML, pp 6275–6284
  * Torres et al [2021] Torres LC, Pereira LM, Amini MH (2021) A survey on optimal transport for machine learning: Theory and applications. arXiv preprint arXiv:210601963
  * Villani [2009] Villani C (2009) Optimal transport: old and new, vol 338. Springer
  * Vrandečić and Krötzsch [2014] Vrandečić D, Krötzsch M (2014) Wikidata: a free collaborative knowledge base. Communications of the ACM 57(10):78–85
  * Wächter and Biegler [2006] Wächter A, Biegler LT (2006) On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical Programming 106:25–57
  * Wang et al [2014] Wang Z, Zhang J, Feng J, et al (2014) Knowledge graph embedding by translating on hyperplanes. In: AAAI, pp 1112–1119
  * Wang et al [2018] Wang Z, Lv Q, Lan X, et al (2018) Cross-lingual knowledge graph alignment via graph convolutional networks. In: EMNLP, pp 349–357
  * Wu et al [2019a] Wu Y, Liu X, Feng Y, et al (2019a) Relation-aware entity alignment for heterogeneous knowledge graphs. In: IJCAI, pp 5278–5284
  * Wu et al [2019b] Wu Y, Liu X, Feng Y, et al (2019b) Jointly learning entity and relation representations for entity alignment. In: EMNLP/IJCNLP, pp 240–249
  * Xu et al [2019a] Xu H, Luo D, Carin L (2019a) Scalable gromov-wasserstein learning for graph partitioning and matching. In: NeurIPS, pp 3046–3056
  * Xu et al [2019b] Xu H, Luo D, Zha H, et al (2019b) Gromov-wasserstein learning for graph matching and node embedding. In: ICML, pp 6932–6941
  * Yang et al [2019] Yang H, Zou Y, Shi P, et al (2019) Aligning cross-lingual entities with multi-aspect information. In: EMNLP/IJCNLP, pp 4430–4440
  * Yang et al [2023] Yang Y, Lv H, Chen N (2023) A survey on ensemble learning under the era of deep learning. Artificial Intellegence Review 56(6):5545–5589
  * Yang et al [2018] Yang Z, Qi P, Zhang S, et al (2018) Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In: EMNLP, pp 2369–2380
  * Zeng et al [2020] Zeng W, Zhao X, Tang J, et al (2020) Collective entity alignment via adaptive features. In: ICDE, IEEE, pp 1870–1873
  * Zhang et al [2021] Zhang B, Wang Y, Hou W, et al (2021) Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. In: NeurIPS, pp 18408–18419
  * Zhao et al [2020] Zhao X, Zeng W, Tang J, et al (2020) An experimental study of state-of-the-art entity alignment approaches. IEEE Transactions on Knowledge & Data Engineering (01):1–1
  * Zhu et al [2017] Zhu H, Xie R, Liu Z, et al (2017) Iterative entity alignment via knowledge embeddings. In: IJCAI, pp 4258–4264
  * Zhu et al [2021] Zhu Y, Liu H, Wu Z, et al (2021) Relation-aware neighborhood matching model for entity alignment. In: AAAI, pp 4749–4756