# APrompt4EM: Augmented Prompt Tuning for Generalized Entity Matching

Yikuan Xia  Key Laboratory of High Confidence Software Technologies, CS,
Peking University, Beijing, China<EMAIL_ADDRESS>,  Jiazun Chen  Key
Laboratory of High Confidence Software Technologies, CS, Peking University,
Beijing, China<EMAIL_ADDRESS>,  Xinchi Li  Big data and Artificial
Intelligence Institute of the China Telecom Research Institute, Beijing, China
<EMAIL_ADDRESS>and  Jun Gao  Key Laboratory of High Confidence Software
Technologies, CS, Peking University, Beijing, China<EMAIL_ADDRESS>
###### Abstract.

Generalized Entity Matching (GEM), which aims at judging whether two records
represented in different formats refer to the same real-world entity, is an
essential task in data management. The prompt tuning paradigm for pre-trained
language models (PLMs), including the recent PromptEM model, effectively
addresses the challenges of low-resource GEM in practical applications,
offering a robust solution when labeled data is scarce. However, existing
prompt tuning models for GEM face the challenges of prompt design and
information gap. This paper introduces an augmented prompt tuning framework
for the challenges, which consists of two main improvements. The first is an
augmented contextualized soft token-based prompt tuning method that extracts a
guiding soft token benefit for the PLMs’ prompt tuning, and the second is a
cost-effective information augmentation strategy leveraging large language
models (LLMs). Our approach performs well on the low-resource GEM challenges.
Extensive experiments show promising advancements of our basic model without
information augmentation over existing methods based on moderate-size PLMs
(average $5.24\%$+), and our model with information augmentation achieves
comparable performance compared with fine-tuned LLMs, using less than $14\%$
of the API fee.

##  1\. Introduction

With the growth of data volume and the increasing demand for high-quality
data, addressing the issue of Entity Matching (EM) has become an urgent need
in many applications, i.e., data integration (Doan et al., 2012) and cleaning
(Chu et al., 2016). The term EM refers to the problem of identifying whether
two data instances correspond to the same real-world entity. In the classical
setting of the EM problem, we aim to identify all pairs of entity records
across two sets that satisfy a matching condition. The well-studied standard
EM pipeline consists of two parts (Papadakis et al., 2020; Konda, 2018), the
blocking part and the matching part. The blocking part involves efficiently
selecting a few entity pairs worthy of comparison, and the matching part is
about a fine-grained comparison of the selected entity pairs. In this paper,
we focus on the matching part of the EM task.

Generalized Entity Matching (GEM) expands upon traditional EM by accommodating
a broader variety of data formats beyond just structured relational tables
(Wang et al., 2021). Conventional EM relies on identical or aligned schemas
and seeks similarity between pairs of identical attributes (Mudgal et al.,
2018). In contrast, GEM aims to handle a broader range of data types,
including relational, semi-structured, and textual data types. Such a
flexibility requirement makes GEM closer to real-world applications where
entities are represented in diverse formats. As EM is a special case of GEM,
and GEM can deal with a wider range of real-world tasks, we will deal with the
GEM problem in this paper. Besides, existing main-stream EM methods rely on
labeled training data heavily (Li et al., 2020; Mudgal et al., 2018), and
obtaining labeled data for GEM is often a challenge because manual labeling of
data is labor-intensive, time-consuming, and requires domain expertise, making
the process costly and slow, especially considering the diverse and dynamic
nature of data sources (Wang et al., 2012; Gokhale et al., 2014). How to build
a GEM system leveraging limited labels is a realistic challenge. So, in this
paper, we mainly focus on the problem of GEM in a low-resource setting.

The GEM problem, as a generalized version of the EM problem, not only inherits
its core challenges, the diverse semantic representations and the noisy data
(Barlaug and Gulla, 2021), but also introduces a novel difficulty: flexible
data format. These difficulties are further deepened, given the scarcity of
labeled resources. For instance, in Fig. 1, the terms ‘computer hardware’ and
‘IT equipment’ from two matching entities have the same semantic meaning
(semantic). The key name ‘Key value pairs’ or the review information are not
helpful in matching the entities (noisy). One source may present data in a
nested JSON format detailing product attributes, while the other could be mere
text describing product features (flexible). From a human’s perspective, the
critical information for discriminating between these two entities, the
product identifier (Mpn, Manufacturer Product Number) and the equipment
parameters (highlighted in different colors in Fig. 1), is either hidden in
the redundant data or missing. As illustrated by such cases, the key challenge
lies in how to exploit the useful semantic information from divergent data
representations.

Figure 1. Different data schemas in GEM and unifying them using natural
language texts. The entity presented by semi-structured and relational data is
a typical RAM product in 2022, while the entity presented in text is a typical
RAM product in 2014.

Tuning Pre-trained Language Models (PLMs) is the mainstream method of
addressing the semantic issue. Previous methods (Li et al., 2020) adopt the
fine-tuning training paradigms to adapt the pre-existing knowledge within PLMs
to the specific matching scenario. To address the low-resource label issue,
prompt-based methods, known for their resource efficiency, lead the
extensively trained PLMs to extract relevant insights with fewer labels (Liu
et al., 2023a). PromptEM (Wang et al., 2022b) is the representative method of
prompt tuning for EM, which uses structural serialization (Li et al., 2020)
and the P-tuning model (Liu et al., 2023b) as its prompt. Given the distinct
advantage of prompt-based methods under limited resource conditions, our
research pivots around this technique.

However, existing prompt strategies are faced with limitations:

Limitation I: Issues related to prompt design. The previous methods do not
adequately address the format flexibility and noisy data. As we have
mentioned, PromptEM’s prompt design consists of structured serialization and
P-tuning soft tokens (Wang et al., 2022b). The currently used prompt of
PromptEM poses two issues. Firstly, the PLM’s input of PromptEM, following the
footsteps of Ditto (Li et al., 2020), serializes entities using identifiers
like [COL] and [VAL]. This serialization does not align with the natural
language patterns encountered during PLMs’ pre-training, as structural or
semi-structural data is limited in PLMs’ training data (Trabelsi et al.,
2022). Secondly, in the GEM landscape, directly serializing the entities
results in mismatches in field types and data formats, which, combined with
the noisy data issue, can diffuse the model’s focus and lead to inferior
outcomes. We have to adjust the prompt to fit the distribution, which the LM
is familiar with. The P-tuning-style soft token method PromptEM uses searches
for better prompts for the task, which partly alleviates this issue. However,
the prompt template is unified for one dataset, and does not benefit each
entity on purpose.

Limitation II: Issues related to semantic information gap. The semantic issue
is not fully addressed, especially when language models (LMs) are tuned using
a few labels. The semantic relationship may be hard for LMs to capture, or the
key information is not provided in the entity. Though PLMs may contain
information in their training corpus, the prior semantic information of
moderate-size PLMs (i.e., BERT (Kenton and Toutanova, 2019), RoBERTa (Liu et
al., 2019), GPT (Radford et al., 2018, 2019)) may not be sufficient for
complex domain-specific EM (Ji et al., 2023). For example, in Fig. 1, we might
expect our model to tell the two products (presented in semi-structured,
textual format) through the information that the mainstream DRAM product in
2014 is not likely to have 64GB of storage. We find out that if we input ‘The
mainstream storage of a 2014 RAM is [MASK] GB.’ to a BERT model (Kenton and
Toutanova, 2019), the output with top-2 probability is 32 and 64, indicating
that the PLM is unaware of the mainstream RAM equipment parameters of each
year. This may deteriorate the matching performance. Conversely, a supplement
based on objective facts can relieve the problem.

In this paper, we introduce a prompt tuning framework, APrompt4EM, which makes
substantial improvements to address these two challenges. To address the first
limitation, we tailor a template generation method for the matching scenario,
converting structured or unstructured data into natural language patterns. The
natural language prompt is more aligned with the PLMs’ training context, which
boosts the model’s performance. Further, we augment the prompt with a soft
token mechanism. Contrary to the task-specific soft tokens in previous works
(Liu et al., 2023b), the contextualized soft token utilizes a neural network
to extract essential portions from the noisy data to form instance-specific
soft tokens, which guides the model’s focus and enables the model to converge
faster.

To address the second limitation, we emphasize the importance of information
augmentation. We propose augmenting sufficient information to bridge semantic
gaps, especially when the PLM’s intrinsic knowledge falls short. Since large
language models (LLMs) have shown commendable performance in understanding
common knowledge (Wei et al., 2022; OpenAI, 2023; Petroni et al., 2019), we
propose a method to gain insights from LLMs, augmenting the essential
information for EM. We also propose an uncertainty-based augmentation strategy
to reduce the token fees.

Our contributions are summarized as follows:

  * •

Augmented Contextualized Soft Token Based Prompts for Entity Matching. We
propose an augmented prompt tuning model for GEM, in which the natural
language templates and contextualized soft token module, that extract
essential information for each entity, can guide the model toward making
better decisions. Our prompt-tuning model based on the contextualized soft
token shows superior results (an average $5.24\%$\+ boost) compared with the
state-of-the-art methods.

  * •

LLM-supported Information Augmentaion. We propose augmenting the information
to enhance the model’s performance. We discuss augmenting information using
LLMs to enhance GEM. The cost of API calls is further reduced based on the
uncertainty estimation strategy. Our cost-effective information augmentation
method can help our base model achieve competitive results compared with a
fine-tuned LLM with less than $14\%$ cost.

Outline. We first introduce the APrompt4EM framework and its design choices in
Sec. 2. We then describe our soft-prompt model and information augmentation
strategy in detail in Sec. 3 and Sec. 4. Finally, we present the experimental
results in Sec. 5, the related works in Sec. 6, and the conclusions in Sec. 7.

Figure 2. Illustration of our APrompt4EM framework.

##  2\. APrompt4EM Framework

In this section, we first discuss the data input for the GEM task. Then we
discuss different design choices, and we present the overall framework of our
Prompt4EM in Fig. 2.

Data input. This paper, like other works (Wang et al., 2021), handles three
major input data types, including structured tables (relational tables), semi-
structured input (JSON format data), and unstructured input (textual data), in
a GEM problem.

We formally describe these data, as we need a unified way to handle them, like
fitting different inputs into a language model. For structured tables with
attribute keys $key_{1},....,key_{n}$, we denote one of its entities $e$ as
$e=[\\{key_{i},val^{e}_{i}\\},i=1,...,n]$. As for semi-structured input, we
consider a nested data structure $S$. To be specific, for an entity $e$, we
denote $e=S^{e}$ as:

$S^{e}=\left\\{\begin{aligned} &[\\{key^{e}_{i},S^{e}_{i}\\},&i=1,...,n]\\\
&[S^{e}_{i},&i=1,...,n]\end{aligned}\right.$

where $S^{e}_{i}$, as the value of a dictionary or the item of a list, is also
the defined nested data structure $S$. For unstructured input, we denote one
of its entities $e$ as $T^{e}$, representing its textual content.

Another perspective on the GEM input is the heterogeneity of the data sources,
which should be taken into account in module design. For homogeneous data
sources, the modules can be designed based on the granularity of the data
source. For heterogeneous data sources, the modules have to be designed at the
level of the data instance.

Design Choices. As we have stated in Sec. 1, we need to handle three main
challenges in GEM, including the semantic issue, the noisy data issue, and the
flexible format issue, in one unified method. We can utilize the advancement
of language models, extract key features, and convert different data to
natural language to handle these three challenges, respectively. Previous
works propose different ways of solving the semantic issue. So in the
following paragraph, we will make an analysis of our choices and conduct
experimental studies for further analysis.

To handle the semantic issues, Pre-trained Language Models (PLMs) are
introduced to understand the semantic meanings of the entities. There are two
kinds of methods to fine-tune PLMs in specific contexts. One is to learn
representation for each entity first and then capture the matching rule
(Reimers and Gurevych, 2019). The other is letting PLMs directly handle the
entity pair (Li et al., 2020). As the pairwise based method can learn the
interaction of features from both entities, it usually achieves better results
than the former one (Paganelli et al., 2022). So we take the pairwise approach
as the matching backbone. In addition, to solve GEM in low-resource settings,
we follow the prompt tuning paradigm to tune the PLM.

We have mentioned in Sec. 1 that the semantic issues are not fully addressed,
as in some cases, PLMs with moderate-size parameters (e.g., BERT) are
insufficient to capture all the semantic information provided in the entity
descriptions. One way to bridge the semantic gap is to use LLM as the backbone
of LM, and another way is to supplement the semantic information we need for
EM. We choose the second way, because although LLM can provide high quality
feedback, it is expensive to fine-tune and infer the model. So we take a more
economic and practical approach, which is mainly based on fine-tuning a
moderate-size PLM, but with enhanced entity information from an LLM.

Framework of APrompt4EM. APrompt4EM is based on prompt-tuning a PLM (RoBERTa
(Liu et al., 2019) in this paper) with the input of two entities in natural
language format, as illustrated in Fig. 2 (upper part). The two entities are
serialized into a natural language, and they are concatenated and sent to the
PLM as a prompt. We obtain the value of [MASK] in the middle of two entities,
from which the match or mismatch between two entities is derived.

There are two key components in APrompt4EM. As there are noises in the
original form and derived form, we introduce the contextualized soft token
module (lower-right part in Fig. 2) to extract the key features of the entity,
through which the impacts of noise data can be alleviated. The soft tokens are
generated with the learned parameters for each entity. Another component, the
information augmention module (lower-left part in Fig. 2), enriches the
features of a given entity from a LLM (ChatGPT in our case). We can see that
some attributes in yellow can be obtained from LLM, which are missing in the
original input.

##  3\. Prompt Tuning for GEM

In this section, we describe our soft-prompt model design in detail. We first
introduce our natural language-based prompt serialization, which serves as a
basic discrete prompt. Then, we propose our contextualized soft token prompt
model, which is a continuous prompt model. Finally, we describe the training
details of the prompt-tuning model.

###  3.1. Natural Language Prompt Templates

Prompt Template. Solving the GEM problem under the prompt-based tuning
paradigm requires converting the entities to a token sequence using the prompt
template. The entities are usually first serialized to a language sequence,
and the serialized sequences are then fit into the prompt template for further
computation. We follow the prompt template proposed in PromptEM (Wang et al.,
2022b). Given two entities $e_{a},e_{b}$, the prompt funtion $f_{pt}$ to be
filled is defined as:

(1) $f_{pt}(e_{a},e_{b})=serialize(e_{a})\ is\ [MASK]\ to\ serialize(e_{b}).$

Previous Structured Entity Serialization. The serialization function
conventions established in Ditto (Li et al., 2020) can be described as
follows. Ditto serializes entities with special tokens. For an entity $e$ as
$e=[\\{key_{i},val^{e}_{i}\\},i=1,...,n]$ in a structured table, such entity
can be serialized as:

$serialize(e)=[COL]\ key_{1}\ [VAL]\ val^{e}_{1}\ ...[COL]\ key_{n}\ [VAL]\
val^{e}_{n}.$

The $[COL]$ and $[VAL]$ tokens are special tokens to help the model identify
the key and value names of the attributes.

As for semi-structured tables, PromptEM extended the serialization method to
semi-structured tables to deal with different input schemas. For nested
attributes in the semi-structured tables, the $[COL]$ and $[VAL]$ tokens are
recursively added in each nested level, for a nested entity
$S^{e}=[\\{key_{i}^{s},S_{i}^{e}\\},i=1,..,n]$:

$sl(S^{e})=[COL]\ key_{1}^{s}\ [VAL]\ sl(S_{1}^{e})\ ...\ [COL]\ key_{n}^{s}\
[VAL]\ sl(S_{n}^{e}),$

where $sl$ is the abbreviation of $serialize$. For list objects, the list
elements are concatenated into one string to present in the template.

Entity Serialization using Natural Language. Previous works have shown that
the prompt templates closer to the LM’s data distributions can help the LM
retrieve its known knowledge (Jiang et al., 2020). Although the serialization
of Ditto enables the template text to encode the structure information, the
use of $[COL]$ and $[VAL]$ tokens is clearly out of the natural language
distribution. Therefore, we propose using natural language prompts to help the
LM extract information better.

For the manual design of the natural language prompt, the basic natural
language serializing module for $e=[\\{key_{i},val^{e}_{i}\\},i=1,...,n]$ is:

$serialize(e)=the\ key_{1}\ is\ val^{e}_{1},...,the\ key_{n}\ is\ val^{e}_{n}$

Similar to the processing method of nested attributes in PromptEM, we
transform the nested structures into text by recursively applying the basic
module. For a nested entity $S^{e}=[\\{key_{i}^{e},S_{i}^{e}\\},i=1,..,n]$, we
serialize one of its nested attribute value pairs as:

$serialize(e)=the\ key_{1}^{e}\ is\ sl(S_{1}^{e}),...,the\ key_{n}^{e}\ is\
sl(S_{n}^{e}),$

where $sl$ is the abbreviation of $serialize$.

Apart from the basic rules, we refer to ChatGPT’s concatenation results for
entities, and we design different prompts based on various attribute
combinations. We use the following prompt to design prompts from ChatGPT:

Concatenate the attributes to form a natural language prompt template for
‘type’ entities. Use a declarative sentence in the simple present tense.
Example: Attribute: ‘title’, ‘brand’, Prompt Design: This ‘title’ is from
‘brand’.
Please design the prompt template: Attribute: {attribute_list}.

where ‘type’ represents the dataset scenario, e.g., book/ music.

Table 1. Discrete Natural Language Prompt Template Design.

Entity Schema | Manual Prompt | Paraphrased Prompt
---|---|---
{‘title’, ‘author’, ‘venue’, ‘year’} | The ‘title’ is authored by ‘author’ and is published in ‘venue’ in the year ‘year’. | The paper entitled ‘title’ is written by ‘author’ and is published in ‘venue’ in ‘year’.
{‘title’, ‘director’, ‘actors’, ‘year’, ‘rating’, ‘information’} |  | The ‘title’ is directed by ‘director’, including ‘actors’.It was released in ‘year’
---
and has received ratings of ‘ratings’. It includes sl(‘information’).
| The ‘title’ is made by ‘director’, featuring ‘actors’. It debuted in ‘year’
---
and has earned ratings of ‘ratings’. It encompasses sl(‘information’).
{‘category’, ‘brand’, ‘identifiers’, ‘keyvaluepairs’, ‘price’} |  | This ‘category’ product is from ‘brand’ priced at ‘price’. It is identified by ‘identifiers’
---
and has key value pairs of sl(‘keyvaluepairs’).
| This ‘category’ product is priced by ‘brand’ ‘price’, it is identified by
‘identifiers’
---
and has key value pairs of sl(‘keyvaluepairs’)
| {‘name’, ‘postalcode’, ‘latitude’, ‘longitude’,
---
‘position’, ‘address’}
| The ‘name’ is located at ‘address’, with latitude ‘latitude’ and longitude
‘longitude’.
---
The position is ‘position’, and the postal code is ‘postalcode’
| The ‘name’ is located in the ‘Address’ with the width ‘Latitude’ and the
length ‘Langitude’,
---
the position is ‘position’ and the postcode is ‘postalcode’.
{‘title’, ‘brand’, ‘price’, ‘pricecurrency’, ‘description’} | The ‘title’ from ‘brand’. It is priced at ‘price’ ‘pricecurrency’. It includes ‘description’. | The ‘title’ from ‘brand’, costs ‘price’ ‘pricecurrency’. It includes ‘description’.
{‘title’, ‘price’, ‘manufacturter’} | The ‘title’ is a product manufactured by ‘manufacturter’ and is priced at ‘price’. | The ‘title’ is a product produced by ‘manufacturer’ and valued at ‘price’.
{‘title’, ‘address’, ‘phone’, ‘category’} | The ‘title’ is a ‘category’ restaurant located at ‘address’. The phone number is ‘phone’. | The ‘title’ is a ‘category’ restaurant at the ‘address’, the telephone number is ‘phone’.
{‘addr’, ‘city’, ‘phone’, ‘type’, ‘class’} |  | This ‘type’ restaurant offers a diverse ‘class’ different types of dishes, located in ‘city’ at ‘addr’.
---
The phone number is ‘phone’.
| This ‘type’ restaurant offers a diverse ‘class’ different types of dishes,
located in ‘city’ at ‘addr’.
---
The telephone number is ‘phone’.
| {‘song_name’, ‘artist_name’, ‘album_name’, ‘genre’,
---
‘price’,‘copyright’,‘time’,‘released’}
| The ‘song_name’ is performed by ‘artist_name’ and is featured on the album
‘album_name’.
---
It falls under the genre of ‘genre’ with a price of ‘price’. The song is
protected by ‘copyright’
and has a duration of ‘time’. It was released on ‘released’.
| The ‘song_name’ is played by ‘artist_name’ and is included on the album
‘album_name’,
---
which falls under the genre of ‘genre’ with a price of ‘price’. The song is
protected by ‘copyright’
and has a length of ‘time’. It was released on ‘released’.
{‘title’, ‘category’, ‘brand’, ‘modelno’, ‘price’} | The ‘title’ is a ‘category’ from ‘brand’ with model number ‘modelno’, priced at ‘price’. | The ‘title’ is a ‘category’ from ‘brand’ with model number ‘modelno’, priced at ‘price’.

Prompt Template Paraphrasing. To generate prompt templates closer to the LM’s
language distribution, we also adopt the paraphrasing approaches proposed in
(Jiang et al., 2020) to mine better prompt templates. We use a T5-based
(Raffel et al., 2020) pre-trained neural translation machine $f_{NMT}$ to
first translate manually designed prompts $p$ to another target language
(i.e., German) prompt $p_{t}$ and back translate it to an English prompt
$p^{\prime}$. To be specific, for each entity in the training set (of size
$n$), we use the beam search strategy to preserve $k_{b}$ translated $p_{t}$
and $k_{b}^{2}$ back-translated result prompts (e.g., $k_{b}=3$). We can parse
the entity values out of all the $n*k_{b}^{2}$ translated result prompts, and
keep all the prompt templates if they are successfully parsed. The prompt
template with the highest sum of generation probability
$p_{NMT}(p_{t}|p)p_{NMT}(p^{\prime}|p_{t})$ ($p_{NMT}$ is the generation
probability output by $f_{NMT}$) is selected as the paraphrased prompt
template. The manual and paraphrased prompt templates are presented in Tab. 1.

###  3.2. Contextualized Soft Token Model

Generally, our contextualized soft token model belongs to the continuous
prompt methods (Liu et al., 2023a). Recall that one of the difficulties in the
GEM problem lies in the noisy data, which may interfere with the PLM to
produce poor results. We need to use suitable prompts to extract critical
information from the entities or guide the LM to distinguish which part of the
information is crucial for matching the two entities. In most previous
continuous soft token approaches (Wang et al., 2022b; Liu et al., 2023b), the
soft tokens are tuned for a task instead of each data instance, which is
insufficient for handling the aforementioned two features.

Therefore, we propose the contextualized soft token model that extracts
information from each entity, which is provided to the LM to relieve such an
issue. Intuitively, comparing different critical aspects of entities can help
boost performance. For example, the relatively important information in a
product entity includes the title, product identifier, and some key device
parameters. We expect our soft token to learn to extract these features, which
can be directly presented to the LM.

In detail, we extend the previous serializing method in Sec. 3.1 with the soft
tokens as follows:

(2) $\text{serialize\\_soft}(e)=\text{serialize}(e)\ \text{the keyword is
}[S_{1}]\dots[S_{K}],$

where $[S_{1}]\dots[S_{K}]$ represents the soft tokens, and $K$ is the hyper-
parameter that controls the number of tokens added. Each of the $K$ different
tokens is expected to represent a different aspect extracted from the
entities. We will present how to compute the embeddings of these $K$ tokens
following.

We first encode the input entity using the standard self-attention transformer
module (Vaswani et al., 2017). Suppose that for an entity $e$,
$\\{t_{1},...,t_{l}\\}$ are the tokens output by the tokenizer of the text
sequence $serialize(e)$. We follow the standard transformer processing
pipeline (token embedding plus position embedding input) and use $N$ self-
attention transformer layers to encode $\\{t_{1},...,t_{l}\\}$ to $E_{1:l}$,
where $E_{1:l}$ represents the output of embeddings after $N$ transformer
layers.

To extract $K$ aspects from a given entity, we aim to fetch the most
semantically relevant tokens for the $K$ different aspects from the entity and
aggregate them. So the aggregated embeddings are the semantic centers we seek
to learn. Specifically, we use an attention module to extract different
aspects of information from the given entity. We propose
$A_{1:K}\in\mathcal{R}^{K\times d_{q}}$ as $K$ learnable aspect parameters,
where $d_{q}=768$ is this parameters’ length. After transforming $E_{1:l}$ to
the key matrix $M_{K}\in\mathcal{R}^{l\times d_{q}}$ and the value matrix
$M_{V}\in\mathcal{R}^{l\times d_{v}}$ ($d_{v}=768$ is the value matrix’s
length) using two linear layers, we conduct the scaled dot-product attention
computation. The aggregated features, according to the attention matrix, are:

(3) $Emb_{[S]}=\text{softmax}(\frac{A_{1:K}M_{K}^{T}}{\sqrt{d_{q}}})M_{V}.$

After the attention layer, the soft token embedding undergoes post-processing,
which includes linear transformations and normalization. The whole pipeline of
our contextualized soft token model is presented in the lower-right part of
Fig. 2.

###  3.3. Training Part of Prompt Tuning

During training, we choose the Prompt+LM tuning paradigm, tuning both the LM
and prompt parameters, which fully activates the model capabilities (Liu et
al., 2023a). The LM part is tuned because the main LM module requires
sufficient training to handle the GEM task (Paganelli et al., 2022). The
prompt part is definitely tuned because a learnable soft token model is
involved. Though the cost of Prompt+LM tuning is usually considered high, the
cost can be neglected due to limited training samples, as we are dealing with
the low-resource GEM task.

We follow the label word selection in PromptEM. The conditional probability of
an entity pair and its label is defined as the sum of the normalized scores in
the corresponding label set with {matched, similar, relevant} as the word set
for the matching label, and {mismatched, different, irrelevant} for the non-
matching label.

We propose an orthogonal regularization loss to guide the model in learning
different soft embeddings, as we expect our learned parameters to represent
diverse key aspects of the given entity. The orthogonal loss is computed as:

(4) $\mathcal{L}_{ortho}=||Emb([S])Emb([S])^{T}-I||/d_{s},$

where $S\in\mathcal{R}^{K\times d_{s}}$ and $I$ is the identity matrix.

The orthogonal loss is combined with the normal cross entropy classification
loss $\mathcal{L}_{CE}$ to be our prompt tuning loss:

(5) $\mathcal{L}=\mathcal{L}_{CE}+\lambda\mathcal{L}_{ortho},$

in which $\lambda$ is a balance hyper-parameter. The choice of $\lambda$ is
further discussed in Sec. 5.

Figure 3. Example of different data augmentation operators. The shuffle
operator from Ditto doesn’t provide additional information, while the
augmentation from GPT 3.5 can supplement new information (red part), and
extract information in structured forms from the description (blue part).

##  4\. Information Augmentation for GEM

In this section, we first introduce the concept of ‘information augmentation’,
offering supplementary information, which can serve as a powerful data
augmentation operator. Subsequently, we propose using LLMs to implement
information augmentation. Lastly, we demonstrate the cost-effectiveness of
this module and propose our uncertainty-based augmentation strategy to further
reduce the LLM’s token fee.

###  4.1. Information Augmentation as a Powerful Data Augmentation Operator

Many data augmentation operators have been proposed to help boost EM’s
performance and stability. The data augmentation methods can help models learn
invariant transformations, guide models to neglect data noises, and serve as a
regularization method to prevent the models from over-fitting (Li et al.,
2021; Feng et al., 2021). For instance, Ditto proposes several data
augmentation operators for EM (e.g., del, shuffle, swap, and the shuffle
operator is presented in Fig. 3) and interpolates the augmented data with the
original data for training. Rotom (Miao et al., 2021) introduces a pre-trained
(on rule-based augmented data) seq2seq model to perform personalized data
augmentation. Sudowoodo (Wang et al., 2023) and SupCon-EM (Peeters and Bizer,
2022) perform data augmentation to provide training samples for self-
contrastive learning. These data augmentation approaches provide rule-based
invariant knowledge and other benefits. However, they typically bring no
entity-aware information as no extra information source is introduced.

Motivated by a series of retrieval-based LM approaches (e.g., KNN-prompt (Shi
et al., 2022), REPULG (Shi et al., 2023)), which have shown that relevant
information retrieved from proper data sources can boost the LM’s few-shot and
zero-shot capabilities dramatically (Asai et al., 2023), we propose the
information augmentation operator. Information augmentation, which provides
additional information for an entity, serves as a new data augmentation
operator. It’s a powerful data augmentation operator, as it introduces
additional information.

There can be various approaches to augmenting information, e.g., knowledge
graphs or open internet data sources. For example, KaER (Fang et al., 2023)
and KnowPrompt (Chen et al., 2022) retrieve information from knowledge graphs
and integrate this information into the language model. However, knowledge
graphs may suffer from issues of data currency and relevance, as rapidly
changing information may not be constantly updated. Besides, locating specific
entities within a knowledge graph is challenging. For example, the digital
product entity in Fig. 3 is not likely to appear in a knowledge graph.
Retrieving from open internet sources also suffers from high costs (API fees
or manual effort) and legal risks (Yu et al., 2020).

After PLMs are proposed, multiple studies have shown the capability of LMs as
knowledge bases (Petroni et al., 2019; AlKhamissi et al., 2022). Recently, the
introduction of LLMs (OpenAI, 2023) has elevated the performance of LM’s fact
probing capabilities to unprecedented levels. Though the augmentation of
entities, such as songs, academic papers, and products, belongs to the long-
tail distribution, which LMs are not familiar with, the large training corpus
for LLMs relieves such an issue. For long-tail distributed data that LLMs have
not been trained on, LLMs can also make guesses based on common sense (OpenAI,
2023). Additionally, LLMs can extract useful information from the entity
descriptions in a well-structured and brief format, which is more friendly to
our prompt tuning framework.

###  4.2. LLM-enhanced Information Augmentation

As we have mentioned above, since LLMs can augment information in structured
forms, we should first determine which attributes are needed for EM. Then, we
should design a prompt used for LLM queries and consider how to integrate new
information into prompt tuning.

Attribute Selection. Considering the heterogeneity of different data sources,
we have to determine the {attribute_list} for each dataset uniquely, where
{attribute_list} is a list of attribute names we desire for a specific entity.
We propose two different ways of determining the attribute keys of an entity.

I. Augmentation at the granularity of the data source. For a highly
homogeneous dataset, we can manually design the attributes that are essential
for the alignment of two entities within it. Advice from ChatGPT can also be
taken into account. We can query ChatGPT for the attributes needed in the WDC
dataset:

List the features important for entity matching in a digital product dataset.

Here, the {attribute_list} for the WDC dataset can be { capacity, color,
frequency, keywords, language, model number, product identifier, release year,
resolution, size, speed, weight }. The important attributes in the list can be
either extracted or recalled by LLMs.

II. Augmentation at the granularity of the data instance. For a heterogeneous
dataset, such as an e-commerce dataset with both digital products and daily
essentials, using a uniform attribute query list is not appropriate.
Therefore, the desired augment attributes should be determined by the entities
to be matched for a given entity. As mentioned in Sec. 1, a blocking pipeline
is usually required before establishing a matching system for entity pairs.
Assume that after the blocking system, the entities to be matched with entity
$e$ are $e_{1},...,e_{n}$. For each $e_{i}$, $KEY_{i}$ is the set of attribute
keys $e_{i}$ has. (For nested semi-structured entities, attribute keys at
every level should be considered.). The query attribute set for entity $e$ can
be defined as: $attribute\\_list_{e}=\bigcup_{i=1}^{n}KEY_{i}$, so that the
key attributes of other entities, that should be compared with entity $e$, can
be collected.

Prompt Design. We use the following prompt to augment structured information
from the LLM:

[{"role": "system", "content": "You are a helpful assistant. Answer in plain
json format only"},
{"role": "user","content": "Please provide some information about the
following entity. The entity is {entity_info}. Please output the
{attribute_list} of the entity.")}]

where {entity_info} is the entity data presented in text (semi-structured data
transformed in JSON format).

Integration with Prompt Tuning. As shown in Fig. 2, the augmented information
in JSON format is serialized as described in Sec. 3.1, and the serialized
sequence is concatenated with the original serialized sequence for the
following computation. It is expected that specific desired attributes in
{attribute_list}do not exist for an entity, so we use regular expressions to
match meaningless augmented information generated by LLMs. The value terms are
then substituted by <pad> in the serialization.

Cost for Augmenting using LLMs. The API fee of original/fine-tuned
GPT3.5-turbo inference is $0.002, 0.016/1k tokens, respectively, in 10/2023
111https://openai.com/pricing, which is still expensive if we aim to match a
large number of entities. So it’s necessary to consider the cost of using
LLMs.

Suppose that we are performing GEM between two sources of $N$ entities, and
each entity has $N_{k}$ pairs of key and value, which consist of $L$ tokens on
average. After the blocking system, each entity from one source has an average
of $B$ counterparts to compare with. If we perform information augmentation on
the $2*N$ entities, we require an input of all the $2N$ entities, which
requires $2*N(N_{k}*L)$ tokens. We further receive an output of the augmented
attributes. If each entity has an $\\{attribute\\_list\\}$ with an average
length $O_{k}$, we receive a total output of $2*N(O_{k}*L)$ tokens.

Since directly using LLMs is a way to bridge the semantic gap, we believe it’s
necessary to compare it with our information augmentation strategy. There are
generally two ways to use these foundation models (Narayan et al., 2022;
Peeters and Bizer, 2023). One is inference in a zero/few-shot (in-context
learning) fashion; the other is traditional fine-tuning. Augmenting
information from LLMs can help boost our matching model’s performance to be
comparable to LLMs’ performance, which will be presented in the Sec. 5. So,
considering that the performance of the models is similar, the cost issue
becomes more critical.

Augmenting information using LLMs generally costs less than directly using
LLMs. A straightforward explanation for the cost-saving of LLM tokens when
adding supplementary information is that during supplementation, all entities
are only traversed once, while directly using LLM for comparison requires
multiple traversals of all entities because a single entity may have multiple
corresponding entities to be judged against. Here, directly using LLMs means
inference from a pre-provided/fine-tuned LLM model, which is the major cost of
using LLMs, given that we are solving low-resource GEM (small training
samples). Besides, inference is indeed a recurring expense. Direct inference
from an LLM approximately requires an input of all the entity pairs, which
consists of $N*B*(2*(N_{k}*L))$ tokens. In comparison, we subtract the two
costs, and the difference is therefore:

$d=2*N*L(B*N_{k}-N_{k}-O_{k}).$

In practice, $B*N_{k}$ is usually higher than $O_{k}$, which makes information
augmentation more cost-effective. Suppose that the hyper-parameter $B$ of the
blocking system is $5$, and the basic attribute number and augmented attribute
number are the same, augmenting information using LLMs can save $60\%$ of the
tokens used by direct inference.

###  4.3. Uncertainty-based Cost-effective Information Augmentation Strategy

In this subsection, we propose an uncertainty-based information augmentation
strategy that further reduces the API fee. It is a common observation that for
some entities, entities from another source are easy to distinguish from,
while for other entities, the matching is quite confusing. This outcome may
result from the dataset’s characteristics, or it could be caused by the
information loss of the data instance. Leveraging this feature, we propose
neglecting the entities that are certain in matching without the information
augmentation procedure to prevent unnecessary API token costs.

During training, we augment all training instances to ensure that the matching
model has the best performance, as the fees can be neglected due to the low-
resource settings. During inference, we augment some of the test instances
according to the uncertainty score. For the entity not augmented, we pad all
the attribute values we query to <pad>. We select instances with similar
classification scores, as they are confusing to the current model. Following
the notations in Sec. 4.2, we define the uncertainty score of an entity $e$:

(6) $Score(e)=max_{e_{i}\in\\{e_{1},...,e_{n}\\}}H(P(y|(e,e_{i}))),$

where $\\{e_{1},...,e_{n}\\}$ are the ones that the blocking module has
selected as potential matches for entity $e$, and $H$ denotes the negative
entropy calculation or the uncertainty metric
$max(P(y=0|(e,e_{i})),P(y=1|(e,e_{i}))$. The higher the score an entity has,
the more confident the current model is about this entity. Then, only the
entities with scores lower than $\tau$ are augmented using LLMs, in which
$\tau$ represents an uncertainty threshold.

Though augmenting only part of the test data may damage the consistent data
distribution between the training set and test set, we find through experiment
that the strategy has little impact on model performance and saves many API
tokens. This is because we convert all entities to natural language,
especially using reverse translation techniques, ensuring that the data
distribution between augmented and original data instances remains similar.

Table 2. Dataset Statistics: The schemas with underlines are relatively long textual attributes, the schemas in braces {} are attributes in iterative dict format, and the schemas in square brackets [] are attributes in list format. The left and right schemas are separated by a vertical bar |. The left and right sources share the same schema if | is missing.  Dataset | #Left | #Right | #Labels | Domain | Data Schema
---|---|---|---|---|---
REL-TEXT | 2,616 | 2,295 | 7,417 | Academic |  abstract | title, authors, venue, year
SEMI-HETER | 22,133 | 23,264 | 1,240 | Book |  | {publication info} | title, authors, {publication info}
---
GEO-HETER | 2,469 | 2,788 | 2,500 | Location |  | name, latitude, longtitude, address, postal code
---
| name, position, address, postal code
SEMI-HOMO | 2,616 | 64,263 | 17,223 | Academic | title, [authors], venue, year
SEMI-TEXT-W(S-T-W) | 9,234 | 9,234 | 5,540 | Product |  | category, identifiers, brand, {key value pairs}, price | info
---
SEMI-TEXT-C(S-T-C) | 20,897 | 20,897 | 12,538 | Product |  | category, identifiers, brand, {key value pairs}, price | info
---
SEMI-REL | 29,180 | 32,823 | 1,309 | Movie |  | title, director, [actors], year, ratings, information | {movie info}
---
REL-HETER | 533 | 331 | 567 | Restaurant |  | title, address, phone, category | address, city, phone, type, class
---
WDC | 1,215 | 1,220 | 2,500 | Product | brand, title, description, price, price currency
GOOGLE-AMAZON | 1,363 | 3,226 | 9,167 | Product | title, manufacturer, price
WALMART-AMAZON | 1,688 | 5,247 | 6144 | Product | category, brand, model number, price
ITUNES-AMAZON | 262 | 436 | 321 | Music |  | song name, artist, album, genre, price, copyright,time, released date
---

##  5\. Experiment

In this section, we conduct extensive experiments on real-world datasets to
evaluate our proposed prompt tuning method and information augmentation
module. Our objective is to address the following research inquiries through
our experiments:

  * •

I1: How does our prompt tuning method perform compared with other state-of-
the-art EM methods in the low-resource GEM task? How does each module affect
the model’s performance?

  * •

I2: From which perspective does the contextualized soft token benefit GEM?

  * •

I3: How does our information augmentation module perform effectively and
economically compared with directly using LLMs? How does this module help GEM?

###  5.1. Experimental Setup

Datasets. We use a total of twelve real-world datasets to conduct our
experiment. The dataset statistics are presented in Tab. 2. The first nine
datasets are GEM tasks with complex and different data schemas, and the rest
are three EM tasks with identical data schemas. For the GEM datasets, we use
seven real-world GEM benchmarks from Machamp (Wang et al., 2021) and GEO-HETER
from PromptEM (Wang et al., 2022b). Further, we include the WDC-product
dataset (Peeters et al., 2024) newly collected in 12/2022, which makes it more
likely to escape the LLM’s training data. We use the hardest test cases of
WDC, which is the test dataset with ‘$80\%$ corner cases’ (hard positives and
negatives) and all unseen cases (entities not appearing in training cases).
For the traditional EM dataset, we select three datasets from the ER-Magellan
dataset (Köpcke et al., 2010), GOOGLE-AMAZON, WALMART-AMAZON, and ITUNES-
AMAZON datasets to show the boost in our performance on the traditional EM
tasks.

Baselines. We compare APrompt4EM with several representative EM methods,
including state-of-the-art methods in low-resource GEM settings. The baselines
compared are listed as follows:

  * •

RoBERTa. RoBERTa represents fine-tuning a RoBERTa (Liu et al., 2019) PLM to
perform EM.

  * •

SentenceBert in EMNLP’ 19 (Reimers and Gurevych, 2019). SentenceBert learns
sentence embeddings using siamese pre-trained LMs to perform EM.

  * •

Ditto in VLDB’ 21 (Li et al., 2020). Ditto is a fine-tuning paradigm EM method
that is improved by injecting domain knowledge, summarizing, and information
augmentation.

  * •

Sudowoodo in ICDE’ 23 (Wang et al., 2023). Sudowoodo is a self-contrastive
learning framework that can learn representations for EM.

  * •

Rotom in SIGMOD’ 21 (Miao et al., 2021). Rotom is a meta-learned seq2seq data
augmentation framework for data processing tasks including EM.

  * •

Machop in aiDM’ 22 (Wang et al., 2022a). Machop is an EM framework with a
structure aware pooling layer that compares each attribute pair.

  * •

PromptEM in VLDB’ 22 (Wang et al., 2022b). PromptEM is the state-of-the-art
low-resource GEM solution, which is based on prompt tuning, enhanced by
pseudo-labels and self-training. We mainly compare its prompt tuning module.

  * •

GPTx. GPTx represents fine-tuning the OPENAI proposed LLM on the GEM task.

Implementation Details. All the backbone LMs of our method and the baselines
are selected as RoBERTa. We implement our model based on the PromptEM source
codes222https://github.com/ZJU-DAILY/PromptEM, Pytorch (Paszke et al., 2019),
the Transformers library (Wolf et al., 2019), and the OpenPrompt Library (Ding
et al., 2022).

The low-resource settings refer to using $10\%$ of the datasets’ labeled data.
We enumerate through the hyper-parameters and report the best results. The
soft token number $K$ is selected from $\\{1,2,4,8\\}$, the number of
transformer encoder layers in the contextualized soft token is searched in
$0-2$, and the hyperparameter $\lambda$ of the orthogonal loss is set to $1$.
The mode of position embeddings in the contextualized soft token is selected
from {COL, POS, w/o }, in which COL means using different position embeddings
for different columns (Nassar et al., 2022), PE means using original position
embeddings, and w/o means not using position embeddings. Following the
training settings of PromptEM, we train our model using the AdamW optimizer
(Loshchilov and Hutter, 2019). The number of training epochs is set to $30$,
the learning rate is set to $2e-5$, and the batch size is set to $24$. The
prompt template we used is selected from Tab. 1. The experiments are conducted
on an Ubuntu 20.04.6 LTS Linux server with an Intel Xeon Silver 4210R CPU,
$256$ GB of RAM, and an NVIDIA RTX 3090 graphics card. The RoBERTa,
SentenceBert, Ditto, Sudowoodo, and Rotom methods are implemented through
their open-source codes333https://github.com/megagonlabs, and the hyper-
parameters are set to the recommended ones. We implemented our own Machop
baseline according to the model structures and hyper-parameters described in
the original paper. The fine-tuning of GPTx models is implemented through the
OPENAI official API using the default hyper-parameters.

Table 3. Main results of comparing methods in the low-resource GEM setting.

| REL-TEXT | SEMI-HETER | GEO-HETER | SEMI-HOMO | S-T-W | S-T-C | SEMI-REL | REL-HETER | WDC | G-A | W-A | I-A
---|---|---|---|---|---|---|---|---|---|---|---|---
Methods |  P |  R |  F |  P |  R |  F |  P |  R |  F |  P |  R |  F |  P |  R |  F |  P |  R |  F |  P |  R |  F |  P |  R |  F |  P |  R |  F |  P |  R |  F |  P |  R |  F |  P |  R |  F
RoBERTa |  19.8 |  35.1 |  25.3 |  25.3 |  40.9 |  31.3 |  83.9 |  83.3 |  83.6 |  94.5 |  93.0 |  93.7 |  30.0 |  1.4 |  2.7 |  76.6 |  71.9 |  74.2 |  43.3 |  95.1 |  59.5 |  100 |  95.5 |  97.7 |  20.9 |  12.6 |  15.7 |  59.0 |  59.0 |  59.0 |  56.2 |  65.8 |  60.6 |  75.9 |  81.5 |  78.6
SentenceBert |  44.0 |  49.5 |  46.8 |  70.2 |  75.5 |  72.7 |  64.0 |  81.5 |  71.7 |  91.8 |  90.4 |  91.1 |  24.5 |  12.8 |  16.8 |  69.7 |  54.4 |  61.1 |  76.8 |  86.9 |  81.5 |  100 |  90.0 |  94.7 |  25.7 |  55.0 |  35.1 |  37.5 |  64.1 |  47.3 |  26.9 |  47.2 |  34.3 |  76.7 |  85.2 |  80.7
Ditto |  37.6 |  51.4 |  43.4 |  92.6 |  62.9 |  74.9 |  78.3 |  86.0 |  82.0 |  91.9 |  95.0 |  93.4 |  29.5 |  31.3 |  30.3 |  56.8 |  47.1 |  51.5 |  86.8 |  97.3 |  91.8 |  100 |  86.4 |  92.7 |  11.7 |  87.8 |  20.7 |  50.3 |  73.1 |  59.6 |  58.0 |  54.4 |  56.1 |  71.4 |  74.1 |  72.7
Sudowoodo |  38.0 |  55.9 |  45.3 |  68.0 |  63.2 |  65.5 |  84.2 |  89.2 |  86.7 |  92.6 |  92.5 |  92.6 |  22.0 |  31.8 |  26.0 |  67.4 |  67.0 |  67.2 |  83.3 |  95.6 |  89.1 |  96.0 |  97.2 |  96.6 |  47.2 |  60.2 |  52.9 |  50.4 |  58.5 |  54.2 |  65.3 |  65.3 |  65.3 |  91.2 |  93.6 |  92.3
Rotom |  61.5 |  50.3 |  55.4 |  70.2 |  45.9 |  55.5 |  72.2 |  56.5 |  63.4 |  86.1 |  91.7 |  88.8 |  21.9 |  20.9 |  21.4 |  86.1 |  59.8 |  70.5 |  74.0 |  93.4 |  82.6 |  98.6 |  100 |  99.3 |  17.8 |  14.0 |  15.7 |  70.2 |  45.9 |  55.5 |  71.7 |  62.3 |  66.7 |  92.1 |  89.7 |  90.9
Machop |  18.0 |  99.2 |  30.5 |  73.6 |  80.5 |  76.9 |  75.6 |  88.1 |  83.1 |  92.7 |  93.2 |  93.0 |  11.6 |  99.1 |  20.7 |  18.6 |  48.2 |  26.8 |  90.4 |  97.8 |  94.0 |  100 |  86.4 |  92.7 |  22.2 |  25.4 |  23.7 |  55.6 |  74.8 |  63.8 |  68.2 |  68.9 |  68.6 |  84.6 |  40.7 |  55.0
PromptEM |  54.9 |  62.8 |  58.6 |  94.0 |  39.6 |  55.8 |  82.9 |  86.5 |  84.7 |  94.2 |  95.0 |  94.6 |  32.9 |  22.8 |  26.9 |  86.2 |  73.4 |  79.3 |  94.5 |  94.5 |  94.5 |  100 |  100 |  100 |  19.9 |  44.4 |  27.5 |  65.6 |  60.3 |  62.8 |  81.3 |  70.0 |  75.2 |  100 |  92.6 |  96.2
APrompt4EM |  66.5 |  63.1 |  64.7 |  81.6 |  78.0 |  79.7 |  83.7 |  90.2 |  86.8 |  94.2 |  96.5 |  95.3 |  76.0 |  52.6 |  62.2 |  84.9 |  73.1 |  78.6 |  92.8 |  98.9 |  95.8 |  100 |  100 |  100 |  53.6 |  56.6 |  55.1 |  63.6 |  70.9 |  67.1 |  80.6 |  79.8 |  80.2 |  100 |  100 |  100
w d-hyp |  59.9 |  65.3 |  62.5 |  81.1 |  73.8 |  77.3 |  86.7 |  85.1 |  85.9 |  95.1 |  94.6 |  94.9 |  68.1 |  46.5 |  55.2 |  83.6 |  71.2 |  76.9 |  93.1 |  95.6 |  94.3 |  100 |  95.5 |  97.7 |  46.6 |  50.2 |  48.3 |  55.1 |  71.8 |  62.3 |  83.1 |  76.2 |  79.5 |  100 |  100 |  100
w/o reg. |  65.2 |  63.3 |  64.2 |  79.4 |  67.9 |  73.2 |  87.4 |  84.4 |  85.9 |  94.8 |  94.3 |  94.6 |  76.0 |  52.6 |  62.2 |  87.6 |  68.4 |  76.8 |  92.8 |  97.8 |  95.2 |  100 |  100 |  100 |  49.1 |  58.4 |  53.4 |  58.3 |  64.5 |  61.3 |  78.7 |  68.9 |  73.5 |  100 |  100 |  100
w/o soft |  60.6 |  68.9 |  64.5 |  80.5 |  59.8 |  68.6 |  84.3 |  82.4 |  83.3 |  92.5 |  96.0 |  94.2 |  20.5 |  36.5 |  26.2 |  77.5 |  73.2 |  75.3 |  92.1 |  95.1 |  93.6 |  100 |  100 |  100 |  36.6 |  50.6 |  42.5 |  52.7 |  63.3 |  57.5 |  80.1 |  73.1 |  76.4 |  100 |  100 |  100
w/o NL |  59.3 |  65.3 |  62.2 |  81.9 |  71.3 |  76.3 |  81.2 |  86.7 |  83.9 |  93.1 |  96.0 |  94.5 |  59.5 |  31.3 |  41.0 |  80.3 |  68.9 |  74.2 |  91.2 |  96.7 |  93.9 |  95.7 |  100 |  97.8 |  43.2 |  62.0 |  50.9 |  51.0 |  75.2 |  60.7 |  73.5 |  80.3 |  76.7 |  93.1 |  100 |  96.4

###  5.2. Results for our Prompt Tuning Model. (I1)

Main Results. We compare the basic APrompt4EM without information augmentation
with other methods in this subsection. The overall performance under the low-
resource GEM task of all involving models is presented in the upper part of
Tab. 3. On all twelve datasets, our model achieves the best results in eleven
of them, and we achieve comparable results on the last dataset, S-T-C.
Compared with the best results of baselines, our prompt tuning approach
achieves an average of $5.24\%$ performance boost. Three main conclusions can
be drawn according to the experiment results. I. Models of the fine-tuning
paradigm (RoBERTa, Ditto, Machop) and contrastive pre-train paradigm
(Sudowoodo) mostly perform worse than models of the prompt tuning paradigm
(PromptEM, APrompt4EM), which verifies our choice of selecting prompt tuning
as our base method. As stated in PromptEM (Wang et al., 2022b), the prompt
tuning paradigm can transfer and adapt information from LMs better than
directly fine-tuning the original LM. II. Compared with the prompt tuning
method PromptEM, APrompt4EM performs better, as the natural language prompt is
closer to the LM’s text distribution, and the contextualized soft tokens
extract the key information. Especially on datasets (SEMI-HETER, WDC, and
S-T-W), which are comparably noisy and full of redundant information, our
model can achieve a significant $20\%+$ F1 improvement compared with PromptEM.
III. APrompt4EM outperforms Rotom consistently on the low-resource GEM task.
Compared with the Seq2seq data augmentation module in Rotom, the
contextualized soft token can compress and extract information better.

Ablation Studies. The ablation studies of the key components of the prompt
tuning module are presented in the lower part of Tab. 3. w d-hyper represents
our model with a default $N=0,K=4$ hyperparameters. w/o reg. represents our
method of removing the orthogonal loss, w/o soft represents our method of
removing the contextualized soft token model, and w/o NL represents removing
our natural language serialize module and using the structured serialize
method proposed by Ditto.

As we can see, the results of w d-hyper slightly decline compared with the
results of the optimized hyperparameters, but APrompt4EM with default
hyperparameters still achieves a performance boost compared with the
baselines. For the other three ablations, we find out that all three key
components are crucial to our final performance. The orthogonal loss could
prevent all the soft tokens from being identical, which is helpful in all
datasets. The contextualized soft token works mainly on the most redundant
dataset, S-T-W, and the natural language module helps improve most on the
S-T-W, as the structured data for one source differs severely from the textual
data for another source, which harms the performance using the structured
prompt template.

---

((a)) GOOGLE-AMAZON

((b)) WDC

Figure 4. F1 results with varying training ratio.

Effectiveness in Different Low-Resource Settings. Our default low-resource
settings use $10\%$ of the labeled training data for each dataset. We test the
effectiveness of our model on the GOOGLE-AMAZON and WDC datasets from $5\%$ to
$100\%$ of the training data. The results are shown in Fig. 4. As we can see,
our method shows promising results in general low-resource settings. The
results of fine-tuning methods (RoBERTa, Ditto) show significant fluctuation
when the training ratio is relatively small, which indicates that when the
training set is tiny, the LM may easily be overfitting while fine-tuning,
leading to unstable results. When the ratio of training data continues to
rise, the gap between prompt tuning and fine-tuning methods gradually narrows,
which is consistent with the hypothesis that prompt-tuning has more advantages
in low-resource settings (Lester et al., 2021). When $100\%$ of the labels are
used, our method is also among the best-performing models, but the results are
very close to the contrastive learning approach (Sudowoodo).

---

((a)) GOOGLE-AMAZON

((b)) WDC

((c)) SEMI-HOMO

Figure 5. F1 results with varying training epoch.

Faster Convergence. Through this experiment, we demonstrate that our natural
language prompt module can help our model converge faster. We present the
results of F1 on the test set for three datasets: GOOGLE-AMAZON, WDC, and
SEMI-HOMO in Fig. 5. As we can see, our method of using natural language
prompts converges consistently faster than PromptEM and achieves the best
results in a relatively early stage of the training process (Epoch $5$, $20$,
and $4$ out of $30$ total epochs, respectively). More importantly, our natural
language prompts can achieve a decent result in the first two epochs of
training. For example, in GOOGLE-AMAZON, where both data sources are
structured tables, the structured template makes the text sequence lose
meaning when the model parameters are not tuned. Converting them to natural
language ensures meaning for each entity without training, which can be
captured by the LM in the very early stage of training.

###  5.3. Further Experiments of the Contextualized Soft Token. (I2)

Figure 6. F1 results with varying soft token number $K$.

Effect of different hyper-parameter variants. We present the performance of
contextualized soft tokens with varying hyper-
parameters to gain a deeper understanding of our key module. The F1 scores for
different numbers of soft tokens $K$ across all datasets are depicted in Fig.
6. The results illustrate that for most datasets, employing multiple soft
tokens/aspect queries is beneficial. This shows that, with the guidance of the
orthogonal loss, multiple soft tokens might represent different aspects
compressed from the entity, which is more helpful to our backbone LM.
Moreover, the results of $8$ tokens are not always the best, which shows that
the number of soft tokens follows ‘not more is better’. Selecting a proper
soft token number can help the contextualized soft token extract information
better.

---

((a)) GEO-HETER

((b)) SEMI-HETER

((c)) SEMI-REL

Figure 7. F1 results with varying position embedding selection in the
contextualized soft token module.

We present the F1 scores of the GEO-HETER, SEMI-HETER, and SEMI-REL datasets
under different position embedding settings in Fig. 7. The results vary for
different datasets. For GEO-HETER, the location dataset and the location
names, which are sensitive to the token orders, are better encoded using the
original position embedding (PE). For SEMI-HETER having both structured data
sources, the column position embedding (COL) is most suitable. As for SEMI-REL
having sources with different data schemas, the difference may make no
position embedding (w/o) better.

---

((a)) GOOGLE-AMAZON

((b)) SEMI-HETER

((c)) S-T-C

Figure 8. F1 results with varying N layers.

Finally, we present the F1 results of modules with different transformers
encoding layer number $N$ in Fig. 8. As we can see, different datasets prefer
different choices of $N$, as the functions of this module with different $N$
layers. We further show that in the following visualization part.

---

((a)) WDC with 1 layer (N=0)

((b)) WDC with 2 layers (N=1)

Figure 9. Visualization of the attention between the aspect queries and the
encoded tokens in the contextualized soft token module. We truncate the
sentence due to page limitations. The redder the color, the greater the
attention weight.

Visualization of the Contextualized Soft Token. We visualize the attention
score of $K=4$ aspect tokens with the encoded text tokens of the WDC dataset
in Fig. 9. The blue color corresponds to smaller attention values, whereas the
red color indicates larger attention values. As we can see, for our
contextualized soft token with one layer ($N=0$), the four tokens focus on
different parts of the text. The last token focuses specifically on the
meaning of <pad>, while the first three tokens have significantly higher
attention values on the key part (key-value pair). This shows that the
extracted information represents the index of information that our backbone LM
should focus on. For contextualized soft tokens with two layers ($N=1$), we
observe variations across the four attention distributions as well. Though the
first token still focuses on the missing information, the rest of the tokens
focus on the encoded meaningful values (key-value pairs). This shows that for
$N>0$, the extracted soft token primarily represents the content of entities,
which can be directly fed to our backbone LM.

###  5.4. Results for the Information Augmentation Module. (I3)

Main Results. We conduct experiments on augmenting information using LLMs.
Considering LLM’s ability to be directly used to GEM, we also compare fine-
tuned LLMs, zero-shot/few-shot inference for LLMs, and using LLMs as a
labeler.

For LLM’s information augmentation, we augment information with GPT3.5 and
conduct the experiments on four datasets (REL-TEXT, S-T-W, S-T-C, and WDC)
that miss information more severely. As we describe in Sec. 4.2, for datasets
with more homogeneity (REL-TEXT, WDC), we augment information with fixed
attribute keys. For REL-TEXT (academic), we augment with {title, abstract}.
For WDC (product), we augment with {capacity, color, frequency, keywords,
language, model number, product identifier, release year, resolution, size,
speed, weight}. For datasets containing heterogeneous data (S-T-W, S-T-C), we
follow the augmentation at the granularity of data instances. We present the
results of our model with augmented information in Tab. 4.

For fine-tuned LLMs, we include the results of fine-tuned GPT3. In Tab. 4,
GPT3 (Fine-tune) represents fine-tuning with structured input, while GPT3
(Fine-tune NL) represents fine-tuning with the natural language serialization.

For zero-shot/ few-shot LLM inference, we list the zero-shot and few-shot
($2$, $5$ shot) results of GPT3.5 in Tab. 4. For zero-shot inference, we
concatenate the two entities with prompts and query the LLM for an answer. For
few-shot ($k$-shot) inference, we retrieve $k$ positive and $k$ negative query
pairs in the labeled training pairs randomly or according to TF-IDF
statistics. The pairs and labels are provided before the query prompts as in-
context samples. The prompts for zero/few-shot are the same as previous work
(Peeters and Bizer, 2023).

For using LLM as a labeler, we use a fine-tuned GPT3 model to provide labels
for the $90\%$ unused training samples. The results are shown in Tab. 4, where
$ax$ labels means we added $a$ times of the original training labels to the
training set.

As we can see for Tab. 4, augmenting information constantly boosts the
performance of our base model. Though directly fine-tuning GPT-3 is promising
on some datasets, our model can achieve comparable results compared with LLMs
with the augmented information from LLMs. For datasets where LLMs achieve a
good result (WDC), the improvement brought by the augmented information is
also larger, which is quite intuitive. For the zero-shot/ few-shot results, we
find out that fine-tuning is necessary even for LLMs for noisy GEM tasks. For
the labeling experiment, we illustrate that training our model with augment
information achieves better results than training our model with LLM provided
labels (up to 3x labels than the original training data).

---

((a)) F1 results augmenting with varying uncertainty score thresholds

((b)) Case study for information augmentation on WDC

Figure 10. Effectiveness of the uncertainty-based strategy and case study for
information augmentation on the WDC dataset.

Effectiveness of the Uncertainty-based Information Augmentation Strategy. We
show the effectiveness of our augmenting strategy for the WDC dataset in Fig.
10 (a). We use $H(P(y|(e,e_{i})))=max(P(y=0|(e,e_{i})),P(y=1|(e,e_{i}))$ in
this experiment. The x-axis represents the threshold of uncertainty scores,
the y-axis represents the F1 results, the number on the bar represents the
percentage of supplementary entities among all tested entities, and the red
line represents APrompt4EM without information augmentation. As we can see,
though augmenting part of the test data harms the consistency of the
distribution between training and test data, augmenting half of the whole
entity can bring comparable results with the base model. Augmenting $85\%$ of
the whole entities can achieve similar results as augmenting all entities, as
many comparing pairs are not confusing.

Table 4. F1 results of methods considering using LLMs.

| REL-TEXT | S-T-W | S-T-C | WDC
---|---|---|---|---
GPT3 (Fine-tune) | 64.8 | 6.3 | 85.7 | 67.5
GPT3 (Fine-tune NL) | 67.5 | 5.5 | 84.7 | 65.9
GPT3.5 (0-shot) | 7.3 | 9.0 | 45.9 | 22.1
GPT3.5 (2-shot,TF-IDF) | 46.6 | 18.7 | 31.8 | 32.1
GPT3.5 (2-shot,random) | 37.9 | 21.6 | 37.7 | 45.4
GPT3.5 (5-shot,TF-IDF) | 55.4 | 16.4 | 15.8 | 33.6
GPT3.5 (5-shot,random) | 39.5 | 20.8 | 36.7 | 39.9
Ours | 64.7 | 62.2 | 78.6 | 55.1
Ours+1x Label | 64.5 | 49.4 | 79.9 | 55.3
Ours+2x Label | 64.5 | 40.6 | 81.8 | 58.4
Ours+3x Label | 64.2 | 29.3 | 82.2 | 58.8
Ours+ aug. | 67.5 | 62.4 | 81.2 | 61.2

---

((a)) Cost of the original dataset

((b)) Cost of the blocking dataset

Figure 11. Cost analysis of our method: cost comparison between our
information augmentation and direct inference.

Cost Analysis of Our Method. We present the cost of our economical information
augmentation method and the fine-tuning method using GPT3.5 in Fig. 11. We
conduct two experiments on the original labeled dataset and a synthetic
blocking dataset. For the blocking dataset, a simple TF-IDF blocker is used
and a threshold is set to make each entity have an average of five candidates
to be compared. We present the approximate token numbers and the total cost of
the two tasks. The approximate token numbers are presented as numbers above
the bars, and the cost is represented as the height of the bar. According to
the official OpenAI pricing, the inference cost of a personal fine-tuned model
is $0.016\$/1k$ tokens, while the inference cost of the original model is
$0.002\$/1k$ tokens. Hence, the cost of inference using a fine-tuned model is
much higher. As we can see, for the partly labeled original dataset, such as
REL-TEXT, S-T-W, and S-T-C, the pairs tested are primarily from different
entities, so our augmentation strategy benefits mainly from the pricing
difference. In WDC, where the labeled samples contain duplicate entities, our
strategy uses only half of the tokens. For scenarios closer to real-world
matching applications, our augmentation method calls for $10\%$-$40\%$ tokens,
as established in Sec. 4.2 and Fig. 11, resulting in an even lower total cost.

Case Study. We present a case study of the WDC dataset. Two initially false
negative and false positive pairs are presented in Fig. 10 (b). After
information augmentation, both pairs are correctly classified. The
augmentation information of the first two entities neglects the redundant
description information of the first entity and points out that the key
information of the two entities is the same. For the second example, though
the same ‘ironwolf seagate 3.5’ information confuses the LM, the augmented
information successfully captures the two entities’ difference in frequency
and capacity in the redundant information, which helps our backbone LM change
its decision. Besides, the information marked in red can be extracted from the
original description, while the information marked in blue is somehow
‘recalled’ by the LLM, which demonstrates the strong ability of the LLM to
supplement information.

##  6\. Related Work

EM, as a crucial and challenging task in data integration, has received much
attention in the past decades. Early works focus on rule-based methods
(Elmagarmid et al., 2014; Wang et al., 2011; Bilenko and Mooney, 2003; Kandhan
et al., 2010), active sampling in EM (Bellare et al., 2013; Meduri et al.,
2020), and human labeling for EM (Gokhale et al., 2014; Wang et al., 2012). In
2017-2018, DeepMatcher (Mudgal et al., 2018) and DeepER (Ebraheem et al.,
2018) first introduce deep learning into EM.

Fine-tuning methods for EM. After 2017, pre-trained language models (PLMs),
which have powerful semantic expression abilities, have shown the capability
of performing multiple NLP tasks (Liu et al., 2023a), including the EM task.
In the groundbreaking work of adopting PLMs on the EM task, Ditto (Li et al.,
2020), the EM task is interpreted as a sequence classification problem. A
structured entity $e$ is first serialized into a language sequence
$serialize(e)$. The serialized sequences for two entity pairs are concatenated
with two special separation tokens, $[CLS]$ and $[SEP]$, and the sequence is
input to a PLM (i.e., BERT (Kenton and Toutanova, 2019), RoBERTa (Liu et al.,
2019), DistilBERT (Sanh et al., 2019)). A task-specific classification layer
is added after the output of the $[CLS]$ token, and the model is tuned on the
training set until convergence with its parameters initialized from the pre-
trained model.

Prompt-tuning for EM. Different from the pre-train, fine-tune training
paradigm, which trains a model for predicting the label distribution of an
input and its label, the prompt-based tuning paradigm directly models the
probability of an input text and computes the label probability referring to
the text probability (Liu et al., 2023a).

The prompt templates used in prompt tuning can be categorized into discrete
prompts and continuous prompts (Liu et al., 2023a). For discrete prompts,
apart from the manual design method, mining (Shin et al., 2020), paraphrasing
(Jiang et al., 2020), and generation methods (Gao et al., 2021; Guo et al.,
2022) are proposed to discover better prompts for downstream tasks. As for
continuous prompts, representative methods include prefix tuning (Li and
Liang, 2021), Optiprompt (Zhong et al., 2021), and P-tuning (Liu et al.,
2023b). Prefix tuning proposes adding a continuous prefix before the prompt
content and tuning it during training, while the other parts of the model
(main prompt, LM) are frozen. Optiprompt proposes adding soft tokens
initialized by discrete prompts. As for P-tuning, trainable variables are
introduced to the prompts through a BiLSTM model.

To address the low-resource challenge, PromptEM (Wang et al., 2022b) first
adopts the P-tuning approach to solve the EM problem. Our work, addressing the
low-resource GEM task, is also based on the prompt tuning paradigm because it
is proven effective in low-resource situations. It is worth noting that
previous works tune one prompt for all data instances in a task, which is
different from our contextualized prompt for each data instance.

Other perspectives for EM. Many works aim at the difficulty of the EM task in
different complex real-world scenarios. Machamp (Wang et al., 2021) first
presents the concept and a benchmark of GEM, and Machop (Wang et al., 2022a)
provides a new framework for GEM considering the alignment at the attribute
granularity. FlexER (Genossar et al., 2023) focuses on matching entities
represented with multiple intents. ZeroER (Wu et al., 2020) and TDMatch
(Ahmadi et al., 2022) focus on more extreme matching scenarios where no
supervision is provided.

Many other works bring in novel techniques to boost EM performance. A series
of works (Loster et al., 2021; Zhao and He, 2019; Tu et al., 2022) focus on
domain adaptation and knowledge transferring in EM. Another series of works
(Miao et al., 2021; Wang et al., 2023; Peeters and Bizer, 2022) discuss data
augmentation in EM: Rotom (Miao et al., 2021) aims at generating augmented
data for data integration; Sudowoodo (Wang et al., 2023) and Supcon-EM
(Peeters and Bizer, 2022) focus on self-contrastive pre-training embeddings
for EM. Recently, the usage of foundation models in EM are discussed in
(Narayan et al., 2022; Peeters and Bizer, 2023).

##  7\. Conclusion

In this paper, we propose APrompt4EM for low-resource GEM, in which the
contextualized soft token prompt in natural language form helps our model
extract critical information from noisy information, and information
augmentation from LLMs helps bridge the semantic gap. Through extensive
experiments and case studies, we report the superior performance of our model
and illustrate the importance of all the techniques that we use in the low-
resource GEM task. We report that the augmented model can achieve comparable
results with fine-tuned LLMs using fewer API fees. We plan to further explore
the possibility of more automatic adding and compressing for prompts in EM,
e.g., via reinforcement learning.

## References

  * (1)
  * Ahmadi et al. (2022) Naser Ahmadi, Hansjörg Sand, and Paolo Papotti. 2022.  Unsupervised matching of data and text. In _2022 IEEE 38th International Conference on Data Engineering (ICDE)_. IEEE, 1058–1070.
  * AlKhamissi et al. (2022) Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. 2022.  A review on language models as knowledge bases.  _arXiv preprint arXiv:2204.06031_ (2022).
  * Asai et al. (2023) Akari Asai, Sewon Min, Zexuan Zhong, and Danqi Chen. 2023.  Retrieval-based Language Models and Applications. In _ACL (tutorial)_. Association for Computational Linguistics, 41–46.
  * Barlaug and Gulla (2021) Nils Barlaug and Jon Atle Gulla. 2021.  Neural networks for entity matching: A survey.  _ACM Transactions on Knowledge Discovery from Data (TKDD)_ 15, 3 (2021), 1–37.
  * Bellare et al. (2013) Kedar Bellare, Suresh Iyengar, Aditya G. Parameswaran, and Vibhor Rastogi. 2013.  Active Sampling for Entity Matching with Guarantees.  _ACM Trans. Knowl. Discov. Data_ 7, 3 (2013), 12:1–12:24.
  * Bilenko and Mooney (2003) Mikhail Bilenko and Raymond J. Mooney. 2003.  Adaptive duplicate detection using learnable string similarity measures. In _KDD_. ACM, 39–48.
  * Chen et al. (2022) Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022.  Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction. In _Proceedings of the ACM Web conference 2022_. 2778–2788.
  * Chu et al. (2016) Xu Chu, Ihab F Ilyas, Sanjay Krishnan, and Jiannan Wang. 2016.  Data cleaning: Overview and emerging challenges. In _Proceedings of the 2016 international conference on management of data_. 2201–2206.
  * Ding et al. (2022) Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Haitao Zheng, and Maosong Sun. 2022.  OpenPrompt: An Open-source Framework for Prompt-learning. In _ACL (demo)_. 105–113.
  * Doan et al. (2012) AnHai Doan, Alon Y. Halevy, and Zachary G. Ives. 2012.  _Principles of Data Integration_.  Morgan Kaufmann.
  * Ebraheem et al. (2018) Muhammad Ebraheem, Saravanan Thirumuruganathan, Shafiq Joty, Mourad Ouzzani, and Nan Tang. 2018.  Distributed representations of tuples for entity resolution.  _Proceedings of the VLDB Endowment_ 11, 11 (2018), 1454–1467.
  * Elmagarmid et al. (2014) Ahmed Elmagarmid, Ihab F Ilyas, Mourad Ouzzani, Jorge-Arnulfo Quiané-Ruiz, Nan Tang, and Si Yin. 2014.  NADEEF/ER: Generic and interactive entity resolution. In _Proceedings of the 2014 ACM SIGMOD international conference on Management of data_. 1071–1074.
  * Fang et al. (2023) Liri Fang, Lan Li, Yiren Liu, Vetle I. Torvik, and Bertram Ludäscher. 2023.  KAER: A Knowledge Augmented Pre-Trained Language Model for Entity Resolution.  _CoRR_ abs/2301.04770 (2023).
  * Feng et al. (2021) Steven Y. Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard H. Hovy. 2021.  A Survey of Data Augmentation Approaches for NLP. In _ACL/IJCNLP_. 968–988.
  * Gao et al. (2021) Tianyu Gao, Adam Fisch, and Danqi Chen. 2021.  Making Pre-trained Language Models Better Few-shot Learners. In _ACL/IJCNLP_. 3816–3830.
  * Genossar et al. (2023) Bar Genossar, Roee Shraga, and Avigdor Gal. 2023.  FlexER: Flexible Entity Resolution for Multiple Intents.  _Proceedings of the ACM on Management of Data_ 1, 1 (2023), 1–27.
  * Gokhale et al. (2014) Chaitanya Gokhale, Sanjib Das, AnHai Doan, Jeffrey F Naughton, Narasimhan Rampalli, Jude Shavlik, and Xiaojin Zhu. 2014.  Corleone: Hands-off crowdsourcing for entity matching. In _Proceedings of the 2014 ACM SIGMOD international conference on Management of data_. 601–612.
  * Guo et al. (2022) Han Guo, Bowen Tan, Zhengzhong Liu, Eric P. Xing, and Zhiting Hu. 2022.  Efficient (Soft) Q-Learning for Text Generation with Limited Good Data. In _Findings of the Association for Computational Linguistics: EMNLP_. 6969–6991.
  * Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023.  Survey of hallucination in natural language generation.  _Comput. Surveys_ 55, 12 (2023), 1–38.
  * Jiang et al. (2020) Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020.  How can we know what language models know?  _Transactions of the Association for Computational Linguistics_ 8 (2020), 423–438.
  * Kandhan et al. (2010) Ramakrishnan Kandhan, Nikhil Teletia, and Jignesh M Patel. 2010.  SigMatch: fast and scalable multi-pattern matching.  _Proceedings of the VLDB Endowment_ 3, 1-2 (2010), 1173–1184.
  * Kenton and Toutanova (2019) Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _Proceedings of NAACL-HLT_. 4171–4186.
  * Konda (2018) Pradap Venkatramanan Konda. 2018.  _Magellan: Toward building entity matching management systems_.  The University of Wisconsin-Madison.
  * Köpcke et al. (2010) Hanna Köpcke, Andreas Thor, and Erhard Rahm. 2010.  Evaluation of entity resolution approaches on real-world match problems.  _Proceedings of the VLDB Endowment_ 3, 1-2 (2010), 484–493.
  * Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant. 2021.  The Power of Scale for Parameter-Efficient Prompt Tuning. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_. 3045–3059.
  * Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021.  Prefix-Tuning: Optimizing Continuous Prompts for Generation. In _ACL/IJCNLP._ 4582–4597.
  * Li et al. (2020) Yuliang Li, Jinfeng Li, Yoshihiko Suhara, AnHai Doan, and Wang-Chiew Tan. 2020.  Deep entity matching with pre-trained language models.  _Proceedings of the VLDB Endowment_ 14, 1 (2020), 50–60.
  * Li et al. (2021) Yuliang Li, Xiaolan Wang, Zhengjie Miao, and Wang-Chiew Tan. 2021.  Data augmentation for ml-driven data preparation and integration.  _Proceedings of the VLDB Endowment_ 14, 12 (2021), 3182–3185.
  * Liu et al. (2023a) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023a.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  _Comput. Surveys_ 55, 9 (2023), 1–35.
  * Liu et al. (2023b) Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2023b.  GPT understands, too.  _AI Open_ (2023).
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.  _CoRR_ abs/1907.11692 (2019).
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled Weight Decay Regularization. In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_.
  * Loster et al. (2021) Michael Loster, Ioannis Koumarelas, and Felix Naumann. 2021.  Knowledge transfer for entity resolution with siamese neural networks.  _Journal of Data and Information Quality (JDIQ)_ 13, 1 (2021), 1–25.
  * Meduri et al. (2020) Venkata Vamsikrishna Meduri, Lucian Popa, Prithviraj Sen, and Mohamed Sarwat. 2020.  A comprehensive benchmark framework for active learning methods in entity matching. In _Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data_. 1133–1147.
  * Miao et al. (2021) Zhengjie Miao, Yuliang Li, and Xiaolan Wang. 2021.  Rotom: A meta-learned data augmentation framework for entity matching, data cleaning, text classification, and beyond. In _Proceedings of the 2021 International Conference on Management of Data_. 1303–1316.
  * Mudgal et al. (2018) Sidharth Mudgal, Han Li, Theodoros Rekatsinas, AnHai Doan, Youngchoon Park, Ganesh Krishnan, Rohit Deep, Esteban Arcaute, and Vijay Raghavendra. 2018.  Deep learning for entity matching: A design space exploration. In _Proceedings of the 2018 International Conference on Management of Data_. 19–34.
  * Narayan et al. (2022) Avanika Narayan, Ines Chami, Laurel Orr, and Christopher Ré. 2022.  Can Foundation Models Wrangle Your Data?  _Proceedings of the VLDB Endowment_ 16, 4 (2022), 738–746.
  * Nassar et al. (2022) Ahmed Nassar, Nikolaos Livathinos, Maksym Lysak, and Peter Staar. 2022.  Tableformer: Table structure understanding with transformers. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 4614–4623.
  * OpenAI (2023) OpenAI. 2023.  GPT-4 Technical Report.  _CoRR_ abs/2303.08774 (2023).
  * Paganelli et al. (2022) Matteo Paganelli, Francesco Del Buono, Andrea Baraldi, Francesco Guerra, et al. 2022\.  Analyzing how BERT performs entity matching.  _Proceedings of the VLDB Endowment_ 15, 8 (2022), 1726–1738.
  * Papadakis et al. (2020) George Papadakis, Dimitrios Skoutas, Emmanouil Thanos, and Themis Palpanas. 2020.  Blocking and filtering techniques for entity resolution: A survey.  _ACM Computing Surveys (CSUR)_ 53, 2 (2020), 1–42.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019\.  Pytorch: An imperative style, high-performance deep learning library.  _Advances in neural information processing systems_ 32 (2019).
  * Peeters and Bizer (2022) Ralph Peeters and Christian Bizer. 2022.  Supervised contrastive learning for product matching. In _Companion Proceedings of the Web Conference 2022_. 248–251.
  * Peeters and Bizer (2023) Ralph Peeters and Christian Bizer. 2023.  Using ChatGPT for Entity Matching. In _New Trends in Database and Information Systems - ADBIS 2023 Short_. 221–230.
  * Peeters et al. (2024) Ralph Peeters, Reng Chiz Der, and Christian Bizer. 2024.  WDC Products: A Multi-Dimensional Entity Matching Benchmark. In _EDBT_. 22–33.
  * Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019.  Language Models as Knowledge Bases?. In _EMNLP-IJCNLP_. 2463–2473.
  * Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018\.  Improving language understanding by generative pre-training.  (2018).
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019\.  Language models are unsupervised multitask learners.  _OpenAI blog_ 1, 8 (2019), 9.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _The Journal of Machine Learning Research_ 21, 1 (2020), 5485–5551.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In _EMNLP-IJCNLP_. 3982–3992.
  * Sanh et al. (2019) Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019.  DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.  _arXiv preprint arXiv:1910.01108_ (2019).
  * Shi et al. (2022) Weijia Shi, Julian Michael, Suchin Gururangan, and Luke Zettlemoyer. 2022.  Nearest neighbor zero-shot inference. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_. 3254–3265.
  * Shi et al. (2023) Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023.  Replug: Retrieval-augmented black-box language models.  _arXiv preprint arXiv:2301.12652_ (2023).
  * Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020.  AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. 4222–4235.
  * Trabelsi et al. (2022) Mohamed Trabelsi, Zhiyu Chen, Shuo Zhang, Brian D Davison, and Jeff Heflin. 2022.  Strubert: Structure-aware bert for table search and matching. In _Proceedings of the ACM Web Conference 2022_. 442–451.
  * Tu et al. (2022) Jianhong Tu, Ju Fan, Nan Tang, Peng Wang, Chengliang Chai, Guoliang Li, Ruixue Fan, and Xiaoyong Du. 2022.  Domain adaptation for deep entity resolution. In _Proceedings of the 2022 International Conference on Management of Data_. 443–457.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Wang et al. (2012) Jiannan Wang, Tim Kraska, Michael J Franklin, and Jianhua Feng. 2012.  CrowdER: Crowdsourcing Entity Resolution.  _Proceedings of the VLDB Endowment_ 5, 11 (2012).
  * Wang et al. (2011) Jiannan Wang, Guoliang Li, Jeffrey Xu Yu, and Jianhua Feng. 2011.  Entity matching: How similar is similar.  _Proceedings of the VLDB Endowment_ 4, 10 (2011), 622–633.
  * Wang et al. (2021) Jin Wang, Yuliang Li, and Wataru Hirota. 2021.  Machamp: A generalized entity matching benchmark. In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_. 4633–4642.
  * Wang et al. (2022a) Jin Wang, Yuliang Li, Wataru Hirota, and Eser Kandogan. 2022a.  Machop: An end-to-end generalized entity matching framework. In _Proceedings of the Fifth International Workshop on Exploiting Artificial Intelligence Techniques for Data Management_. 1–10.
  * Wang et al. (2022b) Pengfei Wang, Xiaocan Zeng, Lu Chen, Fan Ye, Yuren Mao, Junhao Zhu, and Yunjun Gao. 2022b.  PromptEM: prompt-tuning for low-resource generalized entity matching.  _Proceedings of the VLDB Endowment_ 16, 2 (2022), 369–378.
  * Wang et al. (2023) Runhui Wang, Yuliang Li, and Jin Wang. 2023.  Sudowoodo: Contrastive self-supervised learning for multi-purpose data integration and preparation. In _2023 IEEE 39th International Conference on Data Engineering (ICDE)_. IEEE, 1502–1515.
  * Wei et al. (2022) Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022\.  Emergent Abilities of Large Language Models.  _Transactions on Machine Learning Research_ (2022).
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019\.  Huggingface’s transformers: State-of-the-art natural language processing.  _arXiv preprint arXiv:1910.03771_ (2019).
  * Wu et al. (2020) Renzhi Wu, Sanya Chaba, Saurabh Sawlani, Xu Chu, and Saravanan Thirumuruganathan. 2020.  Zeroer: Entity resolution using zero labeled examples. In _Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data_. 1149–1164.
  * Yu et al. (2020) Linxuan Yu, Yeli Li, Qingtao Zeng, Yanxiong Sun, Yuning Bian, and Wei He. 2020.  Summary of web crawler technology research. In _Journal of Physics: Conference Series_ , Vol. 1449. IOP Publishing, 012036.
  * Zhao and He (2019) Chen Zhao and Yeye He. 2019.  Auto-em: End-to-end fuzzy entity-matching using pre-trained deep models and transfer learning. In _The World Wide Web Conference_. 2413–2424.
  * Zhong et al. (2021) Zexuan Zhong, Dan Friedman, and Danqi Chen. 2021.  Factual Probing Is [MASK]: Learning vs. Learning to Recall. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. 5017–5033.