Rabia Azzi and Gayo Diallo 11institutetext: BPH Center/INSERM U1219, Univ.
Bordeaux, F-33000, France,
11email<EMAIL_ADDRESS>
# AMALGAM: A Matching Approach to fairfy tabuLar data with knowledGe grAph
Model

Rabia Azzi  Gayo Diallo

###### Abstract

In this paper we present AMALGAM, a matching approach to fairify tabular data
with the use of a knowledge graph. The ultimate goal is to provide fast and
efficient approach to annotate tabular data with entities from a background
knowledge. The approach combines lookup and filtering services combined with
text pre-processing techniques. Experiments conducted in the context of the
2020 Semantic Web Challenge on Tabular Data to Knowledge Graph Matching with
both Column Type Annotation and Cell Type Annotation tasks showed promising
results.

###### keywords:

Tabular Data, Knowledge Graph, Entity Linking

##  1 Introduction

Making web data complying with the FAIR111FAIR stands for: Findable,
Accessible, Interoperable and Reusable principles has become a necessity in
order to facilitate their discovery and reuse [1]. The value for the knowledge
discovery of implementing FAIR is to increase, data integration, data
cleaning, data mining, machine learning and knowledge discovery tasks.
Successfully implemented FAIR principles will improve the value of data by
making them findable, accessible and resolve semantic ambiguities. Good data
management is not a goal in itself, but rather is the key conduit leading to
knowledge discovery and acquisition, and to subsequent data and knowledge
integration and reuse by the community after the data publication process [2].

Semantic annotation could be considered as a particular knowledge acquisition
task [3, 4, 5]. The semantic annotation process may rely on formal metadata
resources described with an Ontology, even sometimes with multiple ontologies
thanks to the use of semantic repositories [6]. Over the last years, tables
are one of the most used formats to share results and data. In this field, a
set of systems for matching web tables to knowledge bases has been developed
[7, 8]. They can be categorized in two main tasks: structure and semantic
annotation. The structure annotation deals with tasks such as data type
prediction and table header annotation [9]. Semantic annotation involves
matching table elements into KG [10] e.g., columns to class and cells to
entities [11, 12].

Recent years have seen an increasing number of works on Semantic Table
Interpretation. In this context, SemTab
2020222http://www.cs.ox.ac.uk/isg/challenges/sem-tab/2020/index.html has
emerged as an initiative which aims at benchmarking systems which deals with
annotating tabular data with entities from a KG, referred as table annotation
[13]. SemTab is organised into three tasks, each one with several evaluation
rounds. For the 2020 edition for instance, it involves: (i) assigning a
semantic type (e.g., a KG class) to a column (CTA); (ii) matching a cell to a
KG entity (CEA); (iii) assigning a KG property to the relationship between two
columns (CPA).

Our goal is to automatically annotate on the fly tabular data. Thus, our
annotation approach is fully automated, as it does not need prior information
regarding entities, or metadata standards. It is fast and easy to deploy, as
it takes advantage of the existing system like Wikidata and Wikipedia to
access entities.

##  2 Related Work

Various research works have addressed the issue of semantic table annotation.
The most popular approaches which deal with the three above mentioned tasks
rely on supervised learning setting, where candidate entities are selected by
a classiﬁcation models [14]. Such systems include (i) MTab [15], which
combines a voting algorithm and the probability models to solve critical
problems of the matching tasks, (ii) DAGOBAH [16] aiming at semantically
annotating tables with Wikidata and DBpedia entities; more precisely it
performs cell and column annotation and relationship identification, via a
pipeline starting from a pre-processing step to enriching an existing
knowledge graph using the table information; (iii) ADOG [17] is a system
focused on leveraging the structure of a well-connected ontology graph which
is extracted from different Knowledge Graphs to annotate structured or semi-
structured data. In the latter approach, they combine in novel ways a set of
existing technologies and algorithms to automatically annotate structured and
semi-structured records. It takes advantage of the native graph structure of
ontologies to build a well-connected network on ontologies from different
sources; (iv) Another example is described in [18]. Its process is split into
a Candidate Generation and a Candidate Selection phases. The former involves
looking for relevant entities in knowledge bases, while the latter involves
picking the top candidate using various techniques such as heuristics (the
‘TF-IDF’ approach) and machine learning (the Neural Network Ranking model).

In [19] the authors present TableMiner, a learning approach for a semantic
table interpretation. This is essentially done by improving annotation
accuracy by making innovative use of various types of contextual information
both inside and outside tables as features for inference. Then, it reduces
computational overheads by adopting an incremental, bootstrapping approach
that starts by creating preliminary and partial annotations of a table using
‘sample’ data in the table, then using the outcome as ‘seed’ to guide
interpretation of remaining contents. Following also a machine learning
approach, [20] proposes Meimei. It combines a latent probabilistic model with
multi-label classifiers.

Other alternative approaches address only a single specific task. Thus, in the
work of [21], the authors focuses on column type prediction for tables without
any metadata. Unlike traditional lexical matching-based methods, they follow a
deep prediction model that can fully exploit tables’ contextual semantics,
including table locality features learned by a Hybrid Neural Network (HNN),
and inter-column semantics features learned by a knowledge base (KB) lookup
and query answering algorithm. It exhibits good performance not only on
individual table sets, but also when transferring from one table set to
another. In the same vein, a work conducted by [22] propose Taipan, which is
able to recover the semantics of tables by identifying subject columns using a
combination of structural and semantic features.

From Web tables point of view, various works could be mentioned. Thus, in [23]
an iterative matching approach is described. It combines both schema and
entity matching and is dedicated to matching large set of HTML tables with a
cross-domain knowledge base. Similarly, TabEL uses a collective classification
technique to disambiguate all mentions in a given table [24]. Instead of using
a strict mapping of types and relations into a reference knowledge base, TabEL
uses soft constraints in its graphical model to sidestep errors introduced by
an incomplete or noisy KB. It outperforms previous work on multiple datasets.

Overall, all the above mentioned approaches are based on a learning strategy.
However, for the real-time application, there is a need to get the result as
fast as possible. Another main limitation of these approaches is their
reproducibility. Indeed, key explicit information concerning study parameters
(particularly randomization control) and software environment are lacking.

The ultimate goal with AMALGAM, which could be categorized as a tabular data
to KG matching system, is to provide a fast and efficient approach for tabular
data to KG matching task.

##  3 The AMALGAM approach

AMALGAM is designed according to the workflow in Fig. 1. There are three main
phases which consist in, respectively, pre-processing, context annotation and
tabular data to KG matching. The first two steps are identical for both CEA
and CTA tasks.

Figure 1: Workflow of AMALGAM.

Tables Pre-Processing. It is common to have missing values in datasets.
Beside, the content of the table can have different types (string, date,
float, etc.)The aim of the pre-processing step is to ensure that loading table
happens without any error. For instance, a textual encoding where some
characters are loaded as noisy sequences or a text field with an unescaped
delimiter will cause the considered record to have an extra column, etc.
Loading incorrect encoding might strongly affect the lookup performance. To
overcome this issue, AMALGAM relies on the Pandas
library333https://pandas.pydata.org/ to fix all noisy textual data in the
tables being processed.

Figure 2: Illustration of a table structure.

Annotation context. We consider a table as a two-dimensional tabular structure
(see Fig. 2(A)) which is composed of an ordered set of x rows and y columns.
Each intersection between a row and a column determines a cell $c_{ij}$ with
the value $v_{ij}$ where $1\leq i\leq x,1\leq j\leq y$. To identify the
attribute label of a column also referred as header detection (CTA task), the
idea consists in annotating all the items of the column using entity linking.
Then, the attribute label is estimated using a random entity linking. The
annotation context is represented by the list of items in the same column (see
Fig. 2(B)). For example, the context of the first column in the Fig. 2 is
represented by the following items: [A1,B1,…,n]. Following the same logic, we
consider that all cells in the same row describe the same context. More
specifically, the first cell of the row describes the entity and the following
cells the associated properties. For instance, the context of the first row in
the Fig. 2 is represented by the following list of items: [A1,A2,A3,A4].

Assigning a semantic type to a column (CTA). The CTA task consists in
assigning a Wikidata KG entity to a given column. It can be performed by
exploiting the process described in Fig. 3. The Wikidata KG allows to look up
a Wikidata item by its title of its corresponding page on Wikipedia or other
Wikimedia family sites using a dedicated API
444https://www.wikidata.org/w/api.php. In our case, the main information
needed from the entity is a list of the instances of (P31), subclass of (P279)
and part of (P361) statements. To do so, a parser is developed to retrieve
this information from the Wikidata built request. For example, ”Grande
Prairie” provides the following results: [list of towns in Alberta:Q15219391,
village in Alberta:Q6644696, city in Alberta:Q55440238]. To achieve this, our
methodology combines wbsearchentities and parse actions provided by the API.
It could be observed that in this task, there were many items that have not
been annotated. This is because tables contain incorrectly spelled terms.
Therefore, before implementing the other tasks, a spell check component is
required.

As per the literature [25], spell-checker is a crucial language tool of
natural language processing (NLP) which is used in applications like
information extraction, proofreading, information retrieval, social media and
search engines. In our case, we compared several approaches and libraries:
Textblob555https://textblob.readthedocs.io/en/dev/, Spark
NLP666https://nlp.johnsnowlabs.com/,
Gurunudi777https://github.com/guruyuga/gurunudi, Wikipedia
api888https://wikipedia.readthedocs.io/en/latest/code.html,
Pyspellchecker999https://github.com/barrust/pyspellchecker,
Serpapi101010https://serpapi.com/spell-check. A comparison of these approaches
could be found in table 1.

Table 1: Comparative of approaches and libraries related to spell-checking. Name |  Category |  Strengths/Limitations
---|---|---
Textblob |  NLP |  Spelling correction, Easy-to-use
Spark NLP |  NLP |  Pre-trained, Text analysis
Gurunudi |  NLP |  Pre-trained, Text analysis, Easy-to-use
Wikipedia api |  Search engines |  Search/suggestion, Easy-to-use, Unlimited access
Pyspellchecker |  Spell checking |  Simple algorithm, No pre-trained, Easy-to-use
Serpapi |  Search engines |  Limited access for free
Figure 3: Assigning a semantic type to a column (CTA).

Our choice is oriented towards Gurunudi and the Wikidata API with a post-
processing step consisting in validating the output using
fuzzywuzzy111111https://github.com/seatgeek/fuzzywuzzy to keep only the
results whose ratio is greater than the threshold of 90%. For example, let’s
take the expression “St Peter’s Seminarz” after using the Wikidata API we get
“St Peter’s seminary” and the ratio of fuzzy string matching is 95%.

We are now able to perform the CTA task. In the trivial case, the result of an
item lookup is equal a single record. The best matching entity is chosen as a
result. In the other cases, where the result is more than one, no annotation
is produced for the CTA task. Finally, if there is no result after the lookup,
another one is performed using the output of the spell check produced by the
item. At the end of these lookups, the matched couple results are then stored
in a nested dictionary [item:claims]. The most relevant candidate, counting
the number of occurrences, is selected.

Input: $Table\ T$

Output: $Annotated\ Table\ T^{\prime}$

foreach  _$\textit{col i}\in\mathcal{T}{}$_ do

${candidates\\_col}\leftarrow\emptyset$

foreach  _$\textit{el}\in\textit{col}{}$_ do

${label}\leftarrow\textit{el.value}$

${candidates}\leftarrow\textit{wd-lookup}\ (\textit{label})$

if  _( $candidates.size=1$)_ then

$candidates\\_col(k,candidates)$

else if  _( $candidates.size=0$)_ then

${new\\_label}\leftarrow\textit{spell-check}\ (\textit{label})$

${candidates}\leftarrow\textit{wd-lookup}\ (\textit{new\\_label})$

if  _( $candidates.size=1$)_ then

$candidates\\_col(k,candidates)$

end if

end if

end foreach

$annotate(T^{\prime}.col.i,getMostCommunClass(candidates\\_col))$

end foreach

Algorithm 1 CTA task

Matching a cell to a KG entity (CEA). The CEA task can be performed by
exploiting the process described in Fig. 4. Our approach reuse the process of
the CTA task and made necessary adaptations. The first step is to get all the
statements for the first item of the list context. The process is the same as
CTA, the only difference is where result provides than one record. In this
case, we create nested dictionary with all candidates. Then, to disambiguate
the candidates entities, we use the concept of the column generated with the
CTA task. Next, a lookup is performed by using the other items of the list
context in the claims of the first item. If the item is found, it is selected
as the target entity; if not, the lookup is performed with the item using the
Wikidata API (if the result is empty, no annotation is produced).

With this process, it is possible to reduce errors associated with the lookup.
Let’s take the value “650“ in row 0 of the table Fig. 4 for instance. If we
lookup directly in Wikidata, we can get many results. However, if we check
first in the statements of the first item of the list, “Grande Prairie“, it is
more likely to successfully identify the item.

Figure 4: Matching a cell to a KG entity (CEA).

Input: $Table\ T$, $TColsContext$

Output: $Annotated\ Table\ T^{\prime}$

foreach  _$\textit{row i}\in\mathcal{T}{}$_ do

${FirstEl\\_properties}\leftarrow\emptyset$

foreach  _$\textit{el}\in\textit{row}{}$_ do

${label}\leftarrow\textit{el.value}$

if  _( $el=0$)_ then

${FirstEl\\_properties}\leftarrow GetProperties({label,ColsContext})$

end if

if  _( $\textit{Prop-lookup}\ (\textit{label})\not=\emptyset$)_ then

$annotate(T^{\prime}.row.i.el,candidates.value)$

else

${candidates}\leftarrow\textit{wd-lookup}\ (\textit{label,ColsContext})$

if  _( $candidates.size=1$)_ then

$annotate(T^{\prime}.row.i.el,candidates.value)$

else if  _( $candidates.size=0$)_ then

${new\\_label}\leftarrow\textit{spell-check}\ (\textit{label})$

${candidates}\leftarrow\textit{wd-lookup}\ (\textit{new\\_label,ColsContext})$

if  _( $candidates.size=1$)_ then

$annotate(T^{\prime}.row.i.el,candidates.value)$

end if

end if

end if

end foreach

end foreach

Algorithm 2 Algorithm of CEA processing task

##  4 Experimental Results

The evaluation of AMALGAM is done in the context of the SemTab 2020
challenge121212http://www.cs.ox.ac.uk/isg/challenges/sem-tab/2020/index.html.
This challenge is subdivided into 4 successive rounds containing respectively
34294, 12173, 62614 and 22387 CSV tables to annotate. For example, Table 2,
lists all Alberta towns with additional information such as the country and
the elevation above sea level. The evaluation metrics are respectively the F1
score and the Precision [26].

Table 2: List of Alberta towns, extracted from SemTab Round 1. col0 | col1 | col2 | col3 | col4 | col5
---|---|---|---|---|---
Grande Prairie | city in Alberta | canada | Sexsmith | 650 | Alberta
Sundre | town in Alberta | canada | Mountain View County | 1093 | Alberta
Peace River | town in clberta | Canada | Northern Sunrise County | 330 | Alberta
Vegreville | town in Alberta | canada | Mundare | 635 | Alberta

Tables 3, 4, 5 and 6 report the evaluation of CTA and CEA respectively for
round 1, 2, 3 and 4. Thus, it could be observed that AMALGAM handles properly
the two tasks, in particular in the CEA task. Regarding the CTA task, these
results can be explained with a new revision of Wikidata created in the item
revision history and there are possibly spelling errors in the contents of the
tables. For instance, ”rural district of Lower Saxony” became ”district of
Lower Saxony” after the 16th April 2020 revision. A possible solution to this
issue is to retrieve the history of the different revisions, by parsing
Wikidata data history dumps, to use them in the lookup. This is a possible
extension to this work. Another observed issue is that spelling errors impacts
greatly the lookup eﬃciency.

Table 3: Results of Round 1. TASK | F1 Score | Precision
---|---|---
CTA | 0.724 | 0.727
CEA | 0.913 | 0.914

Table 4: Results of Round 2. TASK | F1 Score | Precision
---|---|---
CTA | 0.926 | 0.928
CEA | 0.921 | 0.927

Table 5: Results of Round 3. TASK | F1 Score | Precision
---|---|---
CTA | 0.869 | 0.873
CEA | 0.877 | 0.892

Table 6: Results of Round 4. TASK | F1 Score | Precision
---|---|---
CTA | 0.858 | 0.861
CEA | 0.892 | 0.914

From the round 1 experience, we specifically focused on the spell check
process of items to improve the results of the CEA and CTA tasks in round 2.
Two API services, from Wikipedia and Gurunudi (presented in Sect. 3.)
respectively were used for spelling correction. According to the results in
Table 4, both F1-Score and Precision have been improved. From these rounds, we
observed that term with a single word is often ambiguous as it may refer to
more than one entity. In Wikidata, there is only one article (one entry) for
each concept. However, there can be many equivalent titles for a concept due
to the existence of synonyms, etc. These synonymy and ambiguity issues make it
difficult to match the correct item. For example, the term “Paris” may refer
to various concepts such as “the capital and largest city of France“, “son of
Priam, king of Troy“, “county seat of Lamar County, Texas, United States“.
This leads us to introduce a disambiguation process during rounds 3 and 4. For
these two last rounds, we have updated the annotation algorithm by integrating
the concept of the column obtained during the CTA task in the linking phase.
We showed that the two tasks can be performed relatively successfully with
AMALGAM, achieving higher than 0.86 in precision. However, the automated
disambiguation process of items proved to be a more challenging task.

##  5 Conclusion and Future Works

In this paper, we described AMALGAM, a matching approach to enabling tabular
datasets to be FAIR compliant by making them explicit thanks to their
annotation using a knowledge graph, in our case Wikidata. Its advantage is
that it allows to perform both CTA and CEA tasks in a timely manner. These
tasks can be accomplished through the combination of lookup services and a
spell check techniques quickly. The results achieved in the context of the
SemTab 2020 challenge show that it handles table annotation tasks with a
promising performance. Our findings suggest that the matching process is very
sensitive to errors in spelling. Thus, as of future work, an improved spell
checking techniques will be investigated. Further, to process such errors the
contextual based spell-checkers are needed. Often the string is very close in
spelling, but context could help reveal which word makes the most sense.
Moreover, the approach will be improved through finding a trade-off between
effectiveness and efficiency.

## References

  * [1] Wilkinson, M., Dumontier, M., Aalbersberg, I. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3, 160018 (2016).
  * [2] Wilkinson, M.-D., et all.: The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data 3(1), 1–9 (2016).
  * [3] Diallo G., Simonet M., Simonet A. (2006) An Approach to Automatic Ontology-Based Annotation of Biomedical Texts. In: Ali M., Dapoigny R. (eds) Advances in Applied Artificial Intelligence. IEA/AIE 2006. Lecture Notes in Computer Science, vol 4031. Springer, Berlin, Heidelberg.
  * [4] Dramé, K., Mougin F., Diallo, G. Large scale biomedical texts classification: a kNN and an ESA-based approaches. J. Biomedical Semantics. Vol:7(40), 2016.
  * [5] Handschuh, S.: Semantic Annotation of Resources in the Semantic Web. Semantic Web Services. Springer, Berlin, Heidelberg, 135–155 (2007).
  * [6] Diallo, G. ”Efficient Building of Local Repository of Distributed Ontologies,” 7th IEEE International Conference on SITIS, Dijon, 2011, pp. 159-166.
  * [7] Subramanian, A., Srinivasa, S.: Semantic Interpretation and Integration of Open Data Tables. In: Geospatial Infrastructure, Applications and Technologies: India Case Studies, pp. 217–233. Springer Singapore (2018).
  * [8] Taheriyan, M., Knoblock, C.-A., Szekely, P., Ambite, J.-L.: Learning the semantics of structured data sources. Web Semantics: Science, Services and Agents on the World Wide Web 37(38), 152–169 (2016)
  * [9] Zhang, L., Wang, T., Liu, Y., Duan, Q.: A semi-structured information semantic annotation method for Web pages. Neur. Comp. and App. 32(11), 6491–6501(2019)
  * [10] Ji, S., Pan, S., Cambria, E., Marttinen, P., Yu, P.S.: A survey on knowledge graphs: Representation, acquisition and applications. CoRRabs/2002.00388(2020)
  * [11] Efthymiou, V., Hassanzadeh, O., Rodriguez-Muro, M., Christophides, V.: Matching Web Tables with Knowledge Base Entities: From Entity Lookups to Entity Embeddings. In: LNCS, pp. 260–277. Springer Int. Publishing (2017).
  * [12] Eslahi, Y., Bhardwaj, A., Rosso, P., Stockinger, K., Cudre-Mauroux, P.: Annotating Web Tables through Knowledge Bases: A Context-Based Approach. In: 2020 7th Swiss Conference on Data Science (SDS), pp. 29–34. IEEE (2020).
  * [13] Hassanzadeh, O., Efthymiou, V., Chen, C., Jimenez-Ruiz, E., Srinivas, K.: SemTab2020: Semantic Web Challenge on Tabular Data to Knowledge Graph Matching - 2020 Data Sets, October 2020.
  * [14] Hassanzadeh, O., Efthymiou, V., Chen, C., Jimenez-Ruiz, E., Srinivas, K.: SemTab2019: Semantic Web Challenge on Tabular Data to Knowledge Graph Matching - 2019 Data Sets (Version 2019).
  * [15] Nguyen, P., Kertkeidkachorn, N., Ichise, R., Takeda, H.: MTab: Matching Tabular Data to Knowledge Graph using Probability Models. Proceedings of the SemTab Challenge co-located with the 18th ISWC conference, 2019.
  * [16] Chabot, Y., Labbe, T., Liu, J., Troncy, R.: DAGOBAH: An End-to-End Context-Free Tabular Data Semantic Annotation System. Proceedings of the SemTab Challenge co-located with the 18th ISWC conference, 2019.
  * [17] Oliveira, D., Aquin, M.: ADOG-Annotating Data with Ontologies and Graphs. Proc. of the SemTab Challenge co-located with the 18th ISWC conference, 2019.
  * [18] Thawani, A., Hu, M., Hu, E., Zafar, H., Divvala, N-.T., Singh, A., Qasemi, E., Szekely, P., Pujara, J.: Entity Linking to Knowledge Graphs to Infer Column Types and Properties. Proc. of the SemTab Challenge co-located with ISWC’19, 2019..
  * [19] Zhang, Z.: Effective and efficient Semantic Table Interpretation using TableMiner+.Semantic Web IOS Press 8(6), 921–-957 (2017).
  * [20] Takeoka, K., Oyamada, M., Nakadai, S., Okadome, T.: Meimei: An Efficient Probabilistic Approach for Semantically Annotating Tables. Proceedings of the AAAI Conference on Artificial Intelligence 33, 281–288 (2019).
  * [21] Chen, J., Jimenez-Ruiz, E., Horrocks, I., Sutton, C.: Learning Semantic Annotations for Tabular Data. Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI-19), 2088–2094 (2019).
  * [22] Ermilov, I., Ngomo, AC.N.: TAIPAN: Automatic Property Mapping for Tabular Data. In: Blomqvist E., Ciancarini P., Poggi F., Vitali F. (eds) Knowledge Engineering and Knowledge Management. EKAW 2016. Lecture Notes in Computer Science, vol 10024. Springer, Cham.
  * [23] Ritze, D., Lehmberg, O., Bizer, C.: Matching HTML Tables to DBpedia. In: Proceedings of the 5th International Conference on Web Intelligence, Mining and Semantics - WIMS '15, pp. 1–-6. ACM Press (2015).
  * [24] Bhagavatula, C.-S., Noraset, T., Downey, D.: TabEL: Entity Linking in Web Tables. In: Proceedings of the The Semantic Web - ISWC 2015, Springer International Publishing, pp. 425–441 (2015).
  * [25] Shashank, S., Shailendra, S.: Systematic review of spell-checkers for highly inflectional languages. Artificial Intelligence Review 53(6), 4051–4092 (2019)
  * [26] Jiménez-Ruiz, E., Hassanzadeh, O., Efthymiou, V., Chen, J., Srinivas, K.: SemTab 2019: Resources to Benchmark Tabular Data to Knowledge Graph Matching Systems. In: Harth A. et al. (eds) The Semantic Web. ESWC (2020).