# DCoM: A Deep Column Mapper for Semantic Data Type Detection

Subhadip Maji
Data Scientist
Optum Global Solutions
Bangalore, India 560103
<EMAIL_ADDRESS>
&Swapna Sourav Rout
Senior Data Scientist
Optum Global Solutions
Bangalore, India 560103
<EMAIL_ADDRESS>
&Sudeep Choudhary
Data Scientist
Optum Global Solutions
Bangalore, India 560103
<EMAIL_ADDRESS>

###### Abstract

Detection of semantic data types is a very crucial task in data science for
automated data cleaning, schema matching, data discovery, semantic data type
normalization and sensitive data identification. Existing methods include
regular expression-based or dictionary lookup-based methods that are not
robust to dirty as well unseen data and are limited to a very less number of
semantic data types to predict. Existing Machine Learning methods extract
large number of engineered features from data and build logistic regression,
random forest or feedforward neural network for this purpose. In this paper,
we introduce DCoM, a collection of multi-input NLP-based deep neural networks
to detect semantic data types where instead of extracting large number of
features from the data, we feed the raw values of columns (or instances) to
the model as texts. We train DCoM on 686,765 data columns extracted from
VizNet corpus with 78 different semantic data types. DCoM outperforms other
contemporary results with a quite significant margin on the same dataset.

_K_ eywords Semantic Data Type Detection $\cdot$ Machine Learning $\cdot$
Natural Language Processing $\cdot$ Semantic Column Tagging $\cdot$ Sensitive
Data Detection $\cdot$ Column Search

##  1 Introduction

Robust data preprocessing pipeline enables organizations to take data driven
business decisions. Most unstructured and semi-structured data after initial
pre-processing is available in tabular format for further processing. These
tabular datasets have to go through a process of data curation and quality
check before consumption. Companies apply standard data quality checks/rules
on business-critical columns to access and maintain the quality of data.
Additionally, some of these datasets may contain sensitive information e.g.
Protected Health Information (PHI), Personally Identifiable Information(PII),
etc and need to be masked/de-identified i.e. identifying all columns which are
Social Security Numbers, First Names, telephone numbers, etc. The first step
in this data curation and quality check process is column level semantic
tagging/mapping. There have been many attempts to automate this process [9,
13, 8, 12, 6, 20]. Traditionally, Semantic tagging is usually done using
handcrafted rule-based systems[4, 9, 16]. In some cases, column descriptions
are also available. However, as the data grows exponentially, the cost
associated with maintaining rule-based system also increases. The semantic
data type tagging is still largely manual where data stewards manually scan
through databases and map columns of interest. Most ML based approaches [20,
6, 11] use corresponding metadata, atomic datatypes, column description along
with handcrafted features as input data for training the model.

We herein, propose DCoM-Deep Column Mapper; a generic collection of Deep
learning based Semantic mappers to map the columns to semantic data types.
These model take the raw values of the columns (or instances) as inputs
considering those as texts and build NLP-based deep learning architectures to
predict the semantic type of the columns (or instances). We also extracted 19
engineered features and used those as auxiliary features to DCoM models. We
refrained ourselves extracting large number of features, because we wanted to
make the DCoM models learn those features along with some interesting high
level features on their own, for better semantic data type detection. As we
are exploring NLP in this problem, we make the leverage of using advanced NLP-
based deep learning layers or models such as Bi-LSTM, BERT, etc.

##  2 Related Work

Trifacta [16], Microsoft Power BI [9] and Google Data Studio [4] use some
regular expression based patterns for dictionary looks ups of column headers
and values to detect a limited number of semantic data types. Venetis et al.
[17] build a database of value-type mappings, then assign semantic types using
a maximum likelihood estimator based on column values. Syed et al. [15] use
column values and headers to build a Wikitology query to map columns to
semantic classes. Ramnandan et al. [13] use heuristics to first separate
numerical and textual types, then describe those types using the Kolmogorov-
Smirnov (K-S) test and Term Frequency-Inverse Document Frequency (TF-IDF),
respectively. Pham et al. [11] use slightly more features, including the Mann-
Whitney test for numerical data and Jaccard similarity for textual data, to
train logistic regression and random forest models. Goel et al. [3] use
conditional random fields to predict the semantic type of each value within a
column, then combine these predictions into a prediction for the whole column.
Limaye et al. [8] use probabilistic graphical models to annotate values with
entities, columns with types, and column pairs with relationships. Puranik
[12] proposes a “specialist approach” combining the predictions of regular
expressions, dictionaries, and machine learning models. More recently, Yan and
He [19] introduced a system that, given a search keyword and set of positive
examples, synthesizes type detection logic from open source GitHub
repositories. Hulsebos et al. [6] built a multi-input deep neural network
model for detecting 78 semantic data types, extracting 686, 765 data columns
from VizNet [5] corpus. They extracted a total of 1,588 features for each
column to train the model, thus resulting a support-weighted F1 score of 0.89,
exceeding that of machine learning baselines, dictionary and regular
expression benchmarks, and the consensus of crowdsourced annotations. Zhang et
al. [20] introduced SATO, a hybrid machine learning model to automatically
detect the semantic types of columns in tables, exploiting the signals from
the context as well as the column values. It combines a deep learning model
trained on a large-scale table corpus with topic modeling and structured
prediction to achieve supportweighted and macro average F1 scores of 0.925 and
0.735, respectively, exceeding the state-of-the-art performance by a
significant margin.

While in this paper we do not experiment with the context information of the
column values, our work is mostly aligned to Hulsebos et al. [6]. Keeping that
in mind, we are planning to research and implement context of the column
values in tables in our DCoM models for our future works. The paper is
organized as follows: Section 3 describes the data used for DCoM models. In
Section 4 we present the details of data preparation and architecture for DCoM
models. Section 5 contains the training, evaluation and inference procedures
of DCoM models while Section 6 discussed about the results of extensive
experiments. In Section 7, we talk about some known limitations from the data
as well as application point of view and finally in Section 8, we present the
concluding remarks and some future directions.

##  3 Data

We have used the dataset prepared by Hulsebos et al. [6] and compared our
model performance with them considering their results to be baselines. This
dataset contains 686,765 instances with 78 unique classes (or semantic data
types). It is divided into 60/20/20 training/validation/testing splits. The
instance count for classes varies from 584 (affiliate class) to 9088 (type
class). The count distribution for classes is shown in Figure 1. To provide a
more clear picture of the data a sample of the dataset is shown in Table 1.

Figure 1: Count of instances for each of the 78 classes in the dataset [6]. Instance | Class
---|---
1, 2, 3, 4, 5, 6, 7, 8 | day
4:59, 2:44, 2:04, 2:05, 1:13, 3:14 | duration
Education, Poverty, Unemployment, Employment | category
c;, end-code, code name, label name | command
31 years, 22 years, –, 39 years, 24 years | age
1, 2 | position
LA, CA, AL, Warwickshire] | state
Deletes the property, Lets you edit the value of the property, Script execution will be stopped | description
Table 1: A sample of the dataset.

##  4 Proposed Method

Prior approaches [17, 8] to semantic type detection trained models, such as
logistic regression, decision trees, feedforward neural network [6],
extracting various features from the data. We, on the other hand, treated the
data as natural language (text) and used the data itself as the input to the
model. We used very small number of hand-engineered features unlike Hulsebos
et al. [6]. Therefore, our DCoM models have two inputs, the values of the
instances as text or natural language and hand-engineered features. We present
two types of DCoM models based on the way we feed the text input to the model.

###  4.1 DCoM with Single Sequence Input

In this subsection, we discuss how the values of each instance can be fed to
the model as a single sequence input to the DCoM. Considering the scenario, we
can not simply pass the inputs separating values of each input with any
separator token. This gives the model wrong information about the sequence of
the data resulting faulty training and poor performance on the unseen data.
For example, if we consider the last example from Table 1 and create the
input, Deletes the property <SEP> Lets you edit the value of the property
<SEP> Script execution will be stopped for the model, the model gets to learn
that the value Lets you edit the value of the property has relative position
between Deletes the property and Script execution will be stopped, which is
quite wrong, because the values in an instance does not have any relative
position between them.

To mitigate the problem we introduce permutations from mathematics. With
permutation we order $r$ items from the set of $n_{i}$ items. Here $n_{i}$
items are all the values of an instance $i$, and $r\in[1,n]$. Doing so, the
above instance can be broken down to multiple subsets. A sample of the subset
is shown in the Table 2. If we feed all the new instances of the subset to the
model, it does not learn any relative positional information of Lets you edit
the value of the property with respect to other values in that instance unlike
earlier. Therefore, for the value Deletes the property, the model only learns
the relative positions of the tokens e.g. Deletes, the, property, etc in a
value, but not the relative position of the values in an instance. This helps
the model getting the actual information present in the data for accurate
prediction on the output. This permutation method also helps in augmenting new
instances which helps in training the data hungry deep learning models with
enriched data. It is not feasible to generate all the possible permutations
before the training because of the huge numbers of subsets. Instead, during
training, we sample $r$ between $1$ and $n$ for each of the instances and
generate one permutation for each instance. More than one instances can be
generated, but training the model for multiple epochs will result same for
both cases.

New Instance class Deletes the property description Lets you edit the value of
the property description Script execution will be stopped description Deletes
the property <SEP> Lets you edit the value of the property description Lets
you edit the value of the property <SEP> Script execution will be stopped
description Deletes the property <SEP> Lets you edit the value of the property
<SEP> Script execution will be stopped description Deletes the property <SEP>
Script execution will be stopped <SEP> Lets you edit the value of the property
description Lets you edit the value of the property <SEP> Deletes the property
<SEP> Script execution will be stopped description

Table 2: Permutations on the values of a single instance from class
description.

###  4.2 DCoM with Multiple Sequences Inputs

Input preparation with this method is straight-forward compared to the earlier
method. In this method, we also use permutations to generate new instances,
but the value of $r$ is fixed during training and it is used as a hyper-
parameter, where $r\in[1,\inf)$. Once we decide the value of $r$, $r$ number
of inputs are used as text inputs to the model. Aggregation of embedding
vectors of the inputs are performed once they are generated using the shared
embedding weights. If $r>n$, where $n$ is the number of values of an instance,
then this scenario can be handled in two ways. The first way is to pad $r-n$
inputs and aggregate the embedding vectors only for the non-padded inputs.
Another way is, while generating new instances use permutation with
replacement to always sample $r$ values out of $n$ values. Therefore, padding
is not required for the latter method. We tried both the approaches and did
not observe any significant difference in the result.

We used 19 out of 27 global statistical features [6] as our engineered
features for the DCoM models. These features are normalized before feeding to
the model. The complete list of these features are shown in Table 6. Once text
and engineered inputs are prepared, we attach LSTM/Transformer/BERT layers to
the text input. The output of the same is aggregated with the engineered
inputs. We use some feed-forward layer after that. Finally we use one softmax
layer with 78 units to get the probability of each of the classes as output.
This is our generalized architecture design of DCoM models. Extensive hyper-
parameter tuning is performed to finalize the number of layers, number of
units in a layer and many other hyper-parameters in the model. This topic is
discussed in detail in section 5. To name the DCoM models, type of the text
input and name of the deep learning layers are used as suffixes with the name
DCoM, e.g. DCoM model with single instance input and LSTM layers is named as
DCoM-Single-LSTM. The architecture design of DCoM models for both single
sequence and multiple sequences inputs are shown in the Figure 2.

Figure 2: Architecture diagram of (a) DCoM-Single and (b) DCoM-Multi models.

##  5 Training, Evaluation and Inference

We trained our model on train dataset, validated it on validation dataset and
finally reported our results on the test dataset. As class imbalance is
present, like Hulsebos et al. [6], we also evaluated our model performance
using the average F1-score, weighted by the number of columns per class in the
test set (i.e., the support).

The DCoM models are trained in Tensorflow 2 [1] and the hyper-parameters are
tuned using keras-tuner[10]. For inputs of DCoM models we tried permutations
with replacement as well as without replacement, but we did not observe any
significant difference in outputs. The value of $r$ for which we observed best
performance for DCoM-Multi models is $45$. For tokenizing text inputs, we
experimented with character based, word based and BERT Wordpiece[18]
tokenizers, and we found out BERT Wordpiece tokenizer stood out to be working
better with respect to the other tokenizers because of the obvious reasons
stated in the paper[18]. We compared our result between with and without pre-
trained embedding weights. It is observed that, though in the final output
there is no considerable difference, training with pre-trained embedding
weights take approximately 40% less amount of time to converge. We
experimented with the small, base and large variations of several BERT
architectures and finalized DistilBERT-base[14] and Electra-small[2] based on
their performances. We used Bi-directional LSTM layers in all the DCoM-LSTM
variants. For Dropout layers we used $0.3$ as our dropout rate. We
experimented mean, sum, concatenation and weighted sum functions for
aggregation, but there was not any significant difference in outputs based on
these variations. We used Adam[7] optimizer with initial learning rate
$10^{-4}$. Along with this, we implement learning rate reducer with factor
0.5, which reduces the learning rate by 50% if the model performance does not
improve on validation dataset after consecutive 5 epochs. Assigning class
weights does not have much effect in the test results. As this is a multi-
class (78 classes) classification problem, the DCoM models are trained with
categorical-crossentropy loss and validated with accuracy and average F1 score
metric.

we tried two approaches for inference on the test dataset. In the first
approach we performed single time inference on each of the instances of the
test dataset. While doing so, we set $r$ to be the total number of values
($n_{i}$) for each instance for DCoM-Single models to perform inference. For
DCoM-Multi, as $r$ values if prefixed, some values will be truncated for
inputs with total values $n>r$. The other approach is to generate $k$ (where,
$k>1$) instances with permutation, sampling $r$ values $k$ times between 1 and
$n_{i}$, where $n_{i}$ is the total number of values for instance $i$.
Therefore, we get $k$ class predictions for each of the instances and finally
with majority voting we pick the prediction class for each instance. We used
$k$ value to be 10 in our case and it is observed that for both DCoM-Single
and DCoM-Multi, we observed $0.2-0.5\%$ improvement in the test average F1
score, but this improvement comes with the price of increased inference time
by approximately $k$ times.

##  6 Results

We compared DCoM against Sherlock and other types of models shown by Hulsebos
et al. [6] assuming those to be our baseline models. Table 3 presents the
comparison of results on test dataset for each of the models. Columns
Engineered Features and k are specific to DCoM models. Engineered Features
says whether the 19 engineered features are used with the text inputs while
training the model. k column tells the number of times inference is performed
on a single test instance. It is discussed in detail in section 5. Runtime
column is the average time in seconds needed to extract features and generate
a prediction for a single sample, and Size column reports the space required
by the trained models in MB. From the table it is seen that several DCoM
models outperform Sherlock [6] in both F1 score and inference runtime with
significant margins.

Hulsebos et al. [6] extracted various features e.g. global statistics,
character-level distributions, word embeddings, paragraph vectors from the
data and used those features to fit a feedforward neural network model. On the
other hand, we treated the data as texts and feed those to more advanced NLP-
based models. This allows the DCoM models to extract and learn more useful
features that Hulsebos et al. [6] were unable to extract using hand-
engineering. Along with this, hand-engineering takes considerable amount of
time to calculate the features which increases the inference time of Sherlock
[6] by 3 to 20 times with respect to DCoM models.

Method | Engineered Features | k | F1 Score | Runtime (s) | Size (MB)
---|---|---|---|---|---
DCoM-Single-LSTM | Yes | 1 | 0.895 | 0.019 $\pm$ 0.01 | 112.1
DCoM-Single-LSTM | Yes | 10 | 0.898 | 0.152 $\pm$ 0.01 | 112.1
DCoM-Single-LSTM | No | 1 | 0.871 | 0.018 $\pm$ 0.01 | 97.8
DCoM-Single-LSTM | No | 10 | 0.877 | 0.141 $\pm$ 0.01 | 97.8
DCoM-Multi-LSTM | Yes | 1 | 0.878 | 0.046 $\pm$ 0.01 | 4.7
DCoM-Multi-LSTM | Yes | 10 | 0.881 | 0.416 $\pm$ 0.01 | 4.7
DCoM-Multi-LSTM | No | 1 | 0.869 | 0.044 $\pm$ 0.01 | 4.6
DCoM-Multi-LSTM | No | 10 | 0.871 | 0.401 $\pm$ 0.01 | 4.6
DCoM-Single-DistilBERT | Yes | 1 | 0.922 | 0.162 $\pm$ 0.01 | 268.2
DCoM-Single-DistilBERT | Yes | 10 | 0.925 | 1.552 $\pm$ 0.01 | 268.2
DCoM-Single-DistilBERT | No | 1 | 0.901 | 0.158 $\pm$ 0.01 | 202.3
DCoM-Single-DistilBERT | No | 10 | 0.904 | 1.492 $\pm$ 0.01 | 202.3
DCoM-Single-Electra | Yes | 1 | 0.907 | 0.093 $\pm$ 0.01 | 53.1
DCoM-Single-Electra | Yes | 10 | 0.909 | 0.894 $\pm$ 0.01 | 53.1
DCoM-Single-Electra | No | 1 | 0.890 | 0.092 $\pm$ 0.01 | 45.7
DCoM-Single-Electra | No | 10 | 0.892 | 0.887 $\pm$ 0.01 | 45.7
Sherlock[6] | - | - | 0.890 | 0.42 $\pm$ 0.01 | 6.2
Decision tree[6] | - | - | 0.760 | 0.26 $\pm$ 0.01 | 59.1
Random Forest[6] | - | - | 0.840 | 0.26 $\pm$ 0.01 | 760.4
Dictionary[6] | - | - | 0.160 | 0.01 $\pm$ 0.03 | 0.5
Regular expression[6] | - | - | 0.040 | 0.01 $\pm$ 0.03 | 0.01
Consensus[6] | - | - | 0.320 | 33.74 $\pm$ 0.86 | -
Table 3: Support-weighted F1 score, runtime at prediction, and size of DCoM
and other benchmarks

###  6.1 Performance for Individual Types

Following Hulseboset al. [6], we also prepared Table 4 which displays the top
and bottom five types, as measured by the F1 score achieved by the best
performing DCoM model, DCoM-Single-DistilBERT for single inference ($k=1$) for
that type. . High performing classes such as grades, industry, ISBN, etc.
contain a finite set of valid values which help DCoM models to extract
distinctive features with respect to the other classes. To understand types
for which DCoM-Single-DistilBERT performs poorly, we include incorrectly
predicted examples for the lowest precision type (Rank) and the lowest recall
type (Ranking) in Table 5. From the table it is observed that purely numerical
values or values appearing in multiple classes, cause a challenge for the DCoM
models to correctly classify the semantic type of the data. This issue is
discussed in detail in section 7.

Type F1 Score Precision Recall Support ISBN 0.988 0.999 0.993 1430 Grades
0.988 0.995 0.992 1765 Birth Date 0.987 0.985 0.986 479 Jockey 0.983 0.988
0.986 2817 Industry 0.984 0.984 0.984 2958

(a) Top 5 types by F1 Score

Type F1 Score Precision Recall Support Person 0.813 0.596 0.688 579 Rank 0.586
0.764 0.663 2978 Director 0.702 0.549 0.616 225 Sales 0.667 0.435 0.526 322
Ranking 0.695 0.348 0.464 439

(a) Bottom 5 types by F1 Score Table 4: Top five and bottom five types by F1
score on the test dataset for DCoM-Single-DistilBERT model variant.

Examples True Type Predicted Type 1, 5, 4, 3, 2 Day Rank 1, 2, 3, 4, 5, 6, 7,
8, 9, 10 Region Rank 1, 2, 3 Position Rank

(a) Low Precision

Examples True Type Predicted Type 41, 2, 36 Ranking Rank 0, 2, 4 Ranking Plays
1, 2, 3, 4, 5, 6, 7, 8, 9, 10 Ranking Rank

(a) Low Recall Table 5: Examples of low precision and low recall types on the
test dataset for DCoM-Single-DistilBERT model variant.

###  6.2 Feature Importance

To calculate the feature importance of 19 engineered features used in trained
DCoM-Single-DistilBERT model, we extracted the learned feature weight matrix,
$W$ from the dense layer used after the engineered features input, where $W\in
R^{19\times D}$. Here $D$ is the number of units used in the dense layer. $W$
contains the weights/contribution of each of the 19 features on each of the
$D$ units, thus forming a $19\times D$ array, which contains both positive and
negative values based on the direction of contribution. We take absolute value
of all the elements of $W$, as we are only interested on the amount of
contribution of each of the engineered features, not the direction. After
that, we take mean across the 19 features that results a 19-dimensional array.
We normalized the array by dividing the maximum value of the array to each of
the elements. Table 6 enlists the importance of the engineered features in
decreasing order. From the table it is seen that Std of # of Numeric
Characters in Cells, Std of # of Alphabetic Characters in Cells and Entropy
are the top 3 important features, where Mean # Alphabetic Characters in Cells,
Median Length of Values and Mode Length of Values are the least 3 important
features for the DCoM-Single-DistilBERT model.

Rank | Aux Features | Score
---|---|---
1 | Std of # of Numeric Characters in Cells | 1.00
2 | Std of # of Alphabetic Characters in Cells | 0.63
3 | Entropy | 0.63
4 | Std of # of Special Characters in Cells | 0.62
5 | Std of # of Words in Cells | 0.53
6 | Mean # Words in Cells | 0.50
7 | Mean # of Numeric Characters in Cells | 0.48
8 | Minimum Value Length | 0.45
9 | Kurtosis of the Length of Values | 0.41
10 | Mean # Special Characters in Cells | 0.39
11 | Number of Values | 0.34
12 | Fraction of Cells with Alphabetical Characters | 0.33
13 | Fraction of Cells with Numeric Characters | 0.31
14 | Sum of the Length of Values | 0.31
15 | Maximum Value Length | 0.30
16 | Skewness of the Length of Values | 0.28
17 | Mean # Alphabetic Characters in Cells | 0.28
18 | Median Length of Values | 0.27
19 | Mode Length of Values | 0.20
Table 6: Feature importance of 19 engineered features used in DCoM-Single-
DistilBERT model.

##  7 Known Limitations

The major limitation of the process is with the pre-processed data prepared by
Hulsebos et al. [6] used to train DCoM models. Same values of instances are
present in multiple classes, thus resulting strong class overlapping among
some classes. It makes the models confusing during training to learn
distinctive features for proper classification. Therefore, for some classes in
the test dataset, the models perform very poorly. Table 7 presents a sample of
examples of class overlapping in the data prepared by Hulsebos et al. [6]. It
is to be noted that the proportion of this class overlap is considerably large
in numbers with respect to the total number of instances. Therefore, it
affects the training as well as the model performance on the test dataset by a
significant margin. Besides the class overlap, faulty or wrong values are
present in some classes.

From the application point of view of semantic data type detection models,
identifying and preparing a good dataset for training is very challenging as
well as time consuming. Non standardized column names are a major challenge
large organizations face in doing semantic detection. For example, social
security numbers can be called as ssn, ssn_id, soc_sec_bnr etc. Non
standardized data columns also pose major challenge in information ambiguity.
For example, Gender column can have values Male, Female, unknown, or 0,1,2.
Some organization can have mixed attributes which poses a major challenge in
data security. For example, use of PII data like Social security Number in
Customer ID columns. Some other challenges like corrupt/missing metadata also
exist.

Values | Class
---|---
F, M | gender
M | sex
F | sex
M | gender
M, F | sex
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 | ranking
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 | position
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 | rank
1, 3, 7, 10, 12, 15, 18 | position
130, 109, 50, 55, 29 | rank
65, 1, 39, 7, 41, 9, 106, 76, 12, 116 | ranking
Fresno State, Oregon, UCLA, South Carolina | team
Southern Maryland, yucatan | team name
Michigan, Indiana, Wisconsin, Purdue | team name
Austria | team
Table 7: A sample example of class overlapping of the data used to build up
models

##  8 Conclusion

DCoM presents a novel permutation based method by which the the instances can
be fed to the deep learning models directly as natural language. This takes
the leverage of using more advanced NLP-based layers/models unlike feedforward
neural network in Sherlock [6]. The permutation based method also helps in
generating large number of new instances from the existing ones, helping DCoM
models to boost its performance effectively, thus outperforming Sherlock [6]
and other recent works by quite a significant margin in both F1 score as well
as inference time. We also present an ensembling approach during inference
time which improves the test average F1 score by $0.2-0.5\%$..

As mentioned earlier, the next steps of it to introduce context of the column
values to DCoM models while predicting the semantic types in relational
tables.

## Acknowledgement

We would like to thank Optum Global Solutions, India for sponsoring this work.
We are thankful to the author of paper Sherlock [6], Madelon Hulsebos and
others for helping us arranging the dataset. We would also like to thank
Vineet Shukla and Ravi Kumar Gottumukkala for their valuable inputs which help
us enhancing the quality of this paper significantly.

## References

  * [1] Martín Abadi et al.  “TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems” Software available from tensorflow.org, 2015  URL: https://www.tensorflow.org/
  * [2] Kevin Clark, Minh-Thang Luong, Quoc V. Le and Christopher D. Manning  “ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators”  In _arXiv e-prints_ , 2020, pp. arXiv:2003.10555  arXiv:2003.10555 [cs.CL]
  * [3] Aman Goel, Craig A. Knoblock and Kristina Lerman  “K.: Exploiting Structure within Data for Accurate Labeling Using Conditional Random Fields”  In _In: Proceedings of the 14th International Conference on Artificial Intelligence (ICAI_ , 2012
  * [4] Google  “Google Data Studio”, 2019  URL: https://datastudio.google.com
  * [5] Kevin Hu et al.  “VizNet: Towards a large-scale visualization learning and benchmarking repository”  In _Proceedings of the 2019 Conference on Human Factors in Computing Systems (CHI)_ ACM, 2019
  * [6] Madelon Hulsebos et al.  “Sherlock: A Deep Learning Approach to Semantic Data Type Detection”  In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_ ACM, 2019
  * [7] Diederik P. Kingma and Jimmy Ba  “Adam: A Method for Stochastic Optimization”  In _arXiv e-prints_ , 2014, pp. arXiv:1412.6980  arXiv:1412.6980 [cs.LG]
  * [8] Girija Limaye, Sunita Sarawagi and Soumen Chakrabarti  “Annotating and Searching Web Tables Using Entities, Types and Relationships”  In _Proc. VLDB Endow._ 3.1–2 VLDB Endowment, 2010, pp. 1338–1347  DOI: 10.14778/1920841.1921005
  * [9] Microsoft  “Power BI, Interactive Data Visualization BI”, 2019  URL: https://powerbi.microsoft.com
  * [10] Tom O’Malley et al.  “Keras Tuner”, https://github.com/keras-team/keras-tuner, 2019
  * [11] Minh Pham, Suresh Alse, Craig Knoblock and Pedro Szekely  “Semantic Labeling: A Domain-Independent Approach”, 2016, pp. 446–462  DOI: 10.1007/978-3-319-46523-4_27
  * [12] N. Puranik  “A specialist approach for the classification of column data”, 2012
  * [13] S.K. Ramnandan, Amol Mittal, Craig A. Knoblock and Pedro Szekely  “Assigning Semantic Labels to Data Sources”  In _The Semantic Web. Latest Advances and New Domains_ Cham: Springer International Publishing, 2015, pp. 403–417
  * [14] Victor Sanh, Lysandre Debut, Julien Chaumond and Thomas Wolf  “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter”  In _arXiv e-prints_ , 2019, pp. arXiv:1910.01108  arXiv:1910.01108 [cs.CL]
  * [15] Zareen Syed, Timothy W. Finin, Varish Mulwad and A. Joshi  “Exploiting a Web of Semantic Data for Interpreting Tables”, 2010
  * [16] Trifacta  “Data Wrangling Tools & Software”, 2019  URL: https://trifacta.com
  * [17] Petros Venetis et al.  “Recovering Semantics of Tables on the Web”  In _Proc. VLDB Endow._ 4.9 VLDB Endowment, 2011, pp. 528–538  DOI: 10.14778/2002938.2002939
  * [18] Yonghui Wu et al.  “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation”  In _arXiv e-prints_ , 2016, pp. arXiv:1609.08144  arXiv:1609.08144 [cs.CL]
  * [19] Cong Yan and Yeye He  “Synthesizing Type-Detection Logic for Rich Semantic Data Types Using Open-Source Code”  In _Proceedings of the 2018 International Conference on Management of Data_ , SIGMOD ’18  Houston, TX, USA: Association for Computing Machinery, 2018, pp. 35–50  DOI: 10.1145/3183713.3196888
  * [20] Dan Zhang et al.  “Sato: Contextual Semantic Type Detection in Tables”  In _arXiv e-prints_ , 2019, pp. arXiv:1911.06311  arXiv:1911.06311 [cs.DB]