PANTONE PANTONE PANTONE3015C PANTONE3015C 1 0.3 0 0.2 PANTONE

Corresponding author: Khurram Azeem Hashmi (e-mail:
Khurram_Azeem.Hashmi@dfki.de).

# Guided Table Structure Recognition through Anchor Optimization

KHURRAM AZEEM HASHMI1,2,3  DIDIER STRICKER1,2, MARCUS LIWICKI4, MUHAMMAD NOMAN
AFZAL5 AND MUHAMMAD ZESHAN AFZAL1,2,3  German Research Center for Artificial
Intelligence, 67663 Kaiserslautern, Germany  Department of Computer Science,
University of Kaiserslautern, 67663 Kaiserslautern, Germany  Mindgrage,
University of Kaiserslautern, 67663 Kaiserslautern, Germany  Luleå University
of Technology, A3570 Luleå, Sweden  Bilojix Soft Technologies, Bahawalpur.
Pakistan

###### Abstract

This paper presents the novel approach towards table structure recognition by
leveraging the guided anchors. The concept differs from current state-of-the-
art approaches for table structure recognition that naively apply object
detection methods. In contrast to prior techniques, first, we estimate the
viable anchors for table structure recognition. Subsequently, these anchors
are exploited to locate the rows and columns in tabular images. Furthermore,
the paper introduces a simple and effective method that improves the results
by using tabular layouts in realistic scenarios. The proposed method is
exhaustively evaluated on the two publicly available datasets of table
structure recognition i.e ICDAR-2013 and TabStructDB. We accomplished state-
of-the-art results on the ICDAR-2013 dataset with an average F-Measure of
95.05$\%$ (94.6$\%$ for rows and 96.32$\%$ for columns) and surpassed the
baseline results on the TabStructDB dataset with an average F-Measure of
94.17$\%$ (94.08$\%$ for rows and 95.06$\%$ for columns).

###### Index Terms:

Deep Neural Network, Mask R-CNN, Document Images, Object Detection, Table
Structure Recognition, Table Structure Extraction, Table Understanding.

##  I Introduction

In this modern age of digitization, several camera-equipped devices [59] have
been operated daily to upload documents which leads to expanding the need for
robust systems that can extract information from raw document images [60]. In
the past, numerous approaches have advertised remarkable results in retrieving
information by applying Optical Character Recognition (OCR) methods on
documents [40, 39, 38]. One of the most appropriate ways to represent the
information in documents is through tables [2]. The table contains highly
important facts and figures stored in a concise and organized manner [10].
These tabular structures are extensively used as a medium to convey valuable
information in domains like finance, administration, research, and even
historical documents [42]. Hence, automated identification of these tabular
structures is a significant problem in the document analysis community [2, 3,
4].

The problem of table analysis can be explained by breaking it down into two
sub-problems: The first problem is to identify the boundary of the table in a
document image whereas the second task is to recognize the structure of the
detected table[10].

The task of table detection is a complex problem because of the diversity in
tabular patterns, for instance, some tables contain ruling lines while others
do not have any kind of information. It is highly probable to detect false
positives while spotting a table because of having similarities between tables
and charts or figures [48]. These challenges demonstrate that custom
heuristics or traditional approaches are not capable of handling the problem
of table detection [42]. The recent development in deep learning based methods
has exceptionally improved state-of-the-art table detection methods. Several
researchers have exploited deep learning algorithms to detect the tabular area
[23, 28, 41]. Object detection algorithms have been proven to surpass the rest
of the techniques and achieved almost perfect results [42, 56].

Figure 1: Table Structure Recognition problem definition and challenges. Red
color defines the bounding box for rows while blue color denotes columns. In
the figure, part(a) and part(b) represents tabular images having rows spanning
multiple lines whereas, in part(c), rows are restricted to a single line.
Columns can be as wide as illustrated in part(a) and part(b) but also as
narrow as shown in part(c). For the sake of clarity, only a few rows and
columns are highlighted.

The task of table structure recognition is the detection of various cells
present in the table [18]. This problem can be further dissolved into
detecting rows and columns in a table that can be later combined to produce
the respective cells [27]. The pre-condition for the task of table structure
recognition is the accurately detected tabular regions [33, 32]. Figure 1
illustrates how the problem of table structure recognition is defined in our
approach. Additionally, the figure depicts the challenges that exist due to
the diversities in structures of rows (columns) in tabular images. Only few
rows (columns) are marked for the sake of clarity.

There are several approaches that have tackled the problem of table structure
recognition by leveraging additional metadata extracted from the PDF files
[13, 19]. However, extracting tabular structures directly from images is
perplexing as compared to operating over digital-born PDFs [27]. Although few
considerable efforts have also been made to recognize the tabular structures
straight from images [28, 32], accurate structural recognition is far from
achievable [33].

This paper extends the idea of treating the problem of table structure
recognition as an object detection problem [33]. In object detection problems,
the elementary task is to find the object in a natural scene image. In our
case, we operate a document as a natural image while the rows and columns in
the table are our targeted objects. While the system DeepTabStr [33] relied on
memory-intensive deformable convolutions [34], our approach consists of
intuitive utilization of Mask R-CNN [37] with optimized anchors along with a
simple and effective post-processing method.

In particular, the contributions of this paper are summarized as follows:

  * •

We have treated the problem of table structure extraction as an object
detection problem by employing the well-known Mask R-CNN model.

  * •

We have implemented a novel anchor optimization technique in a region-based
convolutional neural network that produces faster network convergence.

  * •

We have introduced a simple and effective post-processing method to remove the
extra white spaces from the predicted rows. This method can be exploited to
recognize tabular structures in realistic scenarios.

  * •

After extensive cross-dataset evaluations, our proposed approach has beaten
the state-of-the-art results on the ICDAR-13 dataset [18] by using same
evaluation metrics proposed by Schreiber et al. [27]. Furthermore, we have
also surpassed the baseline results on the TabStructDB dataset [33].

The rest of the paper is organized as follows: In the beginning, we discuss
some of the previous work closely related to our approach in Section II; In
Section III we explain our proposed approach and discuss the ideas used in the
experiments; Section IV provides a brief overview about the datasets which are
exploited in the proposed approach; Along with a brief detail over evaluation
metrics, we present our results in Section V; Finally, Section VI concludes
the paper.

##  II RELATED WORK

In this section, we highlight the most relevant related work in field of table
structure analysis. The contributions can be divided into pre and post-deep
learning era as described in the following sections. For an exhaustive state-
of-the-art overview in the closely related research area of table
understanding, refer to [1, 2, 3, 4, 5, 6, 7, 8, 9, 10].

Figure 2: The proposed pipeline for Table Structure Recognition. Optimized
anchors are given to the region proposal network of Mask R-CNN. After
regressing coordinates by the network, the predicted bounding boxes for row
detection are further enhanced by employing the post-processing technique.

###  II-A Traditional Approaches

Kieninger et al. [11, 14, 12], who are the pioneers for working in table
structure extraction, tackled the problem by leveraging the traditional
approaches. Their proposed system T-Recs gathered the words into columns by
calculating their horizontal ruling lines. Subsequently, horizontal lines were
split into respective cells based on column margins.

Wang et al. [15] proposed a system which can generate a large number of table
ground truths which are beneficial for table recognition systems. The author
used a novel table analysis algorithm along with an X-Y cut algorithm to
extract table structure by detecting the respective cells. Later, another
data-driven system proposed by Wang et al. [16] which was based on joint
probability distributions and deals with both detection and structure
decomposition of tables. Their algorithm was analogous to a well famous X-Y
cut algorithm [17].

The problem of table structure extraction caught attention when a table
structure recognition competition is organized at ICDAR in 2013 [18]. While
the first part of the competition was to detect the boundary of the table, the
second part of this competition was to recognize the tabular structure by
reconstructing the cellular structure of a table. The cell-level metrics were
used to evaluate the performance of the systems. It is important to note that
apart from one candidate, all of the participants in the competition vastly
used the PDF metadata. However poor results achieved by the pure image-based
system depict that cell-level metrics are not suitable for the evaluation of
image-based table analysis systems.

Another approach that leverages PDF-metadata to detect the structure of tables
is published by Klampfl et al. [19]. The system employed the blend of
unsupervised learning techniques and hand-crafted heuristics to perform table
structure recognition. Kasar et al. [20] came up with a query-based system to
extract structure of the tables. The system converts the input query taken
from the user into a relational graph which is then compared using a graph
matching algorithm to fetch the required information.

Shigarov et al. [21] performed an exhaustive evaluation on various algorithms
with different thresholds and custom heuristics to tackle the problem of table
structure recognition. Their approach was heavily dependent on the PDF meta-
data as well. Another approach relying heavily on PDF-metadata is proposed by
Rastan et al. [22]. Along with recognizing the structure of tables, their
system TEXUS can also extract the content from tabular structures.

All these techniques are heavily dependent on the meta-data available in
digital-born PDFs. Since our approach works on the scanned document images,
these techniques are not directly comparable with our approach.

###  II-B Deep Learning based Approaches

####  II-B1 Graph Neural Networks

Recently, Chi et al. [23] has exploited graph neural networks [28] to perform
the task of table structure recognition on PDF documents. Another approach
powered by graph neural networks is published by Qasim et al. [24]. Their
model combines the capabilities of convolutional neural networks and graph
neural networks to extract tabular structures. Xue et al. introduced a bottom-
up approach by reconstructing the table structure using a cell relationship
network. The system ReS2TIM [25] employed a distance-based weight technique to
retrieve a syntactic table structure.

####  II-B2 Recurrent Neural Networks

Recurrent neural networks [29] have also been employed to handle the problem
of table structure extraction [31, 30]. However, most of the prior approaches
have utilized PDF meta-data. Since we deal with natural document images, they
are not directly comparable to our approach.

####  II-B3 Convolutional Neural Networks

To the best of our knowledge, Schreiber et al. [27] published the first
natural image-based deep learning system which explored the problem of table
structure analysis. The system leverages the Fully Convolutional Network (FCN)
[61] to segment the table into rows and columns. Later in 2019, TableNet has
been proposed by Shubham et al. [28]. The authors tackled the problem of table
structure extraction through a semantic segmentation technique. Another
approach powered by semantic segmentation to extract tabular structures from
document images is published by Siddiqui et al. [32].

Figure 3: Visualisation of anchors traditionally used for object detection
techniques against optimized anchors used in our approach. (a) Anchors
traditionally used for object detection. (b) Optimized anchors for row
detection. (c) Optimized anchors for column detection. Traditional anchors are
transformed into optimized anchors using K-Means Clustering technique.

All of these approaches have either used semantic segmentation or FCN to solve
the problem of document images. Contrarily, we have chosen to handle the task
of table structure recognition as an object detection problem. Although
Siddiqui et al. [33] has treated the table structure analysis as an object
detection problem, there are various considerable differences between the two
methods. The system DeepTabStr [33] has adopted Faster R-CNN [35] with
deformable convolutions [34] while our proposed approach works with Mask R-CNN
[37] exploiting optimized anchors to directly detect boundaries of respective
rows and columns in a tabular image.

##  III METHOD

We have devised the problem of table structure recognition as an object
detection problem. Object detection is a famous problem in computer vision
that studies how an object can be recognized from a natural scene image.
Recent progress in deep learning has remarkably enhanced the state-of-the-art
object detection systems [35, 37]. To achieve the ultimate goal of table
structure recognition, we have decomposed our problem into two sub-problems
where the first one is about detecting rows in tables while the second sub-
problem deals in detecting columns.

###  III-A Model

Our proposed approach could be implemented in two ways:

  1. 1.

Separate model for both rows and columns.

  2. 2.

Single combined model to handle both the problems.

Considering the diversity in the structures of rows and columns, it has been
settled that the separate model performs better [33]. Hence, we have decided
to go for two segregated models to solve the problem of table structure
recognition.

We have adopted Mask R-CNN [37] as our model to identify the rows and columns
in a table. Mask R-CNN is one of the accurate object detection algorithms and
the latest member to the group of Region-based Convolutional Neural Networks
(RCNN) [44]. Mask R-CNN is a two-phase model and has shown compelling
performance on the PASCAL VOC [46] and COCO [45] datasets. Mask R-CNN has been
exploited before to identify various graphical objects in document images
[48].

To execute the training process of the deep neural network, it requires an
extensive amount of data which we lack specifically in the domain of table
structure extraction. To tackle this problem, we have leveraged the
capabilities of transfer learning in our approach. The backbone of our Mask
R-CNN is a pre-trained model on ImageNet [47] dataset.

Figure 2 illustrates the complete pipeline of our proposed approach. Analogous
to Faster R-CNN [35], Mask R-CNN [37] follows the two-phase procedure with one
addition. The first phase consists of Region Proposal Network (RPN) which
proposes regions of interest in a document image whereas the second phase
deals in the classification of labels and regression of bounding boxes
including the binary masks of each region of interest. Now, we will discuss in
detail about the different components of our proposed pipeline presented in
Figure 2.

In the first stage, the combination of ResNet-101 [49] and Feature Pyramid
Network (FPN) [50] which is acting as a backbone in our case, extracts the
features from the document image. These features are further propagated to
Region Proposal Network (RPN). RPN is a lightweight neural network that scans
some regions in an image and tries to filter out the ones which are more
likely to contain objects. These input regions for RPN are known as anchors.
Anchors are defined as a set of rectangular regions with a predefined set of
scales and aspect ratios [58]. The RPN generates two kinds of outputs for each
anchor:

  1. 1.

The class of an anchor which states whether an anchor is an object or
background.

  2. 2.

Bounding box refinement which is the change in the position of the bounding
box to precisely fit the object in the proposed region of interest.

###  III-B Anchor Optimization

The concept of anchors was introduced in the Faster R-CNN by Ren et al. [35]
which is transported into Mask R-CNN [37] as well. Contrary to the hand-
crafted approach of selecting anchors in Mask R-CNN, we have applied the
K-means clustering technique to retrieve fine anchors as explained in the
approach proposed by Redmon et al. [51]. The anchors traditionally used in
object detection consist of various width to height ratios to deal with
objects having diverse shapes [62]. However, in the case of detecting rows, we
are aware that the width of the anchor will always be equal or greater than
the height of an anchor while it is the other way around for columns. Hence,
anchors having customized sizes and aspect ratios will lead to better
performance as compared to anchors commonly used for object detection
techniques. It is important to mention that the euclidean distance was not
used as a distance metric in our K-means clustering technique but the
following distance metric [56] is used :

$D(box,centroid)=1-IoU(box,centroid)$ (1)

where the box represents bounding box as a data sample that needs to be
clustered and centroid is the center of a cluster which will be retrieved at
the end of clustering. IoU (Intersection over Union) is an evaluation metrics
which is explained in Section V. The purpose of choosing this metric over
euclidean distance is that the bigger boxes will lead to more errors as
compared to smaller ones which is not the main concern in our scenario [56].
The traditional anchors are given as input to K-means clustering technique
along with the training datasets of ICDAR 2013 [18] and TabStructDB [33] in
order to retrieve optimized anchors for each dataset.

Figure 3 illustrates the comparison between original anchors used for object
detection and the optimized anchors for the row (column) detection. The anchor
ratios (0.5, 1 and 2) are used in Figure 3(a) while for Figure 3(b) and Figure
3(c), we have used four different anchor ratios (50, 25, 10 and 3) and (0.1,
0.3, 0.5 and 1) respectively. It can be perceived that optimized anchors 3(b)
and 3(c) are well suited to execute the task of table structure recognition as
compared to the anchors traditionally used for object detection 3(a) .

Figure 4: Network training loss comparison between Optimized anchors and
traditional anchors for row detection. Blue line chart represents network
training loss for row detection using traditional anchors. Red line chart
represents network training loss for row detection using optimized anchors.
The Y axis of the graph shows the loss values whereas the X axis displays the
number of epochs. It can be seen that the network with optimized anchors
achieves the loss value of less than 0.1 right after the 30 epochs whereas the
network with traditional anchors is unable to achieve the same loss value even
after 50 epochs.

It is important to mention that RPN scans these optimized anchors on the
feature maps instead of an actual document image. This enables the RPN to
reuse extracted features efficiently. The proposed anchor optimization
technique not only improves the performance of the model but also facilitates
the network to converge faster, making our approach even more efficient.
Figure 4 illustrates how optimized anchors help the network to achieve better
results in less time. Along with faster network convergence, the performance
of our model is significantly improved after exploiting optimized anchors. The
performance comparisons between the models for row and column detection
employed with traditional and optimized anchors are explained in Figure 5 and
their F-Measure scores are summarized in Table I.

Model | Row Detection | Column Detection
---|---|---
Traditional Anchors | 0.8710 | 0.8920
Optimized Anchors | 0.9206 | 0.9632
TABLE I: Comparison of F-Measure scores for rows and column detection between
traditionally used anchors and our optimized anchors. Figure 5: Performance
comparison between the models trained with traditional anchors and optimized
anchors. We have experienced a noticeable increase in the F-measure score for
both rows and columns detection after employing optimized anchors.

###  III-C Layout Based Post Processing

Once the rows and columns are detected by the Mask R-CNN, we noticed that
while the network managed to detect the columns properly, it was unable to
recognize the precise boundaries of rows. In the case of row detection, we
observed that the height of predicted bounding boxes is identical to ground
truth, however, the network struggled to predict the accurate width of a
bounding box. This either generates extra white spaces in the bounding box or
drops some valuable information from the rows. To tackle the problem, we came
up with a simple and effective post-processing algorithm that can resize the
width of a bounding box on the basis of few constraints.

We are aware of the fact that for most of the documents, the information is
written in black color. Our proposed method improves precision and recall in
two ways:

  1. 1.

Incorporating the important information which was overlooked by the network by
increasing the width of a bounding box close to the last set of black pixels.

  2. 2.

Removing extra white spaces by decreasing the width of the bounding box to the
nearest set of black pixels.

The pseudo-code for the proposed method is explained in Algorithm 1. It is
important to mention that method does not work in a few of the cases where the
text is not displayed in black pixels. However, the proposed method has shown
significant improvements in the IoU of bounding boxes in case of row detection
which is summarized in Table II. Figure 6 depicts how the performance of row
detection can be improved from simple post-processing.

Figure 6: Explaining through example about how the IoU for row detection in
document images can be further improved with simple post-processing. Detected
rows in part (B) are either stretched or reduced to produce accurate
boundaries as illustrated in part (C). Algorithm 1 Resize the width of
bounding box by identifying black pixels

0:

0:

1: $blackPT\leftarrow BlackPixelThreshold$

2: $Area\leftarrow ImageSpecficArea$

3: $R\leftarrow I$

4: for $xValue$ of $R$ to end of image do {checking forward for both xmin and
xmax}

5: if $blackPixelfound$ then

6: Compute blackPixel count in that $Area$

7: if $blackPixelCount>=blackPT$ then

8: $xmaxofR\leftarrow xValue$

9: end if

10: end if

11: end for

12: for $xValue$ of $R$ to beginning of image do {checking backward for both
xmin and xmax}

13: if $blackPixelfound$ then

14: Compute blackPixel count in that $Area$

15: if $blackPixelCount>=blackPT$ then

16: $xminofR\leftarrow xValue$

17: end if

18: end if

19: end for

20: return $R$

###  III-D Experiment Details

We have worked with the combination of ResNet-101 [49] and FPN[50] model as a
backbone for both the row and columns detection. Apart from the aspect ratios
anchors, the rest of the hyperparameters were identical for both of the
models. For row detection, we have used four different anchor ratios (50, 25,
10, and 3) whereas, for columns, we have picked four different anchors ratios
with (0.1, 0.3, 0.5, and 1). However, we have used five different anchor
scales (16, 32, 64, 128, 256) for both of the networks. Both of the models
were optimized for 50 epochs where each epoch consists of 100-time steps. The
maximum image size was limited to $1024\times 800$ and the images exceeding
this size were resized to the maximum dimension. We used a batch size of 2 on
a single NVIDIA $1080$ Ti GPU. Our model works on stochastic gradient descent
having a momentum value of 0.9 and a learning rate of 0.0001. Gradients are
clipped to 5.0 and weights are decayed by 0.0001 at each epoch. In order to
prevent the problem of overfitting, we have applied augmentation techniques
like random rotations, Gaussian blurring, and random horizontal and vertical
flips on the training dataset. We have implemented this work in Keras [53]
with Tensorflow [54] as a backend.

Approach | Precision | Recall | F-Measure
---|---|---|---
Before Post-processing | 0.9106 | 0.9326 | 0.9206
After Post-processing | 0.9468 | 0.9452 | 0.9460
TABLE II: Performance comparison for row detection before and after applying
post-processing technique. After applying our post-processing method, we have
seen a significant increase in an F-Measure score in case of row detection in
document images.

c2cm

Training Dataset |  Testing
Dataset |  Model | Row | Column |  Average F-Measure
---|---|---|---|---|---
Precision |  Recall |  F-Measure |  Precision |  Recall |  F-Measure

ICDAR-13
(Training set) |
ICDAR-13
(Test set) |  Faster R-CNN |  0.8974 |  0.9154 |  0.9063 |  0.9456 |  0.9488 |  0.9510 |  0.9286
Deformable Faster R-CNN |  0.9071 |  0.9221 |  0.9145 |  0.9511 |  0.9588 |  0.9549 |  0.9347
Deformable Faster R-CNN$\dagger$ |  0.8817 |  0.4097 |  0.4531 |  0.9520 |  0.9497 |  0.9497 |  0.7014
|  Mask R-CNN |  0.9106 |  0.9326 |  0.9206 |  0.9605 |  0.9659 |  0.9632 |  0.9419
|  TabStructDB
(Complete) |  Faster R-CNN |  0.6968 |  0.6632 |  0.6796 |  0.6845 |  0.6973 |  0.6908 |  0.6852
|  Deformable Faster R-CNN |  0.7034 |  0.6972 |  0.7003 |  0.7883 |  0.7561 |  0.7719 |  0.7361
|  Deformable Faster R-CNN$\dagger$ |  0.5545 |  0.2785 |  0.4531 |  0.7681 |  0.7489 |  0.7533 |  0.6032
|  Mask R-CNN |  0.7189 |  0.6850 |  0.7034 |  0.7037 |  0.7157 |  0.7097 |  0.7011
|  TabStructDB (Test Set) |  Faster R-CNN |  0.6788 |  0.6634 |  0.6710 |  0.7041 |  0.7255 |  0.7146 |  0.6928
|  Deformable Faster R-CNN |  0.6925 |  0.6812 |  0.6868 |  0.7152 |  0.7377 |  0.7263 |  0.7066
|  Deformable Faster R-CNN$\dagger$ |  0.5492 |  0.2622 |  0.3009 |  0.7687 |  0.7462 |  0.7501 |  0.5255
|  Mask R-CNN |  0.7142 |  0.6937 |  0.7039 |  0.7376 |  0.7525 |  0.7484 |  0.7237

TabStructDB
(Training set) |
ICDAR-13
(Complete) |  Faster R-CNN |  0.7577 |  0.7322 |  0.7447 |  0.6954 |  0.7125 |  0.7038 |  0.7242
Deformable Faster R-CNN |  0.7821 |  0.7514 |  0.7664 |  0.7023 |  0.7344 |  0.7180 |  0.7422
Deformable Faster R-CNN$\dagger$ |  0.6048 |  0.5507 |  0.5660 |  0.7308 |  0.7518 |  0.7422 |  0.6541
|  Mask R-CNN |  0.8263 |  0.7729 |  0.7987 |  0.7143 |  0.7226 |  0.7184 |  0.7677
|  ICDAR-13
(Test Set) |  Faster R-CNN |  0.7321 |  0.7144 |  0.7231 |  0.6543 |  0.6411 |  0.6476 |  0.6853
|  Deformable Faster R-CNN |  0.7932 |  0.6350 |  0.7053 |  0.6621 |  0.6721 |  0.6671 |  0.6862
|  Deformable Faster R-CNN$\dagger$ |  0.5279 |  0.4625 |  0.4818 |  0.6701 |  0.6768 |  0.6705 |  0.5761
|  Mask R-CNN |  0.8296 |  0.7586 |  0.7925 |  0.6453 |  0.6307 |  0.6379 |  0.7354
|  TabStructDB (Test Set) |  Faster R-CNN |  0.9074 |  0.9254 |  0.9163 |  0.9556 |  0.9588 |  0.9572 |  0.9367
|  Deformable Faster R-CNN |  0.9111 |  0.9285 |  0.9197 |  0.9605 |  0.9659 |  0.9632 |  0.9414
|  Deformable Faster R-CNN$\dagger$ |  0.8921 |  0.9125 |  0.8945 |  0.9585 |  0.9682 |  0.9594 |  0.9269
|  Mask R-CNN |  0.9314 |  0.9504 |  0.9408 |  0.9523 |  0.9489 |  0.9506 |  0.9417
TABLE III: Table Structural Segmentation Performance on cross-dataset
evaluations. In this table, $\dagger$ represents the only approach that did
not utilize the optimized anchors and the results are taken from DeepTabStR
[33] in order to have a direct comparison. Reset of the models operate on
optimized anchors.

##  IV Datasets

We have used two publicly available table structure recognition datasets to
conduct the experiments. The particulars of these datasets are explained
below.

###  IV-A ICDAR-2013

ICDAR-2013 [18] dataset has been used to standardize the state-of-the-art
results for the task of table detection and table structure recognition [27,
32]. There is a total of 238 pages in the dataset out of which 156 contains
tabular structures. Originally, the dataset contains labels for cells in a
table. However, we have used the transformed version of the dataset
111ICDAR-2013 dataset is publicly available at : https://bit.ly/2RLgFYu
published by Siddiqui et al. [32]. The authors have converted the cell-based
annotations into the corresponding labeling for rows and columns. We have used
the identical test split as employed by Schreiber et al. [27] in order to
implement a direct comparison against the similar approaches [27, 32, 33]. A
sample tabular image is illustrated in Figure 1.

###  IV-B TabStructDB

A Page Object Detection (POD) competition was arranged in ICDAR 2017. The task
of this competition was to detect graphical page objects in documents like a
table, figures, charts, and equations [55]. By leveraging this dataset,
Siddiqui et al. [32] has published a new dataset for table structure
recognition known as TabStructDB 222TabStructDB is publicly available at:
https://bit.ly/2XonOEx. The dataset contains structural information of each
table present in the ICDAR-2017 POD dataset. In the dataset, each complete row
has been annotated separately regardless of the textual region in order to
maintain consistency. Hence, making this dataset ideal for the object
detection approach. To keep the coherence with the ICDAR-2017 POD dataset, the
same dataset split has been preserved. The dataset comprised of 731 tabular
regions for training whereas 350 tabular regions are preserved for the testing
part. A sample tabular image is illustrated in Figure 1.

##  V Evaluation

Model | Row | Column | Average
---|---|---|---
Precision |  Recall |  F-Measure |  Precision |  Recall |  F-Measure |  Precision |  Recall |  F-Measure
DeepDeSRT [27] |  - |  - |  - |  - |  - |  - |  0.9593 |  0.8736 |  0.91444
TableNet [28] |  - |  - |  - |  - |  - |  - |  0.9307 |  0.9001 |  0.9151
DeepTabStR [33] |  0.8845 |  0.8945 |  0.8861 |  0.9688 |  0.9630 |  0.9655 |  0.9319 |  0.9308 |  0.9298
Siddiqui et al. [32] |  0.9233 |  0.9203 |  0.9190 |  0.9281 |  0.9341 |  0.9288 |  0.9257 |  0.9272 |  0.9239
Proposed System
(With post-
processing) |  0.9468 |  0.9452 |  0.9460 |  0.9605 |  0.9659 |  0.9632 |  0.9537 |  0.9556 |  0.9546
Proposed System
(Without post-
processing) |  0.9106 |  0.9326 |  0.9206 |  0.9605 |  0.9659 |  0.9632 |  0.9355 |  0.9441 |  0.9419
TABLE IV: Table structural recognition performance comparison on ICDAR-2013
dataset. Outstanding results are highlighted. Our proposed system has out-
smarted the prior approaches even without the post-processing included.

In order to compare our approach with state-of-the-art methods [27, 32, 33],
we have used the identical evaluation metrics which are explained below:

###  V-A Intersection over Union (IoU)

Intersection over Union is a famous evaluation metric used to determine the
performance of object detection algorithms. It defines as a measure of a
predicted region overlapped with the actual ground truth region. We have used
an IoU threshold of 0.5 for the detections. The formula for computing IoU is
mentioned below :

$\frac{\text{Area of Overlap region}}{\text{Area of Union region}}$ (2)

###  V-B Precision

Precision is defined as the ratio of correctly predicted region and the total
predicted region. The formula for precision is explained below :

$\frac{\text{Predicted area in ground truth}}{\text{Total area of predicted
region}}=\frac{\text{TP}}{\text{TP $+$ FP}}$ (3)

###  V-C Recall

Recall is calculated as the ratio of correctly predicted region and the total
ground truth region. The formula for recall is explained as follows :

$\frac{\text{Predicted area in ground truth}}{\text{Total area of ground truth
region}}=\frac{\text{TP}}{\text{TP $+$ FN}}$ (4)

###  V-D F-Measure

Harmonic mean of precision and recall is known as the F-meausre. The formula
for F-measure is :

$\frac{\text{$2\times$ Precision $\times$ Recall}}{\text{Precision $+$
Recall}}$ (5)

It is important to understand that the precision, recall, and F-measure are
calculated independently for each document followed by taking an average over
the complete dataset. This evaluation criterion reduces the bias from a single
document containing several rows and columns.

###  V-E Experiments

As described in Section IV, we have evaluated our proposed approach on the two
publicly available datasets i.e. ICDAR-2013 table structure recognition
dataset and TabStructDB. Apart from evaluating the datasets on their
respective test sets, we have appraised the generalization potential of our
approach through the cross-dataset evaluation. The obtained results are
highlighted in Table .

####  V-E1 ICDAR-2013

Since we are using the modified version of the ICDAR-2013 dataset and we
report results on the basis of rows and columns, our approach cannot be
directly compared with any of the participants of ICDAR-2013 table competition
[18] and other methods operating on cell-level information. Hence, we compare
our approach with the other image-based models who has reported results on
rows and columns. To enable the direct comparison with those approaches, we
have used the same train/test split proposed by Schreiber et al. [27].

Table IV summarizes the results of image-based table structure recognition
methods on ICDAR-2013 dataset. Results depict that our proposed approach both
(with and without the involved processing method) has outperformed the
previous state-of-the-art techniques with an average F-Measure of almost
$0.95$. Although results on the column detection of our model are comparable
with the DeepTabStR [33], our anchor optization method has surpassed the
performance of row detections resulting in noticeable improvement on the
average results. For cross-dataset evaluation, the model is trained on the
TabStructDB dataset and tested on the complete as well as the test set of the
ICDAR-2013 dataset. The average F-measure of almost $0.74$ for the test set
and almost $0.77$ for the complete dataset in Table III present the diversity
between the two datasets and indicate that there is still room in generalizing
the system over various datasets.

Figure 7 portrays fragments of correctly recognized tabular structures whereas
Figure 8 depicts some of the cases where rows and columns are not properly
detected by the system. In case of incorrect recognition, the model fails to
detect few rows in Figure 8(a) and 8(c) because of having several rows with
small width in a document image. In another case in Figure 8(b), the system
was unable to recognize the row spanning in multiple lines. Although most of
the columns are correctly detected by the model, there are few instances where
the system either not capture the whole column area or merge multiple small
columns into a single column (Figure 8(d-f)).

Figure 7: Correctly Recognized Table Structures. Model | Row | Column |  Average F-Measure
---|---|---|---
Precision |  Recall |  F-Measure |  Precision |  Recall |  F-Measure
DeepTabStR [33] |  0.9093 |  0.9404 |  0.9186 |  0.9560 |  0.9628 |  0.9559 |  0.9372
Proposed System |  0.9314 |  0.9504 |  0.9408 |  0.9523 |  0.9489 |  0.9506 |  0.9457
TABLE V: Table structural recognition performance comparison on TabstructDB
dataset. Outstanding results are highlighted.

####  V-E2 TabStructDB

Along with the ICDAR-2013 dataset, we have also compared our approach to the
TabStructDB dataset. It is evident in the Table V that our proposed system has
outperformed the baseline results established by the DeepTabStR [33] with an
average F-Measure of 0.9417. It is important to mention that since we have
trained our models for rows and columns separately, we have compared our
results with theirs achieved on separate training methods with the same
train/test split of the dataset.

For the cross-dataset evaluation, a noticeable fall in performance can be
perceived in Table III when the system (trained on ICDAR-2013) is evaluated on
TabStructDB complete set and test-set. One of the main reasons for this
decline is the disparity in the annotation scheme. The annotations of
ICDAR-2013 are limited to textual regions only while TabStructDB has been
labeled with complete rows and columns without considering the textual regions
at all. Since this is an unrealistic scenario, we have not applied the
proposed post-processing method while evaluating the performance of our system
on the TabStructDB dataset.

Figure 8: Examples representing incorrectly recognized row and column
detection. Green colour shows true positives, blue colour depicts false
positives and red colour portrays false negatives for both rows and columns.

##  VI Conclusion and Future Work

We have proposed a novel approach that employs object detection as a base and
adds intelligent automatic estimation of anchor boxes that are suitable for
table structure recognition. In this paper, we exhibit that current object
detectors that have already shown remarkable improvements in resolving the
problem of table detection [27, 42], are also extremely effective in improving
the performance of table structure recognition systems. We have adopted the
anchor optimization technique that facilitates the object detection process
with a faster network convergence, a simple post-processing method at the end
has further enhanced the performance of our table recognition system. The
achieved results have evidently outperformed the state-of-the-art image-based
table structure recognition system on the publicly available ICDAR-2013
dataset with an average F-Measure of $95.05\%$ and surpassed the baseline
results on the publicly available TabStructDB dataset with an average
F-Measure of $94.17\%$. The obtained results recommend the idea of exploiting
object detectors for table recognition systems.

Although our proposed method is nearly applicable to all of the document
images, some exceptional cases do exist. Hence, better post-processing methods
should be developed. Our model had a hard time detecting rows spanning for
multiple lines in the table. An interesting direction could be to detect the
cells directly instead of rows and columns. Instead of using the traditional
convolutional neural networks, recently proposed CoordConv [57] could also be
exploited in the object detection algorithms in order to provide the system
with extra contextual information. This paper tackles the problem of table
structure recognition in business-like scanned document images. It would be
interesting to examine this approach for the datasets that contains historical
document images such as ICDAR-2019 (cTDaR) [32].

## References

  * [1] Hu, Jianying, Ramanujan S. Kashi, Daniel Lopresti, and Gordon T. Wilfong. ”Evaluating the performance of table processing algorithms.” International Journal on Document Analysis and Recognition 4, no. 3 (2002): 140-153.
  * [2] Zanibbi, Richard, Dorothea Blostein, and James R. Cordy. ”A survey of table recognition.” Document Analysis and Recognition 7, no. 1 (2004): 1-16.
  * [3] e Silva, Ana Costa, Alípio M. Jorge, and Luís Torgo. ”Design of an end-to-end method to extract information from tables.” International Journal of Document Analysis and Recognition (IJDAR) 8, no. 2-3 (2006): 144-171.
  * [4] Embley, David W., Matthew Hurst, Daniel Lopresti, and George Nagy. ”Table-processing paradigms: a research survey.” International Journal of Document Analysis and Recognition (IJDAR) 8, no. 2-3 (2006): 66-86.
  * [5] Coüasnon, Bertrand, and Aurélie Lemaitre. ”Recognition of tables and forms.” (2014).
  * [6] Khusro, Shah, Asima Latif, and Irfan Ullah. ”On methods and tools of table detection, extraction and annotation in PDF documents.” Journal of Information Science 41, no. 1 (2015): 41-57.
  * [7] Lopresti, Daniel, and George Nagy. ”A tabular survey of automated table processing.” In International Workshop on Graphics Recognition, pp. 93-120. Springer, Berlin, Heidelberg, 1999.
  * [8] Lopresti, Daniel, and George Nagy. ”Automated table processing: An (opinionated) survey.” In Proceedings of the Third IAPR Workshop on Graphics Recognition, pp. 109-134. 1999.
  * [9] Dougherty, Edward R. Electronic imaging technology. Vol. 60. SPIE Press, 1999.
  * [10] Hurst, Matthew Francis. ”The interpretation of tables in texts.” PhD diss., University of Edinburgh, 2000.
  * [11] Dougherty, Edward R. Electronic imaging technology. Vol. 60. SPIE Press, 1999.
  * [12] Kieninger, Thomas, and Andreas Dengel. ”The t-recs table recognition and analysis system.” In International Workshop on Document Analysis Systems, pp. 255-270. Springer, Berlin, Heidelberg, 1998.
  * [13] Kieninger, Thomas, and Andreas Dengel. ”Applying the T-RECS table recognition system to the business letter domain.” In Proceedings of Sixth International Conference on Document Analysis and Recognition, pp. 518-522. IEEE, 2001.
  * [14] Kieninger, Thomas, and Andreas Dengel. ”A paper-to-HTML table converting system.” In Proceedings of document analysis systems (DAS), vol. 98, pp. 356-365. 1998.
  * [15] Wangt, Yalin, Ihsin T. Phillipst, and Robert Haralick. ”Automatic table ground truth generation and a background-analysis-based table structure extraction method.” In Proceedings of Sixth International Conference on Document Analysis and Recognition, pp. 528-532. IEEE, 2001.
  * [16] Wang, Yalin, Ihsin T. Phillips, and Robert M. Haralick. ”Table structure understanding and its performance evaluation.” Pattern recognition 37, no. 7 (2004): 1479-1497.
  * [17] Nagy, George, and Sharad C. Seth. ”Hierarchical representation of optically scanned documents.” (1984).
  * [18] Göbel, Max, Tamir Hassan, Ermelinda Oro, and Giorgio Orsi. ”ICDAR 2013 table competition.” In 2013 12th International Conference on Document Analysis and Recognition, pp. 1449-1453. IEEE, 2013.
  * [19] Klampfl, Stefan, Kris Jack, and Roman Kern. ”A comparison of two unsupervised table recognition methods from digital scientific articles.” D-Lib Magazine 20, no. 11 (2014): 7.
  * [20] Kasar, Thotreingam, Tapan Kumar Bhowmik, and Abdel Belaid. ”Table information extraction and structure recognition using query patterns.” In 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 1086-1090. IEEE, 2015.
  * [21] Shigarov, Alexey, Andrey Mikhailov, and Andrey Altaev. ”Configurable table structure recognition in untagged PDF documents.” In Proceedings of the 2016 ACM Symposium on Document Engineering, pp. 119-122. 2016.
  * [22] Rastan, Roya, Hye-Young Paik, and John Shepherd. ”Texus: A unified framework for extracting and understanding tables in pdf documents.” Information Processing and Management 56, no. 3 (2019): 895-918.
  * [23] Chi, Zewen, Heyan Huang, Heng-Da Xu, Houjin Yu, Wanxuan Yin, and Xian-Ling Mao. ”Complicated table structure recognition.” arXiv preprint arXiv:1908.04729 (2019).
  * [24] Qasim, Shah Rukh, Hassan Mahmood, and Faisal Shafait. ”Rethinking table recognition using graph neural networks.” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 142-147. IEEE, 2019.
  * [25] Xue, Wenyuan, Qingyong Li, and Dacheng Tao. ”ReS2TIM: reconstruct syntactic structures from table images.” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 749-755. IEEE, 2019.
  * [26] Tensmeyer, Chris, Vlad I. Morariu, Brian Price, Scott Cohen, and Tony Martinez. ”Deep splitting and merging for table structure decomposition.” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 114-121. IEEE, 2019.
  * [27] Schreiber, Sebastian, Stefan Agne, Ivo Wolf, Andreas Dengel, and Sheraz Ahmed. ”Deepdesrt: Deep learning for detection and structure recognition of tables in document images.” In 2017 14th IAPR international conference on document analysis and recognition (ICDAR), vol. 1, pp. 1162-1167. IEEE, 2017.
  * [28] Paliwal, Shubham Singh, D. Vishwanath, Rohit Rahul, Monika Sharma, and Lovekesh Vig. ”Tablenet: Deep learning model for end-to-end table detection and tabular data extraction from scanned document images.” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 128-133. IEEE, 2019.
  * [29] Cleeremans, Axel, David Servan-Schreiber, and James L. McClelland. ”Finite state automata and simple recurrent networks.” Neural computation 1, no. 3 (1989): 372-381.
  * [30] Li, Minghao, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou, and Zhoujun Li. ”Tablebank: Table benchmark for image-based table detection and recognition.” In Proceedings of The 12th Language Resources and Evaluation Conference, pp. 1918-1925. 2020.
  * [31] Khan, Saqib Ali, Syed Muhammad Daniyal Khalid, Muhammad Ali Shahzad, and Faisal Shafait. ”Table structure extraction with bi-directional gated recurrent unit networks.” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1366-1371. IEEE, 2019.
  * [32] Siddiqui, Shoaib Ahmed, Pervaiz Iqbal Khan, Andreas Dengel, and Sheraz Ahmed. ”Rethinking semantic segmentation for table structure recognition in documents.” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1397-1402. IEEE, 2019.
  * [33] Siddiqui, Shoaib Ahmed, Imran Ali Fateh, Syed Tahseen Raza Rizvi, Andreas Dengel, and Sheraz Ahmed. ”DeepTabStR: Deep Learning based Table Structure Recognition.” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1403-1409. IEEE, 2019.
  * [34] Dai, Jifeng, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. ”Deformable convolutional networks.” In Proceedings of the IEEE international conference on computer vision, pp. 764-773. 2017.
  * [35] Ren, Shaoqing, Kaiming He, Ross Girshick, and Jian Sun. ”Faster r-cnn: Towards real-time object detection with region proposal networks.” arXiv preprint arXiv:1506.01497 (2015).
  * [36] Azawi, Mayce Al, Muhammad Zeshan Afzal, and Thomas M. Breuel. ”Normalizing historical orthography for OCR historical documents using LSTM.” In Proceedings of the 2nd International Workshop on Historical Document Imaging and Processing, pp. 80-85. 2013.
  * [37] He, Kaiming, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. ”Mask r-cnn.” In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969. 2017.
  * [38] Hashmi, Khurram Azeem, Rakshith Bymana Ponnappa, Syed Saqib Bukhari, Martin Jenckel, and Andreas Dengel. ”Feedback Learning: Automating the Process of Correcting and Completing the Extracted Information.” In 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), vol. 5, pp. 116-121. IEEE, 2019.
  * [39] Smith, Ray. ”An overview of the Tesseract OCR engine.” In Ninth international conference on document analysis and recognition (ICDAR 2007), vol. 2, pp. 629-633. IEEE, 2007.
  * [40] Mokhtar, Kareem, Syed Saqib Bukhari, and Andreas Dengel. ”OCR Error Correction: State-of-the-Art vs an NMT-based Approach.” In 2018 13th IAPR International Workshop on Document Analysis Systems (DAS), pp. 429-434. IEEE, 2018.
  * [41] Li, Yibo, Liangcai Gao, Zhi Tang, Qinqin Yan, and Yilun Huang. ”A GAN-based feature generator for table detection.” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 763-768. IEEE, 2019.
  * [42] Siddiqui, Shoaib Ahmed, Muhammad Imran Malik, Stefan Agne, Andreas Dengel, and Sheraz Ahmed. ”Decnt: Deep deformable cnn for table detection.” IEEE Access 6 (2018): 74151-74161.
  * [43] Girshick, Ross. ”Fast r-cnn.” In Proceedings of the IEEE international conference on computer vision, pp. 1440-1448. 2015.
  * [44] Girshick, Ross, Jeff Donahue, Trevor Darrell, and Jitendra Malik. ”Rich feature hierarchies for accurate object detection and semantic segmentation.” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580-587. 2014.
  * [45] Lin, Tsung-Yi, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. ”Microsoft coco: Common objects in context.” In European conference on computer vision, pp. 740-755. Springer, Cham, 2014.
  * [46] Everingham, Mark, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. ”The pascal visual object classes (voc) challenge.” International journal of computer vision 88, no. 2 (2010): 303-338..
  * [47] Russakovsky, Olga, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang et al. ”Imagenet large scale visual recognition challenge.” International journal of computer vision 115, no. 3 (2015): 211-252.
  * [48] Saha, Ranajit, Ajoy Mondal, and C. V. Jawahar. ”Graphical object detection in document images.” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 51-58. IEEE, 2019.
  * [49] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. ”Deep residual learning for image recognition.” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778. 2016.
  * [50] Lin, Tsung-Yi, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. ”Feature pyramid networks for object detection.” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2117-2125. 2017.
  * [51] Redmon, Joseph, and Ali Farhadi. ”YOLO9000: better, faster, stronger.” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7263-7271. 2017.
  * [52] Abdulla, Waleed. ”Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow; 2017.” Avaiblable at https://github. com/matterport/Mask_RCNN (2017).
  * [53] Ketkar, Nikhil. ”Introduction to keras.” In Deep learning with Python, pp. 97-111. Apress, Berkeley, CA, 2017.
  * [54] Abadi, Martín, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado et al. ”Tensorflow: Large-scale machine learning on heterogeneous distributed systems.” arXiv preprint arXiv:1603.04467 (2016).
  * [55] Gao, Liangcai, Xiaohan Yi, Zhuoren Jiang, Leipeng Hao, and Zhi Tang. ”ICDAR2017 competition on page object detection.” In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 1417-1422. IEEE, 2017.
  * [56] Huang, Yilun, Qinqin Yan, Yibo Li, Yifan Chen, Xiong Wang, Liangcai Gao, and Zhi Tang. ”A YOLO-based table detection method.” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 813-818. IEEE, 2019.
  * [57] Liu, Rosanne, Joel Lehman, Piero Molino, Felipe Petroski Such, Eric Frank, Alex Sergeev, and Jason Yosinski. ”An intriguing failing of convolutional neural networks and the coordconv solution.” arXiv preprint arXiv:1807.03247 (2018).
  * [58] Wang, Jiaqi, Kai Chen, Shuo Yang, Chen Change Loy, and Dahua Lin. ”Region proposal by guided anchoring.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2965-2974. 2019.
  * [59] Afzal, Muhammad Zeshan, Martin Kramer, Syed Saqib Bukhari, Faisal Shafait, and Thomas M. Breuel. ”Improvements to uncalibrated feature-based stereo matching for document images by using text-line segmentation.” In 2012 10th IAPR International Workshop on Document Analysis Systems, pp. 394-398. IEEE, 2012.
  * [60] Krämer, Martin, Muhammad Zeshan Afzal, Syed Saqib Bukhari, Faisal Shafait, and Thomas M. Breuel. ”Robust stereo correspondence for documents by matching connected components of text-lines with dynamic programming.” In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 734-737. IEEE, 2012.
  * [61] Long, Jonathan, Evan Shelhamer, and Trevor Darrell. ”Fully convolutional networks for semantic segmentation.” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431-3440. 2015.
  * [62] Jiang, Huaizu, and Erik Learned-Miller. ”Face detection with the faster R-CNN.” In 2017 12th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2017), pp. 650-657. IEEE, 2017.
  * [63] Gao, Liangcai, Yilun Huang, Hervé Déjean, Jean-Luc Meunier, Qinqin Yan, Yu Fang, Florian Kleber, and Eva Lang. ”ICDAR 2019 competition on table detection and recognition (cTDaR).” In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1510-1515. IEEE, 2019.

|  Khurram Azeem Hashmi received his bachelor’s degree in computer science
from the National University of Computer and Emerging Sciences, Pakistan in
2016, and the M.S. degree from the Technical University of Kaiserslautern. He
is currently pursuing a Ph.D. degree with the German Research Center for
Artificial Intelligence (DFKI GmbH) and the Technical University of
Kaiserslautern, under the supervision of Prof. Dr. Didier Stricker. His areas
of interest include deep learning for computer vision specifically in object
detection and activity recognition. He is also interested in the area of
pattern recognition and document analysis. Previously, he has worked in the
field of document layout understanding and post-OCR error corrections. He is
also a Reviewer for IEEE Access.
---|---
|  Marcus Liwicki received his M.S. degree in Computer Science from the Free
University of Berlin, Germany, in 2004, his PhD degree from the University of
Bern, Switzerland, in 2007, and his habilitation degree at the Technical
University of Kaiserslautern, Germany, in 2011. Currently he is chaired
professor at Luleå University of Technology and a senior assistant in the
University of Fribourg. His research interests include machine learning,
pattern recognition, artificial intelligence, human computer interaction,
digital humanities, knowledge management, ubiquitous intuitive input devices,
document analysis, and graph matching. He is a member of the IAPR, editor or
regular reviewer for international journals, including IEEE Transactions on
Pattern Analysis and Machine Intelligence, IEEE Transactions on Audio, Speech
and Language Processing, International Journal of Document Analysis and
Recognition (editor), Frontiers of Computer science (editor), Frontiers in
Digital Humanities (editor), Pattern Recognition, and Pattern Recognition
Letters. He is a member of governing board the International Graphonomics
Society and a member of the International Association for Pattern Recognition
where he is Vice president of the Technical Committee 6. He chaired several
International Workshops on Automated Forensic Handwriting Analysis and the
International Workshop on Document Analysis Systems 2014. Furthermore he
serves as program committee member and reviewer of various International
Conferences and workshops in the area of Computer Vision, Pattern Recognition
and Document Analysis as well as Machine Learning and E-Learning.
---|---
|  Didier Stricker is professor with the University of Kaiserslautern and
scientific director with the “German Research Center for Artificial
Intelligence” (DFKI) in Kaiserslautern where he leads the research department
Augmented Vision. From 2002 to June 2008, he lead the department “Virtual and
Augmented Reality” at the Fraunhofer Institute for Computer Graphics
(Fraunhofer IGD) in Darmstadt, Germany. In this function, he initiated and
participated to many national and international projects in the areas of
computer vision and virtual and augmented reality. In 2006, he received the
Innovation Prize of the German Society of Computer Science. He serves as
reviewer for different European or national research organizations, and is a
regular reviewer for the most important journals and conferences in the areas
of VR/AR and computer vision.
---|---
|  Muhammad Noman Afzal completed his bachelor’s degree in Computer Science
from Islamia University of Bahawalpur, Pakistan. He is currently involved in
research and development in the area of Artificial Intelligence. He is a deep
learning enthusiast. He has over 7 years of work experience with different
types of deep learning techniques. However, he is mostly interested in object
detection. His general interests are deep learning in challenging
environments. He has also worked with deploying artificial intelligence at the
edge. He has vast experience in mobile development. Furthermore, he is
involved in academia where he delivers lectures on machine learning.
---|---
|  Muhammad Zeshan Afzal received his Masters degree from the University of
Saarland, Germany majoring in Visual Computing in 2010 and his Ph.D. degree
from the University of Technology, Kaiserslautern, Germany majoring in
Artificial Intelligence in 2016. His research interests include deep learning
for vision and language understanding using deep learning. At an application
level, his experience includes generic segmentation framework for natural,
human activity recognition, document and, medical image analysis, scene text
detection, and recognition, on-line and off-line gesture recognition.
Moreover, a special interest in recurrent neural networks for sequence
processing applied to images and videos. He also worked with numerics for
tensor valued images. He worked both in the industry (Deep Learning and AI
Lead Insiders Technologies GmbH) and academia (TU Kaiserslautern). He received
the gold medal for the best graduating student in Computer Science from IUB
Pakistan in 2002 and secured a DAAD(Germany) fellowship in 2007. He is a
member of IAPR.
---|---