# Review of Computational Epigraphy

Vishal K

###### Abstract

Computational Epigraphy refers to the process of extracting text from stone
inscription, transliteration, interpretation, and attribution with the aid of
computational methods. Traditional epigraphy methods are time consuming, and
tend to damage the stone inscriptions while extracting text. Additionally,
interpretation and attribution are subjective and can vary between different
epigraphers. However, using modern computation methods can not only be used to
extract text, but also interpret and attribute the text in a robust way. We
survey and document the existing computational methods that aid in the above-
mentioned tasks in epigraphy.

##  1 Introduction

###  1.1 Epigraphs

Stone inscriptions are important artifacts in the field of archaeology.
Although several cultures follow different methods as primary forms of
writing, for example, palm leaf manuscripts by Dravidians, papyrus manuscripts
by Egyptians, and Animal Hide manuscripts by several European civilizations,
stone inscriptions remained a robust secondary form of writing across all the
civilizations that practiced writing. This is mainly due to the robustness
associated with the medium, as it is impossible to manipulate, change the
stone inscriptions and very difficult to mutilate them. Therefore, several
historically important documents such as treaties, grants, and tombstones are
engraved in stones.

###  1.2 Interpretation of Epigraphs

Traditional epigraphy practice involves the use of inks to take the imprint of
the inscription. This process has a high probability of mutilating the
inscription. After that, an expert epigrapher will try to interpret the
characters in the inscription.

###  1.3 Challenges in Interpreting Epigraphs

Traditional epigraphy practice involves the use of inks to take the imprint of
the inscription. This process has a high probability of mutilating the
inscription. After that, an expert epigrapher will try to interpret the
characters in the inscription.

###  1.4 Time and Place of origin

Adding on to that, though written within the same script, inscriptions can
vary according to the location and time of origin, since people who lived in
different eras and different parts of the same territory will practice
different dialects of a script. Therefore, attributing the inscription with
their time and place of origin is also important while interpreting the
inscription. But it is challenging because several stone inscriptions have
been moved from their location of origin in due course of history.

###  1.5 Mutilation

Similarly, mutilation of inscriptions also proved to be a serious factor while
interpreting the inscriptions. As we have seen above, misinterpreting a few
characters might change the meaning of the inscription. But stone inscriptions
are supposed to undergo mutilation due to wear and tear and due to humans
(invading armies, accidents, etc.)

###  1.6 Computational Epigraphy

The recent rise of computational imaging and computational linguistics, which
are in turn caused by the rise of machine learning, has begun to be used for
epigraphy. Hence, a new field of computational epigraphy is at its infancy.
These technologies don’t aim to replace the epigraphers. But they are aimed to
assist them in their work. We have divided computational epigraphy into two
domains. One is to transliterate the characters present in the inscription.
This involves taking the image of the stone inscription, pre-processing,
binarizing, denoising, segmenting individual characters and recognizing those
characters. The second part focuses on assigning attributes to the
transliterated text. These attributes can be time and place of origin,
identities of the named entities in the text, finding the missing text,
predicting the sequence of multiple texts, etc.

##  2 Literature Review

We have compiled a list of papers in the field of computational epigraphy.
This does not include papers from adjacent field of Handwritten character
recognition, because, even though they’re similar in some ways, stone
inscriptions provide unique challenges due to the medium. In addition, we have
collected the research regardless of the language and script being used. The
following figure shows the distribution of languages/scripts in which
computational epigraphy is performed in our literature.

###  2.1 Transliteration

Transliteration is the process of identifying individual characters in the
stone epigraph. It may involve mapping the characters from the old script to
the modern script of the same language, if that language still exists.

For example, while transliterating Tamil epigraphs, the characters are mapped
to modern Tamil characters, as the phonetics of the characters and grammar of
the language remained the same. However, it is not always possible, as some
languages became extinct and do not have modern form. For example, Egyptian
hieroglyphs cannot be mapped to modern characters.

Transliteration in Computational Epigraphy often involves, imaging the
inscription, processing the image, segmentation at line, word and character
level and identifying individual characters. As mentioned earlier several
interpretations of the same character can exist; therefore, it is desirable to
have explanations for each interpretation.

####  2.1.1 Cryptography

Transliteration of Indus script poses an unique challenge. Because, unlike
most other scripts in this paper, the Indus script has not been deciphered
yet. There are no multilingual epigraphs identified belonging to the Indus
Valley Civilization. Also, the scripts evolved from Indus script like Brahmi
are widely changed from the original Indus script. Therefore, it remains
undecipherable till the date. Figure 1 shows an inscription discovered at one
of the excavation sites, which has 34 Indus Valley Script characters, which is
longest discovered till the date

However, several statistical studies have been made to identify patterns in
Indus script, which we will see in later section. In this paper, [1] , the
authors try to decipher Indus Script using Cryptography.

Cryptography was a field emerged during World War II. It is discovered as a
tool to decipher the secret messages of Nazi Germany from Enigma Machine [2]
by Alan Turing and others.

Here the author considers an Indus script similar to Enigma code and try to
decipher the key using crypt analysis. They consider each symbol to be a
mathematical code and try to decipher them in a known language. Though they
made some progress, eventually it failed as Indus script proved to be too
complex to be solved using symbolic mathematics.

Figure 1: Copper plate with Indus Valley Script

####  2.1.2 Cuneiform Transliteration

Another script that is challenging to transliterate is Cuneiform. It is a form
of writing practiced in Central Asia and Far East. It involves writing in clay
tablets using wedge-shaped letters and baking those tablets.

Clay tablets, unlike stone, are more fragile. Therefore, even though the
cuneiform script is cracked, it is difficult to transliterate the inscription
due to wear and tear.

In [3], the authors propose a rule-based method to decode cuneiform. They
convert Cuneiform imprints to ”intensity curves” where a graph is drawn based
on pixel intensities across a particular axis. Based on the shape of graph,
the characters are classified into one of the known character sets.

Another rule-based approach is used by [4], converts the characters into
bezier splines and a rule based system is constructed to classify the control
points of the splines.

A rule tree is constructed based on the several features extracted using
different image processing techniques, to classify the Cuneiform Letters in
[5]

Some cuneiform tablets had broken into several pieces. [6], they use 3d CAD
modeling to reconstruct the tablets and extract the text using photogrammetry.

Paper |  Method |  Remarks
---|---|---
Al-Bayatti et al. |  Intensity Curves |  Intensity curves are obtained by getting pixel intensities of the characters in a particular axis. Characters are classified by their unique intensity curves.
Bogacz et al. |  Bezier Splines |  Each stroke of the character is converted into a Bezier spline. Then they used a rule based system to classify those characters from the control points.
Aktas et al. |  Multiple Image Processing methods and Rule based system |  They process the individual Cuneiform characters using multiple image processing techniques. They use a rule tree to classify those characters.
Mara et al. |  3D modelling and CAD |  This method uses 3D scan to construct the 3D model of the Tablet. Then they use CAD techniques to extract and identify the characters.
Table 1: Cuneiform Character Recognition Figure 2: Imprint of Cuneiform Tablet

####  2.1.3 Supervised Learning

Supervised learning has proven to be a valuable tool for character recognition
and transliteration, particularly when the script is already deciphered. In
this section, we review several techniques that employ supervised learning for
character recognition and transliteration in various scripts.

One approach is the use of template images to denoise inscriptions obtained
from stone inscriptions, as demonstrated in [7]. The authors perform image
correlation on a noiseless template image of an inscription to enhance its
clarity.

In [8], the authors utilize gradient and intensity-based filters to extract
features from ancient Kannada language inscriptions. The features are then
transliterated to modern Kannada symbols using fuzzy logic.

Scale Invariant Feature Transform is used to extract features and identify the
decoration and the text areas in ancient manuscripts in [9]

In [10, 11], complex transforms such as Shape and Hough transforms are used to
extract features from epigraph images, and characters are classified using
swarm optimization algorithms like Group Search Optimization and Firefly
optimization. Also along with [11, 12] uses Median fuzzy filters to classify
the characters.

The use of Gabor filters for feature extraction in Old Latin and French texts
is described in [13]. The authors evaluate word segmentation using several
methods and show that Support Vector Machines (SVMs) perform well in this
task.

[14] discusses a method, where set of invariant features are extracted from
documents of middle age Persian origin. Then these features are used to
classify the characters using Minimum Mean Distance Classsifiers and K-Nearest
Neighbour Classifiers.

SVMs are also used for character classification in cuneiform scripts in [15,
16]. In [15], multiple SVM classifiers are used as an ensemble to classify
cuneiform characters based on multiple extracted features. In [16], SVMs are
used to classify individual character images, which are then used in a Hidden
Markov Model for further attribution.

In [17], a Fisher vector is used for feature extraction, and a neural network
is used for character classification. Chain code and Fourier transform
features are used in [18], and two CNNs are utilized for character
classification.

Various versions of Convolution Neural Networks (CNNs) are used for character
recognition and classification in [19, 20, 21, 22, 23, 24, 25, 26, 27, 28]. In
[20], CNNs are used to compare Indus characters that are similar to Brahmi and
Phoenician characters. The study demonstrates that Indus script visually
resembles the Phoenician script more than the Brahmi script.

In [29], the authors extract feature vectors from the image patches of rock
carvings through Convolutional Neural Networks. The attention values of these
feature vectors are calculated and these vectors are processed by LSTMs. The
LSTM predicts the character in that patch of the image.

However, the authors of [30], used one-shot approach to classify the
characters and symbols. They train a Siamese Similarity Network to build a
feature space which optimally places all the character classes for
classification. They introduced a new loss called Soft Similarity Contrast
Loss in addition to traditional Contrastive loss, to train the Siamese
network. The resulting network classifies the characters even though it had
seen very few examples of the character

The use of CNN-based auto-encoder architectures, such as U-Net and ResUNet,
for transliterating ancient Japanese characters into modern Japanese is
described in [31].

A semi-supervised method to label the characters of new scripts is introduced
in [32], where every character is clustered into buckets based on multiple
levels of abstraction automatically. Then multiple human experts label those
buckets and vote among themselves to decide whether a particular bucket is
acceptable. This only consumes 0.5 times the time and effort taken in manually
annotating all those labels.

In [33], Capsule Networks are used instead of CNNs for character
classification.

Finally, [34] utilizes Google Le Net for classifying whether or not a
character belongs to the Indus script. In [35], weakly supervised CNNs are
employed, with individual CNNs for line alignment, line and word segmentation,
character segmentation, and character recognition.

Table 2: Supervised Character Classification methods Paper |  Method |  Remarks
---|---|---
Preethi et al. |  Image Correlation |  Use a template image of a noiseless inscription and perform image correlation to denoise and classify the inscription obtained from a stone inscription
Soumya and Kumar |  Filtering |  Gradient and Intensity filters are used to extract features and classify the ancient Kannada inscriptions
Alirezaee et al. |  Invariant Moments |  A set of invariant moments are used to classify Persian Documents from Middle ages.
Suganya and Murugavalli |  Group Search Optimization and Firefly algorithm |  Shape and Hough Transforms are used to extract features from the characters. GSO and Firefly algorithms are used to classify them.
Mahalakshmi and Sharavanan, Devi and Maheswari |  Particle Swarm Optimization |  Image of scripts are enhanced using Contourlet transform, denoised using fuzzy median filters and classified using Particle Swarm optimization
Garz et al. |  SIFT |  Scale invariant Feature transform is used to detect the text areas from decorative elements in manuscripts.
RajaKumar and Bharathi |  Support Vector Machines |  Features are extracted from Old Latin and French texts which doesn’t have inter-word spaces using Gabor Filter. The features extracted are then classified using Support Vector Machines
Bogacz et al. |  SVM and Hidden Markov Models |  Characters of Cuneiform texts are segmented and ensemble of SVM and Hidden markov model is used to learn the sequence of the characters and classify them.
Amato et al. |  Fisher Vector and Neural Network |  Fisher Vector of the individual characters are extracted as the features and they are classified using Neural Networks.
Chacko and Dhanya |  Convolutional Neural Networks |  Chain code and Fourier transform features extracted and used in two CNNs to classify characters
Avadesh and Goyal, Daggumati and Revesz, Sudarsan et al., Giridhar et al., Gautam et al., Magrina, Narang et al., Wijerathna et al., Rajnish et al., M et al. |  Various CNN models |  Various CNN models such as AlexNet, ResNet, LeNet are used to classify the individual characters
Liu et al. |  Siamese Network |  One shot character Recognition using Siamese similarity network and Soft similarity Contrast Loss
Daggumati and Revesz |  CNNs |  Indus Script characters compared to Brahmi and Phoenician characters using CNNs, Indus script shown to be visually closer to Phoenician script
Ezhilarasi et al. |  CNN-LSTM Network |  The visual features are extracted using CNNs and those feature vectors are then processed by LSTMs to classify the characters.
Clanuwat et al. |  Autoencoder |  Ancient Japanese characters transliterated to modern Japanese using CNN-based Auto-encoder architectures such as U Net and ResUnet
Vajda et al. |  Semi-supervised Learning |  Images are clustered based on different abstractions and human experts vote on to the labels assigned.
Kumar et al. |  Capsule Network |  Characters classified using Capsule Network instead of CNNs
Palaniappan and Adhikari |  LeNet |  Characters classified whether they belonging to Indus script using Google Le Net
Dencker et al. |  Weakly supervised CNNs |  Line alignment, segmentation of lines, words, characters and character recognition using individual CNNs.

###  2.2 Attribution

Attribution refers to the task of assigning metadata to transliterated texts.
This can be sequence/pattern prediction in case of undeciphered texts, place,
missing text prediction, sequence prediction of broken tablets, place and time
of origin prediction, etc. In some cases, the meaning of the epigraph can also
be translated into modern language.

Even though the characters are interpreted in the transliteration part,
attribution enables the epigrapher to interpret the meaning of an inscription.

Traditionally, the epigrapher will use his domain knowledge to interpret an
inscription, which is subjective and always debatable.

In computational epigraphy, attribution is usually done by a system that
understands the distribution of the lexicons in the particular script. This
could be anything ranging from a frequency table to a deep learning NLP model.

####  2.2.1 Statistical Analysis of Indus Script

As discussed above, Indus script is undeciphered, therefore several
statistical analyses are done on those inscriptions attempting to decode it.
[36, 37, 38, 39, 40, 41, 42] are research which attempted at deciphering Indus
script using Statistical analysis. While most of them drew frequency tables
for each Indus script character and tried to predict the next character based
on probability, [37] used n-grams (considering an n-character window at a
time) and tried to predict the characters. These studies revealed several
underlying patterns within the Indus script. For example, one of the results
of [36] is that the frequency tables of the Indus script and the Brahmi script
(which is deciphered) are related. Similarly [38] showed several recurrent
pattern of characters in Indus script. This shows that the Indus script is
indeed a semantic script and has proper grammar, unlike pictorial scripts like
Egyptian hieroglyphs. Similarly [43] used evolutionary algorithms to optimize
KL divergence between distributions of Indus and Brahmi scripts and found the
respective Brahmi characters that are most similar to Indus Characters in
terms of semantics.

[44] designed a Bayesian Classifier to predict the probability of next
character in Indus texts.

[45] attempts to group the characters in Indus Script into commonly used
phenomes using K-Means clustering. [46] uses same K-Means clustering algorithm
to group Indus script into recurrent patterns.

####  2.2.2 Reconstruction of Cuneiform Tablets

We have mentioned above that Cuneiform script is written on clay tablets. Due
to the nature of the medium, a lot of clay tablets are broken into several
pieces. [47, 48] discusses methods of reconstructing multiple pieces of
cuneiform tablets in 3D CAD software by matching text on both the pieces.

In particular, [48] reconstructs the famous Atrahasis Cuneiform Tablet of
which two pieces are present in Geneva and London. Due to large distance
between the location of two pieces and fragility of the medium, there has not
been any attempt to reconstruct both pieces into full tablet. This paper
successfully confirmed that both pieces belong to same tablet and extracted
the cuneiform text from it.

A framework called Gigamesh is proposed by [6], where the broken cuneiform
tablets are 3D scanned and transliterated using Giza++ program.

In contrast, [49] discusses the use of stigmerty, a property that living
organisms such as termites and bacterial colonies use to construct themselves
into the most efficient shapes to reconstruct Cuneiform tablets.

[50] uses photogrammetry,i.e., analyses the 2D images of Cuneiform tablet
imprints and reconstructs them into full texts. Naive Bayes, Maximum Entropy
classifier is used in [51] to predict the probability that the text lines in
one piece of a cuneiform tablet match the other piece.

Paper |  Method |  Remarks
---|---|---
Collins et al. |  3D CAD software reconstruction |  Reconstructs multiple pieces of cuneiform tablets in 3D CAD software by matching text on both pieces.
Collins et al. |  3D CAD software reconstruction |  Reconstructs the Atrahasis Cuneiform Tablet by confirming that both pieces belong to the same tablet and extracting the cuneiform text from it.
Mara et al. |  3D scanning and transliteration |  Proposes a framework called Gigamesh, where broken cuneiform tablets are 3D scanned and transliterated using Giza++ program.
Ch′ng et al. |  Stigmerty-based reconstruction |  Discusses the use of stigmerty to reconstruct Cuneiform tablets.
Lewis and Ch’ng |  Photogrammetry-based reconstruction |  Uses photogrammetry to analyze 2D images of Cuneiform tablet imprints and reconstruct them into full texts.
Tyndall |  Naive Bayes, Maximum Entropy classifier-based reconstruction |  Predicts the probability that the text lines in one piece of a cuneiform tablet match the other piece using Naive Bayes, Maximum Entropy classifier.
Table 3: Cuneiform Character Recognition

####  2.2.3 Predicting Metadata of the Inscriptions

Figure 3: Imprint of Maangulam Tamil-Brahmi Inscription

Epigraphs contain valuable information beyond what is explicitly written on
them. For instance, Recchia et al. [52] accurately predicted the place of
origin and associated artifacts of Indus script inscriptions by using Semantic
Latent Analysis, a commonly used method in computational linguistics.

Based on the success of Handwriting recognition, the writers of Greek stone
inscriptions are identified in [53]. The statistical features are extracted
from the images are characters and based on those features, the writers are
classified. Fractal Dimension Method is used to classify Ancient Arabic
documents from Latin documents using [54]

The era on which the epigraphs are written in Kannada Language epigraphs are
found using Support vector Machines according to [55] and [56]. Here visual
features are extracted and era of the epigraphs are classified by Support
Vector Machine in [55] and Transductive SVMs in [56]

Figure 4: Architecture of model proposed by [57] to restore and attribute
Greek inscriptions

In a similar vein, Pabasara et al. and Surasinghe and Kokul [59, 58] predicted
the period of origin of Sinhala epigraphs and extracted visual features of
ancient Sinhalese using convolutional neural networks. Meanwhile, Vani et al.
[60] used handcrafted features like Zernike Moments and HoG features to
extract visual information from epigraphs. Similarly [61] uses Zernike Moments
and Normalized Central Moments extracted from segmented characters and uses
Random Forest to Classify the characters.

Figure 5: Architecture propsed by [62], to do POS tagging and Lemmatization of
Tamil Inscriptions

Bi-directional LSTMs are used for Lemmatization and Part of Speech tagging in
texts from ancient epigraphs in [62]

Transliterated epigraphs often suffer from missing characters due to wear and
tear, making them difficult to interpret. To address this problem, Assael et
al. [57, 63] utilized Long Short-Term Memory networks and Transformers to
predict missing words in Greek epigraphs and even predict the time and date of
their origin. Saret et al. [64] employed LSTM and BERT to fill missing letters
in Akkadian texts, while Fetaya [65] used LSTM networks to predict missing
characters in Babylonian script. Generative Adversarial Networks (GANs) along
with Charbonnier Loss Function to denoise stone inscriptions and inpaint the
missing segments of them in [66]

In terms of translating epigraphs, Pagé-Perron et al. [67] used traditional
NLP techniques such as stemmatization, morphological analysis, and POS tagging
to translate Sumerian epigraphs to English, while Park [68] utilized
Transformers to translate Ancient Korean to English.

Figure 6: Architecture of Siamese Similarity Network proposed by [30]. It
mainly includes multi-scale fusion backbone structure (MSB), embedding
structure (EB), fusion distance layer (D), similarity layer (S) and soft
similarity contrast loss (SSCL)

Finally, Choo et al. [69] used Transformer architecture to restore and
translate ancient Korean historical documents into modern English, and
performed topic modeling to identify major themes in the documents.

Table 4: Attribution of Epigraphy Paper |  Method |  Remarks
---|---|---
Recchia and Louwerse |  Semantic Latent Analysis |  Accurately predicts place of origin and associated artifacts using Indus script text
Pabasara and Kokul, Surasinghe and Kokul |  Convolutional Neural Networks |  Predicts period of origin of Sinhala epigraphs and extracts visual features using CNNs
Vani and Ananthalakshmi |  Handcrafted Features |  Uses Zernike Moments and HoG features
Zaghden et al. |  Fractal Dimension Method |  Fractal Dimension Method is used to classify Arabic Documents from Latin Documents.
Soumya and Kumar, Kumar |  Support Vector Machines |  SVMs are used to predict the periods of Tamil and Kannada Epigraphs
Soumya and Kumar |  Random Forests |  Zerinike and Normalized central moments are extracted from the images and Random Forests are used to classify them.
Papaodysseus et al. |  Writer Classification by statistical methods |  Finds the writer of the Greek inscription based on the statistical values of the segmented character.
Assael et al., Res |  LSTM, Transformer |  Predicts missing characters in Greek epigraphs, predicts time and date of origin, provides explanations based on saliency maps
Ezhilarasi and Maheswari |  Bidirectional LSTMs |  Lemmatization and Part of speech tagging is done by Attention and Bi-directional LSTMs.
Saret et al. |  LSTM, BERT |  Fills missing letters in Akkadian texts
Fetaya et al. |  LSTM |  Predicts missing characters in Babylonian script
Pagé-Perron et al. |  Traditional NLP Techniques |  Translates Sumerian epigraphs to English
Zhang et al. |  GANs and charbonnier Loss |  GANs and Charbonnier Loss function is used to denoise images of epigraphs and inpain the missing segments.
Park et al. |  Transformer |  Translates Ancient Korean to English
Choo et al. |  Transformer, Topic Modeling |  Restores ancient Korean historical documents, translates to modern English, identifies major themes

##  3 Conclusion

We have discussed in detail the use of computation to aid in Epigraphical and
Archaeological discoveries. We have seen that computational methods are so
beneficial to epigraphers as they have not only simplified the tasks of
epigraphers but also opened new ways to interpret epigraphs. For example, we
have seen several computational ways to decipher Indus script which has not
been deciphered with traditional methods. We have also seen that it is
important to attribute the epigraphs along with transliterations in order to
obtain meaningful interpretations of epigraphs.

The advent of Machine Learning Results seem to provide huge breakthroughs in
the field of Computational Archaeology, especially in the field of Epigraphy.
We have seen several models have provided deciphered previously undecipherable
inscriptions and also solved historical debates. Also, most models simplify
the pipeline of interpreting those inscriptions which by conventional methods
is a very complex process. Therefore we conclude that Computational methods
are slowly becoming the default way of approaching epigraphy.

##  4 References

## References

  * Chadha [2010] A. Chadha, “Cryptographic imagination Indus script and the project of scientific decipherment,” _Indian Economic and Social History Review_ , 2010\.
  * Cry [2022] “Cryptanalysis of the Enigma,” _Wikipedia_ , Nov. 2022.
  * Al-Bayatti et al. [2006] H. M. Y. Al-Bayatti, A. M. Rahma, and H. Alani, “Cuneiform symbols recognition using intensity curves.” _The International Arab Journal of Information Technology_ , 2006.
  * Bogacz et al. [2015] B. Bogacz, M. Gertz, and H. Mara, “Character retrieval of vectorized cuneiform script,” ser. International Conference on Document Analysis and Recognition, 2015\.
  * Aktas et al. [2019] A. Z. Aktas, B. Yesiltepe, and T. Asuroglu, “Computerized hittite cuneiform sign recognition and knowledge-based system application examples,” _European Scientific Journal, ESJ_ , 2019.
  * Mara et al. [2010] H. Mara, S. Krömker, S. Jakob, and B. Breuckmann, “GigaMesh and gilgamesh: –3D multiscale integral invariant cuneiform character extraction,” ser. International Conference on Virtual Reality, 2010\.
  * Preethi et al. [2020] P. Preethi, A. Kasi, M. M. Shetty, and H. R. Mamatha, “Multiscale template matching to denoise epigraphical estampages,” 2020.
  * Soumya and Kumar [2015] A. Soumya and G. H. Kumar, “Feature extraction and recognition of ancient kannada epigraphs,” 2015.
  * Garz et al. [2010] A. Garz, M. Diem, and R. Sablatnig, “Detecting text areas and decorative elements in ancient manuscripts,” 2010.
  * Suganya and Murugavalli [2017] T. S. Suganya and S. Murugavalli, “Feature selection for an automated ancient Tamil script classification system using machine learning techniques,” _2017 International Conference on Algorithms, Methodology, Models and Applications in Emerging Technologies (ICAMMAET)_ , 2017.
  * Mahalakshmi and Sharavanan [2013] M. Mahalakshmi and M. Sharavanan, “Ancient Tamil script and recognition and translation using LabVIEW,” ser. International Conference on Communication and Signal Processing, 2013.
  * Devi and Maheswari [2019] K. D. Devi and P. U. Maheswari, “Digital acquisition and character extraction from stone inscription images using modified fuzzy entropy-based adaptive thresholding,” _Soft Computing_ , 2019.
  * RajaKumar and Bharathi [2012] S. RajaKumar and V. S. Bharathi, “Eighth century tamil consonants recognition from stone inscriptions,” ser. International Conference on Recent Trends in Information Technology, 2012.
  * Alirezaee et al. [2004] S. Alirezaee, H. Aghaeinia, M. Ahmadi, and K. Faez, “Recognition of middle age Persian characters using a set of invariant moments,” 2004.
  * Homburg and Chiarcos [2016] T. Homburg and C. Chiarcos, “Word segmentation for akkadian cuneiform,” ser. Language Resources and Evaluation, 2016.
  * Bogacz et al. [2017] B. Bogacz, M. Klingmann, and H. Mara, “Automating transliteration of cuneiform from parallel lines with sparse data,” ser. International Conference on Document Analysis and Recognition, 2017.
  * Amato et al. [2016] G. Amato, F. Falchi, and L. Vadicamo, “Visual recognition of ancient inscriptions using convolutional neural network and fisher vector,” _ACM Journal on Computing and Cultural Heritage_ , 2016.
  * Chacko and Dhanya [2015] A. M. M. O. Chacko and P. M. Dhanya, “Combining classifiers for offline malayalam character recognition,” 2015.
  * Avadesh and Goyal [2018] M. Avadesh and N. Goyal, “Optical character recognition for sanskrit using convolution neural networks,” ser. Document Analysis Systems, 2018.
  * Daggumati and Revesz [2018] S. Daggumati and P. Z. Revesz, “Data mining ancient script image data using convolutional neural networks,” ser. International Database Engineering and Applications Symposium, 2018.
  * Sudarsan et al. [2018] D. Sudarsan, P. Vijayakumar, S. Biju, S. Sanu, and S. K. Shivadas, “Digitalization of malayalam palmleaf manuscripts based on contrast-based adaptive binarization and convolutional neural networks,” ser. International Conference on Wireless Communications and Signal Processing, 2018.
  * Giridhar et al. [2019] L. Giridhar, A. Dharani, and V. Guruviah, “A novel approach to OCR using image recognition based classification for ancient tamil inscriptions in temples,” 2019.
  * Gautam et al. [2020] N. Gautam, S. S. Chai, and J. Jose, “Recognition of brahmi words by using deep convolutional neural network,” 2020.
  * Magrina [2020] M. M. Magrina, “Convolution neural network based ancient tamil character recognition from epigraphical inscriptions,” 2020.
  * Narang et al. [2021] S. R. Narang, M. Kumar, and M. K. Jindal, “DeepNetDevanagari: A deep learning model for Devanagari ancient character recognition,” _Multimedia Tools and Applications_ , 2021.
  * Wijerathna et al. [2019] K. N. Wijerathna, R. Sepalitha, T. Indika, H. Athauda, P. Suranjini, J. Silva, and A. Jayakodi, “Recognition and translation of Ancient Brahmi Letters using deep learning and NLP,” _2019 International Conference on Advancements in Computing (ICAC)_ , 2019.
  * Rajnish et al. [2023] P. Rajnish, K. P. Kamath, B. Kumar, M. Nishanth, and P. Preethi, “Improving the quality and readability of ancient brahmi stone inscriptions,” 2023.
  * M et al. [2022] P. S. M, H. J, B. V. T, V. K. S, I. N, J. M, and I. K, “CNN based character recognition and classification in tamil palm leaf manuscripts,” 2022\.
  * Ezhilarasi et al. [2023] S. Ezhilarasi, P. UmaMaheswari, and S. Raghavi, “Recognition of characters using PCE based convolutional LSTM networks from palaeographic writings,” 2023.
  * Liu et al. [2022] X. Liu, G. Weizhe, L. Rankang, Y. Xiong, X. Tang, S. Chen, X. Liu, G. Weizhe, L. Rankang, Y. Xiong, X. Tang, and S. Chen, “One shot ancient character recognition with siamese similarity network,” 2022.
  * Clanuwat et al. [2019] T. Clanuwat, A. Lamb, and A. Kitamoto, “KuroNet: Pre-modern japanese kuzushiji character recognition with deep learning.” 2019.
  * Vajda et al. [2011] S. Vajda, A. Junaidi, and G. A. Fink, “A semi-supervised ensemble learning approach for character labeling with minimal human effort,” 2011.
  * Kumar et al. [2021] A. Kumar, F. Rizk, D. Rizk, and R. Rizk, “A hybrid capsule network-based deep learning framework for deciphering ancient scripts with scarce annotations: A case study on phoenician epigraphy,” _2021 IEEE International Midwest Symposium on Circuits and Systems (MWSCAS)_ , 2021.
  * Palaniappan and Adhikari [2017] S. Palaniappan and R. Adhikari, “Deep learning the indus script,” 2017.
  * Dencker et al. [2020] T. Dencker, P. Klinkisch, S. M. Maul, and B. Ommer, “Deep learning of cuneiform sign detection with weak supervision using transliteration alignment.” _PLOS ONE_ , 2020.
  * Kak [1988] S. C. Kak, “A FREQUENCY ANALYSIS OF THE INDUS SCRIPT,” _Cryptologia_ , 1988.
  * Yadav et al. [2010] N. Yadav, H. Joglekar, R. P. N. Rao, M. N. Vahia, R. Adhikari, and I. Mahadevan, “Statistical analysis of the indus script using n-Grams,” _PLOS ONE_ , 2010.
  * Yadav et al. [2008a] N. Yadav, M. N. Vahia, I. Mahadevan, and H. Joglekar, “A STATISTICAL APPROACH FOR PATTERN SEARCH IN INDUS WRITING,” _IJDL. International journal of Dravidian linguistics_ , 2008.
  * Yadav et al. [2008b] ——, “SEGMENTATION OF INDUS TEXTS,” _IJDL. International journal of Dravidian linguistics_ , 2008.
  * Oakes [2019] M. P. Oakes, “Statistical analysis of the tables in mahadevan’s concordance of the indus valley script,” _Journal of Quantitative Linguistics_ , 2019.
  * Yadav and Vahia [2011] N. Yadav and M. N. Vahia, “Classification of patterns on Indus objects,” 2011\.
  * Daggumati and Revesz [2021] S. Daggumati and P. Z. Revesz, “A method of identifying allographs in undeciphered scripts and its application to the Indus Valley Script,” 2021\.
  * Das et al. [2009] S. Das, A. Ahuja, B. Natarajan, and B. K. Panigrahi, “Multi-objective optimization of Kullback-Leibler divergence between Indus and Brahmi writing,” ser. Nature and Biologically Inspired Computing, 2009.
  * Snyder et al. [2010] B. Snyder, R. Barzilay, and K. Knight, “A statistical model for lost language decipherment,” ser. Meeting of the Association for Computational Linguistics, 2010.
  * Yadav et al. [2017] N. Yadav, A. Salgaonkar, and M. Vahia, “Clustering indus texts using K-means,” _International Journal of Computer Applications_ , 2017.
  * Yadav et al. [2014] N. Yadav, A. Salgaonkar, and M. N. Vahia, “Computational techniques for inferring the syntax of un-deciphered scripts.” 2014.
  * Collins et al. [2014] T. Collins, S. I. Woolley, L. H. Munoz, A. Lewis, E. Ch’ng, and E. Gehlken, “Computer-assisted reconstruction of virtual fragmented cuneiform tablets,” ser. Virtual Systems and Multimedia, 2014.
  * Collins et al. [2017] T. Collins, S. I. Woolley, E. Gehlken, and E. Ch’ng, “Computational aspects of model acquisition and join geometry for the virtual reconstruction of the atrahasis cuneiform tablet,” _2017 23rd International Conference on Virtual System & Multimedia (VSMM)_, 2017.
  * Ch′ng et al. [2013] E. Ch′ng, A. Lewis, R. E. Gehlken, and S. I. Woolley, “A theoretical framework for stigmergetic reconstruction of ancient text,” _Visual Heritage in the Digital Age_ , 2013.
  * Lewis and Ch’ng [2012] A. Lewis and E. Ch’ng, “A photogrammetric analysis of cuneiform tablets for the purpose of digital reconstruction,” _International Journal of Heritage in the Digital Era_ , 2012.
  * Tyndall [2012] S. Tyndall, “Toward automatically assembling hittite-language cuneiform tablet fragments into larger texts,” ser. Meeting of the Association for Computational Linguistics, 2012.
  * Recchia and Louwerse [2016] G. L. Recchia and M. M. Louwerse, “Archaeology through computational linguistics : Inscription statistics predict excavation sites of indus valley artifacts,” _Cognitive Science_ , 2016.
  * Papaodysseus et al. [2010] C. Papaodysseus, P. Rousopoulos, D. Arabadjis, F. Panopoulou, and M. Panagopoulos, “Handwriting automatic classification: Application to ancient Greek inscriptions,” 2010.
  * Zaghden et al. [2011] N. Zaghden, R. Mullot, and A. Alimi, “Characterization of ancient document images composed by Arabic and Latin scripts,” ser. 2011 International Conference on Innovations in Information Technology, 2011.
  * Soumya and Kumar [2011] A. Soumya and G. Kumar, “SVM CLASSIFIER FOR THE PREDICTION OF ERA OF AN EPIGRAPHICAL SCRIPT,” 2011.
  * Kumar [2014] S. K. Kumar, “An efficient period prediction system for tamil epigraphical scripts using transductive support vector machine,” 2014.
  * Assael et al. [2019] Y. M. Assael, T. Sommerschield, and J. Prag, “Restoring ancient text using deep learning: A case study on Greek epigraphy,” ser. Empirical Methods in Natural Language Processing, 2019.
  * Surasinghe and Kokul [2021] P. Surasinghe and T. Kokul, “Period prediction of sinhala epigraphical scripts using convolutional neural networks,” ser. International Conference on Advances in ICT for Emerging Regions, 2021.
  * Pabasara and Kokul [2021] S. Pabasara and T. Kokul, “Period Prediction of Sinhala Epigraphical Scripts using Convolutional Neural Networks,” in _2021 21st International Conference on Advances in ICT for Emerging Regions (ICter)_. IEEE, 2021, pp. 141–146.
  * Vani and Ananthalakshmi [2020] V. Vani and S. R. Ananthalakshmi, “Soft computing approaches for character credential and word prophecy analysis with stone encryptions,” ser. Soft Computing, 2020.
  * Soumya and Kumar [2014] A. Soumya and G. Kumar, “Classification of ancient epigraphs into different periods using random forests,” ser. 2014 Fifth International Conference on Signal and Image Processing, 2014.
  * Ezhilarasi and Maheswari [2021] S. Ezhilarasi and P. Maheswari, “Depicting a neural model for lemmatization and POS tagging of words from palaeographic stone inscriptions,” ser. 2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS), 2021.
  * [63] “Restoring and attributing ancient texts using deep neural networks — Nature,” https://www.nature.com/articles/s41586-022-04448-z.
  * Saret et al. [2021] B. Saret, A. Yehudai, K. Lazar, N. Wasserman, G. Stanovsky, and W. Horowitz, “Filling the gaps in ancient akkadian texts: A masked language modelling approach,” _EMNLP_ , 2021.
  * Fetaya et al. [2020] E. Fetaya, Y. Lifshitz, E. Aaron, and S. Gordin, “Restoration of fragmentary Babylonian texts using recurrent neural networks.” _Proceedings of the National Academy of Sciences of the United States of America_ , 2020.
  * Zhang et al. [2021] H. Zhang, Y. Qi, X. Xue, and Y. Nan, “Ancient stone inscription image denoising and inpainting methods based on deep neural networks,” 2021.
  * Pagé-Perron et al. [2017] É. Pagé-Perron, M. Sukhareva, I. Khait, and C. Chiarcos, “Machine translation and automated analysis of the sumerian language,” _LaTeCH@ACL_ , 2017.
  * Park et al. [2020] C. Park, C. Lee, Y. Yang, and H. Lim, “Ancient korean neural machine translation,” _IEEE access : practical innovations, open solutions_ , 2020\.
  * Choo et al. [2021] J. Choo, Y. Kim, S. Jang, S. Yang, K. Jin, and K. Kang, “Restoring and mining the records of the joseon dynasty via neural language modeling and machine translation,” _NAACL_ , 2021.


# Accurate and Robust Scale Recovery for Monocular Visual Odometry Based on
Plane Geometry

Rui Tian1, Yunzhou Zhang1∗, Delong Zhu2∗, Shiwen Liang1, Sonya Coleman3,
Dermot Kerr3 ∗The corresponding author of this paper. 1Rui Tian, Yunzhou Zhang
and Shiwen Liang are with College of Information Science and Engineering,
Northeastern University, Shenyang 110819, China (Email:
zhangyunzhou@mail.neu.edu.cn).2Delong Zhu is with the Department of Electronic
Engineering, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR,
China (Email: zhudelong@link.cuhk.edu.hk).3Sonya Coleman and Dermot Kerr are
with School of Computing and Intelligent Systems,Ulster University, N.
Ireland, UK.This work was supported by National Natural Science Foundation of
China (No. 61973066, 61471110) , Fundation of Key Laboratory of Aerospace
System Simulation(6142002200301), Fundation of Key Laboratory of Equipment
Reliability(61420030302), Fundamental Research Funds for the Central
Universities(N182608004, N2004022) and the Distinguished Creative Talent
Program of Liaoning Colleges and Universities (LR2019027).

###### Abstract

Scale ambiguity is a fundamental problem in monocular visual odometry. Typical
solutions include loop closure detection and environment information mining.
For applications like self-driving cars, loop closure is not always available,
hence mining prior knowledge from the environment becomes a more promising
approach. In this paper, with the assumption of a constant height of the
camera above the ground, we develop a light-weight scale recovery framework
leveraging an accurate and robust estimation of the ground plane. The
framework includes a ground point extraction algorithm for selecting high-
quality points on the ground plane, and a ground point aggregation algorithm
for joining the extracted ground points in a local sliding window. Based on
the aggregated data, the scale is finally recovered by solving a least-squares
problem using a RANSAC-based optimizer. Sufficient data and robust optimizer
enable a highly accurate scale recovery. Experiments on the KITTI dataset show
that the proposed framework can achieve state-of-the-art accuracy in terms of
translation errors, while maintaining competitive performance on the rotation
error. Due to the light-weight design, our framework also demonstrates a high
frequency of 20 Hz on the dataset.

##  I INTRODUCTION

Monocular Visual Odometry (MVO) is a popular method for camera pose
estimation, but due to the scale ambiguity [1, 2, 3, 4, 5], the MVO system
cannot provide real odometry data. Therefore, an accurate and robust scale
recovery algorithm is of great significance in the application of MVO [6]. The
key of scale recovery is to integrate absolute reference information, such as
the gravity orientation from IMU [7] or the depth measured by Lidar [8, 9].
The baseline of stereo cameras can also serve as such a reference [10].
However, these sensors are not always available in real-world applications.
Moreover, a complicated sensor calibration and fusion process is needed to
align the MVO system with other sensors.

Another frequently used method for removing scale ambiguity is to take as
reference the height of a mounted camera above the ground plane, which remains
a stable signal during the navigation of vehicles. The idea is to estimate a
ratio between the real camera height and the relative one calculated from
image features. The ratio then can be used to recover the real scale. The
advantages of this method are significant since it does not depend on other
sensors and is with high feasibility. The method is also regarded as one of
the most promising solutions in this research area.

Prior work like [11, 2, 1, 12] typically leverages the results of feature
matching to calculate the homography matrix and then decompose the matrix to
estimate the parameters of the ground plane, based on which the relative
camera height can be obtained. The major deficiency of this method is that the
decomposition is very sensitive to noises and multiple solutions exist, which
requires additional operations to eliminate the ambiguity. Some other work
like [13, 12] chooses to directly fit the ground plane using feature points
that lie on the ground, e.g., the center regions of a road. By removing low-
quality image features outside the target region, the robustness is improved.
However, the target region may be occluded sometimes, which interferes with
the detection of image features, thus degrading the accuracy of scale
recovery.

In recent work [14, 15, 16, 17, 18], deep learning based MVO algorithms are
proposed, in which the camera pose with a real scale is directly predicted by
the neural network in an end-to-end manner. Such methods have received much
attention in recent years, but their generalization ability across different
scenarios is very limited [18]. Some other deep learning based methods take
scale recovery as an independent problem. For instance, DNet [16] is proposed
to perform ground segmentation and depth regression simultaneously. Based on
the predicted depth points within the ground region, a dense geometrical
constraint is then formulated to help recover the scale. In [17], a scale-
recovery loss is developed based on the idea of enforcing the consistency
between the known camera height and the predicted one. Constrained by this
loss, the neural network can predict more accurate ego poses. Nonetheless,
these methods usually require a large-scale training process, and the
computational cost is prohibitively expensive.

In this paper, we propose a light-weight method for accurate and robust scale
recovery based on plane geometry. The method includes an efficient Ground
Point Extraction (GPE) algorithm based on Delaunay triangulation [19] and a
Ground Points Aggregation (GPA) algorithm for aggregating ground points from
consecutive frames. Based on these two algorithms, a large number of high-
quality ground points are selected. For scale recovery, we first formulate a
least-square problem to fit the ground plane and then estimate the relative
camera height to calculate the real scale. By leveraging the high-quality
points and a RANSAC-based optimizer, the scale can be estimated accurately and
robustly. Benefiting from the light-weight design of the algorithms, our
method can achieve a 20Hz running frequency on the benchmark dataset.

The main contributions of this work are as follows:

  * •

We propose a GPE algorithm based on Delaunay triangulation, which can
accurately extract ground points.

  * •

We propose a GPA algorithm that can effectively aggregate local ground points
and perform robust optimization of the ground plane parameters.

  * •

Based on the proposed algorithms, we implement a real-time MVO system with
accurate and robust scale recovery, aiming to reduce scale drift and provide
accurate odometry in long-distance navigations without loop closure.

Figure 1: Overview of the proposed system. There are two parallel threads in
the system: 1) The MVO thread takes image frames as input and estimates the
current camera pose, 2) The GPE-GPA thread fetches image features from the MVO
thread and selects high-quality points for ground plane estimation.

##  II System Overview

The notations used in this paper are as follows:

  * •

$T_{t}\in{R^{4\times 4}}$ \- The camera pose of image frame $I_{t}$ in the
global frame, which is composed of a camera orientation $R_{t}\in{R^{3\times
3}}$ and a translation $\boldsymbol{t}_{t}\in{R^{3\times 1}}$.

  * •

$T_{t,t-1}$ \- The relative pose of frame $I_{t-1}$ w.r.t. frame $I_{t}$.

  * •

$K$ \- The intrinsic matrix of a pinhole camera model.

  * •

$\mathbf{x}_{i},\mathbf{u}_{i}$ \- The 3D map point in camera frame and its
corresponded 2D point on image plane after projection.

  * •

$\mathbf{n}_{t},h_{t}$ \- The plane parameters, i.e.,
$\mathbf{n}_{t}\cdot\mathbf{x}_{i}-h_{t}=\mathbf{0}$, where $\mathbf{n}_{t}$
is the normal vector and $h_{t}$ is the distance to the plane.

  * •

$h,{h}^{\dagger},h^{*}$ – The calculated camera height form image features,
the estimated camera height after scale recovery, and the real camera height.

###  II-A Problem Definition

Given consecutive image frames from a calibrated monocular camera, our goal is
to estimate the absolute scale of camera poses and then recover the real
camera trajectory by making use of the prior known camera height $h^{*}$.
Under scale ambiguity, the camera height $h$ calculated from image features
maintains a ratio with the real one, i.e., $s=h^{*}/h$. Therefore, scale
recovery is essentially to compute $s$, and the key lies in the accurate
estimation of the ground plane.

###  II-B System Architecture

The proposed system in this work is shown in Fig. 1. There are two parallel
threads in the system: The first one is the MVO thread, which takes
consecutive images as input and estimates the current camera pose, e.g., the
ORB-SLAM2 framework. The second thread is used to run the GPE and GPA
algorithms for scale recovery. The proposed system is based on such an
assumption that the ground is locally flat and can be approximated by a plane
with a surface normal. The workflow of the second thread is as follows.

As shown in the red block in Fig.1, for each image frame from the MVO thread,
the Delaunay triangulation is first applied to segment the matched feature
points into a set of triangles. Each triangle is then back-projected into the
camera frame, and the associated plane parameters are also estimated. After
that, several geometrical constraints are leveraged to select and then refine
ground points.

Note that selected ground points are not enough for an accurate estimation of
the plane parameters. We thus propose the GPA algorithm to aggregate ground
points from multiple frames using a sliding windows method, as shown in the
orange block of Fig.1. Based on the aggregated local points, a robust
parameter estimation procedure is then performed to fit the ground plane.
Accordingly, the relative camera height of each frame can be estimated, and
the absolute camera trajectory is recovered, shown in the blue block of Fig.1.

##  III Ground Plane Estimation

###  III-A Ground Point Extraction

For a given set of matched feature points $\mathbf{u}^{t}_{i}$,
$i\in\\{1,2,\dots,N\\}$, in the current image frame $I_{t}$, the Delaunay
triangulation uses each of the feature points as a triangle vertex. We back-
project the triangles from the image plane into the current camera frame and
denote them by $\Delta_{i}^{t},i\in\\{1,2,\dots M\\}$ associated with a set of
vertices $\mathbf{x}^{t}_{ij},j\in\\{1,2,3\\}$. The normal vector
$\mathbf{n}_{i}^{t}=({n}_{i,x}^{t},{n}_{i,y}^{t},{n}_{i,z}^{t})$ of each
triangle can be obtained by the cross product,

$\mathbf{n}_{i}^{t}=\frac{(\mathbf{x}^{t}_{i1}-\mathbf{x}^{t}_{i2})\times(\mathbf{x}^{t}_{i1}-\mathbf{x}^{t}_{i3})}{||(\mathbf{x}^{t}_{i1}-\mathbf{x}^{t}_{i2})\times(\mathbf{x}^{t}_{i1}-\mathbf{x}^{t}_{i3})||_{2}},$
(1)

where $\mathbf{n}_{i}^{t}$ has an unit length. For each vertex of the
triangle, the following geometrical constraint then holds:

$\mathbf{n}^{t}_{i}\cdot\mathbf{x}^{t}_{ij}-h^{t}_{i}=0.$ (2)

Therefore, $h^{t}_{i}$ can also be estimated. Here, we also add two additional
constraints, i.e., ${n}_{i,y}^{t}>0$ and $h^{t}_{i}>0$, based on the fact that
the camera is mounted on the top of the vehicle and is above the ground plane,
as shown in Fig.1.

Note that the triangles are scattered in the whole image plane, hence we need
to identify the ones located on the ground plane, named ground triangles, for
estimating the plane parameters. Based on the fact that the normal of a ground
triangles is orthogonal to camera translation $\boldsymbol{t}_{t,t-1}$, and
that the pitch angle of the camera is zero, the ground triangles can be
identified by testing with the following constraints,

$\displaystyle\arccos(\mathbf{n}^{t}_{i},\boldsymbol{t}_{t,t-1})$
$\displaystyle=0,$ (3) $\displaystyle|\arctan(-\frac{R_{32}}{R_{33}})|=0,$
$\displaystyle R_{33}\neq 0.$

In practice, the equality condition cannot be strictly satisfied. We thus set
a tolerance value of $5^{\circ}$ in the actural implementation.

For ground triangles that satisfy the above constraints, their vertices are
categorized into a new point set $\tilde{\mathbf{x}}^{t}_{ij},i\in\\{1,2,\dots
K\\},j\in\\{1,2,3\\},K\small{<}M$. Since the same vertex point may be shared
by multiple triangles, we also need to remove the repetitive ones from the
point set. This will ensure the same contribution of each point to the ground
plane estimation.

The ground points are now initially segmented out, denoted by
$\tilde{\mathbf{x}}^{t}_{k}\in\mathcal{G}$, but there may still exist some
outliers introduced by moving objects and some remote points. To further
improve the quality of $\mathcal{G}$, a RANSAC-based method is leveraged to
optimize $\tilde{\mathbf{x}}^{t}_{k}$, which minimizes a plane-distance error
as follows,

$\min_{\tilde{\mathbf{x}}^{t}_{g}\in\mathcal{G}}\;\sum_{g=1}^{|\mathcal{G}|}||\mathbf{n}^{t}\cdot\tilde{\mathbf{x}}^{t}_{g}-h^{t}||_{2}.$
(4)

In the implementation, we randomly sample three points to estimate a new plane
with eq. (1)-(2), and then we calculate the total distance of the remaining
points to the estimated plane. Such a process repeats $Z$ times, and the plane
that induces the minimum total distance error is reserved. The points with a
distance larger than $0.01$m to the reserved plane are then removed. This is
actually a more strict criterion for ground point selection. After this
process, only high-quality ground points are reserved. In Alg. 1, we present a
complete procedure of the GPE algorithm, which gives more details about the
proposed implementation.

Input: $\mathbf{u}^{t}_{i}$ , $R_{t,t-1}$, $\boldsymbol{t}_{t,t-1}$

Output: $\\{\tilde{\mathbf{x}}^{t}_{g}\\}$

1

2$\\{\Delta_{i}^{t}\\}$$\leftarrow$
DelaunayTriangulation($\mathbf{u}^{t}_{i}$)

3triangles points set $\\{\tilde{\mathbf{x}}^{t}_{ij}\\}\leftarrow\emptyset$

4segmented points set $\\{\tilde{\mathbf{x}}^{t}_{k}\\}\leftarrow\emptyset$

5ground points set $\mathcal{G}_{best}\leftarrow\emptyset$

6temp ground points set $\mathcal{G}_{k}\leftarrow\emptyset$

7for  _each $\mathbf{u}_{ij}^{t}\in\\{\Delta_{i}^{t}\\},j=\\{1,2,3\\}$ _ do

8

9 back-project $\mathbf{x}^{t}_{ij}$=$K^{-1}\mathbf{u}_{ij}^{t}$

10 calculate $\mathbf{n}_{i}^{t}$ by Eq.(1)

11 calculate $h_{i}^{t}$ by Eq.(2)

12 if  _$|\arctan(-\frac{R_{32}}{R_{33}})| <\theta_{1}$ & $h^{t}_{i}>0$_ then

13 if  _$\arccos(\mathbf{n}^{t}_{i},\boldsymbol{t}_{t,t-1}) <\theta_{2}$_ then

14

15 $\\{\tilde{\mathbf{x}}^{t}_{ij}\\}\cup\mathbf{x}^{t}_{ij}$

16 end if

17

18 end if

19

20 end for

21

22delete repeate vertices and get
$\\{\tilde{\mathbf{x}}^{t}_{k}\\}\subset\\{\tilde{\mathbf{x}}^{t}_{ij}\\}$

23/* _ensure enough points_ */

24 while  _size ( $\\{\tilde{\mathbf{x}}^{t}_{k}\\}$) $>$ 5 _ do

25

26 for  _iterations $<Z$_ do

27 randomly select
$\mathbf{x}^{t}_{1},\mathbf{x}^{t}_{2},\mathbf{x}^{t}_{3}\in\\{\tilde{\mathbf{x}}^{t}_{k}\\}$

28 calculate $\mathbf{n}^{t}$ by Eq.(1)

29 for  _$\tilde{\mathbf{x}}^{t}_{g}$ in $\\{\tilde{\mathbf{x}}^{t}_{k}\\}$_
do

30 $D\leftarrow|\mathbf{n}^{t}\cdot\tilde{\mathbf{x}}^{t}_{g}-h^{t}|$ by
Eq.(4)

31 if  _$D <d_{thresh}$_ then

32 $\mathcal{G}_{k}\cup\\{\tilde{\mathbf{x}}^{t}_{g}\\}$

33 end if

34

35 end for

36 if  _size $(\mathcal{G}_{k})$ $>$ size $(\mathcal{G}_{best})$ _ then

$\mathcal{G}_{best}\leftarrow\mathcal{G}_{k}$ // best model select

37

38 end if

39

40 end for

41 $\\{\tilde{\mathbf{x}}^{t}_{g}\\}\leftarrow\mathcal{G}_{best}$

42 return _$\\{\tilde{\mathbf{x}}^{t}_{g}\\}$_

43 end while

Algorithm 1 Ground Point Extraction (GPE)

###  III-B Ground Point Aggregation

Due to the strict criteria by GPE, the inliers are not enough for accurate
estimation of the ground plane. Therefore, we propose the GPA algorithm to
aggregate ground points from consecutive image frames. As shown in Fig. 2, we
leverage the sliding window method to select image frames, and a frame buffer
is maintained to store the camera poses and ground points in the current
window. At each time step, with the arrival of a new image frame, we update
the buffer and then estimate the ground plane by solving a least-squares
problem.

Figure 2: Illustration of the GPA algorithm. In the bottom-left figure, the
red dots indicate ground points, and the green line segment is the normal of
each triangle. In the bottom-right figure, the red quadrilateral is the
estimated ground plane based on the aggregated ground points in the current
window.

From the MVO thread, we can get the pose $T_{t}$ and the inliers
$\mathbf{x}^{t}_{g}$ of each frame from the buffer. We then can transform each
of the inliers, denoted by $\mathbf{x}^{t}_{gi}$, into the global frame,

$\mathbf{p}_{i}=T_{t}\,[\mathbf{x}^{t}_{gi},1]^{T}.$ (5)

Suppose there are $N$ local ground points in the buffer, the least-squares
problem that minimizes the plane-distance error of these points is formulated
as follows:

$\displaystyle\min_{\boldsymbol{\mu}}\sum_{i=1}^{N}\|\boldsymbol{\mu}^{T}\mathbf{p}_{i}\|_{2}$
$\displaystyle=\min_{\boldsymbol{\mu}}\boldsymbol{\mu}{P_{t}}{P_{t}}^{T}\boldsymbol{\mu}^{T},$
(6) $\displaystyle\boldsymbol{\mu}=$
$\displaystyle[\mathbf{n}_{t},-h_{t}]^{T},$

where ${P_{t}}=[\mathbf{p}_{1},\mathbf{p}_{2},\cdots,\mathbf{p}_{N}]\in
R^{4\times N}$. Equation (6) can be rewritten as follows:

$\displaystyle\min_{\boldsymbol{\mu}}\boldsymbol{\mu}Q_{t}\boldsymbol{\mu}^{T},$
(7) $\displaystyle Q_{t}$ $\displaystyle={P}_{t}{P}_{t}^{T}\in R^{4\times 4},$

which can then be efficiently solved by the SVD method.

To further improve the estimation accuracy of $\boldsymbol{\mu}$, we also
introduce a weighting matrix $\Sigma={\sigma^{-2}_{z}}I$, where $\sigma_{z}$
is the normalized distance of the point depth to their mean value. As a
result, the matrix $Q$ in Eq. (7) becomes,

$Q_{t}={P}_{t}\Sigma{P}_{t}^{T}\in R^{4\times 4}.$ (8)

Another important refinement on the estimated plane parameter is to conduct a
RANSAC-based optimization, which shares the same idea as Eq. (4). In each
iteration of the optimization, we first estimate $\boldsymbol{\mu}$, and then
calculate the distance between $\mathbf{p}_{i}$ and the estimated plane.
Points with a distance larger than $0.01$m are removed, and the remaining is
then leveraged to estimate a new $\boldsymbol{\mu}$. Such a process continues
until convergence. We denote the final plane normal by $\mathbf{n}_{t}^{*}$
and the reserved ground points by $\mathbf{p}_{k}^{*},k\in\\{1,2,\cdots,K\\}$.
The relative camera height then can be calculated by projecting the camera
center to the ground plane:

$h^{t}_{j}=\mathbf{n}_{t}^{*}\cdot(\mathbf{p}_{c}-\mathbf{p}_{k}^{*}),$ (9)

where $\mathbf{p}_{c}$ is the camera center of frame $I_{t}$ in the global
frame. It is worth noting that there are $K$ estimated camera heights, which
will be further processed to recover a smooth scale. Details of the GPA
algorithm are presented in Alg. 2.

1

Input: $I_{t}$, $\\{\mathbf{x}^{t}_{g}\\}$, $T_{t}$

2

Output: $\\{h^{t}\\}$

3

4buffer $\\{queue\\}\leftarrow\emptyset$

5inliers in global frame $\\{\mathbf{p}_{t}\\}\leftarrow\emptyset$

6inliers in current frame $\\{\mathbf{x}_{g}^{t}\\}\leftarrow\emptyset$

7reserved ground points $\\{\mathbf{p}_{k}^{*}\\}\leftarrow\emptyset$

8camera heights $\\{h_{t}\\}\leftarrow\emptyset$

9while  _$I_{t}$ is not empty_ do

10 $\\{queue\\}\cup\\{I_{t}\\}$

11 if  _$size(\\{queue\\}) >4$_ then

12

13 pop$\\_$front($\\{queue\\}$)

// fixed number of frames

14 for  _$each$ $I_{t}$ in $\\{queue\\}$_ do

15 calculate $\mathbf{p}_{i}$ by Eq.(5)

16 $\\{\mathbf{p}_{t}\\}\cup\\{\mathbf{p}_{i}\\}$

17 end for

18

19 Matrix $P_{t}$

calculate $\Sigma={\sigma^{-2}_{z}}I$ // Weighting Matrix

20

21 calculate $Q_{t}$ by Eq.(8)

22 $\boldsymbol{\mu}\leftarrow\textsc{SVDDecomposition}(Q)$

23 end if

24

25 for  _iteration $<Z$_ do

26 Randomly select $\mathbf{p}_{s}\in\\{\mathbf{p}_{t}\\}$

27 $D\leftarrow\boldsymbol{\mu}^{T}\cdot\mathbf{p}_{s}$

28 if  _$D >d_{thresh}$_ then

29

30 Remove $\mathbf{p}_{s}$ in $\\{\mathbf{p}_{t}\\}$

31 end if

32 $\\{\mathbf{p}_{k}^{*}\\}\leftarrow\\{\mathbf{p}_{t}\\}$

33 calsulate $Q_{s}$ by Eq.(8)

34 $Q\leftarrow Q_{s}$

35 $\boldsymbol{\mu_{s}}\leftarrow\textsc{SVDDecomposition}(Q)$

$\boldsymbol{\mu}\leftarrow\boldsymbol{\mu_{s}}$ // update model

36

37 end for

38

39 for  _$\mathbf{p}^{*}$ in $\\{\mathbf{p}_{k}^{*}\\}$_ do

40 calculate $h_{j}^{t}$ by Eq.(9)

41 $\\{h^{t}\\}\cup\\{h_{j}^{t}\\}$

42 end for

43 return _$\\{h^{t}\\}$_

44 end while

45

Algorithm 2 Ground Points Aggregation (GPA)

###  III-C Filter-Based Scale Recovery

After we compute the relative camera height $h$ of each frame, the scale
factor is then obtained by $s_{t}=h^{*}/h$, the motion scale of each frame is
recovered by

$\boldsymbol{t}^{\dagger}_{t,t-1}=s_{t}\cdot\boldsymbol{t}_{t,t-1}.$ (10)

Corresponding to the multiple $h$ values in Eq. (9), there are also multiple
estimated scales.

By plotting the scaled camera heights of each frame in the figure, shown in
Fig. 3(a), we find the data do not strictly follow a Gaussian distribution.
Therefore, we choose the median point as the scale of the current frame. In
the time domain, a moving average filter is applied, shown in Fig. 3(b), which
can give a more smooth result.

(a) The distribution of the scaled camera heights.

(b) The estimated camera height on sequence-02 and -05.

Figure 3: Demonstration of the filter-based scale recovery. The green points
are the scaled camera heights. The red curve is the smoothed one.

##  IV Experiments

We conduct experiments to evaluate the performance of our proposed method. The
MVO system used in the experiments is implemented based on ORB-SLAM2, and the
proposed scale recovery method is integrated as an independent thread. The
system architecture is demonstrated in Fig. 1. The KITTI dataset [20] is
adopted as the benchmark dataset, in which sequence-01 is not used since it
fails most feature-based VO systems. All the experiments are conducted using a
laptop with Intel(R) Core(TM) i5-6300HQ CPU @ 2.30 GHz.

Figure 4: The re-scaled trajectories on KITTI dataset. The blue and green
trajectories are generated by our system and ORB-SLAM2-noLC, respectively.

###  IV-A Qualitative Evaluation

The qualitative evaluation results of the proposed method are visualized in
Fig. 4. The trajectories outputted by the system are recovered using the
proposed method, which means similarity transformation is not necessary when
compared with ground-truth trajectories[21]. The baseline trajectories,
indicated by green color in Fig. 4, are generated by ORB-SLAM2 with loop
closure detection disabled, denoted by ORB-SLAM2-noLC. We can see that our re-
scaled trajectories can eliminate scale drift and form correct loops, which
demonstrates the effectiveness of the proposed method.

The comparison of trajectory length between the ground truth and the proposed
method is shown in Table I, in which the Relative Length Error (RLE) is
computed by $e=|l_{gt}-l_{ours}|/|l_{gt}|$, where $l$ is the length of a
trajectory. For sequence-02, -04, -06, -07, and -10, the RLE is less than
1$\%$. The high performance is due to the fact that most roads in these
sequences are straight lines and contain rich features. Sequence-00 and -08
are more complicated cases, in which the scenario is composed of a lot of
curves and turns. The path lengths are relatively longer than other sequences.
The RLE is thus slightly higher, 2.17$\%$ and 2.72$\%$, respectively.
Nevertheless, the results show that the proposed system can estimate an
accurate scale of the entire trajectory.

TABLE I: Comparison of re-scaled trajectory
length with ground truth Seq | GT (m) | Ours (m) | RLE (%)
---|---|---|---
00 | 3645.213 | 3724.419 | 2.173
02 | 5071.151 | 5067.233 | 0.757
03 | 547.774 | 560.888 | 0.558
04 | 391.861 | 393.645 | 0.455
05 | 2175.159 | 2205.576 | 1.398
06 | 1235.982 | 1232.876 | 0.251
07 | 709.065 | 694.697 | 0.767
08 | 3137.398 | 3222.795 | 2.722
09 | 1738.601 | 1705.051 | 1.929
10 | 911.399 | 919.518 | 0.890
TABLE II: comparison of average translation errors and rotation errors with the latest visual odometry methods on kitti dataset Seq | ORB-SLAM2-noLC [10] | VISO2-M[22] | VISO2-Stereo[22] | Song et.al[1] | Zhou et.al[11] | Brandon et.al[17] | DNet[16] | Ours |
---|---|---|---|---|---|---|---|---|---
Trans | Rot | Trans | Rot | Trans | Rot | Trans | Rot | Trans | Rot | Trans | Rot | Trans | Rot | Trans | Rot |
(%) | (deg/m) | (%) | (deg/m) | (%) | (deg/m) | (%) | (deg/m) | (%) | (deg/m) | (%) | (deg/m) | (%) | (deg/m) | (%) | (deg/m) |
00 | 20.8 | $-$ | 11.91 | 0.0209 | 2.32 | 0.0109 | 2.04 | 0.0048 | 2.17 | 0.0039 | 1.86 |  | 1.94 | $-$ | 1.41 | 0.0054 |
01 | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ |
02 | 9.52 | $-$ | 3.33 | 0.0114 | 2.01 | 0.0074 | 1.50 | 0.0035 | $-$ | $-$ | 2.27 | $-$ | 3.07 | $-$ | 2.18 | 0.0046 |
03 | $-$ | $-$ | 10.66 | 0.0197 | 2.32 | 0.0107 | 3.37 | 0.0021 | 2.70 | 0.0044 | $-$ | $-$ | $-$ | $-$ | 1.79 | 0.0041 |
04 | $-$ | $-$ | 7.40 | 0.0093 | 0.99 | 0.0081 | 2.19 | 0.0028 | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | 1.91 | 0.0021 |
05 | 18.63 | $-$ | 12.67 | 0.0328 | 1.78 | 0.0098 | 1.43 | 0.0038 | $-$ | $-$ | 1.50 | $-$ | 3.32 | $-$ | 1.61 | 0.0064 |
06 | 18.98 | $-$ | 4.74 | 0.0157 | 1.17 | 0.0072 | 2.09 | 0.0081 | $-$ | $-$ | 2.05 | $-$ | 2.74 | $-$ | 2.03 | 0.0044 |
07 | 13.82 | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | 1.78 | $-$ | 2.74 | $-$ | 1.77 | 0.0230 |
08 | 22.06 | $-$ | 13.94 | 0.0203 | 2.35 | 0.0104 | 2.37 | 0.0044 | $-$ | $-$ | 2.05 | $-$ | 2.72 | $-$ | 1.51 | 0.0076 |
09 | 12.74 | $-$ | 4.04 | 0.0143 | 2.36 | 0.0094 | 1.76 | 0.0047 | $-$ | $-$ | 1.50 | $-$ | 3.70 | $-$ | 1.77 | 0.0118 |
10 | 4.86 | $-$ | 25.20 | 0.0388 | 1.37 | 0.0086 | 2.12 | 0.0085 | 2.09 | 0.0054 | 3.70 |  | 5.09 |  | 1.25 | 0.0031 |
Avg | 18.17 | $-$ | 14.39 | 0.0245 | 2.32 | 0.0095 | 2.03 | 0.0045 | 2.32 | 0.045 | 2.03 | $-$ | 3.17 | $-$ | 1.72 | 0.0068 |

###  IV-B Quantitative Evaluation

The quantitative comparison between our method and the baseline methods,
including [17, 16, 1, 11, 22, 10], is presented in Table II. The average
translation error and rotation error are adopted as evaluation metrics.

We can see that ORB-SLAM2-noLC and VISO2-M have the worst performance due to
the lack of loop closure detection. The scale drift of the two methods induces
a large translation error, 18.17$\%$ and 14.39$\%$ respectively, while the VO
systems with scale recovery all maintain a low translation error, $<4\%$. It
can also be seen that a MVO system with scale recovery [1, 11, 16, 17] can
exhibit competitive performance with a stereo VO system like VISO2-M [22],
which significantly demonstrates the importance of scale recovery for MVO.

In terms of monocular systems, we can see our proposed method achieves the
minimum translation error while maintaining a competitive performance on
rotation error. The methods proposed by Song et al. [1] and Zhou et al. [11]
can not work with sequence-07, because they both rely on a fixed region to
extract ground points, whereas occlusions by moving vehicles occur frequently
in this sequence. In contrast with [1, 11], the proposed method works well
with sequence-07 with a translation error of 1.77$\%$, benefiting from the GPA
algorithm.

In [16], a deep neural network, named DNet, is proposed for monocular depth
prediction and scale recovery. Compared with this method, our method shows a
better accuracy in all the sequences. In [17], a real-time MVO system is
implemented based on self-supervised learning techniques. This method can
slightly outperform our proposed method in sequence-05 and -09, but has a much
lower accuracy in sequence-00, -08, and -10. A similar phenomenon can be
observed when comparing with [1]. This indicates a significant variance on the
performance of [17]. Actually, this is the limitation of most deep learning
based methods, which has been discussed in detail by [18].

The comparative experiments in Table II significantly verify the effectiveness
of our method and demonstrates the advantages over the latest methods in the
literature.

###  IV-C Efficiency Evaluation

Another significant advantage of our method lies in its high efficiency. We
evaluate the run-time of our system on all the KITTI sequences mentioned
above, and the experiment repeats five times. The median run-time is reported
in Fig. 5. In all the experiments, the MVO thread requires $50$-$55$ ms, while
the GPE and GPA requires less than $10$ ms, which makes the system suitable
for real-time applications.

##  V CONCLUSIONS

In this work, we present a light-weight MVO system with accurate and robust
scale recovery, aiming to reduce scale drift and provide accurate odometry in
long-distance navigations without loop closure. We solve the scale ambiguity
for MVO by implementing our GPE-GPA algorithm for selecting high-quality
points and optimizing them in a local sliding window. Sufficient data and
robust optimizer provide accurate metric trajectory leveraging the ratio of
the estimated camera height and the real one. Extensive experiments show that
our proposed framework can achieve state-of-the-art accuracy and recover a
metric trajectory without additional sensors. The system is designed to be a
light-weight framework, which can achieve real-time performance with 20 Hz
running frequency. Our proposed light-weight MVO system facilitates the
localization and navigation of low-cost autonomous vehicles in long-distance
scenarios.

Further study into integrating the uncertainty of plane estimation will be
considered, which will further improve the accuracy of scale recovery. The
light-weight neural network for ground segmentation will also be considered to
help constrain the extraction of high-quality ground points.

Figure 5: Run-time of our MVO system on KITTI dataset. The time costs of MVO
thread, the GPE algorithm, and the GPA algorithm are reported.

## References

  * [1] S. Song, M. Chandraker, and C. C. Guest, “High accuracy monocular sfm and scale correction for autonomous driving,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 38, no. 4, pp. 730–743, 2015.
  * [2] D. Zhou, Y. Dai, and H. Li, “Reliable scale estimation and correction for monocular visual odometry,” in _2016 IEEE Intelligent Vehicles Symposium (IV)_. IEEE, 2016, pp. 490–495.
  * [3] Y. Wu, Y. Zhang, D. Zhu, Y. Feng, S. Coleman, and D. Kerr, “Eao-slam: Monocular semi-dense object slam based on ensemble data association,” _arXiv preprint arXiv:2004.12730_ , 2020.
  * [4] D. Scaramuzza, F. Fraundorfer, M. Pollefeys, and R. Siegwart, “Absolute scale in structure from motion from a single vehicle mounted camera by exploiting nonholonomic constraints,” in _IEEE International Conference on Computer Vision_ , 2009.
  * [5] H. Strasdat, J. M. M. Montiel, and A. J. Davison, “Scale drift-aware large scale monocular slam,” in _Robotics: Science and Systems VI, Universidad de Zaragoza, Zaragoza, Spain, June 27-30, 2010_ , 2010.
  * [6] D. Zhu, Y. Du, Y. Lin, H. Li, C. Wang, X. Xu, and M. Q.-H. Meng, “Hawkeye: Open source framework for field surveillance,” in _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2017, pp. 6083–6090.
  * [7] T. Qin, P. Li, and S. Shen, “Vins-mono: A robust and versatile monocular visual-inertial state estimator,” _IEEE Transactions on Robotics_ , vol. 34, no. 4, pp. 1004–1020, 2018.
  * [8] J. Zhang, M. Kaess, and S. Singh, “Real-time depth enhanced monocular odometry,” in _2014 IEEE/RSJ International Conference on Intelligent Robots and Systems_. IEEE, 2014, pp. 4973–4980.
  * [9] T. Li, D. Zhu, and M. Q.-H. Meng, “A hybrid 3dof pose estimation method based on camera and lidar data,” in _2017 IEEE International Conference on Robotics and Biomimetics (ROBIO)_. IEEE, 2017, pp. 361–366.
  * [10] R. Mur-Artal and J. D. Tardós, “Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras,” _IEEE Transactions on Robotics_ , vol. 33, no. 5, pp. 1255–1262, 2017.
  * [11] D. Zhou, Y. Dai, and H. Li, “Ground-plane-based absolute scale estimation for monocular visual odometry,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 21, no. 2, pp. 791–802, 2019.
  * [12] Qifa Ke and T. Kanade, “Transforming camera geometry to a virtual downward-looking camera: robust ego-motion estimation and ground-layer detection,” in _2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings._ , vol. 1, 2003, pp. I–I.
  * [13] J. Gräter, T. Schwarze, and M. Lauer, “Robust scale estimation for monocular visual odometry using structure from motion and vanishing points,” in _2015 IEEE Intelligent Vehicles Symposium (IV)_. IEEE, 2015, pp. 475–480.
  * [14] X. Yin, X. Wang, X. Du, and Q. Chen, “Scale recovery for monocular visual odometry using depth estimated with deep convolutional neural fields,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 5870–5878.
  * [15] L. Andraghetti, P. Myriokefalitakis, P. L. Dovesi, B. Luque, M. Poggi, A. Pieropan, and S. Mattoccia, “Enhancing self-supervised monocular depth estimation with traditional visual odometry,” in _2019 International Conference on 3D Vision (3DV)_. IEEE, 2019, pp. 424–433.
  * [16] F. Xue, G. Zhuo, Z. Huang, W. Fu, Z. Wu, and M. H. Ang Jr, “Toward hierarchical self-supervised monocular absolute depth estimation for autonomous driving applications,” _arXiv preprint arXiv:2004.05560_ , 2020\.
  * [17] B. Wagstaff and J. Kelly, “Self-supervised scale recovery for monocular depth and egomotion estimation,” _arXiv preprint arXiv:2009.03787_ , 2020.
  * [18] W. Wang, D. Zhu, X. Wang, Y. Hu, Y. Qiu, C. Wang, Y. Hu, A. Kapoor, and S. Scherer, “Tartanair: A dataset to push the limits of visual slam,” _arXiv preprint arXiv:2003.14338_ , 2020.
  * [19] J. R. Shewchuk, “Triangle: Engineering a 2d quality mesh generator and delaunay triangulator,” in _Workshop on Applied Computational Geometry_. Springer, 1996, pp. 203–222.
  * [20] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in _2012 IEEE Conference on Computer Vision and Pattern Recognition_. IEEE, 2012, pp. 3354–3361.
  * [21] M. Grupp, “evo: Python package for the evaluation of odometry and slam.” https://github.com/MichaelGrupp/evo, 2017.
  * [22] A. Geiger, J. Ziegler, and C. Stiller, “Stereoscan: Dense 3d reconstruction in real-time,” in _Intelligent Vehicles Symposium (IV), 2011 IEEE_ , 2011.