# OpenStreetMap: Challenges and Opportunities in Machine Learning and Remote
Sensing

John E. Vargas-Muñoz∗, Shivangi Srivastava∗, Devis Tuia, ,
and Alexandre X. Falcão John E. Vargas Muñoz, and Alexandre X. Falcão are with
the Laboratory of Image Data Science, Institute of Computing, University of
Campinas, Brazil. E-mail<EMAIL_ADDRESS>(corresponding
author)Shivangi Srivastava and Devis Tuia are with the Laboratory of Geo-
information Science and Remote Sensing, Wageningen University & Research, the
Netherlands.∗These authors contributed equally to this manuscript.

###### Abstract

This is the preprint version. To read the final version please go to IEEE
Geoscience and Remote Sensing Magazine on IEEE XPlore:
https://doi.org/10.1109/MGRS.2020.2994107. OpenStreetMap (OSM) is a community-
based, freely available, editable map service that was created as an
alternative to authoritative ones. Given that it is edited mainly by
volunteers with different mapping skills, the completeness and quality of its
annotations are heterogeneous across different geographical locations. Despite
that, OSM has been widely used in several applications in Geosciences, Earth
Observation and environmental sciences. In this work, we present a review of
recent methods based on machine learning to improve and use OSM data. Such
methods aim either 1) at improving the coverage and quality of OSM layers,
typically using GIS and remote sensing technologies, or 2) at using the
existing OSM layers to train models based on image data to serve applications
like navigation or land use classification. We believe that OSM (as well as
other sources of open land maps) can change the way we interpret remote
sensing data and that the synergy with machine learning can scale
participatory map making and its quality to the level needed to serve global
and up-to-date land mapping. A preliminary version of this manuscript has been
presented in the first author’s dissertation [1].

###### Index Terms:

OpenStreetMap; machine learning; remote sensing, volunteered geographic
information; deep learning.

##  I Introduction

Mapping systems need to be reliable and frequently updated, which makes them
costly to be maintained. Due to limited budget, authoritative maps are usually
not fully updated at regular time intervals, and present temporal, spatial,
and completeness inaccuracies. Recently, Volunteered Geographic Information
(VGI) [2] has appeared as an alternative to authoritative map services. VGI
collects mapping information from individuals, usually volunteers, and stores
the information in a database which is often freely available. OpenStreetMap
(OSM) is one of the most successful VGI projects. It started in 2004 and
currently counts more than 5 million users 111https://osmstats.neis-one.org/
from different parts of the world. This gives OSM the potential to provide
updated mapping data at global scale.

OSM information is represented by four types of data: nodes, ways, relations,
and tags, which are constantly edited by volunteers.

  * -

A node is a location on the earth’s surface, as determined by latitude and
longitude coordinates. Points of Interests (POIs) such as bus stations can be
represented by nodes.

  * -

A way is a list of nodes forming polylines, that can represent road networks
or areal objects (closed ways) like buildings.

  * -

A relation represents the relationship among objects — e.g., a group of road
segments can represent a bus route.

  * -

A tag is a key-value pair that contains information of an object — e.g., a
restaurant can be represented by a way with a tag “amenity=restaurant”.

Although the OSM data is constantly under improvement, the completeness and
quality of the annotations in different regions are affected by the number and
mapping skills of the volunteers [3]. As reported in [4], the spatial coverage
of OSM is heterogeneous in different geographical regions — i.e., urban areas
are more regularly updated than rural areas. In road networks, missing roads
are reported in [5] and inaccurate road tags are reported in [6]. The
positional accuracy of building footprints in OSM sometimes requires
corrections [7]. Several works in the literature have studied methods to
assess the quality of OSM data by quantifying: data completeness [8],
positional accuracy [9], semantic tag acccuracy [10], and topological
consistency [11]. Some works focus on meta analysis of OSM, like the analysis
of the contributors’ activities [12, 13] and the quality assessment of the OSM
data [14, 15].

Despite its completeness and quality issues, OSM has been widely used for
several applications: e.g., validation of land cover maps [16]; land
cover/land use classification [17, 18, 19]; navigation and routing
applications like traffic estimation [20] and pedestrian, bicycle, and
wheelchair routing [21, 22]; detection of buildings and roads in aerial
imagery [23, 24]; 3D city modelling [25]; indoor mapping [26]; and location-
based map services [27].

In the recent years, the automation of tasks involving OSM data has received
increasing attention: on one hand, research aiming at the improvement of the
OSM layers has turned to Earth observation and machine learning algorithms as
automatic ways to complete footprints in missing areas and verify specific
annotations, mostly to ease and decrease the workload of volunteers. On the
other hand, the rise of deep learning [28] has found in OSM a very valuable
source of label information to train large models for image recognition from
Earth Observation data, but also brought issues related to the quality,
standardization, and completeness of the data used for training the models. In
both cases, and in countless applications, the alliance of machine learning,
Earth Observation, and OSM layers is proving to be an enabling factor for
tackling global challenges in new ways.

To present the potential and opportunities of OSM for geoscience and remote
sensing research, in this paper we present a review of methods based on
machine learning to improve and use OSM for applications in different domains.
Note that, we do not intend to cover an exhaustive list of OSM applications,
but we focus on the ones that involve machine learning techniques only.

In Section II, we review methods based on machine learning to improve the
completeness and quality of the OSM objects, such as building footprints,
street networks, and points of interest. Section III reviews the works based
on machine learning for applications like land cover/land use classification,
navigation, and fine-scale population estimation. Section IV summarizes the
discussion and draws promising future research areas at the interface of ML
and VGI. Section V states the conclusion about this paper.

##  II Improving OSM data with machine learning

The next sections describe methods based on machine learning to improve
different types of OSM annotations: building footprints, street networks,
semantic tags, and points of interest.

###  II-A Building footprints

Detecting geometric mismatches

Buildings are one of the most widely annotated objects in OSM. Although the
geometrical features and tags of the buildings in OSM are usually correct
(especially in urban areas), there are cases where the building footprints are
not accurately mapped by volunteers. Figure 1

presents examples of incomplete OSM building annotations in the cities of São
Paulo and Amsterdam. An autoencoder neural network method is proposed in [7]
to measure the accuracy of OSM building annotations with respect to official
governmental data in the city of Toronto. The authors extract geometrical
mismatch features to train an autoencoder neural network. Then, the
reconstruction error predicted with the trained model is interpreted as a
score that represents the quality of the annotation for a particular region.
This method could be useful to import building footprints from other sources
to OSM, since the proposed score could be used to identify where are the most
mismatched regions that need to be carefully analyzed by annotators.

|
---|---
a) | b)
Figure 1: Examples of OSM building annotations (regions highlighted in blue)
with completeness errors superimposed over Bing aerial imagery: a) incomplete
annotation of buildings in the city of São Paulo, b) incomplete annotation of
buildings in the city of Amsterdam.

Detecting vandalism behavior

Some building annotations in OSM are intentionally edited with wrong
geometries, such cases are known as digital vandalism. In order to identify
vandalism in OSM data, the authors in [29] propose a rule-based system that
analyzes temporal data of user annotations. For the case of buildings, in
particular the authors in [30] propose a clustering-based method to detect
vandalism of building annotations. This method extracts geometrical features
from the OSM vectorial building data (e.g., perimeter, elongation, convexity,
and compacity) and then finds groups in the feaure space to detect outliers,
which are assumed to be possible vandalized building footprints.

Correct and create new annotations

In addition to the geometrical features of OSM building annotations, other
works in the literature use aerial imagery to correct building annotations
[23, 31]. In [23], the authors propose a methodology to correct rural building
annotations in OSM. The paper points out three common problems in OSM building
annotations in rural areas: they are geometrically misaligned (see Fig. 2a),
some annotations do not correspond to buildings in the updated aerial imagery
(Fig. 2b), and some OSM annotations are missing for buildings that are present
in the updated aerial imagery (Fig. 2c). The authors propose solutions for the
three issues by using Markov Random Fields (MRF) to align annotations and
remove annotations using a building probability map obtained by a
Convolutional Neural Network (CNN). The last step of the method is the
prediction of new building annotations that are missing by using a CNN model
that predicts building footprint with predefined shape priors. The method in
[31] aims at correcting OSM building annotations by using contour information
from image segmentation of oblique images, acquired by Unmanned Aerial
Vehicles (UAV). The paper uses contour information of multiple-view images and
3D building models to correct OSM building annotations. Some companies have
also made great efforts to improve the geometrical completeness of OSM.
Microsoft has used deep learning models to compute new building footprints by
processing satellite imagery in the United States of America
222https://blogs.bing.com/maps/2018-06/microsoft-releases-125-million-
building-footprints-in-the-us-as-open-data.

|  |
---|---|---
a) | b) | c)
Figure 2: Examples of rural building annotations errors found in OSM
superimposed over Bing aerial imagery (polygons in blue represent building
annotations): a) misalignment annotation errors, b) annotations that do not
match with any building, c) missing building annotations.

###  II-B Street Network

The quality of street network information in OSM is crucial for several
applications. The quality of street type tags and the completeness of the road
network data are critical for route planning, while the street names are
important to perform queries on the OSM map. As reported in [11, 32], road
networks in OSM present heterogeneous quality and some completeness errors
even in urban areas.

Correcting topology

In [5] the authors propose a method to improve the completeness of road
networks. Specifically, they present a method based on machine learning to
identify missing roads between candidate locations (two nodes of the OSM road
network). The method extracts several features from each pair of candidates
from OSM data such as connectivity, street type, and node degree in the OSM
road network. The work shows empirically some evidence that the shortest path
distance between two nodes in an OSM road network is correlated with the
straight line distance. The extracted features are then used to train a
Logistic Regression classifier to predict missing roads. The last step
involves the pruning of some predictions to increase their precision.

Extracting roads from aerial images

The automatic extraction of road networks has also been attempted by analyzing
remote sensing imagery. The work proposed in [33] performs per pixel
classification using a CNN-based method and later obtains the centerline of
the roads. The authors of [34] also use CNNs and centerline computation, but
additionally they correct some gaps in the extracted road network by
generating several possible missing road candidates and selecting some of them
with the help of another CNN. Recently, more accurate results have been
obtained by some methods [35, 36] that iteratively construct the road network
graph, by adding new edges to the graph. The authors in [37] propose a method
that is as accurate as [35] but much more efficient. This CNN-based method
output road directions for each pixel to create a road network.

The authors of [38] propose a CNN-based method that uses aerial imagery as
well as ground-based pictures, in the city of Karlsruhe, Germany, to extract
road networks and other objects such as parking spots and sidewalks that could
be integrated into the OSM database. Facebook has also implemented deep
learning methods to analyze satellite imagery but for the detection of new
road networks in developing countries in the OSM map
333https://wiki.openstreetmap.org/wiki/AI-Assisted_Road_Tracing

Figure 3: Three different street types (secondary, tertiary and residential
colored in red, blue, and cyan, respectively) over the OSM map in the city of
Wageningen, the Netherlands.

Assigning attributes: road types

Topological and geometrical features extracted from road networks can be very
useful to predict street types. Figure 3 depicts three types of streets (i.e.,
secondary, tertiary, and residential streets) over the OSM map. It can be
observed that residential street segments are small in length (distinctive
geometrical feature) and that tertiary roads are connected with several
residential streets (distinctive topological feature). A solution to fix
incorrect street type tags in OSM is then presented in [39]. The authors
extract topological features from OSM road network data to train a neural
network classifier that predicts if a street is of type residential or
pedestrian. This classifier can be useful to find inconsistent street type
tags in OSM.

A multi-granular graph representation of street networks is proposed in [40].
This structure combines the primal (where nodes are road intersections) and
dual representations (where nodes are fragments of roads) of road networks.
This multi-granular representation is used in [6] to extract features and
train a Random Forest classifier that is able to classify streets to 21
different street categories in OSM. The method uses Bag of Words computed over
geometrical and topological features of the analyzed streets and their
neighbors. The method in [41] uses graphical models with geometrical and
spatial features, such that the parameters of the model are learned by
Structured Support Vector Machines (SSVM) [42]. More recently, the authors in
[43] propose a multi-layer CRF (Conditional Random Field) model to perform
hierarchical classification of street types into coarse and fine-grained
classes. In [44] the authors propose a graph convolutional network based
method for driving speed estimation of road segments. The output of this
method could be used as an additional feature for models that classify road
types.

Assigning attributes: structural

Detecting multilane roads is important to model traffic in urban areas.
However, the tag ’lanes’ that is used to specify how many traffic lanes a road
has is usually empty in OSM. Therefore, some works [45, 46] have developed
methods to detect multilane roads in OSM data by analyzing the polygons formed
by the road network. This is possible because frequently multilane roads are
digitized as multiple parallel roads with terminals in road crossings. In [45]
the authors propose to extract geometrical features (e.g., area, perimeter,
and compactness) from polygons obtained from the road network and train SVM
classifier to predict if a road has multiple lanes. After that a
postprocessing step is performed, by using a region growing algorithm, to
analyze if roads connected to the predicted multilane roads are also multilane
roads. The method proposed in [46] also uses geometrical features to train a
classifier, a Random Forest in this case, but the predictions are used to
train a second Random Forest classifier that uses geometrical and topological
features, such as the percentage of neighboring roads that are classified as a
multilane road by the first classifier. In [47] the authors use an MRF and
data extracted from remote sensing images (e.g., edge information, cars
detected, and contextual information) to correct OSM road centerline locations
and estimate the width of OSM roads.

Extracting road data from GPS locations

Sequences of GPS positions (also called tracking data) of users can be used to
enrich OSM data. This information can be obtained by GPS locations of cars or
applications installed in the volunteers’ mobile devices. In [48], the authors
propose to find errors in OSM data by analyzing patterns extracted from GPS
positions and OSM mapping information. For instance, indoor corridors wrongly
labeled as tunnels in OSM can be detected by verifying if the trajectory data
comes from a pedestrian or a car.

In [49], the authors propose to use GPS positions and machine learning models
for recomending the addition of new objects to OSM. For instance, GPS
positions can help us predict a missing street in OSM by observing a linear
shaped agglomeration of points at some location, where there is not a street
in OSM but it is close to the road network. Analyzing the spatiotemporal GPS
positions one can also identify, for example, that a road in OSM is a motorway
because of the high velocity of objects derived from the GPS information data.
The authors in [49] extract several features from GPS information, such as the
density of nodes in the trajectory and speed of movement. Two types of
classifiers are trained: one to predict the geometry and the other to predict
object attributes, such as motorway, bicycle lanes, one or two-way street. For
geometry classification, the K-Nearest Neighbours (KNN) classifier performed
better than other algorithms like Logistic Regression and Random Forest. It is
observed that the KNN model obtains poor results when detecting polygonal
geometries, because of the lack of data along the boundaries of polygonal
objects. For the classification of geographical object attributes, the Random
Forest classifier outperforms the compared traditional machine learning
methods.

In [50], the authors utilize GPS information for the reconstruction of road
network geometries. The extracted road networks from updated GPS locations can
be useful to improve the OSM map. One issue is obtain accurate road geometries
since the GPS locations present errors in the range of 5-20 meters. This
shortcoming can be mitigated by using multiple trajectories obtained from the
same road segment. The authors observe that the accuracy of geometries
increase with the number of GPS samples for each road segment.

###  II-C Semantic tags

The annotation of a geographical object in OSM consists in the digitization of
the object geometry (e.g., polygons, lines, or points) and also the
attribution of a tags to it. OSM does not provide a rigorous classification
system of the geographical objects. It just gives some recommendations and a
set of predefined tags that can be used to annotate objects. Thus, the final
label attributed to the OSM objects is defined by the volunteers based on
their knowledge about the objects under annotation. This can lead to incorrect
tag annotations since sometimes it is difficult for inexperienced users to
differentiate between similar classes. The decision if a water body is a lake
or a pond will depend on the knowledge of the volunteer and on his/her
analysis of the aerial imagery or in-situ information.

Recommendation systems

OSM also allows the assignment of tag values that are not in the set of
recommended OSM tag values, which detriments standardization of OSM data. The
authors in [51] propose a method to identify recommended OSM tags that are
equivalent to new tags created by annotators. The paper proposes an
unsupervised method that uses tag usage statistics and geometry type
information to compute a similarity measure between a given tag value and a
set of common tag values recommended by OSM. This approach just uses OSM data,
in contrast with the method proposed in [52], which uses external data, such
as information from the OSM wiki website. Other works [53, 54] have
implemented tag recommendation tools as plugins of JOSM
444https://josm.openstreetmap.de/, a widely used editor of OSM data. In [53],
the authors proposed the tool called OSMRec that uses geometrical and textual
features to train a Support Vector Machine classifier that is used to
recommend a set of tags for new objects that are being digitized by
annotators. The tool OSMantic is proposed in [54] and uses semantic similarity
and tag frequency to recommend tags.

Tags verification

As shown in the previous section, several methods propose machine learning
models based on properties of OSM objects. Then, the trained classifiers can
be applied to another set of OSM objects to find possible annotation errors.
In [55], the authors present three strategies of how such learned classifiers
can be applied (see an illustration of them in Fig. 4).

  1. 1.

Consistency checking, where the classifier is applied while the user is
editing and assigning tags to OSM objects. In this case, the editing tool can,
for example, inform the volunteer that the assigned tag value is inconsistent
with what the classifier predicted. Then, the annotator can modify the
annotation if required, by taking into consideration the classifier’s
recommendation

  2. 2.

Manual checking, in which the classifier is applied over a selected set of
objects already registered in OSM. Then, the objects whose tags present
inconsistencies with the predictions of the classifier are manually validated
by the users

  3. 3.

Automatic checking, in which a classifier is used to automatically correct
annotations based on its predictions without human verification.

---
a)
b)
c)
Figure 4: Three strategies used with a classifier to improve OSM data,
according to [55]. a) contribution checking, b) manual checking and c)
automatic checking

The method proposed in [55] aims to find errors in tags used for annotating
green area objects (i.e., meadow, garden, grass and park). The authors
observed that these four types of green area objects are some times mislabeled
by OSM annotators. Figures 5a-b illustrate a case where a grassland area in
the center of a roundabout was wrongly labeled as a park. Figures 5c-d depict
a case where a grassland area with some trees is wrongly labeled as a forest.
The technique proposed in [55] extracts geometrical, topological, and
contextual properties (e.g., object area and features based on the
9-Intersection model [56]) and trains a KNN classifier to analyze the labels
of the four types of green area objects in OSM. The authors in [55] perform an
experiment that consists in asking users to manually verify/correct objects
with possible erroneous labels. These objects are detected by a classifier,
and the experiment shows the effectiveness of the approach to detect
mislabeled green areas. Another approach proposed in [57] tries solving this
problem (disambiguation of green areas with the same four classes) by
extracting rules from the OSM dataset using the algorithm proposed in [58].
These rules are extracted based on topological relations between geographical
objects.

|
---|---
a) | b)
|
c) | d)
Figure 5: Misannotated green area objects in the OSM map alongside with the
Bing imagery of the corresponding location: a-b) A grassland area at the
center of a roundabout mislabeled as a park in the city of Campinas, Brazil,
c-d) A grassland area with some trees (located at the bottom right of the
image) is mislabeled as a forest in the city of São Gonçalo, Brazil.

###  II-D Points of Interest

Points of Interest (POIs) are key elements in OSM. They indicate the location
of geographical objects that are commonly used in the city, such as bus
stations, cafes, restaurants, ATMs, etc. Thus, the quality control of new
added POIs to the OSM database is very important. Some OSM editors, like JOSM
implement basic rules to avoid errors (e.g., duplicate elements) while editing
objects in OSM. However, this type of topology quality control verifications
does not take into account the spatial relationship between a new POI and
neighboring geographical objects in OSM.

Plausibility of new additions

In [59], the authors propose a recommendation tool that evaluates the
positional plausibility of a new registered POI with a certain category label.
That work proposes to use spatial co-existence patterns for computing a
plausibility score. Confidence scores are computed based on the frequency of
occurrence of each pair of POI categories (i.e., {ATM, Bar} or {Bank,
supermarket}). Then, the similarity score of two POIs is defined based on the
confidence values of their POI categories. Finally, the plausibility score of
a new POI is computed as the sum of the similarity score of the new POI and
its neighbors. In order to compute the confidence values of pairs of POI
categories, the authors recommend using POIs of the same city where the tool
has to be validated. This is because different cities may have different
patterns.

In [59], a case study is shown by evaluating the plausibility of a new ATM
being added to four locations in Paris (besides a river and bridge, Paris
downtown, middle of a park, and outside the city). The plausibility values
obtained are coherent with what is expected – e.g., the plausibility score of
a new ATM located in Paris downtown is much larger than the other
alternatives.

Tags prediction

In [60] the authors propose a method that can predict tags of Points of
Interest (POIs) based on their names. This method can be useful to extract tag
information for POIs that lack of tagging information. For instance, a POI
with name “Chicken Palace” probably should have the tag “amenity=restaurant”.
This work used the number of occurences of k-grams, substrings of a given
size, extracted from the POI names to create feature vectors. Then, a Random
Forest classifier was trained with OSM reference data to food, shop and
tourism related POIs in OSM obtaining accurate prediction for some food
related classes.

##  III Using OSM data with machine learning algorithms

This section presents methods based on machine learning to use OSM in other
applications, namely land use/ land cover classification, building detection
and segmentation, navigation, traffic estimation, and fine-scale population
estimation.

###  III-A Land use and land cover

Land cover/land use mapping has been attempted by governmental organizations
(e.g., Urban Atlas 555https://land.copernicus.eu/local/urban-atlas),
commercial services (e.g., Google maps) and crowdsourced projects (e.g.,
OpenStreetMap). Several governmental surveys are freely available. However,
the quality of the land use maps depends on the city and country and also this
data is of few use when is not updated frequently. Commercial services like
Google maps are more frequently updated but great part of the geographical
information in such services are not openly available. In contrast,
crowdsourced projects, like OSM provide access to all the collected
geographical information and they are regularly updated in several cities.

The data quality of some land use types in OSM is comparable to governmental
surveys. In [61], the authors compare the accuracy of OSM data for land use
mapping in seven large European metropolitan regions. The thematic accuracy
and degree of completeness of OSM data are compared to the available Global
Monitoring for Environment and Security Urban Atlas (GMESUA) datasets.
Evaluation of several land use types suggests that some OSM classes have good
quality, such as forest, water bodies, and agricultural areas, and could be
used for land use planning.

Several works have proposed methods to predict land cover/land use labels by
using remote sensing imagery and OSM land use labels as reference data to
train a classifier. The authors in [62] use time-series Landsat imagery and
OSM annotations (i.e., object boundary delineations and land use labels) to
train and evaluate several supervised methods for land cover classification,
considering six classes (e.g., impervious, farm, forest, grass, orchard, and
water). The authors in [63] use aerial imagery and a large amount of building
and road annotations from OSM as training data for supervised classification.
The collected annotations are selected without any quality verification and
thus the authors observe several cases of low-quality annotations. The authors
show that Convolutional Neural Network (CNN) methods trained with this type of
data can achieve high accuracies as compared with methods that use a
relatively small amount of good quality data. The authors in [18] propose a
CNN based method that combines aerial imagery and rasterized OSM data to
predict per-pixel land use labels. In [64], The authors propose a method for
land use mapping at large scale (approx. 34000 $km^{2}$ in the region of
NorthRhine-Westfalia, Germany). Each spatial unit is a 200m x 200m cell
characterized by multimodal features: remote sensing data (RapidEye bands and
texture features, see Fig. 6a-b), 3D models (see Fig. 6c) and OSM data
(locations of Point of interests and road networks (see Fig. 6d), both encoded
as street densities at the cell level (see Fig. 6e)).

Figure 6: Data sources and local climate zones prediction map in the region of
North Rhine Westfalia [64]. All the subfigures correspond to the region in the
blue square in the right most map.

Other works use OSM data and ground-based pictures of a set of OSM objects, to
teach a model how to predict the land use (e.g., museums, parks, educational
institutions, sports centers, and hotels) of other OSM objects [65, 17, 19].
The method proposed in [65] use pictures, obtained from Google Street View
(GSV), which capture multiple viewpoints of OSM objects (see the last three
columns in Fig. 7) and use a pre-trained CNN model to extract features and
perform label prediction of 13 land use types. The method proposed in [17]
presents an extension of [65] and considerably improves the prediction
accuracy, by training a CNN with multiple ground-based pictures, extracting
and combining features to identify the proper land use class (among 16 land
use categories) of an OSM object.

More recently, the authors in [19] propose a CNN-based method that combines
aerial imagery and ground-based pictures information to perform land use
prediction, using OSM as reference data. That method greatly improves the
accuracy obtained by the method in [17] which uses only ground-based pictures.
Additionally, that work also proposes a strategy to deal with the cases when
ground-based pictures are missing for an OSM object. Figure 7 illustrates
aerial and ground-based images corresponding to two OSM objects in the city of
Paris. The first row shows the images of a church. If we just observe the
aerial imagery, it is difficult to be confident in predicting the object as a
church. Ground-based pictures can give additional visual features to predict
the correct land use label. The second row shows images of a sport facility in
OSM. Because of the fences around the building, sometimes it is difficult to
recognize a sport facility. In this case, aerial imagery is usually more
valuable to predict the correct land use label.

Figure 7: Aerial imagery, from Google maps, and ground-based pictures, from
Google Street View, for two different OSM objects in the city of Paris. The
pictures in the first row correspond to a church. The pictures in the second
row correspond to sport facility.

###  III-B Building detection and segmentation

Building annotations are widely available in OSM and they are usually of
considerable quality in several urban areas. Thus, OSM building annotations
have been used as reference data to train CNN-based building segmentation
methods [66, 24, 67]. In [68] the authors use a Recurrent Neural Network (RNN)
based method that improves the building classification map in several
iterations. The building segmentation maps produced by these methods are not
directly usable for Geographical Information Systems (GIS) because they are
raster images. However, some methods [69, 70] have been recently proposed to
output vectorial building polygons suitable for GIS software. The method
proposes in [69] uses Active Contour Models with its parameters learned by a
CNN to output vectorial footprints. The authors of [71] and [70] propose to
convert binary building classification maps into vectorial outputs by using a
mesh-based approximation method.

In contrast to urban buildings, rural buildings are sparsely located in large
geographical areas and their annotations in OSM are less frequent and of lower
quality as compared to buildings in urban areas. Some techniques have proposed
to improve the detection of locations of rural buildings in aerial imagery
[72, 73]. In [72], the authors propose a CNN-based method to detect buildings
in aerial imagery using an iterative process, in which new samples are
selected for annotation by an active learning method and the model is
retrained. In [73] the authors use multiple sources of crowdsourced
geographical data (namely OSM, MapSwipe 666https://mapswipe.org/, and OsmAnd
777https://osmand.net/) and an active learning strategy to train a CNN model
that detects image patches with buildings. Furthermore, the authors perform an
experiment in MapSwipe (smartphone-based application for humanitarian mapping)
asking the volunteers to just verify the tiles that are selected by a trained
classifier, saving considerably the user effort and obtaining an accurate
classifier.

###  III-C Navigation

In several urban areas, building and road network data in OSM have similar
quality as commercial map services. Thus, OSM has been used for
navigation/routing applications [74, 75, 76]. In [74], the authors show that
routing services based on OSM data can attain real-time shortest path
computation in large areas for web-based applications as well as in hand-held
devices. In [75], the authors show that OSM data can be used for pedestrian
routing. The paper proposes a solution to efficient routing in open spaces
(e.g., squares, parks, and plazas). A more recent and extensive analysis of
different strategies to deal with routing that consider open spaces is
presented in [76].

In [77], the authors propose a method for accurate global vehicle
localization. That work uses visual odometry and OSM data obtaining better
localization results than methods that just rely on visual odometry
approaches. A probabilistic model for autonomous robot navigation is proposed
in [78]. It uses a Markov-Chain Monte-Carlo method to combine semantic terrain
information extracted from 3D-LiDAR data and OSM information.

###  III-D Traffic estimation

Traffic prediction is a challenging task that can be very useful for
congestion management and car navigation. The authors in [79] propose a method
to predict four classes of traffic (i.e., good, slow, congested, and extremely
congested) in four cities in China. That work uses data obtained from POIs of
Baidu maps and geographical objects from OSM (the number of POIs in OSM is
limited in China). This geographical data together with other features, like
weather, temperature, and house pricing are used to train a machine learning
model for traffic prediction. The authors use traffic data from Baidu maps as
reference data to train a Support Vector Machine (SVM) classifier. In that
work, the authors observe that traffic congestion data is very unbalanced,
because most of the time the traffic is good considering all the time
intervals in one day (with exception of the rush hours 9h and 17h). Thus, the
authors assign higher weights to the classes with less number of samples. The
results show that even using class weighting, the accuracy of the model is
high for the class good and poor for the other classes. Additionally, that
method is compared to the traffic prediction system of Baidu maps
outperforming it in some time intervals of the day and obtaining reasonably
good performances when the model trained in one city is used for prediction in
another city.

In [20], the authors deal with a similar problem, predicting traffic speed
using a regression method. A public dataset named UIUC New York City Traffic
Estimates 888https://lab-work.github.io/data/ is used for their experiments.
This dataset contains hourly average traffic speed measurements in the New
York City road network, obtained from car trips. Several geographical features
are obtained from OSM, such as road length, number of roads connected to the
analyzed road, number of neighbor nodes and roads in the area, and also
temporal features (e.g., time and whether it is a workday). In order to
predict traffic speed in the target areas, the authors of that work propose
the method Cluster-based Transfer Model for Prediction (CTMP), which first
clusters the road features of the source and target areas. Then, the traffic
speed of the target area is computed based on the nearest neighbor roads data
of the source area, which contains traffic speed information. CTMP shows
better results than other baseline methods, such as Neural Networks and
Support Vector Regression. More recently in [80], the authors propose a deep
learning method that models the road network topology to predict traffic flow
in the city of Chengdu, China. The authors use neural networks to model road
network topology and residual learning [81] to model spatio-temporal
dependencies. One limitation of that work is that it requires traffic flow
historic reference data of the target location to be able to predict the
traffic flow in a different time interval.

###  III-E Fine-scale population estimation

Population distribution at the building level is important for several tasks,
like urban planning and business development. Population estimation at the
building level scale can be obtained by areal interpolation. Although this
technique usually requires 3D building models, obtained by LiDAR data, the
authors in [82] propose to use building footprints and POIs from OSM to
predict population distribution by using areal interpolation. The authors in
[83] use a Random Forest model to predict population at the grid level (i.e.,
the area of study is divided into grid cells) by using Baidu POIs, mobile user
density data, and road networks from OSM. Then, the grid level estimations are
transformed to building level estimations.

In [84], the authors propose a CNN-based method to perform population density
estimation at the grid level. First, the area of study is divided into grid
cells of size 200 $\times$ 200 meters. Then, for each cell, they compute
several urban features, such as building area, number of buildings, and number
of POIs. Finally, the authors use a fully convolutional neural network,
applied over the urban features of the grid cells, to obtain the population
estimation of the corresponding grid cells. The experimental results show that
by training the model with data from 14 French cities, the model attains low
error rates in the validation data extracted from the city of Lyon.

##  IV Discussion and future works

In this section, we discuss potential promising research avenues at the
interface of OSM and machine learning.

Applications with multimodal data

Although the use of several sources of data (multimodal approaches) have
proven to be beneficial for solving several problems it has been only applied
in relatively few works described in this manuscript, such as in [19]. We
believe that the performance of supervised methods used to improve OSM data
can be greatly improved with the use of several data sources, such as images,
tracking data, and social media data. This has been also pointed out in [85]
for the particular case of POIs, where the authors recommend the use of
several data sources, like OSM geographical data, ground-based pictures, and
historical data to create more accurate models for POI label prediction. Among
these external data sources, ground-based pictures obtained from Google Street
View (GSV) have found to be particularly useful to enrich OSM data, for
example, for land use prediction [17] and crosswalk localization [86].
Recently, crowdsourced ground-based images collected by the TeleNav’s project,
called OpenStreetCam999https://openstreetcam.org, has also been used to
improve OSM. For instance, pictures obtained from OpenStreetCam have been used
to detect traffic signs101010https://blog.improveosm.org/en/2018/02/detecting-
traffic-signs-in-openstreetcam/. Similarly, Mapillary has released a dataset
of ground-based pictures111111https://www.mapillary.com/dataset/trafficsign
for traffic sign detection [87]. Although nowadays the coverage of
crowdsourced ground-based pictures platforms is not as complete as GSV, they
have the potential to obtain more updated data and to be available for
everyone at no cost. This data can then be used to obtain up-to-date OSM data
of objects like traffic lights and road signs, which can greatly benefit
navigation applications.

Very recent works [88, 89] have explored the use of OSM data combined with
social media textual information, obtained from Twitter, to perform land use
classification. These works use natural language processing techniques (e.g,
fastText [90]) together with neural networks to process geolocalized tweets.
The authors in [91] use Long short-term memory (LSTM) for feature extraction
of Twitter data for the classification of OSM urban-objects into three
categories: residential, non-residential and mixed-use. In contrast to works
that use image data for finegrained land use classification [19], these works
present results with datasets that contain only a few land use classes, but
have the advantage of the massive amounts of data that can be retrieved from
social media. Also, working with Twitter, it creates the need of specialized
workflows, typically aiming at denoising data, retrieve useful Tweets, or
avoiding spourious geolocation due to re-Tweets. We found these issues very
exciting and believe that textual data should be used in a multimodal fashion:
as suggested in [88], textual information and remote sensing data could be
used together to perform more precise land use classification and this is, in
our opinion, one of the avenues that future research could focus on in the
next years.

Multimodal techniques

In [19], the authors use a CNN that extracts features from ground-based
pictures and remote sensing images to perform land use classification of OSM
urban objects. The features extracted from these two different image
modalities are stacked and a fully connected layer is used to predict the land
use category. The authors in [19] also propose a solution to the case when the
data of one of the modalities is missing. This is done by using Cannonical
Correlation Analysis (CCA) to create a latent space and projection matrices
that allow, starting from a satellite patch, to retrieve a plausible feature
vector for the ground-based missing images. Another example of feature
stacking, but using governmental data, is proposed in [92]. In that work, the
authors propose a method for per-pixel land use classification. The proposed
method stacks features extracted from remote sensing data and ground-based
pictures, but keep the spatial information. Finally, a Multilayer Perceptron
(MLP) is used to perform per-pixel classification. In [93], the authors
perform local climate zones classifications by fusing multi-temporal and
multispectral satellite images that have different spatial resolutions. The
authors propose a method for weighted voting of the output of the classifiers
trained with the different data modalities and show more accurate results than
feature stacking. The methods that we have commented in this review perform
the fusion of data of two modalities. We believe that a challenging topic of
future research is the creation of methods that handle the fusion of more than
two modalities, dealing well with the problem of missing modality.

Supporting users via interaction and skills estimation

Mapping information obtained by machine learning models applied with Earth
Observation data (e.g., building footprints, and road networks) could contain
some errors. Thus, an alternative to performing automatic updates in OSM is to
use a human-computer interactive approach. In this strategy, the machine
learning model is used to minimize the effort of the users during the
annotation process. This strategy has been already applied in [37] to improve
road network completeness. In that work, an automatic method is used to
extract major roads (in places with a few road annotations) and missing roads
(in places where major roads are already annotated) and the user is asked to
verify or correct if needed the extracted roads. This work shows
experimentally that such an interactive approach is more efficient than
traditional manual annotation. A related and also effective approach that
involves the user in the process is active learning [94, 95]. Active learning
is an iterative process that consists of intelligently selecting a small
number of samples for user annotation that allow training an effective
classifier. We think that these approaches that involve the interaction
between a machine learning model and the annotator could be applied to improve
other aspects of OSM data with fewer annotators’ effort, especially if user’s
skills are involved in the process, as shown in concurrent research in
crowdsourcing [96, 97]. Although active learning methods have been applied to
OSM data of rural areas [72, 73], they can be also applied to OSM data of
urban areas (e.g., cities in developing countries).

Between January and May of 2019, the number of active contributors per month
in OSM was less than 1% of the total registered OSM users. However, a few
efforts have been made to encourage volunteers to frequently edit data.
Gamification strategies could be applied to solve the problem, by assigning
annotation tasks to volunteers with a game-like interface and scoring systems.
Some gamification projects have been listed in the OSM wiki
121212https://wiki.openstreetmap.org/wiki/Gamification but they are not of
widespread use.

Contributor analysis in OSM have been studied in several works [98]. It has
been observed that volunteers’ experience and familiarity with the area edited
in the map are good proxies to estimate the quality of their annotations in
OSM. The authors in [99] propose a measure for estimating annotation trust, by
using annotation statistics obtained from volunteers’ activities, object
geometries, and temporal data. A trust index could be also learned with
machine learning methods using OSM data statistics with some reference data.
Such trust index could be used to improve methods that are created to verify
the correctness of labels of objects registered in OSM [55, 57].

New ways of searching and interacting with OSM

Search tools are important features of mapping services. However, the search
tool provided by the OSM website has limited capabilities, basically just
trying to find the location of a given place name or address. In [100], the
authors propose NLmaps 131313https://www.cl.uni-
heidelberg.de/statnlpgroup/nlmaps/, a natural language interface to query data
in OSM. This service can answer textual questions about geographical facts in
OSM. The response is a text and a map with geographical objects of interest
highlighted. For instance, NLmaps can answer the following question: “What is
the closest supermarket from the Royal Bank of Scotland in Edinburgh?”. Figure
8 illustrates the search results in NLmaps for the question: “Which museums
are there in Heidelberg?”. First, the names of the museums found are shown and
then their locations are shown with markers in the OSM map of the city of
Heidelberg.

Figure 8: NLmaps search results for the query: “Which museums are there in
Heildelberg?”.

This work uses a semantic parser for the OSM data, proposed in [101] and
extended in [102], to transform natural language text to a Machine Readable
Language formula (MRL) [103]. This representation is used to create a
structured query and retrieve OSM information using an extension of the
Overpass API, called Overpass MLmaps
141414https://github.com/carhaas/overpass-nlmaps. Although NLmaps can handle
several common questions, it is not able to answer complex questions like
“Where are 4 star hotels in Berlin?”. The authors of [102] also propose to
improve NLmaps performance by using user feedback. A simple example of
feedback is to ask the user if the result of the query is helpful or not. More
complex feedback can be obtained from expert users, with knowledge of OSM and
the Overpass API. For instance, one can ask the expert user if the
intermediate results of the query processing pipeline are correct or not. We
believe that the implementation of a more intelligent tool for answering
natural language queries in the OSM website could potentially attract more
users.

Recently, the authors of [104] use OSM data for the Visual Question Answering
(VQA) problem, a task that consists in providing a natural language answer for
a given image and a natural language question about the image. The authors
create a dataset using OSM data and Sentinel-2 satellite imagery for training
a VQA model. VQA is a challenging problem and its application for remote
sensing imagery is still in its infancy. VQA can enable to perform some
queries in places where there is incomplete mapping data, but aerial imagery
is available.

OSM-enabled augmented reality

Recent applications of OSM data can be further improved by using machine
learning methods. For instance, augmented reality (AR) has great potential to
improve the way we experience the cities, especially for tourism. In [105], an
augmented reality tool is presented to recommend Points Of Interest (POIs) in
the city by taking into account the profile of the users. The authors propose
a mobile application that shows POIs registered in OSM in real time with the
mobile camera view as background. A case study in the city of Trani, Italy,
shows how a tourist interested in local architectural work can visualize in
the mobile application the POIs, within a distance radius, marked as colored
circles with labels superimposed over the mobile camera view. In [105], a hand
engineered rule is proposed to define which POIs to show to the user. For this
type of AR applications, reference data could be used by asking users’
feedback (e.g., asking to the user to add POIs visited in a favorites list).
Thus, obtaining reference data can enable the use of more effective supervised
learning methods to predict adequate POIs for new users.

##  V Conclusion

In this paper, we reviewed synergetic efforts involving OpenStreetMap and
machine learning. In the first part, we review works that use machine learning
to improve OSM data. These methods deal with the three main object geometric
types in OSM: points (e.g., ATMs), lines (e.g., roads) and polygons (e.g.,
buildings). The reviewed works use frequently traditional machine learning
methods (e.g., Support Vector Machines and Random Forest), but also several of
them have used state-of-the-art methods, such as deep learning based
techniques, especially when dealing with remote sensing image data. Although
several methods could be integrated into OSM editors (e.g., iD editor
151515http://ideditor.com/ and JOSM) just a few works [53, 54] have
implemented their methods in such tools.

The second part of the manuscript reviewed works that have used machine
learning based techniques to use OSM data for applications in other domains.
We identified two groups of works. The first group uses OSM data as reference
data to train machine learning models, for examples several works that perform
land use classification [17, 19] and building segmentation [66, 24]. The
second group uses OSM data to extract features for training the machine
learning model (e.g., fine-scale population estimation [84]).

We believe that a mixture of automatic and human-interactive approaches could
lead to obtaining accurate data for OSM with efficient use of the annotators’
labor. The strong links with machine learning and the ever increasing
availability of up-to-date remote sensing data open countless opportunities
for research in this exciting interface among disciplines.

## Acknowledgment

This research was funded by Fundação de Amparo à Pesquisa do Estado de São
Paulo (FAPESP, grant 2016/14760-5 and 2014/12236-1), Conselho Nacional de
Desenvolvimento Científico e Tecnológico (CNPq, grant 303808/2018-7) and
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES,
finance code 001).

## References

  * [1] J. E. Vargas-Muñoz, “Machine learning applied to open geographical data,” Ph.D. dissertation, University of Campinas, 10 2019.
  * [2] M. F. Goodchild, “Citizens as sensors: the world of volunteered geography,” _GeoJournal_ , vol. 69, no. 4, pp. 211–221, 2007.
  * [3] P. Mooney and P. Corcoran, “The annotation process in OpenStreetMap,” _Transactions in GIS_ , vol. 16, no. 4, pp. 561–579, 2012.
  * [4] M. Haklay, “How good is volunteered geographical information? A comparative study of OpenStreetMap and Ordnance Survey datasets,” _Environment and planning B: Planning and design_ , vol. 37, no. 4, pp. 682–703, 2010.
  * [5] S. Funke, R. Schirrmeister, and S. Storandt, “Automatic extrapolation of missing road network data in OpenStreetMap,” in _Proceedings of the 2nd International Conference on Mining Urban Data-Volume 1392_ , 2015, pp. 27–35.
  * [6] M. Jilani, P. Corcoran, and M. Bertolotto, “Automated highway tag assessment of OpenStreetMap road networks,” in _Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems_ , 2014, pp. 449–452.
  * [7] Y. Xu, Z. Chen, Z. Xie, and L. Wu, “Quality assessment of building footprint data using a deep autoencoder network,” _International Journal of Geographical Information Science_ , vol. 31, no. 10, pp. 1929–1951, 2017.
  * [8] T. Koukoletsos, M. Haklay, and C. Ellul, “Assessing Data Completeness of VGI through an Automated Matching Procedure for Linear Data,” _Transactions in GIS_ , vol. 16, 08 2012.
  * [9] H. Fan, A. Zipf, Q. Fu, and P. Neis, “Quality assessment for building footprints data on OpenStreetMap,” _International Journal of Geographical Information Science_ , vol. 28, pp. 700–719, 04 2014.
  * [10] J. F. Girres and G. Touya, “Quality assessment of the French OpenStreetMap dataset,” _Transactions in GIS_ , vol. 14, no. 4, pp. 435–459, 2010.
  * [11] P. Neis, D. Zielstra, and A. Zipf, “The street network evolution of crowdsourced maps: OpenStreetMap in Germany 2007–2011,” _Future Internet_ , vol. 4, no. 1, pp. 1–21, 2011.
  * [12] P. Neis and D. Zielstra, “Recent developments and future trends in volunteered geographic information research: The case of OpenStreetMap,” _Future Internet_ , vol. 6, no. 1, pp. 76–106, 2014.
  * [13] J. J. Arsanjani, P. Mooney, M. Helbich, and A. Zipf, “An exploration of future patterns of the contributions to OpenStreetMap and development of a Contribution Index,” _Transactions in GIS_ , vol. 19, no. 6, pp. 896–914, 2015.
  * [14] H. Senaratne, A. Mobasheri, A. L. Ali, C. Capineri, and M. Haklay, “A review of volunteered geographic information quality assessment methods,” _International Journal of Geographical Information Science_ , vol. 31, no. 1, pp. 139–167, 2017.
  * [15] M. Jilani, M. Bertolotto, P. Corcoran, and A. Alghanim, “Traditional vs. Machine-Learning Techniques for OSM Quality Assessment,” in _Geospatial Intelligence: Concepts, Methodologies, Tools, and Applications_. IGI Global, 2019, pp. 469–487.
  * [16] C. C. Fonte, L. Bastin, L. See, G. Foody, and F. Lupia, “Usability of VGI for validation of land cover maps,” _International Journal of Geographical Information Science_ , vol. 29, no. 7, pp. 1269–1291, 2015.
  * [17] S. Srivastava, J. E. Vargas-Muñoz, S. Lobry, and D. Tuia, “Fine-grained landuse characterization using ground-based pictures: a deep learning solution based on globally available data,” _International Journal of Geographical Information Science_ , vol. 0, no. 0, pp. 1–20, 2018.
  * [18] N. Audebert, B. L. Saux, and S. Lefèvre, “Joint Learning from Earth Observation and OpenStreetMap Data to Get Faster Better Semantic Maps,” in _IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2017, Honolulu, HI, USA, July 21-26, 2017_ , 2017, pp. 1552–1560.
  * [19] S. Srivastava, J. E. Vargas-Muñoz, and D. Tuia, “Understanding urban landuse from the above and ground perspectives: A deep learning, multimodal solution,” _Remote Sensing of Environment_ , vol. 228, pp. 129–143, 2019\.
  * [20] B. Y. Lin, F. F. Xu, E. Q. Liao, and K. Q. Zhu, “Transfer Learning for Traffic Speed Prediction: A Preliminary Study,” in _Workshops at the Thirty-Second AAAI Conference on Artificial Intelligence_ , 2018.
  * [21] S. Schmitz, A. Zipf, and P. Neis, “New applications based on collaborative geodata—the case of routing,” in _Proceedings of XXVIII INCA international congress on collaborative mapping and space technology_ , 2008.
  * [22] P. Neis, “Measuring the Reliability of Wheelchair User Route Planning based on Volunteered Geographic Information,” _Transactions in GIS_ , vol. 19, no. 2, pp. 188–201, 2015.
  * [23] J. E. Vargas-Muñoz, S. Lobry, A. X. Falcão, and D. Tuia, “Correcting rural building annotations in OpenStreetMap using convolutional neural networks,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 147, pp. 283 – 293, 2019.
  * [24] V. Mnih and G. E. Hinton, “Learning to label aerial images from noisy data,” in _International Conference on Machine Learning_ , 2012, pp. 567–574.
  * [25] M. Over, A. Schilling, S. Neubauer, and A. Zipf, “Generating web-based 3D City Models from OpenStreetMap: The current situation in Germany,” _Computers, Environment and Urban Systems_ , vol. 34, no. 6, pp. 496–507, nov 2010.
  * [26] M. Goetz and A. Zipf, “Using Crowdsourced Geodata for Agent-Based Indoor Evacuation Simulations,” _ISPRS International Journal of Geo-Information_ , vol. 1, no. 2, pp. 186–208, 2012.
  * [27] A. Schilling, M. Over, S. Neubauer, P. Neis, G. Walenciak, and A. Zipf, “Interoperable Location Based Services for 3D cities on the Web using user generated content from OpenStreetMap,” _Urban and regional data management: UDMS annual_ , pp. 75–84, 2009.
  * [28] X. Zhu, D. Tuia, L. Mou, G. Xia, L. Zhang, F. Xu, and F. Fraundorfer, “Deep learning in remote sensing: A comprehensive review and list of resources,” _IEEE Geosci. Remote Sens. Mag._ , vol. 5, no. 4, pp. 8–36, 2017.
  * [29] P. Neis, M. Goetz, and A. Zipf, “Towards automatic vandalism detection in OpenStreetMap,” _ISPRS International Journal of Geo-Information_ , vol. 1, no. 3, pp. 315–332, 2012.
  * [30] Q. Truong, G. Touya, and C. De Runz, “Towards Vandalism Detection in OpenStreetMap Through a Data Driven Approach,” in _GIScience 2018_ , 2018\.
  * [31] X. Zhuo, F. Fraundorfer, F. Kurz, and P. Reinartz, “Optimization of OpenStreetMap Building Footprints Based on Semantic Information of Oblique UAV Images,” _Remote Sensing_ , vol. 10, no. 4, p. 624, 2018.
  * [32] C. Barrington-Leigh and A. Millard-Ball, “The world’s user-generated road map is more than 80% complete,” _PloS one_ , vol. 12, no. 8, p. e0180698, 2017.
  * [33] G. Cheng, Y. Wang, S. Xu, H. Wang, S. Xiang, and C. Pan, “Automatic road detection and centerline extraction via cascaded end-to-end convolutional neural network,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 55, no. 6, pp. 3322–3337, 2017.
  * [34] G. Máttyus, W. Luo, and R. Urtasun, “Deeproadmapper: Extracting road topology from aerial images,” in _IEEE International Conference on Computer Vision_ , 2017, pp. 3438–3446.
  * [35] F. Bastani, S. He, S. Abbar, M. Alizadeh, H. Balakrishnan, S. Chawla, S. Madden, and D. DeWitt, “Roadtracer: Automatic extraction of road networks from aerial images,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 4720–4728.
  * [36] C. Ventura, J. Pont-Tuset, S. Caelles, K. K. Maninis, and L. V. Gool, “Iterative deep learning for road topology extraction,” in _British Machine Vision Conference_ , 2018.
  * [37] F. Bastani, S. He, S. Abbar, M. Alizadeh, H. Balakrishnan, S. Chawla, and S. Madden, “Machine-assisted map editing,” in _Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems_ , ser. SIGSPATIAL ’18. ACM, 2018, pp. 23–32.
  * [38] G. Máttyus, S. Wang, S. Fidler, and R. Urtasun, “Hd maps: Fine-grained road segmentation by parsing ground and aerial images,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 3611–3619.
  * [39] M. Jilani, P. Corcoran, and M. Bertolotto, “Automated quality improvement of road network in OpenStreetMap,” in _Agile Workshop (Action and Interaction in Volunteered Geographic Information)_ , 2013, p. 19.
  * [40] ——, “Multi-granular street network representation towards quality assessment of OpenStreetMap data,” in _Proceedings of the Sixth ACM SIGSPATIAL International Workshop on Computational Transportation Science_. ACM, 2013, p. 19.
  * [41] ——, “Probabilistic graphical modelling for semantic labelling of crowdsourced map data,” in _Intelligent Systems Technologies and Applications_. Springer, 2016, pp. 213–224.
  * [42] I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun, “Large margin methods for structured and interdependent output variables,” _Journal of Machine Learning Research_ , vol. 6, no. Sep, pp. 1453–1484, 2005.
  * [43] M. Jilani, P. Corcoran, and M. Bertolotto, “A Multi-layer CRF Based Methodology for Improving Crowdsourced Street Semantics,” in _Proceedings of the 11th ACM SIGSPATIAL International Workshop on Computational Transportation Science_. ACM, 2018, pp. 29–38.
  * [44] T. S. Jepsen, C. S. Jensen, and T. D. Nielsen, “Graph convolutional networks for road networks,” in _Proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems_ , 2019, pp. 460–463.
  * [45] Q. Li, H. Fan, X. Luan, B. Yang, and L. Liu, “Polygon-based approach for extracting multilane roads from OpenStreetMap urban road networks,” _International Journal of Geographical Information Science_ , vol. 28, no. 11, pp. 2200–2219, 2014.
  * [46] Y. Xu, Z. Xie, L. Wu, and Z. Chen, “Multilane roads extracted from the OpenStreetMap urban road network using random forests,” _Transactions in GIS_ , vol. 23, no. 2, pp. 224–240, 2019.
  * [47] G. Máttyus, S. Wang, S. Fidler, and R. Urtasun, “Enhancing road maps by parsing aerial images around the world,” in _IEEE International Conference on Computer Vision_ , 2015, pp. 1689–1697.
  * [48] A. Basiri, M. Jackson, P. Amirian, A. Pourabdollah, M. Sester, A. Winstanley, T. Moore, and L. Zhang, “Quality assessment of OpenStreetMap data using trajectory mining,” _Geo-spatial Information Science_ , vol. 19, no. 1, pp. 56–68, 2016.
  * [49] A. Basiri, P. Amirian, and P. Mooney, “Using crowdsourced trajectories for automated osm data entry approach,” _Sensors_ , vol. 16, no. 9, p. 1510, 2016\.
  * [50] C. Kuntzsch, M. Sester, and C. Brenner, “Generative models for road network reconstruction,” _International Journal of Geographical Information Science_ , vol. 30, no. 5, pp. 1012–1039, 2016.
  * [51] I. Majic, S. Winter, and M. Tomko, “Finding Equivalent Keys in Openstreetmap: Semantic Similarity Computation Based on Extensional Definitions,” in _Proceedings of the 1st Workshop on Artificial Intelligence and Deep Learning for Geographic Knowledge Discovery_ , ser. GeoAI ’17. New York, NY, USA: ACM, 2017, pp. 24–32.
  * [52] A. Ballatore, M. Bertolotto, and D. C. Wilson, “Geographic knowledge extraction and semantic similarity in OpenStreetMap,” _Knowledge and Information Systems_ , vol. 37, no. 1, pp. 61–81, 2013.
  * [53] N. Karagiannakis, G. Giannopoulos, D. Skoutas, and S. Athanasiou, “OSMRec tool for automatic recommendation of categories on spatial entities in OpenStreetMap,” in _Proceedings of the 9th ACM Conference on Recommender Systems_. ACM, 2015, pp. 337–338.
  * [54] A. Vandecasteele and R. Devillers, “Improving volunteered geographic information quality using a tag recommender system: the case of OpenStreetMap,” in _OpenStreetMap in GIScience_. Springer, 2015, pp. 59–80.
  * [55] A. L. Ali, F. Schmid, R. Al-Salman, and T. Kauppinen, “Ambiguity and plausibility: managing classification quality in volunteered geographic information,” in _Proceedings of the 22nd ACM SIGSPATIAL international conference on advances in geographic information systems_ , 2014, pp. 143–152.
  * [56] M. J. Egenhofer and R. D. Franzosa, “On the equivalence of topological relations,” _International Journal of Geographical Information Systems_ , vol. 9, no. 2, pp. 133–152, 1995.
  * [57] A. L. Ali, Z. Falomir, F. Schmid, and C. Freksa, “Rule-guided human classification of Volunteered Geographic Information,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 127, pp. 3–15, 2017.
  * [58] R. Agrawal and R. Srikant, “Fast Algorithms for Mining Association Rules in Large Databases,” in _Proceedings of the 20th International Conference on Very Large Data Bases_ , ser. VLDB ’94. Morgan Kaufmann Publishers Inc., 1994, pp. 487–499.
  * [59] A. Kashian, A. Rajabifard, K. F. Richter, and Y. Chen, “Automatic analysis of positional plausibility for points of interest in OpenStreetMap using coexistence patterns,” _International Journal of Geographical Information Science_ , vol. 33, no. 7, pp. 1420–1443, 2019.
  * [60] S. Funke and S. Storandt, “Automatic Tag Enrichment for Points-of-Interest in Open Street Map,” in _Web and Wireless Geographical Information Systems_. Cham: Springer International Publishing, 2017, pp. 3–18.
  * [61] J. J. Arsanjani and E. Vaz, “An assessment of a collaborative mapping approach for exploring land use patterns for several european metropolises,” _International Journal of Applied Earth Observation and Geoinformation_ , vol. 35, pp. 329–337, 2015.
  * [62] B. A. Johnson and K. Iizuka, “Integrating OpenStreetMap crowdsourced data and Landsat time-series imagery for rapid land use/land cover (LULC) mapping: Case study of the Laguna de Bay area of the Philippines,” _Applied Geography_ , vol. 67, pp. 140–149, 2016.
  * [63] P. Kaiser, J. D. Wegner, A. Lucchi, M. Jaggi, T. Hofmann, and K. Schindler, “Learning aerial image segmentation from online maps,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 55, no. 11, pp. 6054–6068, 2017.
  * [64] D. Tuia, G. Moser, M. Wurm, and H. Taubenböck, “Land use modeling in North Rhine-Westphalia with interaction and scaling laws,” in _Joint Urban Remote Sensing Event (JURSE)_ , March 2017, pp. 1–4.
  * [65] S. Srivastava, J. E. Vargas-Muñoz, S. Lobry, and D. Tuia, “Land-use characterisation using Google Street View pictures and OpenStreetMap,” 2018\.
  * [66] E. Maggiori, Y. Tarabalka, G. Charpiat, and P. Alliez, “Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification,” _IEEE Transaction on Geoscience and Remote Sensing_ , vol. 55, no. 2, pp. 645–657, Feb 2017.
  * [67] J. Yuan, P. K. R. Chowdhury, J. McKee, H. L. Yang, J. Weaver, and B. Bhaduri, “Exploiting deep learning and volunteered geographic information for mapping buildings in Kano, Nigeria,” _Scientific data_ , vol. 5, p. 180217, 2018.
  * [68] E. Maggiori, G. Charpiat, Y. Tarabalka, and P. Alliez, “Recurrent neural networks to correct satellite image classification maps,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 55, no. 9, pp. 4962–4971, 2017.
  * [69] D. Marcos, D. Tuia, B. Kellenberger, L. Zhang, M. Bai, R. Liao, and R. Urtasun, “Learning deep structure active contours end-to-end,” in _Computer Vision and Pattern Recognition (CVPR)_ , 2018.
  * [70] O. Tasar, E. Maggiori, P. Alliez, and Y. Tarabalka, “Polygonization of Binary Classification Maps Using Mesh Approximation with Right Angle Regularity,” in _IEEE International Geoscience and Remote Sensing Symposium (IGARSS)_ , 7 2018, pp. 6404–6407.
  * [71] E. Maggiori, Y. Tarabalka, G. Charpiat, and P. Alliez, “Polygonization of remote sensing classification maps by mesh approximation,” in _IEEE International Conference on Image Processing (ICIP)_ , 2017, pp. 560–564.
  * [72] J. Chen and A. Zipf, “DeepVGI: Deep learning with volunteered geographic information,” in _Proceedings of the 26th International Conference on World Wide Web Companion_. International World Wide Web Conferences Steering Committee, 2017, pp. 771–772.
  * [73] J. Chen, Y. Zhou, A. Zipf, and H. Fan, “Deep Learning From Multiple Crowds: A Case Study of Humanitarian Mapping,” _IEEE Transactions on Geoscience and Remote Sensing_ , pp. 1–10, 2018.
  * [74] D. Luxen and C. Vetter, “Real-time Routing with OpenStreetMap Data,” in _Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems_ , ser. GIS ’11, 2011, pp. 513–516.
  * [75] A. Graser, “Integrating Open Spaces into OpenStreetMap Routing Graphs for Realistic Crossing Behaviour in Pedestrian Navigation,” vol. 1, pp. 217–230, 06 2016.
  * [76] S. Hahmann, J. Miksch, B. Resch, J. Lauer, and A. Zipf, “Routing through open spaces – A performance comparison of algorithms,” _Geo-spatial Information Science_ , vol. 21, no. 3, pp. 247–256, 2018.
  * [77] G. Floros, B. van der Zander, and B. Leibe, “OpenStreetSLAM: Global vehicle localization using OpenStreetMaps,” in _2013 IEEE International Conference on Robotics and Automation_ , May 2013, pp. 1054–1059.
  * [78] B. Suger and W. Burgard, “Global outer-urban navigation with OpenStreetMap,” in _IEEE International Conference on Robotics and Automation (ICRA)_ , May 2017, pp. 1417–1422.
  * [79] F. F. Xu, B. Y. Lin, Q. Lu, Y. Huang, and K. Q. Zhu, “Cross-region traffic prediction for China on OpenStreetMap,” in _Proceedings of the 9th ACM SIGSPATIAL International Workshop on Computational Transportation Science_. ACM, 2016, pp. 37–42.
  * [80] Y. Ren, T. Cheng, and Y. Zhang, “Deep spatio-temporal residual neural networks for road-network-based data modeling,” _International Journal of Geographical Information Science_ , vol. 0, no. 0, pp. 1–19, 2019.
  * [81] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 770–778.
  * [82] M. Bakillah, S. Liang, A. Mobasheri, J. Jokar Arsanjani, and A. Zipf, “Fine-resolution population mapping using OpenStreetMap points-of-interest,” _International Journal of Geographical Information Science_ , vol. 28, no. 9, pp. 1940–1963, 2014.
  * [83] Y. Yao, X. Liu, X. Li, J. Zhang, Z. Liang, K. Mai, and Y. Zhang, “Mapping fine-scale population distributions at the building level by integrating multisource geospatial big data,” _International Journal of Geographical Information Science_ , vol. 31, no. 6, pp. 1220–1244, 2017.
  * [84] L. Gervasoni, S. Fenet, R. Perrier, and P. Sturm, “Convolutional neural networks for disaggregated population mapping using open data,” in _2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA)_. IEEE, 2018, pp. 594–603.
  * [85] G. Touya, V. Antoniou, A. M. Olteanu-Raimond, and M. D. Van Damme, “Assessing crowdsourced POI quality: Combining methods based on reference data, history, and spatial relations,” _ISPRS International Journal of Geo-Information_ , vol. 6, no. 3, p. 80, 2017.
  * [86] R. F. Berriel, F. S. Rossi, A. F. de Souza, and T. Oliveira-Santos, “Automatic large-scale data acquisition via crowdsourcing for crosswalk classification: A deep learning approach,” _Computers & Graphics_, vol. 68, pp. 32–42, 2017.
  * [87] C. Ertler, J. Mislej, T. Ollmann, L. Porzi, and Y. Kuang, “Traffic sign detection and classification around the world,” _arXiv preprint arXiv:1909.04422_ , 2019.
  * [88] M. Häberle, M. Werner, and X. X. Zhu, “Geo-spatial text-mining from twitter–a feature space analysis with a view toward building classification in urban regions,” _European journal of remote sensing_ , vol. 52, no. sup2, pp. 2–11, 2019.
  * [89] ——, “Building type classification from social media texts via geo-spatial textmining,” in _IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium_. IEEE, 2019, pp. 10 047–10 050.
  * [90] P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” _Transactions of the Association for Computational Linguistics_ , vol. 5, pp. 135–146, 2017.
  * [91] R. Huang, H. Taubenböck, L. Mou, and X. X. Zhu, “Classification of settlement types from Tweets using LDA and LSTM,” in _IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium_. IEEE, 2018, pp. 6408–6411.
  * [92] S. Workman, M. Zhai, D. J. Crandall, and N. Jacobs, “A unified model for near and remote sensing,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 2688–2697.
  * [93] G. Zhang, P. Ghamisi, and X. X. Zhu, “Fusion of heterogeneous earth observation data for the classification of local climate zones,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 57, no. 10, pp. 7623–7642, Oct 2019.
  * [94] B. Settles, “Active learning literature survey,” University of Wisconsin-Madison Department of Computer Sciences, Tech. Rep., 2009.
  * [95] M. M. Crawford, D. Tuia, and L. H. Hyang, “Active learning: Any value for classification of remotely sensed data?” _Proceedings of the IEEE_ , vol. 101, no. 3, pp. 593–608, 2013.
  * [96] R. Gomes, P. Welinder, A. Krause, and P. Perona, “Crowdclustering,” in _Proceedings of Advances in Neural Information Processing Systems (NIPS)_ , 2011.
  * [97] D. Tuia and J. Muñoz-Marí, “Learning user’s confidence for active learning,” _IEEE Transaction on Geoscience and Remote Sensing_ , vol. 51, no. 2, pp. 872–880, 2013.
  * [98] P. Neis and A. Zipf, “Analyzing the contributor activity of a volunteered geographic information project - The case of OpenStreetMap,” _ISPRS International Journal of Geo-Information_ , vol. 1, no. 2, pp. 146–165, 2012.
  * [99] J. Severinsen, M. de Roiste, F. Reitsma, and E. Hartato, “VGTrust: measuring trust for volunteered geographic information,” _International Journal of Geographical Information Science_ , pp. 1–19, 2019.
  * [100] C. Lawrence and S. Riezler, “NLmaps: A Natural Language Interface to Query OpenStreetMap,” in _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations_ , 2016, pp. 6–10.
  * [101] C. Haas and S. Riezler, “A Corpus and Semantic Parser for Multilingual Natural Language Querying of OpenStreetMap,” in _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , 2016, pp. 740–750.
  * [102] C. Lawrence and S. Riezler, “Improving a neural semantic parser by counterfactual learning from human bandit feedback,” _arXiv preprint arXiv:1805.01252_ , 2018.
  * [103] J. Andreas, A. Vlachos, and S. Clark, “Semantic parsing as machine translation,” in _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , vol. 2, 2013, pp. 47–52.
  * [104] S. Lobry, J. Murray, D. Marcos, and D. Tuia, “Visual Question Answering from Remote Sensing Images,” in _IEEE International Geoscience and Remote Sensing Symposium (IGARSS)_ , 2019, to appear.
  * [105] M. Ruta, F. Scioscia, D. De Filippis, S. Ieva, M. Binetti, and E. Di Sciascio, “A semantic-enhanced augmented reality tool for OpenStreetMap POI discovery,” _Transportation Research Procedia_ , vol. 3, pp. 479–488, 2014\.

John E. Vargas-Muñoz received the B.Sc. degree in informatics engineering from
the National University of San Antonio Abad in Cusco, Cusco, Peru, in 2010,
and the master’s degree in computer science from the University of Campinas,
Campinas, Brazil, in 2015. He is currently pursuing the Ph.D. degree with the
University of Campinas. His research interests include machine learning, image
processing, remote sensing image classification, and crowdsourced geographic
information analysis.
---
Shivangi Srivastava received her B.E. in Electrical Engineering at Delhi
Technological University (Formerly Delhi College of Engineering), India, in
2009. In 2013, she received her European Master of Science in Nuclear Fusion
and Engineering Physics and in 2015, Masters by Research in Mathematics Vision
and Learning at Centrale Suplélec, France. She commenced her Ph.D. at the
University of Zurich, Switzerland in 2015 and is continuing the same at
Wageningen University and Research, the Netherlands. Currently, her research
focuses on applications of machine learning, in particular deep learning,
along with multi-modal data (remote sensing, geospatial information, and
social media) to understand urban land-use characterization.
---
Devis Tuia (S’07, M’09, SM’15) received the Ph.D in environmental sciences at
the University of Lausanne, Switzerland, in 2009. He was a Postdoc at the
University of Valéncia, the University of Colorado, Boulder, CO and EPFL
Lausanne. Between 2014 and 2017, he was Assistant Professor at the University
of Zurich. He is now Full Professor at the Geo-Information Science and Remote
Sensing Laboratory at Wageningen University, the Netherlands. He is interested
in algorithms for information extraction and data fusion of geospatial data
(including remote sensing) using machine learning and computer vision. He
serves as Associate Editor for IEEE TGRS and the Journal of the ISPRS. More
info on http://devis.tuia.googlepages.com/
---
Alexandre X. Falcão is full professor at the Institute of Computing,
University of Campinas, Campinas, SP, Brazil. He received a B.Sc. in
Electrical Engineering from the Federal University of Pernambuco, Recife, PE,
Brazil, in 1988. He has worked in biomedical image processing, visualization
and analysis since 1991. In 1993, he received a M.Sc. in Electrical
Engineering from the University of Campinas, Campinas, SP, Brazil. During
1994-1996, he worked with the Medical Image Processing Group at the Department
of Radiology, University of Pennsylvania, PA, USA, on interactive image
segmentation for his doctorate. He got his doctorate in Electrical Engineering
from the University of Campinas in 1996. In 1997, he worked in a project for
Globo TV at a research center, CPqD-TELEBRAS in Campinas, developing methods
for video quality assessment. His experience as professor of Computer Science
and Engineering started in 1998 at the University of Campinas. His main
research interests include image/video processing, visualization, and
analysis; graph algorithms and dynamic programming; image annotation,
organization, and retrieval; machine learning and pattern recognition; and
image analysis applications in Biology, Medicine, Biometrics, Geology, and
Agriculture.
---