# Reference study of IFC software support: the GeoBIM benchmark 2019 — Part I

Francesca Noardo  3D Geoinformation, Delft University of Technology, Delft,
The Netherlands —<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Thomas Krijnen  3D Geoinformation, Delft University of
Technology, Delft, The Netherlands —<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Ken Arroyo Ohori  3D
Geoinformation, Delft University of Technology, Delft, The Netherlands —
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Filip
Biljecki  Department of Architecture, National University of Singapore,
Singapore —<EMAIL_ADDRESS>Department of Real Estate, National University
of Singapore, Singapore  Claire Ellul  Department of Civil, Environmental and
Geomatic Engineering, University College London, London, UK —
<EMAIL_ADDRESS>Lars Harrie  Department of Physical Geography and Ecosystem
Science, Lund University, Sweden — (lars.harrie<EMAIL_ADDRESS>Helen Eriksson  Department of Physical Geography and Ecosystem Science, Lund
University, Sweden — (lars.harrie<EMAIL_ADDRESS>Lorenzo Polia
Architect —<EMAIL_ADDRESS>Nebras Salheb  3D Geoinformation, Delft
University of Technology, Delft, The Netherlands —<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Helga Tauscher  Faculty of
Spatial Information, Dresden University of Applied Sciences, Dresden, Germany
— (helga.tauscher<EMAIL_ADDRESS>Jordi van Liempt  3D
Geoinformation, Delft University of Technology, Delft, The Netherlands —
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Hendrik Goerne  GMX, Germany —<EMAIL_ADDRESS>Dean Hintz  Safe
Software, Surrey, Canada —<EMAIL_ADDRESS>Tim Kaiser  Faculty of Spatial
Information, Dresden University of Applied Sciences, Dresden, Germany —
(helga.tauscher<EMAIL_ADDRESS>Cristina Leoni  Department of
Civil, Constructional and Environmental Engineering, Sapienza Univerisity of
Rome, Rome —<EMAIL_ADDRESS>Artur Warchoł  Institute of Technical
Engineering, PWSTE Bronisław Markiewicz State University of Technology and
Economics in Jarosław, Poland / ProGea 4D sp. z o.o. —
<EMAIL_ADDRESS>Jantien Stoter  3D Geoinformation, Delft
University of Technology, Delft, The Netherlands —<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

IFC, the buildingSMART open standard for BIM, is underused with respect to its
promising potential, since, according to the experience of practitioners and
researchers working with BIM, issues in the standard’s implementation and use
prevent its effective use. Nevertheless, a systematic investigation of these
issues has never been performed, and there is thus insufficient evidence for
tackling the problems. The GeoBIM benchmark project is aimed at finding such
evidence by involving external volunteers, reporting on tools behaviour about
relevant aspects (geometry, semantics, georeferencing, functionalities),
analysed and described in this paper. Interestingly, different IFC software
with the same standardised datasets yield inconsistent results, with few
detectable common patterns, and significant issues are found in their support
of the standard, probably due to the very high complexity of the standard data
model. This paper is in tandem with Part II, describing the results of the
benchmark related to CityGML, the counterpart of IFC within geoinformation.

KEYWORDS: Industry Foundation Classes, Building Information Models, open
standards, software support, interoperability, GeoBIM.

This is the author’s version of the work.

It is posted here only for personal use, not for redistribution and not for
commercial use.

The definitive version is published in the journal _Transactions in GIS_.

Noardo, F., Krijnen, T., Arroyo Ohori, K., Biljecki, F., Ellul, C., Harrie,
L., Eriksson, H., Polia, L., Salheb, N., Tauscher, H., van Liempt, J., Goerne,
H., Hintz, D., Kaiser, T., Leoni, C., Warchol, A., Stoter, J. (2021).
Reference study of IFC software support: the GeoBIM benchmark 2019 – Part I.
_Transactions in GIS_.
doi: https://doi.org/10.1111/tgis.12709

Full details of the project: https://3d.bk.tudelft.nl/projects/geobim-
benchmark/

##  1 Introduction

In the Architecture, Engineering and Construction (AEC) fields, as well as in
further disciplines, interoperability is of increasing importance, in order to
enable re-use and exchange of data and information, including in the strictly
related asset and facility management field. Furthermore, it is essential for
reciprocal integration of data having different nature. One of the current
research topics is, for example, the integration of building information
models (BIMs) with 3D city models, for supporting several use cases, e.g.
building permits issuing, 3D cadastre, complex assets and facility management.
International standards are conceived as a solution for fostering such
interoperability. The most popular standard data models, considered for
obtaining such integration, are usually the OGC CityGML for 3D city models and
buildingSMART Industry Foundation Classes (IFC) for BIMs.

However, although a part of the building designers and professionals,
supporting openBIM, are IFC enthusiasts, many others working with BIMs still
seldom use IFC as a first choice for exchanging their models, and only export
them to IFC when required explicitly by law, when trying to integrate with
software from different vendors, or when no other solutions are available, as
emerged from several personal communications. This often happens because they
are aware of some limitations in the IFC format (as for example documented
more than ten years ago by Pazlar and Turk (2008), or in Barbini et al.
(2019)), although they might acknowledge its potential as an open exchange
format.

In practice, limited support for IFC might not been considered as a major
issue, since some very widespread software, such as the Autodesk Revit format,
is often used as de facto standard for BIMs exchange among designers, as well
as for integrating the BIM model with further systems (Baik et al. ; Kensek
(2014); Petersen et al. (2018); Lv (2018); Kamari et al. (2018); Papadopoulos
et al. (2017); Lamartina and Sa (2019); Aksamija (2018); Farid Mohajer and
Aksamija (2019) are only few examples), while the actual use of BIMs by
authorities, besides visualization, is still limited, for the moment. This
solves the most immediate interoperability issues, even though the Revit
format is a proprietary binary format, which hinders re-usability in an open
way and across time.

A specific study of the limitations of IFC was part of the GeoBIM benchmark
project111https://3d.bk.tudelft.nl/projects/geobim-benchmark/ (see Section 2)
and is reported in this paper. Within the project, the approach to the study
of the support for the two standards involved in the GeoBIM integration (IFC
and CityGML) was conceived in parallel, also with the aim of understanding if
one of the two offered more effective solutions that could be possibly
borrowed by the other one in future developments. However, the final outcomes
of the two different tasks are very specific for each standard and deserve to
be presented and discussed separately, considering the specificities of each
case. For these reasons, this paper, which focuses on the results about the
benchmark Task 1 — support for IFC, is written in tandem with Noardo et al.
(2020d), which describes Task 3, covering the support for CityGML. In order to
allow each paper to be read on its own, the two papers share some information
(i.e. Section 2 explaining the general context and motivation of the study;
Section 3.1 covering the initial part of the methodology about the entire
GeoBIM benchmark set-up, and Section 3.3 concerning some similarities in the
methodology). One further paper explores the parts of the project more
directly related to the subject of integration itself, namely, conversion
procedures and useful tools to georeference IFC models (Noardo et al., 2020c).

##  2 The GeoBIM needs and the concept of this study

Two increasingly developed, studied and used 3D information systems have
revealed their potential in the related fields:

  * •

3D city models, which are used to represent city objects in three dimensions
and advance previous 2D maps and other cartographic products, in order to
support city analysis and management, city planning, navigation, and so on
(e.g. Biljecki et al. (2015); Kumar et al. (2017); Egusquiza et al. (2018);
Jakubiec and Reinhart (2013); Liang et al. (2014); Bartie et al. (2010);
Peters et al. (2015); Nguyen and Pearce (2012));

  * •

Building Information Models (BIM), which are used in the architecture,
engineering and construction fields (AEC) to design and manage buildings,
infrastructure and other construction works, and which also have features
useful to project and asset management (e.g. Petri et al. (2017); Haddock
(2018); Azhar (2011)).

Several international standards exist to rule the representation of the built
environment in a shared way, to foster interoperability and cross border
analysis and, consequently, actions, or to reuse tools, analysis methods and
data themselves for research and, possibly, government. Some example of
international standards are: the European Directive for an infrastructure for
spatial information in Europe (INSPIRE)222https://inspire.ec.europa.eu, aimed
at the representation of cross border pieces of land in Europe, for common
environmental analysis; the Land and Infrastructure standard
(LandInfra)333https://www.ogc.org/standards/landinfra, by the Open Geospatial
Consortium (OGC), aimed at land and civil engineering infrastructure
facilities representation; and the green building data model
(gbXML)444https://www.gbxml.org, aimed at the representation of buildings for
energy analysis.

Nonetheless, the two dominant reference open standards for those two models
are CityGML555citygmlwiki.org, by the OGC, focusing on urban-scale
representation of the built environment, and the Industry Foundation Classes
(ISO, 2013)666https://technical.buildingsmart.org/standards/ifc/, by
buildingSMART, aimed at the very detailed representation of buildings and
other construction works for design and construction objectives, first, but
also intended to enable project management throughout the process, and asset
and facility management in a following phase. Those standards are both
intended to be very comprehensive and are therefore very wide and articulated.
They both use complex data models allowing for a wide variety of models using
object-oriented representations, even if that comes at a cost of slower and
more inconsistent implementations.

Due to the overlapping interests in both fields (meeting in the building-level
representation), increasing attention is being paid to 3D city model-BIM
integration (GeoBIM), where the exchange of information between geospatial (3D
city models) and BIM sources enables the reciprocal enrichment of the two
kinds of information with advantages for both fields, e.g. automatic updates
of 3D city models with high-level-of-detail features, automatic representation
of BIM in their context, automated tests of the design, and so on (Liu et al.,
2017; Fosu et al., 2015; Aleksandrov et al., 2019; Kumar et al., 2019; Niu et
al., 2019; Noardo et al., ; Arroyo Ohori et al., 2018; Kang and Hong, 2015;
Stouffs et al., 2018; Lim et al., 2019).

The GeoBIM subject can be divided into several sub-issues.

  1. 1.

The harmonization of data themselves, which have to concretely fit together,
with similar (or harmonizable) features (e.g. accuracy, kind of geometry,
amount of detail, kind of semantics, georeferencing).

  2. 2.

Interoperability, which is a fundamental key in the integration. It is
important to note here, that before enabling the interoperability among
different formats (e.g. GIS formats and BIM formats), which is the theme of
point three below, the interoperability GIS-to-GIS and BIM-to-BIM itself is
essential. That means that the formats of data have to be understood and
correctly interpreted uniquely by both any person and any supporting software.
Moreover, an interoperable dataset is supposed to remain altogether unchanged
when going through a potentially infinite number of imports and exports by
software tools, possibly converting it to their specific native formats and
exporting it back.

  3. 3.

The effective conversion among different formats, i.e. transforming one
dataset in a (standardised) format to another one in compliance with the end
format specifications and features.

  4. 4.

The procedures employing 3D city models and the ones based on BIM should be
changed in order to obtain better advantages by the use of both, integrated,
since those systems enable processes which are usually more complex than just
the simple representations.

The many challenges implied by the points above are still far from being
solved, and one of the essential initial steps is actually to outline such
challenges more sharply.

In particular, the second point (interoperability and involved standards) is
often considered to be solved by standardization organizations. It is
desirable to rely on open standards for this, because the well-documented
specifications of open standards would enable longer-term support, as well as
their genericity with respect to different software vendors, as opposed to
closed point-to-point solutions that merely connect one proprietary system to
another (and might be discontinued or stop working at any moment). However,
our previous experiences suggest that the support for open standards in
software often include shortcomings.

The researchers promoting this study (as users of data, advocates of open
standards and developers of tools adopting such standards) have noticed, over
their research and work activities, how the use of those standards in data and
their implementations in software were not always straightforward and not
completely consistent with the standard specifications either. Many tools,
when managing standardized data, do not adequately support features or
functionalities as they do when the data is held in the native formats of the
software. In addition, software tools have limitations with respect to the
potential representation (geometry, semantics, georeferencing) of data
structured following these standards, or can generate errors and erroneous
representations by misinterpreting them.

The standards themselves are partly at fault here, since they often leave some
details undefined, with a high degree of freedom and various possible
interpretations. They allow high complexity in the organization and storage of
the objects, which does not work effectively towards universal understanding,
unique implementations and consistent modelling of data. This is probably due
to the fact that such standards often originate as amalgamations of existing
mechanisms and compromises between the various stakeholders involved. These
experiences have been informally confirmed through exchanges within the
scientific community and especially with the world of practitioners, who are
supposed to work with (and have the most to gain from) those standardized data
models and formats. However, more formal evidence on the state of
implementation of these open standards and what problems could be connected to
the standard themselves have not been compiled so far.

For this reason, the GeoBIM benchmark
project777https://3d.bk.tudelft.nl/projects/geobim-
benchmark/,888https://www.isprs.org/society/si/SI-2019/TC4-Noardo_et_al_WG-
IV-2-final_report.pdf was proposed and funded in 2019 by the International
Society for Photogrammetry and Remote Sensing (ISPRS)999https://www.isprs.org
and the European association for Spatial Data Research
(EuroSDR)101010http://www.eurosdr.net. The aim of the benchmark was to get a
better picture of the state of software support for the two open standards
(IFC and CityGML) and the conversions between them, in order to formulate
recommendations for further development of the standards and the software that
implements them. In addition, we tested two known major technical issues
related to GeoBIM integration and which are known to be solved only partially
in practice: the ability of tools and methods to georeference IFC and the
conversion procedures between IFC and CityGML (Noardo et al., 2020c).

The relevant outcomes regarding the support of software for buildingSMART IFC
standard are the subject of this paper.

###  2.1 Industry Foundation Classes and knotty points

The buildingSMART Industry Foundation Classes (IFC)
standard111111https://technical.buildingsmart.org/standards/ifc/ is an open
standard data model for Building Information Modelling (BIM) to be shared and
exchanged through software applications, domains and use cases, within the
Architecture Engineering and Construction (AEC) and Facility Management (FM)
fields. It includes classes for describing both physical and abstract concepts
(e.g. cost, schedule, etc.) concerning AEC-FM for buildings, mainly (recent
versions are extending it for including infrastructures and other kinds of
constructions)121212https://technical.buildingsmart.org/standards/ifc/ifc-
schema-specifications/. It has also been adapted as the ISO 16739
international standard (ISO, 2013).

The standard includes relevant constructs for a wide variety of disciplines,
use cases and processes associated to the construction domain, most
prominently the semantic description and geometric representation of typical
construction elements and their relationships.

The IFC is structured in a deeply hierarchical data model, furthermore
organized in several and deep and complex meronymic (part-of) trees too. The
spatial composition (Site/Building/Storey/Space/Zone) is one more kind of
aggregation, different from the element (meronymical) composition one (e.g. a
stair and the assembled elements in it). Moreover, nesting is a slightly
different kind of element composition, representing the products which are
specifically designed as complementary by manufacturers. Finally, subtraction
relationships are also part of the IFC model, representing openings by means
of the voiding mechanism. A great number of further relationships are added to
this complexity (e.g. to associate materials, geometric representation or
other property information and so on).

An additional complexity to the semantic model is given by the possibility to
store the same kind of object by means of several entities. For example the
layers within a compound wall object can be represented by means of an
associated IfcMaterialLayerSet, but also as a more generic decomposition where
every wall layer is modelled as a distinct IfcBuildingElementPart.

A great number of attributes can be associated with entities (and inherited by
the parent-ones), both directly or through property sets.

All this semantic complexity is intended to represent faithfully the buildings
as functional to the standard designed scope. However, the implementation and
use of such theoretically precise model is difficult and can result in
inaccuracies or under-use of it, besides hindering interoperability for
leaving too high freedom in filling the information in and in choosing the
kind of representation to be used.

Additional terms which can be used in IFC, are defined within the
buildingSMART Data Dictionaries (bsDD) and are modelled according to the
International Framework for Dictionaries
(IFD)131313http://bsdd.buildingsmart.org. It is based on the standard ISO
12006–3.

The IFC current versions are: IFC2x3, which was released in 2005 (with the
latest corrigendum in 2007) and the IFC4.1 from 2018. At the time of writing,
the most implemented and used version is still (by far) IFC2x3. For this
reason, both versions were considered in this study.

A third part of the standards is the Information Delivery Manual (IDM), which
defines the workflow and the information exchange specifications and
requirements in the processes involved in the construction life cycle.

In an IDM, a set of Model View Definitions (MVD) can be defined for
identifying the portion of the IFC model which is needed for a particular
information exchange in the IDM to be fulfilled. This can define a use-case
oriented part of the wide IFC model, to be implemented in software.

Interesting to note is that this mechanism is in a way opposite of CityGML
Application Domain Extensions (ADEs) (Biljecki et al., 2018). In an IFC MVD a
subset of definitions is selected from the monolithic schema, instead of an
addition. On the other hand, for property set definitions, the usage of MVD
and ADE are analogous, as both nominate additional attributes.

IFC derives many aspects from ISO (2014), informally known as STEP. The
majority of geometry definitions are derived from ISO 10303–42 and the typical
exchange formats are based on STEP Physical File (SPF, ISO 10303–21) and an
XML encoding (ISO 10303–28).

Parametric modelling is usually employed in BIM and IFC, which makes it
possible to encode many kinds of geometries. This includes Boolean operations
and complex sweeps, for example the sweep of an arbitrary profile along a
curve while constraining the normal vector. Also explicit geometries are
supported in the form of Boundary Representations and (added in IFC4)
efficient support for triangulated meshes. The implementation of the former
type of geometry is that supporting the full stack of geometric procedures in
IFC is a major implementation effort and due to implementation choices can
sometimes lead to different results in importing applications. The complexity
can, therefore, have consequences on interoperability and the way different
pieces of software read and re-export the same geometry.

####  2.1.1 IFC geometries

The particular issues described in this section guided the methodological
choice of providing a specific set of IFC geometries among the benchmark
datasets, in order to point out specific related behaviour (Figure 1 is an
example of uncontrolled geometries transformations within some of the tested
software).

Figure 1: Example of unusual behaviour of the geometries in some of the tested
software.

IFC is a complex standard and the fidelity of both importers and exporters is
still evolving. Implementations are based on varying paradigms and geometric
modelling principles: Boundary Representation, Polyhedra or Triangular meshes.
Some implementations use so called toolkits that offer complete support for
STEP and EXPRESS (the modelling language in which the IFC schema is specified;
with an embedded complex Turing-complete constraint language to specify WHERE
rules and FUNCTIONS). Other implementations use custom in-house developments
and, for that reason, typically have less support for the automatic validation
of IFC models based on the WHERE rules. The IFC schema is evolving as
well141414as seen in repositories such as
https://github.com/buildingSMART/NextGen-IFC mostly aiming at the improvement
of reliability, reuse of industry-standard best-practices and the reduction of
implementation effort.

Given these considerations, the permutations of IFC definitions (some of which
assessed by this set of analytical geometries) span a three-dimensional matrix
along the axes (Figure 2) that specify:

  1. 1.

whether the situation is known to be exported by authoring tools

  2. 2.

whether the situation is handled successfully by importing applications and

  3. 3.

whether that situation is valid according to the IFC schema.

As such we can summarize this state in a tuple of three elements. For example,
for a situation that is exported by authoring tools, successfully imported,
but not valid according to the schema, the tuple would be $<Y,Y,N>$.

From these states we want to highlight three that are meaningful:

  * •

$<N,Y/N,Y/N>$ is a situation that is never exported and can hence be removed
from the schema to reduce the implementation effort of a fully compliant
application.

  * •

$<Y,Y,N>$ are situations that are successfully imported, but not valid
according to the schema, schema constraints may be loosened to actually allow
such situations.

  * •

$<Y,N,Y/N>$ is the biggest problem for practitioners, i.e. geometries that are
exported, but not successfully imported (whether valid or not).

An additional concern is that while validity according to the schema is fairly
constant and universal, import success depends on the application used, and
with many different tools used by practitioners, the diverging results of
importers may cause misconceptions and disputes. Also note that validity is
not as universal as it might seem at first glance. The IFC schema defines a
point equality tolerance (IfcGeometricRepresentationContext.Precision) that
defines “the tolerance under which two given points are still assumed to be
identical”. How this is handled in geometry kernels is implementation
dependent. Additionally there are WHERE rules that specify that an extrusion
length needs to be positive (IfcExtrudedAreaSolid.Depth is of type
IfcPositiveLengthMeasure which has a rule WR1 : SELF > 0.) but at the same
time EXPRESS does not specify restrictions on the floating point number type,
so an infinitesimal value can be supplied that is greater than zero, but when
mapped to the number type in the importing application cannot be distinguished
from it. Or an extrusion depth can be supplied that is below the model
tolerance and therefore may fail in applications that adhere to this
tolerance.

NYNYNYexportedimportedvalid Figure 2: Three-dimensional matrix representing
the IFC geometries conditions: can be exported / can be imported / is valid
according to the IFC standard.

A significant set of geometries with a variation of such three features was
added as part of the tested IFC datasets, as described in Section 3.2.1.

####  2.1.2 IFC georeferencing

Proper georeferencing of an IFC file facilitates a link between the local
Cartesian coordinate system used by IFC, with their corresponding real-world
coordinate reference system (CRS), and thus to place the model of a single
building or construction within its context and environment. There are several
options to store georeferencing information in IFC, with varying level of
detail. These options range from basic address information over the
specification of the geolocation of a reference point to the definition of an
offset between the project coordinate system and the global origin of a
coordinate reference system and the corresponding rotation of the XY-Plane.
Some of the georeferencing options are described and classified into Levels of
Georeferencing (LoGeoRefxx) by Clemen and Görne (2019) (Table 18). This
classification scheme is not officially defined in the IFC standard but can
help practitioners to assess which georeferencing information is available in
the IFC file. Common for the presented options are that coordinates for a
reference point (which is usually the origin of the local Cartesian system
where the model is designed) are stored and sometimes complemented with a
direction of the axis in the local system.

LoGeoRef |  Supported CRS |  Storing entities
---|---|---
LoGeoRef10 |  No CRS, approximate location by means of the address. |  IfcPostalAddress referenced by either IfcSite or IfcBuilding.
LoGeoRef20 |  WGS84 EPSG:4326 |  Attributes RefLatitude, RefLongitude, RefElevation within IfcSite
LoGeoRef30 |  Any Cartesian CRS, including projected coordinates (CRS not specified in the file) |  IfcCartesianPoint referenced within IfcSite (defining the projected coordinates of the model reference point);IfcDirection attribute of IfcSite (stores rotations regarding project or global north).151515Ad-hoc solution used by several tools.
LoGeoRef40 |  Any Cartesian CRS, including projected coordinates (CRS not specified in the file) |  Attribute WorldCoordinateSystem storing the coordinates of the reference point in any Cartesian CRS (including the projected ones) and direction TrueNorth. Both are stored within IfcGeometricRepresentationContext.161616Most official IFC2x3-way to store the reference system.
LoGeoRef50 |  Specific projected CRS, specified by means of the EPSG code |  IFC v.4 only. Coordinates of the reference point stored in IfcMapConversion using the attributes Eastings, Northings and OrthogonalHeight for global elevation. Rotation for the XY-plane stored using the attributes XAxisAbscissa and XAxisOrdinate. The CRS used is specified by IfcProjectedCRS in the attribute Name by means of the proper EPSG code.
Table 1: List of georeferencing options in IFC classified as LoGeoRefs (Clemen
and Görne, 2019). EPSG is a coding list for CRS.181818https://epsg.io/

The georeferencing of BIM has not been a priority for designers and software
developers. Therefore the georeferencing information is not regularly stored
and used, and not always read and exported in a completely consistent way.
Moreover, the available tools to georeference BIMs are not perfectly optimized
to acquire and store the correct information in the foreseen attributes and
entities in the IFC files. For this reason, the IFC georeferencing, which is
fundamental for GeoBIM integration, was investigated within the benchmark,
from two main points of view.

On the one hand, the ability of software to correctly interpret the
georeferencing information provided within the datasets was checked, with the
focus on the georeferencing level LoGeoRef20 and LoGeoRef30. Since LoGeoRef30
is more accurate, allowing the storage of Cartesian projected coordinates, we
asked participants to check if this one could be read and interpreted by
software correctly, within the Task 1 (see Section 3.1).

On the other hand, it is quite unexplored if and how the tools can edit the
georeferencing information of IFC models: normally there is no or only little
control of how georeferencing information is stored in the final model. For
this reason, we tested this issue separately, within the benchmark Task 2 (see
Section 3.1, (Noardo et al., 2020c)).

##  3 Methodology

###  3.1 The GeoBIM benchmark general set-up

The benchmark was intended as a way to combine the expertise of many people
with different skills, coming from several fields and interests, in order to
describe the present ability of current software tools to use (i.e. read,
visualize, import, manage, analyse, export) CityGML and IFC models and to
understand their performance while doing so, both in terms of information
management functionalities, and possible information loss. Moreover, since the
large size of such standardised datasets often generates difficulties in their
computational management, the ability to handle large datasets was a further
part of the tests.

In particular, the four topics investigated in the benchmark are:

$Task1$

What is the support for IFC within BIM (and other) software?191919This Task is
the object of this paper.

$Task2$

What options for geo-referencing BIM data are available?

$Task3$

What is the support for CityGML within GIS (and other) tools?

$Task4$

What options for conversion (software and procedural) (both IFC to CityGML and
CityGML to IFC) are available?

For this purpose, a set of representative IFC and CityGML datasets were
provided (Noardo et al., 2019a) and used by external, voluntary, participants
in the software they would like to test in order to check the support in it
for the considered open standard (Noardo et al., 2019b).

Full details about the tested software and a full list of participants can be
found in the respective pages of the benchmark
website202020https://3d.bk.tudelft.nl/projects/geobim-benchmark/software.html
for the tested software and https://3d.bk.tudelft.nl/projects/geobim-
benchmark/participants.html for the list of participants.. The significant
number of participants, balance in skills, fields of work, levels of
confidence about the tested software (asked them to be declared) offered the
possibility to limit the bias in the results.

The participants described the behaviour of the tested tools following
detailed instructions and delivered the results in a common template with
specific questions, provided as online forms. In the end, they delivered both
their observations and the models as re-exported back to the original
standardised format (CityGML or IFC).

In order to cover the widest part of the list of software potentially
supporting the investigated standards, we completed the testing ourselves, by
searching the online documentation of both the standards and the potential
software.

In the final phase of the project, the team coordinating the study analysed
the participants’ observations, descriptions and delivered further
documentation (screenshots, log files, related documents and web pages). From
this review, an assessment of the performances and functionalities of the
tested tools was derived. Moreover, the delivered models were validated and
analysed using available tools, when possible, and/or through manual
inspection (Section 3.3). This approach allowed the inquiry about the level of
interoperability given by the standard and its software implementation, by
comparing the results of the export with the imported model features.

It is important to notice that the test results are not intended to substitute
the official documentation of each software. Moreover, there were no expertise
nor skill requirements to participate in the benchmark tests. Therefore, some
information could be wrong or inaccurate, due to little experience with the
tested software or the managed topics. The declaration of the level of
expertise was intended to lower this possible bias. Moreover, the benchmark
team and the authors tried to double check the answers (at least the most
unexpected ones) as much as possible, but the answers reported in the data
were generally not changed from the original ones. The eventual discrepancies
between the best potential software performances and what was tested could
anyway be showing a low level of user-friendliness of tools (and thus a degree
of difficulty in achieving the correct result).

###  3.2 The provided IFC datasets

We have selected IFC datasets allowing the testing of the main issues
regarding the support of IFC (Table 2): one dataset was the architectural IFC
model of a small two-floor building, intended to test the support for IFC
features in the most reliable way (Myran.ifc). The second one was a large and
complex architectural IFC model of a big tower in Rotterdam (UpTown.ifc),
intended to test the software’s performance; and the last one was the
architectural IFC model of a medium-size building in IFC4 (Savigliano.ifc),
which was chosen to test also the support for IFC4 files and eventual
discrepancies with the management of the other datasets, which were provided
in IFC version 2x3 (which our previous experience suggests is better
supported). All the IFC files where originally modelled in Autodesk Revit and
thereafter exported as IFC files.

Moreover, since performing tests of IFC geometries could be tricky in complex
files combining many geometry types in an uncontrolled manner, we chose to
monitor this aspect better by means of synthetically generated geometries
(using IfcOpenShell212121http://ifcopenshell.org) to test the support of
software for specific kinds of geometry modelling (more detail in Section
3.2.1). Those geometries are provided in both IFC2x3 and IFC4.

Name |  Description |  IFC vers. |  Size (MB) |  Source |  Aim
---|---|---|---|---|---
Myran.ifc |  Model of a small 2-floor building in Sweden, by Swedish architects. Georeferenced. |  2x3 |  27.14 |  Mondo arkitekter, (SE) |  Test of the main functionalities of software and common procedures.
UpTown.ifc |  Model of a big complex tower in Rotterdam, by Dutch architects. |  2x3 |  241.04 |  Municipality of Rotterdam (NL) |  Test of the software’s performance.
Savigliano.ifc |  Model of a building in Italy, by an Italian architect within a research environment. |  4 |  21.55 |  Arch. Lorenzo Polia (IT) |  Test of the support for IFC v.4 and to enable the tests of procedures and tools working with IFC v.4.
Specific IFC geometries |  Set of geometries modelled using a range of the modelling alternatives allowed in IFC (see section 3.2.1). |  2x3; 4 |  0.31 (v.2x3); 0.27 (v.4) |  Synthetically generated with IfcOpenShell. |  Test of the support and behaviour of software with respect to these specific geometries.
Table 2: Summary of descriptions, features and aims of the provided IFC
datasets.

####  3.2.1 IFC geometry sets

The geometries used in the BIM models can have a huge numbers of variations
and a complete validation of them, as used in models, is still an unsolved
task. Moreover, IFC admits a number of geometry types that can be useful to
modellers, but are sometimes not supported or can be interpreted in different
ways by software. On the other hand, IFC puts validity constraints on certain
geometries, but some software implements workarounds aiming at reading those
invalid geometries too, which are often undocumented, and there is therefore
often little possibility to keep track of these solutions. For these reasons
and in order to explore the issues detailed in Section 2.1.1, a specific set
of geometries was provided among the benchmark datasets (Figure 3,Table
LABEL:tab:ifcgeometries).

Figure 3: The dataset with synthetically generated IFC geometries. Table 3: Description of each object included in the IFC geometries set, in both IFC 2x3 and IFC 4 (the leftmost column refers to the row and column in Figure 3). *1 not valid due to violation of the positive length measure requirement on IfcExtrudedAreaSolid.Depth *2 not valid due to violation of the where rule ValidExtrusionDirection *3 not valid as the parameter range for the sweep is outside the parametric range of the curve † not included in the IFC 4 dataset due to removed entities in the schema. ‡ valid, but using a non-normalized extrusion depth which may render differently in applications ‖ this shape should have fillet radii, but not all viewing software displays those  |  IFC Definition |  Description
---|---|---
A1 |  IfcBooleanResult_1 |  Result of boolean subtraction with two cube operands with partial overlap
A2 |  IfcBooleanResult_2 |  Result of boolean intersection with two cube operands with partial overlap
A3 |  IfcBooleanResult_3 |  Result of boolean union with two cube operands with partial overlap
A4 |  IfcBooleanClippingResult_1 |  Result of boolean clipping operation with a cube and a halfspace operand
A5 |  IfcShellBasedSurfaceModel_1 |  A shell based surface model, an explicit collection of faces
B1 |  IfcFacetedBrep_1 |  A faceted boundary representation, an explicit collection of faces
B2 |  IfcExtrudedAreaSolid_1 |  Extrusion of a rectangular profile
B3*1 |  IfcExtrudedAreaSolid_2 |  Extrusion of a rectangular profile, negative extrusion depth
B4*1 |  IfcExtrudedAreaSolid_3 |  Extrusion of a rectangular profile, zero extrusion depth
B5 |  IfcExtrudedAreaSolid_4 |  Extrusion of a rectangular profile, non-normalized direction vector
C1*2 |  IfcExtrudedAreaSolid_7 |  Extrusion of a rectangular profile, direction vector parallel to profile
C2 |  IfcExtrudedAreaSolid_10 |  Extrusion of a rectangular profile, slanted direction vector
C3 |  IfcExtrudedAreaSolid_13 |  Extrusion of a elliptical profile
C4 |  IfcExtrudedAreaSolid_16 |  Extrusion of a elliptical profile, non-normalized direction vector
C5*2 |  IfcExtrudedAreaSolid_19 |  Extrusion of a elliptical profile, direction vector parallel to profile
D1 |  IfcExtrudedAreaSolid_22 |  Extrusion of a elliptical profile, slanted direction vector
D2‖ |  IfcExtrudedAreaSolid_25 |  Extrusion of a I-shape profile
D3‖ |  IfcExtrudedAreaSolid_28 |  Extrusion of a I-shape profile, non-normalized direction vector
D4‖*2 |  IfcExtrudedAreaSolid_31 |  Extrusion of a I-shape profile, direction vector parallel to profile
D5‖ |  IfcExtrudedAreaSolid_34 |  Extrusion of a I-shape profile, slanted direction vector
E1 §† |  IfcExtrudedAreaSolid_37 |  Extrusion of a crane rail (A-shape) profile
E2 §† |  IfcExtrudedAreaSolid_40 |  Extrusion of a crane rail (A-shape) profile, non-normalized direction vector
E3*2§† |  IfcExtrudedAreaSolid_43 |  Extrusion of a crane rail (A-shape) profile, direction vector parallel to profile
E4 §† |  IfcExtrudedAreaSolid_46 |  Extrusion of a crane rail (A-shape) profile, slanted direction vector
E5 |  IfcRevolvedAreaSolid_1 |  Revolution of a rectangular profile
F1 |  IfcRevolvedAreaSolid_2 |  Revolution of a elliptical profile
F2‖ |  IfcRevolvedAreaSolid_3 |  Revolution of a I-shape profile
F3 §† |  IfcRevolvedAreaSolid_4 |  Revolution of a crane rail (A-shape) profile
F4 §† |  IfcSweptDiskSolid_1 |  Swept disk
F5*3§† |  IfcSweptDiskSolid_2 |  Swept disk with parameter range outside of curve definition

The structure of the results template guiding the test about such datasets is
as follows, for every geometry in the set. First there is a general question
whether an object is displayed in that slot after import. Then what follows
are questions about the positioning relative to the $Z=0$ plane (mostly to
distinguish how the negative extrusion depth is handled), the rendering of
curved surfaces and what shape is displayed (to identify some specific
situations such as the non-normalized extrusion direction).

###  3.3 Answers analysis about the support for IFC

The methodology for analysing the results about the support of software for
IFC (Task 1) and CityGML (Task 3) are very similar, since they were also
conceived to test similar issues concerning interoperability and the ability
of software to keep files consistent with themselves after import-export
phases.

The initial part of results analysis (Section 4.2) is qualitative, providing
the description of software support and functionality based on the delivered
answers.

The complete answers and documents delivered in the online
templates222222https://doi.org/10.5281/zenodo.3964445 (Noardo et al., 2020b)
were double checked for correctness and consistency with respect to the asked
questions. However, due to the nature of the initiative, we trusted the
delivered information about the software, double checking it with new tests
only in cases of inconsistent answers in different tests about the same
software, or possibly, unexpected answers. In these cases, we also considered
the level of expertise of the participant to assess if further checks were
actually needed.

The delivered answers in the templates were critically assessed, cross-
checking them with the different tests about the same software and the
attached screenshots. A score about each aspect considered for the assessment
of general support and software functionalities is assigned, as: 1-full
support; 0.5-partial support; 0- no support. Those are synthesized (Table 5),
in order to more easily deduce possible patterns across many issues for a
single software package or across many software packages for a single issue.

The definition of software groups are getting increasingly fuzzy, since the
functionalities of all of them are continuously being extended and now tend to
overlap with each other. However, in the tables, and more generally, in the
analysis, in order to help the detection of possible patterns, the tested
software are classified considering the criteria that usually guide the
choices made by users, based on their different needs for specific tasks:

  * •

GIS are expected to combine different kinds of geodata and layers and make
analysis on them, structured in a database, in a holistic system;

  * •

‘Extended’ 3D viewers are likely software that were originally developed for
visualising the 3D semantic models, including georeferencing, and query them.
They were (sometimes later) extended with new functions for applying symbology
or making simple analysis.

  * •

Extract Transform and Load (ETL) software, and conversion software, are
expected to apply some defined transformations or computations to data;

  * •

3D modelling tools have good support for geometry editing, but is not
originally intended to manage georeferenced data nor semantics;

  * •

Analysis software are intended specifically for few kinds of very specific
analysis (e.g. energy analysis);

  * •

BIM software, are intended to design buildings or infrastructures according to
the BIM method.

The investigated issues, reflected in the different sections of the provided
templates, regarded mainly the support of the software for the two standards
(how the software read and visualise the datasets) and the functionalities
allowed by the software with standardised datasets (what is it possible to do
with such data). In particular, the test about the support was intended to
check: how is the georeferencing information in the files read and managed
(Section 4.2.1); how are the semantics read, interpreted and kept after the
import (Section 4.2.2); and how is the geometry after the import (Section
4.2.3).

The last questions were related to the kind of functionalities (Section 4.3)
that are offered by the software for the management of IFC data:

  * •

What kind of visualization is enabled (3D, 2D, with textures, with specific
themes);

  * •

What kind of editing is possible (attributes, geometry, georeferencing);

  * •

What kind of query (query the single object to read the attributes, selection
by conditions on attributes, spatial query, computation of new attributes);

  * •

What analysis are allowed. This topic is more complex, since very different
analysis can be possible. Therefore we summarized it by defining two analysis
types: ‘Type 1’ is any kind of analysis regarding the model itself (like
geometric or semantic validation), and ‘Type 2’ are the simulations and
analysis about the performances of the represented object (e.g. a building)
with respect to external factors, in the city or environment (e.g. shadow,
noise, energy, etc.).

  * •

Final issue: Is it possible to export back to IFC?

Moreover, the support for each of the delivered datasets were noted, given the
specific features: IFC2x3 building model (Myran.ifc), IFC4 building model
(Savigliano.ifc), very heavy model (UpTown.ifc).

This first parts provide a reference about the tools themselves for people
intending to use standardised information. In addition, the most challenging
tasks and most frequent issues for the management of standards were supposed
to be pointed out.

A second, more quantitative, part of the analysis considers the delivered
models exported back to IFC from the tested software (Sections 4.4 and 4.5).
The exported models of UpTown.ifc were visually inspected but could not be
completely analysed since the used tools (such as the NIST IFC analyser)
crashed without producing any useful result, probably given the big size of
these datasets. For this reason the results regarding UpTown.ifc are not
described in the paper. The models can be though downloaded together with the
other data from Noardo et al.
(2020a)232323https://doi.org/10.5281/zenodo.3964368.

The numbers and types of features of such files were calculated and compared
to the same features in the initial datasets that were provided for the test.
The NIST IFC analyser242424https://www.nist.gov/services-
resources/software/ifc-file-analyzer, calculating the number of IFC entities
of each type in the dataset, was used for this check. Moreover, manual
inspection of the files within 3D viewers (BIM Vision, RDF IfcViewer) was used
to check the apparent problems (e.g. missing elements) and further changes
(e.g. grouping in storeys, transformation of building elements in
IfcBuildingElementProxy, change to some kind of element to others and so on).
The inspection of the text format was also helpful to check some elements
(e.g. the more formal ones, documenting organization, applications, IFC
version and so on).

This allowed us to assess the level of interoperability that the connected
standards-tools can actually reach in the different cases: i.e. can the data
be imported and re-exported without any change?

A further assessment (Section 4.7) was intended to evaluate the software and
hardware connected performance. The times declared by the testers were
compared for each datasets to see how much their computational weight could
affect their management within software.

Given the complexity of measuring software performance to the closest second,
this was not requested of the users. Instead, they were asked to provide an
approximate timing value for each test, according to a classification that was
proposed following the way they could affect the perception or the work of a
user, as explained in the following list:

  * •

It is almost immediate (good!)

  * •

Less than a minute (ok, I will wait)

  * •

1–5 minutes (I can wait, if it is not urgent)

  * •

5–20 minutes (in the meantime I do other things)

  * •

20 minutes–1 hour (I cannot rely on it for frequent tasks)

  * •

more than 1 hour (I launch my process and go home, definitely ineffective for
regular work)

Other options included reporting if the software crashed or if the task was
not possible with the software provided, and participants were also asked to
provide information about the specification of the machine, as this may impact
overall performance of the software.

##  4 Results: support of software for Industry Foundation Classes

###  4.1 Tested software against support for Industry Foundation Classes

For the benchmark we tested 31 different software packages (Table 4), trying
to cover all the possible solutions to manage IFC, although a huge number of
software packages and other tools exist, considering that the BIM-related
industry is quite developed and still growing. In the table, they are
organised based on the kind of software and divided into: open source,
proprietary and freeware (but not open source) software. Moreover, the levels
of expertise of participants making the tests (from Level 1 the least to Level
4 the most expert) are also reported.

Some pieces of software were tested several times, with different levels of
expertise: 2 tests with eveBIM (Level 1, beginner and Level 4, developer); 2
tests with FZK viewer (Levels 1 and 2); 3 tests with Safe Software FME (all
experts); 2 tests with FreeCAD (Levels 1 and 2); 3 tests with ArchiCAD, Levels
1 and 2; and many tests about Revit, as expected, since very popular software
(Levels 1, 2 and 3).

In addition, more (unsuccessful) tools were considered, for example within the
Tekla suite, Tekla Structure is supposed to be the only tool supporting IFC
(and finding the trial versions of others was anyway tricky); Autodesk Fusion,
which is however not supposed to support IFC; bimspot252525https://bimspot.io
crashed when trying to import any of the datasets.

Some other software, found on the internet or initially proposed by the
participants that were found to be unable to support IFC were ACCA Solarius
and Solarius PV, iTown, BimView and bimvie. There were also issues with
finding the trial versions of other software, such as BuildingReconstruction,
OpenDesignAlliance IFC SDK, only allowing some of the tools to be tried, or
not reading IFC correctly (e.g. other Bentley software, ACCA EdiLus). However,
the most popular tools were covered and tested thoroughly.

|  Open Source |  Proprietary |  Freeware
---|---|---|---
GIS Software |  |  Bentley Map Enterprise [L1] ESRI ArcGIS Pro [L3] |
‘Extended’ 3D viewers |  Datacomp Sp. z.o.o. BIM Vision [L1] RDF IFC Viewer [L1] |  CSTB eveBIM [L4] CSTB eveBIM Viewer [L1] TeamSystem STR Vision IFC Viewer [L1] Solibri Anywhere [L1] |  FZKViewer [L1+L2]
ETL and conversion software |  |  FME Desktop [3L3] |
3D modeling software (CAD) |  FreeCAD [L1+L2] Bricsys Blender [L1] |  Bentley MicroStation+TerraSolid [L3] Trimble SketchUp [L1] Cadwork Lexocad [L1] Bricsys BricsCAD Ultimate [L1] |
Analysis software |  |  ACCA PriMus-IFC [L1] Dlubal RFEM [L1] |
BIM software |  BimServer [L2] |  Autodesk Civil 3D [L1] Solibri Office [L1] Tekla Structures [L1] ACCA usBIM.viewer+ [L1] CadLine Ltd ARCHline.XP [L1] Simplebim [L1] Allplan [L1] AutoCAD Architecture [L1] ACCA Edificius [L1] Autodesk Revit [3L3+3L1+1L2] Vectorworks Designer [2L1] Graphisoft ArchiCAD [2L1+L2] DDS-CAD [L1] Autodesk Infraworks [L1] |
Table 4: Software tools tested against support for IFC within the benchmark
Task 1. The participant Level 1 of expertise is abbreviated L1, etc.

###  4.2 Software support for IFC

In Table 5, the more qualitative analysis of the participants’
reports262626https://doi.org/10.5281/zenodo.3964445 (Noardo et al., 2020b) is
synthesized. The result is discussed below focusing on the themes
georeferencing, semantics and geometry.

Table 5: Synthesis table of the delivered tests regarding support for IFC
(benchmark Task 1). The colour scale from green to red is assigned according
to the scores from 1-full support to 0-no support.

####  4.2.1 Load of IFC data: Georeferencing information

It is possible to notice that in most of cases, georeferencing information, as
well as elevation, is not read correctly and the models are moved to a local
system having the origin in (0,0,0), using different units of measure (mainly
metres and millimetres). In some cases it is possible to change this. For
example in eveBIM, by default, it would load the model with a local coordinate
system; it is however possible to change the settings to have it use the
model’s previously defined local coordinate system.

Some tools work partially, for example, FME uses RefLatitude RefLongitude in
IfcSite, although more accurate coordinates are assigned through an
IfcLocalPlacement (LoGeoRef30) or the project’s representation context
(LoGeoRef40).

In other cases, the CRS is misinterpreted, for example Bentley
Microstation+Terrasolid moved all the models to 0,0,0 in the CRS EPSG:3152. In
other software, some georeferencing information is reported in very high
detail, including datum, ellipsoid and so on. However, this information is not
correct.

With some other software, like Revit or ArchiCAD different tests give
different answers, therefore it is possible that it depends on import
settings.

To summarize, only 30% of the tools correctly use the georeferencing
information with proper global coordinates (as provided by LoGeoRef30 or 40).
Very few changes are reported about the tests with the IFC4 dataset, which
does not use the extended IFC4 georeferencing features anyway. Orientation of
the model is instead kept correctly in most of the cases (90%), although it
was difficult to assess if the added True North information was read.

####  4.2.2 Load of IFC data: Focus on Semantics

The semantics are read more consistently than the georeferencing information,
with general good support, although some inaccuracies are anyway found, mostly
consisting in: partial recognition of entities272727As examples, in Bentley
Microstation+TerraSolid, for the Myran.ifc model, only IfcWall and
IfcWallStandardCase were translated; in ACCA Edificius “opening, door, and
curtain wall are detected by the software; IFCBeam and IFCCovering are loaded
as an IFC Proxy Object; It also has vertical/horizontal envelopes which are
actually not IFC classifications”; for Revit: “some classes that are different
in IFC are the same in Revit, following the standard settings; it is possible
to set own Revit Categories (therefore set them to the correct IFC Class
name)”; in ArchiCAD, “Some classes/layers are missing, most notably are the
roofs and windows”; Tekla structures does not import all the entities,
although the present ones are consistent.; loss (or partial loss) of
relationships282828In FME, hierarchy and relationships are kept by means of
parent ids; for Revit, the loss of many relationships is reported, probably
due to the need to separate elements while importing; same for BIM Vision; in
FreeCAD, “spatial aggregation and containment is properly retained (although
in debug mode, aggregations with more than 10 objects are skipped), Type
objects (IfcRelDefinedByType) seem not to be considered,
IfcRelConnectsPathElements (connection between walls) is ignored”; In Autodesk
civil 3D, only storeys aggregation is kept; in ACCA PriMus-IFC, it is only
possible to find many “IfcRelConnectsPathElements” relations.; partial
inconsistencies in the reading and interpretation or loss of
attributes292929In the FZK Viewer sometimes attributes are different from the
reference ones, although generally well-managed and interpreted; Bentley
Microstation+TerraSolid can read consistent attributes for the Uptown.ifc
dataset but cannot for Myran.ifc; DDS-CAD is supposed to read attributes
correctly, however, the function did not work, therefore it was not possible
to check; in Revit, some of the attributes were consistent with the IFC ones,
but most of them were missing; in ArchiCAD, some objects are not recognized
correctly in terms of attributes and semantics..

Approximately, 80% of the tools manage semantics in a satisfactory way,
especially the 3D viewers group performed well. However, many inaccuracies
need to be solved to reach a good interoperability. In particular, it is
noticeable how Revit reports inconsistencies with respect to the imported
original data, although being the software where all the datasets were
modelled and from where they were exported.

####  4.2.3 Load of IFC data: Focus on Geometry

The software tools in this study are considered to support the geometry if
there are no apparent errors; that is no detailed study of geometry was
conducted on the building models, since this is a very complex issue. However,
reported errors generally regard the loss of some elements or their change so
that intersections, deformations or modifications (e.g. in normals)
occurred.303030Specific reported features include: FZK Viewer, generally works
well, except with the Myran.ifc model, whose geometrical representation looks
a bit distorted in one of the tests, while the geometry is not visualized at
all in the other one; Moreover, with the Savigliano.ifc (IFC4) file, some
elements (roof pitches) are not visualised; eveBIM detected wrong normals,
although we do not know if they were wrong in the beginning or were changed
during import, and with respect to IFC4 file, the IfcAdvancedBrep is not
implemented so the geometry is not shown in this case; about Bentley
MicroStation +TerraSolid, some slight changes in dimensions are reported;
BIMServer seems to read geometry correctly, except that the roof at the top of
the tower is missing, and possibly windows; about Revit issues are reported:
some walls intersecting with the floors and beams that do not join correctly,
subtracting the volumes; mistakenly display of subtraction solids inside the
families of doors and windows, which are used to pierce the walls; often the
stratigraphy of walls is not correct in the corners and intersections; in
Vectorwork a scale can be applied to the data, thus, it is necessary to check
that it is correctly set.

In addition, Table 5 reports whether the software is able to load and work
with the different datasets. Issues include: failures most likely connected to
the computational requirements in the case of Uptown.ifc; to the IFC version
in case of Savigliano.ifc; and maybe to possible geometry issues in the case
of Myran.ifc, which is not read correctly in a few cases.

For example, the version 0.18 of FreeCAD is not able to import the Myran.ifc
model, and one of the tests with FZK viewer fails with Myran.ifc as well. In
addition, other import errors are reported in Table 6.

Software |  Import error
---|---
FZK Viewer |  with Savigliano.ifc: The software crashes when trying to add an element with Uptown.ifc: Many errors reported with Myran.ifc: Error 776: Boolean Operation - clipping plane generation for CdgisModel geometry failed; Geometry face - Invalid outer loop; Geometry polyline - Polyline contains collinear points, points removed; Geometry polylines - Double point removed
eveBIM |  with Savigliano.ifc: some geometries can’t be generated. Error message is: “Warn - IFCProduct (#565376): error IfcRepresentationItem (#564617) Error - this representation Item IfcAdvancedBRep (#564922) is not implemented” (this will be a further development of the software)
RFEM |  Many elements were not imported because said to be “not relevant for structural analysis”.
Tekla structure |  You have to convert it to Tekla’s own structure first. It could not convert quite many objects.
Allplan |  It ignored 14 objects for Myran and 440 for Savigliano, but does not state which ones these are.
AutoCAD Architecture |  Despite there not being any errors during the import, not all elements shown on the data page are present.
ACCA Edificius |  It reported the following kind of errors: incorrect element referencing, unexpected error while calculation Entity 2409 cross beam, a critical error has occurred in the structural model calculation.
Revit |  IFC opening elements not imported and needed to be discarded.
Table 6: Errors in the imports for some of the software tools.

####  4.2.4 IFC geometries interpretation

The dataset is synthetically generated for both the IFC2X3 schema as well as
the IFC4 schema, with the IFC4 geometries being a subset of the ones supplied
for IFC2X3. For that reason one would expect similar import success ratios for
the IFC2X3 and IFC4. This is not the case, as shown in Figures 4 and 5. A
possible explanation is that applications supporting IFC4 are newer or better
and therefore have higher success rates.

Figure 4: Item visibility ratio for IFC2X3. The labels on the horizontal axes
refers to the column and row of the geometry in Figure 3 Figure 5: Item
visibility ratio for IFC4. The labels on the horizontal axes refers to the
column and row of the geometry in Figure 3

As mentioned in the discussion in earlier paragraphs, one of the main concerns
of the authors regarding the state of IFC support in importing tools is the
divergence in how situations are handled. On the one hand this can be
explained as evolving support for more complicated geometry types or types
that are not considered native to the domain of the importing tool. On the
other hand, what can be observed in practice is that some tools import invalid
situations without notifying the user. In some cases these are due to tools
not performing checks on the entity instance attribute values or on the
resulting geometries. For example, in the case of the negative extrusion
depth, when the IFC parser does not validate WHERE rules automatically,
detecting the negative extrusion depth requires an explicit statement in the
program code that may have been omitted accidentally. In other cases there is
a deliberate effort to “heal” certain invalid situations. For example the
IfcOpenShell software library provides functionality to detect self-
intersections in face boundaries definitions and discard all but the largest
cycle of
edges313131https://github.com/IfcOpenShell/IfcOpenShell/blob/df81490/src/ifcgeom/IfcGeomFunctions.cpp#L3656.
While on the one hand this is implemented with good intentions, this behaviour
does deviate from strict standard compliance and results in different flavours
or dialects of IFC being supported among applications, which in the long run
will hurt interoperability. The divergence of importing applications is
displayed in Figure 6.

Figure 6: Consistency among respondents, defined as the pair-wise equality of
all combinations of answers (1.0 in case all respondents answer the same, 0.0
when all different, although that’s generally due to the nature of the
questions). Consistency is averaged over the three follow-up questions
(regarding position, surface shading and shape type) and excludes answers of
the respondents that indicated the shape is not displayed.

The least consistency among the IFC2X3 results is seen in D4, C1, C5 (various
profiles with invalid extrusion direction). Naively one would expect
consistent results here given that this is a violation of a schema constraint,
but this shows the state of implementation in the industry that support for
these constraints in parser libraries cannot be assumed, especially not the
complicated ones as in this case (the IfcDotProduct function). D3, D5, D2 are
also inconsistent which stems from the question on whether fillet radii are
incorporated and whether the surfaces are shaded smooth or faceted.
Interesting to highlight is that there is quite some consistency in F4 and F5
as virtually no application took the StartParam and EndParam of
IfcSweptDiskSolid into account.

###  4.3 Using IFC data

BIM tools are usually very complex and making a very detailed report about
them is difficult. However, the general functionalities regarding
visualization, editing, query, analysis and export possibilities are tested
(Table 5). All the software offer visualisation functions, mainly only 3D,
usually also 2D within the BIM and 3D modelling software groups. Those same
two groups are the ones mainly allowing editing of geometry or attributes.
Usually, query functionalities are partial, within most of the tools323232
They are foreseen within GIS software; 3D viewers usually have partial query
abilities, for example BIM Vision provides general pre-defined ways to view
the information, listed by structure, by types, by groups (system, zones or
other) and by layers. For each, properties, location, classification and
relations can be shown at the object level; also FZK Viewer offers mainly pre-
defined queries; Lexocad can only query based on object type (e.g. wall, slab,
wall standard, …); in Autodesk civil 3D, as well as in Allplan and AutoCAD
Architecture it is only possible to select properties which are in the native
software format; in Vectorworks, query tools are present, however query is
possible if the project is connected to a database; object selection and
information retrieval is possible; finally, ArchiCAD presents a “find-and-
select” tool, but issues are reported. .

Looking at analysis, very few software packages allow it: only 20% for the
analysis concerning the model itself, Type 1, and 30% analysing the designed
building performances, Type 2, which is very little for something which is
supposed to be an operational tool. Since many different kinds of analysis can
be within the two categories (Type 1 and 2), and the tested tools offered very
various features, only the general support for the two types are reported in
Table 5. In some cases, the support is partial, for example, some Type 1
analysis are sometimes foreseen but not working, as it is the case of Solibri
Anywhere, or only in specific cases, like Vectorworks, working only with
NURBS. With the Type 2 analysis the pattern is similar, with some tools giving
issues, for example, in Allplan, Revit and ArchiCAD, the Type 2 analysis
sometimes do not work with the datasets, or with IFC files in general,
although available. It is quite difficult to give an exhaustive overview of
this, though.

80% of the software can export to IFC and only 30% of them allow some
customization about the IFC version and Model View Definition to be used.
Additional notes and warnings were reported to the export operations,
regarding the possibility to customize the mapping of entities, in some cases,
and a few errors about the way geometry and other features could result from
the export.333333 in FME, there were a number of warnings related to
inappropriate geometry types; in FreeCAD, the default settings yielded an
error regarding colours (to be checked in later versions and reported for
fixing if necessary). By changing the settings to use the IfcOpenShell
serializer, the error did not appear; for ACCA PriMus, it is possible to add
additional properties; in Revit it is possible to customize the export by
means of manual entity mapping; similarly in Vectorworks, a layer mapping pre-
process was needed to export the file. Each layer had to be selected
separately and assigned to the correct “vector story name”.

No sharp connection between the IFC certification and the software support and
functionalities is verifiable.

###  4.4 Writing of IFC2x3 files: analysis of the exported Myran.ifc models

The models which were exported from the tested pieces of software (Noardo et
al., 2020a)343434https://doi.org/10.5281/zenodo.3964368 were analysed by means
of the NIST IFC analyser, in order to formally check them. The tool counts the
number of entities, relationships and properties and summarizes it in reports.
From comparing the summaries, counting the features in the exported files,
with the ones regarding the original datasets, we obtain general information
about the way in which the software modified the models and we can point out
which are the entities that tools have limited support for.

Some of the delivered Myran.ifc models were re-exported by the tools without
errors, but apparently the export failed, since the building is not
represented anymore: this is the case of one of the ArchiCAD tests, where only
the signboard of the building is stored; the test with Tekla structure, where
only one beam is there; and one of the models exported by Revit, which is
empty. In addition, the models exported by FZKViewer could not be read by BIM
Vision nor NIST IFC analyser to be analysed and inspected, and RDF IfcViewer
only shows the IfcSite geometry, as terrain, and other two random (wrong)
objects in one of the FZKViewer-exported cases, and a completely wrong
building in the other one (Figure 7).

In the models shown in Figure 7, issues in the exported models mainly regard a
change in the model properties, loss of elements, change in the grouping of
entities and some occasional apparent change in geometry.353535 The model
exported by FreeCAD gives some problems when trying to load it into viewers:
RDF IfcViewer can show it only very far and small (probably some change in the
coordinate reference system happened), and it can be hardly handled in such
software; BIM Vision is not able to visualise it; Revit gives a warning
(“IFC:only 2 points in polyloop #702627, expected ¿=3”), but then shows it
quite correctly. There, it is possible to see that some elements are missing
(e.g. the beams in the exterior stairs). In Allplan, the difference that could
be noticed with respect to the original is the lack of the external stair and,
in the grouping, the IfcSite is stored separately than the three other
storeys, which is actually a better storage, but anyway different from the
original benchmark-provided file. In AutoCAD Architecture, the three storeys
are collapsed in a single one, big parts of objects belonging to the side
building are missing (e.g. walls, doors), the roofs’ colour changed for some
reason, probably due to the loss of some properties, and some of the walls
that were cut at the roof level, are not, so that they protrude above the
roof. In AutoCAD Civil, the three storeys are collapsed in only one, the
terrain model constituting the IfcSite is split in its different components
(parking places, street, different parts of the terrain), which are
recognizable by a human eye, but were not labelled differently in the original
dataset. All of these are represented by means of IfcBuildingElementProxy
entities. Also in this case, the roofs and the walls are changed in the same
way as in the AutoCAD Architecture model. Moreover, one of the walls has a
part protruding towards the exterior in a clear error.In the model exported by
Archline XP, the windows and doors seem to be disappeared, even if they rest
below the most exterior layers; probably the reason was some issues in the use
of IfcOpenings to subtract the volumes corresponding to doors and windows in
all the concerned walls and coverings. The roof slab is also missing.

Figure 7: Exported Myran models.

The two Revit ones, the two ArchiCAD ones, the ACCA PriMus one, the ACCA
usBIM.viewer one (only showing something weird, similar to a duplication in
the wireframe representing the site), in eveBIM, Simplebim, FME and Bricscad
look similar to the original dataset, and one could assume they are
consistent.

However, a closer look into the kind and number of entities included in the
models, by means of the NIST IFC analyser analysis (Table 7), shows they are
actually not.

Table 7: Differences in the NIST analysis results between the Myran models
exported by the tools and the provided one.

The only two tools leaving the model completely unchanged (same number of each
feature) are: ACCA Primus, which is a software supporting calculation of
materials for costs assessment, and ACCA usBIM.viewer. The only IFC entities
which are left unchanged across all the models are IfcUnitAssignment and
IfcBuilding (which is 1 for all the models).

The section regarding some general information, from IfcApplication to IfcSite
(blue-green), is among the most consistent ones. It includes subcategories of
IfcSpatialStructures information, together with some meta-data level
information. Apparently, it is quite easy for the software to keep track of
such entities, interpreting and writing them back in a correct way: the number
of IfcBuildings remains the same (one). Moreover, for example, eveBIM and
Simplebim add one IfcApplication to the file, probably intended to document
that the file passed also through such tools. Therefore it is not a mistake in
the writing of the file, but a wise update of the file metadata. The same
applies for IfcOwnerHistory, IfcOrganization, IfcPerson,
IfcPersonAndOrganization. However, the syntax used to add the information in
these two files is not the same. Although both are not wrong, this flexibility
in the writing of the new file would for sure be a difficult condition for the
interpretation of the file by third applications.

In the section representing entities concerning quantities (dark yellow),
which are not present in the original file, almost all the models remain the
same, except for two of the ArchiCAD tests (which report different numbers,
though). Also the units representation (orange) is kept quite consistent, and
the inconsistencies could be due to the different software settings for export
(both for the original file and for the tested ones), not necessarily meaning
inconsistency in the model. IfcDerviedUnit and IfcDerivedUnitElement entities
are the least kept ones by the re-exported models, which lose them completely
in 7 out of 16 cases.

The most interesting part of this analysis is however concerning the IFC
entities contained in the ‘Shared Building Elements’ and ‘Product extension’
parts of the IFC structure (white).

Models exported by the same software by different participants (e.g. the two
coming from Revit, the two coming from ArchiCAD) present very similar results
in the number of entities, although not exactly the same even. The same
happens with AutoCAD Architecture and AutoCAD civil, which we can suppose
using similar algorithms.

Moreover, it is not possible to find any balance in the lost/gained entities
considering neither the groups of entities being subclasses and superclasses,
or that can be considered as alternative representation of the same object
(e.g. IfcWall, IfcWallStandardCase, IfcWallType or IfcStair, IfcStairFlight,
IfcStairFlightType), nor the specific entities and the IfcBuildingElementProxy
and IfcBuildingElementProxyType.

It is also curious how some additional elements appear even if the similar
ones are kept consistently. For example, in the case of ‘stairs’ entities, in
both ArchiCAD tests and in Bricscad, although the IfcStairFlight entities are
still in the same number, other 4 IfcStairFlightType objects (same number of
occurrences of IfcStairFlights) are added (only three in one of such cases,
for some reason). In the models exported by Revit, 19 more IfcStair entities
are added.

Similar reasoning is possible when considering IfcWall, IfcWallStandardCase
and IfcWallType, although in this case in (only) one of the ArchiCAD models
and in the Allplan case the balance is kept, at least, with
IfcWallStandardCases becoming IfcWalls in the ArchiCAD case and IfcWallType
becoming IfcWalls in Allplan. Such change in the choice of an entity whose
meaning is similar, although losing the optimization advantages for which
different entities were proposed in the data model, would keep the model
semantically consistent. However, it is not possible to verify a consistent
behavior with respect to such choice in software.

The models exported by FreeCAD lose parts of object (as we could see from the
visual inspection, for example, part of the external stairs are no more
represented), as well as Allplan (e.g. again the external stair). Moreover,
Revit-exported files, FreeCAD, Allplan, AutoCAD Architecture, AutoCAD Civil
lose all the IfcMemberType entities.

But looking at the final balance in the number of entities, we can see that
their number raises. Therefore, assuming that no objects or parts of the model
were not exported at all, we can just report on a major number of entities
describing the building elements, that means that probably some of the objects
were split into several ones. The increase in the amount of entities is also
confirmed by the number of geometric objects, whose numbers vary
substantially, losing many instances of several geometric entities, while some
of them are more than doubled (especially the most simple ones, such as
IfcCartesianPoint, IfcFace or IfcPolyLoop). Again, a pattern is difficult to
be found. Similarly, for the parts concerning the other groups of entities, it
is difficult to find similarities across the software or the same entities.

The increase in the number of entities is also reflected in the file sizes,
that change from the original 27 MB to a maximum 94 MB for both the Revit
models.

In addition, the size decreases to 24 MB in the models exported by eveBIM,
SimpleBIM, Bricscad. This could mean that either those ones are better
optimized than the original dataset or that something went lost in the
conversions. In both cases it does not depict the best situation for
interoperability, even if it maybe not too bad per se.

###  4.5 Writing of IFC4 files: analysis of the exported Savigliano.ifc models

There are similar considerations for the exported Savigliano.ifc models as for
the Myran.ifc model.

The models exported by BIMserver, eveBIM and SimpleBIM look good. Again, the
models exported by FZKViewers can’t be opened in BIM Vision, nor it is shown
in the RDF IfcViewer or analysed by the NIST IFC analyzer.

In the other models, the loss of elements and properties was the main issue,
besides a few changes in some geometries363636 In the model exported by
Allplan, the upper part representing the roofs is missing, and all the windows
as well. Moreover some of the external walls and some slabs are coloured in a
different way (blue), but it is not very clear what is changed. The model
exported by AutoCAD Architecturelooks very similar to this last one: the top
part of the building is almost completely missing, even a bigger part than in
the Allplan model. Moreover, the windows are missing and, in this case, the
openings in the walls, where the windows and doors would be supposed to be
inserted, are missing too, presenting plain filled walls. Even the ramp
leading to the underground parking is filled and appears as the continuation
of the site. Moreover, other elements are missing, such as the windows in the
stair towers, closing elements. In addition, the colour of some elements,
especially slabs are missing, so that probably some property was lost. The
AutoCAD civil one is very similar to the AutoCAD Architecture one, but in
addition, a slab and a door are completely displaced and the site colour
changes. The ramp to access the underground parking is however well
represented. One possible explanation is that, being the software intended to
represent civil works, those ramps can be better interpreted with respect to
the same in AutoCAD Architecture, although probably the two tools being based
on similar algorithms. The model exported by ArchiCAD also misses the top part
and windows. Moreover, the site around the building is missing.An apparent
error is a sort of beam (classified however as wall) that goes from one of the
ground slabs into the ramp going to the parking (in green in Figure 8).
Another model, exported by Vectorworks, is very similar, although without the
beam invading the ramp.(Figure 8).

Figure 8: The exported Savigliano models.

When looking at the NIST analysis, it is again possible to notice several more
inconsistencies, except for the model exported by BIMServer, which leaves the
model exactly unchanged.

Table 8: Differences in the NIST analysis results between the Savigliano.ifc
models exported by the tools and the provided one.

Simplebim comes second, considering that all the others perform very bad. In
this case there are no differences in the number of building elements, and
very little difference among the geometric entities, where many IfcDirections,
25% of IfcCartesianPoints and some other ones are lost. In this case the main
differences are found in properties, materials and relationships.

As in the case of Myran.ifc, it is very difficult to find patterns in the
loss/gain of entities. In the model exported by Bricscad it is possible to see
718 IfcWalls lost and 718 IfcWallStandardCases gained, which were probably
resulting from the conversion of these ones. The numbers do not correspond
perfectly, but similar behaviour can be seen also in ArchiCAD and Allplan,
while AutoCAD Architecture loses 35 IfcWalls and gain 23 IfcWallTypes. The
most general entity being IfcWall, among them, this is a weird result though,
since the opposite direction of the conversion would be expected to be the
safest one for the sake of consistency.

Very little other patterns can be found. In this case many entities are lost,
contrary to the Myran.ifc case, maybe due to the little support for some IFC4
new entities. We could also see from the visual inspection that many objects
were lost and this is even clearer by the entities count, where we can see
that many are lost completely. The tendencies are generally not
straightforward, even if it is possible to notice that the added entities are
mainly IfcPropertySingleValue, IfcFace, IfcPolyLoop, but the behaviour varies
case to case, and it is necessary to look at the Table 8 to check the details.

###  4.6 Writing of IFC geometries: analysis of the exported IFCgeometries.ifc
and IFC4geometries.ifc models

The differences in the NIST analysis results between the IFCgeometries.ifc
models exported by the tools and the provided one is shown in Table 9 and
those for IFC4geometries.ifc in Table 10. The information in the two tables
indicates that there is no relevant difference between the test with the
dataset in IFC v.2x3 and the one in IFC v.4. In the case of these datasets,
the original files consisted only of geometries, with only minimal semantic
information. Therefore, when it is present in the exported files (e.g.
Materials, Styles and property sets), it means that the software which
exported them had attached such information based on its default settings.

Table 9: Differences in the NIST analysis results between the
IFCgeometries.ifc models exported by the tools and the provided one.

Table 10: Differences in the NIST analysis results between the
IFC4geometries.ifc models exported by the tools and the provided one.

Few tools are able to re-export the data leaving them completely consistent
with the imported ones, namely: ACCA Primus, ACCA usBIM.viewer and ArchiCAD.
eveBIM can also be included in the list of the good ones, since some elements
are added regarding IfcPropertySets and metadata, but no elements are lost.
Simplebim follows, with only few entities missing.

Many geometries resulting from parametric modelling are completely lost, as it
is possible to see in correspondence of the lines in light green in Table 9
and Table 10 (e.g. IfcRevolvedAreaSolid, IfcCircleProfileDef,
IfcIShapeProfileDef, IfcSweptDiskSolid) as well as some of the entities which
are in the other green rows in the tables, such as IfcBooleanResults. Instead,
it is possible to observe that the number of other more generic entities
increases a lot, e.g.: IfcCartesianPoint, IfcFace, IfcFaceOuterBound,
IfcPolyLoop, and so on. Apparently, the parametric geometries tend to be
simplified in the export, by most of the software, in favour of a more
explicit representation. It is also visible in the analysis of the other
exported models, but in these datasets it appears very clearly.

###  4.7 Software performances with IFC

A total of 43 different reports were returned, for 33 different software
packages. In particular, multiple results were submitted for FreeCAD (2 sets),
FZKViewer (2 sets), Autodesk Revit (6 sets, multiple versions), Vectorworks
Designer (2 sets), Archicad (3 sets). These offer the opportunity for timing
comparisons to investigate the impact of hardware on software performance. The
graph in Figure 9 gives a summary of the success rates returned for the tests
on the three datasets. The graph in Figure 10 gives the count of the different
timing values for the successful tests.

Figure 9: Graph reporting the number of tools associated to their success rate
in the indicated tasks. Figure 10: Graph reporting the number of tools
associated to their timing results in the indicated tasks.

Note that in some cases users reported results for some of the tests but not
for all of them (“No result reported”). Additionally, some users typed in
comments such as “no error” instead of giving specific timing. These are
included in the “No result reported” count.

For the Myran.ifc dataset, none of the tasks took more than 5 minutes to
execute, and the large majority of the software packages tested could zoom and
pan the data immediately. Data loading times of less than a minute were also
dominant (65% in total). 28 out of the 43 participants reported successful IFC
export, with 86% of these exports being executed in less than a minute.

The impact of the more complex dataset (adopting IFC v.4) on the tests is
visible in the results, with only 58% of the software packages able to carry
out the basic timed test that involved opening the dataset in the software,
and 35% reporting that the software crashed. For the Savigliano.ifc dataset,
in contrast to the Myran.ifc dataset, 9 of the successful attempts to open the
data (36%) took 5 or more minutes to execute, although once opened zooming and
panning the data was immediate, and the large majority of the software
packages tested could zoom and pan the data immediately. Data loading times of
less than a minute were also dominant (65% in total). 16 out of the 43
participants reported successful IFC export, with all of these taking less
than 5 minutes to execute.

The results with the UpTown.ifc data demonstrate clearly the impact of the
larger dataset on the tests carried out, with only 47% of the software
packages able to carry out the basic timed test that involved opening the
dataset in the software, and 41% reporting that the software crashed. In
addition, in contrast to the Myran.ifc dataset, 9 of the 20 successful
attempts to open the data (55%) took 5 or more minutes to execute, with 2 of
the participants reporting test times of between 20 minutes and 1 hour, and 3
of over one hour. Interestingly, once open, the vast majority of the testers
reported sub-minute execution times for zooming and panning. Only 15 out of
the 43 participants reported successful IFC export, with more than 50% of them
taking over 20 minutes.

####  4.7.1 Multiple Tests on Same Software Packages

As noted above, the crowdsourced approach taken in this project resulted in
multiple participants testing the same software, providing the opportunity for
comparison. For both FreeCAD and FZK viewer, one out of the two respondents
reported that they were unable to open the Myran.ifc dataset. Vectorworks
results are similarly consistent, with the only minor difference being that
one participant reported that the Myran.ifc data opened immediately and
another that it took less than one minute (both on machines with 16GB RAM).

Identical timing results are reported for the zoom/pan/rotate/query tasks for
the remaining three packages (Autodesk, ArchiCAD), and as noted above, the
large majority of software packages managed these functions in near immediate
time. As different analytical tasks were selected by participants these are
not comparable.

Differences do, however, arise when execution time for the remaining
functionality is considered. Table 11 summarises the remaining timing results
for these packages for the Myran.ifc dataset, which was chosen as this proved
the least problematic of the three datasets in terms of successful execution
of tasks.

|  Import |  Inspect Linked Objects |  Export to IFC
---|---|---|---
Autodesk Revit 2019 — Result 1 |  1–5 minutes |  it’s almost immediate |  it’s almost immediate
Autodesk Revit 2018 — Result 2 |  1–5 minutes |  it’s almost immediate |  1–5 minutes
Autodesk Revit 2018 — Result 3 |  1–5 minutes |  the software does not allow this |  1–5 minutes
Autodesk Revit 2018 — Result 4 |  1–5 minutes |  the software does not allow this |  crashes
Autodesk Revit 2019.2 — Result 5 |  1–5 minutes |  the software does not allow this |  it’s almost immediate
Autodesk Revit 2020 — Result 6 |  less than a minute |  less than a minute |  it’s almost immediate
ArchiCAD — Result 1 |  less than a minute |  the software does not allow this |  less than a minute
ArchiCAD — Result 2 |  less than a minute |  the software does not allow this |  less than a minute
ArchiCAD — Result 3 |  1–5 minutes |  it’s almost immediate |  less than a minute
Table 11: Inconsistent timing results that were submitted for the Myran.ifc
dataset

For Revit, there is an apparent performance improvement when opening Myran.ifc
data in the 2020 version of the software, and an improvement of export time
from 1–5 minutes in the 2018 version, through to “almost immediate” in the
2019 and 2020 versions of the software. In terms of hardware, all tests were
carried out on Windows 10 machines. However, the machine used for Revit 2020
had 64GB of RAM, with Result 1 obtained with 16GB RAM, and Result 2, 4 and 5
with 8GB RAM. Contradictory results in the linked object query make these
difficult to compare. For ArchiCAD, all users report an IFC export time for
Myran.ifc of less than a minute, but two report that it was not possible to
query linked information. Examining the hardware used, Result 3 for ArchiCAD
–– file opening time of 1–5 minutes –– was obtained on a machine with 8GB RAM,
whereas the others were tested on machines with 16GB RAM (all Windows 10
machines, all with dedicated graphics cards).

##  5 Discussion

What is apparent while analysing the results of the exported files, is that it
is very difficult to find a pattern or a sensible explanation about what
happened in the export phase, since there are very few cases in which the IFC
entities are completely kept / lost / gained in all the exported models. The
most frequent scheme is: some models keep all — some models lose all — some
models keep part — some entities are added. The very high complexity of both
the semantic data model and the geometry management is for sure related to the
issue.

Similarly, the investigation about loading IFC files suggests a general
difficulty in reading semantics correctly, very little support for
georeferencing and quite good support for general geometry, as visualized,
which is not unambiguously read though, when looking at the IFC geometries
datasets results. But no clear pattern is found, that could make it possible
to better understand the specific issues of IFC interoperability. A clear
result is just that little interoperability is actually reached, since there
are very few tools able to read the standardised datasets correctly and even
fewer that are able to export them consistently. The ability to uniquely
interpret the models and to leave them consistent through the import-export
phases is absolutely essential for interoperability and what it enables (data
exchange, data re-use and so on). At this stage it is not possible to fully
trust standardised models though, even just for simple file exchange.

Results were reported for 33 software packages, highlighting the wide range of
tools available to users. Different versions of Revit also demonstrated that
Autodesk is making improvements to the interoperability of the software —
Revit 2019 gave results for all data, but Revit 2018 could not handle two of
the three datasets (Uptown.ifc and Savigliano.ifc). Myran.ifc could be opened
by 40 participants, Savigliano.ifc by 25 and Uptown.ifc by 20. The twelve
participants who reported that they were not able to open Uptown.ifc also
reported the same issue with Savigliano.ifc. Ten participants reported
problems with Uptown.ifc but none with Savigliano.ifc and 4 with
Savigliano.ifc but not with Uptown.ifc, indicating that the size of the
dataset is, overall, more problematic than the IFC version.

Regarding performance, the impact of the size of the dataset on overall
performance is marked – with 55% of the software packages that managed to open
the Uptown.ifc dataset reporting a time of 5 minutes or more. While it is not
possible to say which software package is the fastest –– the approach to
timing used general timing categories rather than requiring the user to
undertake the onerous task of time measurement — and performance will also
depend both on the hardware being used, we can report that none of the
software packages managed to carry out all the visualisation tasks in under a
minute for the Uptown.ifc dataset, although 9 packages achieved this with the
Myran.ifc dataset and 9 with the Savigliano.ifc dataset. For IFC export, 24
software packages managed to export the data in 1 minute or less for
Myran.ifc, 14 for Savigliano.ifc but only two for Uptown.ifc.

The set of available functionalities of software to work with IFC models is
moreover quite limited. The analysis about the model itself (Type 1) are
essential to make sure a model is valid and suitable to support further
analysis. On the other hand, tools are necessary to use IFC models for the
operational intentions it was conceived for. However, not many tools are now
able to work with IFC effectively.

The fact that the inquiry is based on voluntary and completely open
contribution can be considered both as a strong point and a limitation of this
work. On the one hand, it is essential to cover the investigated object in the
most thorough way: as many software packages as possible, with many experts
involved, but also with the inclusion of less expert users to test also user-
friendliness. However, one outcome of this approach is some incompleteness in
the tools that are assessed373737e.g. the open source xBIM
(https://docs.xbim.net/index.html), the very recent BlenderBIM
(https://blenderbim.org/) and some of the niche engineering tools were not
tested.. This one was limited by the ex-post integration of tests about the
software not originally considered. Moreover, the tests reporting suspicious
results according to the promoting team experiences (for both too good or too
bad performances), were double-checked with new tests or by asking for
clarifications. A further issue could be the inexperience of some testers,
reporting about tools behaviour in an inaccurate way. To lower this
eventuality, it was checked that all the delivered answers were described with
sufficient care, whatever the level of expertise could be. Once verified this,
eventual conflicting answers with respect to the tools actual potential, could
indicate a deficiency in the suitability of the tool to be used by any
inexpert user (which would be anyhow necessary for the models to be used in
practice).

The involvement of a large part of the community (twice the number of
participants delivering the results initially registered to the initiative) is
also important to perceive the relevance of the topic, although dealing with a
somehow hidden issue among the high-level standardization and academic
communities.

###  5.1 Lessons learnt and consequent suggestions

The insight and data gained with this study can be the base for reasoning
about improvements to the IFC model, the common behaviour of the models and
the most urgent issues to be tackled by software, for example, the suitable
implementation of georeferencing.

Control in export and import would be critical to make the choice explicit
about the used possible alternative to store information in IFC. Although
possible ambiguities and equivalent alternatives should be avoided as much as
possible, in some cases, applications could require information to be stored
according to specific structures, considering the slight differences between
them (as an example, see the difference between LoGeoRef30 and LoGeoRef40, or
the generalization level of semantics required by applications).

Such control, for georeferencing would consist in the choice about the
LoGeoRef to be read, when alternative ones are stored (including LoGeoRef50
enabled by IFC4). It could be both a choice, to be selected during import, and
an instruction, as a requirement for the model to be input. A reasoning could
be made about the opportunity of allowing the reading of N-E, H and rotation
separately, according to different LoGeoRefs.

The georeferencing issue is even more articulated. First, regardless of the
country and its level of advancement in implementing the BIM idea in ongoing
projects, the little use by designers and architects of BIM models with the
right georeference is similar. Almost all designers expect a model in their
local coordinate system, despite the fact that the point cloud obtained for
this purpose has the correct coordinates, consistent with the national system.
To make it worse, in most cases, “long” coordinates are truncated due to
software problems with “long” coordinates. Therefore, in addition to
developing effective methods for storing georeferencing in IFC files, it seems
necessary to provide education to designers and architects and a clear
guideline about how to deal with it.

With respect to semantics, probably a smaller set of entities, to be used
according to stricter rules would help in implementing a suitable
interpretation in software. At least BIM software, supposed to have the most
general scope, should try to implement internal structures of entities/
families in the native format in agreement with the IFC data model, as much as
possible. This would be not supposed to be a top-down directive from
buildingSMART, but probably an agreement coming from the collaboration of
software developers with standardization institution, starting from the needs
of practitioners and users.

If such a stricter collaboration would be implemented, probably also the IFC
model could have an indication for selecting its most important core part to
be maintained, improved and likely simplified or made less ambiguous. The
parts of the model more specific of different application domains could be
cared together with the specific expert domains-practitioners-developers.

Control and transparency would be the second requirements as well. It would be
an added value in the case of export: How will the software export the data?
For example, a software-specific but explicit MVD could describe exactly how
the internal model of the software is mapped on which part of the IFC model.
And it would help the import phase as well: how will be the IFC classes
interpreted? Which ones are supposed to be interpreted exactly, since part of
the software data model? Which ones could instead be misinterpreted and how:
maybe a choice could be provided to map the un-recognized classes, or they
could be represented as their parent class, or as a generic class. This should
be a straightforward, guided and clear process instead of an advanced option
of the software likely supposed to be used mainly by experts.

Since it is not possible to detect any pattern in the behaviour of software,
the issue is more identifiable in such indefiniteness than in one specific
aspect of the standard or of the software.

One unique way of writing the IFC files should also be imposed, to avoid
redundancy or ambiguity in the storage of information. Semantics would have
advantages from simpler and clearer rules about how to structure entities,
priorities and limits. For implementations, and therefore practice, it is for
sure preferable to rely on a simpler model than a very complex and expressive
but inconsistent one.

Finally, reducing the set of geometries in the schema, together with clear
constraints on how and when to use these geometries would enable software
developers to implement them more robustly, avoiding loss of information,
intersections and so on. As is the aim already in future versions of IFC, this
should also be made more use-case dependent. When an export is intended to
function as a static reference for visualization, coordination or inference
checking, procedural and parametric representation should be largely avoided.
When transfer of design intent and parametric behavior is required the more
procedural geometry definitions can be used. In addition, the non-compliances
possibly implemented by some software, should at least give an error message
and make the user explicitly choose if using a non-compliant IFC file anyway,
to the detriment of interoperability. This should be part of the IFC
certification process too. One potential avenue to detect flaws in the
geometric interpretation is to serialize a set of descriptive properties (such
as center of mass, surface area, volume, surface genus) along with the
geometry during export that the importing application can calculate from its
reinterpreted geometry representation and compare, so called geometric
validation properties from Pratt et al. (2005) in the case of STEP.

Again, implementing some control on the export of geometries by modelling
software would help in making the (likely few) possible choices explicit,
fostering re-usability and interoperability. The storage of such choices in
metadata would also be useful.

As a general rule, the use of default settings while importing or exporting
files to overcome the presence of undefined information should always be
avoided, since it is source of many misleading inconsistencies (e.g. inferring
CRS).

In some case, it could be possible to object that the IFC aim is different
from some of the aspects tested here. For example, someone could say that
editing is not a priority or not even a planned possibility for IFC files,
that should only be exchanged as reference: e.g. for different professionals
designing the same building, for which semantics are barely needed; only for
analysis (asset and facility management, project management, project analysis,
computations and so on), where semantics and quality do matter. If this is the
case, the purpose of IFC itself should be clearly stated and maybe reduced.

##  6 Conclusions

The described study in this paper, part of the GeoBIM benchmark project, was
intended to point out and provide evidence about the support for and issues of
available software for standardised information in IFC. Interoperability is
essential for a number of use cases, and even for simply exchanging and re-
using data. Standards are supposed to be enabling such interoperability, and
standardization is the essential premise to the development of any
integration, including the GeoBIM one.

The Industry Foundation Classes, by buildingSMART, is the official reference
open standard for BIM management and storage. However, we have personally
stumbled across a number of issues while using IFC, and many more have been
often informally reported to us by practitioners. These prevent the effective
use of IFC in practice, without the issues being clear to users, who usually
rely on proprietary formats (such as the Revit native format), using them as
often more reliable de-facto standards. Moreover, although we were aware that
some data or functionality loss was prevalent when using IFC, no systematic
proof was available in the literature. This project aimed to create such
proof, which could be used as the base of future improvements in
implementations, in data modelling and in the standard itself.

The results show how difficult it is for software tools to read, manage and
export the IFC datasets consistently, in a manner that woud take full
disadvantage of interoperability, and that would enable the effective
usability of datasets in use cases too.

Besides the internal interoperability of IFC in BIM software, the potential
integration with 3D geoinformation and respective standards is additionally
hindered. It is clear that if even IFC software cannot consistently read and
write IFC, multi-format software will do even worse, especially when dealing
with the areas where both domains diverge (e.g. georeferencing and semantics).
In fact, it is almost impossible to plan, design and implement effective
solutions for mapping, conversions and object transformation to other formats
without relying on a stable and trustworthy initial format.

We are aware that there is a possible bias in this study’s results, which
could be produced by the lack of expertise of some participants, or by the
initial inaccuracies in the provided datasets. However, such a bias would also
reflect the way in which users use software and product data in practice.
Nevertheless, in order to minimise potential bias, the datasets were validated
and improved as much as possible for the purpose of the benchmark, in order to
limit their influence on the quality and reliability of results. Therefore, if
there are still problems in the datasets, it would likely reflect additional
drawbacks of the standard itself, for the little clarity about its use for the
modelling of actual datasets and the difficulty in implementation, which could
produce little intuitive tools.

This study shows the drawbacks of the current IFC standard and implementation,
and the related difficulties, also due to the challenges to which it is
intended to respond (e.g. representation of the information regarding a vast
and complex field, flexibility to multiple needs). A particularly important
point is that the outcomes of the study are officially documented here, in a
manner that they can be the base for future research in the field and
development of concrete solutions, such as the addition of constraints and
specific guidelines, simpler ways to store geometry, better selection of
useful semantics, and so on.

Giving clear guidance towards changes in the IFC schema is out of scope in the
present paper. However, it is clear that from the point of view of software
support, more consistent implementations of the existing schema are needed.
There are many possibilities that could help in this regard, e.g. the creation
of high quality open source libraries to read/write IFC, the creation of
reference implementations, and the writing of more extensive documentation
that shows how the IFC data model is mapped to software’s internal data
models. All of these would enable the reuse of source code and the definition
of best practices.

Considering the results of this study as evidence, future work should be aimed
at filling in gaps in the results (e.g. through the study of additional
software or new reference datasets), updating the results with respect to new
software, and the solution of the outlined issues.

For example, a clear area of opportunity is the definition of clear guidelines
about the use of complex semantics, fixing priorities and criteria for their
use. Moreover, specific kinds of geometries should be selected for specific
cases, adding constraints to guarantee that software can import, read, use and
re-export them without any change. In addition, when considering the
performance related to the computational requirements, we can easily
understand that the reduction of data size is urgent.

buildingSMART promisingly began working towards solving such issues, trying to
work for less complex models offering more straightforward choices, easier to
implement (see the “Ten principles for the future IFC” by
buildingSMART383838https://github.com/buildingSMART/NextGen-IFC/wiki/Ten-
principles-for-a-future-IFC) and in the work towards IFC v.5. Thanks to this
study, it was possible to gain a higher awareness of the specific issues to be
tackled in future research in order to foster an improved adoption of the IFC
standard.

##  7 ACKNOWLEDGEMENTS

This work was possible thanks to the collaboration of the whole GeoBIM
benchmark team (with their work as in-kind contribution to the project), all
the data providers and the participants making the tests, listed in the GeoBIM
benchmark website393939https://3d.bk.tudelft.nl/projects/geobim-
benchmark/participants.html.

The benchmark was funded by ISPRS and EuroSDR. This project has also received
funding from the European Research Council (ERC) under the European Union’s
Horizon 2020 Research & Innovation Programme (grant agreement no. 677312,
Urban modelling in higher dimensions) and Marie Skłodowska-Curie (grant
agreement No. 707404, Multisource Spatial data Integration for smart City
Applications).

## References

  * Aksamija [2018] Ajla Aksamija.  Methods for integrating parametric design with building performance analysis.  In _ARCC Conference Repository_ , 2018.
  * Aleksandrov et al. [2019] M Aleksandrov, A Diakité, J Yan, W Li, and S Zlatanova.  Systems architecture for management of BIM, 3D GIS and sensors data.  _ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences_, 4, 2019.
  * Arroyo Ohori et al. [2018] Ken Arroyo Ohori, Abdoulaye Diakité, Thomas Krijnen, Hugo Ledoux, and Jantien Stoter.  Processing BIM and GIS models in practice: Experiences and recommendations from a GeoBIM project in the Netherlands.  _ISPRS International Journal of Geo-Information_ , 7(8), aug 2018.
  * Azhar [2011] Salman Azhar.  Building information modeling (BIM): Trends, benefits, risks, and challenges for the AEC industry.  _Leadership and management in engineering_ , 11(3):241–252, 2011.
  * [5] AHA Baik, Reda Yaagoubi, and Jan Boehm.  integration of jeddah historical bim and 3d gis for documentation and restoration of historical monument.  International Society for Photogrammetry and Remote Sensing (ISPRS).
  * Barbini et al. [2019] Ambra Barbini, Giada Malacarne, Giovanna Massari, Gabriele Pasetti Monizza, and Dominik T Matt.  BIM objects library for information exchange in public works: The use of proprietary and open formats.  _WIT Transactions on The Built Environment_ , 192:269–280, 2019.
  * Bartie et al. [2010] Phil Bartie, Femke Reitsma, Simon Kingham, and Steven Mills.  Advancing visibility modelling algorithms for urban environments.  _Computers, Environment and Urban Systems_ , 34(6):518–531, 2010.
  * Biljecki et al. [2015] Filip Biljecki, Jantien Stoter, Hugo Ledoux, Sisi Zlatanova, and Arzu Çöltekin.  Applications of 3D city models: State of the art review.  _ISPRS International Journal of Geo-Information_ , 4(4):2842–2889, 2015.
  * Biljecki et al. [2018] Filip Biljecki, Kavisha Kumar, and Claus Nagel.  CityGML Application Domain Extension (ADE): overview of developments.  _Open Geospatial Data, Software and Standards_ , 2018.
  * Clemen and Görne [2019] Christian Clemen and Hendrik Görne.  Level of Georeferencing (LoGeoRef) using IFC for BIM.  _Journal of Geodesy_ , (10):15–20, 2019.
  * Egusquiza et al. [2018] A Egusquiza, JL Izkara, and A Gandini.  Energy efficiency improvement in historic urban environments: From decision support systems to co-creation strategies.  In _The 3rd International Conference on Energy Efficiency in Historic Buildings (EEHB2018), Visby, Sweden, September 26th to 27th, 2018._ , pages 576–584. Uppsala University, 2018.
  * Farid Mohajer and Aksamija [2019] Mahsa Farid Mohajer and Ajla Aksamija.  Integration of Building Energy Modeling (BEM) and Building Information Modeling (BIM): Workflows and Case Study.  _Building Technology Educator’s Society_ , 2019(1):37, 2019.
  * Fosu et al. [2015] Richelle Fosu, Kamal Suprabhas, Zenith Rathore, and Clark Cory.  Integration of Building Information Modeling (BIM) and Geographic Information Systems (GIS)–a literature review and future needs.  In _Proceedings of the 32nd CIB W78 Conference, Eindhoven, The Netherlands_ , pages 27–29, 2015.
  * Haddock [2018] Amber Haddock.  Building information modelling for asset and facilities management.  2018\.
  * ISO [2013] ISO.  _Industry Foundation Classes (IFC) for data sharing in the construction and facility management industries_.  International Organization for Standardization, March 2013.
  * ISO [2014] ISO.  _Industrial automation systems and integration - Product data representation and exchange_.  International Organization for Standardization, August 2014.
  * Jakubiec and Reinhart [2013] J. Alstan Jakubiec and Christoph F. Reinhart.  A method for predicting city-wide electricity gains from photovoltaic panels based on LiDAR and GIS data combined with hourly Daysim simulations.  _Solar Energy_ , 93:127–143, 2013.
  * Kamari et al. [2018] Aliakbar Kamari, Christian Laustsen, Steffen Petersen, and Poul Henning Kirkegaard.  A BIM-based decision support system for the evaluation of holistic renovation scenarios.  _Journal of Information Technology in Construction_ , 23:354–380, 2018.
  * Kang and Hong [2015] Tae Wook Kang and Chang Hee Hong.  A study on software architecture for effective BIM/GIS-based facility management data integration.  _Automation in construction_ , 54:25–38, 2015.
  * Kensek [2014] KM Kensek.  Integration of Environmental Sensors with BIM: case studies using Arduino, Dynamo, and the Revit API.  2014\.
  * Kumar et al. [2017] K. Kumar, H. Ledoux, T. J. F. Commandeur, and J. E. Stoter.  Modelling urban noise in CityGML ADE: Case of the Netherlands.  In _ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences_ , volume IV-4/W5, pages 73–81, 2017.
  * Kumar et al. [2019] Kavisha Kumar, Anna Labetski, Ken Arroyo Ohori, Hugo Ledoux, and Jantien Stoter.  The LandInfra standard and its role in solving the BIM-GIS quagmire.  _Open Geospatial Data, Software and Standards_ , 4(1):1–16, 2019.
  * Lamartina and Sa [2019] Livio Lamartina and Joao Sa.  Structural pre-analysis through implementing revit structures and robot to develop and compare engineered timber structural families.  _WIT Transactions on The Built Environment_ , 192:229–243, 2019.
  * Liang et al. [2014] Jianming Liang, Jianhua Gong, Wenhang Li, and Abdoul Nasser Ibrahim.  A visualization-oriented 3D method for efficient computation of urban solar radiation based on 3D–2D surface mapping.  _International Journal of Geographical Information Science_ , 28(4):780–798, 2014.
  * Lim et al. [2019] J Lim, H Tauscher, and F Biljecki.  Graph transformation rules for IFC-to-CityGML attribute conversion.  _ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci._ , IV-4/W8:83–90, 2019.
  * Liu et al. [2017] Xin Liu, Xiangyu Wang, Graeme Wright, Jack CP Cheng, Xiao Li, and Rui Liu.  A state-of-the-art review on the integration of Building Information Modeling (BIM) and Geographic Information System (GIS).  _ISPRS International Journal of Geo-Information_ , 6(2):53, 2017.
  * Lv [2018] Siqing Lv.  Study on BIM Modeling Method of Bridge Railway Integration Based on Revit and Civil3D.  In _2018 7th International Conference on Energy and Environmental Protection (ICEEP 2018)_. Atlantis Press, 2018.  ISBN 978-94-6252-558-0.  doi: https://doi.org/10.2991/iceep-18.2018.37.  URL https://doi.org/10.2991/iceep-18.2018.37.
  * Nguyen and Pearce [2012] Ha T. Nguyen and Joshua M. Pearce.  Incorporating shading losses in solar photovoltaic potential assessment at the municipal scale.  _Solar Energy_ , 86(5):1245–1260, 2012.
  * Niu et al. [2019] Sanyuan Niu, Yi Yang, and Wei Pan.  Logistics Planning and Visualization of Modular Integrated Construction Projects Based on BIM-GIS Integration and Vehicle Routing Algorithm.  _Modular and Offsite Construction (MOC) Summit Proceedings_ , pages 579–586, 2019.
  * [30] F. Noardo, C. C. Ellul, L. Harrie, I. Overland, M. Shariat, K. Arroyo Ohori, and J. Stoter.  opportunities and challenges for geobim in europe: developing a building permits use-case to raise awareness and examine technical interoperability challenges.  _Journal of Spatial Science_.  doi: 10.1080/14498596.2019.1627253.
  * Noardo et al. [2019a] F. Noardo, K. Arroyo Ohori, F. Biljecki, C. Ellul, L. Harrie, T. Krijnen, and J. Stoter.  Geobim benchmark 2019: Design and initial results.  In _ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences_ , volume 42, pages 1339–1346. ISPRS, 2019a.  doi: 10.5194/isprs-archives-XLII-2-W13-1339-2019.  URL https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W13/1339/2019/.
  * Noardo et al. [2019b] Francesca Noardo, Filip Biljecki, Giorgio Agugiaro, Ken Arroyo Ohori, Claire Ellul, Lars Harrie, and Jantien Stoter.  GeoBIM Benchmark 2019: Intermediate Results.  In _14th 3D GeoInfo Conference 2019_ , volume XLII-4 of _ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences_ , pages 47–52. ISPRS, 2019b.  doi: 10.5194/isprs-archives-XLII-4-W15-47-2019.
  * Noardo et al. [2020a] Francesca Noardo, Ken Arroyo Ohori, Filip Biljecki, Claire Ellul, Lars Harrie, Thomas Krijnen, and Jantien Stoter.  IFC files exported within the GeoBIM benchmark 2019 - Task 1, July 2020a.  URL https://doi.org/10.5281/zenodo.3964368.
  * Noardo et al. [2020b] Francesca Noardo, Ken Arroyo Ohori, Filip Biljecki, Claire Ellul, Lars Harrie, Thomas Krijnen, and Jantien Stoter.  Reports about the software behaviour delivered within the GeoBIM benchmark 2019 - Task 1, July 2020b.  URL https://doi.org/10.5281/zenodo.3964445.
  * Noardo et al. [2020c] Francesca Noardo, Lars Harrie, Ken Arroyo Ohori, Filip Biljecki, Claire Ellul, Thomas Krijnen, Helen Eriksson, Dogus Guler, Dean Hintz, Mojgan A Jadidi, Maria Pla, Santi Sanchez, Ville-Pekka Soini, Rudi Stouffs, Jernej Tekavec, and Jantien Stoter.  Tools for BIM-GIS Integration (IFC Georeferencing and Conversions): Results from the GeoBIM Benchmark 2019.  _ISPRS International Journal of Geo-Information_ , 9(9):502, 2020c.
  * Noardo et al. [2020d] Francesca Noardo, Ken Arroyo Ohori, Filip Biljecki, Claire Ellul, Lars Harrie, Thomas Krijnen, Helen Eriksson, Jordi van Liempt, Maria Pla, Antonio Ruiz, et al.  Reference study of CityGML software support: the GeoBIM benchmark 2019–Part II.  _arXiv preprint arXiv:2007.10938_ , 2020d.
  * Papadopoulos et al. [2017] Nicolas Alexandros Papadopoulos, Elisa Dominguez Sotelino, Luiz Fernando Martha, Daniel Luiz Mattos Nascimento, and Pedro Saieg Faria.  Evaluation of integration between a BIM platform and a tool for structural analysis.  _Sist. Gestão_ , 12:108–116, 2017.
  * Pazlar and Turk [2008] Tomaž Pazlar and Žiga Turk.  Interoperability in practice: geometric data exchance using the IFC standard.  _Journal of Information Technology in Construction (ITcon)_ , 13(24):362–380, 2008.
  * Peters et al. [2015] Ravi Peters, Hugo Ledoux, and Filip Biljecki.  Visibility analysis in a point cloud based on the Medial Axis Transform.  In _Eurographics Workshop on Urban Data Modelling and Visualisation 2015_ , pages 7–12, Delft, Netherlands, November 2015.
  * Petersen et al. [2018] Steffen Petersen, Andreas Bøving, Christine Nielsen, and Pil Brix Purup.  Integration of thermal simulations in revit for decision support and national building code compliance.  In _4th Building Simulation and Optimization Conference of International Building Performance Simulation Association_ , pages 407–413. International Building Performance Simulation Association, 2018.
  * Petri et al. [2017] Ioan Petri, Sylvain Kubicki, Yacine Rezgui, Annie Guerriero, and Haijiang Li.  Optimizing energy efficiency in operating built environment assets through building information modeling: a case study.  _Energies_ , 10(8):1167, 2017.
  * Pratt et al. [2005] Michael J. Pratt, Bill D. Anderson, and Tony Ranger.  Towards the standardized exchange of parameterized feature-based cad models.  _Computer-Aided Design_ , 37(12):1251 – 1265, 2005.  ISSN 0010-4485.  doi: 10.1016/j.cad.2004.12.005.
  * Stouffs et al. [2018] Rudi Stouffs, Helga Tauscher, and Filip Biljecki.  Achieving Complete and Near-Lossless Conversion from IFC to CityGML.  _ISPRS International Journal of Geo-Information_ , 7(9):355, 2018.