###### Abstract

_Smart Cities_ already surround us, and yet they are still incomprehensibly
far from directly impacting everyday life. While current Smart Cities are
often inaccessible, the experience of everyday citizens may be enhanced with a
combination of the emerging technologies Digital Twins (DTs) and Situated
Analytics. DTs represent their Physical Twin (PT) in the real world via
models, simulations, (remotely) sensed data, context awareness, and
interactions. However, interaction requires appropriate interfaces to address
the complexity of the city. Ultimately, leveraging the potential of Smart
Cities requires going beyond assembling the DT to be comprehensive and
accessible. Situated Analytics allows for the anchoring of city information in
its spatial context. We advance the concept of embedding the DT into the PT
through Situated Analytics to form Fused Twins (FTs). This fusion allows
access to data in the location that it is generated in an embodied context
that can make the data more understandable. Prototypes of FTs are rapidly
emerging from different domains, but Smart Cities represent the context with
the most potential for FTs in the future. This paper reviews DTs, Situated
Analytics, and Smart Cities as the foundations of FTs. Regarding DTs, we
define five components (Physical, Data, Analytical, Virtual, and Connection
environments) that we relate to several cognates (i.e., similar but different
terms) from existing literature. Regarding Situated Analytics, we review the
effects of user embodiment on cognition and cognitive load. Finally, we
classify existing partial examples of FTs from the literature and address
their construction from Augmented Reality, Geographic Information Systems,
Building/City Information Models, and DTs and provide an overview of future
directions.

###### keywords:

Digital Twin; Smart Cities; Internet of Things; Sensor Networks; Remote
Sensing; Fused Twins; Immersive Analytics; Embodiment; Visualisation

1 1 1 https://doi.org/10.48550/arXiv.2202.07104 The Hitchhiker’s Guide to
Fused Twins: A Review of Access to Digital Twins in situ in Smart Cities
Access to Digital Twins in situ in Smart Cities: A Review Jascha Grübel
1,2,3,4,5†*, Tyler Thrash6, Leonel Aguilar 1,7, Michal Gath-Morad 8,9, Julia
Chatain2,10, Robert W. Sumner2, Christoph Hölscher1 and Victor R. Schinazi11
Jascha Grübel, Tyler Thrash, Leonel Aguilar, Michal Gath-Morad, Julia Chatain,
Robert W. Sumner, Christoph Hölscher and Victor R. Schinazi Grübel, J.;
Thrash, T.;Aguilar, L.; Gath-Morad, M.; Chatain, J.; Sumner, R.W., Hölscher,
C.; Schinazi, V.R. Correspondence<EMAIL_ADDRESS>Current address: Visual
Computing Group, Harvard University

##  1 Introduction

a)
b)

Figure 1: Comparison of a Digital Twin and Fused Twins. In both rows, the same
lobby of a building is shown. a) the DT of a sensor system with 390 nodes is
visualised in Virtual Reality (VR) Grübel et al. (2021, 2016). The white areas
visualise the areas of observation and the coloured sphere encode the type of
sensor that is located at the respective position in the building. b) the FTs
Grübel et al. (2021) are visualised in Mixed Reality (MR) Speicher et al.
(2019) by embedding the DT _in situ_ in its PT thus fusing the two. The fused
representation enables Situated Analytics Thomas et al. (2018) that
facilitates the user understanding of analysis through spatial cues of the
environment.

Can Digital Twins (DTs)Grieves (2014); Grieves and Vickers (2017) really make
cities smart? The pressure to transform cities into _Smart_ Cities is mounting
amongst urban policymakers worldwide Deakin and Al Waer (2011). Initially, the
hope was that _smartification_ would be instilled in cities by simply creating
citywide Internet of Things (IoT) installations and remote sensing
applications Su et al. (2011); Hung (2017); Deng et al. (2021). The idea
behind this thinking is simple yet alluring: to have everything in the city
attached to the Internet and make it proclaim its own state as if it were a
smart object Sánchez López et al. (2012) remotely observing everything Su et
al. (2011); Deng et al. (2021). Once observed and connected, everything should
become measurable. In such a smart city, every minute detail, if measured, can
be included in the analysis. This would allow for the uncovering of new
insights Tao et al. (2018) and give stakeholders a deeper understanding of
processes that were previously thought to be fully understood.

However, the complexity of the collected data, the observed processes, and the
city itself requires a unifying view. A DT Batty (2018) provides a platform on
which cities can become really smart by shifting attention from data
collection to data use. Here, a digital representation of the city allows for
modelling and predicting its ever-changing state via simulations Kaur et al.
(2020); Melgar et al. (2014); Jacob et al. (2014); Aguilar et al. (2017,
2019). Unfortunately, until now, the virtual and physical environments of
cities remain insufficiently interconnected sol . At best, DTs are remotely
displayed in city administration offices to obtain insights Batty (2018); Lock
et al. (2019). However, DTs have the potential beyond administration to inform
decision-making on all levels from citizens to politicians and planners Grübel
et al. (2021). To realise such a wide-ranging implementation of a DT for a
Smart City, its data must be accessible _in situ_ within its physical context.
Situated visualisations White and Feiner (2009); Thomas et al. (2018) enable
sensing and actuating in the environment with which it is combined.

We introduce the Fused Twins (FTs)111On a grammatical side note, FTs are
always plural because they consist of both the Physical Twin and the Digital
Twin and their interaction. paradigm (see Figure 1) paradigm to enable the
conceptualisation of an accessible Smart City. We argue that there is a need
to clarify the definitions of DTs, Situated Analytics, and Smart cities with
an emphasis on the visualisation and development of a practical interface. The
FTs concept aims to embed the DT in its Physical Twin (PT), fusing them via
Extended Reality (XR) applications Grübel et al. (2021). Throughout this
paper, we follow the working hypothesis that _DTs combined with Situated
Analytics enable lower cognitive load, enhanced cognition, and easier
embodiment; and therefore, FTs can make Smart Cities more accessible to users
ranging from managers and policy makers to citizens_. Ultimately, only when
data is accessible where it is most needed and understandable to those who
consult it, can DTs fulfil the promise that IoT alone could not: to power
Smart City developments by interlinking the physical and virtual environments.

This review is structured in four main sections. To define the term FTs, we
need a thorough understanding of Smart Cities, Situated Analytics, and DTs.
The first three sections cover the necessary background to develop,
understand, and apply the FTs paradigm. We dedicate the Section 2 to gain an
understanding of how Smart Cities arose as a research field and why they
converge towards DT research. Section 3 covers Situated Analytics as a novel
data exploration approach for Smart Cities. The similarity in the
implementation structure leads us to the following section on DTs. The Section
4 elaborates on DTs. We provide a definition following Tao and Zhang, where a
DT is constructed using five components (see Figure 2). DTs are a vague term
that requires multiple clarifications in terms of composability (Subsection
4.1), servicisation (Subsection 4.2), and differences among cognate terms
(i.e., functionally and historically related but slightly different terms
Carroll (1992); Subsection 4.3). Finally, in Section 5, we cover the FTs
concept in depth including the required steps to implement FTs and classify
examples of _de facto_ FTs in the recent literature. Section 6 provides an in-
depth discussion on Smart Cities (Subsection 6.1), Situated Analytics
(Subsection 6.2), DTs (Subsection 6.3), and the newly introduced FTs paradigm
(Subsection 6.4).

Figure 2: Components of a Digital Twin. A DT is often thought of as the
interaction between the PT and the DT Grieves and Vickers (2017), but any DT
implements five core components, even if only partially. These components are
the Physical Environment, the Data Environment, the Analytical Environment,
the Virtual Environment, and the Connection Environment Tao and Zhang (2017).
The PT is measured through the Physical Environment of the DT, and the raw
data is stored in the Data Environment. The Analytical Environment provides
simulations, services and automatisation. Users often only perceive the DT
represented in the Virtual Environment as a model of the PT without being
aware of the involvement of all the other components. The Connection
Environment is the invisible glue that holds the different components together
and enables composition with other DTs if properly implemented. Generated with
http://draw.io.

Beyond the definition for FTs, we also provide several new points on a wide
range of topics. These topics have often been covered shallowly in shorter
research papers which has left a great potential for confusion. In this
review, we try to discern some of these, especially in the context of DTs.

##  2 Smart Cities

Smart Cities are an inherently technological approach to resolve urban issues
that spawn economic, social, environmental, and governmental aspects Tompson
(2017) (see Figure 3). Smart cities are constituted using bench-marking
indicators that can be constructed from technological sources Vanolo (2014).
This technologist approach only provides a narrow perspective Mattern (2017)
that is often divided along disciplinary boundaries Luque-Ayala and Marvin
(2019). The weakness in the Smart City concept is juxtaposed by its necessity
because technological advances are one of the few options to manage the
complexity of modern cities Tompson (2017). However, at the same time, it must
be acknowledged that there are other perspectives or “urban imaginaries”
Huyssen (2008) beyond Smart Cities such as resilient cities Newman et al.
(2009), intelligent cities Vanolo (2014), and sustainable cities Haughton and
Hunter (2004). In the reminder of this section, we will expand on the
evolution of “smart cities” from the nineteenth century all the way to its
portrayal in contemporary urban discourse.

Figure 3: The Smart City with a Digital Twin Foundation. Smart Cities rely on
effective data collection, analysis, and communication to work. A DT can
provide the required processing of the data collection systems to formalise
the PT. In a Smart City, a DT provides analysis for governance, management,
and entertainment. While governance refers to directing the city’s future,
management refers to the present state of the city, and entertainment refers
to the use of the city. Generated with
%****␣template.tex␣Line␣250␣****http://draw.io.

Throughout their evolution, advances in ICT have repeatedly helped cities to
cope with growing complexity Castells (2011); Townsend (2013); Gath-Morad et
al. (2016). During the nineteenth century, industrial cities were faced with
the management challenges of large industrial enterprises as well as new
administration challenges due to unprecedented population growth. The need to
process information and communicate it quickly over great distances led to the
invention of the telegraph, which enabled the industrial city to keep growing
Castells (2011); Gottmann (1977).

This transformation, from a society in which communication was impeded by
distance to a society in which distance is addressed using ICTs, has forever
changed humanity and cities Gottmann (1977). On the one hand, critical
obstacles to the growth of cities were removed with the integration of
technologies that enabled businesses to flourish and municipalities to govern
more effectively. On the other hand, futurists such as Toffler Toffler and
Alvin (1981)and Negroponte Negroponte (1995) predicted that these technologies
will lead to the “death of distance” because it is possible that improved
telecommunications technologies will make cities, and space itself, obsolete.
Their basic proposition was that electronics will eliminate the need for face-
to-face interactions and that the cities at their core facilitate those
interactions and thus would themselves become obsolete Toffler and Alvin
(1981); Negroponte (1995). Clearly, this is not yet the case. Major cities
have become points of intense investment in ICT because the production,
exchange, and dissemination of information were and still are critical to
their function. Paraphrasing Abler, the shift from goods handling to service
and information industries has made cities into communication systems that are
central to information flow in the global economy Abler (1970).

Today, the integration of ICTs into Smart Cities has become a complex man-made
creation that is unparalleled in its cascading effects on every aspect of the
city and its citizens Townsend (2013); Castells (2011). Two innovations in the
20th and 21st centuries have enabled this evolution: the rise of the Internet
infrastructure for networked computers and mobile personal computing. In the
words of Townsend, “The democrati[s]ation of computing power that started with
the PC in the 1970s and leap[t] onto the Internet in the 1990s is now spilling
out into the streets Townsend (2013)” which can be interpreted as the rise of
IoT as part of the Smart City landscape.

Contemporary ICTs have restored the importance of space to the previous
discourse on urban telecommunication Picon (2015). Everything and everyone in
cities generate geo-tagable information which can reveal the position of a
multitude of stationary or moving objects in real time Goodchild (2007); Picon
(2015). Geo-tags enrich space with contextualised electronic content and are a
fundamental dimension of a sort of return to space, or _spatial turn_ , to use
the expression coined by urban geographer Soja Soja (1989). This spatial turn
of digital technologies has been reinforced by the proliferation of electronic
interfaces and wireless communications, which realise the notion of Ubiquitous
Computing or Ambient Intelligence Picon (2015); Townsend (2013); Flügel and
Gehrmann (2009); Dohr et al. (2010); Cubo et al. (2014); Darwish and Hassanien
(2018) and ultimately underpin a DT.

Sometimes visible, but more often hidden, countless chips and sensors allow
objects, people, and vehicles to be located; consumption levels and
transactions to be recorded; and temperatures, pollution levels, population
densities and flows to be measured Weiser (1991); Atzori et al. (2010); Puiu
et al. (2016). The possibility of transforming every Thing in the built
environment into something more “smart” appears more feasible than ever
before. Low cost, miniature sensors enable everything to sense and be sensed.
The ability to perform analytics over the data collected has become
ubiquitous, enabling Things to “think”, while wireless communication
technology enables Things to “talk” to one another as part of the Internet of
Things (IoT) Goodchild (2007); Negroponte (1995); Picon (2015); Townsend
(2013). However, as note earlier, this dream of Smartification has not been
realised due to the increasing complexity of cities. It increasingly looks as
if a DT is required to provide the necessary abstractions to enable the Smart
City.

Ambient Intelligence has taken on many forms that resemble the functionality
of DTs, but they remain disconnected and often disjoint. This leaves it up to
the observer to connect the dots, forming the meta-analysis on the city level.
Examples of Ambient Intelligence range from trash cans and parking lots to
streetlights. Smart waste disposal containers in Barcelona are able to know
when they have reached their maximum capacity and must be serviced Nikitin et
al. (2016). Smart parking uses sensors to know if a parking lot is free or
not. Citizens can then be directed to available parking in the London borough
of Westminster Westminster City Council (2014). In the Future City Glasgow
project Leleux and Webster (2018), streetlights can be equipped to sense and
analyse what is happening below them. Sensors collect data on temperature,
humidity, noise, and movement, and interpret the data into a functional
“information-scape”. The streetlights can then respond in real time by
adapting their light intensity or by communicating the interpreted data to the
nearby police station, reporting any ongoing crime.

There are also even more ambitions approaches towards constructing a DT. The
Spanish city of Santander with 180,000 inhabitants placed 20,000 sensors to
monitor temperature, luminosity, ground humidity, and available parking slots
Sánchez et al. (2013). Paris’s 120,000 trees have each been equipped with a
radio-frequency identification (RFID) chip itr . In a large number of towns,
vehicular traffic is monitored in real time through technology ranging from
induction loops, to wireless sensors, to video cameras. In Singapore, this
monitoring enabled the introduction of one of the first dynamic pricing
systems, which consists of making drivers paying in proportion to the
desirability and, more importantly, the congestion levels of thoroughfares
Picon (2015). This new capacity of cities to detect, measure, and record what
is happening within them is truly unprecedented in their history Picon (2015);
Townsend (2013); Goodchild (2007) and yet requires new infrastructure to
manage the sheer complexity Botín-Sanabria et al. (2022); Bilal et al. (2020).
In the words of Foth: “If aerial photography revealed the skeletal structure
of the city during the nineteenth century, the revolution brought about by
contemporary ICTs is likely to reveal its circulatory and nervous systems – as
it reveals the “real-time city”. For the first time in history, cities can be
understood as a whole, the way biologists see a cell – instantaneously and in
excruciating detail, but also alive Foth (2008)”.

This new understanding of real-time urban dynamics is not (yet) centralised,
nor is it ensured to be the property of city governments and planners. If
servicised properly and arranged as DT components, it may remain distributed
in nature with a low barrier to entrance and can be accessed from nearly
everywhere at nearly any time. The ongoing rise in personal mobile computing,
especially smartphones, is transforming how individuals perceive and interact
with the urban space that surrounds them. Situated Analytics powered by DT
infrastructure would provide citizens, tourists, and other users unprecedented
insights into the city to help them in their every day life and in planning
how to use the city in the future. Using DTs to power Smart Cities will make
analytical tools available to everybody and will make them accessible in a
spatial context that will democratise the understanding and use of the city.
To realise this vision of a Smart City and deal with the complexity that is
already plaguing current implementations of Smart Cities, DTs offer a bedrock.
The composability of separate DTs into a city-wide DT, underpinning the Smart
City, is a promising approach (see Figure 4).

Figure 4: Composition of Digital Twins to describe higher order complex
systems. Multiple DTs are composed into higher order systems such as cities.
Here an open stationary city DT is composed of different open or closed and
mobile or stationary DTs such as building DTs, transportation DTs and
Infrastructure DTs. Each of the DTs contains complex nested DTs of different
types such as for healthcare or industrial settings. The complexity level can
be arbitrarily increased through nesting. Generated with http://draw.io.

##  3 Situated Analytics

Figure 5: Immersive Analytics compared to Situated Analytics. The real world
is schematically represented in grey. AR content is shown in blue. On the left
side, a user is displaying a smart city environment (blue model) in non-
situated AR. Interactables are highlighted in white, and several pop-ups
showing information (white squares) are opened. Immersive Analytics make use
of 3D space to visualise content but can be independent of the spatial context
surrounding them. On the right side, a user (in yellow) is walking through the
physical environment of the smart city. In Situated Analytics, interactables
are AR content that is displayed in situ of the physical environment that they
describe. Similarly to a), pop-ups (blue squares) are showing content to the
user in AR space. Generated with https://miro.com and https://rhino3d.com with
royalty-free models from https://cgtrader.com.

Augmented Reality (AR) combines computer-generated visual content in 3D with
the real world by superimposing virtual content onto a view of the real world
Azuma (1997); Milgram et al. (1995). The principles of an AR system are three-
fold, combining real and virtual elements, registering in three dimensions,
and, importantly, providing interactions in real-time Azuma (1997). There are
at least two categories of approaches to produce AR. First, screen-based
approaches, or “see-through AR”, can be divided into the optical combination
of a screen with the user’s perception of the real world or 2D displays that
replay a live recording of the real world and superimposes virtual content
Azuma (1997); Van Krevelen and Poelman (2007). Second, projector-based
approaches, or “spatial AR”, consist of projecting the virtual content
directly onto the real-world, benefiting from projection mapping algorithms
and hardware Van Krevelen and Poelman (2007); Grundhöfer and Iwai (2018);
Brock et al. (2018). The last decades have refined see-through AR and, today,
the main lines of AR systems are either based on the HoloLens for optical
systems and ARKit or ARCore for display-based AR on smartphones and tablets.
With the systems maturing to the point where it is both easy to use and easy
to generate content, AR applications are going mainstream Carmigniani et al.
(2011) and find applications across all industries Lavingia and Tanwar (2020),
commerce Scholz and Smith (2016), entertainment Stapleton et al. (2002),
education Elmqaddem (2019), and medicine Eckert et al. (2019).

As AR becomes more than a gadget, it opens the door for immersive analytics
Marriott et al. (2018) of data and information either in situ or remotely (see
Figure 5). Whereas the systems to display AR have been developed over the last
decades, the visualisation of information in this new media is still in its
infancy and yet rapidly evolving Dwyer et al. (2018). For example, traditional
visualisation on 2D screens and print is a well-established field with many
well-known representations, toolboxes, and high literacy among producers and
consumers. However, most spatial 3D visualisation are still hand-coded and
often imitate 2D visualisations. 3D visualisations also offer the opportunity
for interaction and embedding in the augmented space Genay et al. (2021);
Gervais et al. (2016), possibly easing cognitive load and enhancing cognition
Willett et al. (2021) through externalisation Scaife and Rogers (1996).

The development of AR technologies also enables new types of interaction with
digital content. Specifically, embodied interaction is a concept that
reconnects the fields of tangible and social computing and argues that
interaction is fundamentally physical and social. Thus, failing to consider
these aspect for design interaction would be a fallacy Dourish (2004); Kirsh
(2013). Embodied interaction is defined as “the creation, manipulation, and
sharing of meaning through engaged interaction with artifacts” and places the
responsibility of meaning-making on users rather than designers Dourish
(2004). The level of responsibility for meaning-making at the object’s level
is driven by different definitions of affordances across research fields that
have yet to be reconciled Michaels (2003). There is an epistemological
difference between affordances as properties of the object or interaction
Gibson (2014) and affordances as properties of the users’ mental
representation Norman (2013). From both Normanian Norman (2013) and Gibsonian
Gibson (2014) perspectives, AR can be viewed as deeply anchored in the
physical world and therefore demonstrate strong affordances for embodied
interaction Azuma (1997). Moreover, AR supports a higher amount of embodiment
compared to 2D displays by enabling high levels of sensorimotor engagement and
immersion Johnson-Glenberg and Megowan-Romanowicz (2017).

For example, anchoring and embedding visualisations in the physical 3D world
can offer tangible ways to explore one’s physiological data Gervais et al.
(2016a) and, to some extent, one’s own “personal” DT. For example, Tobe is an
AR tangible humanoid DT to explore one’s physiological data such as heart rate
and breathing patterns and was used in a social context to let two users relax
together via data observation Gervais et al. (2016b). Similarly, Teegi is an
AR tangible humanoid DT used to explore one’s brain activity via
electroencephalogram visualisations Frey et al. (2014). Non-humanoid
representations have also been explored. Inner Garden is an augmented sandbox
embodying one’s physiological state, representing breathing patterns as waves
and cardiac coherence as weather Roo et al. (2017). In the wider context of
embodied interaction, users’ bodies have also been explored as canvases for
digital content Spiel (2021), for example, by co-locating display and
interaction on the users’ hands Chatain et al. (2020) or utilising embodied
metaphors, such as mimicking scissors with one’s hand to cut through digital
content Pei et al. (2022). Tests of embodied AR approaches have also been
initiated for data exploration in the context of geographical data for
visually-impaired users Thevin and Brock (2018); Brulé et al. (2018); Albouys-
Perrois et al. (2018).

The integration of users’ bodies and their physical context has been
conceptualised during interaction with digital content. Towards that end,
different physicalities of embodiment have been described: body-centred,
object-centred, and environment-centred embodiment Melcer and Isbister (2016);
Ottmar et al. (2019). We offer examples of embodied DTs and FTs following this
categorisation in Figure 6. These categories themselves include direct-
embodied, enacted, manipulated, surrogate, and augmented approaches Melcer and
Isbister (2016); Ottmar et al. (2019). The direct-embodied approach considers
the users’ bodies as primary constituents of cognition and therefore the body
state is used to convey information. In the enacted approach, bodily actions
are considered a form of enacting knowledge through movement. The manipulated
approach uses the body to directly manipulate a representative object. The
surrogate approach uses a physical avatar that performs an interaction with
the object. Finally, in the augmented approach, the body is used to manipulate
an avatar to interact with the larger environment. The impact of the
physicality of embodiment on interaction and cognition is not yet clear.
However, these considerations are of importance because a mismatch between
activity design and type of embodiment can result in decreased movements,
persistence, and learning outcomes Chatain et al. (2022).

Figure 6: Examples of Embodied Digital Twins and Fused Twins. We observe the
difference between non-situated AR (top row) and situated AR (bottom row) for
anchoring content at the body of the user, an object, or the immediate
environment. The distinction between body-, object-, and environment-centred
AR is inspired by the classification of different physicalities of embodiment
offered by Melcer and Isbister (2016); Ottmar et al. (2019). With a body
anchor, the DT is anchored on the users’ bodies, similar to the directly-
embodied approach. With an object anchor, the DT is anchored on an object with
which the user can interact, similar to the manipulated approach. With an
environment anchor, the DT is anchored in a room, similar to the augmented
approach. In the situated environment (implementing the FTs paradigm), the
relationship between the DT and the representation is homologous such that the
body, object, or environment is used to anchor its own DT in a one-to-one
correspondence. However, in the non-situated environment, this relationship is
non-homologous because the mapping from the DT to the representation is
transformation instead of a one-to-one correspondence.

When discussing the spatial context of data visualisation and data embodiment,
we also ought to consider the role of the users’ bodies in the interaction.
Often, the users’ bodies are solely considered as physical entities (i.e.,
_Körper_) utilised to push buttons and perform actions, rather than feeling
entities (i.e., _Leib_) Mueller et al. (2018). Novel approaches such as
somaesthetic appreciation design Höök et al. (2016); Hook (2018) describe the
manner in which users’ bodies may be integrated in the design process as well
as in the experience with digital content and data. Empirical evidence
suggests that bodies’ representations impact the sense of embodiment Kilteni
et al. (2012); Nimcharoen et al. (2018), as well as the types of gestures
performed to interact with a system and the time spent observing and
interacting with data Trajkova et al. (2020). In this regard, the challenge of
FTs extends beyond meaningfully anchoring virtual data onto a physical context
to include the design of the interaction with the augmentation from an
embodied, somaesthetic perspective Dourish (2004); Höök et al. (2016); Hook
(2018).

Two further problems arise when visualising embodied information in augmented
space. First, most traditional authoring tools are focused on 2D
visualisations, and new toolkits such as DXR Sicat et al. (2018) and RagRug
Fleck et al. (2022) have only provided the basis to display 2D visualisations
of data in 3D. Second, the benefit of the third dimension is difficult to
grasp and even more difficult to implement. Most modern immersive analytics
actually rely on embedding 2D displays into the real world Prouzeau et al.
(2020); Fleck et al. (2022); Grübel et al. (2021); Sicat et al. (2018).
Visualisations in 3D are typically derived from the previous 2D media, but
there is a trend towards exploring new possibilities. For example, multiple 2D
visualisations can be arranged to use the third dimension to filter
information and visualise this filtration Hubenschmid et al. (2021).

Figure 7: Situated Analytics Thomas et al. (2018); Willett et al. (2016)
within a DT context. Situated Analytics use the Digital/Physical Divide to
define the generative steps in situ immersive analytics in the physical world
(bottom row) or digital world (top row). There is a remarkable similarity
between the components of DTs (blue boxes) and the generation of situated
analytics (white boxes). The mapping is visually nested to be highlight. A
observer in the physical world sees the physical referent or PT (in the DT
context). The observer also sees the physical presentation of information
(Virtual Environment of a DT). In the digital world, the raw data is received
from the referent, which covers the processes of gathering data by the
Physical Environment and storing data by the Data Environment. The
Visualisation Pipeline produces a visual output (Analytical Environment of a
DT). Lastly, the data is placed into the physical presentation (Virtual
Environment). Generated with http://draw.io.

Situated visualisations is less represented in the literature. Recently, there
has been some defining work that characterised the types of enhancements that
immersive analytics may provide through enhanced vision and visual synesthesia
Willett et al. (2021). Addressing the question of how situated visualisations
might improve cognition is an ongoing research area. The supposition has been
put forth that cognition may be improved by enhanced attention, enhanced
numeracy, enhanced recall, enhanced comparison, and enhanced prediction
Willett et al. (2021). Moreover, theoretical frameworks such as embodied,
situated, and socially situated cognition support the idea that cognition is
deeply rooted in its physical and social context Wilson and Golonka (2013);
Brown et al. (1989); Smith and Semin (2004). However, there are no empirical
evaluations yet because, so far, researchers have not focused on the
embodiment of the user in the analytics or understanding how the spatial
context can reduce cognitive load or enhance cognition.

Some research is starting to address the impact of the third immersive
dimension on visualisation possibilities Merino et al. (2020). For example,
one small study demonstrated that a toolkit to create new immersive
visualisations helped to keep users engaged for longer and produce more
elaborate immersed visualisations. We have a clear understanding of how 2D
visualisation improve cognitive load through externalisation Scaife and Rogers
(1996), but for AR, the evidence is less clear Turan et al. (2018); İbili
(2019); Thees et al. (2020); Buchner et al. (2021, 2022). Previous work has
proposed the following mechanisms for how 3D visualisations can improve
cognition by reducing task switching with visual cues Neumann and Majoros
(1998); Bonanni et al. (2005) and context-sensitive displays Kim and Dey
(2009). Moreover, recent work also explores the potential of AR to reduce
negative split-attention effects in multimedia activities Chatain et al.
(2020); Altmeyer et al. (2020, 2021). Split attention effects Mayer and Moreno
(1998) occur when two related sources of information are spatially or
temporally distant. However, split-attention is not always negative. For
example, indexing (i.e., connecting multiple data representations through
pointing gestures) is investigated as a mechanism for sense-making Hornecker
(2016). The distance between information sources can also be reduced using AR,
in particular the FTs paradigm, because it co-locates physical and virtual
entities in real-time Sweller et al. (2011); Kapp et al. (2020); Altmeyer et
al. (2020, 2021); Azuma (1997). In addition, overwhelming displays of
information have been shown to decrease performance Tang et al. (2003). There
is no definite answer as to how immersive analytics ultimately improves users’
performance, but theoretic arguments keep this research area active.

Furthermore, the spatial context in the augmented space has often been used as
an empty canvas Sicat et al. (2018) instead of a source of information Lobo
and Christophe (2020). Most commonly, this spatial context has been used for
geographic representations to show the built environment either in the past
through remote sensing or as planned in the future Lobo and Christophe (2020).
The subfield of Situated Analytics Thomas et al. (2018) combines data with its
spatial context to achieve a form of virtual data physicalisation Dragicevic
et al. (2020) and embodied interaction Dourish (2004); Kirsh (2013); Chatain
et al. (2020); Genay et al. (2021); Pei et al. (2022) that enables humans to
maximise the use of all senses to perceive the data (see Figure 7). Indeed,
information on cities can be embedded in the situated context with meaningful
3D representations that users can explore. The real world can become the
canvas to help users make sense of the information _in situ_ Bressa et al.
(2021).

##  4 The Digital Twin

Before we explore how FTs can make DTs more accessible, we will review DTs,
their historical background, and their relations to other adjacent
technologies and the wide range of definitions that accompany them.
Historically, the conceptualisation of DTs originated from the industrial
branch of IoT developments in 2003 Grieves (2014). The concept was loosely
defined at the time but already included the most important components: a
physical object, a virtual object, and their connections Grieves and Vickers
(2017). Throughout this section, we will expand this understanding to reach
our definition of a DT based on composability and the five components the
Physical Environment, the Data Environment, the Analytical Environment, the
Virtual Environment, and the Connection Environment (see Figure 2).

Historically, there was a lag in the adoption of DTs because the technologies
necessary to effectively implement a DT were not yet developed Tao et al.
(2018). In the 2010s, the DT concept outgrew its industrial roots by capturing
cyber-physical integration as well as representing the physical world
virtually Lee (2008); Rajkumar et al. (2010). As DTs grow in size from
factories and cities Lee et al. (2022) to the globe Nativi et al. (2021),
remote sensing with drones and satellites plays an increasingly important role
to keep DT representations up-to-date but also offer an opportunity for
interactions with the city through interactive mobile displays technology
Brock et al. (2018). For a full review on current DT applications, we refer to
Botín-Sanabria et al. (2022). Here, we will elaborate the definitions of DTs
to form a common basis from where we can develop the concept of FTs.

DTs have been “hyped” according to Gartner as one of the top-ten most
promising technological trends of the previous decade Pettey (2017); Cearley
et al. (2017, 2018); Panetta (2018); Tao et al. (2018). Beyond the hype,
across many different definitions, DTs fuzzily describe any software aiming to
closely represent something in reality Boschert and Rosen (2016). These
qualities are in line with common and older definitions such as Cyber-Physical
Systems (CPS) Wiener (1961), Ubiquitous Computing (UC) Weiser (1991), and
Mirror Worlds Gelernter (1993), which begs the question of whether we can (and
should) draw a line between DTs and these concepts. Here, we argue that DTs
should be approached as an analogy to a mental model that helps to structure
information and actions and to attach them to the real world through
appropriate mechanisms such as the FTs concept.

In this review, we next situate DTs in the context of its predecessors. DTs
are a specialised form of Cyber-Physical Systems (CPS) Wiener (1961) because
they focus on the accurate representation and manipulation of processes in the
real world with a model. DTs are similar to Ubiquitous Computing Weiser (1991)
because they attach computations to real world objects, but without the FTs
paradigm, DTs are ultimately not Ubiquitous Computing because the computations
are detached from the embodiment of the represented object. Fusing the PT and
DT Grübel et al. (2021); Prouzeau et al. (2020) joins CPS properties with the
PT to produce truly Ubiquitous Computing as envisioned by Weiser Weiser
(1991).

Whereas Ubiquitous Computing has no clear requirements for the kind of
computations to be performed, there is a common theme across different
definitions of DTs with respect to the need for simulations to predict a
system’s response to critical events. The aim to predict often goes beyond
simulating known events towards understanding deviations between the actual
observations and the predictions to indicate unknown issues Glaessgen and
Stargel (2012); Aguilar et al. (2016). Furthermore, DTs are often expected,
not only to report on the state of the system, but to also actively manipulate
the system through actuators Grieves and Vickers (2017); Botín-Sanabria et al.
(2022); Aguilar et al. (2016). The tension between the accuracy of DT
predictions and computational capacity is captured by the idea of Mirror
Worlds Gelernter (1993), which perfectly imitate reality similar to a mirror.
Early ideations of the DT therefore required the underlying model to support
similitude Glaessgen and Stargel (2012), while similitude requires the
implementation of a multi-physics, multi-scale, probabilistic, ultra-fidelity
simulation that mirrors the PT based on historical data, real-time sensor
data, and a physical model. The “ideal DT” requires similitude to become as
close as physically possible to mirroring the real world. Indeed, DTs need to
model the mutual dependency of hardware and software to tightly couple the PT
and the DT, producing high-quality simulations. Until recently, most
simulations were feasible only for certain sub-components of complex systems
that relied on scientific knowledge and real observations to produce a more
accurate offline simulation. Therefore, the requirements of similitude have
only been satisfied in specialised cases such as with the components of flying
vehicles Glaessgen and Stargel (2012). New computational methods increase
simulation performance such that events in the DT can be simulated fast enough
for predictive purposes in the real world Gabor et al. (2016). However, even
with respect to aircraft, there are only plans to produce a DT of a whole
vehicle Chowdhury et al. (2019); The Economist (2022) which are expected to be
completed in the coming decade. Regarding the sheer scale of cities, the
mirror worlds that would underlie a smart city seems to be a distant dream.
Due to the simulation complexity of large-scale systems such as cities,
researchers have since called for a less rigorous model definition to make DTs
relevant to real world applications Batty (2018).

In other words, stepping away from ideal DTs to “practical DTs” can allow for
more practical applications. Easing any of the core aspects of similitude
allows for the implementation of many useful concepts such as analytical
assessment, predictive diagnosis, and performance optimisation Tao et al.
(2018), even in large-scale systems. Limiting the functionality of DTs to a
synchronous representation of reality can also allow for the monitoring of
complex production processes, adjust these processes, and possibly optimise
them Weyer et al. (2016). Ideally, the model underlying a DT is complex enough
to enable a system to continuously react to dynamic changes in physical space
Rosen et al. (2015), including unforeseen/unexpected changes Vachálek et al.
(2017). DTs also open the door to continuous design improvement through data
feedback loops Canedo (2016) enabling data assimilation into a model Wikle and
Berliner (2007); Reichle (2008). These incomplete but practical DTs can become
especially useful if they are composable to cover more features of the PT.
Next, we will introduce specialised, practical DTs which answer questions on
particular topics and can be composed to resemble ideal DTs.

In our definition, we side-step the discussions surrounding ideal DTs by
focusing on structural features of practical DTs. We refine the definition of
a DT based on earlier work Tao et al. (2018) as a five component system: the
Physical Environment, the Data Environment, the Analytical Environment, the
Virtual Environment, and the Connection Environment (see Figure 2). In the
Physical Environment, IoT systems Atzori et al. (2010) and remote sensing
units Xiao et al. (2018) try to capture the PT and provide interactions with
the PT through actuators. The Data Environment contains the DT’s memory and is
usually located somewhere in the cloud but may also be stored locally or at
the edge Shi et al. (2016); Hossain et al. (2018). For the Analytical
Environment, the processes and models of the DT are computed Schluse et al.
(2018), often in the cloud, and insights and predictions are communicated back
to the Data Environment. The Virtual Environment makes the DT accessible to
human users and possibly other machines Grübel et al. (2021); Prouzeau et al.
(2020); Swetina et al. (2014). Finally, the Connection Environment provides an
Application Programming Interface (API) to transfer data as required between
the components. We consider two ways of designing the connection environment,
1) from a service perspective, where the Connection Environment is not an
independent component but refers to the APIs exposed by the different
components, and 2) from a compatibility/hypervisor layer perspective where the
connection environment is an independent component that mediates between the
other different components in the system. Each component could be implemented
in many different ways, and making a sensible choice may be overwhelming. This
review gives an overview to guide newcomers through the most important
components and decisions that are required to implement the FTs concept with a
DT.

To understand DTs, it is necessary to comprehend the function and goal of the
core components in relation to the literature. Specifically, we connect our
definition of the Physical Environment to research on IoT and remote sensing,
as well as our definitions of Analytical and Virtual Environments to research
on simulation and interaction with DTs. Our definitions of the Data and
Connection Environments are not broadly covered in previous research because
previous implementations have used case specific data solutions and had were
not focused on interoperability.

The Physical Environment component captures the PT and associated interactions
via IoT and remote sensing infrastructure. The composition of IoT stack is
regularly repeated but rarely analytically decomposed with respect to its
trade offs. For most applications, the stack is designed with the purpose of
underlining where and why a specific new technology is needed Xu et al.
(2014); Lee et al. (2015); Knappmeyer et al. (2013); Guinard et al. (2011).
Technically, this is a useful approach to help experts situate a technology in
the literature. However, these stacks typically make many unspecified
assumptions that we disentangle here. We start with the notion that the DT
perspective can be mapped to aspects of IoT and remote sensing. Following the
semantic decomposition of the term IoT Huang and Li (2010), Things themselves
are not actually _in_ the Internet, but information about them is perceived
either with attached sensing units or with remote sensing and then shared via
the Internet. The DT can be used as the necessary representation of the
perceived Thing in the Internet.

Previous resource-oriented and service-oriented architectures have represented
IoT hardware as practical DTs without using this particular nomenclature
Spiess et al. (2009); Guinard et al. (2010a, b, c, 2011); Kindberg et al.
(2002); Blackstock and Lea (2012). In contrast to IoT definitions, the DT
perspective does not necessitate the digital information to be shared globally
as may be implied by the term Internet. The potential local scope of DTs is
reflected in their origins in cyber-physical systems Wiener (1961); Grieves
and Vickers (2017) where the application scope is usually limited to an
industrial production site. Nonetheless, the DT perspective provides us with a
useful definition of the properties that a Thing should have when represented
digitally. A broad range of example IoT applications have been addressed in
previous publications Atzori et al. (2010); Sánchez López et al. (2012);
Borgia (2014); Islam et al. (2015); Wang et al. (2016); Darwish and Hassanien
(2018) and can be generalised to DT solutions.

The Analytical and Virtual Environments in DTs are difficult to precisely
define because their development is multi-disciplinary and reaches beyond the
simulation perspective. These paradigms have been advanced by independent
research in the areas of information science, production engineering, data
science, and computer science Tao et al. (2018) as well as electrical
engineering Akyildiz et al. (2002). A DT is a highly complex system that
combines infrastructure, simulation, data processing, human-computer
interaction, and services Tao et al. (2018). These approaches give rise to
another understanding of DTs as an advanced digitalisation of a system that
extends representation of the PT with operations. This produces a rich digital
model that has many desired properties such as scalability, interoperability,
extensibility, and high fidelity Schleich et al. (2017). To arrive at a
digital model, digitalisation in a space can also be understood as a
virtualisation process Moreno et al. (2017). Here, the system is represented
in 3D space via modelling and remote sensing. First, behaviours are extracted
and knowledge representations are created. Second, the interactions between
its constituent elements are modelled and integrated into the knowledge
representations. Third, operations and their effects are modelled. Lastly,
everything is assembled in a simulation to make the system match the PT as
close as possible. Because digitalisation/virtualisation only creates the
Analytical Environment (see Figure 2), most research has clearly focused on
the analysis and not interaction with the DT. Most research on DTs also
follows the simple physical-digital split into PT and DT components Boschert
and Rosen (2016); Grieves and Vickers (2017), but this split often fails to
capture the difference between the Analytical Environment for computation and
the Virtual Environment for interaction (but see Tao and Zhang (2017)). The
FTs paradigm offers a new perspective that highlights the unique contribution
of the Virtual Environment, resulting in a more intuitive use of the DT in
nearly every application domain.

There may also be issues with many definitions of DTs because of the
unrecognised difference between an universal ideal DT and a partial practical
DT. Ultimately, universal ideal DTs are unachievable because of the need to
integrate the complex requirements of similitude and to comply with the many
definitions across different literatures. Therefore, any practical DT is only
partially implemented compared to ideal DTs. Confusion regarding this
distinction has led to a heated debate and has resulted in the differentiation
of DTs according to their degree of completeness from digital models and
digital shadows to (ideal) DTs Schluse et al. (2018); Sepasgozar (2021), see
Figure 8.

Figure 8: Digital Shadows and Digital Models in the context of Digital Twins.
The left side demonstrates the relations between a digital shadow, a digital
model and a DT Schluse et al. (2018). The right side demonstrates the mapping
to our definition. The Digital Shadow corresponds to the Data Environment and
relies on the Physical Environment to collect data. The Engineering, Digital
Model and their DT simulation corresponds to the Analytical Environment. The
experimentable DT is akin to the Virtual Environment. In our definition, all
blocks are components of the DT whereas in the other definition, the DT is a
piece in the diagramm. The reduction of the DT to a simulations removes the
HCI from the system and requires terms such as digital shadow and digital
models to differentiate DTs. Under our definition, theses terms reflect
different aspects of a DT and map to different components. Generated with
%****␣template.tex␣Line␣550␣****http://draw.io.

A digital model is an independent representation of a real-world system that
may be used in simulations but that is generally not fed real-time data.
According to our interpretation, a digital model reflects the implementation
of the Analytical Environment. A Digital Shadow is the assembly of data from
the real-world system for descriptive purposes. By our interpretation, a
digital shadow reflects the Physical Environment of the DT. Thus, our
definitions can shift the discussion on digital shadows and digital models
from whether something is a DT to what component of a DT is implemented. While
a completely implemented, ideal DT at the urban scale is probably out of scope
for now, it is feasible to implement and compose partial practical DTs that
are capable of performing important tasks for Smart Cities. To elucidate IoT,
DTs, and Smart Cities, the present paper adopts the FTs perspective because it
unites three views on DTs that are often confounded: Why to generate the DT
(i.e., business case for the Analytical Environment), how to generate the DT
(i.e., technological implementation of the Physical Environment), and how to
use the DT (i.e., end user interaction of the Virtual Environment).

###  4.1 Composition of Digital Twins

Composed DTs offers an integrative perspective to understand and address the
complexity of the PT by using multiple abstractions and performing different
tasks on each abstraction (see Figure 9). Towards this end, DTs are composable
through their Connection Environment allowing both for the nesting and
recombination of components. This approach allows us to re-conceptualise any
sensors in the Physical Environment as virtually belonging to all associated
composites of the DTs. Composites of the DTs can be produced to offer
previously unavailable information via new virtual sensors Madria et al.
(2014) or virtual sensor networks Ocean et al. (2006) Khan et al. (2016).

Figure 9: Example Composition of DTs. Multiple Smart Objects (SOs) and their
DTs are composed into a DT of a Household. Remote sensing provides the
external context to the building. Locally FTs (SOs) provide physical user
interfaces enabling end users to directly interact with the DT in a natural
way. Generated with http://draw.io.

For DTs to be composable, interoperability between components is the most
important feature Bröring et al. (2017); Desai et al. (2015); Al-Fuqaha et al.
(2015); Schleich et al. (2017); Raza et al. (2018). An example composition of
the different components is shown across associated DTs in Figure 10. The
Physical Environment contains sensory equipment within arbitrary boundaries to
capture properties of the PT, which is immutable and equally anchors all
overlapping DTs. The Data Environment is a heterogeneous composition of
storage architectures that best befits the data requirements. The Analytical
Environment is usually defined per task per DT. In addition, it may also have
feedback loops to other DTs via the Data Environment. Outputs of the
Analytical Environment are fed back into the Data Environment as virtual
sensors to provide the information for display and interaction at a later
time. The Virtual Environment forms a local Mirror World Gelernter (1993) that
allows for the visual inspection of information in the DTs. The scope of the
Virtual Environment is limited to the current composition of DTs and allows
for access to and interaction with both the data from the Physical Environment
and the data from the Analytical Environment. The Connection Environment
manages access rights between components and ensures that users (both humans
and other DTs) have the appropriate access rights to use the respective
environments, addressing concerns regarding trust, security, and privacy Das
et al. (2018); Keoh et al. (2014); Granjal et al. (2015); Kouicem et al.
(2018); Zhou et al. (2008); Chen et al. (2009); Zhang et al. (2014); Singh et
al. (2016); Heer et al. (2011); Yaqoob et al. (2017); Zhao and Ge (2013);
Sarma and Girão (2009); Roman et al. (2011); Sadeghi et al. (2015); Raza et
al. (2018).

Figure 10: Composition of Components to form Digital Twins. Each DT is
represented as a column. The Connection Environment (represented by arrows)
connects instances in all environments across DTs. The other environments are
aligned in rows to show the implementation across DTs. Instances are shown in
each environment as rectangles, cylinders, triangles or circles. Data from the
Data Environment can processed in the Analytical Environment or displayed
directly in the Virtual Environment. DTs can also be siloed (e.g., DT C) but
more often will use the Connection Environment with complex dependencies
across all environments. For example, DT A uses Data 3 and Analysis 3 from PT
B. PT B has two DTs where DT B2 is composed into DT B1 in the Virtual
Environment by combining interface 2 and 3. DT B1 & B2 are using data that is
collected from all PTs. Despite DT C being siloed, PT C is evaluated in DT A
through Data 3 and Analysis 2. Note that this abstraction makes no claims
about the mode of data collection in the Physical Environment that may be with
either local sensors or remote sensing. Generated with http://draw.io.

In practice, DTs are applicable to all fields in which IoT has been applied
Akyildiz et al. (2002); Atzori et al. (2010); Borgia (2014), including
industry Spiess et al. (2009); Bi et al. (2014), health care Dohr et al.
(2010); Doukas and Maglogiannis (2012); Amendola et al. (2014); Islam et al.
(2015); Catarinucci et al. (2015); Kumari et al. (2018); Cubo et al. (2014),
urban management Vlacheas et al. (2013); Krylovskiy et al. (2015); Puiu et al.
(2016), vehicular application Al-Turjman and Lemayian (2020), and mobile end-
user applications Kamilaris and Pitsillides (2016). However, the composability
of DTs suggests that, by extension, these applications should be composable,
too. For example, DTs in industry, health care, and transportation could
exchange vital information to improve their effectiveness, but, to the best of
our knowledge, this has not been accomplished in practice. To understand how
DTs of these varied systems could be composed in the context of truly smart
cities, we categorise each application by its system type and location type to
form a taxonomy of DT contexts (see Table LABEL:tab:iot:application:taxonomy).

[!htbp] Taxonomy of Digital Twin Context Types System Type Open Closed
Location Type Stationary Cities, Economies & Countries Industrial processes
Mobile End users, goods & vehicles Body (health) & machines (internal)

The system type of a DT is either open or closed Bertalanffy (1969). Open
systems are characterised by an exchange with the wider environment in which
they are embedded, whereas closed system do not directly exchange with the
environment and are usually observed in isolation. In addition, closed systems
usually supervise well-defined processes, whereas open systems try to
characterise ill-defined processes. Obviously, no system is entirely isolated,
but industrial processes or health applications can be considered relatively
closed systems with usually well-defined interactions with the environment. In
contrast, cities and traffic Raza et al. (2018) have no clear limit to what is
entering from the surrounding environment, and congestion may be a side effect
of this ill-defined boundary.

The location type of a DT describes the relation between the unit of
observation and the sensors as active or passive sensing Moussaïd et al.
(2018). In stationary systems, the sensors are fixed in the environment and
observe throughput. In mobile systems, the sensors are moving around either
with the observed unit or independent of it. They may also capture
environmental conditions around the observed unit such as in human navigation
Kiefer et al. (2014) and health monitoring Petäjäjärvi et al. (2017).

When composing different types of DTs, it is important to understand their
characteristics. Mobile DTs may only temporarily compose with Stationary DTs,
while Closed DTs often form units within Open DTs. Open Stationary DTs often
form the overarching type of DT that contains other types of DTs, whereas
Mobile Closed DTs usually describe atomic units in composed DTs. Typically,
Mobile Closed DTs can be part of Mobile Open DTs when a person has a Body Area
Network (BAN) Singh and Prasad (2021) for health care applications and a
Personal Area Network (PAN) Kushalnagar et al. (2007) to interact with the
environment. In this example, the PAN will provide a wide range of
interactions that can be composed with the Open Stationary DT of a Smart City,
whereas the BAN will only share limited information. Similarly, a Closed
Stationary DT for a Smart Factory within Open Stationary DT of a Smart City
will only share limited information.

###  4.2 Servicisation of Digital Twins

While the word DT suggests some form of unitary entity, it would be more
accurate to understand DTs as context-aware representations of Things in the
cloud composed of services that act upon the abstraction. Services can range
from interacting with the environment to representing the sensed and form the
bedrock of ubiquitous computing Weiser (1991). Basic services again can be
composed into higher-order services to address more complex issues.

With digital representation by the DT, it is possible to turn everything into
a service—or _servicise_ White et al. (1999); Toffel (2008); Plepys et al.
(2015); Agrawal and Bellos (2016). In other words, the customer buys the
temporary use of a service associated with a product instead of the product
itself. The cost of developing and maintaining a DT’s infrastructure justifies
this approach because it is more efficient to reuse resources rather than to
duplicate functionalities Agrawal and Bellos (2016). Therefore, DTs can be
freely composed of services that conform to the requirements of the Connection
Environment .

The three most common themes in “as a Service” (*aaS) are Infrastructure as a
Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS).
Newer types of services can be subsumed in these categories, including Sensing
as a Services (SenaaS) Alam et al. (2010); Distefano et al. (2015), Semantics
as a Service (SemaaS) Desai et al. (2015), and Robots as a Service (RaaS) Chen
and Hu (2013). Notably, Infrastructure, Platform, and Software “as a Service”
form an interesting interaction with the DT components because they can
provide abstractions for particular components (see Figure 2). From an
abstraction perspective, each replaces the lower layers of the stack with an
interface that hides implementation complexity. In this section, we will use
the example of weather services as part of a smart city to explain how IaaS,
PaaS, and SaaS can map to the components of a DT (see Figure 11).

Figure 11: Mapping of the “as a Service” paradigm to the components of a
Digital Twin and the weather services example. This example application is an
alert for a nearby weather event. Presented from left to right, the PT forms
the Physical Environment of the DT and is captured by local sensors and
networks and remote sensing as IaaS. The Data Environment of the DT serves as
a PaaS. The Analytical Environment of the DT models the weather and calculates
the proximity to the predicted event as a SaaS. The Virtual Environment is
implemented as a smartphone application that provides an alert on an abstract
dashboard (see Figure 14 for more details). The Connection Environment spans
across all services and environments providing an API that allows for one
environment to be replaced by another. This makes the DT both modular and
composable. Generated with http://draw.io.

####  4.2.1 Infrastructure as a Service (IaaS)

A DT requires the creation of a Physical Environment as the representation of
the PT. The Physical Environment provides access to and measurement of the PT
using infrastructure to collect raw data (see Figure 2). Creating and
maintaining the DT infrastructure is the most physically laborious part of
developing a DT. This severely limits the number of entities that can afford
to create a DT on a city scale. However, provided as a service, it may be
possible to finance the required infrastructure in aggregate and reuse it
across multiple DTs. When creating a DT, the IaaS approach allows for the
reuse of a service provider’s communication network, sensor network, and
remote sensing units rather than having to build one’s own system from
scratch. Therefore, IaaS enables to have alternative competing and composable
DTs based on the same or different data source(s). There are also more
specialised intrastructure services that are available such as Sensing as a
Services (SenaaS) Alam et al. (2010); Distefano et al. (2015), Remote Sensing
as a Service (RSaaS) Toulios (2015), and Robots as a Service (RaaS) Chen and
Hu (2013).

In our weather service example (see Figure 11), the sensors, network, and
remote sensing units in a smart city could be servicised. The city government
or a private provider could implement a policy to maintain such an IaaS to
make digital services in a city more accessible, enabling the creation of
different city DTs. This would allow new entrants to easily develop their own
DTs that provide large-scale digital services without having to build physical
hardware with redundant functionality Toffel (2008).

At the same time, IaaS may raise some concerns regarding security. The IoT
used for IaaS consists of billions of locatable, addressable, and readable
devices Roman et al. (2011). There are three avenues by which security in IoT
must be enforced: information, physical infrastructure, and management
practices Zhao and Ge (2013). Security breaches from unexpected behaviour may
arise from data collection and management via privacy protection problems,
heterogeneous network authentication, and access control problems Sarma and
Girão (2009); Roman et al. (2011); Zhao and Ge (2013); Ouaddah et al. (2017).
The fundamental challenges for all of these topics are questions of trust: how
to give whom access for what purpose and for what duration. Over the last
decade, the IEEE and IETF have laid the foundations to build a security
infrastructure along the IEEE 802.15.4 standard Association et al. (2011,
2012) and the IETF LLN stack Kushalnagar et al. (2007). These efforts include
sensor network security Chen et al. (2009), wireless sensor network security
Zhou et al. (2008), and dedicated secure protocols such as CoAP/Lithe Bormann
et al. (2012); Raza et al. (2013), 6LowPSec Glissa and Meddeb (2019) and RPL
Mayzaud et al. (2016). Ultimately, such as with the Internet, security will be
an ongoing process that must be attended at all times.

####  4.2.2 Platform as a Service (PaaS)

PaaS extends access to hardware by simplifying interfaces. PaaS usually
packages digital representations into a Data Environment and thus reduces the
complexity of accessing the data. Platforms vary largely and are usually tied
to one of the five IoT application areas industry Spiess et al. (2009); Bi et
al. (2014), health care Dohr et al. (2010); Doukas and Maglogiannis (2012);
Amendola et al. (2014); Islam et al. (2015); Catarinucci et al. (2015); Kumari
et al. (2018); Cubo et al. (2014), urban management Vlacheas et al. (2013);
Krylovskiy et al. (2015); Puiu et al. (2016), vehicular applications Al-
Turjman and Lemayian (2020), and end-user applications Kamilaris and
Pitsillides (2016). The concept of PaaS can be vague, and its use may
sometimes be difficult to differentiate from IaaS or SaaS. In the context of a
DT, a PaaS ought to provide only _access to data_ as a Data Environment in
contrast to an IaaS that collects _raw data_ from the physical world as a
Physical Environment and a SaaS that _derives higher-order data_ from
previously collected data as a Analytical Environment.

A DT probably cannot rely on a single PaaS to source all of the required data.
More often, a DT will mix and match PaaS sources Villegas et al. (2012) to
fulfil its task. The Data Environment is more complex than simply making data
available and needs to be curated, indexed, and integrated into a knowledge
graph based on context-awareness Knappmeyer et al. (2013), ontologies Hachem
et al. (2011); Wang et al. (2012); Song et al. (2010), and middleware Ngu et
al. (2017); Razzaque et al. (2016); Mineraud et al. (2016). In reality, IaaS
and PaaS are often mixed and sometimes offer a subset of services that should
be associated with SaaS Linthicum (2017).

In our weather service example, the PaaS stores the information of the
weather-related sensor data from sensing units throughout the city, including
their spatiotemporal context. Specifically, the PaaS does not include any
analytical components but offers a raw data source upon which higher-order
data services may be built. A clear distinction is necessary between services
to disambiguate their different responsibilities in the DT and ensure
composability.

####  4.2.3 Software as a Service (SaaS)

According to the classical SaaS literature, SaaS is often exposed directly to
an end user Kamilaris and Pitsillides (2016) and ought to follow a singular
use case. However, in the context of the DT, SaaS could be mapped to both the
Analytical Environment and the Virtual Environment. Following the “as a
service” paradigm, we suggest limiting the use of SaaS to the Analytical
Environment in DTs to reduce redundant functionality and to enable more
servicisation in end-user applications (see Figure 2). Conventionally, one may
consider SaaS as built upon PaaS, but the relationship between the Data and
Analytical Environments may actually be bidirectional. The results of the
analyses may be written back to the Data Environment before being forwarded to
the Virtual Environment for user interaction. Thus, there may be an ambiguity
that makes it difficult to separate SaaS from PaaS. The difference lies in
whether one considers the results of an analysis to be newly computed as a
SaaS or retrieved from memory as a PaaS.

In our weather service example, the classification of the predictive weather
data to an end user could be qualified as SaaS if it is being calculated
directly. However, if the predictive weather data is provided as a component
of a smart city environmental observation system, it would be qualified as a
PaaS. In general, a SaaS conducts some analytical task and represents the
Analytical Environment of DTs because it generates the models to mirror
reality in the DT.

Figure 12: Google search trends by search term in web searches. Interests are
averaged with LOESS across months from 2003 to early 2022. In general,
Internet of Things has been falling since 2017, and Digital Twin has been
steadily rising. This year, Digital Twin has become the most searched term
among the cognates other than Internet of Things. Terms for underlying
technologies and previous system names are stagnating or declining in use.
Data collected from https://trends.google.com on the 8th of February 2022.

###  4.3 The Cognates of Digital Twins

DTs are a relatively new perspective that shares a common but amorphous root
with a variety of other perspectives . While this root may be difficult to
trace precisely and may differ from the intent of the author, it may have
originated with various thinkers such as Wiener Wiener (1961), Weiser Weiser
(1991), or even Tesla (Atzori et al., 2017, p. 122). Given this complexity, we
borrowed the term “cognate” from linguistics Carroll (1992) that refers to
words with a formal resemblance or a common historical root. The confusing
overlap among DT cognates is driven by developments across a diversity of
research backgrounds such as information science, production engineering, data
science, and computer science Tao et al. (2018). The different components of
the DT are prioritised differently in each field, which results in similar yet
slightly different meanings. For instance, the cognate term IoT is dominating
the public sphere according to search word trends (see Figure 12). However, as
hype gives way to pragmatism, simply having the physical infrastructure
necessary for the creation of DTs is insufficient by itself. The two cognates
IoT and DTs are often confused, but IoT may form the basis of DTs in a similar
manner that the Internet forms the basis of the Web. Although it is too early
to determine, search word trends may eventually place DTs or another cognate
as the ultimate concept to describe these systems. The following sections
clarify the nuances associated with DTs and their cognates across different
disciplines.

####  4.3.1 Cyber-Physical Systems

> A Cyber-Physical System (CPS) represents physical objects as virtual
> objects. Users can with either physical or virtual objects to change the
> state of both. A DT is a special instance of a CPS that includes a model of
> the PT for predictive operations.

The terms DT is relatively new because the original idea of digital
representation was first described in detail by Wiener Wiener (1961) in the
field of control theory. Later, the term Cyber-Physical Systems (CPSs) Theory
Baheti and Gill (2011); Lee (2015) was coined to describe a _physical_ object,
homologous to a “Thing” Huang and Li (2010) in the IoT context and the PT in
the DT context. This physical object has a _cyber_ correspondence, which is
homologous to the “Digital Twin” Grieves and Vickers (2017). _A priori_ , CPSs
have no predetermined (physical) scope and are an idealised form to describe
possible interactions with entities Rho et al. (2016); Darwish and Hassanien
(2018). Therefore, the CPS literature is rarely linked directly to DTs, remote
sensing, or the IoT (but see Alam and El Saddik (2017); Um (2019)). CPSs are
increasingly used to capture social data as well Sheth (2016). Their high
level of abstraction makes CPS a suitable paradigm to theorise DTs, and
findings in the CPS literature are very relevant to DTs.

####  4.3.2 (Wireless) Sensor (and Actuator) Networks

> (Wireless) Sensor (and Actuator) Networks (WSN/SN/SAN/WSAN) provide sensing
> and actuating infrastructure that processes the signals to and from the
> physical world to enable a CPS or DT.

A DT cannot be conceptualised without having first digitised the PT. While the
DT is usually presented as agnostic with respect to its underlying hardware,
the DT cannot be constructed without it. Creating this representation requires
sensing to comprehend the physical world and actuating to change the physical
world Akyildiz et al. (2002, 2002). Sensor networks ground any Cyber-Physical
System in the physical world by provided readings thereof. For each device,
questions of how to implement “autonomicity, scalability, energy efficiency,
heterogeneity in terms of user equipment capabilities, complexity and
environments” Rawat et al. (2016) need to be answered. Sensor networks have
been researched long before the origination of the terms IoT or DT. However,
contemporary research focuses more on how sensor networks are seen as a
component of an IoT system Mainetti et al. (2011).

In a sensor network context, there are active and passive devices that can be
placed in the environment. Passive devices called Radio Frequency
Identification (RFID) harvest energy from the transmission signal via
induction to return a signal Welbourne et al. (2009). The signal can be used
as an identification of an object in order to trigger a software-based service
or to determine its location depending on the RFID reader Li et al. (2019).
Typically, RFID cannot return environmental information (but see Shao et al.
(2018)). Active devices are usually battery powered and have actual
computational infrastructure, sensory equipment, and possibly actuators to
interact with the environment. Whether active or passive devices are required
depends on the task. A DT will often require a combination of both active and
passive devices to efficiently represent the PT and the processes therein. A
comparison of tasks and the technology that best captures them can be found in
Table LABEL:tab:iot:devices:tasks.

[!htbp] Task Classification for Active and Passive Devices Task Description
Active Passive Accessing Accessing information on an object linked to the
device. Actuating Triggering an action of the object linked to the device.
Locating Identifying the location of an object linked to the device. Sensing
Measure characteristic of the object linked to the device.

Without using the term DT, RFID implement a similar abstraction because a
“physical object is accompanied by a rich, globally accessible virtual object
that contains both current and historical information on that object’s
physical properties, origin, ownership, and sensory context Welbourne et al.
(2009)”. RFID helps to embed access to knowledge in the environment and can
make access to the DT context-driven Welbourne et al. (2009). For example, if
users and environment are both tagged, the appropriate DT services can be
called to facilitate the users’ task. At the same time, the system can track
such interactions to help model activity for the DT. However, RFID tags could
reveal sensitive information, and it is important to manage access control
Rastogi et al. (2008).

In contrast to passive devices, active devices usually have local
computational power but may be constrained, which should be classified
according to usability. Resource constraints on active devices can be grouped
into six categories: battery, communication range, bandwidth, memory storage,
processing capability, half duplex communication Raza et al. (2018). The IETF
has classified devices based on hardware capabilities Bormann et al. (2014),
and each class requires different operating systems (OS) to function optimally
Hahm et al. (2016); Javed et al. (2018). They range from Class 0 devices to
Class 2 devices. Class 0 devices have a tailored OS that provides no
functionality beyond the required features. In contrast, Class 1 devices and
above may already run a generalised if bare OS fulfilling the criteria of a
small memory footprint, support for heterogeneous hardware, network
connectivity, energy efficiency, real-time capabilities, security, and
opportune scheduling policies Hahm et al. (2016); Javed et al. (2018). Active
devices can be fully integrated into DTs and can be managed from within the
DT, whereas passive devices currently only provide anchor points for DTs.

####  4.3.3 Smart Objects

> Smart Objects are physical objects that have embedded computational power to
> provide interactions for its virtual object to make it more interesting and
> may represent the fusion of the PT and DT.

A DT can be understood as a highly accurate model of a physical system
Boschert and Rosen (2016); Grieves and Vickers (2017); Batty (2018), whereas a
Smart Object (SO) offers interesting interactions with a particular object
through embedded or remote computing power Sánchez López et al. (2012). Both
DTs and SOs represent a CPS Lee (2015). It could be argued that the concept of
an SO either incorporate or encapsulate the concept of a DT. However, a DT is
usually required to mirror the role of an actual object in a larger context,
whereas SOs only provide end-users with interactions with the particular
object Sánchez López et al. (2012). Nonetheless, many SOs will represent the
underlying physical object sufficiently close to use them interchangeably for
a DT. Assuming that an SO qualifies as a DT and that the SO can offer physical
access to its digital representation, it can be considered a dual _embodiment_
for both the object (i.e., PT) and the DT, providing a physical instance of
the FTs paradigm (see Figure 9).

However, a DT may also describe an entity that is composed of multiple SOs,
other Things, processes, and high-order entities. These entities may entail
buildings, cities, or regions Li et al. (2011); Su et al. (2011); Sanchez et
al. (2014); Perera et al. (2014) modelled through remote sensing Lee et al.
(2022); Botín-Sanabria et al. (2022); Kothari et al. (2022). Moreover, a DT
may also be formed of Things that are immaterial or at least incomprehensible
in the physical world such as phantom traffic jams Gazis and Herman (1992).
These can be considered purely virtual objects in contrast to virtualised
objects Nitti et al. (2016), which would have a mapping back to a physical
object. Composition allows for the formation of even higher order virtual
objects that form the basis of a DT in a complex system such as the processes
or aggregates representing a city Rathore et al. (2016); Kaur et al. (2020),
traffic flow Anda et al. (2021), or production line Tao and Zhang (2017).
Furthermore, a DT may also be produced exclusively by remote sensing without
having local sensors attached to the PT Botín-Sanabria et al. (2022). A DT is
thus a more encompassing concept than a SO.

####  4.3.4 Ubiquitous and Pervasive Computing / Ambient Intelligence

> Ubiquitous Computing is a paradigm that expects computation to be available
> for all objects to form a CPS, whereas Pervasive Computing specifies the
> computation to occur at the object or close to it. SO and Ambient
> Intelligence are instances of Ubiquitous Computing in CPSs in which the
> sensing and actuating capacity is directly embedded into the object that is
> described by the corresponding DT.

Ubiquitous and pervasive computing are usually considered interchangeable. The
former is an older term to link computations to any object in the physical
world Weiser (1991). The latter is a younger term and associated with placing
computing power within (or close to) any object in the physical world
Satyanarayanan and Others (2001). Without loss of generality, we will only
refer to Ubiquitous Computing in the remainder of the present paper. For a DT,
any data collected in the physical world must be represented in “cyberspace”
and thus requires computation. Many ubiquitous computing applications pre-date
the IoT paradigm but have been subsumed by it (see Ma et al. (2005)).

The closeness between the Thing and computations performed on sensor data from
the Thing has often resulted in them being regarded as equivalent to each
other Atzori et al. (2010). Similarly, the concept of a DT is often used as a
synonym for both the PT (homologous to the Thing) and the Physical Environment
with WSAN (homologous to computations on sensor data from the Thing) from
which the DT is created. Ubiquitous computing offers us a lens to distinguish
between PT and DT by considering the difference between computations on the
state of an object and the actual object. SOs Kortuem et al. (2010); Fortino
et al. (2012); Sánchez López et al. (2012); Atzori et al. (2014) are a typical
example where the distinctions between PT, DT, sensors, actuators, and
computations are blurred. SOs are expected to have awareness of their
environment and have some model of the ongoing activities, policies, and
processes around them. At the same time, SOs should represent functions,
rules, and flows that allow for interactivity Kortuem et al. (2010); Sánchez
López et al. (2012). This formulation clearly indicates that the SOs have a
Physical Environment in which they are embedded, an Analytical Environment to
model their behaviour, a Data Environment to maintain their state, and a
Virtual Environment in which their interactivity can be triggered. When the
computations on the object or Thing become unrecognisable to humans because
they are “woven into the background”, as predicted by Weiser Weiser (1991),
the term Ambient Intelligence becomes more relevant Flügel and Gehrmann
(2009); Dohr et al. (2010); Cubo et al. (2014); Darwish and Hassanien (2018).
Making a clear distinction between a Thing, a SO, and Ambient Intelligence is
probably neither possible nor desirable.

####  4.3.5 Internet of Things

> The IoT is an amalgam of competing technologies with the goal of digitally
> representing the physical world. It interconnects WSANs to be combined into
> local CPSs that in turn compose larger DTs.

The idea behind IoT was first hyped Tan and Wang (2010); Khan et al. (2012);
Stankovic (2014), then overestimated, and then slowly became practical
Hurlburt et al. (2012). The term IoT was probably coined towards the end of
the last millennia Ashton and Others (2009), but its roots are older, albeit
unnamed. The functional similarity of CPSs Wiener (1961) and ubiquitous
computing Weiser (1991), as well as stipulations from early thinkers such as
Nikolai Tesla (Atzori et al., 2017, p. 122), make its historical provenance a
question of choosing a remotely related cognate. Following Atzori and
colleagues Atzori et al. (2017), we take the probable origin of the term
“Internet of Things” by Ashton in 1999 Ashton and Others (2009) as a starting
point to limit ourselves to cases where the general idea of IoT was already
the goal of research.

Nearly anything remotely related to Information and Communication Technologies
(ICT) Hurlburt et al. (2012) is included in modern IoT research Atzori et al.
(2010); Gluhak et al. (2011); Chen (2012); Gubbi et al. (2013); Borgia (2014);
Singh et al. (2014); Want et al. (2015); Al-Fuqaha et al. (2015); Whitmore et
al. (2015); Madakam et al. (2015); Palattella et al. (2016); Lin et al.
(2017); Qiu et al. (2018); Javed et al. (2018); Shafique et al. (2020) because
IoT promises to be the invisible glue to digitally represent the world and
offers new means of analysing and understanding Weiser (1991). In part, this
may be due to the well-known term _Internet_ that immediately allows people to
associate it with global information infrastructure Huang and Li (2010). On
the other side, a _Thing_ is vague enough to label any entity that can be
distinguished by some measure Huang and Li (2010). The Intranet of Things in a
WSAN has been transformed through ICT Zorzi et al. (2010); Wu and Zhao (2017);
Shafique et al. (2020) and cloud-centric processing Gubbi et al. (2013);
Alamri et al. (2013); Misra et al. (2017) to the IoT. This loose description
is what allowed the IoT to soak up many kinds of ICT developments for the last
decade (see Figure 12).

While the term IoT has recently become the most common term for the linkage
between the virtual and the real, it was not always the case. Previously, a
set of competing ideas (e.g., CPS, WSAN, and Ubiquitous Computing) co-created
the ecosystem that today is generously labelled IoT and which laid the
hardware foundation for DTs. These ideas were closely related but ultimately
based on different assumptions because they originated from different fields.
According to public perception, the terminology has settled on IoT even as the
underlying technology is still quickly diversifying and expanding. With
current trends, DTs appear to be a concept that is set to overtake the IoT in
terms of public perception (see Figure 12).

Another issue is that IoT is a buzzword that is difficult to grasp because it
is overloaded with meaning. Simply tapping into the literature is overwhelming
and, at the same time, highly specific. The many technologies peddled under
the umbrella-term IoT would be more distinctive if labelled specifically
according to their function. While the technological umbrella-term is now
well-established, if ill-defined, there is still a lingering sense that
different fields of application are uniquely distinct and require special
attention. Research and development is thus heavily directed towards five
distinct real-world applications that promise the biggest gains: industrial
application, urban management, health care, vehicular applications, and end-
user applications Atzori et al. (2010); Borgia (2014); Kamilaris and
Pitsillides (2016). This ingrained belief regarding the distinction leads to
difficulties in synthesising common knowledge across applications. Overcoming
this barrier could help the interdisciplinary fields of IoT and DTs in many
ways. Not only would it enable learning from each other to clarify their
theoretical backgrounds, but it would also make the use of IoT as a basis for
DTs more accessible. Indeed, a clearly communicated and unified approach to
IoT would lower the barriers of entry tremendously.

Weiser Weiser (1991) postulated the goal of Ubiquitous Computing to make the
computations disappear in the mundane of the environment. To obtain services
that hide in plain sight, three categories of design considerations have been
brought forward that need to be considered for IoT to work: fundamental
properties of a Thing, network communication properties, and real-time
interaction properties. The fundamental properties of a Thing include
existence, sense of self, connectivity, interactivity, dynamicity and
sometimes environmental awareness Roman et al. (2011); Rawat et al. (2016);
Raza et al. (2018). Much research has also been conducted on properties to
make network communication more efficient, including energy, latency,
throughput, modular design for improved scalability, topology, security of
information, safety Flügel and Gehrmann (2009); Rawat et al. (2016); Raza et
al. (2018), deployment, mobility, cost, efficient resource usage,
heterogeneity, communication modality, infrastructure, coverage, network size,
and lifetime Rawat et al. (2016); Raza et al. (2018). Real-time requirements
in industrial and healthcare settings have brought forth another set of
properties that need to be considered to enable real-time interaction with IoT
networks: predictable delays and latencies, congestion free communication,
accurate time synchronisation, real-time assurance, interoperability with
existing infrastructure, and prioritised communication Raza et al. (2018).

The historic development IoT and the evolution of definitions has been
addressed in detail by Atzori et al. Atzori et al. (2017). The work identified
three generations of IoT development with different foci that have resulted in
the current diffusive definitions (see Table LABEL:tab:iot:generations). An
additional new generation is identified in this review paper that exemplifies
the most current trends of DT research in IoT that was not reported in Atzori
et al. (2017). Notably, while these four generations focus on distinctive
features, they are effectively contemporary because these generations are
still actively researched. To make sense of these concurrent generations, it
is best to understand each generation as a necessary condition to start work
on the following generations.

[!t] Waves of IoT generations Generation I Generation II Generation III
Generation IV Earliest Mention 1999 2010 2011 2012 Technologies RFID Inter-
networking Cloud computing Fog Computing M2M Web of Things ICN DTs Object
integrations Constrained devices Semantic IoT Opportunistic IoT Focus
Digitalisation Networking Centralisation Decentralisation Source Ashton and
Others (2009); Atzori et al. (2017) Pfisterer et al. (2011); Atzori et al.
(2017) Atzori et al. (2017); Gubbi et al. (2013) own definition, Bonomi et al.
(2012); Guo et al. (2013); Grieves and Vickers (2017)

The first generation is driven by the rise of RFID tags Ashton and Others
(2009) to digitally distinguish Things. At this time, IoT was often taken to
be synonymous with RFID. This generation is tightly coupled to Ubiquitous
Computing Weiser (1991), Machine-to-Machine (M2M), and CPSs as the RFID
technologies ultimately derive from these research fields. However, by the
late 2000s, most of these technologies had been tagged IoT. The second
generation is driven by interconnecting Things instead of just centralising
the information flow. It focuses on the network component of IoT devices and
realistically brings the _Internet_ into the IoT Pfisterer et al. (2011).
Control of information becomes more central as well, reinforcing the
connections to its roots in CPSs Wiener (1961). The third generation is driven
by a re-centralisation around cloud technologies that matured meanwhile Gubbi
et al. (2013); Alamri et al. (2013); Misra et al. (2017). This generation
focuses on processing data and generating knowledge. The fourth generation is
driven by a second wave of decentralisation due to time-sensitivity of
applications, fog computing Bonomi et al. (2012), and the rise of DTs Grieves
and Vickers (2017) and servicisation Agrawal and Bellos (2016). As questions
of processing have been addressed, the newest generation brings more
efficiency to IoT. IoT is often mentioned as a multiplier that makes other
technologies more attractive Atzori et al. (2017), contributing to the
diffusion of the term.

####  4.3.6 M2M

> Machine-to-Machine (M2M) communication allows Things and network components
> to interact automatically with each other without human actions, thus
> enabling automated formation of CPSs or DTs on different scales.

The complexity of an IoT system underpinning a DT requires much communication
between each involved Thing and the network infrastructure to support a
control system Wiener (1961); Paraskevakos (1974); Lee (2015). These
communication protocols have been aggregated and named M2M communication. M2M
is an ongoing research field and is usually required for all other IoT-related
and DT-related activities Datta et al. (2014); Andreev et al. (2015); Gazis
(2017); Patil and Gokhale (2021). Essentially, M2M describes how IoT
components interact with each other and can be understood as part of the
implementation of the Connection Environment of a DT. As such, M2M, IoT, and
DTs cannot be meaningfully separated, although they do not constitute the same
concept. Before IoT became the defining word for ICT, M2M was a contender for
being the umbrella-term for these technologies.

The number of devices connected to IoT today surpasses the number of devices
manually connected to the Internet. Thus, it is impossible to manually manage
the communication between devices by connecting them or maintaining them
Baronti et al. (2007); Yick et al. (2008); Khorov et al. (2015); Raza et al.
(2018). This aspect triggered the development of the M2M concept in the early
2000s Wang and Fapojuwo (2017); Gazis (2017) followed by standardisations such
as oneM2M Swetina et al. (2014) in the 2010s. At the centre of M2M activities
are the protocols that are used to exchange information between devices.
Protocols describe predefined procedures of how to initiate, maintain, and
reestablish communication Swetina et al. (2014). While some protocols are
high-level, they are often bound by the physical limitations of the
communication capabilities of the constrained devices. The complexity of the
classical standard Internet Protocol (IP) was quickly disqualified as a
platform Ishaq et al. (2013) because of the assumptions that the computing
infrastructure was powerful, wired, and connected and that communication was a
negligible overhead. However, constrained devices have been highly optimised
in terms of energy consumption, size, and costs such that communication has
become the major bottleneck Sheng et al. (2013).

The physical infrastructure to connect constrained devices to the network is
usually referred to as physical / PHY Granjal et al. (2015), and the data link
/ MAC address is used to uniquely identify single units Bachir et al. (2010);
Rajandekar and Sikdar (2015); Sotenga et al. (2020). These terms originate
from the OSI model and refer to the physical and data link layers between
devices International Organization for Standardization (2016). Originally, a
link would be a copper or fibre cable. However, wireless transmission has
adopted the same terminology, so PHY usually refers to some kind of antenna
that uses a pattern of signals to encode information Kothari et al. (2022). To
enable a more efficient use of bandwidth and potentially allow for more
devices, Cognitive Radios allow secondary users to use communication channels
while primary users are idle Chen and Lien (2014); Aijaz and Aghvami (2015);
Andreev et al. (2015). The main trade-off for the PHY connection is between
data rates, connection distances, and energy consumption. Currently,
technology can be approximately grouped into four application areas: Body Area
Networks (BAN), Personal Area Networks (PAN), Local Area Networks (LAN), and
Wide Area Networks (WAN) (see Table LABEL:tab:phy:technologies for an
overview).

[!htbp] Communication technologies for PHY layer Scope Abbreviation
Technologies Reviews Body Area BAN BLE, Zigbee Akyildiz and Jornet (2010);
Akyildiz et al. (2015); Gravina et al. (2017); Miraz et al. (2018); Singh and
Prasad (2021) Personal Area PAN UWB, Bluetooth, WiFi Arslan et al. (2006);
Tozlu et al. (2012) Local Area LAN WiFi, LTE Tozlu et al. (2012); Ratasuk et
al. (2016) Wide Area WAN LoRaWAN, Sigfox, NB-IoT Raza et al. (2017); Nolan et
al. (2016); Oliveira et al. (2011); Bor et al. (2016); Petäjäjärvi et al.
(2017); Georgiou and Raza (2017); Grübel et al. (2021)

The large standardising bodies, IEEE and IETF, approach protocols in
constrained environments based on expansion of the IEEE Standard 802.15
Palattella et al. (2013); Kurunathan et al. (2018) and the IETF LLN protocol
Sheng et al. (2013); Ishaq et al. (2013) to handle IP. In addition, there are
technological niches that have not been fully covered by those standardising
organisations such as the Low Power Wide Area Networks (LPWAN) where protocols
vary and are highly incompatible, representing a low level of industry
cooperation Raza et al. (2017); Nolan et al. (2016); Oliveira et al. (2011);
Bor et al. (2016). However, several consortia have developed around the
different core technologies associated with each protocol, and only subsequent
years will show how prevalent each WAN technology has become.

Figure 13: Computing Hydrology of Computation-on-the-wire. Computing power can
be located locally at the sensor/actuator or more remotely at the gateway, a
local cluster, or remote server. The computations form the basis for a DT.
Computing in IoT and smart cities is often conceptualised with a hydrological
terminology (i.e., cloud, fog, mist, and dew) that maps to the computing
location and encapsulates different trade-offs in computational intensity,
traffic, latency, accuracy, energy consumption, and security Flügel and
Gehrmann (2009); Roman et al. (2011). These trade-offs are best evaluated
under the subsidiarity principle to decide where to place the computation
physically. Edge or grid computing Shi et al. (2016); Hossain et al. (2018) is
another terminology to covers all non-cloud computing. DTs rely heavily on
these different data and computing sources while at the same time abstracting
away their existence in the end users’ view. Generated with http://draw.io.

The commonality among all protocols is that they are simplified to move data
from the constrained devices to some kind of translation unit that transfers
the data into the IP protocol, making it widely available on the Internet Hui
and Culler (2008); Hong et al. (2010); Rodrigues and Neves (2010). This
translation is necessary to cope with the reduced computing capacity of
constrained devices as well as questions of how to transmit data and scale
local IoT networks for use in a DT. A difficulty arises for these protocols
because the translation unit of constrained devices do not only send data
(such as with sensors) but may also receive data (such as with actuators).
This aspect requires a bidirectional translation protocol that can uniquely
address every node from the outside Imani et al. (2018).

####  4.3.7 Computation-on-the-wire / Dew / Mist / Fog / Cloud and Edge
Computing

> Computation-on-the-wire computes all necessary transformations on data at
> the best possible point in the network according to the _subsidiarity
> principle_ to deliver Ubiquitous Computing for CPSs such as DTs.

The trend to locally compute, distribute, and store data in end-user devices
and local clusters rather than centralised data centres Bonomi et al. (2012,
2014) is a reaction to the third Generation of IoT’s _cloudification_ Atzori
et al. (2017). At the same time, the trend away from cloudification is the
foundational moment for the fourth Generation of IoT. Furthermore, sensors
generally pose a trade-off between computational power, latency, cost,
accuracy, energy consumption, and security Flügel and Gehrmann (2009); Roman
et al. (2011); Stojmenovic and Wen (2014); Jalali et al. (2016). To balance
these trade-offs, additional computational power was introduced physically
_closer_ to the sensors, which also limited data transmission and reduced the
load on the network. The placement of computational units in the physical
world to provide the necessary computing capacity for Ubiquitous Computing
produced a set of _hydrologically_ inspired terms to describe different trade-
offs. The division of remote computing into cloud Soliman et al. (2013); Botta
et al. (2014); Fortino et al. (2014); Cavalcante et al. (2016); Singh et al.
(2016); Alam and El Saddik (2017); Bittencourt et al. (2018), fog (also edge
or grid) Bonomi et al. (2012); Aazam and Huh (2014); Satyanarayanan et al.
(2015); Shi et al. (2016); Chiang and Zhang (2016); Mao et al. (2017); Naha et
al. (2018); Stojmenovic and Wen (2014); Vaquero and Rodero-Merino (2014);
Giang et al. (2015); Cavalcante et al. (2016); Dastjerdi and Buyya (2016);
Aazam et al. (2018); Bittencourt et al. (2018); Sarkar et al. (2018); Hossain
et al. (2018); Negash et al. (2019); Alam et al. (2019), mist Liyanage et al.
(2016); Dogo et al. (2019), and dew Hong et al. (2013); Liyanage et al.
(2016); Rindos and Wang (2016); Ray (2017); Gushev (2020) is an attempt to
follow this principle (see Figure 13). While the analogy may have been taken a
bit too far, this simile is quite adept at equating the distance from the
observer to the water droplet and the distance from the device to the
computation location. At the same time, the high level of differentiation and
the unclear boundaries between these concepts—sometimes used
interchangeably—makes the discussion more complicated. For example, fog, mist,
and dew computing are often used interchangeably across papers, and edge and
grid computing often fit partial definitions of the hydrological terminology.

Another, less metaphorically laden, view focuses on _Computation-on-the-wire_
to describe the same phenomena Bhattacharyya et al. (2018). This perspective
assumes that data needs to be manipulated between the data generation point
(i.e., a Thing at a physical location) and the data consumption point (i.e., a
user inquiring about something related to the PT via its DT). To be the most
cost-effective, the computation is required to take place as close to the
generation and consumption points as sufficient and as far away as necessary.
Therefore, the computation-on-the-wire concept is essentially the application
of the subsidiarity principle. Taken from political science, subsidiarity
implies that a central authority should only perform tasks that cannot be
performed more locally Oxford Living Dictionary (2018). The principle is most
famously applied in the European Union and should be taken as the guiding
principle for IoT as well. On a more abstract level, it implies that actions
regarding an entity should be taken _a)_ as close as possible to an entity and
at the same time _b)_ as far as necessary to successfully execute the action.
Applied to Computation-on-the-wire and DTs, this principle ipmlies that
computations for DTs should occur _a)_ as close to a device, a requester, or
the PT as possible and _b)_ as far into the cloud as necessary to successfully
compute the output. The concept elegantly joins the hydrological simile and
the underlying trade-offs by applying the subsidiarity principle Oxford Living
Dictionary (2018).

####  4.3.8 Remote Sensing

> Remote sensing has two meanings. First, it provides observations recorded by
> a distant measurement device with some form of camera. Second, remote
> sensing refers to distant access to a local measurement device. Both can be
> processed to represent the physical environment around a CPS (such as a DT)
> and to possibly interact with the physical environment.

Remote sensing can be easily misunderstood as either exclusively meaning
distant measurements based on some kind of camera Campbell and Wynne (2011) or
using remote access to sensing units Viani et al. (2012); Sutaria and
Govindachari (2013) to measure local properties from afar such as in
agriculture Triantafyllou et al. (2019), healthcare Ruiz-Fernández et al.
(2017), smart cities Demetri et al. (2019), or everywhere Abdelwahab et al.
(2014). Traditionally, remote sensing exclusively refers to camera-based,
often airborne, approaches Campbell and Wynne (2011). However, the secondary
meaning of remote sensing as a shorthand for “remote access to local sensing
units” has become prevalent across nearly all applications that do not apply
traditional remote sensing. Whereas there is a clear difference in the
meanings of remote and sensing in both interpretations, recent applications
combining the two such as sensor networks on Unmanned Aerial Vehicles (UAV)
often linked to ground stations and the Internet Ismail et al. (2018); Fraga-
Lamas et al. (2019) may make the difference irrelevant in the long term. More
recently, for the context of the DT, the precise sourcing of information from
the Physical Environment is not important. Instead, the accuracy of the
measurements must be reported to allow for a decision-making process that
accounts for uncertainty. Throughout this review, we understand remote sensing
in both its meanings as the main source for information from the Physical
Environment. Furthermore, we assume that any form of traditional remote
sensing can be made available in real-time as a virtual sensor in an IoT
network Srivastava et al. (2005); Wang et al. (2014); Xiao et al. (2018).

##  5 The Fused Twins Concept

DTs require visualisation to unfold their potential. Naturally, it is the
Virtual Environment Grübel et al. (2021, 2016); Prouzeau et al. (2020) of a DT
through which information be accessed. To make sense of the different forms of
visualisations, we use the Reality-Virtuality-Continuum Milgram and Kishino
(1994), see Figure 14 to classify a broad range of human interface devices.

Figure 14: Devices to access Digital Twins on the Reality-Virtuality-Continuum
Milgram and Kishino (1994). On the Continuum, FTs span instances of Extended
Reality (XR) input modalities and media from Augmented Reality (AR) via Mixed
Reality (MR) to Augmented Virtuality (AV). Typical media include tablets,
phones but also holographic displays and smart objects. Reality-embedded
access to DT focuses on abstract dashboards whereas Virtuality-embedded access
focuses on remote analysis of the DT. The FTs Spectrum covers direct
interaction with the PT that is fused with augmentations from the DT either
displayed via AR on tablet, MR on HoloLens or haptically through AV smart
objects. Generated with http://draw.io.

Commonly used implementations of DTs make use of visualisation types on both
ends of the continuum which includes the Dashboard and Virtual Reality
(VR)-based digital models. Dashboards Batty (2015) reside on the Reality end
of the spectrum because they display information independent of the physical
world on screens which display numeric visualisations222Note that reality here
is the continuum instance in which the user is located and not the medium
through with the application is experienced.. The visualisation is not
embedded directly in reality and is accessed in a non-immersive context. On
the other end of the spectrum, remote monitoring Bednarz et al. (2020);
Spielhofer et al. (2021) is located in the Virtuality end and can be
implemented both as VR and non-local AR. The information usually has a digital
model that is displayed in the virtual space with no direct relations to the
physical space in which the data is consumed.

The FTs concept captures the situation where the DT is queried from within the
PT allowing for a fused _in situ_ representation. A media is used to connect
the PT with the DT through the display of Situated Analytics Fleck et al.
(2022). Possible media range from tablet-based (AR) and HoloLens (MR) to SOs
(AV). AR and MR devices can be used to overlay immersive analytics of the PT
with the DT. In contrast, SOs Sánchez López et al. (2012) are interactables in
the environment that allow users to manipulate the state of both the PT and
the DT.

Despite the similiarites, FTs are different from situated visualisations
Thomas et al. (2018). FTs require a fully-fleshed DT behind the situated
visualisation that includes all five types of environments and the ability to
compose the underlying DT with other DTs that may be spatially or logically
connected. FTs use the situated visualisation as an entry point to interact
with all systems that are connected to that particular point in the PT to
provide access to the whole PT through the DT. In contrast, many situated
visualisations focus on local applications such as a smart switches for IoT
devices Hua et al. (2021). However, at first sight, the situated visualisation
is often indistinguishable from FTs for end users. Nonetheless, continuous
interaction with the tool eventually reveals the limited functionality. Thus,
any situated visualisation can be expanded to implement the FTs paradigm by
integrating it into a DT model.

This section will outline the DT Requirements to implement the FTs paradigm.
The particular features of the main environments are discussed, but the Data
Environment and the Connection Environment are not discussed in detail. The
Connection Environment has not been implemented as interoperability between
DTs or their components is as of yet uncharted territory. Consequently, the
Data Environment is usually a proprietary implementation that requires no
further discussion here. An open standard for DTs would open up the research
on the Connection Environment and the Data Environment. Lastly, while
discussing the Virtual Environment, we reflect upon current situated
visualisations that are close to implement the full FTs concept.

###  5.1 The Physical Environment of the Digital Twin

Figure 15: Comparison of Digital Twin Architectures. Data flows from the
bottom (Thing) to the top (User). The functionality of each layer is displayed
in comparison to other architectures where possible by spawning each layer
vertically such that it contains all functionality that similar layers in
other architectures provide. For example, Presentation and Session in the OSI
Reference Model are subsumed in Application in the IETF LLN Stack. The labels
at the top provide the name of the models, and the labels at the bottom
references the origin of the respective model. _From left to right:_ OSI
Reference Model, Variation of IETF Internet Model Kurunathan et al. (2018),
Digital Twin Grieves and Vickers (2017), CPS Lee (2015), ETSI M2M Borgia
(2014) & SENSEI Tsiatsis et al. (2010), SoA Xu et al. (2014), 5C Lee et al.
(2015), and Context Aware Knappmeyer et al. (2013). Generally, management and
security columns spawning all layers are not shown. Our own definition is
mapped to these architectures in the last column. Generated with
http://draw.io.

The Physical Environment encapsulates the infrastructure to cast the digital
shadow Schluse et al. (2018) of the PT (see Figure 8) into the Data
Environment. When assembling an IoT or remote sensing system for a DT, it is
important to be able to determine the dimensions which matter most for the
functioning of the DT and choose an architecture accordingly. Most research on
architecture has been done in the context of IoT. However, as the time
progressed, architectures focused more on how to deal with the data and
consequently today, these architectures are also used in DT cognates and DTs
themselves.

International institutions have developed IoT reference architectures Weyrich
and Ebert (2016); Di Martino et al. (2018); Lynn et al. (2020). Architectures
are usually presented as layers. However, multiple architectures exist with a
similar number of layers but different functional goals. Going into detail of
how each layer is supposed to work sometimes contradicts other architectures
with the same number of layers (see Figure 15). On the one hand, there is no
unified architecture and due to the sheer endless number of application
scenarios across different branches of research. On the other hand,
architectures are sometimes compatible, and we provide a mapping of the
different components in Figure 15. These architectures may appear as fully-
fleshed DTs at first glance but on closer inspection will reveal themselves to
be mostly concerned with the Physical Environment and the Data Environment.
However, depending on the overall goal of the architecture, the upper layers
may actually contain features of the Analytical Environment and the Virtual
Environment. IoT architectures are the basis for most current DT applications
and will probably morph into fully-fleshed DTs in the coming decade.

Architectures aim to be somewhat self-explanatory and ideally each layer
covers exactly one functionality on transforming the data. Realistically,
layers help conceptually to assign tasks to different parts of the real
system. However, most implementations rely on a messier, less clean
interaction that takes place between the components. To choose the layer, it
is typical to consider aspects of security, energy consumption, data
collection density (spatial and temporal), maintenance, availability (real
time vs long-term evaluation), interactivity (explicit reaction versus
implicit reaction), and heterogeneity Flügel and Gehrmann (2009); Roman et al.
(2011); Rawat et al. (2016); Raza et al. (2018). Lastly, management of the
infrastructure and security concerns relate to every layer and are often
represented as vertical columns to the side of the layers Borgia (2014);
Atzori et al. (2010); Javed et al. (2018), but they are not shown in Figure 15
for brevity. When comparing this to our DT design, it is those management
layers that map to the Connection Environment that would ultimately allow to
compose DTs given appropriate permissions.

While the number of architectures is infinite, we will focus on a few of the
most common approaches in more detail here. The simplest architecture divides
the world into two layers, the Physical layer and some abstract layer such as
Digital Grieves and Vickers (2017) or Cyber Lee (2015); Alam and El Saddik
(2017). Mapping these realms to our DT model, the Physical covers the Physical
Environments and the Digital/Cyber covers the Data, Analytical, and Virtual
Environments. The Connection Environment maps most closely to the Management
and Security columns in previous architectures but the focus is more on
interoperability while also satisfying management and security requirements.
This architecture has no representation of how data is collected, transported,
transformed, and attributed in the DT. Nonetheless, it is a powerful vision
that is easily understood despite the technological intricacies necessary to
make such a vision possible. This perspective strengthens ontological
approaches to describe processes and qualities of objects in such a way that
allows for interaction. This concept is often applied in Industrial IoT (IIoT)
where it is usually used as an end-user perspective on a Service-Oriented
Architecture (SOA) Xu et al. (2014).

For practical implementations, often a three-layer architecture is used that
describes the detailed work in the middle Tsiatsis et al. (2010); Borgia
(2014). These architectures are already able to address how the DT arises and
as such a third layer is brought to attention: the middleware. The physical
world is sensed in a perception layer mapping to the Physical Environment.
Then, the data is structured into semantically meaningful units in the
middleware layer mapping to the Data Environment and the Analytical
Environment and finally presented to end-users in an application layer mapping
to the Virtual Environment. This architecture belongs to the earlier models
and is focused more on the process of how to create basic IoT infrastructure.
The perception layer describes the process of data acquisition usually
provided from sensors, actuators, and tags. The middleware layer aggregates
the data and provides APIs to access necessary information. When comparing the
three-layer model to the two-layer model, middleware is the hidden layer that
converts the real world’s PT into its DT. However, most real systems stop
short of complex analysis based on simulation and often limit itself to
present raw data or aggregates.

The application layer offers useful services on top of the data. In a sense,
those services are offered in the Virtual Environment of the DT (see Figure
2), but they heavily rely on the Analytical Environment to produce interesting
services. Naming conventions for the three layers vary, but their basic
functionality remains the same such as in ETSI M2M Borgia (2014), SENSEI
Tsiatsis et al. (2010). Additional layers always focus on very specific
features and mostly subdivided the Application layer for Service-oriented
Architecture Xu et al. (2014), Resource-oriented Architecture Guinard et al.
(2010c) business applications Lee et al. (2015), or Context Awareness
Knappmeyer et al. (2013).

###  5.2 The Analytical Environment of the Digital Twin - Models and Services

The Analytical Environment is often considered to be the DT itself Schluse et
al. (2018) because it simulates the PTs behaviour. However, throughout this
review paper, we have outlined more specific tasks for the Analytical
Environment such as running simulations, producing aggregations, and providing
services. We also have elaborated why a DT is more than just a simple
simulation with real-world data. Nonetheless, it is the Analytical Environment
that usually gives DTs and their Cognates the Smart label such as in Smart
City.

The word _smart_ generally allures to the superiority of a (connected,
digitised) Thing compared to its dumb (unconnected, analogue) counterpart.
Since connectivity and digitalisation on their own do not make anything
_smart_ but only provide accessibility, the smartness implies easy interaction
with the Thing, new insights into the Thing, and new applications of the
Thing. To attain any of these goals, models are required to represent reality,
predict the future, and monitor ongoing processes. For DTs, this means that
services De et al. (2011); Asghari et al. (2018) are provided on top of the
data through the Analytical Environment to enrich a PT’s functionality and
simplify interactions. In general, most Things will require a DT to become
smart.

However, the term smart has been miss-used. The previous description of
_smart_ is applicable to any meaningful service. The use of “ _Smart_ ” as a
prefix for services has proliferated to the point where it has become
superfluous. Following the above argument, the prefix has been attached to
anything that could have a DT or has implemented features that are
indistinguishable from a DT. In the simplest case, a Smart Object provides
Awareness, Representation, Interactivity, and Identity Kortuem et al. (2010);
Sánchez López et al. (2012). Smart Objects are concrete implementations of a
DT (and even the FTs paradigm), but they limit the Thing to be a small-scale
object with a fixed use. Smart Objects are also a form of data physicalisation
Dragicevic et al. (2020) that provides the opportunity to interact with data
beyond a visual medium.

As objects increase in size and become places, the terminology adapts to Smart
Environment Bonomi et al. (2014), Smart Home Hui et al. (2017); Laput and
Harrison (2019), Smart Office Laput and Harrison (2019), Smart Museum Chianese
and Piccialli (2014), Smart City Su et al. (2011); Suciu et al. (2013);
Soliman et al. (2013); Vlacheas et al. (2013); Zanella et al. (2014);
Krylovskiy et al. (2015); Rathore et al. (2016); Puiu et al. (2016); Hui et
al. (2017); Alavi et al. (2018); Kaur et al. (2020) and even Smart Regions
Matern et al. (2020) and Smart Communities Li et al. (2011); Sun et al.
(2016). Conceptually, these Smart Places Glaeser and Berry (2006) do not
differ from Smart Objects Sánchez López et al. (2012) but only generalise it
to sets of Things. This scaling aligns with the composability of DTs and
allows for these concepts to be nested.

However, there is a difference to be made between physical Things and non-
physical Things such as processes. The Thing in this context is a non-physical
entity but interacts with physical entities in a defined way allowing to
define smart processes such as Smart Governance Meijer and Bolívar (2016),
Smart Manufacturing Shrouf et al. (2014); Lu et al. (2020), Smart Construction
Kochovski and Stankovski (2018), Smart Health Care Services Rohokale et al.
(2011), Smart Mobility Faria et al. (2017), and Smart Experiences (for
touristic activities)Kontogianni and Alepis (2020).

With everything labelled smart, it becomes more meaningful to talk about the
kind of actions a Smart Thing can provide. Interestingly, it is these
interactions that are also sought after in the immersive analytics context
Marriott et al. (2018). We locate actions on a scale of how independently they
can react to external input:

  1. 1.

Sensing: Only produce data

  2. 2.

Reacting: Use threshold _on collected data_ to trigger action

  3. 3.

Interacting: Use _additional input_ beyond collected data to change behaviour

  4. 4.

Analysing: Use a _model to interpret collected data_ when choosing actions

  5. 5.

Proposing: Use a _decision model_ to propose actions based on data models

  6. 6.

Deciding: _Autonomously perform_ all necessary action to interact with data.

The action space of Smart Things is constantly expanded and we cannot
enumerate everything here. However, all of these developments can be
understood as the Analytical Environment of a DT. Research has covered many
approaches to provide the Analytical Environment needed to transform data into
insights (see Table LABEL:tab:iot:reserach:ongoing).

[!htbp] Research Areas for Digital Twin Analytics Domain Description Review
Ontologies Make semantic sense of the data and produce knowledge and context
Song et al. (2010); Hachem et al. (2011); Wang et al. (2012); Szilagyi and
Wira (2016); de Matos et al. (2020); Knappmeyer et al. (2013); Perera et al.
(2014) Middleware Infrastructure to identify, process, store, secure and
present information Ngu et al. (2017); Razzaque et al. (2016); Mineraud et al.
(2016); Fremantle and Scott (2017); Almusaylim and Zaman (2019) Data
preprocessing Data aggregation and data fusion Rajagopalan and Varshney
(2006); Wang and Liu (2011); Li et al. (2013); Zhang et al. (2010); Alsheikh
et al. (2014); Bagaa et al. (2014); Gravina et al. (2017); Al-Turjman (2017)
Data mining Cloud-based Big Data analysis on unstructured data Bin et al.
(2010); Tsai et al. (2014); Ganz et al. (2015); Chen et al. (2015); Ahmed et
al. (2017); Shadroo and Rahmani (2018); Ge et al. (2018); Hadi et al. (2018)
AI&ML Machine learning and deep learning Kulkarni et al. (2011); Alsheikh et
al. (2014); Al-Garadi et al. (2020); Atitallah et al. (2020) Blockchain
Security, trust, and privacy through smart contracts Christidis and
Devetsikiotis (2016); Zheng et al. (2017); Dorri et al. (2017); Khan and Salah
(2018); Mistry et al. (2020) HCI Human-Computer Interaction for improved
access Kranz et al. (2010); Koreshoff et al. (2013); Atzori et al. (2011,
2014); Chatain et al. (2020); Genay et al. (2021); Pei et al. (2022) ABM
Agent-based modelling Objects Vinyals et al. (2011); Wu et al. (2014); Kataria
(2021); Gath-Morad et al. (2020)

###  5.3 The Virtual Environment of the Digital Twin - Access, Interaction and
Representation through the Fused Twins Paradigm

The last environment we discuss provides the Human-Computer Interaction Jacko
(2012); Helander (2014) with the DT. When implementing the FTs paradigm for a
DT, the main focus lies on AR interactions with the DT. AR is only beginning
to arrive in the world of SOs, the IoT, remote sensing, and DTs but has
already been accepted as a crucial feature in the near future Chettri and Bera
(2019). AR is of interest for FTs as it explicitly supports embodied
interaction: interaction that is not agnostic to its physical and social
context Dourish (2004); Gervais et al. (2016a). AR can also reduce negative
split-attention effects Mayer and Moreno (1998); Sweller et al. (2011), and
appropriate visualisations of data can lower cognitive load Scaife and Rogers
(1996) and enhance cognition Willett et al. (2021). The expectation is that AR
will offer a new avenue to simplify interaction with SOs, the IoT, and DTs
Grübel et al. (2021). The FTs paradigm is a formalisation of the interaction
between AR and DTs in a situated context.

Because the combination of technologies is only emergent now, most published
material towards this end is either a prototype Jo and Kim (2016); Park et al.
(2019); Hua et al. (2021); Prouzeau et al. (2020); Grübel et al. (2021) or a
proposal for future research directions Michalakis et al. (2018); Carneiro et
al. (2018); Jo and Kim (2019); Chen et al. (2021). The main reason is that
applying AR on top of the already complex combination of SO, IoT, remote
sensing, and DT stacks requires many software systems to run smoothly that are
currently only being prototyped.

The Fusion of the the digital representation and the physical entity lies at
the core of the FTs paradigm Grübel et al. (2021) but also of the more general
Situated Analytics Thomas et al. (2018). Interestingly, other research on the
topic of Situated Analytics has often implemented FTs to some degree but
without the explicit mentioning of the DT aspect Jo and Kim (2016); Park et
al. (2019); Hua et al. (2021). A notable exception is the ‘Corsican Twins”
Prouzeau et al. (2020) implementation of a FTs platform with an explicit DT
powering the Situated Analytics. Short papers on future research directions in
Situated Analytics clearly point towards FTs-like implementations Michalakis
et al. (2018); Carneiro et al. (2018); Jo and Kim (2019); Chen et al. (2021).
We summarise ongoing and future research that is already covering a wide
variety of topics (see Table 5.3). Some of the research explicitly mentioned
the DT Approach or an Situated Analytics approach but often simply uses the
available technology to demonstrate that their problem can be addressed Madsen
and Madsen (2013); Chen et al. (2021); Carneiro et al. (2018), implicitly
developing a DT and applying Situated Analytics.

[!htbp] Recent and Ongoing Research Applying the Fused Twins Paradigm
Environments Application Type Year Physical Data Analytical Virtual Connection
Infrastructure Management Schall et al. (2009) 2009 * * - City Information
White and Feiner (2009) 2009 * - History Madsen and Madsen (2013) 2013 * - -
City Information (concept) Xu et al. (2014) 2014 * * * * * Shopping ElSayed et
al. (2015) 2015 * * - IoT device interaction Jo and Kim (2016) 2016 - * City
Information Zollmann et al. (2016) 2016 * * - Shopping Abao et al. (2018) 2018
* * - Smart Campus Dave et al. (2018) 2018 Evacuation Lochhead and Hedley
(2018) 2018 * - - - Medicine Pratt et al. (2018) 2018 - - Facility Management
Stojanovic et al. (2018) 2018 - Rehabilitation Lee et al. (2019) 2019 * - IoT
device visualisation Park et al. (2019) 2019 - - - Seat Selection Guarese et
al. (2020) 2020 * * - Equipment Maintenance Mourtzis et al. (2020) 2020 * - -
In situ visualisation Prouzeau et al. (2020) 2020 * * * Education Vidal-Balea
et al. (2020a) 2020 * Training & Assistance Vidal-Balea et al. (2020b) 2020 *
* - * Firefighting Chen et al. (2021) 2021 - - - General Grübel et al. (2021)
2021 * IoT device interaction Hua et al. (2021) 2021 - General Fleck et al.
(2022) 2022 * Note: * Partially implemented or insufficient information

Since the FTs concept in particular and DT concept in general is not well
defined yet, we analysed the literature for traces of it. To construct Table
5.3, we analysed the papers for the presence of implementations of any of the
five components. For the Physical Environment, we considered it as fully
implemented (i.e., we gave it a check-mark [ ] in Table 5.3) if active sensor
data collection was reported, or we considered it as partially implemented
(i.e., we gave it a star [*] in Table 5.3) if passive sensors or markers were
reported (e.g., QR code stickers). For the Data Environment, we considered it