# A Multi-Layout Design for Immersive Visualization of Network Data

David Bauer<EMAIL_ADDRESS>UC DavisUnited States ,  Chengbo Zheng
HKUSTChina<EMAIL_ADDRESS>,  Oh-Hyun Kwon<EMAIL_ADDRESS>UC
DavisUnited States and  Kwan-Liu Ma<EMAIL_ADDRESS>UC DavisUnited States

###### Abstract.

Visualization plays a vital role in making sense of complex network data.
Recent studies have shown the potential of using extended reality (XR) for the
immersive exploration of networks. The additional depth cues offered by XR
help users perform better in certain tasks when compared to using traditional
desktop setups. However, prior works on immersive network visualization rely
on mostly static graph layouts to present the data to the user. This poses a
problem since there is no optimal layout for all possible tasks. The choice of
layout heavily depends on the type of network and the task at hand. We
introduce a multi-layout approach that allows users to effectively explore
hierarchical network data in immersive space. The resulting system leverages
different layout techniques and interactions to efficiently use the available
space in VR and provide an optimal view of the data depending on the task and
the level of detail required to solve it. To evaluate our approach, we have
conducted a user study comparing it against the state of the art for immersive
network visualization. Participants performed tasks at varying spatial scopes.
The results show that our approach outperforms the baseline in spatially
focused scenarios as well as when the whole network needs to be considered.

network community structure, graph layout, head-mounted display, immersive
environment

††ccs: Human-centered computing Graph drawings††ccs: Human-centered computing
Information visualization††ccs: Human-centered computing Virtual reality††ccs:
Human-centered computing User studies

Figure 1. A view of a hierarchical network rendered with our system. A single
group is expanded. Its nodes and intra-group connectivity can be inspected in
detail while preserving inter-group context.

figure description

##  1\. Introduction

Network data represents relations of interest between entities in a domain
under study. It is commonly found in a variety of areas such as sociology,
computational biology, ecology, to epidemiology and many others. Building
effective visualizations is essential for the analysis and understanding of
network data. Many network datasets are quite complex and visualizing them is
often the only way to produce insightful analyses. Such networks contain a
myriad of nodes and links which themselves form group and sub-group structures
within the data. Group structures often arise from similarities in certain
node attributes or are formed based on strong connectivity in parts of a
network. Depending on the network’s semantics, these groups—often called
communities—can represent anything from circles of friends in a social network
to similar types of bacteria in a biological network.

Exploring the community structure of a network using the methods commonly
found in most graph drawing disciplines might yield sub-optimal results (Yang
et al., 2013; Vehlow et al., 2017). The community structure might not be
apparent, and conventional representations and interaction methods might not
suffice to provide deeper insight. For example, looking at a social network as
a conventional node-link diagram, where the nodes are simply connected by
lines, might not reveal the intricate interactions between social sub-
communities that are inherent to the data.

There have been efforts to cater to analysts’ needs by developing layout
techniques and visual de-cluttering methods that improve the legibility and
highlight the community structure of a network (Vehlow et al., 2017). However,
most of the methods are designed for 2D displays. For instance, a common task
for group-based network visualization is to find the number of neighboring
communities or to identify the group with the highest group-degree (connected
to the most other groups) (Saket et al., 2014). Such tasks require users to
navigate among multiple communities. In a 2D setting, users need to pan and
zoom frequently in large networks due to visual clutter.

Recent years have seen an advance in three-dimensional rendering and layout of
networks (Barahimi and Wismath, 2014). Particularly, stereoscopic displays and
head-mounted (HMD) virtual reality (VR) devices and similar immersive
technologies were successfully used to facilitate network analysis (Ware and
Franck, 1996; Ware and Mitchell, 2005, 2008). Although they rarely go beyond
simple node-link representations, these works pioneered the field,
underscoring the potential of such applications. More recent work has started
exploring in-depth the possibilities of network rendering in VR (Kwon et al.,
2016). With the devices that offered these new prospects came a new means of
interaction. Motion controls promoted the development of novel interaction
designs for virtual environments (Huang et al., 2017). Even in immersive
space, there is no single optimal graph layout or interaction method for all
tasks. Despite this fact, prior work has only been focused on rendering
networks using single, static layouts with minimal interaction, relying solely
on the perceptual advantages of immersive VR to justify their designs.

In our work, we introduce a multi-layout approach to visualizing network data
in VR. We created an interactive suite of graph layouts that helps solve a
wide variety of tasks and offers the cognitive advantages of immersive VR.
Using the space available in VR we are able to present, combine and juxtapose
multiple graph layouts—something that is not easily possible in screen-based
applications. This kind of approach caters to cognitive requirements that are
associated with analysing a network at different scopes. Layouts range from
overview summaries of the whole network to detailed projections of single
communities - all while visually preserving their context in the dataset. The
choice of layouts is informed by the idea of an extended overview-plus-detail
workflow. Each layout is designed to be explored from an egocentric
perspective. We use several previously established presentation techniques
(Kwon et al., 2016; Itoh et al., 2009; Didimo and Montecchiani, 2014; Holten,
2006; Munzner, 1997; Jacomy et al., 2014) to create effective and clutter-free
visualizations and our interactions let users engage with and navigate between
the different layouts.

We conducted a user study comparing our design against the state of the art
for immersive network visualization. The results of our evaluation show that
although our multi-layout approach might not be suited for all types of
community-based tasks, it outperforms the state of the art approach in the
majority of cases. Users achieve significantly higher accuracy in the same
time in spatially focused tasks as well as in scenarios that require attention
to the whole network. Our work is another pivotal step towards establishing
virtual reality as an effective means for information visualization.

##  2\. Related Work

Our work is related to the use of virtual reality for data visualization,
networks with community structures, interaction design and community-based
tasks. Here, we discuss existing studies related to these topics.

###  2.1. Virtual Reality

Ever since 1965 when Sutherland et al. have introduced the Ultimate Display
(Sutherland, 1965)—one of the first virtual reality (VR) devices—the world has
seen a multitude of devices and applications of VR. However, its uses have
long been confined to highly specialized research and industry applications
due to high cost of devices and development (Van Dam et al., 2000). Today, VR
has matured and is marketed heavily by the entertainment industry (HTC
Corporation, 2020; Sony Interactive Entertainment LLC., 2020; SAMSUNG
ELECTRONICS CO., LTD., 2020; Facebook Technologies, LLC., 2020) which has led
to the development of cost-effective and widely available devices.

With the recent advances, the scientific community has also embraced VR as a
means for data visualization, education, and training (Brooks, 1999). Early
adopters such as NASA have shown the value of using VR in wind tunnel
simulations (Bryson, 1993). Later on, it has been used for volume rendering
(Hanel et al., 2015; Scholl et al., 2018, 2019), trajectory analysis (Hurter
et al., 2019), diagnostic colonoscopy (Mirhosseini et al., 2015, 2019) and
even archaeology (Bennett et al., 2015), to name a few. We refer the
interested reader to Mariott et al.’s (Marriott et al., 2018) detailed
analysis of the potential uses of immersive visualizations for more examples.
This new domain of research is far from exhausted. As more and more
researchers engage with VR, it becomes clear what type of tasks and
representations work in VR and which ones do not (McIntire and Liggett, 2015).
While there exist several recommendations for how VR may be applied to data
visualization (Bryson, 1993; Bowman et al., 2004; Van Dam et al., 2000), there
are no unified guidelines to do so.

Table 1. Overview of existing work in the domain of immersive graph
visualizations. Works are shown in terms of used graph layouts and supported
tasks. Task names are based on Lee et al.’s taxonomy of graph tasks (Lee et
al., 2006) and Saket et al.’s work on group-level tasks (Saket et al., 2014)
and tasks found in cited work have been catorized to best match items in this
table. Red cells indicate task-layout combinations that have not been
discussed in prior work, while yellow highlights items that have been
considered but are not supported by our design. Hues of green indicate items
that are supported by our design. Layouts
---
|  | 2D Force-Directed Layout | 3D Force-Directed Layout | 3D Spherical Layout
Tasks | Low-Level | Ours, (Alper et al., 2011), (Halpin et al., 2008) | Ours, (Drogemuller et al., 2019), (Huang et al., 2017) | Ours
Adjacency | Ours, (Alper et al., 2011) | Ours | Ours, (Kwon et al., 2016)
Common Connection | Ours, (Alper et al., 2011), (Halpin et al., 2008) | Ours, (Huang et al., 2017) | Ours, (Kwon et al., 2016)
Connectivity | Ours | Ours, (Drogemuller et al., 2019), (Kotlarek et al., 2020), (Buschel et al., 2019), (Huang et al., 2017) | Ours, (Kwon et al., 2016)
Attributes on Nodes | (Halpin et al., 2008) | (Drogemuller et al., 2019), (Huang et al., 2017) | —
Follow Path | — | (Ware and Franck, 1996), (Ware and Mitchell, 2005), (Ware and Mitchell, 2008) | —
Revisit | — | (Kotlarek et al., 2020) | (Kwon et al., 2016)
Group Only Tasks | Ours, (Alper et al., 2011) | Ours | Ours
Group-Link Tasks | Ours | Ours | Ours

###  2.2. Network Communities and Tasks

There are many factors that can add complexity to a simple network. Nodes can
have a set of attributes, and edges can be weighted and/or directed. All of
these additional data can be used to infuse the network with more structure. A
common practice is to group nodes based on shared attribute values. The result
is a hierarchical structure—a community structure. Nodes are said to be part
of the same community if they share similarities in some attribute. The leaf
nodes in the hierarchy correspond to the nodes of the original network. The
higher-level nodes in the hierarchy are called meta-nodes, which form groups
by aggregating their lower levels.

Tasks performed on regular networks have been thoroughly categorized (Lee et
al., 2006; Amar et al., 2005). These taxonomies, however, do not concern
themselves with community-based networks. Saket et al. (Saket et al., 2014)
have compiled a taxonomy specific to community-related tasks. The tasks they
describe are the ones commonly used to evaluate visualizations of networks
with group structures. In other cases, it is important to identify communities
in networks rather than operating on them. Fortunato et al. (Fortunato, 2010)
give an exhaustive overview of this topic. Greffard et al. (Greffard et al.,
2015) have investigated the potential of stereoscopy while Cordeil et al.
(Cordeil et al., 2017) have compared different types of immersive technology
for these types of tasks. Although this proves the potential of VR in that
area, we limit ourselves to the type of tasks formerly described (Saket et
al., 2014), where communities are already established.

###  2.3. Community Drawing

Drawing networks with complex community structures poses a challenge. General
networks are traditionally drawn using force-directed layouts (Kobourov,
2012). These common methods often fail to communicate the community structure
of a hierarchical network. There have been efforts to develop layouts to
accommodate a hierarchical network (Vehlow et al., 2017). Several studies,
such as Johnson et al. (Johnson and Shneiderman, 1991), Bourqui et al.
(Bourqui et al., 2007), Itoh et al. (Itoh et al., 2009) or Didimo et al.
(Didimo and Montecchiani, 2014), have introduced layout methods for the
disjoint, hierarchical type of networks described by Vehlow et al. (Vehlow et
al., 2017). The proposed methods emphasize the community structure by either
leaving out some edges and trying to maximally utilize the available space
(Johnson and Shneiderman, 1991), by using recursive space division to group
similar nodes using multi-level force-directed layouts (Bourqui et al., 2007),
or by combining different methods to layout the graph and communities
separately (Itoh et al., 2009). Others have focused on improving performance
of existing approaches (Didimo and Montecchiani, 2014).

Another challenge in network visualization is handling a large number of
edges. A common technique to overcome visual complexity is edge bundling.
Edges will appear as bundles in parts where they share similar trajectories. A
number of authors have developed ways to achieve this bundling effect (Phan et
al., 2005; Ersoy et al., 2011; Holten, 2006; Holten and van Wijk, 2009).
Holten (Holten, 2006) has proposed a method to apply edge bundling to
hierarchical networks by taking advantage of common meta-nodes among leaf
nodes that share the same attributes. We deal with networks that are
inherently hierarchical, however rendering the network in 3D requires special
considerations for occlusions, layout, and projections as shown by Kwon et al.
(Kwon et al., 2016). In this project we adapt some of the layout as well as
bundling techniques (Kwon et al., 2016; Holten, 2006; Itoh et al., 2009;
Didimo and Montecchiani, 2014) to de-clutter the network in VR space.

###  2.4. 3D Network Visualization

Visualizing abstract data in three dimensions bears many problems. Although
there have been attempts of visualizing networks in 3D (Robertson et al.,
1991; Rekimoto and Green, 1993; Munzner, 1997; Young, 1996), they invariably
suffer from occlusion and the need for additional depth cues (Herman et al.,
2000) when displayed on two-dimensional viewing devices (such as a standard
computer monitor or display wall). This is why the topic has not received as
much attention in recent years. The depth cues available to 3D renderings
projected to 2D space are limited to occlusions, motion parallax and shading
(Ware, 2019; Cutting and Vishton, 1995). However, the introduction of
stereoscopic viewing and VR has brought new possibilities to this field. These
new technologies allow users to freely move about and explore the data. Most
importantly, immersive displays offer an additional depth cue: binocular
disparity. McIntire et al. (McIntire and Liggett, 2015) have shown that this
can improve task performance in visualization tasks. Network visualizations
are particularly suitable, as improved task performance was observed in
multiple studies (Ware and Franck, 1996; Ware and Mitchell, 2005, 2008). Aside
from low-level tasks like the ones that the former authors investigated,
community identification tasks have also been shown to benefit from
stereoscopy (Greffard et al., 2015).

Works in this area have since matured and explored more ways to handle
networks in three-dimensional space. Kwon et al. (Kwon et al., 2016) have
analyzed different projection methods for spherical layouts to maximize space
utilization in VR. Büschel et al. (Buschel et al., 2019) have compared the
suitability of different edge styles in AR. Others have investigated
possibilities of novel interaction design for those environments (Alper et
al., 2011; Kotlarek et al., 2020; Huang et al., 2017; Halpin et al., 2008;
Sorger et al., 2019). In terms of interaction, our work is most closely
related to the works of Alper et al. (Alper et al., 2011), Kotlarek et al.
(Kotlarek et al., 2020), and Sorger et al. (Sorger et al., 2019). Their
proposed interaction techniques have inspired us to build on top of the ideas
of overview and detail and stereoscopic highlighting. Huang et al. (Huang et
al., 2017) have explored the feasibility of gesture controls in virtual
reality. Different navigation interactions have been compared by Drogemueller
et al. (Drogemuller et al., 2019). Both the gestures and navigation paradigms
lend themselves excellently for more complex interactions or for systems that
feature a broad array of simultaneously available functions.

Figure 2. (a) The network is shown as a miniature floating in front of the
user. The H3 hyperbolic layout (Munzner, 1997) is used to layout the nodes.
(b) In the spherical view, the network is projected onto a large sphere. The
view is scaled to fill the horizon and take up about 180 degrees of the user’s
field of view. (c) Selected communities descend from the spherical layout for
the user’s close inspection. The group expands, and the rendering and layout
of the community changes to reveal more detail about intra-community
connections while emphasizing its place in the context of the whole network.
(d) Expanded communities can be projected to the floor to lay bare details of
intra-community connectivity that might not be apparent in the three-
dimensional view.

##  3\. Methods

Virtual reality (VR) allows for intuitive exploration of 3D space. We leverage
the additional depth cues to present the data to the user. However, no graph
layout method is universally applicable for all possible tasks, even with
these perceptual advantages. We present a multi-layout approach that lets
users interactively adapt the layout of a hierarchical network and its
communities as she sees fit for the task at hand. Specifically, we propose a
suite of four layouts that maximize the utility of available space in VR. To
the best of our knowledge and in contrast to prior work on immersive network
visualization, our novel visualization design is the only one to support a
broad variety of tasks and layouts (Table 1).

###  3.1. Multi-Layout Suite

Our system comprises four views. Each view shows all the data but emphasizes
different aspects of the network and is intended to support different tasks.
Prior work is limited to single layouts or simple overview plus detail
concepts, supporting only a limited number of tasks. With our approach, we
hope to bridge the gaps and provide users with a system that facilitates
efficient graph exploration and task completion. For our design, we picked
four views that are orthogonal in their utility to that end. Each layout
reveals a different aspect of the network, and the flow between them is
organized by level of detail, ranging from a broad overview down to a layout
that allows detailed inspection of single nodes and links. This workflow
expands on the idea of overview plus detail by breaking it up into multiple
levels. In addition to that, the choice for these four layouts and the drill-
down workflow was informed by available task taxonomies (Lee et al., 2006;
Saket et al., 2014) which suggest the need for attention on multiple levels of
detail–especially for networks with group structures. With this suite of
layouts, we expect users to be able to efficiently solve such tasks. Table 1
highlights the capabilities of our systems and terms of graph layouts and
supported network tasks. It puts our contribution into perspective with prior
work in network analysis.

To get a general impression of the data, a miniature overview of the network
is placed in front of the user (Figure 2 (a)). After an initial inspection of
the network, the user can expand this view into a larger spherical projection
that stretches the horizon and makes optimal use of the available space
(Figure 2 (b)). In this view, communities and their general connectivity
become more apparent. The user might decide to explore one or more of them in
more detail. Selecting communities will expand its nodes into a separate
three-dimensional force-directed layout that descends to the user for closer
inspection (Figure 2 (c)). Finally, this floating layout can be projected to
the ground below the user, where it further expands into a flat 2D layout
taking up most of the user’s field of view (Figure 2 (d)).

In conjunction, these layouts constitute a continuous workflow that takes the
user from a high-level view of the whole network down to the level of single
nodes within a community. In the following sections, we discuss each layout
and their interactions in more detail. Each section is accompanied by a real-
life use-case scenario based on one of our datasets (Watts and Strogatz,
1998). The data represents the nervous system of a C. Elegans worm encoded as
a network. Each node represents a neuron. Edges indicate functional
connections between those neurons. We show the workflow of finding neurons
with strong and diverse inter-community connectivity. For a visual tour
through this workflow, please refer to the supplemental video.

###  3.2. Overview Layout

The overview is meant to convey a general impression of the structural
properties of the hierarchical network (Figure 2 (a)). We use the H3
hyperbolic layout (Munzner, 1997) to calculate node positions. The layout was
chosen as it is especially suited for visualizing large networks with inherent
group structures which is the type of network we use in this project. Links
are drawn as straight lines, and color is used to encode communities. The
network is scaled down to fit the user’s view comfortably and is placed in
front of her at about shoulder’s height in VR space. As this view is meant to
introduce the viewer to the dataset, the placement and scale were chosen such
that she can freely walk around the network. The experience is similar to
looking at a museum art piece in a display case from all angles. At the touch
of a button on the motion controller, the miniature expands into a larger
spherical layout.

Use Case The user identifies the overall structure of the neural network at a
glance. Color coding of communities allows her to quickly identify 13 major
functional groups of the nervous system. She is also able to see that some
groups are more densely connected than others.

###  3.3. Spherical Layout

Virtual reality environments offer a vast space that surrounds the user. This
layout facilitates this space to the highest degree. Spherical projections
have been shown to be a suitable method of visualizing a network in virtual
reality environments (Kwon et al., 2016). They use the available space by
placing the user in the center of the projection and therefore providing an
equidistant placement of elements in a network in relation to the user’s point
of view. Aside from this factor, the field of view (FOV) utilized by the
projection can impact user performance and comfort. Note that we use the term
FOV to mean the horizontal and vertical angular space taken up by the layout
on the sphere. The FOV of the human eye (i.e., the angular range perceptible
to us at any given point in time) is fixed for each individual. By varying the
FOV, the user’s level of immersion ranges from seeing the network just in
front of oneself (low utilization of the sphere, low FOV) to being completely
enveloped by elements of the network (full utilization of the sphere, high
FOV).

Limiting the FOV can reduce exaggerated head movement and increase task
performance and comfort, as shown in previous studies (Kwon et al., 2016;
Youdas et al., 1992). Therefore, we decided to fixate the FOV to a value of
just under $180^{\circ}$ both horizontally and vertically.

The spherical layout constitutes only part of the technicalities since it
merely projects an exiting 2D layout onto a sphere. The base layout can also
affect the outcome visually. The one we chose is derived from the works of
Itoh et al. (Itoh et al., 2009) and Didimo et al. (Didimo and Montecchiani,
2014), who both investigated layout methods for hierarchically clustered
networks. These networks are the same type of community-based networks that we
are treating. Communities are groups of leaf nodes in a hierarchically
clustered network. Nodes that share the same parent are said to be in the same
community.

To calculate the clustering, we use the Louvain method for community detection
(Blondel et al., 2008). For the layout, we employ much of what Itoh et al.
(Itoh et al., 2009) and Didimo et al. (Didimo and Montecchiani, 2014) have
already proven to be effective. We first compute a node hierarchy with the
aforementioned technique (Blondel et al., 2008) which is followed by a
combination of a force-directed layout (Jacomy et al., 2014) for the group
level and a treemap layout for the overall placement of communities. The
resulting layout spans the horizon around the user (Figure 2 (b)) and edge
bundling visually emphasizes inter-community connection patterns.

Use Case After gaining an overview of the data, the user expands the layout
into the spherical view. She identifies several candidate groups that stand
out due to their strong bundles of edges. These groups might play a central
role in the worm’s nervous system.

###  3.4. Floating Community Layout

In its initial state after expansion, the network and all of its communities
are subject to the spherical layout described above. Upon selecting a
community of interest, its layout changes from the treemap-based hybrid
technique to a separate, three-dimensional, force-directed layout (Figures 1,
2 (c), 3 (b), (d), (e)). In contrast to the overall network structure, the
selected community’s layout is calculated and updated continuously as it
descends from the spherical projection closer towards the user. Dynamically
updating the layout adds aesthetic value through fluidity in motion and smooth
transitions from the group’s initial state to its final constellation. Aside
from aesthetics, it offers a sense of continuity. While the new layout
unravels, the user sees the nodes and their connections in different
constellations. This aids the user’s understanding of the intra-group
structure. Furthermore, the user can adjust the layouts arrangement—moving
nodes around to get a better view of parts of the community that might be
occluded. Links within the community change their style to straight edges and
are rendered in clear white color. Similarly, links that connect outward to
other communities are emphasized too. Connections unrelated to that particular
group are subdued.

Use Case In this view, it is easy to discern neurons with strong external
connectivity. By expanding multiple groups simultaneously, the user can find
nodes that share a connection, make judgements about the interplay of neuron
clusters, and identify communities that connect to a large number of other
clusters.

###  3.5. Projected Community Layout

While inspecting a floating community, the user might want more information
about the connections within a group. Depth cues available in VR can help
understand a community’s structure, but there are cases where a traditional 2D
projection reveals additional details. For such cases, the user can project
the group of nodes down to the ground below her. The community changes to a
flat, horizontally arranged, force-directed layout (Jacomy et al., 2014) that
fills the space below the user’s position (Figure 2 (d)). It can be viewed
comfortably by looking down. Similar to the floating layout, intra-community
links are pronounced. Inter-community links and unrelated connections still
connect to the spherical layout above but are both greatly subdued, enabling
the user to focus completely on the single community.

Use Case The user invokes the projected layout for a strongly connected
community to spread it out across the floor. Here she has enough space to
closely inspect all the nodes one by one. She identifies several nodes that
have strong inter-community connectivity and records their names.

###  3.6. Interaction

Although interaction design is not a focus of this work, it was important for
us to aim for high usability. Part of our design goals was to provide a set of
interactions that are easy to execute yet effective at aiding the user in
exploring the visualization. We decided to use the motion controls provided by
the HTC Vive (HTC Corporation, 2020). These controls allow developers to
implement natural interactions like grabbing, pushing or pulling. In this work
we deliberately decided against providing users with such gesture-based
interactions. On the one hand, this was done to keep required training to a
minimum. All the interactions in our system can be triggered by pressing a
small set of buttons on the controllers. The other main reason for this
decision was that we wanted the results of our study to be comparable to
previous network visualization work in VR (Alper et al., 2011; Kotlarek et
al., 2020; Sorger et al., 2019; Kwon et al., 2016), where inputs were also
mostly realized with simple button presses. In the system, a ray is cast,
originating at the controller’s position and extending towards where the user
points with the controller. Previous studies have made successful use of
similar techniques to enable interaction in immersive network visualizations
(Kotlarek et al., 2020; Huang et al., 2017). They enable users to point at and
select objects, which is the main focus of our design. Much of the user’s
interaction centers around selecting and inspecting nodes, communities, and
their connectedness. With these things taken into consideration, we offer the
following interactions:

  * •

Expand Network \- Transitioning from the overview to the spherical layout
(Figure 2 (a), (b)).

  * •

Highlight Community \- Pointing at a community with the ray. The community
will be highlighted by a colored circle (Figure 3 (a)).

  * •

Expand Community \- Expanding a highlighted community by pressing a button.
The community will descend towards the user’s position stopping right in front
of her (Figures 1, 2 (c), 3 (b), (d), (e)).

  * •

Project Community \- Projecting an expanded community will change it to a flat
force-directed layout. The layout descends beneath the user taking up the
available space there (Figure 2 (d)).

  * •

Reset Community \- Resetting an expanded community back to its condensed
state. The community will float back to its original position. Nodes in the
community will contract, and the rendering style of intra-community edges will
revert back to curved splines.

  * •

Highlight Node \- Nodes can be highlighted much like the way communities are
highlighted. The node and all of its edges will change color to emphasize its
relation to other nodes and communities (Figure 3 (c)). This applies to all
layouts so that highlighted links will always stand out even if their
neighbors are heavily subdued.

Figure 3. Some of the interaction techniques in our system. (a) Highlight
Community. (b) Expand Community. (c) Highlight Nodes. (d) Effect of expanding
a single community. Links between communities are emphasized while the rest of
the network is subdued. (e) Effect of expanding multiple communities. Similar
effect to (d). Links between expanded communities are rendered straight.

###  3.7. Edge Bundling

The large number of edges that we display in our system and, more-so, the
unique community structure that these networks exhibit demand special
considerations for rendering. Hierarchical edge bundling has been successfully
used in prior work to effectively improve the legibility of hierarchical
networks (Holten, 2006). Recently, these techniques have been expanded to
accommodate bundles in spherical layouts (Kwon et al., 2016).

Our approach to bundling is inspired by previous work (Kwon et al., 2016;
Holten, 2006). However, these methods do not satisfy the need to distinguish
between intra-community edges and inter-community edges. In our design, edges
between nodes in the same community are bundled locally. On the other hand,
edges that connect nodes in two different communities are rendered using the
previously proposed hierarchical edge bundling techniques (Holten, 2006; Kwon
et al., 2016). We vary bundling strength depending on the currently active
layout. Bundling strength ranges from $0.0$ to $1.0$. In the overview, edge
bundling is disabled ($0.0$ bundling strength). The spherical view uses
moderate bundling strength ($0.7$) to reduce visual clutter and emphasize
overall connectedness between communities. When communities are expanded,
their intra-community links are rendered as straight-line edges since their
number is limited enough to still provide a good overview (Yoghourdjian et
al., 2018). Other links in the remaining graph stay bundled ($0.7$ bundling
strength). Upon invoking the projected community layout, bundling strength in
the main graph is increased ($0.9$ bundling strength) to remove clutter
further and help shift the focus to the projected group of nodes. In this
view, intra-community links are again rendered as straight lines.

###  3.8. Rendering

For rendering nodes, we use GPU instancing. This can effectively reduce the
number of draw calls by magnitudes. Edge positions and subsequent spline
interpolations are calculated using compute shaders. Edge drawing is done by
procedural draw calls available through Unity3D. The compute shader results
are shared in GPU memory and are therefore available for drawing without
having to transfer the data back to the CPU. To improve performance further,
all edges are rendered in a single draw call. We use a regular shader to
access compute shader results and generate edge geometry for the entire scene.
To reduce the number of vertices, we employ billboarding so that each line
segment requires only two triangles.

Figure 4. The largest network in our tests. It contains 2646 nodes and 10455
edges.

The largest network in our tests consisted of 2646 nodes and 10455 edges
(Figure 4). We achieved real-time frame-rates in VR while providing full
interactivity.

##  4\. User Study

The purpose of this study is to investigate the utility of our multi-layout
approach to immersive visualization of network data. Previous studies have
already shown that people tend to perform better in graph tasks when working
in immersive environments as compared to 2D or even monoscopic 3D environments
(Ware and Franck, 1996; Ware and Mitchell, 2005, 2008; Kwon et al., 2016;
Kotlarek et al., 2020). However, these studies only consider single, static
graph layouts with minimal interactivity. In this study we examine the effects
of multiple, interactively connected layouts to perform such graph tasks in
immersive environments.

###  4.1. Study Design

We conducted a study with 20 participants to investigate how our multi-layout
system performs against the state of the art of immersive network
visualizations (Kwon et al., 2016). To construct a baseline and ensure
comparability, we chose a subset of features from our system that most closely
represents the functionality of prior work (Kwon et al., 2016). The experiment
was designed as a between-subject study that had participants complete several
network-related tasks. There were two conditions in total. Each participant
completed 1 condition $\times$ 3 networks $\times$ 5 tasks. Networks were
presented in ascending order of complexity. The sequence of tasks was
randomized among subjects. We recorded accuracy, time to completion, total
number of interactions, and separate counts for each type of interaction.

####  4.1.1. Visualization Conditions

For this study, we considered two conditions:

  * •

MULTI \- Multi-layout suite in virtual reality. Displayed on a stereoscopic
head-mounted display.

  * •

BASE \- State-of-the-art in immersive network visualization (Kwon et al.,
2016) which offers a treemap-based spherical layout for the whole network and
a detail view of individual groups.

For both conditions, we used the same basic settings in our system. These
includes values like edge bundling strength, edge opacity, or animation speed
within and between different layouts.

The MULTI condition offers all the features described earlier while the BASE
condition is modeled after Kwon et al. (Kwon et al., 2016) who offered a
treemap-based spherical layout in which single groups could be brought closer
to the user by selecting them. Our BASE condition offers this same set of
capabilities.

Both conditions use the same underlying hybrid layout technique for the
spherical view, which is a combination of a treemap-based layout for the
placement of communities and a force-directed approach (Jacomy et al., 2014)
on the group-level. The floating layout also uses force-directed methods, each
appropriate to the spatial dimensions they are shown in.

Some tasks require us to visually highlight nodes or groups of interest. Such
targets are marked by in a distinct, colored indicator in both conditions. In
tasks involved communities, a red circle is placed around those groups. For
tasks that require the attention to single nodes, we use a red fill color
which is easily distinguishable from regular colors used in the rendering. In
addition, nodes are also emphasized through a small halo of that same color
around their perimeter.

####  4.1.2. Network Data

We used three different networks for the study with an additional fourth
network for initial training sessions. The networks are shown in ascending
order of complexity. For the purpose of this study, we label these different
levels of complexity as easy, medium, and hard. The network used for training
is equivalent to the easy graph in terms of complexity. The exact metrics are
shown in Table 2.

Table 2. Network complexities in terms of number of nodes, number of edges, number of communities, and the maximum hierarchical depth of the graphs used in this study. | Nodes | Edges | Communities | H. Depth
---|---|---|---|---
Easy | 115 | 613 | 12 | 2
Medium | 297 | 2148 | 13 | 3
Hard | 1133 | 5451 | 25 | 3

####  4.1.3. Tasks

Our tasks are based on taxonomies proposed earlier (Saket et al., 2014; Lee et
al., 2006; Amar et al., 2005) to maximize comparability among similar studies.
In our study, we categorize them based on the domain the user has to act upon.
We therefore define the following task categories:

  * •

Intra-Community \- Task is performed in the context of a single community.
Based on Lee et al. (Lee et al., 2006), Amar et al. (Amar et al., 2005), and
Saket et al. (Saket et al., 2014).

  * •

Inter-Community \- Task has user focused on elements in multiple communities.
Based on Saket et al. (Saket et al., 2014)

  * •

Epi-Community \- Task performed by looking at the whole population of
communities. Based on Saket et al. (Saket et al., 2014)

Building upon the established taxonomies (Saket et al., 2014; Lee et al.,
2006; Amar et al., 2005) and our newly introduced categories we, chose the
following five tasks for this study.

  * •

CommonNeighbors \- Find Common Neighbors. In a given community, find the nodes
that are shared neighbors between two nodes. (intra-community)

  * •

CountNodes \- Count Nodes in a Community. Given a community, count the number
of nodes in it. (intra-community)

  * •

MaxDegreeNode \- Find Maximum Degree Node. Given four nodes in the network,
find the one with the highest degree. (inter-community)

  * •

MaxDegreeCommunity \- Find the Maximum Degree Community. Among all
communities, find the one with the highest number of outbound links. (inter-
community)

  * •

CountCommunities \- Count number of neighboring Communities. Given a
community, count the number neighboring communities. (epi-community)

Our choice of tasks was informed by two considerations. The majority of tasks
should be specific to networks with group structures. Such tasks were adopted
from the domain-specific task taxonomies mentioned above (Saket et al., 2014).
In addition to that, we wanted to establish a link to prior work by including
some tasks from Kwon et al.’s study (Kwon et al., 2016) for the purpose of
comparability.

###  4.2. Hypotheses

Based on our task categories and related work, we defined the following
hypotheses:

  * •

H1 \- Our MULTI design outperforms the BASE version in intra-community tasks.

  * •

H2 \- Our MULTI design does not outperform the BASE version in inter-community
tasks.

  * •

H3 \- Our MULTI design outperforms the BASE version in epi-community tasks.

We justify our choice for H1 and H3 based on the affordances that our layout-
suite offers. The projected community layout allows users to gain more
detailed insights into intra-community constellations, while the overview
layout offers an at-a-glance view of the data, which we expect to be conducive
to solving such tasks. For H2, we suspect that the split attention that is
required to solve inter-community tasks outweighs the benefits of having
multiple views of the data (Miller, 1956; Sweller et al., 1998; Kotlarek et
al., 2020).

###  4.3. Apparatus

Both conditions were implemented in Unity3D. We used a HTC Vive (HTC
Corporation, 2020) head-mounted display device (HMD) with both of the included
motion controllers. The application was rendered on an NVIDIA GeForce RTX 2080
TI graphics processor and maintained a smooth framerate of about 78 frames per
second. Participants were standing in both conditions to allow for more
freedom of movement. The space available for them to move around in was about
60 square feet.

###  4.4. Participants

For the study, we recruited 20 participants (12 female, 8 male). Ages ranged
from 18 to 34. None of the participants were colorblind. Eight wore glasses,
and three wore contacts. The other nine participants reported no visual
impairments. The design of the HMD allowed participants to leave on their
glasses when wearing it. We adjusted the fit and ocular distance of the HMD
for each participant to meet individual needs.

Four rated themselves as experienced with network data, nine reported moderate
knowledge, six had little knowledge and one had never worked with graphs
before. As for VR, three participants were experienced users, while eight had
at least tried VR before. Nine participants had never used this technology, of
which one had never heard of it.

###  4.5. Procedure

The group of participants was randomly divided into two equally-sized
subgroups of which one performed the study in the BASE condition while the
other subgroup used the MULTI version. Before the start of the study, each
person was informed about the possible adverse effects of using a head-mounted
VR device. These include cybersickness, vertigo, eye strain, or
disorientation. They were told that they could pause or exit the study at any
point in time if they felt uncomfortable for any reason. Due to recent public
health events, all equipment involved in the study was thoroughly disinfected
between each use, proper social distancing was practiced and face masks were
worn at all times.

####  4.5.1. Questionnaire

Participants were asked to fill out a preliminary questionnaire on
demographics and prior experience with networks and virtual reality. In-
between tasks, participants were required to answer questions about their
experience through a set of NASA TLX (Hart and Staveland, 1988) questions.
After the completion of all the tasks, participants answered several questions
to summarize their experience in the experiment. This also gave them the
opportunity to provide some unstructured feedback.

####  4.5.2. Protocol

Each participant was allowed to verbalize their actions and thoughts during
the completion of tasks. This was limited to a degree that would not impact
the user’s performance. We created audio recordings to capture the dialog
between the conductor and participants and ”think-aloud” events during the
experiment.

####  4.5.3. Training

Before engaging in any real tasks, participants were instructed to complete a
short training. It had users perform the same types of tasks found in the real
experiment on a training network. This network was less complex than most of
the ones used in the experiment and served to familiarize the participants
with the nature of the tasks, the interactions and the layouts used in each
condition. They were able to ask questions to clarify any issues and get
feedback about the correctness of each task.

####  4.5.4. Experiment

In the experiment, participants were asked to complete the same kind of tasks
that they knew from the training. The tasks were performed on three different
networks. The networks were given in ascending order of complexity. The task
order was randomized between subjects. No feedback about the correctness of
the participant’s answers was given. Participants were allowed to ask the
conductor for information from the training slides. After the completion of
each task category, participants were asked to provide feedback concerning
this specific type of task. Responses were provided verbally and recorded by
the experiment conductor as to not disrupt immersion and save time on the
subsequent execution of further tasks.

##  5\. Results

Null hypothesis testing (NHT) has been under some criticism in different
fields due to its limitations in reporting evaluation results (Harlow et al.,
2016; Cumming, 2014). We therefore provide confidence intervals (CI) and
effect sizes in addition to traditional NHT results. Cohen’s $d$ was used to
calculate effect sizes. Quantitative results, for the most part, pass
normality tests and, considering the relatively low sample count, are assumed
to be normal distributed. CIs were taken from the respective t-distribution.

Overall, our results show that MULTI outperforms BASE in terms of accuracy for
the majority of tasks. Time to completion was comparable in both conditions
for most tasks. This means that users were able to perform tasks more
accurately in the same time when using our multi-layout approach. Both
CommonNeighbors and CountNodes were significantly better in the MULTI
condition, confirming H1. The inter-community tasks did not show any
significant difference, which confirms H2. Lastly, H3 is confirmed by the
significantly better performance in CountCommunities for MULTI. User feedback
indicates a general preference for MULTI with criticism mostly directed
towards the positioning of graphs in space. In this section, we highlight the
most important findings. For more data and analyses, please refer to the
supplemental material.

###  5.1. Intra-Community Tasks

Intra-community tasks (Figure 5 (a), (b)) are locally focused and do not
require users to adjust their view much. We saw significantly better overall
accuracy in the MULTI condition for CommonNeighbors (Cohen’s $d=1.467$,
$p<0.01$) and CountNodes (Cohen’s $d=0.659$, $p<0.05$). For CommonNeighbors,
overall accuracy in BASE was $0.478$ ($SD=0.275$) and $0.878$ ($SD=0.262$) for
MULTI while CountNodes results show $0.869$ ($SD=0.173$) for BASE and $0.957$
($SD=0.069$) for MULTI. Time to completion was not significantly different in
either condition with Cohen’s $d=-0.074$, $p=0.775$ and $d=0.134$, $p=0.605$
and with averages of $136.821$ ($SD=105.114$) and $68.156$ ($SD=48.198$) for
BASE and $129.789$ ($SD=79.94$) and $75.221$ ($SD=54.934$) for MULTI for
CommonNeighbors and CountNodes respectively.

###  5.2. Inter-Community Tasks

These tasks (Figure 5 (c), (d)) require users to look around and adjust their
view more frequently. For this category, we saw no significant difference in
accuracy in either of the tasks. For MaxDegreeNode, Cohen’s was $d=0.155$,
$p=0.549$ and averages were $0.734$ ($SD=0.442$) for BASE and $0.8$ ($SD=0.4$)
for MULTI. MaxDegreeCommunity had a Cohen’s $d=0.382$, $p=0.145$ and averages
of $0.433$ ($SD=0.496$) for BASE and $0.622$ ($SD=0.477$) for MULTI.
Completion times for MaxDegreeNode were significantly higher in MULTI (Cohen’s
$d=1.339$, $p<0.01$) with averages of $65.292$ ($SD=38.204$) and $136.325$
($SD=63.089$) for BASE and MULTI. There was no difference in completion time
for the MaxDegreeCommunity tasks with Cohen’s $d=0.051$, $p=0.843$ and
averages of $99.733$ ($SD=113.098$) and $104.7$ ($SD=72.932$) for BASE and
MULTI.

###  5.3. Epi-Community Tasks

Participants need to inspect the whole network in this task (Figure 5 (e)).
Overall accuracy was significantly higher in MULTI (Cohen’s $d=1.091$,
$p<0.01$) with averages of $0.744$ ($SD=0.202$) for BASE and $0.923$
($SD=0.106$) for MULTI. Completion times were comparable in both conditions
(Cohen’s $d=0.362$, $p=0.167$). The overall averages were $73.619$
($SD=52.338$) and $102.272$ ($SD=96.984$) for BASE and MULTI.

Figure 5. Performance results for all tasks given as the mean over all
participants. The error bars denote 95% CIs. Tasks are aligned in columns
ranging from intra-community (a) and (b), over inter-community (c) and (d) to
epi-community (e). The top row shows values for accuracy while the bottom
lists task completion time in seconds. A star in the legend marks a
significant difference in the overall results. Figure 6. User’s assessment of
the tasks. Each graph shows one item of the NASA TLX (Hart and Staveland,
1988) score with two bars for each task. Top bars are the BASE condition,
bottom bars are MULTI. Color encodes selected value from red (values 1-3) to
blue (values 5-7). Bars are centered around neutral ratings (4). Length
denotes the frequency of that score in the assessment.

###  5.4. User Feedback

Participants were asked to provide unstructured feedback at the end of the
study.

For the individual tasks, users were asked several questions related to user
experience. These questions were a subset of the NASA TLX (Hart and Staveland,
1988) evaluation scheme. They were presented on a 1 to 7 Likert scale in which
negative ratings were indicated towards 1 (e.g. not easy, low confidence,
etc.), and positive ratings were on the 7 side (e.g. very easy, high
confidence, etc.). These results are shown in Figure 6. More detailed
analysis, including CI and effect sizes in addition to NHT, similar to the
task results, can be found in the supplemental material.

Overall, users felt more confident (Cohen’s $d=0.687$, $p<0.05$) about their
answers using the MULTI condition while perceiving less mental (Cohen’s
$d=-0.437$, $p<0.05$) and physical demand (Cohen’s $d=-1.503$, $p<0.01$).
While perceived easiness (Cohen’s $d=0.130$, $p=0.517$) of the tasks are the
same in both conditions, users also felt less hurried (Cohen’s $d=-0.800$,
$p<0.01$) and less irritated (Cohen’s $d=-0.596$, $p<0.01$) using our system.

Participants noted some criticism of the system:

  * •

”Pulling groups closer felt too close.”

  * •

”My map was always located above me, so I had to constantly be looking at the
ceiling.”

The MULTI condition also received several positive comments:

  * •

”Miniature graph overview is very helpful for tasks that require a rough
estimate.”

  * •

”The projection [layout] is good to find common neighbors of two nodes […]”

  * •

”[…] the function where you could expand the nodes on the ”floor” […] was very
helpful in some of the tasks.”

##  6\. Discussion

The results of our study confirm our hypotheses and hint towards several
trends. Users were more accurate in spatially focused tasks (intra-community)
using the MULTI condition while taking the same time as the baseline. These
tasks primarily concerned single nodes or nodes within the same community. Due
to this, users in both conditions tended to expand the community in question.
In the multi-layout approach, most expansions were followed by a projection to
see even more details. This underlines the utility of the projected layout.

On a larger scale—for tasks that focus on multiple communities (inter-
community)—users in the MULTI condition did not perform better than the
baseline. This is reflected in the interaction statistics. Aside from
community expansion, which helped users track external links from expanded
communities back to their source in the main network more easily, the
remaining layouts were used very little. Interestingly, users took
significantly longer to solve the MaxDegreeNode task in the MULTI condition.
We suspect this to be due to the relatively higher utilization of the
projected layout in this task. Users tended to spend a long time inspecting
nodes but were ultimately unable to perform better than the baseline. These
results indicate that the additional views on the data were not helpful to
solving the tasks. We attribute this to the fact that the spherical view
balances overview and detail quite well, which reduces the amount of split
attention required to solve the task and can therefore be beneficial when
estimating node or group degrees. Due to the lack of layout utilization, the
equality in performance in both conditions is not surprising.

Lastly, the epi-community task required the users to focus on the whole
network. Our multi-layout approach again outperformed the baseline for this
type of task. The projected view was utilized more often than the overview,
which is surprising since we initially suspected the overview to be more
useful here. Our data indicates that the projected view helped users
distribute a large number of inter-community links more broadly, making the
counting task more manageable. Contrary to our expectations, users in the BASE
condition made almost no use of the expansion feature despite having
experienced its relative benefits when compared to the spherical view.

To summarize, our results and user feedback indicate that the larger the
network and the more spatially complex the task, the better the results can be
expected from the multi-layout approach. Accuracy varied among tasks and
conditions, while average completion times were generally comparable. These
findings align with observations made by Greffard et al. (Greffard et al.,
2015) in that users tend to take the same time to complete tasks while
approaching them differently in terms of focus and interactions.
Unsurprisingly, the narrower the scope of a task, the less frequent the
different layouts were used. As tasks increased in complexity and scope,
layouts tended to be switched more often on average. Overall, the number of
interactions varied greatly between users, hinting at vastly different
problem-solving approaches. Ultimately, the tools we offered in our method
enabled users to perform significantly better in most tasks which shows that a
multi-layout approach performs well in a wide range of tasks. These results
are encouraging insofar as they confirm prior observations about the viability
of immersive visualization for network analysis (Ware and Franck, 1996; Ware
and Mitchell, 2005, 2008; Kwon et al., 2016; Kotlarek et al., 2020). Moreover,
they show that by carefully designing layout and interaction suits for VR, we
are able to build analysis systems that can greatly improve users’
productivity. These results are proof that we have not reached the full
potential of what immersive visualization has to offer. Nonetheless, through
our study, we show the superior performance of an immersive multi-layout suite
over prior work in VR network analysis and transitively over 2D screen-based
network analysis tools. We attribute these results to the fact that prior VR
work has not made extensive use of the available space and that there is great
potential in both spatial arrangement, choice of layouts, and interaction
design. We hope to provide a direction for future research in this area since
there are still several unexplored areas, as hinted at in Table 1.

##  7\. Limitations & Future Work

We believe that the multi-layout method presented here is only the first step
toward understanding the potential of interactive, immersive graph layouts. We
presented a combination of four possible views, which we show to perform quite
well. Although we make informed choices in our design, we do not claim this
particular arrangement of views to be flawless. The presented approach was
shown to outperform prior work in immersive network visualization, but future
studies need to further investigate the effect of various layout methods and
different combinations thereof (Table 1).

One limitation of the current evaluation was the fixed FOV and static
placement of the layouts. This results in some overwhelmingly big
presentations of smaller networks while it creates clutter in more dense
datasets. It also leads users to assume uncomfortable head positions when
performing certain tasks. These restrictions increase the risk of fatigue
which could impair performance and enjoyment of using the system. This issue
is reflected in user feedback, as it is often one of the first concerns
mentioned by our participants. Future work can address this drawback by
adjusting the FOV to achieve homogeneous node and community density and by
allowing the user to adjust layout positions manually.

Another possible direction this work can be expanded on is by introducing more
intricate interaction methods like gestures (Erra et al., 2019; Huang et al.,
2017) or eye-tracking to accommodate certain tasks and reduce fagitue
(Masopust et al., 2021).

##  8\. Conclusion

We introduce a multi-layout approach to networks in VR. Through this work, we
hope to enrich existing efforts to prove the potential of immersive
visualization for network tasks. The literature on interactions, layout, and
rendering of networks in VR environments is ever-growing. We contribute to
this body of work by introducing a novel and effective design for network
analysis in immersive space which covers a broad range of layouts and supports
a vast array of common network tasks.

Our study results show that the multi-layout approach outperforms the state-
of-the-art of immersive network visualization—especially for complex and
spatially demanding tasks in hierarchical network structures. Building on top
of users’ feedback about interaction and layout methods, the next step is to
refine our system for further studies. Specifically, we want to further
evaluate scenarios that were not covered by our current design (Table 1 yellow
cells) and extend its capabilities to investigate layout-task combinations
that were previously not considered by literature (Table 1 red cells).
Additionally, more studies on the efficacy of different views, interactions,
and layout suite sizes will become necessary as we explore the potential uses
of immersive visualizations further. Understanding the suitability of
different representations and interactions for this type of data is crucial to
driving the design research of information visualization in virtual reality.

## References

  * (1)
  * Alper et al. (2011) Basak Alper, Tobias Höllerer, Joann Kuchera-Morin, and Angus Forbes. 2011.  Stereoscopic highlighting: 2D graph visualization on stereo displays.  _IEEE Transactions on Visualization and Computer Graphics_ 17, 12 (2011), 2325–2333.  https://doi.org/10.1109/TVCG.2011.234
  * Amar et al. (2005) Robert Amar, James Eagan, and John Stasko. 2005.  Low-level components of analytic activity in information visualization. In _IEEE Symposium on Information Visualization, 2005. INFOVIS 2005._ IEEE, 111–117.  https://doi.org/10.1109/INFVIS.2005.1532136
  * Barahimi and Wismath (2014) Farshad Barahimi and Stephen Wismath. 2014.  3D graph visualization with the Oculus Rift.  _Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)_ 8871, 3 (2014), 519–520.
  * Bennett et al. (2015) Rebecca Bennett, David J. Zielinski, and Regis Kopper. 2015\.  Comparison of interactive environments for the archaeological exploration of 3D landscape data.  _2014 IEEE VIS International Workshop on 3DVis, 3DVis 2014_ (2015), 67–71.  https://doi.org/10.1109/3DVis.2014.7160103
  * Blondel et al. (2008) Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. 2008.  Fast unfolding of communities in large networks.  _Journal of statistical mechanics: theory and experiment_ 2008, 10 (2008), P10008.  https://doi.org/10.1088/1742-5468/2008/10/p10008
  * Bourqui et al. (2007) Romain Bourqui, David Auber, and Patrick Mary. 2007\.  How to draw clustered weighted graphs using a multilevel force-directed graph drawing algorithm.  _Proceedings of the International Conference on Information Visualisation_ (2007), 757–764.  https://doi.org/10.1109/IV.2007.65
  * Bowman et al. (2004) Doug Bowman, Ernst Kruijff, Joseph J LaViola Jr, and Ivan P Poupyrev. 2004.  _3D User interfaces: theory and practice, CourseSmart eTextbook_.  Addison-Wesley.
  * Brooks (1999) Frederick P. Brooks. 1999\.  What’s real about virtual reality?  _IEEE Computer Graphics and Applications_ 19, 6 (1999), 16–27.  https://doi.org/10.1109/38.799723
  * Bryson (1993) Steve Bryson. 1993\.  Virtual reality in scientific visualization.  _Computers and Graphics_ 17, 6 (1993), 679–685.  https://doi.org/10.1016/0097-8493(93)90117-R
  * Buschel et al. (2019) W. Buschel, S. Vogt, and R. Dachselt. 2019.  Augmented Reality Graph Visualizations.  _IEEE Computer Graphics and Applications_ 39, 03 (may 2019), 29–40.  https://doi.org/10.1109/MCG.2019.2897927
  * Cordeil et al. (2017) M. Cordeil, T. Dwyer, K. Klein, B. Laha, K. Marriott, and B. H. Thomas. 2017\.  Immersive Collaborative Analysis of Network Connectivity: CAVE-style or Head-Mounted Display?  _IEEE Transactions on Visualization and Computer Graphics_ 23, 1 (2017), 441–450.  https://doi.org/10.1109/TVCG.2016.2599107
  * Cumming (2014) Geoff Cumming. 2014\.  The New Statistics: Why and How.  _Psychological Science_ 25, 1 (2014), 7–29.  https://doi.org/10.1177/0956797613504966
  * Cutting and Vishton (1995) James E Cutting and Peter M Vishton. 1995.  Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth.  In _Perception of space and motion_. Elsevier, 69–117.  https://doi.org/10.1016/B978-012240530-3/50005-5
  * Didimo and Montecchiani (2014) Walter Didimo and Fabrizio Montecchiani. 2014.  Fast layout computation of clustered networks: Algorithmic advances and experimental analysis.  _Information Sciences_ 260 (2014), 185–199.  https://doi.org/10.1016/j.ins.2013.09.048
  * Drogemuller et al. (2019) Adam Drogemuller, Andrew Cunningham, James Walsh, Bruce Thomas, Maxime Cordeil, and William Ross. 2019\.  Examining Virtual Reality Navigation Techniques for 3D Network Visualisations.  _Journal of Visual Languages & Computing_ 56 (12 2019).  https://doi.org/10.1016/j.cola.2019.100937
  * Erra et al. (2019) Ugo Erra, Delfina Malandrino, and Luca Pepe. 2019\.  Virtual Reality Interfaces for Interacting with Three-Dimensional Graphs.  _International Journal of Human-Computer Interaction_ 35, 1 (2019), 75–88.  https://doi.org/10.1080/10447318.2018.1429061
  * Ersoy et al. (2011) Ozan Ersoy, Christophe Hurter, Fernando Paulovich, Gabriel Cantareiro, and Alex Telea. 2011\.  Skeleton-Based Edge Bundling for Graph Visualization.  _IEEE Transactions on Visualization and Computer Graphics_ 17, 12 (2011), 2364–2373.  https://doi.org/10.1109/TVCG.2011.233
  * Facebook Technologies, LLC. (2020) Facebook Technologies, LLC. 2020\.  Oculus — Headsets & Equipment.  https://www.oculus.com/.  [Online; accessed 22-June-2020].
  * Fortunato (2010) Santo Fortunato. 2010\.  Community detection in graphs.  _Physics Reports_ 486, 3-5 (2010), 75–174.  https://doi.org/10.1016/j.physrep.2009.11.002
  * Greffard et al. (2015) Nicolas Greffard, Fabien Picarougne, and Pascale Kuntz. 2015\.  Beyond the classical monoscopic 3D in graph analytics: An experimental study of the impact of stereoscopy.  _2014 IEEE VIS International Workshop on 3DVis, 3DVis 2014_ (2015), 19–24.  https://doi.org/10.1109/3DVis.2014.7160095
  * Halpin et al. (2008) Harry Halpin, David J. Zielinski, Rachael Brady, and Glenda Kelly. 2008.  Exploring semantic social networks using virtual reality.  _Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)_ 5318 LNCS (2008), 599–614.  https://doi.org/10.1007/978-3-540-88564-1-38
  * Hanel et al. (2015) Claudia Hanel, Benjamin Weyers, Bernd Hentschel, and Torsten W. Kuhlen. 2015.  Interactive volume rendering for immersive virtual environments.  _2014 IEEE VIS International Workshop on 3DVis, 3DVis 2014_ 73 (2015), 73–74.  https://doi.org/10.1109/3DVis.2014.7160104
  * Harlow et al. (2016) L. Harlow, S. Mulaik, and J. Steiger. 2016.  _What If There Were No Significance Tests?_ Routledge, Chapter Eight Common but False Objections to the Discontinuation of Significance Testing in the Analysis of Research Data.
  * Hart and Staveland (1988) Sandra G Hart and Lowell E Staveland. 1988.  Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research.  In _Advances in psychology_. Vol. 52. Elsevier, 139–183.  https://doi.org/10.1016/S0166-4115(08)62386-9
  * Herman et al. (2000) Ivan Herman, Guy Melançon, and M Scott Marshall. 2000\.  Graph visualization and navigation in information visualization: A survey.  _IEEE Transactions on Visualization and Computer Graphics_ 6, 1 (2000), 24–43.  https://doi.org/10.1109/2945.841119
  * Holten (2006) Danny Holten. 2006\.  Hierarchical edge bundles: Visualization of adjacency relations in hierarchical data.  _IEEE Transactions on Visualization and Computer Graphics_ 12, 5 (2006), 741–748.  https://doi.org/10.1109/TVCG.2006.147
  * Holten and van Wijk (2009) Danny Holten and Jarke van Wijk. 2009.  Force-Directed Edge Bundling for Graph Visualization.  _Comput. Graph. Forum_ 28 (06 2009), 983–990.  https://doi.org/10.1111/j.1467-8659.2009.01450.x
  * HTC Corporation (2020) HTC Corporation. 2020\.  VIVE.  https://www.vive.com/us/.  [Online; accessed 22-June-2020].
  * Huang et al. (2017) Yi Jheng Huang, Takanori Fujiwara, Yun Xuan Lin, Wen Chieh Lin, and Kwan Liu Ma. 2017.  A gesture system for graph visualization in virtual reality environments.  _IEEE Pacific Visualization Symposium_ (2017), 41–45.  https://doi.org/10.1109/PACIFICVIS.2017.8031577
  * Hurter et al. (2019) C. Hurter, N. H. Riche, S. M. Drucker, M. Cordeil, R. Alligier, and R. Vuillemot. 2019\.  FiberClay: Sculpting Three Dimensional Trajectories to Reveal Structural Insights.  _IEEE Transactions on Visualization and Computer Graphics_ 25, 1 (2019), 704–714.  https://doi.org/10.1109/TVCG.2018.2865191
  * Itoh et al. (2009) Takayuki Itoh, Chris Muelder, Kwan-Liu Ma, and Jun Sese. 2009\.  A hybrid space-filling and force-directed layout method for visualizing multiple-category graphs. In _2009 IEEE Pacific Visualization Symposium_. IEEE, 121–128.  https://doi.org/10.1109/PACIFICVIS.2009.4906846
  * Jacomy et al. (2014) Mathieu Jacomy, Tommaso Venturini, Sebastien Heymann, and Mathieu Bastian. 2014.  ForceAtlas2, a Continuous Graph Layout Algorithm for Handy Network Visualization Designed for the Gephi Software.  _PLOS ONE_ 9, 6 (06 2014), 1–12.  https://doi.org/10.1371/journal.pone.0098679
  * Johnson and Shneiderman (1991) Brian Johnson and Ben Shneiderman. 1991.  Tree-maps: A space-filling approach to the visualization of hierarchical information structures.  _Proceedings of the 2nd Conference on Visualization 1991, VIS 1991_ (1991), 284–291.  https://doi.org/10.1109/visual.1991.175815
  * Kobourov (2012) Stephen G. Kobourov. 2012\.  Spring Embedders and Force Directed Graph Drawing Algorithms.  (2012), 1–23.  arXiv:1201.3011 http://arxiv.org/abs/1201.3011
  * Kotlarek et al. (2020) Joseph Kotlarek, Oh-Hyun Kwon, Kwan Liu Ma, Peter Eades, Andreas Kerren, Karsten Klein, and Falk Schreiber. 2020.  A Study of Mental Maps in Immersive Network Visualization.  _IEEE Pacific Visualization Symposium_ 2020-June (2020), 1–10.  https://doi.org/10.1109/PacificVis48177.2020.4722 arXiv:2001.06462
  * Kwon et al. (2016) Oh-Hyun Kwon, Chris Muelder, Kyungwon Lee, and Kwan Liu Ma. 2016.  A study of layout, rendering, and interaction methods for immersive graph visualization.  _IEEE Transactions on Visualization and Computer Graphics_ 22, 7 (2016), 1802–1815.  https://doi.org/10.1109/TVCG.2016.2520921
  * Lee et al. (2006) Bongshin Lee, Catherine Plaisant, Jean-Daniel Fekete, Cynthia Sims Parr, and Nathalie Henry. 2006\.  Task Taxonomy for Graph Visualization Categories and Subject Descriptors.  _Proceedings of the AVI Workshop Beyond Time and Errors: Novel Evaluation Methods for Information Visualization_ (2006), 1–5.  https://doi.org/10.1145/1168149.1168168
  * Marriott et al. (2018) Kim Marriott, Jian Chen, Marcel Hlawatsch, Takayuki Itoh, Miguel A. Nacenta, Guido Reina, and Wolfgang Stuerzlinger. 2018.  _Immersive Analytics: Time to Reconsider the Value of 3D for Information Visualisation_.  Springer International Publishing, Cham, 25–55.  https://doi.org/10.1007/978-3-030-01388-2
  * Masopust et al. (2021) Lukas Maximilian Masopust, David Bauer, Siyuan Yao, and Kwan-Liu Ma. 2021.  A Comparison of the Fatigue Progression of Eye-Tracked and Motion-Controlled Interaction in Immersive Space. In _2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)_. 460–469.  https://doi.org/10.1109/ISMAR52148.2021.00063
  * McIntire and Liggett (2015) John P. McIntire and Kristen K. Liggett. 2015.  The (possible) utility of stereoscopic 3D displays for information visualization: The good, the bad, and the ugly.  _2014 IEEE VIS International Workshop on 3DVis, 3DVis 2014_ (2015), 1–9.  https://doi.org/10.1109/3DVis.2014.7160093
  * Miller (1956) George A. Miller. 1956\.  The magical number seven, plus or minus two: some limits on our capacity for processing information.  _Psychological Review_ 63, 2 (1956), 81–97.  https://doi.org/10.1037/h0043158
  * Mirhosseini et al. (2015) Koosha Mirhosseini, Qi Sun, Krishna C. Gurijala, Bireswar Laha, and Arie E. Kaufman. 2015.  Benefits of 3D immersion for virtual colonoscopy.  _2014 IEEE VIS International Workshop on 3DVis, 3DVis 2014_ Vc (2015), 75–79.  https://doi.org/10.1109/3DVis.2014.7160105
  * Mirhosseini et al. (2019) Seyedkoosha Mirhosseini, Ievgeniia Gutenko, Sushant Ojal, Joseph Marino, and Arie Kaufman. 2019\.  Immersive Virtual Colonoscopy.  _IEEE Transactions on Visualization and Computer Graphics_ 25, 5 (2019), 2011–2021.  https://doi.org/10.1109/TVCG.2019.2898763
  * Munzner (1997) Tamara Munzner. 1997\.  H3: laying out large directed graphs in 3D hyperbolic space.  _Proceedings of the IEEE Symposium on Information Visualization_ (1997), 2–10.  https://doi.org/10.1109/infvis.1997.636718
  * Phan et al. (2005) Doantam Phan, Ling Xiao, Ron Yeh, and Pat Hanrahan. 2005\.  Flow map layout. In _IEEE Symposium on Information Visualization, 2005. INFOVIS 2005._ IEEE, 219–224.  https://doi.org/10.1109/INFVIS.2005.1532150
  * Rekimoto and Green (1993) Jun Rekimoto and Mark Green. 1993.  The information cube: Using transparency in 3d information visualization. In _Proceedings of the Third Annual Workshop on Information Technologies & Systems (WITS’93)_. 125–132.
  * Robertson et al. (1991) George G. Robertson, Jock D. Mackinlay, and Stuart K. Card. 1991\.  Cone Trees: Animated 3D Visualizations of Hierarchical Information. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_ (New Orleans, Louisiana, USA) _(CHI ’91)_. Association for Computing Machinery, New York, NY, USA, 189–194.  https://doi.org/10.1145/108844.108883
  * Saket et al. (2014) Bahador Saket, Paolo Simonetto, and Stephen Kobourov. 2014\.  Group-Level Graph Visualization Taxonomy. In _EuroVis - Short Papers_ , N. Elmqvist, M. Hlawitschka, and J. Kennedy (Eds.). The Eurographics Association.  https://doi.org/10.2312/eurovisshort.20141162
  * SAMSUNG ELECTRONICS CO., LTD. (2020) SAMSUNG ELECTRONICS CO., LTD. 2020\.  Samsung Gear.  http://www.samsung.com/global/galaxy/gear-vr/.  [Online; accessed 22-June-2020].
  * Scholl et al. (2019) Ingrid Scholl, Alexander Bartella, Cem Moluluo, Berat Ertural, Frederic Laing, and Sebastian Suder. 2019\.  _MedicVR: Acceleration and Enhancement Techniques for Direct Volume Rendering in Virtual Reality_.  152–157.  https://doi.org/10.1007/978-3-658-25326-4_32
  * Scholl et al. (2018) Ingrid Scholl, Sebastian Suder, and Stefan Schiffer. 2018\.  Direct volume rendering in virtual reality.  _Informatik aktuell_ d, 211279 (2018), 297–302.  https://doi.org/10.1007/978-3-662-56537-7_79
  * Sony Interactive Entertainment LLC. (2020) Sony Interactive Entertainment LLC. 2020\.  PlayStation.  https://www.playstation.com/en-us/explore/playstation-vr/.  [Online; accessed 22-June-2020].
  * Sorger et al. (2019) Johannes Sorger, Manuela Waldner, Wolfgang Knecht, and Alessio Arleo. 2019.  Immersive analytics of large dynamic networks via overview and detail navigation.  _Proceedings - 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2019_ (2019), 144–151.  https://doi.org/10.1109/AIVR46125.2019.00030 arXiv:1910.06825
  * Sutherland (1965) Ivan E. Sutherland. 1965\.  The Ultimate Display. In _Proceedings of the IFIP Congress_. 506–508.
  * Sweller et al. (1998) John Sweller, Jeroen J. G. van Merrienboer, and Fred G. W. C. Paas. 1998\.  Cognitive Architecture and Instructional Design.  _Educational Psychology Review_ 10, 3 (01 Sep 1998), 251–296.  https://doi.org/10.1023/A:1022193728205
  * Van Dam et al. (2000) Andries Van Dam, Andrew S. Forsberg, David H. Laidlaw, Joseph J. LaViola, and Rosemary M. Simpson. 2000.  Immersive VR for scientific visualization: A progress report.  _IEEE Computer Graphics and Applications_ 20, 6 (2000), 26–52.  https://doi.org/10.1109/38.888006
  * Vehlow et al. (2017) Corinna Vehlow, Fabian Beck, and Daniel Weiskopf. 2017\.  Visualizing Group Structures in Graphs: A Survey.  _Computer Graphics Forum_ 36, 6 (2017), 201–225.  https://doi.org/10.1111/cgf.12872
  * Ware (2019) Colin Ware. 2019\.  _Information visualization: perception for design_.  Morgan Kaufmann.
  * Ware and Franck (1996) Colin Ware and Glenn Franck. 1996.  Evaluating Stereo and Motion Cues for Visualizing Information Nets in Three Dimensions.  _ACM Transactions on Graphics_ 15, 2 (1996), 121–140.  https://doi.org/10.1145/234972.234975
  * Ware and Mitchell (2005) Colin Ware and Peter Mitchell. 2005.  Reevaluating Stereo and Motion Cues for Visualizing Graphs in Three Dimensions. In _Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization_ (A Coroña, Spain) _(APGV ’05)_. Association for Computing Machinery, New York, NY, USA, 51–58.  https://doi.org/10.1145/1080402.1080411
  * Ware and Mitchell (2008) Colin Ware and Peter Mitchell. 2008.  Visualizing graphs in three dimensions.  _ACM Transactions on Applied Perception (TAP)_ 5, 1 (2008), 1–15.
  * Watts and Strogatz (1998) Duncan J. Watts and Steven H. Strogatz. 1998.  Collective dynamics of ‘small-world’ networks.  _Nature_ 393, 6684 (01 Jun 1998), 440–442.  https://doi.org/10.1038/30918
  * Yang et al. (2013) Jing Yang, Yujie Liu, Xin Zhang, Xiaoru Yuan, Ye Zhao, Scott Barlowe, and Shixia Liu. 2013.  PIWI: Visually exploring graphs based on their community structure.  _IEEE Transactions on Visualization and Computer Graphics_ 19, 6 (2013), 1034–1047.  https://doi.org/10.1109/TVCG.2012.172
  * Yoghourdjian et al. (2018) Vahan Yoghourdjian, Daniel Archambault, Stephan Diehl, Tim Dwyer, Karsten Klein, Helen C. Purchase, and Hsiang Yun Wu. 2018.  Exploring the limits of complexity: A survey of empirical studies on graph visualisation.  _Visual Informatics_ 2, 4 (2018), 264–282.  https://doi.org/10.1016/j.visinf.2018.12.006 arXiv:1809.00270
  * Youdas et al. (1992) James W Youdas, Tom R Garrett, Vera J Suman, Connie L Bogard, Horace O Hallman, and James R Carey. 1992\.  Normal Range of Motion of the Cervical Spine: An Initial Goniometric Study.  _Physical Therapy_ 72, 11 (11 1992), 770–780.  https://doi.org/10.1093/ptj/72.11.770
  * Young (1996) Peter Young. 1996\.  Three Dimensional Information Visualization.  _Technical Report 12/96, University of Durham_ (1996).