# Submerse: Visualizing Storm Surge Flooding Simulations in Immersive Display
Ecologies

Saeed Boorboor, Yoonsang Kim, Ping Hu, Josef M. Moses, Brian A. Colle, and
Arie E. Kaufman Boorboor, Kim, Hu, and Kaufman are with the Department of
Computer Science, Stony Brook University.
E-mail: {sboorboor, yoonsakim, pihu<EMAIL_ADDRESS>Moses and Colle
are with the School of Marine and Atmospheric Sciences, Stony Brook
University. Manuscript received xxxxx; revised xxxxx.

###### Abstract

We present Submerse, an end-to-end framework for visualizing flooding
scenarios on large and immersive display ecologies. Specifically, we
reconstruct a surface mesh from input flood simulation data and generate a to-
scale 3D virtual scene by incorporating geographical data such as terrain,
textures, buildings, and additional scene objects. To optimize computation and
memory performance for large simulation datasets, we discretize the data on an
adaptive grid using dynamic quadtrees and support level-of-detail based
rendering. Moreover, to provide a perception of flooding direction for a time
instance, we animate the surface mesh by synthesizing water waves. As
interaction is key for effective decision-making and analysis, we introduce
two novel techniques for flood visualization in immersive systems: (1) an
automatic scene-navigation method using optimal camera viewpoints generated
for marked points-of-interest based on the display layout, and (2) an AR-based
focus+context technique using an auxiliary display system. Submerse is
developed in collaboration between computer scientists and atmospheric
scientists. We evaluate the effectiveness of our system and application by
conducting workshops with emergency managers, domain experts, and concerned
stakeholders in the Stony Brook Reality Deck, an immersive gigapixel facility,
to visualize a superstorm flooding scenario in New York City.

###### Index Terms:

Immersive visualization, Flooding simulation, Camera navigation, Mixed
Reality.

##  1 Introduction

The increase in extreme weather events [1] has led to a growing need for
visualization systems that can aid scientists, emergency managers, and
concerned stakeholders to effectively prepare for the impacts of potential
catastrophic events. Of particular concern is coastal and inland flooding.
Studies have shown that floods account for 43% of all disasters and are the
most frequently occurring natural disasters worldwide [2]. Although there have
been advances in weather forecasting models, its usefulness depends greatly on
the human decision-making dimension [3].

Predicted flooding scenarios using numerical simulation models are typically
rendered as 2D visualizations superimposed onto 2D maps. While a 2D
representation can convey the birds-eye view extents of the flooding, it
relies on the user to perceive depth information based on cues such as color.
Extending the visualizations to 3D [4, 5] enhances the user’s understanding of
flooding depth, intensity, and impact with respect to its spatial
surroundings. However, when analyzing a vast virtual scene at a high level-of-
detail (LoD), the limited field-of-view (FoV) of a single-screen desktop
workstation can impel a user to engage in frequent pan-and-zoom interactions.
This has motivated us to develop Submerse, an application and system for flood
visualization that harnesses the screen real-estate and visual acuity of large
immersive visualization systems.

Specifically for scientific analysis, large, high-resolution displays, such as
CAVEs and immersive tiled displays have been shown to improve user performance
in various visualization tasks [6, 7, 8]. In essence, utilizing large screen
real-estate allows for a wider FoV. This is particularly beneficial for
applications such as flood visualization, where immediate access to local and
global data context is vital for well-rounded analysis and decision-making.
Moreover, leveraging high-resolution graphics and high-performance
capabilities of such facilities enhances the detail and granularity of
information as well as the realism of the visualizations, which has been shown
to positively influence perceptions and behaviors in preparing for and
responding to natural hazards [9].

Submerse addresses three important design challenges. First, visualizing
simulation data coupled with elements of a geographic information system (GIS)
requires constructing and rendering large 3D meshes from data arranged on
structured and unstructured grids. Thus, a system infrastructure with low
latency and real-time synchronization across display nodes is required for
effective interactivity. To this end, we have developed _Submerse_ as a
modular and scalable system that abstracts visualization design at an
application level from its deployment across a multi-node multi-display
system. By providing the layout and cluster configuration of a display system,
Submerse determines appropriate camera projections for each physical display
in the setup, along with view-dependent data extents for rendering the virtual
scene. At the application level, we render the simulated flood as a 3D surface
mesh with propagating waves to visualize flow direction. To optimize
computation time and memory, and to support LoD rendering, we achieve this by
representing the simulation data on an adaptive heightfield grid [4, 10] using
a dynamic quadtree datastructure.

Second, flood visualization applications require frequent data interaction.
Unlike a desktop setup, where applications have a graphical user interface and
users have access to input devices, interaction in immersive settings usually
adopts a natural mode, such as walking and hand gestures, and occasionally by
using hand-held controllers. To allow user-interface controls, Submerse
provides a unique focus-and-context interaction by integrating a video see-
through augmented-reality (AR) auxiliary display to the immersive setup. In
integrating a hand-held device as an auxiliary display, Submerse enables
augmenting additional information and visualizations atop the surrounding
displays, facilitating local contextual analysis while simultaneously having
access to global data. Moreover, to ensure an interactive user experience, we
have included a remote rendering protocol in Submerse that offloads
computation and rendering tasks from the auxiliary device to a server.

Finally, effective navigation is an essential element for gaining a complete
understanding of the scene. This becomes additionally challenging for large
and cluttered scenes, where multiple points-of-interest (POIs) have to be
observed. Determining appropriate views and designing methods for navigation
in immersive systems is an active ongoing area of research [11] To this end,
we have developed an optimal camera view-finding algorithm that maximizes view
coverage of user-defined POIs based on the layout of the display system. Using
the camera views, we additionally facilitate scene navigation by determining a
camera path that smoothly transitions along subsequent viewpoints by
connecting them using straight lines and quadratic Bézier curves such that it
avoids collision with 3D scene objects, minimizes motion sickness for
immersive environments [12], and maintains a camera look-at vector in the
direction of the shoreline to ensure visibility of the coastal and inland
flooding.

Submerse is developed as a collaboration between computer scientists and
atmospheric scientists and feedback from flood managers. In this paper, we
demonstrate the visualization of a flooding scenario in New York City (NYC)
and deploy it on the Stony Brook University Reality Deck (RD), the largest
immersive gigapixel display facility in the world [13]. We evaluate our system
based on feedback from workshops conducted with the NYC flood managers,
meteorologists, domain scientists, graduate students in atmospheric sciences,
and other stakeholders.

We summarize our contributions as follows:

  * •

An end-to-end scalable and modular system designed specifically for
visualizing flooding data on large immersive display setups.

  * •

An AR-based auxiliary display medium for focus+context visualization in
immersive settings.

  * •

A novel automatic camera viewpoint finding algorithm that facilitates
effective scene exploration.

Figure 1: For a defined display system (a) and input flooding and GIS data
(b), Submerse manages data extents and generates automatic camera viewpoints
and navigation trajectory (c) to render 3D immersive visualization of the
flooding scenario (d).

##  2 Related Works

Visualization of Flooding Simulation Data: Tools for visualizing flooding
scenarios generally feature a geographical map interface embedded with
rasterized information from numerical simulation models [14, 15, 16]. However,
it has been shown that 2D representation of flooding data limits the full
spectrum perception of additional multivariate geospatial attributes [17], and
can lead to the lack of understanding of the severity and ineffective
preparation for flooding scenarios [9]. In integrating flood data with 3D
objects, Cornel et al. [4] have developed visual encodings on 3D SCENE
objects, establishing an intuitive relation between objects and flooding
events information. To fully integrate geospatial attributes such as terrains,
Cornel et al. [5] have developed fast interpolation of adaptive grids to
reconstruct a surface mesh. Additionally, they present an algorithm for
realistic water shading with depth perception cues. In Submerse, we derive our
mesh interpolation and wave synthesis techniques from Cornel et al. [5] and
expand it to adapt to distributed visualization systems and immersive
settings. Particular to the VR domain, several game applications exist for
training and behavior analysis in the event of flooding [18, 19], and to aid
non-scientific stakeholders in visualizing and understanding flood simulation
data without prior technical knowledge [20]. In contrast, Submerse provides a
more wholesome framework for VR by presenting a novel interaction interface
for collaborative spaces, and camera view-finding and path-planning algorithms
specific to the domain requirements.

Interaction in Immersive Systems: Large display systems pose two important
challenges for data interactivity: (1) individual-level exploration of the
data visualized collectively with other users, and (2) implementing scalable
interaction metaphors. Reipschlager et al. [21] have addressed the former by
utilizing an AR head-mounted display (HMD) device to provide analytical
information on top of the visualization being presented. Chen et al. [22] have
developed a system that supports the independent exploration of global data
using a smartphone. Beyond the screen space, Nishimoto and Johnson [23] have
developed a method to extend a user’s field of regard by displaying additional
information using an AR HMD. For the latter, the most widely adopted technique
has been to utilize tracking hardware in the display facility to translate
human gestures into interaction [24, 25, 26]. Another interaction metaphor,
using a touch-enabled surfaces, have been used to interact with the data
directly [27, 28, 29]. Recent works have investigated the use of commodity
hardware for interaction. Siddhpuria et al. [30] have explored the use of a
personal smart device as a pointing device. Babic et al. [31] leveraged off-
the-shelf smartphone cameras for hand, head, body, and gaze tracking to
interact with a distant display. Particularly for large display systems,
Langner and Dachselt [32] have developed a system to use spatially-aware
mobile devices in tiled-display ecology. Chen et al. [22] have introduced a
unique method of interaction whereby a user takes a photograph of the display
using a mobile device and subsequently uses the photo to explore, edit, and
navigate the device. In contrast to existing works presented, our auxiliary
display system combines components of exploration, extended visualization, as
well as interaction using a novel framework that utilizes AR to seamlessly
bridge interaction metaphors between the physical world and virtual scene.

View-finding and Path-Planning: Early on, Kamada and Kawai [33] have
formulated rules for determining a good view of basic shapes by maximizing the
projection of lines on the objects. We refer the reader to a comprehensive
review by Bonaventura et al. [11] on viewpoint selection methods for polygonal
data. More recently, information entropy [34] has been considered a measure
for determining visible information. Arbel et al. [35] have formulated view
selection based on Shannon entropy maps by measuring object recognition in a
monochrome view. Vázquez et al. [36] modeled view selection as an object
visibility maximization problem. Specific to landscape scenes, Stoev et al.
[37] have developed a method for automatic camera positioning for terrain
datasets by maximizing scene depth and the projected area. In terms of
navigation, Huang et al. [38] have introduced a camera motion design method,
particularly for urban scenes. Given a set of viewpoints, a path is determined
by their interests according to the number and the shapes of the visible
buildings. Furthermore, Lin et al. [39] have developed a framework for large-
scale scene data perception and reconstruction. In contrast to existing works,
our view-finding algorithm is two-fold: determining a view that maximizes
information visibility and also integrates the surrounding information and
display configuration in its formulation. Based on the viewpoints, we develop
a camera path control method for large-scale visualization, also taking into
consideration the domain information.

##  3 Domain Background and Requirements

Understanding and analyzing numerical models that describe and predict real-
life environment around us makes atmospheric science a very visual field.
While data collection methods and forecasting models have become better and
more sophisticated, tools for their visualization are still primarily based on
2D projections and limited to pan-and-zoom interactions. To effectively use
data for studying unprecedented weather extremes from climate change, as well
as for collaborating with stakeholders to plan resilience measures, Submerse
has been designed and developed based on the following requirements identified
by domain experts:

R1 – 3D visualization of flooding scenarios: To better understand the flooding
intensity, a visualization system is needed to stimulate a realistic
perception of the flooding depth and severity with respect to its geographic
surroundings and progression over time.

R2 – A collaborative space for diverse stakeholders: A method to facilitate a
fully integrative and collaborative space for stakeholders, such that users
can experience a shared understanding as well as interact independently with
the scientific findings of the climate catastrophe.

R3 – Optimizing POI views and navigation in large scenes: One of the
challenges of 3D visualization is scene navigation and adjusting the camera to
achieve good views, especially in large urban scenes. To this end, for a set
of input POIs, the system should assist with providing effective views for
analyzing the progression of coastal flooding with minimal viewing occlusion.
For a collection of POIs, the system should aid in navigating the scene.

Figure 2: An example of constructing a water surface mesh using an adaptive
grid. For flood datapoints defined on a geographic location (a), we discretize
the data on a quadtree (b) and interpolate the values to construct a smooth
and continuous field as shown in (c) and (d). A cell without a datapoint (e)
is defined by averaging the values of its neighboring defined cells (f). To
interpolate a value for the red point in (g), we apply bilinear interpolation
on the cells that bound the point. However, since one of the sampling cells,
marked in red, has a lower cell resolution, we divide the cell (h) and assign
a value by upsampling. The upsampling is done by applying bilinear
interpolation on the neighboring same-resolution cells (i). A lower resolution
value is calculated using downsampling, that is, the average value of all its
child higher resolution cells. Finally, a value for the point is constructed
by applying bilinear interpolation on the same-resolution cells (j).

##  4 Submerse

Based on the domain requirements, we have designed Submerse as an application
and system to visualize storm surge flooding simulations by specifically
utilizing immersive display facilities. Fig. 1 provides an overview of
Submerse application and system pipeline.

For an input simulation data, we present an interactive visualization of the
flooding scenario by integrating the flooding level into a to-scale 3D virtual
model of the corresponding geographic area (R1). The virtual scene is set up
using GIS data, such as digital elevation map (DEM) and satellite imagery, and
supplementary scene objects, such as 3D buildings and reference 3D objects. We
describe our method for the processing of the input simulation data to
reconstruct an animated water surface, visualizing the flooding level and
direction, in Sec. 4.1.

High-resolution display facilities such as CAVEs and immersive tiled-displays
have been shown to enhance data realism and allow a more effective exploration
of three- and higher-dimensional data by tapping into the human peripheral
vision [8] (R1). Moreover, these visualization systems, depending on the
layout, present a natural space for multi-user collaboration and interaction
(R2). In Sec. 4.2, we present the Submerse system design which enables the
deployment of the visualization application to a multi-node, multi-GPU display
facility. Specifically, based on the configuration and geometry of a display
setup, Submerse manages synchronized instances of the application for each
viewport in the system and optimizes computational and memory resources by
determining view-dependent data extents. Additionally, we have developed a
novel AR-based auxiliary interaction method for immersive facilities, which we
describe in Sec. 4.3. Using an auxiliary display device, Submerse enables
users to perform localized visual analytics seamlessly atop the displayed
visualization (R2).

Finally, to facilitate an effective exploration of large scenes (R3), we have
designed an energy function that automatically generates optimal camera
viewpoints for user-defined POIs, based on parameters such as the
visualization display layout, POI visibility, and the distance from the user.
Using the set of calculated viewpoints, we generate a smooth camera path for
navigating to the POI views. Our camera path algorithm is designed to avoid
occlusion with scene objects during navigation and aims to minimize sickness
due to motion in immersive settings. The camera view-finding optimization
function and path generation algorithm are presented in Sec. 4.4.

###  4.1 Flood Simulation Rendering

The flood simulation data, which serves as input to our application, is a time
series data of scalar water elevation values and 2D tidal velocity,
discretized over a vast irregular grid of latitude and longitude positions.

####  4.1.1 Quadtree-based Height-Field Reconstruction

Although using a high-resolution grid is desirable for generating a smooth and
continuous surface, it has a high memory and performance overhead with
increasing spatial extents. To this end, we adopt adaptive height-field grid
which is a discretized representation of data on an unstructured grid of
varying cell sizes [40]. By interpolating cell-centered values on the grid, a
surface mesh can be reconstructed for a particular resolution level. Fig. 2
illustrates how this process is applied to our application. For a collection
of water elevation datapoints, shown in Fig. 2(a), we discretize the data
using a quadtree datastructure, as shown in Fig. 2(b). The main advantage of
using a quadtree is its low memory footprint, which is particularly important
for limited GPU memory, as well as its hierarchical data representation, which
enables LoD rendering. Each leaf node in the quadtree corresponds to a cell of
the height-field grid, and the value of each cell, illustrated using shades of
blue in Fig. 2(b), is defined using the datapoint value bounded by the leaf
node. To enable LoD rendering, we assign parent nodes to have a value equal to
the average value of their children. Finally, using a regular triangular mesh,
we render a water surface where the height of each vertex is determined by
applying 2D bilinear interpolation on the four cells that enclose the vertex.
A planar result of the interpolated values is shown in Fig. 2(c).

Notice in Fig. 2(b) that in sub-dividing a quadtree node, a child node may not
contain a datapoint, resulting in an undefined value in the height-field cell.
Moreover, the simulation output can also have dry datapoints, that is, the
spatial location does not flood for certain timepoints, and in some cases, the
entire duration of the simulated flood (this is specifically to define a
boundary for inland spatial locations beyond which no flooding occurs). For
dry cells, we use the average height of the underlying terrain obtained from
the input DEM, whereas for the undefined cells, we assign an average value of
all the defined cells in the neighborhood.

To correctly interpolate height values from varying cell sizes, we modify the
adaptive reconstruction introduced by Cornel et al. [4]. In their method, if a
neighboring cell is of a different size, an imaginary cell with an equivalent
size is reconstructed by upsampling (for a higher resolution level) or
downsampling (for a lower resolution level). In our variation, the downsampled
value is simply the averaged value assigned to the corresponding node’s
parent, as in our quadtree reconstruction above. To reconstruct an upsampled
value, a similar 2D bilinear interpolation is applied using the lower
resolution values of the four bounding cells. Fig. 2(e)-(f) shows this process
schematic representation. One caveat to this approach is that the cell
neighbors can differ by, at most, one level of resolution. We implement this
constraint in our quadtree construction step.

We furthermore extend this technique to facilitate LoD rendering. Although the
hierarchical datastructure of quadtrees supports LoD by design, we restrict
the lowest renderable resolution of a sub-tree to be the node whose children
belong to the same type, that is, all four children should either be flooded
or dry. This condition prevents dry cells from being incorrectly estimated to
have flooding. Note that for leaf nodes that correspond to undefined cells,
their extrapolated flooding value is used to propagate to the parent node,
thus categorizing it as a flooded node.

Figure 3: (a) shows vertex motion following a Gerstner wave and (b)
illustrates the difference in calculating analytical and mesh normals.

####  4.1.2 Flooding Direction Visualization

To visualize tasks that have a behavioral attribute of flow, for instance,
inundation, information cannot be perceived by observing a static mesh. This
becomes particularly true when time-series data is visualized while paused for
a single instance. A natural instinct then is to repetitively toggle back and
forth to observe change and variation. Moreover, visual metaphors such as
glyphs may add to the complexity of the visualization [4], especially amongst
diverse stakeholders.

To enable an intuitive perception of flow for a single timepoint, we animate
the water to emulate wind-driven surface waves. Typically in computer
graphics, realistic waves are synthesized using the Navier-Stokes equation.
However, for a more efficient and simplified approach, we apply Gerstner wave
synthesis model [41] using velocity vectors from the simulation data. Gerstner
waves produce nearly realistic-looking waves based on a fundamental
observation that, unlike a particle system where particles move with the wave,
mesh vertices can be displaced along a trochoidal function. Specifically, as
the crest of a wave approaches, in addition to being vertical displacement
defined by a sine function, a point moves horizontally towards the direction
of the crest to simulate the phenomenon of water particles filling up the
space under the crest. Conversely, as the crest passes, the point slides back
to its resting position on the horizontal axis. A schematic representation of
this displacement is shown in Fig. 3.

For a vertex point $P_{xyz}=(x,y,z)$ on a surface mesh, its displacement along
a Gerstner wave is calculated as follows:

$W(P_{xyz},\vec{v},\lambda,t)=\begin{cases}P_{xy}+\frac{s}{k}\cos\left(kD-
ct\right)&\textrm{for }(x,y)\\\ P_{z}+\frac{s}{k}\sin(kD-ct)&\textrm{for
}z\end{cases}$ (1) $k=\frac{2\pi}{\lambda}\textrm{,}\quad
D=P_{xy}\cdot\vec{v}\textrm{,}\quad c=\sqrt{\frac{g}{k}}$ (2)

where $\lambda$ is the wavelength, $g$ is gravity, $\vec{v}$ is the 2D unit
vector of the simulation tidal velocity, $s$ is a defined wave steepness
parameter, and $t$ is the animation time interval.

Directly sampling wave orientations from the corresponding quadnodes result in
tiling artifacts. Moreover, rendering a single uniform train of waves
traveling across the water surface presents a non-realistic feature. Existing
works have suggested superimposing multiple waves sampled from increasing
spatial extents, especially for open water bodies [42, 41]. This not only
maintains a degree of realism but also allows the visualization of a regional
principal flow as well as local flow direction.

In our implementation, we use the overlapping bounds suggested by Cornel et
al. [4]. An imaginary tile, $T$ around $P_{xy}$, for a spatial extent spanning
$\lambda$, is defined as $T=[x_{0},x_{1}]\times[y_{0},y_{1}]$, where

$(x_{0},y_{0})=2\lambda\left\lfloor\dfrac{(x,y)}{2\lambda}\right\rfloor\textrm{,}\quad(x_{1},y_{1})=(x_{0},y_{0})+(2\lambda,2\lambda)$
(3)

A wave for $T$ is then calculated using its center position and an average
tidal direction, sampled from each of its four corners. To get an overlapping
region, $T$ is then shifted horizontally and vertically by $\lambda$ such that
the centers of the four tiles
$[T,T+(\pm\lambda,0),T+(0,\pm\lambda),T+(\pm\lambda,\pm\lambda)]$ enclose
$P_{xy}$ in a box of length $\lambda$. Thus, a wave $W_{\lambda}$ for $P_{xy}$
is calculated as the waves bilinear interpolation at each tile center, using
the following horizontal and vertical weights:

$\alpha=\frac{1}{2}-\frac{1}{2}\cos\left(\frac{\Delta
x}{2\lambda}\right)\textrm{,}\quad\beta=\frac{1}{2}-\frac{1}{2}\cos\left(\frac{\Delta
y}{2\lambda}\right)$ (4)

where $\Delta x$ and $\Delta y$ are the distance of $P_{xy}$ along the $x$ and
$y$ directions with respect to the tile centers. For superimposing multiple
waves of increasing spatial extents, waves for $n$ different wavelengths are
generated using $\lambda_{i}=r_{i}2^{i}\lambda_{0}$, where $\lambda_{0}$ is
the shortest considered wavelength and $r_{i}\in[0.8,1.0]$ is a random value
to avoid frequency doubling. The final vertex displacement is determined
using:

$W=W_{\lambda_{0}}+\sum_{i=1}^{n}W_{\lambda_{i}}$ (5)

Since the waves are generated on a triangular mesh with discrete intervals, we
use analytical normals instead of vertex normals for continuous lighting and
shadow effects (see Fig. 3 for this comparison). The normal $\vec{n}$ for each
vertex is the spatial derivative of Eq. 1:

$\vec{n}=\begin{bmatrix}1-D^{2}_{x}s\sin f\\\ D_{x}s\cos f\\\ -D_{x}D_{z}s\sin
f\end{bmatrix}\times\begin{bmatrix}-D_{x}D_{z}s\sin f\\\ D_{z}s\cos f\\\
1-D^{2}_{z}s\sin f\end{bmatrix}$ (6)

Finally, we address two implementation details. First, an averaged
neighborhood tidal velocity is assigned to an undefined node, however since
dry nodes do not have tidal velocities, we set $W_{\lambda_{0}}=0$. Second, to
reduce loss of tidal direction due to averaging, we impose a further
constraint for determining the lowest resolution LoD subtree, such that the
angles between all pair-wise unit vectors of the subtree children should not
exceed 15°.

###  4.2 Immersive Display System

Submerse is intended to be generalized for any display layout and
configuration – a single desktop to a large and immersive setup, a single node
to a multi-node, multi-GPU configuration. For an input display setup, we (1)
determine the scene data extents for each display and use dynamic quadtrees to
allow interactive data update (Sec. 4.2.1), (2) set up a localization and
rendering protocol for the auxiliary display (Sec. 4.3), and (3) configure
optimal camera views for marked POIs (Sec. 4.4).

A display system is defined using the position, orientation, and dimensions of
each screen in the layout, along with a head position. The head is primarily a
reference point that represents a unified position of the display system in
the virtual scene and is used to calculate the camera projection matrix for
each display. At a systems level, Submerse is designed as a client-server
architecture, where the server manages scene synchronization and sends
interaction commands across the client (display) nodes.

Figure 4: Two examples (a) and (b) of our dynamic quadtree update based on
camera motion. Red represents the deleted quadrants and the newly added nodes
are shaded in green.

####  4.2.1 Data Extents and Scene Update

To optimize data retrieval, we have designed our quadtree to contain data for
spatial bounds proportional to the display’s viewing frustum, which is
dynamically updated during scene navigation. Specifically, the bounds of a
quadtree are initialized such that the virtual position of the display lies in
the center of the quadtree and it spans the maximum horizontal extents of the
camera-view frustum pyramid $+$ a buffer region. The camera view frustum for a
screen is determined using its dimensions, relative position from the head,
and camera parameters.

For maintaining an interactive framerate during scene exploration, we have
designed the first-level quadrants of the quadtree (children of the root node)
to be dynamic. That is, the quadrants are selectively updated based on the
camera movement direction. By tracking the displacement of the physical screen
in the virtual scene, we determine which quadrant needs to be updated as the
screen approaches close the quadtree outer boundary. Consequently, instead of
constructing an entire quadtree with new spatial extents, we discard the
quadrants that are beyond a threshold distance from the screen position,
rearrange the pointers of the preserved quadrants to reflect the new root
bounds, and populate nodes for the new quadrants. In Fig.4, we illustrate this
process using two examples of camera movement. In our implementation, the
quadnode updates are performed in parallel using multi-threading and are
either triggered as the user manually navigates through the virtual scene or
scheduled sequentially based on a predefined camera path generated using our
algorithm described in Sec. 4.4.

###  4.3 Auxiliary Display

To support the analysis and decision-making of large datasets, visualization
applications typically provide users with tools that can systematically
present a global overview of the data as well as a detailed depiction of a
selected local area. Naturally, large and immersive display facilities are
most functional when it comes to projecting vast extents of global data,
compared to personalized display screens and desktop setups. However, the
larger the screen real-estate, the more challenging it becomes to design tools
for localized visualization. To this end, we have designed a novel mixed-
reality focus+context technique where the global data is projected on the
surrounding immersive display facility while the localized contextual
visualizations are shown on a hand-held AR-enabled device (which we term an
auxiliary display in the Submerse system).

####  4.3.1 Device Setup

The seamless integration of an auxiliary display into an immersive display
system requires tracking and localization in both the physical space and the
virtual world. In other words, specifically for immersive facilities, the
auxiliary system must accommodate two domains: the user’s physical movements,
including device orientation, and the position of the virtual camera in the
virtual scene. For localization in the physical space, we leverage recent
advances in AR that effectively maps the position of a device in real-time.
Using the defined head position, we are able to determine the physical-to-
virtual space transformation of the auxiliary device. Since the server node is
responsible for the data, scene, and interaction synchronization in the system
workflow, the auxiliary device is likewise connected to the server, which
solves for the physical-to-virtual transformation and determines the
visualizations that need to be rendered on the auxiliary display. Fig. 5
presents a schematic representation of the system protocol.

Here we address an important challenge related to AR-based rendering of the
visualizations on the auxiliary display. Unlike typical AR applications that
render virtual objects onto the physical world, we render virtual objects that
are meant to be augmented onto a scene defined by a virtual world, displayed
on a physical plane (screens). This mapping introduces scale disparities since
two cameras are involved, the auxiliary display camera and the Submerse
virtual camera. To solve this mapping, we utilize render textures on the
server. First, based on the device localization, the required visualizations,
such as the dip-stick or custom data visualizations shown in Fig. 5, are
rendered in an empty virtual scene with extents similar to the flooding scene.
Next, we use the Submerse camera parameters to bake a view of the rendered
visualization onto a virtual texture that replicates the size and layout of
the display facility. Thus, instead of rendering virtual objects from the
scene, the auxiliary display rasterizes the visible parts of the render
texture based on the device camera parameters.

One final matter in the system workflow deserves attention. Given the limited
GPU and compute resources of a mobile device, user experience can be
significantly affected when rendering complex scenes and managing large
datasets. To offload this burden from the auxiliary device, we have
implemented a web-based remote rendering module into our system. As a result,
the rendering is performed on the server and the final result is a series of
textures received by the device over the network.

Figure 5: System protocol of our AR-based auxiliary display. (a) and (b) are
two example applications that show the flooding level with respect to building
height (on a diverging color map of green to yellow to red, with red being
most sever) and a heatmap of population density, respectively.

###  4.4 Camera View Finding and Navigation Trajectory

We have designed an automatic camera viewpoint-finding algorithm based on
factors that maximize POI visibility with respect to the virtual scene and
physical display layout. Below, we describe each term of our objective
function, followed by an algorithm we have designed for refining the optimal
views based on user-defined thresholds and generating a path for navigating
through the viewpoints. Mathematically, a POI is defined using its 3D position
and a scalar value indicating its expansion radius, and a camera viewpoint,
$v$ is defined using its 3D position, and yaw and pitch angles. For all the
terms described below, $E_{X}$ is a large-valued penalty score to constrain
the system to satisfy the respective user-defined threshold value or
condition.

View coverage: To anchor a POI in the center of the camera view frustum, we
define the term view coverage, $R(v)$ as the combination of the eccentricity
of a POI’s projection on the camera view, $H_{e}(v)$, and the alignment
between the camera up vector and $v$, denoted as $H_{u}(v)$:

$R(v)=\begin{cases}E_{R},&\text{if $a_{p}>A_{p}$}\\\
\omega_{u}\left\|\frac{2a_{p}}{\pi}\right\|^{2}-\left(\overrightarrow{pq}\cdot\vec{d}\right),&\text{otherwise}\end{cases}$
(7)

where $\omega_{u}$ is a weight to control the relative influence between
eccentricity and viewing direction alignment, $a_{p}$ is the pitch angle of
the camera pose, $p$ and $q$ are the positions of the camera and POI
respectively, $\vec{d}$ is the camera look-at direction, and
$A_{p}\in[0,\pi/2]$ is a user-defined maximum allowable pitch angle for the
camera.

POI visibility: To ensure effective visibility of a POI, we penalize a
candidate view where the POI is occluded by a building, scene object, or
another POI using:

$O(v)=\begin{cases}E_{O},&\text{if this POI is occluded}\\\
0,&\text{otherwise}\end{cases}$ (8)

We use a discretized visibility grid to efficiently check if the POI is
occluded by any cell when projected onto the camera view. A cell in the
visibility grid is defined using a cuboid with a height equal to the maximum
height of the 3D objects occupying the cell bounds (except the water mesh).
Moreover, we define an additional term $D(v)$ to control the closeness of the
POI to the display:

$D(v)=\begin{cases}E_{D},&\text{if $\frac{||q-p||^{2}}{k}>K$ }\\\
\frac{||q-p||^{2}}{k},&\text{otherwise}\end{cases}$ (9)

where $k$ is a constant to normalize the distance calculate and $K$ is the
user-defined maximum allowable distance of the POI from the camera.

Coastal vicinity: Often, for POIs on the coastline, it is preferred that the
camera view is in the direction of shore-to-POI, so that the effect of the
progression of the coastal flood approaching the POI may be observed. To this
end, we model this preference as location preference $L(v)$.

$L(v)=\begin{cases}0,&\text{if in offshore locations on map}\\\
E_{L},&\text{otherwise}\end{cases}$ (10)

Final objective: Thus, to find a set of optimal viewpoints, $C$, for $M$ POIs,
we define the following objective function with customizable weights:

$\displaystyle\begin{split}C&=\operatorname*{argmin}E(v)\end{split}$ (11)
$\displaystyle\begin{split}E(v)&=\sum^{M}{(\Omega+R_{i}(v)+\omega_{o}O_{i}(v)+\omega_{d}D_{i}(v)+\omega_{l}L_{i}(v))}\end{split}$
(12)

We empirically determine the weights to be
$\omega_{u}=0.5,\Omega=10.0,\omega_{o}=1.0,\omega_{d}=1.0,\omega_{l}=0.50$ and
the penalties $E_{X}=100.0$. For $N$ displays, the total objective is
calculated as:

$\displaystyle\begin{split}E(v)=\sum^{N}_{i}{\sum^{N^{{}^{\prime}}}_{j}{E_{i}(v_{j})}}\quad\quad
N^{{}^{\prime}}=\\{x\in N\mid x\neq i\\}\end{split}$ (13)

Eq. 13 additionally accumulates a score for all remaining displays in the
layout by rotating a candidate camera viewpoint $v$, along its look-up axis,
with the relative yaw angle between the display pairs. This allows us to
obtain a set of minimum camera viewpoints for multiple POIs based on the
display layout. We further demonstrate this using 2 layouts in our results
presented in Sec. 6.

The arbitrary terrain elevation and building heights create a complex search
space for resolving a minima since the derivative of a sample point
contributes little to locating an optimal candidate. Therefore, we chose
Particle Swarm Optimization (PSO) [43], a derivative-free optimization method
that does not depend on a sampled gradient or covariance matrix of the sample
set, to find a set of optimal viewpoints with minimum objective function
score.

Given a set of viewpoints next we calculate the transition trajectory between
viewpoints pairs. A straight line segment is preferred for connecting
viewpoint pairs, as it avoids unnecessary rotation that can lead to VR
sickness [12]. However, in our design, we also aim to avoid occlusion with
scene objects (buildings in particular). Therefore, if a straight line
trajectory is interrupted, we transform the path to a quadratic Bézier curve.
The middle control point of the curve is determined by a vector rooted at a
point off the coast or the further point that results in no intersection.
Choosing a point off the coast is a preference expressed by domain experts so
that while navigating, one may also get a chance to view how the impact of
coastal flooding – synonymous with disaster analysis protocol undertaken by
emergency managers. To reduce VR sickness due to sharp curvature [12], we set
the vector length to be three times the distance between the center point and
the straight line.

Figure 6: We demonstrate Submerse on three display systems: (a) single-screen,
(b) tiled-display, and (c) Reality Deck, a room-sized gigapixel immersive
facility. Inset (d) shows a result of our wave synthesis and (e) illustrates
the detail that can be resolved approaching the facility walls.

##  5 Implementation Details

Submerse is developed using Unity 3D [44] game engine. The modular and drag-
and-drop interface of Unity automatically provides an interactive framework
for configuring and amending the individual components of Submerse listed in
the previous section or adding new ones based on application needs. Moreover,
using Unity AR Foundation [45] we are able to perform the localization on any
auxiliary device, irrespective of the underlying operating system. In keeping
up with the agnosticism of devices, we implement our remote rendering
framework using the WebRTC package for Unity [46].

To set up the display system layout, we interactively define the physical
width, height, and relative placement and orientation of the virtual screens
using quads as a visual guide (see Fig. 6). A virtual screen is a canvas that
can comprise either a single or multiple monitors, typically configured at the
operating system or the GPU driver level. For each virtual screen, a set of
attributes need to be configured, namely, for multi-node systems, its network
address, and for multi-GPU systems, the GPU port number. Using these
configurations, we can launch instances of Submerse on the respective display
nodes for the respective virtual screens. A server application then connects
to each client instance and performs its synchronization and interaction tasks
in parallel, implemented using Unity’s networking protocol.

To ensure an interactive framerate, we perform the computations for both
vertex height interpolation and wave generation, on the GPU using shaders. At
the application startup and during the scene update, a triangular mesh with
intervals equal to the quadnode length is generated. By initializing a coarse
mesh on the CPU, we optimize the number of triangle draw calls on the GPU by
performing hardware-level tessellation. To this end, we apply two tessellation
passes on each triangle. For the first pass, we subdivide a triangle based on
its distance from the camera. The tessellation factor is maximum when the
camera is at a minimum defined distance from the water surface (e.g., our
default value is 100 meters), and gradually decreases until it reaches a
maximum defined distance (e.g., our default value is 2000 meters). If the
triangles in the initial mesh have largely varying edge lengths, the output
from the distance-based tessellation pass will result in smaller triangles
being tessellated more than larger triangles. Thus, in the second tessellation
pass, we further refine the tessellated triangles in the camera frustum to
have a maximum allowable edge length. Although we empirically set this value
to be one-tenth of the smallest quadnode length, this can be changed based on
available GPU resources.

A limitation of using height-field grids is that information is cell-centered
rather than spatially localized. For new vertices generated during
tessellation, it becomes challenging to check for intersections with other
scene objects accurately. This is important to avoid rendering artifacts where
the wave is only partially visible on the terrain. Thus, in our shader design,
we cache the interpolated height of each vertex, along with its crest and
trough heights, and perform a depth test using a depth buffer. Since the wave
amplitude in our design is solely to induce a perception of the direction of
flow and does not hold any numerical significance, we perform the following
operations in the surface shader: (1) if the cached interpolated height is
projected higher in the depth buffer than other scene objects and either the
crest or trough is occluded, the amplitude is reduced to a minimal value (in
our case, 1mm), (2) if the interpolated height is occluded in the depth
buffer, it is shaded transparent.

##  6 Results and Evaluation

We demonstrate Submerse for a simulated flooding scenario of superstorm Sandy
impacting NYC [47], visualized on three kinds of display modalities: a single-
screen desktop, a single-wall multi-GPU tiled-display setup (tiled-display),
and the Stony Brook Reality Deck (RD), a room-sized 1.5 gigapixel-resolution
immersive display facility. The specifications and layout for each display
system are illustrated in Fig. 6(a)-(c), respectively. All three systems are
equipped with NVIDIA Quadro RTX6000 GPU hardware and Intel Xenon 2.80GHz CPU
with 128GB of RAM.

The flooding data is generated using the ADCIRC simulation model and spans the
NYC area ($302.6\text{mi}^{2}$), containing 908,000 datapoints $\times$ 288
timepoints (30 mins interval). For constructing a NYC virtual scene, we used
publically available datasets [48] to integrate a DEM, texture map it using
the highest available resolution aerial NYC images [49], and 3D buildings.
Consequently, for this scenario, Submerse managed a total data size of 15GB
(excluding aerial textures which we directly streamed from the internet at
runtime).

In terms of runtime performance, the startup time for the single-display,
tiled-display, and RD, at the highest LoD, were 66s, 110s, and 145s,
respectively. This accounts for 3D object loading, quadtree construction, and
floodwater surface rendering. The texture mapping was performed in parallel,
streamed directly from the internet, and therefore not included in the time
performance. Once initialized, Submerse was able to synthesize and render the
animated waves, at a framerate of 40 frames-per-second (fps). The inset (d) in
Fig. 6 shows an example of waves synthesized using our approach, for a
waterbody area in the scene. During manual scene navigation, updating the
quadree using our dynamic approach maintained 25-30 fps. However, navigating
the scene automatically using our camera path algorithm did not affect the
framerate, since the path is pre-calculated and the quadtree is updated
preemptively in parallel, as the camera is moved along the path.

Figure 7: Camera views and navigation path generated for (a) a single screen
and (b) four-walled RD. The red boxes are user-marked POIs.

Three areas in NYC were of particular interest to our domain collaborators for
analysis: the road and open space for a potential evacuation point next to the
coast at the eastern side of Manhattan, the Battery park and tunnel area, and
road infrastructure at both ends of a bridge on the western end of Manhattan.
For the POIs marked using red boxes in Fig. 7, we show the results for our
automatic camera viewpoints and path generation for the tiled-display RD in
(a) and (b), respectively. Since the tiled-display is a single-screen setup
with a relatively small FoV, we adjust the default weights for the viewpoint
optimization such that the POI is positioned towards the center of the display
($\omega_{u}=0.9$, $A=\pi/12$) and nearer to the camera ($\omega_{d}=0.75$,
$K=500m$). Conversely, utilizing the very large FoV of the RD, we adjusted the
weights such that the POI could be anywhere within the view of the four walls
($\omega_{u}=0.1$, $A=\pi/6$), and enable comparison by allowing multiple POIs
to be viewed at an instance ($\omega_{d}=0.25$, $K=1000m$). For both displays,
1000 particles were randomly initialized for the PSO algorithm. It took 120s
to determine camera views for the single display and 75s for the RD.

Finally, in Fig. 8 we demonstrate how users can explore the data by either
performing natural pan-and-zoom interactions, as in Fig. 8(a), or the
auxiliary display system to view additional contextual information, as shown
in Fig. 8(b). Additionally, using a game controller, users can manually
navigate the scene or use our automatic navigation path to view the marked
POIs, and update the timepoint of the flooding scenario. Based on domain
expert feedback, we have implemented the following visualization and
interactions for focus+context on the auxiliary display:

Figure 8: Interaction with data in the RD can be performed either (a)
naturally or (b) using our auxiliary display system.

Dip-stick metaphor: Although color and opacity transfer functions provide
both, a qualitative and quantitative visual sense of the flooding depth,
meteorologists additionally prefer to note quantitative measurement values.
Desktop-based applications achieve this by providing a legend or designing a
mouse-based tool-tip to display the depth value. To adopt a natural way of
measuring depth, we render a ruled dip-stick on the auxiliary display, as
shown in Fig. 8(b), that shows water markings on the measurements when the
device is pointed to the water surface mesh.

Selection and occlusion removal for buildings: Since the auxiliary display can
complement interaction metaphors such as selection, we provide users the
ability to point and select a specific building to view additional
information. Fig. 8(c) demonstrates such an example where for a selected
building, we change its color to represent the flooding level severity
relative to its height (also see Fig. 5(a), and a label with information such
as its name, location (address, along with latitude and longitude), and any
additional related information provided by the user. Moreover, to enhance
visibility, we remove scene objects and buildings that occlude the selected
building in the auxiliary display camera view and in a radius around the
building.

Visualizing additional contextual information: For any domain-specific
additional data, we allow users to (1) select the 3D scene object class for
which the data is to be visualized, namely, water surface mesh, 3D buildings,
or a new plane spanning the scene extents, and (2) configure a transfer
function for the rendering. Fig. 5(b) is an example of this feature,
demonstrating a heatmap of NYC population density atop the flooding scenario.

###  6.1 Domain Expert Feedback

To evaluate our work and the effectiveness of using Submerse in an immersive
setting, we conducted two workshop spaces, one specialized with domain experts
and the other with college students without domain knowledge.

For the specialized workshop, we invited 10 participants with backgrounds in
research and academia, NYC emergency managers, and National Weather Services
(NWS). In following NWS protocol, an informational orientation of an upcoming
storm was presented to the participants to imitate a situation where experts
engage in preparedness ahead of a weather event. The identity of the storm
simulation dataset was not revealed to the participants as it would have
influenced their perception due to prior knowledge or memory. The participants
were first asked to express and follow the steps they would have normally
taken to analyze the provided information. Following that, they were
collectively shown Submerse in the RD, for POIs marked in Fig. 7. All experts
appreciated the visualizations presented to them. One unanimous comment was
that the 3D visualization and sense of realism, coupled with immersion,
positively transformed the way they planned for preparation, evacuation, and
resilience measures, in contrast to currently adopted 2D methods.
Specifically, the to-scale rendering aided them in having a clearer spatial
perception of the flooding levels with respect to the surroundings, for
instance, the buildings, streets, and open areas. To this end, the
participants found most effective what they called the street-view, as shown
in Fig. 8 and Fig. 9(a)-(c), as they could visualize the flood level with
respect to their own height. One participant reflected that “the display of
flooding in 3D with actual buildings was a much more effective method of
communicating risks, rather than a numerical forecast or a 2D risk map.”
Another participant commented that the visualizations in the RD felt like “a
helicopter ride during a catastrophe where [they] get a wholesome view of the
city, and then go down at street level to investigate its intensity.”
Moreover, they expressed that the shared experience exploring the data allowed
for greater insights as participants were able to instantaneously take into
account the diversity of expertise and domains for their analysis and
decisions. An example of this was witnessed when an atmospheric scientist
explained to an emergency manager how an incoming storm surge would result in
a drastic increase in flooding levels of an area and thus may be considered a
priority in evacuation plans.

Feedback from the first workshop enabled us to address some shortcomings,
consequently directing us in conducting our research and developing the
auxiliary display framework, automatic viewpoint and scene navigation, and
synthesizing the tidal waves. We conducted a second specialized workshop with
10 students in atmospheric sciences, particularly emphasizing the new Submerse
features. The auxiliary display was welcomed by the participants, especially
since they could now make quantitative measurements and seamlessly integrate
their domain data visualization for personal-level analysis as well as share
their findings collaboratively. Occlusion removal was often utilized to get a
clearer view of the surrounding landscape and road network. Compared to manual
navigation, all participants preferred the automatic camera views and
navigation. At some instances, participants requested to change the camera
views for exploring the area around the POI. A few participants however
experienced VR sickness during the navigation and one suggested that they
would have preferred an instant scene change from one POI to the next.
Finally, the participants greatly acknowledged wave synthesis for flow
direction. Results from Fig. 9(a)-(c), the Battery Tunnel area, were of a
particular learning experience. For each incremental timepoint, the
participants were able to observe, for the first time, how tidal surge from
different directions contributed to the increasing flooding levels in the
area. This information was not perceivable otherwise using a still water
surface mesh rendering.

A separate workshop with 80 college students was conducted to evaluate the
impact of standard weather forecasting graphics compared to Submerse
visualizations. Following a pre-recorded narration of a storm weather
forecast, the participants were presented with a hurricane flooding scenario
for a fictitious college campus. We deployed Submerse on a single-screen
desktop setting, without an auxiliary display. In terms of taking action in
favor of preparedness, participants agreed that the visualizations aided their
decision-making. A comment by a participant that led to general consensus was
that the ability to “see” the flooding, which would otherwise be narrated
using words, such as “we are expecting 3-5 feet of storm surge flooding,” or
2D graphs, not only allowed them to understand the forecast severity but also
made them more likely to avoid that situation.

Figure 9: Scenes from the specialized workshop showing NYC in (a)-(c), and the
college workshop showing a fictitious scene in (d)-(f). For both, we show no
flooding, mid-level, and maximum flooding, respectively.

##  7 Conclusion and Future Work

We have presented Submerse, an application for visualizing flooding scenarios
rendered in 3D using numerical simulation and GIS data, along with a scalable
framework for deploying the application in custom immersive display systems.
For rendering the flooding level, we have developed a real-time approach that
constructs a smooth surface mesh, coupled with animated tidal waves to depict
the flooding direction. We achieve this by constructing a dynamic quadtree
datastructure that interactively updates based on the camera view frustum and
is utilized to implement an adaptive grid for sampling and interpolation of
the mesh height values and wave direction. Specifically for immersive visual
systems, we have introduced two novel interaction modalities: an automatic
camera view-finding and path-generation algorithm, and an AR-based auxiliary
display system. We find optimal camera views for marked POIs based on the
screen layout and viewing parameters of the POI with respect to the layout.
For traversing through the views, our path-generation algorithm is designed to
avoid collision and reduce VR-sickness, while also facilitating effective
scene exploration during the navigation. The auxiliary display is a unique
design that compliments the visualizations on the shared screen space by
displaying additional contextual information about the virtual scene, using
AR. The resulting visualizations and particularly the utility of an immersive
display system have been positively evaluated by domain experts and other
stakeholders.

Based on feedback received during evaluation, we identified a limitation that
while Submerse enhances a sense of realism , it does not fully achieve the
fearures of a decision system. Therefore, for future work, we plan to
integrate tools that will allow stakeholders to visualize outcomes by
interactively applying scenarios for mitigation and evacuation measures.
Moreover, one natural limitation of large immersive facilities is their
immobile nature. To this end, we plan to extend the Submerse system to support
VR HMDs and include a framework for co-located as well as remote collaboration
across all devices. Finally, we aim to conduct further research to extend the
Submerse visualization and system framework for stereo display systems.

## Acknowledgments

This project was supported in part by NSF grants CNS1650499, OAC1919752,
ICER1940302, and IIS2107224.

## References

  * [1] C. B. Field, V. R. Barros, M. D. Mastrandrea, K. J. Mach, M.-K. Abdrabo, N. Adger, Y. A. Anokhin, O. A. Anisimov, D. J. Arent, J. Barnett _et al._ , “Summary for policymakers,” in _Climate change 2014: impacts, adaptation, and vulnerability._ Cambridge University Press, 2014, pp. 1–32.
  * [2] “DAT: The International Disasters Database,” https://www.emdat.be/, accessed: 2022-02-25.
  * [3] A. H. Murphy, “What is a good forecast? an essay on the nature of goodness in weather forecasting,” _Weather and Forecasting_ , vol. 8, no. 2, pp. 281–293, 1993.
  * [4] D. Cornel, A. Konev, B. Sadransky, Z. Horvath, E. Gröller, and J. Waser, “Visualization of object-centered vulnerability to possible flood hazards,” _Comp. Graph Forum_ , vol. 34, no. 3, pp. 331–340, 2015.
  * [5] D. Cornel, A. Buttinger-Kreuzhuber, A. Konev, Z. Horváth, M. Wimmer, R. Heidrich, and J. Waser, “Interactive visualization of flood and heavy rain simulations,” _Comp. Graph Forum_ , vol. 38, no. 3, pp. 25–39, 2019\.
  * [6] R. Ball and C. North, “Realizing embodied interaction for visual analytics through large displays,” _Computers & Graphics_, vol. 31, no. 3, pp. 380–400, 2007.
  * [7] C. Andrews, A. Endert, B. Yost, and C. North, “Information visualization on large, high-resolution displays: Issues, challenges, and opportunities,” _Info. Vis._ , vol. 10, no. 4, pp. 341–355, 2011.
  * [8] B. Laha, K. Sensharma, J. D. Schiffbauer, and D. A. Bowman, “Effects of immersion on visual analysis of volume data,” _IEEE Trans Vis Comput Graph_ , vol. 18, no. 4, pp. 597–606, 2012.
  * [9] J. M. Mol, W. W. Botzen, and J. E. Blasch, “After the virtual flood: Risk perceptions and flood preparedness after virtual reality risk communication,” _Judgm. Decis. Mak._ , vol. 17, no. 1, p. 189, 2022.
  * [10] B. Kim and P. Tsiotras, “Image segmentation on cell-center sampled quadtree and octree grids,” _Wavelet Applications in Industrial Processing VI_ , vol. 7248, pp. 139–147, 2009.
  * [11] X. Bonaventura, M. Feixas, M. Sbert, L. Chuang, and C. Wallraven, “A survey of viewpoint selection methods for polygonal models,” _Entropy_ , vol. 20, no. 5, p. 370, 2018.
  * [12] P. Hu, Q. Sun, P. Didyk, L.-Y. Wei, and A. E. Kaufman, “Reducing simulator sickness with perceptual camera control,” _ACM Trans. Graph._ , vol. 38, no. 6, p. Article 210, Nov. 2019.
  * [13] C. Papadopoulos, K. Petkov, A. E. Kaufman, and K. Mueller, “The Reality Deck–an Immersive Gigapixel Display,” _IEEE Computer Graphics and Applications_ , vol. 35, no. 1, pp. 33–45, 2015.
  * [14] J. G. Leskens, C. Kehl, T. Tutenel, T. Kol, G. d. Haan, G. Stelling, and E. Eisemann, “An interactive simulation and visualization tool for flood analysis usable for practitioners,” _Mitigation & Adaptation Strategies for Global Change_, vol. 22, no. 2, pp. 307–324, 2017.
  * [15] National Oceanic and Atmospheric Administration , “Sea Level Rise Viewer,” https://coast.noaa.gov/slr/#, accessed Oct 26, 2022.
  * [16] D. Leedal, J. Neal, K. Beven, P. Young, and P. Bates, “Visualization approaches for communicating real-time flood forecasting level and inundation information,” _Journal of Flood Risk Management_ , vol. 3, no. 2, pp. 140–150, 2010.
  * [17] M. Vuckovic, J. Schmidt, T. Ortner, and D. Cornel, “Combining 2D and 3D Visualization with Visual Analytics in the Environmental Domain,” _Information_ , vol. 13, no. 1, p. 7, 2021.
  * [18] N. Lipp, N. Dużmańska-Misiarczyk, A. Strojny, and P. Strojny, “Evoking emotions in VR: schema activation via a freeze-frame stimulus,” _Virtual Reality_ , vol. 25, no. 2, pp. 279–292, 2021.
  * [19] K. Kroesl, H. Steinlechner, J. Donabauer, D. Cornel, and J. Waser, “Master of Disaster: Virtual-Reality Response Training in Disaster Management,” _ACM International Conference on Virtual-Reality Continuum and its Applications in Industry_ , 2019.
  * [20] M. T. Oyshi, V. Maleska, J. Schanze, F. Bormann, R. Dachselt, and S. Gumhold, “Floodvis: Visualization of climate ensemble flood projections in virtual reality,” _EG Workshop on Visualisation in Environmental Sciences_ , pp. 27–359, 2022.
  * [21] P. Reipschlager, T. Flemisch, and R. Dachselt, “Personal augmented reality for information visualization on large interactive displays,” _Transactions on Visualization and Computer Graphics_ , vol. 27, no. 2, pp. 1182–1192, 2020\.
  * [22] S. Chen, H. Wu, Z. Lin, C. Guo, L. Lin, F. Hong, and X. Yuan, “Photo4action: phone camera-based interaction for graph visualizations on large wall displays,” _Journal of Visualization_ , vol. 24, no. 5, pp. 1083–1095, 2021\.
  * [23] A. Nishimoto and A. E. Johnson, “Extending virtual reality display wall environments using augmented reality,” in _Symposium on Spatial User Interaction_ , 2019, pp. 1–5.
  * [24] J. Davis and X. Chen, “Lumipoint: Multi-user laser-based interaction on large tiled displays,” _Displays_ , vol. 23, no. 5, pp. 205–211, 2002.
  * [25] C. Ganser, A. Steinemann, and A. Kunz, “Infractables: Multi-user tracking system for interactive surfaces,” in _IEEE Virtual Reality Conference_ , 2006, pp. 253–256.
  * [26] R. Langner, U. Kister, and R. Dachselt, “Multiple coordinated views at large displays for multiple users: Empirical findings on user behavior, movements, and distances,” _IEEE Trans Vis Comput Graph_ , vol. 25, no. 1, pp. 608–618, 2018.
  * [27] M. R. Morris, A. Huang, A. Paepcke, and T. Winograd, “Cooperative gestures: multi-user gestural interactions for co-located groupware,” _ACM CHI_ , pp. 1201–1210, 2006.
  * [28] S. Malik, A. Ranjan, and R. Balakrishnan, “Interacting with large displays from a distance with vision-tracked multi-finger gestural input,” _ACM UIST_ , pp. 43–52, 2005.
  * [29] J. O. Wobbrock, M. R. Morris, and A. D. Wilson, “User-defined gestures for surface computing,” _ACM CHI_ , pp. 1083–1092, 2009.
  * [30] S. Siddhpuria, S. Malacria, M. Nancel, and E. Lank, “Pointing at a distance with everyday smart devices,” _ACM CHI_ , pp. 1–11, 2018.
  * [31] T. Babic, H. Reiterer, and M. Haller, “Understanding and creating spatial interactions with distant displays enabled by unmodified off-the-shelf smartphones,” _Multimodal Technologies and Interaction_ , vol. 6, no. 10, p. 94, 2022.
  * [32] R. Langner and R. Dachselt, “Towards visual data exploration at wall-sized displays by combining physical navigation with spatially-aware devices,” in _IEEE VIS Poster_ , 2018.
  * [33] T. Kamada and S. Kawai, “A simple method for computing general position in displaying three-dimensional objects,” _Computer Vision, Graphics, and Image Processing_ , vol. 41, no. 1, pp. 43–56, 1988.
  * [34] M. Feixas, E. Del Acebo, P. Bekaert, and M. Sbert, “An information theory framework for the analysis of scene complexity,” _Computer Graphics Forum_ , vol. 18, no. 3, pp. 95–106, 1999.
  * [35] T. Arbel and F. P. Ferrie, “Viewpoint selection by navigation through entropy maps,” _IEEE Intl Conf. Computer Vision_ , vol. 1, pp. 248–254, 1999.
  * [36] P.-P. Vázquez, M. Feixas, M. Sbert, and W. Heidrich, “Viewpoint selection using viewpoint entropy.” _Sixth International Fall Workshop on Vision, Modeling, and Visualization_ , pp. 273–280, 2001.
  * [37] S. L. Stoev and W. Straßer, “A case study on automatic camera placement and motion for visualizing historical data,” _IEEE Visualization_ , pp. 545–548, 2002.
  * [38] H. Huang, D. Lischinski, Z. Hao, M. Gong, M. Christie, and D. Cohen-Or, “Trip synopsis: 60km in 60sec,” _Computer Graphics Forum_ , vol. 35, no. 7, pp. 107–116, 2016.
  * [39] L. Lin, Y. Liu, Y. Hu, X. Yan, K. Xie, and H. Huang, “Capturing, reconstructing, and simulating: the UrbanScene3D dataset,” _European Conference on Computer Vision_ , pp. 93–109, 2022.
  * [40] Q. Liang, “A structured but non-uniform Cartesian grid-based model for the shallow water equations,” _International Journal for Numerical Methods in Fluids_ , vol. 66, no. 5, pp. 537–554, 2011.
  * [41] D. Hinsinger, F. Neyret, and M.-P. Cani, “Interactive animation of ocean waves,” _EG Symp on Computer Animation_ , pp. 161–166, 2002.
  * [42] S. Thon, J.-M. Dischler, and D. Ghazanfarpour, “Ocean waves synthesis using a spectrum-based turbulence function,” _IEEE Proceedings Computer Graphics International_ , pp. 65–72, 2000.
  * [43] J. Kennedy and R. Eberhart, “Particle swarm optimization,” in _International Conf. on Neural Networks_ , vol. 4, 1995, pp. 1942–1948.
  * [44] Unity Technologies, “Unity3D,” https://unity3d.com/unity/.
  * [45] Unity, “Unity ar foundation,” https://unity.com/unity/features/arfoundation, online; accessed Oct 26, 2022\.
  * [46] Unity, “WebRTC for Unity,” https://github.com/unity-technologies/com.unity.webrtc, accessed Oct 26, 2022.
  * [47] B. A. Colle, M. J. Bowman, K. J. Roberts, M. H. Bowman, C. N. Flagg, J. Kuang, Y. Weng, E. B. Munsell, and F. Zhang, “Exploring water level sensitivity for metropolitan new york during sandy (2012) using ensemble storm surge simulations,” _Journal of Marine Science and Engineering_ , vol. 3, no. 2, pp. 428–443, 2015.
  * [48] NYC Planning, “NYC BYTES of the BIG APPLE,” https://www.nyc.gov/site/planning/data-maps/open-data.page, online; accessed Oct 26, 2022.
  * [49] Open Street Maps, “Open StreetMap,” https://wiki.openstreetmap.org/wiki/Main_Page, accessed Oct 26, 2022.

|  Saeed Boorboor is currently persuing a Ph.D. degree in Computer Science at
Stony Brook University. He received his Bachelor degree in Computer Science
from School of Science and Engineering, Lahore University of Management
Sciences, Pakistan. His research interests include scientific visualization,
medical imaging, and computer graphics.
---|---
|  Yoonsang Kim is currently persuing a Ph.D. degree in Computer Science at
Stony Brook University. He received his Bachelor’s degree in Computer Science
from Soongsil University, South Korea, and Master’s degree in Computer Science
from Stony Brook University. His research interests include augmented reality,
virtual reality, data visualization, computer graphics, and human-computer
interaction.
---|---
|  Ping Hu is currently pursuing a Ph.D. degree in Computer Science at Stony
Brook University. She received her Bachelor degree of Physics at Shandong
University, China. Her research focuses on computational camera control in
scientific visualization and computer graphics.
---|---
|  Josef Moses is currently persuing a Bachelor degree and Master degree in
Atmospheric Science at Stony Brook University. His research interests include
meteorology, risk communication, severe weather communication, and social
science.
---|---
|  Brian A. Colle is a Professor of Atmospheric Sciences and the Division Head
for the Atmospheric Sciences Division (2018-present) at Stony Brook
University, School of Marine and Atmospheric Sciences (SoMAS). He has
published over 125 papers during the last 25 years on various aspects of
extreme weather and climate, including the application and improvement of
weather prediction models for these events. He is a Fellow of the American
Meteorological Society. He received his Ph.D. in Atmospheric Sciences from the
University of Washington (1997).
---|---
|  Arie E. Kaufman is a Distinguished Professor of Computer Science, Director
of Center of Visual Computing, and Chief Scientist of Center of Excellence in
Wireless and Information Technology at Stony Brook University. He served as
Chair of Computer Science Department, 1999-2017. He has conducted research for
$>$40 years in visualization, VR and graphics and their applications, and
published $>$350 refereed papers. He was the founding Editor-in-Chief of IEEE
TVCG, 1995-98. He is an IEEE Fellow, ACM Fellow, National Academy of Inventors
Fellow, recipient of IEEE Visualization Career Award (2005), and inducted into
Long Island Technology Hall of Fame (2013) and IEEE Visualization Academy
(2019). He received his Ph.D. in Computer Science from Ben-Gurion University,
Israel (1977).
---|---