0 Research

# C-D ratio in multi-display environments

Travis Gesslein
Mixed Reality Labor e-mail<EMAIL_ADDRESS>Hochschule Coburg  Jens
Grubert
Mixed Reality Labor e-mail<EMAIL_ADDRESS>Hochschule Coburg

###### Abstract

Research in user interaction with mixed reality environments using multiple
displays has become increasingly relevant with the prevalence of mobile
devices in everyday life and increased commoditization of large display area
technologies using projectors or large displays. Previous work often combines
touch-based input with other approaches, such as gesture-based input, to
expand the possible interaction space or deal with limitations of other two-
dimensional input methods. In contrast to previous methods, we examine the
possibilities when the control-display (C-D) ratio is significantly smaller
than one and small input movements result in large output movements. To this
end one specific multi-display configuration is implemented in the form of a
spatial-augmented reality sandbox environment, and used to explore various
interaction techniques based on a variety of mobile device touch-based input
and optical marker tracking-based finger input. A small pilot study determines
the most promising input candidate, which is compared to traditional touch-
input based techniques in a user study that tests it for practical relevance.
Results and conclusions of the study are presented.

H.5.2HCIMultimedia Information SystemsInput Methods; I.3.7Computer
GraphicsThree-Dimensional Graphics and RealismVirtual reality, augmented
reality, mixed reality

Introduction

When using any arbitrary input device, some process has to map the performed
input onto some output (such as position). This mapping process is not limited
to virtual or augmented reality environments, but can be achieved in the real
world as well. The relation of how some input is mapped onto output is the C-D
ratio:

$CD_{ratio}=\frac{\Delta x}{\Delta X}$ (1)

It maps a movement distance $\Delta x$ in physical space to a mapped movement
distance $\Delta X$ in the output space. C-D ratios of one or close to one are
commonly used in application such as smartphones where a touch on the display
surface is usually mapped directly to, for example, a virtual button press in
the same position on the display. In contrast, applications where small inputs
are mapped onto large outputs become relevant when the space being interacted
with is larger than the input device. C-D ratios close to one can still be
used in these cases, but it is desirable to investigate C-D ratios that
directly map some input on the small input device onto a target in the larger
display space in one step. One application field where such problems appear
are multi-display environments, where multiple displays can be combined to
overcome limitations of single-display technologies. The experimentation setup
described here in particular uses an spatial augmented reality environment
based on the RoomAlive Toolkit [19] which combines a projector that projects
an image onto some arbitrarily shaped surface with a commodity smartphone that
is able to show part of the same projected environment on its screen in higher
resolution, while also offering the ability to interact with the space
directly using touch controls. In the following we show separate but related
simple mathematical models for mapping positional input onto output positions.
Several novel input methods that map small input spaces onto large output
spaces based on the described mathematical frameworks are presented and
implemented. Developed input methods are evaluated and put in contrast with
each other in a user study, which offers insight into practical relevance of
small C-D ratio mappings.

##  1 Related Work

[3] suggests that user performance is dependent on movement in the physical
control space (also called motor space), as opposed to output movement that
results from the C-D ratio mapping on the target display. This presents an
opportunity for improvement for problems such as the occlusion problem of
touch interfaces presented in [25] by decoupling output movement from input
movement. [16] investigates zooming as a potential C-D ratio modifier by
introducing a method to automatically scale a document that is read by the
user based on the current scrolling speed through the document. The technique
is similar to Z-Scaling, presented here later. One popular mode of 3D input
are gesture-based methods. Previous work such as [24] [29] [14] [17] [17]
describe techniques where input of traditional input devices is augmented with
hand-gestures that are recognized using the support of added hardware and
algorithms. [23] introduces the idea of a continuous interaction space, and
publications in the following years use this concept, such as[4] and [27],
which seek to unify gesture and touch-based input, or [11] which expands the
touch input area using air above and around a smartphone. Instead of gesture
based input, the work in this thesis makes use of positional finger
information above a smartphone surface to implement movement akin to mouse
movement. Much research into 3D target acquisition exists in the fields of
virtual and augmented reality. [1] surveys more than 30 of these VR and AR
object selection techniques and presents some conclusions as to the main
concerns and limitations of their design. In [8], Paul Fitts presents a
predictive mathematical model for human movement in target selection, known as
Fitts’ law, and [21] introduces the now commonly used reformulation of the
index of difficulty appearing in Fitts’ Law in terms of the Shannon-Hartley
theorem [13]

$ID=log_{2}(\frac{D}{W}+1)$ (2)

in order to deal with limitations of the original formulation, where D and W
are distance to the target and W its width (or radius), respectively. This
thesis does not examine the correctness of results in tests of the presented
input methods with respect to Fitts’ law, but the definition above is still
used as a guideline for the relationship between the size and distance of a
target that is shown to the user. [19] introduces the RoomAlive toolkit, a
software library that enables spatial augmented reality (SAR) applications in
projector-camera systems by providing the necessary algorithms to correctly
render virtual environments onto arbitrarily shaped surfaces. The toolkit is
used in work presented here in order to experiment with and implement various
input methods in an SAR environment. [10] establishes a mobile multi-display
environment based on commodity smartphones by tracking the user’s head and
face and using it as a reference, thus showing mobile possibilities of multi-
display environments. The work presented in this thesis uses optical marker-
based tracking in order to track various objects. Previous work such as [22]
suggest gluing a marker onto a subject’s finger nails, joints and other hand
positions in order to reconstruct a model of the user’s hand skeleton via
inverse kinematics. [6] suggests attaching markers to commodity gloves that
are then worn by users for hand and finger tracking. However, it is unclear
whether gloves are possibly a confounding factor in user performance. [30],
[18] and [15] suggest that gloves have no effect on performance in the
particular input methods tested in the respective papers, however, research in
other scientific fields such as [31], [5] and [2] suggest influence of gloves
on various hand-related task. To remove these uncertain variables, the work
presented here aims at a glove-less tracking solution.

##  2 Concept

Figure 1: Two different C-D ratio cases of input.

The driving idea behind the work presented here is the investigation of
various C-D ratio mappings in order to improve interaction over large
distances using small input movements in mixed reality or VR. One critical
component is the interaction with relatively small input devices (compared to
the display space), as these imply a small motor space and small movements to
traverse this space. Compare the two cases shown in figure 1, where the
available motor space when using the smartphone totals less than 15cm on each
coordinate axis, which can be utilized completely by finger, wrist or hand
movements. In the case of the PC touch display, the user has to move the
entire arm to reach corners of the display. On larger displays or VR
applications that are not bound to physical screen size restrictions, the user
might have to move with their feet to reach distant objects. This thesis
explores the possibilities of controlling these large output spaces with small
input movements, and to determine whether these smaller movements in motor
space for small displays translates to faster or more accurate control than
large movements.

Touch-based input devices like smartphones have the limitation of a two-
dimensional input space. Fingers can use this space, but can also reach
several centimeters into the air. We seek to exploit this opportunity of
expanding the motor space into the third dimension using optical tracking-
based finger movement.

##  3 Input methods

Four modes of input are presented in the following. (1.) Basic large scale
touch-screen input with C-D ratio of exactly one. Here, a PC monitor with
touch input serves as a baseline test for situations where the user can
directly select targets on the screen. (2.) Smartphone touch-screen input,
used to explore different C-D ratio mappings with an input method that average
users are likely to be familiar with. (3.) Z-Mapping: Finger-movement optical
tracking-based input. Here, the user can move the environment around by moving
the hand and finger. (4.) Z-Scaling: Environment zooming and navigation based
on finger-movement. The user can zoom in and out of the environment by moving
their finger up or down, scaling the distance needed to reach a target instead
of modifying the C-D ratio. The following sections introduce more precise
mathematical definitions of these input methods.

###  3.1 Method 1: 1:1 mapped direct input

Inputs such as physical touch locations can be directly mapped onto the
virtual display position below the user’s finger in a one to one fashion. It
is defined as

$f(\vec{p})=\vec{p},f:\mathbb{R}^{2}\to\mathbb{R}^{2}$ (3)

where $\vec{p}$ is a two-dimensional input position. It serves as a baseline
because it is functionally equivalent to having no C-D ratio mapping at all.

###  3.2 Method 2: 1:N mapping

Method 1 can be extended such that the input position $p_{t}$ is mapped to
output depending on a custom C-D ratio mapping function $H$. $\vec{p}$ can be
scaled by a factor $N$ by introducing

$H(N)=\frac{1}{N},H:\mathbb{R}\to\mathbb{R}$ (4)

and modifying $f$ in the following manner:

$f(\vec{p})=\vec{p}\cdot H(N),f:\mathbb{R}^{2}\to\mathbb{R}^{2}$ (5)

###  3.3 Method 3: Z-Mapping

Figure 2:  Fundamental principle of Z-Mapping. Left: A small input movement
$\Delta p$ a small distance above the surface results in a nearly identical
output movement $\Delta f$ on the target display. Right: The same input
movement, but further up above the display results in a larger output movement

Z-Mapping extends the mappings above by extending the input space to three
dimensions and using the extra coordinate to adjust the C-D ratio dynamically.
The function $H$ then operates on the new coordinate as input. The idea is to
interpret the new dimension as input height, and to have users be able to
smoothly modify the speed at which they control the environment by moving
their fingers in arcs of different heights, higher arcs representing larger
”jumps” in display space. Figure 2 shows two examples. The final target
selection still happens with touch input (typically on a smartphone). Compared
to previous methods, $f$ is extended with one parameter:

$f(z,\Delta\vec{p})=\vec{f}_{t-1}+\Delta\vec{p}_{xy}\cdot
H(p_{z}),f:\mathbb{R}\times\mathbb{R}^{3}\to\mathbb{R}^{2}$ (6)

where $\Delta\vec{p}$ is the change in input positions since the last frame,
with $\Delta\vec{p}_{xy}$ being the change in the x-y plane, $\vec{f}_{t-1}$
is the mapped position of $f$ from the previous frame, and $H$ is the new C-D
ratio mapping function that linearly modifies the ’C-D ratio based on
movements in the new third dimension $z$ of $\vec{p}$:

$H(z)=(1+\frac{z\cdot N}{M}),H:\mathbb{R}\to\mathbb{R}$ (7)

$M$ and $N$ are constants. The two functions are now described in detail:
Since the resulting mapped position is not supposed to change when the input
position is moved up or down, and is only supposed to change when the position
is changed in the x and y directions, $f$ formulates the output position
relative to the previously mapped position $\vec{f}_{t-1}$, meaning that if
$\Delta\vec{p}_{xy}=\vec{0}$ the result of the right term is $\vec{0}$ and the
output position does not change. $H$ then only modifies the C-D ratio of input
in the x-y directions. The goal of $H$ is to modify the C-D ratio by a factor
of $M$ for every $N$ units which the user moves in the $z$ direction.

As an example from the context of a practical application, when a large one
dimensional display space of size $s_{l}$ is controlled with an input space
that is limited to a smaller size $s_{s}$, $M$ and $N$ might be chosen such
that the maximum height a user’s finger can reach above the smaller device
changes the input ratio such that one movement across the entire range $s_{s}$
maps onto a movement across the entire range of $s_{l}$. In this case, N and M
would be

$N=h_{max}-h_{min}$ $M=\frac{s_{l}}{s_{s}}$

where $h_{max},h_{min}$ are the maximum and minimum finger input height of the
user, respectively, and the resulting function $H$ is

$H(z)=(1+s_{s}\cdot
h_{max}-h_{min})\cdot\frac{z}{s_{l}},H:\mathbb{R}\to\mathbb{R}$ (8)

This precise case is used in the user study experiment design for the
evaluation of Z-Mapping.

###  3.4 Method 4: Z-Scaling

Figure 3:  Visualization of the scaling concept. Left: With the finger point
$\vec{p}$ at a low input height, the environment is visible near or at its
original scale on the output display. Right: At a higher input height, the
environment is scaled down, until it is eventually visible entirely on the
smaller display.

The last examined input method does not change the C-D ratio directly, and
instead relies on scaling the environment (zooming in and out) to reduce the
amount of motor space that has to be traversed for some output movement.
Figure 3 shows a graphical representation of the finger scaling concept. The
user first scales the environment down until a desired target is visible on
the touch input screen, then traverses the necessary distance to the target in
motor space, with the environment still scaled down, and then scales the
environment back up, with the scaling happening relative to the target. After
this, the target can be selected via touch input.

In this method, movements in motor space are mapped directly to movements in
display space (similar to method 3.1), with the driving idea being that
reducing the amount of motor space that has to be covered to reach a certain
target is functionally similar to increasing the C-D ratio such that the same
motor space is covered but the output distance is increased instead, resulting
in reduced movement times to reach targets.

The scaling used here is a simple uniform scale transformation, i.e.

$S=\begin{bmatrix}s&0&0\\\ 0&s&0\\\ 0&0&s\end{bmatrix}$

where $s$ is the scaling factor, with $s<1$ effectively ”zooming out” and
$s>1$ ”zooming in”. After applying this transformation, any point $\vec{p}$ in
the environment is shifted to the new location

$\vec{p_{s}}=S\cdot(\vec{p}-\vec{c})+\vec{c}=s\cdot(\vec{p}-\vec{c})+\vec{c}$
(9)

where $\vec{c}$ is the center position of the environment.

In order to specify a target location, the foot $\vec{p_{f}}$ of the
perpendicular from the users finger position $\vec{p_{i}}$ to the touch
screens plane (typically from a smartphone), which is centered at
$\vec{p_{s}}$ with plane normal vector $\vec{n}$, is used:

$\vec{p_{f}}=\vec{p_{i}}-((\vec{p_{i}}-\vec{p_{s}})\bullet\vec{n})\cdot\vec{n}$
(10)

The equation above computes the foot by subtracting the part of the input
user’s position vector that is perpendicular to the touch screen display’s
plane normal vector. From the user’s perspective, the target is on the
smartphone screen location ”below the finger”. The scaling is then applied
such that the location at the foot of this perpendicular remains the same
after scaling, i.e. the user’s target remains below the finger. This is
achieved by translating the entire environment in the opposite direction
corresponding to how much the target location moved after scaling, i.e. the
environment’s center $\vec{c}$ is moved to its post-scale location
$\vec{c_{s}}$ via

$\vec{c_{s}}=\vec{c}-(\vec{p_{fs}}-\vec{p_{f}})$ (11)

where $\vec{p_{f}}$ is the perpendicular’s foot location before scaling, and
$\vec{p_{fs}}$ is its location after scaling which can be determined using
equations 9 and 10 above. In the default implementation of z-scaling the above
is applied to both upwards and downwards motions of the user’s finger, i.e.
the target location is kept under the finger when zooming in as well as when
zooming out, which allows the user to finely adjust their target positioning
by moving their fingers up and down in small intervals, for example to improve
landing accuracy on the target.

##  4 Practical Applications

Figure 4: SAR environment displaying a basic prototyping environment.  Figure
5: Running example of the multi-display environment of the user study.

The described input methods are implemented in two practical applications, an
SAR environment seen in figure 4, which was used for prototyping and
development, and a separate setup used in the user study seen in figure 5,
where a PC touch monitor is used for input and as the larger output space
instead of a projector. Both setups share other hardware and configuration
aspects, such as a central smartphone that displays a cutout of the larger
environment. Targets are shown and selected via touch input.

##  5 Implementation

The following sections will details about the various components involved in
the implementation of the input methods described previously.

###  5.1 Hardware

Deployed hardware components consist of the following parts:

  * •

Microsoft Kinect (v2) which is mainly used during calibration by the RoomAlive
toolkit to recreate a virtual model of the real-life working space, most
prominently featuring the desk upon which the experiments are conducted. The
Kinect is only used for 3D scanning, and not used during the actual runtime of
the Unity applications, as optical tracking is used to determine locations of
real world objects such as fingers and smartphones.

  * •

Optoma GT 1070 projector which is used both in the calibration process of the
RoomAlive toolkit, as well to project the final running application
environment onto the real world workspace.

  * •

Eight OptiTrack Prime 13 camera sensor units used for optical marker tracking.

  * •

22 inch Dell S2340T PC touch screen, required for the user study.

  * •

Workstation consisting of a wooden desk that is used as a display surface for
the projector used in the user study and input method prototyping, and a
consumer PC used for developing input methods as well as running the actual
user study Unity applications.

  * •

PC hardware which consists of an Intel Xeon E5-2620 CPU with 6 cores of 2.00
GHz base frequency each, an NVIDIA GTX 1070 graphics card, and 32GB of DDR3
RAM.

###  5.2 Optitrack object and finger tracking

Optitrack and its control software allows the definition of so-called rigid
bodies, a user-defined collection of markers in the real world whose position
and orientation relative to each other remains fixed (i.e. they cannot be
deformed). In addition to allowing the tracking of rotations which is not
possible with a single marker due to lack of available degrees of freedom,
tracking a single object with multiple markers also increases reliability of
tracking as a rigid body’s location can be tracked even when one or multiple
of its markers are not visible to the cameras, although a minimum count of 3
markers is generally still required. In the following, the exact marker setup
for the tracked smartphone as well as for finger tracking are described.

####  5.2.1 Smartphone Rigid Body

Figure 6: Two rigid body configurations side-by-side.

A rigid body is defined in Optitrack to allow tracking of the smartphone. Two
possible solutions are shown in figure 6 side by side, with the setup on the
right being used in the user study. The setups are tradeoffs: The left
configuration can cause problems when a user’s hand occludes markers partially
or completely, and the setup on the right is mechanically more bulky, which is
undesirable for mobile setups (which is not a problem in the user study where
all hardware components are static).

####  5.2.2 Finger tracking

Figure 7: Marker attached to finger via double sided bonding tape.

One finger of the user is tracked so it can be used with input methods from
sections 3.3 and 3.4. A single spherical marker is used (see figure 7) and
attached to the finger via double sided bonding tape. The set of markers read
from the Optitrack system are filtered according to application-defined logic
to identify the relevant finger marker when other markers are visible.

###  5.3 Transformation acquisition

Figure 8:  Special purpose rigid body that is used to represent a coordinate
system transform from the Optitrack coordinate system to the RoomAlive
Toolkit’s and Unity’s coordinate system.

In order to figure out the transformation between the Optitrack and Unity
coordinate system, a custom calibrator rigid body (see 8) is aligned with the
Kinects position in the real world (which serves as Unity’s coordinate origin)
and its orientation$Q_{T}$ and position $\vec{x}_{T}$ are used as the rotation
and translation part of the coordinate transform.

####  5.3.1 Transformation usage

After acquiring rotation quaternion $Q_{T}$ and translation vector
$\vec{x}_{T}$, they are applied by

$Q^{\prime}=Q_{T}^{-1}\cdot Q$

where$Q$ and $Q^{\prime}$ are an objects rotation before and after the
transformation, respectively. The final position $\vec{x}^{\prime}$ with
original position $\vec{x}$ is given by

$\vec{x}^{\prime}=Q_{T}^{-1}\cdot(\vec{x}-\vec{x}_{T})+\vec{x}_{T}$

##  6 Evaluation

Several C-D ratio mappings presented previously were subjected to a user study
in order to compare their performance. Three candidates (direct 1:1 mapping
via smartphone touch dragging, Z-Mapping and Z-Scaling) were initially
selected based on their performance during their development phase. In
addition, the 1:1 mapped direct PC touch monitor input method (see figure 1)
was added as a baseline. Z-Mapping and Z-Scaling were then compared in the
user study environment as part of an informal pilot study with two expert
users. As a result of that pilot study, Z-Mapping was chosen as the novel
input method to be tested with more subjects, due to higher performance of
both users in the pilot study. The following section describes the conducted
experiment, presents results and then offers interpretation of these results.

###  6.1 Experiment description

As an evaluation environment, the setup described in section 4 was used.
Subjects were asked to acquire targets using three different input methods:

  * •

Direct (1:1 C-D ratio mapped) PC-display touch input, where users could
physically select targets on the screen.

  * •

1:1 C-D ratio mapped smartphone touch-input, where users had to move targets
into the view of the smartphone screen by dragging environment towards it
using touch input.

  * •

Z-Mapping as described in section 3.3, where users had to move targets into
view of the smartphone screen by dragging the environment towards it using
finger movements in the air. The minimum and maximum height as described in
section 3.3 were acquired using a simple height calibration where users placed
their fingers at the lowest and highest possible position they could reach
with their fingers with a hand placed down on the large PC touch screen, which
were then recorded.

###  6.2 Procedure

After initial introductions and signing of a participant agreement waiver and
release form, subjects filled out a demographics questionnaire. All subjects
had a marker attached to their index finger that was used for optical tracking
as described in section 5.2.2. After explaining the basic experiment setup and
rules, for each input method, subjects then executed a target acquisition task
(described below), where circular targets had to be selected on one or two of
the used displays. Before each input method, subjects were provided with a
short demonstration of the respective input technique’s concept. Subjects then
executed an initial training sequence in order to familiarize themselves with
the input techniques, followed by a main run as described below in section
6.2.2. Subjects executed the task in a seated position in front of the PC
touch screen. After each input method, participants were asked to fill out an
after-scenario-questionnaire [20] and the NASA TLX questionnaire [12], and a
preference questionnaire after all input method tasks were completed. Finally,
an informal interview was held in order to allow each subject to expand on
their opinion of conducted tasks in their own words.

####  6.2.1 Target acquisition task

Circular targets are displayed on the two used displays (PC touch screen, and
smartphone screen). One display was used for the PC-display touch input case,
and two displays for both other input methods, where the smartphone is placed
centrally on the larger screen and displays a cutout of its environment.
Initially, both a red and blue target are displayed. Upon clicking on the red
target, the blue target is activated for selection and turned green, after
which users can navigate to it and acquire it using direct touch input (for
all methods). In the smartphone touch input case and Z-Mapping, users first
have to relocate the target such that it is visible on the smartphone display,
as PC display touch-screen input was disabled for those input methods. After
acquiring a green target, the environment is reset to its starting location
and the red target, in addition to the next blue target, is displayed again.
The purpose of this starting position is for the user to be able to take time
and visually recognize the position of the new target, as movement times and
accuracy are not tracked during this period. This process repeats until all
required samples have been gathered. Subjects were specifically instructed to
first locate the new (blue) target, and only after visually acquiring it,
select the red target and then select the green target. This is to prevent
targer search times from entering into the recorded movement times.

####  6.2.2 Design

The experiment used a repeated measures within-subject design. The independent
variables were input method type(direct-touch-PC, direct-touch-smartphone,
z-mapping), the direction in which targets appear (8 directions in total, each
45 degrees apart in a circle around the large display center), as well as the
index of difficulty of those targets resulting from their sizes and distances
to the smartphone (average of 15 targets for each direction). 120 targets had
to be selected per input method with 360 main samples gathered over 3 input
methods in total for each subject, in addition to 180 training samples for
each subject over all three input methods. For the index of difficulty, values
of 2-5 were targeted, but with a minimum target width of 1cm, the highest
possible $ID$ values reachable were 4.85 (in the corners of the PC touch
screen display) and 4.67 (near the edge of the PC touch screen on the right
and left sides in the middle), which were used to represent the $ID$ category
5. Participants performed the procedure described above in one session lasting
roughly 50 minutes. Each of the three input methods was performed one after
the other, with counterbalanced order permutations. The main sequence of 120
samples per input method was executed in 4 blocks of 30 targets each. Each
subject selected the same targets, but the order of targets was randomized
per-subject using selection without replacement. After each block in both the
training and main sequences, the subjects were offered an optional short break
in case of fatigue.

####  6.2.3 Participants

A total of 20 (4 female, 16 male) subjects participated in the study, in the
age range of 21-34. 14 subjects worked in the field of or studied computer
science, 2 subjects were electrical engineers, 3 subjects worked or studied in
health sciences and 1 subject was a product designer. 9 subjects were visually
impaired, with all of them using corrective glasses or contact lenses. 18
subjects were right-handed, and 2 subjects left handed. All 20 subjects
specified that they use smartphones on a daily basis. Subjects were
volunteers, but with the exception of one were awarded with a gift certificate
worth 10 euros (the subjects were told of this when signing up for the test).

####  6.2.4 Apparatus

The study was executed on a PC running the target acquisition test software
with an Intel Xeon E5-2620 CPU with 6 cores of 2.00 GHz base frequency each,
an NVIDIA GTX 1070 graphics card, and 32GB of DDR3 RAM. An Amazon Fire
smartphone was used as the smartphone device, and a 2 inch Dell S2340T PC
touch screen display served as the larger display and for the PC touch screen
input method. The Optitrack M13 based optical tracking environment is
described in section 5.1.

###  6.3 Results

The following section presents results of the experiment. For the indices of
difficulty range of 2 through 5, four $ID$ categories are defined. They are:
Category 2 with $1.5<ID\leq 2.5$, category 3 with $2.5<ID\leq 3.5$, category 4
with $3.5<ID\leq 4.5$ and category 5 with $4.5<ID\leq 5.5$. Data was also
examined by each executed 30 sample block. User preferences as determined by
the preference questionnaire are also presented. The ranks given by subjects
in their answers are converted to ordinal samples using their ranking number
(from 1 = best, to 3 = worst). These results are then tested for statistical
significance.

####  6.3.1 Statistical significance testing

Samples from each grouping (overall, by ID, by block) were first averaged per
user and then subjected to significance testing, resulting in sample count
$n=20$ for each input method. Normality was assumed and tested using the
Shapiro-Wilkes normality test [26] with $\alpha=0.05$. ANOVA was used to
determine statistical significance for normally distributed samples, and
Friedman test [9] was used for non-normally distributed samples, both with
$\alpha=0.05$. When passed, the alpha level was adjusted for multiple
comparisons using conservative Bonferroni correction [7], with resulting
$\alpha_{b}=0.017$. Finally, post-hoc tests for normal data used paired
Student’s t-tests [28], and Wilcoxon signed-rank tests [32] for non normal
data, both with the modified alpha level resulting from Bonferroni correction.

The tested hypothesis was that there is a relationship (in movement time or
accuracy, respectively) between on the input method used to perform the task
and the resulting metric being measured. In the case of the preference
questionnaire, the hypothesis tested for is that users have differing
preferences between input methods with respect to certain criteria (perceived
speed and accuracy, as well as overall preference).

####  6.3.2 Data Collection and Analysis

Two primary types of data are collected and used to judge subject performance:
Movement time and accuracy. Movement time is recorded from touch-up of the red
target to touch-up of the green target.The accuracy of task execution $a$ is
defined by the number of a subject’s average hits $n_{h}$ and misses $n_{m}$:

$a=\frac{n_{h}}{n_{h}+n_{m}}$

####  6.3.3 Accuracy and movement times

Figures 9 through 14 show results of average means and standard deviations for
movement times and accuracy. Tables 7 through 12 show results of significance
tests for both metrics. Input methods are abbreviated: ZM = Z-Mapping, PT = PC
touch input, ST = smartphone touch input.

Figure 9: Box plot showing summarized movement times of different input methods over all blocks and indices of difficulty. Figure 10: Box plots showing movement times of different input methods for separated blocks 1 (top left) through 4 (bottom right), proceeding left to right, top to bottom. Figure 11: Box plots showing movement times of different input methods for separated ID categories 1 (top left) through 4 (bottom right), proceeding left to right, top to bottom. Figure 12: Box plot showing summarized accuracies of different input methods over all blocks and indices of difficulty. Figure 13: Box plots showing accuracy of different input methods for separated blocks 1 (top left) through 4 (bottom right), proceeding left to right, top to bottom. Figure 14: Box plots showing accuracy of different input methods for separated ID categories 1 (top left) through 4 (bottom right), proceeding left to right, top to bottom. | ZM | ST | PC
---|---|---|---
Overall | $\mu_{zm}=1.64s,\sigma_{zm}=0.20$ | $\mu_{st}=1.76s,\sigma_{st}=0.15$ | $\mu_{pt}=0.62s,\sigma_{pt}=0.09$
Table 1: Means and standard deviations for movement times for overall data.  | ZM | ST | PC
---|---|---|---
Block 1 | $\mu=1.74,\sigma=0.24$ | $\mu=1.76,\sigma=0.11$ | $\mu=0.62,\sigma=0.10$
Block 2 | $\mu=1.62,\sigma=0.21$ | $\mu=1.74,\sigma=0.19$ | $\mu=0.61,\sigma=0.09$
Block 3 | $\mu=1.60,\sigma=0.24$ | $\mu=1.75,\sigma=0.18$ | $\mu=0.61,\sigma=0.12$
Block 4 | $\mu=1.60,\sigma=0.30$ | $\mu=1.77,\sigma=0.18$ | $\mu=0.62,\sigma=0.11$
Table 2: Means and standard deviations for movement times by block.  | ZM | ST | PC
---|---|---|---
ID category 2 | $\mu=1.07,\sigma=0.12$ | $\mu=1.37,\sigma=0.13$ | $\mu=0.43,\sigma=0.07$
ID category 3 | $\mu=1.46,\sigma=0.19$ | $\mu=1.61,\sigma=0.14$ | $\mu=0.52,\sigma=0.08$
ID category 4 | $\mu=1.82,\sigma=0.25$ | $\mu=1.88,\sigma=0.18$ | $\mu=0.66,\sigma=0.11$
ID category 5 | $\mu=1.24,\sigma=0.39$ | $\mu=2.31,\sigma=0.22$ | $\mu=0.92,\sigma=0.16$
Table 3: Means and standard deviations for movement times by ID categories.  | ZM | ST | PC
---|---|---|---
Overall | $\mu_{zm}=0.77,\sigma_{zm}=0.10$ | $\mu_{st}=0.98,\sigma_{st}=0.02$ | $\mu_{pt}=0.92,\sigma_{pt}=0.04$
Table 4: Means and standard deviations for accuracy for overall data.  | ZM | ST | PC
---|---|---|---
Block 1 | $\mu=0.76,\sigma=0.10$ | $\mu=0.98,\sigma=0.03$ | $\mu=0.92,\sigma=0.05$
Block 2 | $\mu=0.77,\sigma=0.11$ | $\mu=0.97,\sigma=0.04$ | $\mu=0.94,\sigma=0.05$
Block 3 | $\mu=0.79,\sigma=0.13$ | $\mu=0.98,\sigma=0.02$ | $\mu=0.93,\sigma=0.06$
Block 4 | $\mu=0.79,\sigma=0.13$ | $\mu=0.98,\sigma=0.03$ | $\mu=0.92,\sigma=0.05$
Table 5: Means and standard deviations for accuracy by block.  | ZM | ST | PC
---|---|---|---
ID category 2 | $\mu=0.89,\sigma=0.09$ | $\mu=0.99,\sigma=0.03$ | $\mu=0.99,\sigma=0.01$
ID category 3 | $\mu=0.79,\sigma=0.10$ | $\mu=0.99,\sigma=0.02$ | $\mu=0.98,\sigma=0.03$
ID category 4 | $\mu=0.74,\sigma=0.14$ | $\mu=0.97,\sigma=0.03$ | $\mu=0.92,\sigma=0.05$
ID category 5 | $\mu=0.70,\sigma=0.13$ | $\mu=0.97,\sigma=0.04$ | $\mu=0.80,\sigma=0.01$
Table 6: Means and standard deviations for accuracy by ID categories.  | Overall | Block 1 | Block 2 | Block 3 | Block 4 | ID 2 | ID 3 | ID 4 | ID 5
---|---|---|---|---|---|---|---|---|---
ZM | 0.9917 | 0.40 | 0.91 | 0.84 | 0.14 | 0.09 | 0.28 | 0.08 | 0.51
ST | 0.06053 | 0.63 | 0.45 | 0.38 | 0.14 | 0.28 | 0.18 | 0.43 | 0.78
PT | 0.3105 | 0.36 | 0.16 | 0.15 | 0.18 | 0.18 | 0.09 | 0.61 | 0.42
Table 7: p-values for Shapiro-Wilk tests of each input method and sample group in order to test for normality.  | ANOVA p-value | Passed?
---|---|---
Overall | $<2.2\cdot 10^{-16}$ |
Block 1 | $<2.2\cdot 10^{-16}$ |
Block 2 | $4.19\cdot 10^{-14}$ |
Block 3 | $2.58\cdot 10^{-13}$ |
Block 4 | $1.36\cdot 10^{-12}$ |
ID category 2 | $<2.2\cdot 10^{-16}$ |
ID category 3 | $<2.2\cdot 10^{-16}$ |
ID category 4 | $1.48\cdot 10^{-15}$ |
ID category 5 | $9.25\cdot 10^{-11}$ |
Table 8: ANOVA test results for movement times. All groupings pass the test. Group | Method 1 | Method 2 | t-test p-value | Passed?
---|---|---|---|---
Overall | ZM | ST | $0.05$ | x
ZM | PT | $3.78\cdot 10^{-14}$ |
PT | ST | $<2.20\cdot 10^{-16}$ |
Block 1 | ZM | ST | $0.69$ | x
ZM | PT | $3.26\cdot 10^{-13}$ |
PT | ST | $<2.20\cdot 10^{-16}$ |
Block 2 | ZM | ST | $0.03$ | x
ZM | PT | $1.50\cdot 10^{-13}$ |
PT | ST | $2.61\cdot 10^{-16}$ |
Block 3 | ZM | ST | $0.05$ | x
ZM | PT | $2.60\cdot 10^{-12}$ |
PT | ST | $6.00\cdot 10^{-16}$ |
Block 4 | ZM | ST | $0.03$ | x
ZM | PT | $1.07\cdot 10^{-11}$ |
PT | ST | $<2.20\cdot 10^{-16}$ |
ID Category 2 | ZM | ST | $3.14\cdot 10^{-8}$ |
ZM | PT | $2.13\cdot 10^{-14}$ |
PT | ST | $<2.20\cdot 10^{-16}$ |
ID Category 3 | ZM | ST | $0.01$ |
ZM | PT | $3.283\cdot 10^{-14}$ |
PT | ST | $<2.20\cdot 10^{-16}$ |
ID Category 4 | ZM | ST | $0.46$ | x
ZM | PT | $1.296\cdot 10^{-13}$ |
PT | ST | $<2.20\cdot 10^{-16}$ |
ID Category 5 | ZM | ST | $0.37$ | x
ZM | PT | $8.943\cdot 10^{-12}$ |
PT | ST | $1.741\cdot 10^{-15}$ |
Table 9: Results of paired t-tests between all combinations of input methods for movement times.  | ZM | ST | PT
---|---|---|---
Overall | $0.09$ | $8.83\cdot 10^{-7}$ | $0.22$
Block 1 | $0.95$ | $4.04\cdot 10^{-5}$ | $0.01$
Block 2 | $0.02$ | $5.18\cdot 10^{-7}$ | $4,00*\cdot 10^{-3}$
Block 3 | $0.09$ | $7.57\cdot 10^{-5}$ | $0.05$
Block 4 | $0.76$ | $1.18\cdot 10^{-5}$ | $0.14$
ID Category 2 | $0.01$ | $7.91\cdot 10^{-7}$ | $2.89\cdot 10^{-7}$
ID Category 3 | $0.29$ | $6.55\cdot 10^{-6}$ | $8.91\cdot 10^{-5}$
ID Category 4 | $0.07$ | $0.001$ | $0.16$
ID Category 5 | $0.45$ | $3.76\cdot 10^{-6}$ | $0.32$
Table 10: p-values for Shapiro-Wilk tests of each input method.  | Friedman p-value | Passed?
---|---|---
Overall | $5.33\cdot 10^{-9}$ |
Block 1 | $2.40\cdot 10^{-8}$ |
Block 2 | $4.03\cdot 10^{-8}$ |
Block 3 | $2.2\cdot 10^{-6}$ |
Block 4 | $8.32\cdot 10^{-7}$ |
ID category 2 | $2.78\cdot 10^{-7}$ |
ID category 3 | $2.04\cdot 10^{-8}$ |
ID category 4 | $1.20\cdot 10^{-7}$ |
ID category 5 | $2.16\cdot 10^{-7}$ |
Table 11: Friedman test results for accuracy. All groupings pass the test.  Group | Method 1 | Method 2 | Wilcoxon p-value | Passed?
---|---|---|---|---
Overall | ZM | ST | $9.56\cdot 10^{-5}$ |
ZM | PT | $0.11\cdot 10^{-4}$ |
PT | ST | $9.56\cdot 10^{-5}$ |
Block 1 | ZM | ST | $9.52\cdot 10^{-5}$ |
ZM | PT | $0.14\cdot 10^{-4}$ |
PT | ST | $0.38\cdot 10^{-4}$ |
Block 2 | ZM | ST | $9.53\cdot 10^{-5}$ |
ZM | PT | $0.20\cdot 10^{-4}$ |
PT | ST | $0.23\cdot 10^{-3}$ |
Block 3 | ZM | ST | $0.11\cdot 10^{-4}$ |
ZM | PT | $0.20\cdot 10^{-4}$ |
PT | ST | $0.26\cdot 10^{-3}$ |
Block 4 | ZM | ST | $0.21\cdot 10^{-4}$ |
ZM | PT | $0.27\cdot 10^{-3}$ |
PT | ST | $0.52\cdot 10^{-4}$ |
ID Category 2 | ZM | ST | $0.19\cdot 10^{-4}$ |
ZM | PT | $0.14\cdot 10^{-4}$ |
PT | ST | $0.31$ | x
ID Category 3 | ZM | ST | $9.42\cdot 10^{-5}$ |
ZM | PT | $9.50\cdot 10^{-5}$ |
PT | ST | $0.42\cdot 10^{-16}$ |
ID Category 4 | ZM | ST | $9.54\cdot 10^{-5}$ |
ZM | PT | $0.11\cdot 10^{-4}$ |
PT | ST | $0.18\cdot 10^{-3}$ |
ID Category 5 | ZM | ST | $9.53\cdot 10^{-5}$ |
ZM | PT | $0.2$ | x
PT | ST | $0.14\cdot 10^{-4}$ |
Table 12: Results of Wilcoxon signed-rank tests between all combinations of
input methods.

####  6.3.4 User preference

Table 13 shows answer results and significance tests for answers in the
preference questionaire. During statistical significance testing, results for
the overall question did not pass the initial Friedman test and thus no post-
hoc tests were conducted. Speed and accuracy passed the Friedman test with
$p=0.12\cdot 10^{-4}$ and $p=4.48\cdot 10^{-6}$, respectively. Post-hoc showed
no significant differences when comparing ZM vs. ST in terms of speed and PT
vs. ST in terms of accuracy ($p=0.22$ and $p=1.64\cdot 10^{-2}$), but showed
significance for ZM vs ST and PT vs. ST in terms of speed ( $p=0.20\cdot
10^{-3}$ and $p=0.44\cdot 10^{-4}$), as well as ZM vs. ST and ZM vs. PT in
terms of accuracy ( $0.17\cdot 10^{-3}$ and $8.09\cdot 10^{-5}$ ).

| ZM | ST | PC
---|---|---|---
Speed | $\mu=2.2,\sigma=0.61$ | $\mu=2.55,\sigma=0.69$ | $\mu=1.25,\sigma=0.55$
Accuracy | $\mu=2.8,\sigma=0.41$ | $\mu=1.90,\sigma=0.64$ | $\mu=1.25,\sigma=0.44$
Overall preference | $\mu=2.1,\sigma=0.91$ | $\mu=2.20,\sigma=0.52$ | $\mu=1.70,\sigma=0.92$
Table 13: Means and standard deviations for answers of different questions in
the preference questionnaire.

##  7 Discussion

In summary, compared to the smartphone touch input method, Z-Mapping shows
better movement times throughout all data groups (overall, by block, by ID),
but also worse accuracy throughout all groups. Further, statistical
significance could only be shown for accuracy. Direct PC touch input method
performs significantly better in terms of movement times than both other input
methods. These results align with participant’s answers in the preference
questionnaire (table 13), where users on average reported a slight perceived
increase in speed for Z-Mapping over the smartphone touch input case, but
almost universally lower perceived accuracy. Further, there is no
statistically significant overall preference between input methods in
subject’s answers. Z-Mapping’s failure to show significance for movement times
could be because the number of subjects in the study could simply have been
too low, or not enough training blocks or non-optimal instructions were
provided. However, movement times grouped by ID show another indication. The
gap between Z-Mapping and the smartphone touch method increases from a
relative speed up of $28\%$ at ID category 2 to a $86\%$ at ID category 5.
While further investigations are necessary to confirm it, it may be possible
that the chosen output space (PC touch display) is too small relative to the
input space to fully show the advantages in movement times of Z-Mapping.
Further, data indicates that Z-Mapping suffered disproportionately from below
average samples, compared to other input methods. During the experiment, users
would often have to make corrective movements after missing a target,
resulting in increased movement times. In interviews, participants also often
stated that Z-Mapping felt ”shaky”, which resulted in problems and frustration
with target selection (confirmed by accuracy results). This hints at a need to
improve the final target selection step of Z-Mapping. Overall, the method
needs further work.

##  8 Future Work

Z-Mapping, but also Z-Scaling, have not been evaluated for larger scale output
spaces in this thesis. The chosen PC touch display is still small enough that
users can reach targets by physically touching their on-screen locations. Of
interest then are environments where users are unable to physically reach
targets, or where doing such would imply very large movement times (e.g.
walking to the other end of the room). One such example are VR spaces in
general, where users may locally only be able to physically reach a very small
fraction of the overall output space. Investigations towards that end could
also confirm or deny whether the general trend of Z-Mapping performing better
at higher IDs continues at higher distances.

One major problem of Z-Mapping as determined during the user study is the
final phase of target acquisition when users try to click on a target. As the
environment keeps moving on downwards finger movements, and users don’t
typically move their fingers in perfect lines that are orthogonal towards the
target, user’s experience a slight shaking of the environment that results in
missed clicks. It may be possible to work with the user’s tracked finger
trajectory to determine the point at which they begin trying to click targets,
and from that point switch to a different input mapping scheme that is
optimized for hitting the target below the finger. Another possible point of
investigation would be techniques that detach the moving of the environment
from the final target selection. In Z-mapping during the evaluated user study,
the environment always moved whenever the user’s fingers moved. Instead, it
might be possible to deactivate this movement, e.g. via secondary controls
activated by a user’s unused hand. The same concepts could be used give the
user a way to quickly adjust their positioning when they miss a target on
their first try, which in the user study required them to touch the smartphone
screen to disable environment movement and move back towards the middle in
order to try again.

##  9 Summary

This thesis presented C-D ratio mappings for commonplace as well as novel
input techniques and implemented the presented concepts in two distinct multi-
display environments. Z-Mapping in particular was then evaluated in a user
study by comparing it against touch based input techniques as they are
commonly used in smartphone applications. As expected, it performs worse
overall compared to the direct PC-touch screen input method that was used as a
baseline comparison. While in the collected data Z-Mapping was shown to
perform better in average movement times compared to the smartphone touch
input method, the difference was not found to be statistically significant. In
addition, the study showed problems with accuracy in target selection. In
informal interviews with the study subjects, this has been determined to
likely be because of target acquisition problems due to the high sensitivity
of the input method in general and a lack of possibility for users to
intuitively adjust the final positioning if they miss the target on the first
try. Finally, possible approaches for future work in order to improve various
aspects of the concepts shown in this thesis were presented.

## References

  * [1] F. Argelaguet and C. Andujar.  A survey of 3d object selection techniques for virtual environments.  Computers & Graphics, 37(3):121–136, 2013.
  * [2] R. R. Bishu and G. Klute.  Investigation of the effects of extravehicular activity (eva) gloves on performance.  1993\.
  * [3] R. Blanch, Y. Guiard, and M. Beaudouin-Lafon.  Semantic pointing: Improving target acquisition with control-display ratio adaptation.  In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’04, pages 519–526, New York, NY, USA, 2004. ACM.
  * [4] X. A. Chen, J. Schwarz, C. Harrison, J. Mankoff, and S. E. Hudson.  Air+touch: Interweaving touch & in-air gestures.  In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ’14, pages 519–525, New York, NY, USA, 2014. ACM.
  * [5] I. Dianat, C. M. Haslegrave, and A. W. Stedmon.  Methodology for evaluating gloves in relation to the effects on hand performance capabilities: a literature review.  Ergonomics, 55(11):1429–1451, 2012.
  * [6] K. Dorfmuller-Ulhaas and D. Schmalstieg.  Finger tracking for interaction in augmented environments.  In Augmented Reality, 2001. Proceedings. IEEE and ACM International Symposium on, pages 55–64. IEEE, 2001.
  * [7] O. J. Dunn.  Multiple comparisons among means.  Journal of the American Statistical Association, 56(293):52–64, 1961.
  * [8] P. M. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement.  Journal of experimental psychology, 47(6):381, 1954.
  * [9] M. Friedman.  The use of ranks to avoid the assumption of normality implicit in the analysis of variance.  Journal of the american statistical association, 32(200):675–701, 1937.
  * [10] J. Grubert and M. Kranz.  Headphones: Ad hoc mobile multi-display environments through head tracking.  In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pages 3966–3971. ACM, 2017.
  * [11] J. Han, S. Ahn, and G. Lee.  Transture: Continuing a touch gesture on a small screen into the air.  In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA ’15, pages 1295–1300, New York, NY, USA, 2015. ACM.
  * [12] S. G. Hart and L. E. Staveland.  Development of nasa-tlx (task load index): Results of empirical and theoretical research.  Advances in psychology, 52:139–183, 1988.
  * [13] R. V. Hartley.  Transmission of information.  Bell Labs Technical Journal, 7(3):535–563, 1928.
  * [14] O. Hilliges, S. Izadi, A. D. Wilson, S. Hodges, A. Garcia-Mendoza, and A. Butz.  Interactions in the air: adding further depth to interactive tabletops.  In Proceedings of the 22nd annual ACM symposium on User interface software and technology, pages 139–148. ACM, 2009.
  * [15] H. Hwangbo, S. H. Yoon, B. S. Jin, Y. S. Han, and Y. G. Ji.  A study of pointing performance of elderly users on smartphones.  International Journal of Human-Computer Interaction, 29(9):604–618, 2013.
  * [16] T. Igarashi and K. Hinckley.  Speed-dependent automatic zooming for browsing large documents.  In Proceedings of the 13th annual ACM symposium on User interface software and technology, pages 139–148. ACM, 2000.
  * [17] S. Izadi, S. Hodges, S. Taylor, D. Rosenfeld, N. Villar, A. Butler, and J. Westhues.  Going beyond the display: a surface technology with an electronically switchable diffuser.  In Proceedings of the 21st annual ACM symposium on User interface software and technology, pages 269–278. ACM, 2008.
  * [18] J. P. Jokinen and J. M. Silvennoinen.  Interactive map interface for controlling bridge crane automation: a cognitive-affective approach.  In Proceedings of the 20th International Academic Mindtrek Conference, pages 95–103. ACM, 2016.
  * [19] B. Jones, R. Sodhi, M. Murdock, R. Mehra, H. Benko, A. Wilson, E. Ofek, B. MacIntyre, N. Raghuvanshi, and L. Shapira.  Roomalive: magical experiences enabled by scalable, adaptive projector-camera units.  In Proceedings of the 27th annual ACM symposium on User interface software and technology, pages 637–644. ACM, 2014.
  * [20] J. R. Lewis.  Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the asq.  ACM Sigchi Bulletin, 23(1):78–81, 1991.
  * [21] I. S. Mackenzie.  Fitts’ Law As a Performance Model in Human-computer Interaction.  PhD thesis, Toronto, Ont., Canada, Canada, 1991.  UMI Order No. GAXNN-65985.
  * [22] J. MacRitchie and A. P. McPherson.  Integrating optical finger motion tracking with surface touch events.  Frontiers in psychology, 6, 2015.
  * [23] N. Marquardt, R. Jota, S. Greenberg, and J. A. Jorge.  The continuous interaction space: interaction techniques unifying touch and gesture on and above a digital surface.  In IFIP Conference on Human-Computer Interaction, pages 461–476. Springer, 2011.
  * [24] J. K. Parker, R. L. Mandryk, and K. M. Inkpen.  Integrating point and touch for interaction with digital tabletop displays.  IEEE Computer Graphics and Applications, 26(5):28–35, 2006.
  * [25] K. Ryall, M. R. Morris, K. Everitt, C. Forlines, and C. Shen.  Experiences with and observations of direct-touch tabletops.  In Horizontal Interactive Human-Computer Systems, 2006. TableTop 2006\. First IEEE International Workshop on, pages 8–pp. IEEE, 2006.
  * [26] S. S. Shapiro and M. B. Wilk.  An analysis of variance test for normality (complete samples).  Biometrika, 52(3/4):591–611, 1965.
  * [27] P. Song, W. B. Goh, W. Hutama, C.-W. Fu, and X. Liu.  A handle bar metaphor for virtual object manipulation with mid-air interaction.  In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, pages 1297–1306, New York, NY, USA, 2012. ACM.
  * [28] Student.  The probable error of a mean.  Biometrika, pages 1–25, 1908.
  * [29] S. Subramanian, D. Aliakseyeu, and A. Lucero.  Multi-layer interaction for digital tables.  In Proceedings of the 19th annual ACM symposium on User interface software and technology, pages 269–272. ACM, 2006.
  * [30] X. Sun, T. Plocher, and W. Qu.  An empirical study on the smallest comfortable button/icon size on touch screen.  Usability and internationalization. HCI and Culture, pages 615–621, 2007.
  * [31] W. Tiefenthaler, S. Gimpl, G. Wechselberger, and A. Benzer.  Touch sensitivity with sterile standard surgical gloves and single-use protective gloves.  Anaesthesia, 61(10):959–961, 2006.
  * [32] F. Wilcoxon.  Individual comparisons by ranking methods.  Biometrics bulletin, 1(6):80–83, 1945.


2021 xx March 2022

# A new versatile method for the reconstruction of scintillator-based muon
telescope events

Raphaël Bajou1  Marina Rosas-Carbajal1  and Jacques Marteau2  1Université de
Paris, Institut de Physique du Globe (CNRS UMR 7154), Paris, France
2Université de Lyon UCBL, Institut de Physique des deux infinis (CNRS UMR
5822), Lyon, France

(14 December 2021)

###### Abstract

This paper presents a new method to process the data recorded with muon
telescopes. We have developed this processing method for the plastic
scintillator-based hodoscopes located around the volcano La Soufrière de
Guadeloupe, in the French Lesser Antilles, in order to perform muon
radiographies of the lava dome region, strongly impacted by the volcanic
hydrothermal activity. Our method relies on particle trajectory
reconstruction, performing a fit of the recorded hits in the impacted
scintillator bars using a Random Sample Consensus algorithm. This algorithm is
specifically built to discriminate outlier points, usually due to noise hits,
in the data. Thus, it is expected to significantly improve the signal/noise
separation in muon track hits and to obtain higher quality estimates of the
particles’ incident trajectories in our detectors. The first analysis of the
RANSAC-reconstructed events offers promising results in terms of average
density maps. To illustrate the performances of this algorithm, we provide
angular resolution and reconstruction efficiency estimates using a GEANT4
simulation of a telescope equipped with four detection matrices. In addition,
we also show preliminary results from open-sky data recorded with such
telescope at La Soufrière de Guadeloupe volcano.

###### keywords:

muon radiographytrack reconstruction volcano imagingRANSAC

††journal: JAIS-ID

##  1 Introduction

The intense hydrothermal activity occurring at La Soufrière de Guadeloupe, in
the French Lesser Antilles, is a major concern for volcanologists at the
Volcanological and Seismological Observatory of Guadeloupe (OVSG) and for the
local population that lives nearby. Partly hosted within the andesitic lava
dome, this hydrothermal system, continuously fueled with meteoric waters, is
indeed responsible for both violent phreatic eruptions (last major event
occurred in 1976-1977), and fast rock alteration due to hot and acid fluid
circulation that worsens the risk of a partial volcano flank collapse.

Since 2015, the deployment of particle trackers developed at IP2I, Lyon, to
perform dynamic muon imaging of the lava dome structure, in addition to other
geophysical methods such as seismic monitoring, has allowed to increase the
knowledge of the hydrothermal system dynamics (see legonidec2019 ). Nowadays
two new generation particle trackers equipped with 4 scintillator panels are
installed at two different locations: South, shown on figure 1(a), and East of
the volcano. This 4-panel configuration allows to increase precision on the 2D
density radiography of the central part of the scanned region and offers two
more 3-panels sub-configurations, which lead finally to three different muon
images. Previously acquired data by our group were based on 3-panel detectors,
and processed using a direct straightness check on the scintillator bars hits
in each panel. Here, we propose a more complex particle reconstruction
algorithm, which takes into account the presence of outlier events and allows
to improve the precision on the reconstructed trajectories.

##  2 Detection technology

The detection panels are composed of an array of 16x16 plastic-scintillator
bars of 5 cm width (detection area 80x80 $cm^{2}$) (fig. 1(b)). Each bar has
its own Wavelength Shifting Fibers (WLS) embedded within the scintillator
medium (fig. 1(c)), to collect the light emitted along the charged particle
path, and guide the signal towards multi-anodes photomultipliers. In addition
to the four detection matrices, a 100 mm thick lead shielding panel has been
installed between the third and the fourth (rear) matrices, satisfying
mechanical constraints on the telescope structure. This passive shielding
allows to stop low-energy particles and also trigger electromagnetic showers
inside our detection volume initiated by non-muonic events characterized with
high hit multiplicity on the rear panel caused by secondary shower particles.
For more information about the detector assembly, we invite the reader to
refer to lesparre2012  and jourde2013 ; for details about the light and
electronics readout system to marteau2010  and marteau2013 .

(a)

(b)

(c)

Figure 1: (a) photo of the 4-panel detector Super Nain Jaune, located near La
Savane à mulets on the South flank of La Soufrière volcano; (b) a scheme of a
scintillator matrix, adapted from lesparre2012 ; (c) a scheme of a plastic
scintillator bar, with its embedded WLS fiber, adapted from opera

.

##  3 Data processing and performances

Within telescope data, the detected muon events outgoing from the volcano are
polluted with noise resulting from the electromagnetic component of
atmospheric showers, and secondary electrons from interactions inside the
detector, as well as an important contribution from low-energy muons scattered
on volcano flanks gomez2017 . To mitigate the noise contribution on
radiographies, raw data are processed event-by-event with a preliminary filter
on the minimum number of impacted detection panels, and hit multiplicity.
Events with a multiplicity larger than 10 on one panel are rejected, to remove
electromagnetic shower-like events triggered after shielding panel crossing.
After filtering, to minimize the impact of the noisy hits on the measurement
of the incident muon direction, we have chosen to reconstruct each track with
a random sampling consensus (RANSAC) fit procedure.

###  3.1 The RANSAC procedure

The RANSAC algorithm ransac  is based on an hypothesize and test iterative
process. The process starts by randomly sampling within the input dataset,
i.e. the hits XYZ coordinates, a subset of points of given size and fit an
hypothesized line model to this subset. It then evaluates which hits of the
whole dataset are consistent with this hypothesized model, computing a fit
error, and counting the number of hits whose orthogonal distance to the model
is below a certain distance threshold (set to one scintillator width). Those
hits forms the consensus set. After N iterations, the algorithm outputs the
model that minimizes the fit error, and maximizes the size of the consensus
set. The hits belonging to the consensus are tagged as ’inliers’ and the rest
of the dataset as ’outliers’. N is set so that the probability of getting a
pure-signal consensus, i.e. only composed of true inliers, at the end of the
process is 0.99.

###  3.2 Numerical simulation

To measure the performances of this RANSAC-based reconstruction, we have
developed a dedicated GEANT4 geant4  simulation with our 4-panel detector
geometry, illustrated on figure 2, interfaced with the MC generator CRY cry
to simulate the incident flux from atmospheric showers (muonic, hadronic and
electromagnetic components). CRY appeared well-suited for processing
performance tests, since it is very fast: $O(1min)$ for $10^{5}$ MC events.

Figure 2: An event display of cosmic muons emitted from a plane surface above
the front panel of the 4-panel detector using the CRY MC generator. The
scintillator matrices are colored in green, the shielding parts are in red,
and a 100-mm-thick lead panel is located between the third and the fourth
panel.

###  3.3 Performances

Using the simulation framework described in 3.2, we first compared the RANSAC
processing output of pure-muon and pure-electron samples of $10^{6}$ particles
each. We then measured, for the muon sample, the reconstruction efficiency and
the angular resolution of the estimated trajectories. We compared these
measurements with the results obtained from a simpler processing algorithm
relying on a track straightness criteria, without any trajectory fit involved.
This former processing was based on the check of the alignment of the most
energetic hit formed in each impacted panel, which is a significant bias since
muons are minimum ionizing particles and therefore would not be expected to
systematically form the highest energy deposit in each detection matrix.

####  3.3.1 Muon vs electron signal

We compare in figure 3 the muon and electron charge signals $dQ$ in the rear
panel of our detector (bottom panel in figure 2), located behind the lead
shielding panel. We observe that 96% of the muon signals on figure 3(b),
fitted with a Landau function, are tagged at as inlier. We also observe a
suppression of noisy hits at lower ADC values. As for electrons, we observe a
large proportion of outlier hits that populates the low ADC values, with a
ratio outlier/inlier around 85%.

(a) Charge signal for muons $dQ_{\mu}$ before processing

(b) $dQ_{\mu}$ after processing

(c) Charge signal for electrons $dQ_{e}$ before processing

(d) $dQ_{e}$ after processing

Figure 3: Comparison of the muon and electron charge signals recorded in the
rear telescope panel, shown before and after RANSAC processing. The charge is
measured here after digitization of the energy deposits measured in GEANT4,
and is given in ADC counts unit.

####  3.3.2 Angular resolution

In figure 4, we show the angular deviation in zenith ($\Delta\theta$) and
azimuth ($\Delta\phi$) between the generated muon straight track and the
reconstructed track. We observe for the RANSAC processing an improvement of
the angular resolutions in both zenith and azimuth of the order of 30%,
compared to the straightness check-based processing.

(a) RANSAC

(b) Straightness check

Figure 4: Angular resolution in Zenith (upper line) and Azimuth (bottom line)
angles for the estimated muon trajectories, for the (4(a)) RANSAC and (4(b))
and straightness check processing.

####  3.3.3 Reconstruction efficiency

In figure 5, we have measured the muon reconstruction efficiency
$\epsilon_{\mu}$ for each 4-panel detector line-of-sight. For each pixel,
representing a given line-of-sight, we show the ratio of the number of
reconstructed tracks and the total number of generated muon straight track. We
observe for the RANSAC processing an increase by 10% of the mean efficiency
$\langle\epsilon_{\mu}\rangle$ with respect to the straightness check-based
processing.

(a) RANSAC

(b) Straightness check

Figure 5: Muon reconstruction efficiency $\epsilon_{\mu}$ maps for (5(a))
RANSAC with a mean efficiency $\langle\epsilon_{\mu}\rangle=0.71\pm 0.11$ and
(5(b)) straightness check processing with
$\langle\epsilon_{\mu}\rangle=0.61\pm 0.10$. Each pixel is a given telescope
line-of-sight.

##  4  Application to an Open-Sky acquisition

We present in this section the application of the RANSAC processing to real
data taken with a 4-panels telescope, as described in 2. The data covers 36
hours and has been recorded in open-sky mode, measuring the vertical cosmic
muon flux centered around zenith $\theta=0^{\circ}$. This run serves as a
calibration step for measuring the experimental detector acceptance, which
allows to take into account detection matrices flaws due notably to some
imperfect optical couplings marteau2016  in the dome-outgoing flux
computation. The integrated acceptance analytical function, presented in
sullivan1971 , on the other hand, is purely geometrical and depends only on
the dimensions of our detector: distance between front and rear matrices,
number, length and width of scintillator bars in each matrices.

(a) Theoretical acceptance

(b) RANSAC experimental acceptance

Figure 6: (6(a)) Theoretical integrated acceptance of the 4-panel telescope
geometry and (6(b)) experimental acceptance obtained after open-sky data
RANSAC processing.

(a) RANSAC

(b) Straightness check

Figure 7: Ratios between experimental and theoretical integrated acceptance
$\mathcal{T}_{exp}$/$\mathcal{T}_{th}$ for (7(a)) RANSAC processing and (7(b))
former straightness check processing .

We observe in figure 7 that the mean ratio between experimental and
theoretical acceptances is $0.38\pm 0.02$ using the RANSAC processing. This
reduction factor is consistent with previous detection efficiency estimation
obtained in lesparre2012 . As for the former processing algorithm, this ratio
is equal to $0.19\pm 0.01$, that is, reduced by a factor 2. This means that
with the new RANSAC method we manage to reconstruct twice more events for real
open-sky data.

##  5 Summary and outlook

In this paper, we have presented a new reconstruction algorithm, based on the
RANSAC procedure, for scintillator-based muon telescope data and compared it
with our former processing algorithm that relied on a track straightness check
criterion without any trajectory fit for the incident particles. The main
drawback of the RANSAC-based algorithm is in an increased processing time,
multiplied by a factor 2 for $10^{5}$ MC muon events on a standard 8Go CPU. In
spite of this, we observe that a RANSAC-based reconstruction of the muon
tracks in the detector offers significant advantages. From the RANSAC
processing of a pure muon MC sample, we highlight :

  * •

a 30% gain in muon angular resolution

  * •

a 10% increase in muon reconstruction efficiency

  * •

a 13% increase in the muon count

Although in a real world data acquisition, we would not be able to perform
precise particle identification, we have already observed for open-sky real
data that the RANSAC-reconstructed event count is doubled compared to the one
obtained with the former processing algorithm, leading to a telescope
effective acceptance estimation closer to the theoretical integrated
acceptance expected for the 4-panel telescope configuration. This increase in
reconstructed event rate will need to be further confirmed with a realistic
CORSIKAcorsika  cosmic flux generation, taking into account all the cosmic
sources of physical noise, that were not present in the current GEANT4
simulation study.

##  6 Future studies

The RANSAC processing will be used to reconstruct the available tomography
datasets taken both with new generation 4-panels telescopes presented in
sec.2, and with former 3-panels telescopes. The 2D density radiographies
obtained for different scanned regions of the lava dome will then be combined
to serve as input for numerical modelling, conjointly with other geophysics
data sets (e.g, from gravimetry surveys), in order to obtain a 3D bulk density
distribution model of the lava dome at an expected unprecedentedly high
resolution thanks to the uniqueness of the different data sets available and
the quality of the event reconstruction. This will lead to a better
characterization of the hydrothermal fluid circulation impact on rock
alteration and the resulting partial flank collapse hazard.

## Acknowledgements

This work and the data shown were acquired in the context of the ANR MEGaMu.
We also wish to thank Axel Le Berrigaud, for his contribution to the first
applications of the RANSAC-based reconstruction during his master thesis at
IPGP, in 2020.

## References

  * [1] Le Gonidec, Y., Rosas-Carbajal, M., Bremond d’Ars, J.d. et al.: Abrupt changes of hydrothermal activity in a lava dome detected by combined seismic and muon monitoring. Sci Rep 9, 3079, https://doi.org/10.1038/s41598-019-39606-3 (2019)
  * [2] Lesparre, N., Marteau, J., Déclais, Y., Gibert, D., Carlus, B., Nicollin, F., Kergosien, B.: Design, operation of a field telescope for cosmic ray geophysical tomography. Geoscientific Instrumentation, Methods, Data Systems, 1(1):33–42. (2012)
  * [3] Jourde, K., Gibert, D., Marteau, J., Bremond d’Ars, J., Gardien, S., Girerd, C., Ianigro, J.-C., Carbone, D.: Experimental detection of up-ward going cosmic particles and consequences for correction of density radiography of volcanoes, 2013, Geophysical Research Letters, 40(24):6334–6339.
  * [4] Marteau, J.: The OPERA global readout, GPS distribution system. Nuclear Instruments, Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors, Associated Equipment, 617(1-3):291–293. (2010)
  * [5] Marteau, J., de Bremond d’Ars, J., Gibert, D., Jourde, K., Gardien, S., Girerd, C., and Ianigro, J.-C.: Implementation of sub-nanoseconds TDC in FPGA: applications to time-of-flight analysis in muon radiography. Measurement Science, Technology, 25(3):035101. (2014)
  * [6] H. Gómez, D. Gibert, C. Goy, K. Jourde, Y. Karyotakis, S. Katsanevas, J. Marteau, M. Rosas-Carbajal, A. Tonazzo: Forward scattering effects on muon imaging. arXiv:1709.05106 [physics.ins-det] (2017)
  * [7] Fischler, Martin A. and Bolles, Robert C., Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. https://doi.org/10.1145/358669.358692, Commun. ACM (1981)
  * [8] S.Agostinelli et al.: Geant4—a simulation toolkit, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment Volume 506, Issue 3, 1 July 2003, Pages 250-303 (2003)
  * [9] C. Hagmann, D. Lange, and D. Wright: Cosmic-ray shower generator (CRY) for Monte Carlo transport codes, IEEE Nuclear Science Symposium Conference Record, volume 2, pages 1143–1146. IEEE. (2007)
  * [10] Marteau, J., de Bremond d’Ars, J., Gibert, D., Jourde, K., Ianigro, J.-C., Carlus, B.: DIAPHANE: Muon tomography applied to volcanoes, civil engineering, archaelogy, arXiv:1612.03905 [physics.geo-ph] (2016)
  * [11] Sullivan, J-D: Geometric factor and directional response of single and multi-element particle telescopes; Nuclear Instruments and Methods, volume 95, 1, pages 5-11, issn 0029-554X, https://doi.org/10.1016/0029-554X(71)90033-4 (1971)
  * [12] OPERA collaboration: The OPERA experiment Target Tracker, arXiv:physics/0701153v1 [physics.ins-det] (2007)
  * [13] D. Heck, J. Knapp, J-N. Capdevielle, G. Schatz, T. Thouw. CORSIKA: A Monte Carlo Code to Simulate Extensive Air Showers., https://ui.adsabs.harvard.edu/abs/1998cmcc.book.....H (1998)