# Immersive Virtual Reality Simulations of Bionic Vision

Justin Kasowski  University of California, Santa BarbaraSanta BarbaraCAUSA
<EMAIL_ADDRESS>and  Michael Beyeler  University of California, Santa
BarbaraSanta BarbaraCAUSA<EMAIL_ADDRESS>
(2022)

###### Abstract.

Bionic vision uses neuroprostheses to restore useful vision to people living
with incurable blindness. However, a major outstanding challenge is predicting
what people “see” when they use their devices. The limited field of view of
current devices necessitates head movements to scan the scene, which is
difficult to simulate on a computer screen. In addition, many computational
models of bionic vision lack biological realism. To address these challenges,
we present VR-SPV, an open-source virtual reality toolbox for simulated
prosthetic vision that uses a psychophysically validated computational model
to allow sighted participants to “see through the eyes” of a bionic eye user.
To demonstrate its utility, we systematically evaluated how clinically
reported visual distortions affect performance in a letter recognition and an
immersive obstacle avoidance task. Our results highlight the importance of
using an appropriate phosphene model when predicting visual outcomes for
bionic vision.

retinal implant, virtual reality, simulated prosthetic vision

††journalyear: 2022††copyright: none††conference: Augmented Humans 2022; March
, 2022; Preprint††booktitle: Augmented Humans 2022 (AHs 2022), March 13–15,
2022, Kashiwa, Chiba, Japan††doi: 10.1145/3519391.3522752††isbn: PRE-PRINT,
PLEASE CITE DOI:††ccs: Human-centered computing Accessibility
technologies††ccs: Human-centered computing Virtual reality††ccs: Human-
centered computing Empirical studies in visualization

Figure 1. Immersive virtual reality simulations of bionic vision. A
microelectrode array is implanted in the eye to stimulate the retina
$\rightarrow$ Anatomical data is used to position a simulated electrode array
on a simulated retina to create a “virtual patient” $\rightarrow$ Visual input
from a virtual reality environment acts as stimulus for the simulated implant
to generate a realistic prediction of simulated prosthetic vision (SPV)
$\rightarrow$ The rendered SPV image is presented to the virtual patient and
behavioral metrics are recorded

Figure 1: Left Panel. An image labelled as “Fit anatomical data, real patient”
showing a fundus image of an implanted retinal prosthesis. An arrow points to
the next panel labelled “Simulate, virtual patient” which shows a simulated
electrode array overtop of a retinal map drawing. An arrow points to the next
panel which is labelled “Stimulate” and shows a person with a virtual reality
headset and the Letter E with a simulated electrode array over it. An arrow
points to the final panel which is labelled “Evaluate” and shows a distorted
version of the parts of the E that activated the simulated electrode array

##  1\. Introduction

The World Health Organization has estimated that by the year 2050, roughly
114.6 million people will be living with incurable blindness and 587.6 million
people will be affected by severe visual impairment (Bourne et al., 2017).
Although some affected individuals can be treated with surgery or medication,
there are no effective treatments for many people blinded by severe
degeneration or damage to the retina, the optic nerve, or cortex. In such
cases, an electronic visual prosthesis (“bionic eye”) may be the only option
(Fernandez, 2018). Analogous to cochlear implants, these devices electrically
stimulate surviving cells in the visual pathway to evoke visual percepts
(“phosphenes”). The phosphenes produced by current prostheses generally
provide users with an improved ability to localize high-contrast objects and
perform basic orientation & mobility tasks (Ayton et al., 2020), but are not
yet able to match the acuity of natural vision.

Despite their potential to restore vision to people living with incurable
blindness, the number of bionic eye users in the world is still relatively
small (roughly $500$ retinal prostheses implanted to date). To investigate
functional recovery and experiment with different implant designs, researchers
have therefore been developing virtual reality (VR) prototypes that rely on
simulated prosthetic vision (SPV). The classical method relies on sighted
subjects wearing a VR headset, who are then deprived of natural viewing and
only perceive phosphenes displayed in a head-mounted display (HMD). This
allows sighted participants to “see through the eyes” of the bionic eye user,
taking into account their head and/or eye movements as they explore a virtual
environment (Kasowski et al., 2021b).

However, most SPV studies simply present stimuli on a computer monitor or an
HMD, without taking into account eye movements, head motion, or locomotion
(Kasowski et al., 2021a). This leads to a low level of immersion (Kardong-
Edgren et al., 2019; Pasch et al., 2009), which refers to technical
manipulations that separate the existence of the physical world from the
virtual world (Miller and Bugnariu, 2016). Seeing the real world, using a
keyboard or joystick to move, and sounds present in the real environment are
factors which lead to lower levels of immersion. However, the role of
immersion for behavioral tasks in SPV is still unclear as no previous study
has assessed whether behavioral performance is comparable across monitor-based
and HMD-based versions of the same task.

In addition, most current prostheses provide a very limited field of view
(FOV); for example, the artificial vision generated by Argus II (Luo and da
Cruz, 2016), the most widely adopted retinal implant thus far, is restricted
to roughly $10\times 20$ degrees of visual angle. This forces users to scan
the environment with strategic head movements while attempting to piece
together the information (Erickson-Davis and Korzybska, 2021). The emergence
of immersive VR as a research tool provides researchers with the ability to
simulate this in a meaningful way.

Moreover, a recent literature review found that most SPV studies relied on
phosphene models with a low level of biological realism (Kasowski et al.,
2021a). It is therefore unclear how the findings of most SPV studies would
translate to real bionic eye users.

To address these challenges, we make three contributions:

  * •

We present VR-SPV, a virtual reality (VR) reality toolbox for simulated
prosthetic vision (SPV) that allows sighted participants to “see through the
eyes” of a bionic eye user.

  * •

Importantly, we use an established and psychophysically validated
computational model of bionic vision (Beyeler et al., 2017a) to generate
realistic SPV predictions.

  * •

We systematically evaluate how different display types (HMD or monitor) affect
behavioral performance in a letter recognition and an obstacle avoidance task.
To the best of our knowledge, this is the first SPV study that uses a within-
subjects design to allow for a direct comparison between display types of the
same tasks.

##  2\. Background

###  2.1. Restoring Vision with a Bionic Eye

The only FDA-approved technology for the treatment of retinal degenerative
blindness are visual neuroprostheses. These devices consist of an electrode
array implanted into the eye or brain that is used to artificially stimulate
surviving cells in the visual system. Two retinal implants are already
commercially available (Argus II: 60 electrodes, Second Sight Medical
Products, Inc. (Luo and da Cruz, 2016); Alpha-AMS (1600 electrodes, Retina
Implant AG, (Stingl et al., 2013)), and many other emerging devices have
reached the clinical or pre-clinical stage (Lorach et al., 2015; Ferlauto et
al., 2018; Ayton et al., 2014). In order to create meaningful progress within
these devices, there is a growing need to understand how the vision these
devices provide differs from natural sight (Erickson-Davis and Korzybska,
2021).

One common fallacy is the assumption that each electrode produces a focal spot
of light in the visual field (Perez-Yus et al., 2017; Sanchez-Garcia et al.,
2019; Kvansakul et al., 2020). This is known as the scoreboard model, which
implies that creating a complex visual scene can be accomplished simply by
using the right combination of pixels (analogous to creating numbers on a
sports stadium scoreboard). On the contrary, recent work suggests that
phosphenes vary in shape and size, differing considerably across subjects and
electrodes (Fine and Boynton, 2015; Luo et al., 2016; Beyeler et al., 2019b).

Figure 2. A simulated map of retinal NFBs (_left_) can account for visual
percepts (_right_) elicited by retinal implants (reprinted with permission
from (Beyeler et al., 2019a)). _Left_ : Electrical stimulation (red circle) of
a NFB (black lines) could activate retinal ganglion cell bodies peripheral to
the point of stimulation, leading to tissue activation (black shaded region)
elongated along the NFB trajectory away from the optic disc (white circle).
_Right_ : The resulting visual percept appears elongated; its shape can be
described by two parameters, $\lambda$ and $\rho$.

Figure 2: Left panel: Simulated retinal implant setup, consisting of an
electrode array situation on top of a drawing of a retina map. A single
electrode is red and the percept it generates is a black streak around the
electrode. Right panel: Predicted Visual Percept, a panel that has isolated
the percept and describes it by its elongation, labelled lambda, and its
spread, labelled rho

Increasing evidence suggests that perceived phosphene shape in an epiretinal
implant is a result of unintended stimulation of nerve fiber bundles in the
retina (Rizzo et al., 2003; Beyeler et al., 2019b). These NFBs follow polar
trajectories (Jansonius et al., 2012) away from the horizontal meridian,
forming arch-like projections into the optic nerve (Fig. 2, _left_).
Stimulating a NFB would result in the activation of nearby retinal ganglion
cells that are upstream in the trajectory, resulting in phosphenes that appear
elongated (Fig. 2, _right_).

Ref. (Beyeler et al., 2019b) demonstrated through simulations that the shape
of elicited phosphenes closely followed NFB trajectories. Their computational
model assumed that an axon’s sensitivity to electrical stimulation:

  1. i.

decayed exponentially with $\rho$ as a function of distance from the
stimulation site,

  2. ii.

decayed exponentially with $\lambda$ as a function of distance from the cell
body, measured as axon path length.

In other words, the values of $\rho$ and $\lambda$ in this model dictate the
size and elongation of phosphenes, respectively. This may drastically affect
visual outcomes, as large values of $\lambda$ are thought to distort phosphene
shape.

###  2.2. Related Work

The use of virtual reality has emerged as a tool for assisting users with low
vision (see (Kasowski et al., 2021a) for a review of recent literature). This
includes not just accessibility aids, but also simulations aimed at
understanding low vision. A number of previous SPV studies have focused on
assessing the impact of different stimulus and model parameters (e.g.,
phosphene size, phosphene spacing, flicker rate) on measures of visual acuity.
Stimuli for these low-level visual function tests were often presented on
monitors (Vurro et al., 2014; Lu et al., 2012) or in HMDs (Wu et al., 2014;
Cao et al., 2017). Some studies also tested the influence of FOV (Sanchez-
Garcia et al., 2020; Thorn et al., 2020) and eye gaze compensation (Titchener
et al., 2018) on acuity. Others focused on slightly more complex tasks such as
letter (Zhao et al., 2011), word (Perez Fornos et al., 2011), face (Denis et
al., 2013; Chang et al., 2012), and object recognition (Zhao et al., 2010;
Wang et al., 2018; Macé et al., 2015). In most setups, participants would view
SPV stimuli in a conventional VR HMD, but some studies also relied on smart
glasses to present SPV in augmented reality (AR).

However, because most of the studies mentioned above relied on the scoreboard
model, it is unclear how their findings would translate to real bionic eye
users. Although some studies attempted to address phosphene distortions (Wu et
al., 2014; Guo et al., 2014; Subramaniam et al., 2012), most did not account
for the neuroanatomy (e.g., NFB trajectories) when deciding how to distort
phosphenes. Only a handful of studies have incorporated a great amount of
neurophysiological detail into their setup (Josh et al., 2013; Vurro et al.,
2014; Wang et al., 2018; Thorn et al., 2020), only two of which (Wang et al.,
2018; Thorn et al., 2020) relied on an established and psychophysically
validated model of SPV. One notable example is the study by Thorn _et al._
(Thorn et al., 2020), which accounted for unintentional stimulation of axon
fibers in the retina by adding a fixed “tail” length to each phosphene.
However, a fixed-length tail is a simplification of the model (Beyeler et al.,
2019b) as the size of phosphenes (and their tails) have been shown to vary
with stimulation parameters such as amplitude, frequency, and pulse duration
(Nanduri et al., 2012).

In addition, being able to move around as one would in real life has shown to
significantly increase the amount of immersion a user experiences (Pasch et
al., 2009). However, the level of immersion offered by most SPV studies is
relatively low, as stimuli are often presented on a screen (Ying et al., 2018;
Wang et al., 2018). In contrast, most current prostheses provide a very
limited FOV (e.g., Argus II: $10\times 20$ degrees of visual angle), which
requires users to scan the environment with strategic head movements while
trying to piece together the information (Erickson-Davis and Korzybska, 2021).
Furthermore, Argus II does not take into account the eye movements of the user
when updating the visual scene, which can be disorienting for the user.
Ignoring these human-computer interaction (HCI) aspects of bionic vision can
result in unrealistic predictions of prosthetic performance, sometimes even
exceeding theoretical acuity limits (as pointed out by (Caspi and Zivotofsky,
2015)).

In summary, previous SPV research has assumed that the scoreboard model
produces phosphenes that are perceptually similar to real bionic vision
(Vergnieux et al., 2014; Zhao et al., 2011), and that findings from an HMD-
based task would more accurately represent the experience of a bionic eye user
than a monitor version (Zapf et al., 2014; Sanchez-Garcia et al., 2020; Thorn
et al., 2020). In this paper we aim to systematically evaluate these
assumptions with a within-subjects (repeated measures) design, allowing for
direct comparisons in performance across different model parameters and
display conditions.

##  3\. Methods

###  3.1. VR-SPV: A Virtual Reality Toolbox for Simulated Prosthetic Vision

The VR-SPV system consisted of either a wireless head-mounted VR headset (HTC
VIVE Pro Eye with wireless adapter, HTC Corporation) or a standard computer
monitor (Asus VG248QE, 27in, 144Hz, 1920x1080p). Both HMD and monitor versions
used the same computer for image processing (Intel i9-9900k processor and an
Nvidia RTX 2070 Super GPU with 16GB of DDR4 memory). All software was
developed using the Unity development platform, consisting of a combination of
C# code processed by the central processing unit (CPU) and fragment/compute
shaders processed by the graphics processing unit (GPU). The entire software
package, along with a more in-depth explanation, is available at
https://github.com/bionicvisionlab/BionicVisionXR.

The workflow for simulating bionic vision was as follows:

  1. i.

Image acquisition: Utilize Unity’s virtual camera to acquire the scene at
roughly 90 frames per second and downscale to a target texture of $86\times
86$ pixels.

  2. ii.

Image processing: Conduct any image preprocessing specified by the user.
Examples include grayscaling, extracting and enhancing edges, and contrast
maximization. In the current study, the image was converted to grayscale and
edges were extracted in the target texture with a $3\times 3$ Sobel operator.

  3. iii.

Electrode activation: Determine electrode activation based on the visual input
as well as the placement of the simulated retinal implant. In the current
study, a $3\times 3$ Gaussian blur was applied to the preprocessed image to
average the grayscale values around each electrode’s location in the visual
field. This gray level was then interpreted as a current amplitude delivered
to a particular electrode in the array.

  4. iv.

Phosphene model: Use Unity shaders to convert electrode activation information
to a visual scene in real time. The current study re-implemented the axon map
model available in _pulse2percept_ (Beyeler et al., 2017a) using shaders.

  5. v.

Phosphene rendering: Render the elicited phosphenes either on the computer
monitor or the HMD of the VR system.

The VR-SPV system is designed to handle any retinal implant by allowing users
to specify the location and size of each electrode in the simulated device. It
can also handle other phosphene models, including cortical models, by
replacing the model provided by _pulse2percept_ with any phosphene model of
their choosing.

While not considered in this study, VR-SPV can also be used to model temporal
interactions by integrating electrode activation from previous frames or by
only rendering at a specific frequency. The software is also capable of
utilizing the VIVE’s eye tracking hardware to elicit a “gaze lock”. This
function moves the rendered image to the center of the user’s gaze, attempting
to replicate the inability of a prosthetic user to scan the presented image
with eye movements. Neither of these optional functions were used in this
study as they were not the focus of the current work, and it was unclear how
these settings would influence any findings on the parameters being studied.
VR-SPV also includes a function to change the source of the visual input from
a virtual environment to the HMD’s front-facing camera for supporting AR
applications.

Figure 3. Letter recognition task. _Top:_ The lights in the virtual room are
turned off and the image seen by the user is passed to the preprocessing
shader which performs edge extraction/enhancement before the axon model shader
renders SPV. Modeled after (Cruz et al., 2013). _Bottom:_ Output of the axon
model shader across the various devices and $\rho$ / $\lambda$ combinations.

Figure 3: A picture of a chair and laptop labelled Virtual Environment. An
arrow points to a zoomed in picture of the laptop screen with the letter “E”
labelled Participant View. An arrow points to a similar picture but only the
laptop screen is visible, labelled Lights Off. An arrow points to a processed
image that is a thicker version of the letter “E” from the previous panel,
labelled Preprocessed Image. An arrow points to a small box with the words
Axon Model Shader. The axon Model shader points to three different panels
showing the outputs of simulated prosthetic vision across different devices
for the preprocessed image

###  3.2. Simulated Prosthetic Vision

The underlying phosphene model for this experiment was a re-implementation of
the pyschophysically validated axon map model (Beyeler et al., 2019b) provided
by _pulse2percept_ (Beyeler et al., 2017a). To support real-time execution, an
initial mapping of each electrode’s effects on the scene were pre-calculated
with _pulse2percept_ before starting the experiment. The shape of the elicited
phosphenes was based on the retinal location of the simulated implant as well
as model parameters $\rho$ and $\lambda$ (see Section 2). As can be seen in
Fig. 2 (_left_), electrodes near the horizontal meridian activated cells close
to the end of the NFBs, limiting the potential of elongation along an axon.
This resulted in more circular phosphenes, whereas other electrodes were
predicted to produce elongated percepts that differed in angle based on
whether they fell above or below the horizontal meridian.

We were particularly interested in assessing how different SPV model
parameters affected behavioral performance. Importantly, $\rho$ and $\lambda$
vary drastically across patients (Beyeler et al., 2019b). Although the reason
for this is not fully understood, it is clear that the choice of these
parameter values may drastically affect the quality of the generated visual
experience. To cover a broad range of potential visual outcomes, we simulated
nine different conditions by combining $\rho=\\{100,300,500\\}$
$\text{\,}\mathrm{\SIUnitSymbolMicro m}$ with
$\lambda=\\{50,1000,5000\\}$$\text{\,}\mathrm{\SIUnitSymbolMicro m}$.

We were also interested in how the number of electrodes in an implant and the
associated change in FOV affected behavioral performance. In addition to
simulating Argus II, we created two hypothetical near-future devices that used
the same aspect ratio and electrode spacing, but featured a much larger number
of electrodes. Thus the three devices tested were:

  * •

Argus II: $6\times 10=60$ equally spaced electrodes situated
$575\text{\,}\mathrm{\SIUnitSymbolMicro m}$ apart in a rectangular grid. To
match the implantation strategy of Argus II, the device was simulated at
$-45\text{\,}\mathrm{\SIUnitSymbolDegree}$ with respect to the horizontal
meridian in the dominant eye.

  * •

Argus III (hypothetical): $10\times 16=160$ electrodes spaced
$575\text{\,}\mathrm{\SIUnitSymbolMicro m}$ apart in a rectangular grid
implanted at $0\text{\,}\mathrm{\SIUnitSymbolDegree}$. A recent modeling study
suggests that this implantation angle might minimize phosphene streaks
(Beyeler et al., 2019a).

  * •

Argus IV (hypothetical): $19\times 31=589$ electrodes spaced
$575\text{\,}\mathrm{\SIUnitSymbolMicro m}$ apart in a rectangular grid
implanted at $0\text{\,}\mathrm{\SIUnitSymbolDegree}$.

###  3.3. Participants

We recruited 17 sighted participants (6 female and 11 male; ages $27.4\pm 5.7$
years) from the student pool at the University of California: Santa Barbara.
Participation was voluntary and subjects were informed of their right to
freely withdraw for any reason. Recruitment and experimentation followed
protocols approved by the university’s Institutional Review Board, along with
limitations and safety protocols approved by the university’s COVID-19
response committee.

None of the participants had previous experience with SPV. Participants were
split into two equally sized groups; one starting with the HMD-based version
of the first experiment while the other started with the monitor-based
version.

In order to get accommodated with the SPV setup, participants began each task
with the easiest block; that is, the scoreboard model
($\lambda$=$50\text{\,}\mathrm{\SIUnitSymbolMicro m}$) with the smallest
possible phosphene size and the highest number of electrodes. The order of all
subsequent blocks was randomized for each participant.

Figure 4. Letter recognition task. Data points represent each subject’s
average performance in a block with boxplots displaying median and
interquartile ranges. _Top:_ Average F1 score across blocks for each subject
within the condition specified by the x-axis. _Bottom_ : Average time across
blocks for each subject within the condition specified by the x-axis.
Statistical significance was determined using ART ANOVA (*¡.05, **¡.01,
***¡.001).

Six box plots for the letter recognition task that are fully explained plot by
plot in the results

##  4\. Experiments and Results

To study the impact of SPV parameters and level of immersion, we replicated
two popular tasks from the bionic vision literature. The first task was a
basic letter recognition experiment (Cruz et al., 2013), tasking participants
with identifying the letter presented to them. The second one was a more
immersive orientation & mobility task, requiring subjects to walk down a
virtual hallway while avoiding obstacles (He et al., 2020).

To allow for a direct comparison across all conditions, we chose a within-
subjects, randomized block design. This systematic side-by-side comparison
minimized the risk of learning effects and other artifacts that may arise from
inhomogeneity between groups, allowing for meaningful statistics with a
relatively small number of subjects.

The procedures and results for each task are presented separately below,
followed by a joint discussion on both experiments in the subsequent sections.

###  4.1. Task 1: Letter Recognition

####  4.1.1. Original Task

The first experiment was modeled after a letter recognition task performed by
Argus II recipients (Cruz et al., 2013). In the original task, following a
short training period, participants were instructed to identify large and
bright white letters presented on a black screen situated
$0.3\text{\,}\mathrm{m}$ in front of them. Participants were given unlimited
time to respond. The experiment was carried out in a darkened room. Both the
initial training period and the actual experiment featured all 26 letters of
the alphabet. The letters were grouped by similarity and tested in batches of
8, 8, and 10 letters.

####  4.1.2. Experimental Setup and Procedure

To emulate the experiment described in (Cruz et al., 2013), we carefully
matched our virtual environment to the experimental setup of the original
task. The setup mainly consisted of a virtual laptop on top of a virtual desk
(Fig. 3). A virtual monitor was positioned $0.3\text{\,}\mathrm{m}$ in front
of the user’s head position. In agreement with the original task, participants
were presented letters that were $22.5\text{\,}\mathrm{cm}$ tall (subtending
$41.112\text{\,}\mathrm{\SIUnitSymbolDegree}$ of visual angle) in True Type
Century Gothic font. For the monitor version of the task, the camera was
positioned at the origin and participants could simulate head movements by
using the mouse.

| F1 Score ($\pm$ Std Dev) | Mean Time (s) ($\pm$ Std Dev)
---|---|---
| HMD | Monitor | HMD | Monitor
06x10 Array | 0.411 ($\pm$ 0.341 ) | 0.344 ($\pm$ 0.339 ) | 8.312 ($\pm$ 5.685 ) | 7.979 ($\pm$ 7.367 )
10x16 Array | 0.628 ($\pm$ 0.361 ) | 0.546 ($\pm$ 0.380 ) | 5.853 ($\pm$ 4.808 ) | 5.567 ($\pm$ 3.608 )
19x31 Array | 0.699 ($\pm$ 0.347 ) | 0.596 ($\pm$ 0.373 ) | 5.379 ($\pm$ 4.164 ) | 5.661 ($\pm$ 4.373 )
$\rho$=100 | 0.570 ($\pm$ 0.388 ) | 0.467 ($\pm$ 0.381 ) | 7.415 ($\pm$ 5.874 ) | 7.007 ($\pm$ 6.120 )
$\rho$=300 | 0.620 ($\pm$ 0.366 ) | 0.540 ($\pm$ 0.379 ) | 6.173 ($\pm$ 4.879 ) | 5.829 ($\pm$ 4.686 )
$\rho$=500 | 0.548 ($\pm$ 0.352 ) | 0.479 ($\pm$ 0.377 ) | 5.956 ($\pm$ 4.267 ) | 6.371 ($\pm$ 5.483 )
$\lambda$=50 | 0.824 ($\pm$ 0.267 ) | 0.750 ($\pm$ 0.329 ) | 4.540 ($\pm$ 3.408 ) | 5.034 ($\pm$ 4.403 )
$\lambda$=1000 | 0.665 ($\pm$ 0.331 ) | 0.543 ($\pm$ 0.362 ) | 5.698 ($\pm$ 4.338 ) | 6.074 ($\pm$ 5.780 )
$\lambda$=5000 | 0.248 ($\pm$ 0.229 ) | 0.193 ($\pm$ 0.188 ) | 9.307 ($\pm$ 5.906 ) | 8.099 ($\pm$ 5.700 )
Table 1. Letter recognition task: Average performance and time per trial
across conditions. Best performances (highest F1/shortest time) for each
grouping are presented in bold.

Each combination of 3 devices $\times$ 3 $\rho$ values $\times$ 3 $\lambda$
values were implemented as a block, resulting in a total of 27 blocks. All 27
blocks were completed twice; once for the HMD version of the task, and once
for the monitor version of the task. Rather than presenting all 26 letters of
the alphabet (as in the original experiment), we limited our stimuli to the
original Snellen letters (C, D, E, F, L, O, P, T, Z) for the sake of
feasibility.

All nine Snellen letters were presented in each block, resulting in a total of
243 trials. Participants were limited to 1 minute per trial, after which the
virtual monitor would go dark and the participant had to select a letter
before the experiment continued.

To acclimate participants to the task and controls, we had them perform an
initial practice trial using normal vision. After that, the lights in the
virtual room were turned off and the VR-SPV toolbox was used to generate SPV.
To mimic the training session of (Cruz et al., 2013), participants completed
three practice trials using SPV at the beginning of each block. Participants
were able to repeat each practice trial until they had selected the correct
letter. To prevent participants from memorizing letters seen during practice
trials, we limited practice trials to the letters Q, I, and N.

Participant responses and time per trial were recorded for the entirety of the
experiment.

####  4.1.3. Performance Evaluation

Perceptual performance was assessed using F1 scores, which represent the
harmonic mean between precision and recall, allowing for a slight penalty
towards false positive choices compared to recall (proportion correct) on its
own. This had the advantage of eliminating bias towards specific letter
choices. F1 values were calculated for each block using the scikit-learn
‘f1_score’ function (Pedregosa et al., 2011). We also measured time per trial
with the assumption that easier trials could be completed faster than trials
that were more difficult.

Due to ceiling and floor effects, neither outcome measure (F1 scores and time
per trial) were normally distributed, violating the assumptions of the
standard ANOVA. We therefore performed a subsequent aligned rank transform
(ART) with the R package ARTool (Wobbrock et al., 2011) for both F1 scores and
time per trial. This method of analysis allows for a factorial ANOVA to be
performed on repeated measures, non-uniform data, and lower subject counts
(Wobbrock et al., 2011). Post-hoc analyses were performed on significant
groups by analyzing the rank-transformed contrasts (Elkin et al., 2021). The
Tukey method (Tukey, 1949) was used to adjust $p$-values to correct for
multiple comparisons. All code used in the analysis, along with the raw data,
is provided at https://github.com/bionicvisionlab/2022-kasowski-immersive.

####  4.1.4. Results

Results from the letter recognition task are summarized in Table 1 and
distributions are plotted in Fig. 4. Group F-values, along with their
significance, are reported in Table 2. Each data point in Fig. 4 represents a
subject’s F1 score (Fig. 4A–C) and time per trial (Fig. 4D–F) across all
letters in a block. F1 score ranged from 0 to 1 with higher values
representing better performance. Assuming a different letter is chosen for
each selection, a chance-level F1 score would equal the probability for
randomly guessing the correct letter ($\frac{1}{9}=0.1111$).

| F1 Score | Time
---|---|---
| F-Value | Signif. | F-Value | Signif.
device | 150.8174 | 9.17E-57 | 25.1232 | 2.51E-11
$\rho$ | 9.9004 | 5.62E-05 | 8.6049 | 2.00E-04
$\lambda$ | 535.8116 | 3.60E-151 | 80.6779 | 8.42E-33
display | 31.5610 | 2.62E-08 | 1.4799 | 2.24E-01
device : $\rho$ | 0.7838 | 5.36E-01 | 0.5371 | 7.09E-01
device : $\lambda$ | 18.2971 | 1.98E-14 | 5.6673 | 1.67E-04
$\rho$ : $\lambda$ | 10.0737 | 5.72E-08 | 0.5573 | 6.94E-01
device : display | 0.3742 | 6.88E-01 | 1.3682 | 2.55E-01
$\rho$ : display | 0.2668 | 7.66E-01 | 0.5586 | 5.72E-01
$\lambda$ : display | 1.9499 | 1.43E-01 | 5.1031 | 6.27E-03
device : $\rho$ : $\lambda$ | 1.3828 | 2.00E-01 | 2.4198 | 1.38E-02
device : $\rho$ : display | 0.3410 | 8.50E-01 | 0.3107 | 8.71E-01
device : $\lambda$ : display | 0.3956 | 8.12E-01 | 0.1496 | 9.63E-01
$\rho$ : $\lambda$ : display | 0.7717 | 5.44E-01 | 0.6527 | 6.25E-01
device : $\rho$ : $\lambda$ : disp | 0.4598 | 8.84E-01 | 0.6592 | 7.28E-01
Table 2. Letter recognition task: F-value table for Aligned Rank Transform
(ART) ANOVA. Values were calculated with the ARTool software package. “device”
refers to the three simulated electrode grids, while “display” refers to the
use of an HMD or monitor.

Figure 5. Obstacle avoidance task. _Left:_ Layout of the virtual hallway
environment modeled after (He et al., 2020). Empty circles represent the
possible locations for obstacles. _Right/Top:_ View of the real environment -¿
participant’s view is passed to the preprocessing shader which performs edge
extraction/enhancement before the axon model shader renders SPV. _Bottom:_
Output of the axon model shader across the various devices and $\rho$ /
$\lambda$ combinations.

Left side is an overhead view of a hallway map. At the bottom are two
locations labelled Monitor Start(4.9 ft) and VR Start(9.8ft). Other positions
are labelled further up on the hallway map at 13 feet, 21 feet, and 29 feet.
At each of these locations, there are two circles on either side of the
hallway labelled as 3 feet apart. A finish line is marked at 30 feet and the
entire hallway is labelled as 8 feet by 34 feet. Right side is a picture of
room with two ropes and a person standing between them with a Virtual Reality
headset (labeled “Real Environment”). An arrow points to the next panel which
has a virtual version of a hallway with three people standing in it (labeled
“Participant View”). An arrow points to a black and white rendering of the
image where borders between people, walls, and doors have been thickened
(labeled “Preprocessed Image”). An arrow points to a square labeled “Axon
Model Shader” that points to three panels displaying simulated prosthetic
vision representation of the hallway across three devices

As expected, increasing the number of electrodes (Fig. 3A) significantly
increased F1 scores in both HMD (light gray) and monitor (dark gray) versions
of the task. It is worth noting that participants were consistently above
chance levels, even with the simulated Argus II ($6\times 10$ electrodes)
device. Increasing the number of electrodes also decreased the time it took
participants to identify the letter (Fig. 3D). However, increasing the number
of electrodes from $10\times 16$ to $19\times 31$ did not further decrease
recognition time.

Contrary to previous findings, F1 scores and recognition time did not
systematically vary as a function of phosphene size ($\rho$, Fig. 3B, E). In
both HMD and monitor-based conditions, median F1 scores were highest for
$\rho=$300\text{\,}\mathrm{\SIUnitSymbolMicro m}$$ (Table 1). However,
participants achieved similar scores with
$\rho=$100\text{\,}\mathrm{\SIUnitSymbolMicro m}$$ in the HMD version and with
$\rho=$500\text{\,}\mathrm{\SIUnitSymbolMicro m}$$ in the monitor-based
version of the task.

The most apparent differences in performance were found as a function of
phosphene elongation ($\lambda$, Fig. 3C, F). Using
$\lambda=$50\text{\,}\mathrm{\SIUnitSymbolMicro m}$$, participants achieved a
perfect median F1 score of $1.0$, but this score dropped to $0.741$ for
$\lambda=$1000\text{\,}\mathrm{\SIUnitSymbolMicro m}$$ and $0.185$ for
$\lambda=$5000\text{\,}\mathrm{\SIUnitSymbolMicro m}$$ (Table 1). Increasing
$\lambda$ also significantly increased the time it took participants to
identify the letter.

A trend toward a higher F1 score when using the HMD was observed across all
conditions (Fig. 4, _Top_), but the trend failed to reach significance for the
device with the lowest number of electrodes ($6\times 10$ array) or across the
larger distortion parameters ($\rho$=$1000\text{\,}\mathrm{\SIUnitSymbolMicro
m}$ and $\lambda$=$5000\text{\,}\mathrm{\SIUnitSymbolMicro m}$) (Fig. 4,
_Top_). While average time per trial was faster across all conditions with the
HMD, the effect was not significant (Fig. 3, _Bottom_).

###  4.2. Task 2: Obstacle Avoidance

####  4.2.1. Original Task

The second task was modeled after an obstacle avoidance experiment performed
by Argus II recipients (He et al., 2020). In this task, participants were
required to walk down a crowded hallway with one to three people located at
one of four fixed distances on either the left or right side of the hallway.
Participants were permitted the use of a cane and were allowed to touch the
walls with the cane (but not the standing persons). Participants were given
unlimited time to complete the task and were closely monitored by the
experimenter to avoid dangerous collisions. For each trial, the experimenter
instructed the participant to stop when they reached the end of the hallway.

####  4.2.2. Experimental Setup and Procedure

To emulate the experiment described in (He et al., 2020), we designed a
virtual hallway (Fig. 5, _Left_) modeled closely after the description and
pictures of the physical hallway.

Participants were tasked with successfully navigating the virtual hallway
while avoiding collisions with obstacles (simulated people). Each trial
consisted of navigating past either two or three obstacles (three trials per
condition, six trials total) located on either the left or right side of the
hallway (Fig. 5).

Figure 6. Obstacle avoidance. Data points represent each subject’s average
performance in a block with boxplots displaying median and interquartile
ranges. _Top:_ Average number of collisions across blocks for each subject
within the condition specified by the x-axis. Red line represents chance level
(1.25 collisions). _Bottom_ : Average time across blocks for each subject
within the condition specified by the x-axis. Statistical significance was
determined using ART ANOVA (*¡.05, **¡.01, ***¡.001).

Figure 6: Six box plots for the hallway task that are fully explained plot by
plot in the results section

To acclimate participants to the task and controls, we had them perform three
initial practice rounds using normal vision. After that, participants
completed three more practice rounds with a high-resolution scoreboard model
($31\times 19$ electrodes, $\rho=$100\text{\,}\mathrm{\SIUnitSymbolMicro m}$$,
$\lambda=$50\text{\,}\mathrm{\SIUnitSymbolMicro m}$$). Participants were
instructed to complete the trials as quickly as possible while avoiding
collisions. They were informed that collisions would result in audio feedback;
a sample of each sound was played at the beginning of the experiment.

Each combination of 3 devices $\times$ 3 $\rho$ values $\times$ 3 $\lambda$
values were implemented as a block, resulting in a total of 27 blocks. Block
order was randomized and participants completed six trials per block for a
total of 162 trials for each version (HMD/monitor) of the task. Participants
were limited to 1 minute per trial, after which vision was returned to normal
and participants walked to the end of the hallway to begin the next trial.

To ensure the safety of participants during the HMD-based version of the task,
we positioned rope at the real-life location corresponding to each wall of the
hallway (Fig 5, _Top, Left_). The rope served to guide the participants safely
along the path while keeping them in bounds, but was also a substitution for
the cane usage in the previous research. This substitution was necessary,
because our testing facility was much larger than the hallway in the original
experiment; thus the virtual walls did not coincide with physical walls.

An experimenter was always nearby to ensure the safety of the participants but
did not otherwise interact with them during the experiment. At the end of each
trial, the screen turned red and on-screen text instructed participants to
turn around and begin the next trial in the other direction.

The monitor version of the task was similar, but each new trial would start
automatically without the subject needing to turn around. Participants were
seated in front of a monitor and were able to use the keyboard to move and the
mouse to look around. The size of the hallway and positions of the obstacles
were identical between versions, but participants started 1.5m closer to the
first obstacle in the HMD version due to size restrictions of the room.

Collisions were detected using Unity’s standard continuous collision detection
software, with each obstacle having a
$$0.7\text{\,}\mathrm{m}$\times$0.4\text{\,}\mathrm{m}$$ hitbox and the
participant having a radius of $0.4\text{\,}\mathrm{m}$. Subject locations and
orientations were continuously recorded. Time per trial, along with individual
positions and timings of each collision, were recorded for each trial.

| Number of Collisions ($\pm$ Std Dev) | Mean Time (s) ($\pm$ Std Dev)
---|---|---
| HMD | Monitor | HMD | Monitor
06x10 Array | 1.279 ($\pm$ 0.515 ) | 1.734 ($\pm$ 0.621 ) | 22.138 ($\pm$ 10.449 ) | 14.045 ($\pm$ 10.726 )
10x16 Array | 1.148 ($\pm$ 0.602 ) |  1.603 ($\pm$ 0.593 ) | 21.483 ($\pm$ 8.216 ) | 11.622 ($\pm$ 7.274 )
19x31 Array |  1.117 ($\pm$ 0.562 ) | 1.663 ($\pm$ 0.498 ) |  21.052 ($\pm$ 7.388 ) |  9.234 ($\pm$ 4.469 )
$\rho$=100 | 1.253 ($\pm$ 0.536 ) | 1.739 ($\pm$ 0.634 ) | 22.000 ($\pm$ 9.950 ) | 13.378 ($\pm$ 10.025 )
$\rho$=300 |  1.083 ($\pm$ 0.558 ) |  1.553 ($\pm$ 0.546 ) | 21.699 ($\pm$ 8.456 ) | 11.594 ($\pm$ 7.725 )
$\rho$=500 | 1.208 ($\pm$ 0.586 ) | 1.709 ($\pm$ 0.523 ) |  20.974 ($\pm$ 7.797 ) |  9.929 ($\pm$ 5.780 )
$\lambda$=50 |  1.037 ($\pm$ 0.573 ) |  1.627 ($\pm$ 0.637 ) | 22.233 ($\pm$ 9.523 ) | 12.026 ($\pm$ 8.412 )
$\lambda$=1000 | 1.209 ($\pm$ 0.610 ) | 1.686 ($\pm$ 0.607 ) | 21.711 ($\pm$ 9.004 ) | 11.495 ($\pm$ 8.183 )
$\lambda$=5000 | 1.297 ($\pm$ 0.473 ) | 1.687 ($\pm$ 0.466 ) |  20.728 ($\pm$ 7.677 ) |  11.379 ($\pm$ 7.850 )
Table 3. Obstacle avoidance task: Average performance and time per trial
across conditions. Best performances (lowest number of collisions/lowest time)
for each grouping are presented in bold.

####  4.2.3. Evaluating Performance

Performance was assessed by counting the number of collisions per trial and
the amount of time to complete a trial, with a lower number of collisions or
lower time per trial expected on easier trials. Analogous to the first task,
these two metrics were averaged across trials in a block for each subject and
analyzed using ART ANOVA. Post-hoc analyses were performed on significant
groups using the Tukey method for multiple comparison adjustments.

####  4.2.4. Results

Results are summarized in Table 3 and Fig. 6. Each data point in Fig. 6
represents a subject’s number of collisions (Fig. 6, _Top_) and time to
completion (Fig. 6, _Bottom_) averaged across repetitions in a block. Group
F-values, along with their significance, are reported in Table 4.

| Num Collisions | Time
---|---|---
| F | Signif. | F | Signif.
device | 4.7538 | 8.85E-03 | 7.2265 | 2.51E-11
$\rho$ | 9.2904 | 1.02E-04 | 25.1790 | 2.00E-04
$\lambda$ | 4.8301 | 8.21E-03 | 19.8199 | 8.42E-33
display | 207.3125 | 3.27E-42 | 335.6442 | 2.24E-01
device : $\rho$ | 1.2885 | 2.73E-01 | 3.6222 | 7.09E-01
device : $\lambda$ | 1.2039 | 3.08E-01 | 3.5733 | 1.67E-04
$\rho$ : $\lambda$ | 0.2015 | 9.38E-01 | 1.1654 | 6.94E-01
device : display | 1.3595 | 2.57E-01 | 6.5119 | 2.55E-01
$\rho$ : display | 0.3381 | 7.13E-01 | 0.9380 | 5.72E-01
$\lambda$ : display | 3.8149 | 2.24E-02 | 9.0722 | 6.27E-03
device : $\rho$ : $\lambda$ | 1.0423 | 4.02E-01 | 3.1542 | 1.38E-02
device : $\rho$ : display | 0.9071 | 4.59E-01 | 1.9217 | 8.71E-01
device : $\lambda$ : display | 0.6814 | 6.05E-01 | 1.2380 | 9.63E-01
$\rho$ : $\lambda$ : display | 1.5045 | 1.99E-01 | 2.0618 | 6.25E-01
device : $\rho$ : $\lambda$ : disp | 0.9815 | 4.49E-01 | 2.2511 | 7.28E-01
Table 4. Obstacle avoidance task: F-value table for Aligned Rank Transform
(ART) ANOVA. Values were calculated with the ARTool software package. “device”
refers to the three simulated electrode grids, while “display”refers to the
use of an HMD or monitor.

Contrary to our expectations, neither the number of electrodes (Fig. 6A) nor
phosphene size (Fig. 6B) had a significant effect on the number of collisions.
Although the number of collisions decreased slightly with higher electrode
counts (Table 3), this did not reach statistical significance. The only
statistical differences could be found between the scoreboard model
($\lambda$=$50\text{\,}\mathrm{\SIUnitSymbolMicro m}$) and axon map models
($\lambda$={100, 300}$\text{\,}\mathrm{\SIUnitSymbolMicro m}$) for the HMD-
based version of the task. However, participants performed around chance
levels in all tested conditions.

The time analysis revealed a downward trend in time (better performance) with
higher electrode counts, but only among the groupings in the monitor version.
This trend in time reached significance for all comparisons within the monitor
version (Fig. 6D). Similarly to comparisons across groupings of $\rho$ values,
there was a slight downward trend across the median time taken as phosphene
distortion increased (Fig. 6, _F_).

A comparison between the two different versions of the task showed a clear
difference in performance, with performance for the HMD version being
drastically higher than the monitor version of the task. This trend reached
significance across any grouping of device, $\rho$, or $\lambda$ (Fig. 6,
_Top_). There was also a difference in time taken between the versions of the
task, with the HMD version taking longer for all groupings (Fig. 6, _Bottom_).

##  5\. Discussion

###  5.1. Using an HMD May Benefit Behavioral Performance

The present study provides the first side-by-side comparison between HMD and
monitor versions of different behavioral tasks using SPV. Importantly, we used
a psychophysically validated SPV model to explore the expected behavioral
performance of bionic eye users, for current as well as potential near-future
devices, and found that participants performed significantly better in the HMD
version than the monitor version for both tasks.

In the letter recognition task, participants achieved a higher mean F1 score
across all conditions (Table 1). However, this trend was only significant for
the hypothetical future devices and smaller phosphene sizes and elongations
(Fig. 4, _Top_). While average time per trial was faster across all conditions
with the HMD, the effect was not significant (Fig. 3, _Bottom_).

The difference in performance was even more evident in the obstacle avoidance
task, where performance (as measured by number of collisions) for the HMD
version was significantly higher than the monitor version across all
conditions (Fig. 6, _Top_). It is also worth pointing out that participants
were able to complete the task faster with higher electrode counts in the
monitor-based version of the task. Since the walking speed was fixed across
all conditions, this likely indicates that the task was easier with higher
electrode counts.

Overall these results suggest that participants were able to benefit from
vestibular and proprioceptive cues provided by head movements and locomotion
during the HMD version of the task, which is something that is available to
real bionic eye users but cannot be replicated by a mouse and keyboard.

###  5.2. Increased Phosphene Elongation May Impede Performance

Whereas previous studies treated phosphenes as small, discrete light sources,
here we systematically evaluated perceptual performance across a wide range of
common phosphene sizes ($\rho$) and elongations ($\lambda$). As expected,
participants performed best when phosphenes were circular (scoreboard model:
$\lambda=$50\text{\,}\mathrm{\SIUnitSymbolMicro m}$$; Tables 1 and 3), and
increasing phosphene elongation ($\lambda$) negatively affected performance.

In the letter recognition task, participants using the scoreboard model
($\lambda$=$50\text{\,}\mathrm{\SIUnitSymbolMicro m}$) achieved a perfect
median F1 score of 1.0 (Fig. 4C), which is much better than the behavioral
metrics reported with real Argus II patients (Cruz et al., 2013). Conversely,
performance approached chance levels when increasing $\lambda$ to
$5000\text{\,}\mathrm{\SIUnitSymbolMicro m}$.

In the obstacle avoidance task, the only significant findings within one
version of the experiment were between the scoreboard model
($\lambda=$50\text{\,}\mathrm{\SIUnitSymbolMicro m}$$) and either of the
larger $\lambda$ values. This suggests that elongated phosphenes make obstacle
avoidance more challenging than the scoreboard model. However, participants
performed around chance levels in all tested conditions, which was also true
for real Argus II patients (He et al., 2020).

Contrary to our expectations, phosphene size ($\rho$) did not systematically
affect performance (Fig. 4B, Fig. 6B). The best performance was typically
achieved with $\rho=$300\text{\,}\mathrm{\SIUnitSymbolMicro m}$$. This is in
contrast to previous literature suggesting smaller phosphene size is directly
correlated with higher visual acuity (Chen et al., 2009; Han et al., 2021)

Overall these findings suggest that behavioral performance may vary
drastically depending on the choices of $\rho$ and $\lambda$. This is
important for predicting visual outcomes, because $\rho$ and $\lambda$ have
been shown to vary drastically across bionic eye users (Beyeler et al.,
2019b), suggesting future work should seek to use psychophysically validated
SPV models when making theoretical predictions about device performance.

###  5.3. Increasing the Number of Electrodes Does Not Necessarily Improve
Performance

As expected, letter recognition performance improved as the size of the
electrode grid (and therefore the FOV) was increased from $6\times 10$ to
$10\times 16$ and $19\times 31$ (Fig. 4A). This performance benefit was also
observed in the time it took participants to recognize the letter (Fig. 4D),
and is consistent with previous literature on object recognition (Thorn et
al., 2020).

However, electrode count did not affect behavioral performance in the obstacle
avoidance task. Whereas there was a slight increase in performance scores for
devices with more electrodes (Fig. 6A), this effect did not reach
significance.

Overall these results are consistent with previous literature suggesting that,
for most tasks, the number of electrodes may not be the limiting factor in
retinal implants (Beyeler et al., 2017b; Behrend et al., 2011).

###  5.4. Limitations and Future Work

Although the present study addressed previously unanswered questions about
SPV, there are a number of limitations that should be addressed in future work
as outlined below.

First, in an effort to focus on the impact of phosphene size and elongation on
perceptual performance, we limited ourselves to modeling spatial distortions.
However, retinal implants are known for causing temporal distortions as well,
such as flicker and fading, which may further limit the perceptual performance
of participants (Beyeler et al., 2017b).

Second, the displayed stimuli were not contingent on the user’s eye movements.
Even though current retinal implants ignore eye movements as well, there is a
not-so-subtle difference between a real retinal implant and a simulated one.
Since the real device is implanted on the retinal surface, it will always
stimulate the same neurons, and thus produce vision in the same location in
the visual field—no matter the eye position. This can be very disorienting for
a real patient as shifting your gaze to the left would not shift the vision
generated by the implant. In contrast, a participant in a VR study is free to
explore the presented visual stimuli with their gaze, thus artificially
increasing the FOV from that offered by the simulated device. Consequently,
the here presented performance predictions may still be too optimistic. In the
future, simulations should make use of eye tracking technologies to update the
scene in a gaze-contingent way.

Third, we did not explicitly measure the level of immersion across the two
display types (HMD and monitor). Instead, we assumed that viewing a scene that
updates with the user’s head movement through an HMD would lead to a higher
level of immersion. Although this may be true for realistic virtual
environments (Miller and Bugnariu, 2016), this has yet to be demonstrated for
SPV studies. Future SPV work should therefore explicitly measure the level of
immersion and/or a user’s sense of presence.

Fourth, the obstacle avoidance task did not have a meaningful time metric.
Although participants performed the task significantly faster in the monitor-
based version, this is likely an artifact due to the walking speed of
participants not being consistent between versions of the task. Participants
moved much slower with the HMD as they were not able to see the real world
around them. Future studies should take this into consideration and correct
for each participant’s walking speed within desktop versions of tasks.

Fifth, the study was performed on sighted graduate students readily available
at the University of California, Santa Barbara. Their age, navigational
affordances, and experience with low vision may therefore be drastically
different from real bionic eye users, who tend to not only be older and
prolific cane users but also receive extensive vision rehabilitation training.

Interestingly, we found vast individual differences across the two tasks
(individual data points in Figs. 4 and 6) which were not unlike those reported
in the literature (Cruz et al., 2013; He et al., 2020). Subjects who did well
in one experiment tended to do well across all versions of both experiments
(data not shown), suggesting that some people were inherently better at
adapting to prosthetic vision than others. Future work should therefore zero
in on the possible causes of these individual differences and compare them to
real bionic eye users. Studying these differences could identify training
protocols to enhance the ability of all device users.

##  6\. Conclusions

The present work constitutes a first essential step towards immersive VR
simulations of bionic vision. Data from two behavioral experiments demonstrate
the importance of choosing an appropriate level of immersion and phosphene
model complexity. The VR-SPV toolbox that enabled these experiments is freely
available at https://github.com/bionicvisionlab/BionicVisionXR and designed to
be extendable to a variety of bionic eye technologies. Overall this work has
the potential to further our understanding of the qualitative experience
associated with different bionic eye technologies and provide realistic
expectations of prosthetic performance.

###### Acknowledgements.

This work was supported by the National Institutes of Health (NIH R00
EY-029329 to MB).

## References

  * (1)
  * Ayton et al. (2020) Lauren N. Ayton, Nick Barnes, Gislin Dagnelie, Takashi Fujikado, Georges Goetz, Ralf Hornig, Bryan W. Jones, Mahiul M. K. Muqit, Daniel L. Rathbun, Katarina Stingl, James D. Weiland, and Matthew A. Petoe. 2020.  An update on retinal prostheses.  _Clinical Neurophysiology_ 131, 6 (June 2020), 1383–1398.  https://doi.org/10.1016/j.clinph.2019.11.029
  * Ayton et al. (2014) Lauren N. Ayton, Peter J. Blamey, Robyn H. Guymer, Chi D. Luu, David A. X. Nayagam, Nicholas C. Sinclair, Mohit N. Shivdasani, Jonathan Yeoh, Mark F. McCombe, Robert J. Briggs, Nicholas L. Opie, Joel Villalobos, Peter N. Dimitrov, Mary Varsamidis, Matthew A. Petoe, Chris D. McCarthy, Janine G. Walker, Nick Barnes, Anthony N. Burkitt, Chris E. Williams, Robert K. Shepherd, Penelope J. Allen, and for the Bionic Vision Australia Research Consortium. 2014\.  First-in-Human Trial of a Novel Suprachoroidal Retinal Prosthesis.  _PLOS ONE_ 9, 12 (Dec. 2014), e115239.  https://doi.org/10.1371/journal.pone.0115239 Publisher: Public Library of Science.
  * Behrend et al. (2011) Matthew R. Behrend, Ashish K. Ahuja, Mark S. Humayun, Robert H. Chow, and James D. Weiland. 2011\.  Resolution of the Epiretinal Prosthesis is not Limited by Electrode Size.  _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ 19, 4 (Aug. 2011), 436–442.  https://doi.org/10.1109/TNSRE.2011.2140132
  * Beyeler et al. (2017a) M. Beyeler, G. M. Boynton, I. Fine, and A. Rokem. 2017a.  pulse2percept: A Python-based simulation framework for bionic vision. In _Proceedings of the 16th Science in Python Conference_ , K. Huff, D. Lippa, D. Niederhut, and M. Pacer (Eds.). 81–88.  https://doi.org/10.25080/shinma-7f4c6e7-00c
  * Beyeler et al. (2019a) Michael Beyeler, Geoffrey M. Boynton, Ione Fine, and Ariel Rokem. 2019a.  Model-Based Recommendations for Optimal Surgical Placement of Epiretinal Implants. In _Medical Image Computing and Computer Assisted Intervention – MICCAI 2019_ _(Lecture Notes in Computer Science)_ , Dinggang Shen, Tianming Liu, Terry M. Peters, Lawrence H. Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap, and Ali Khan (Eds.). Springer International Publishing, 394–402.
  * Beyeler et al. (2019b) Michael Beyeler, Devyani Nanduri, James D. Weiland, Ariel Rokem, Geoffrey M. Boynton, and Ione Fine. 2019b.  A model of ganglion axon pathways accounts for percepts elicited by retinal implants.  _Scientific Reports_ 9, 1 (June 2019), 1–16.  https://doi.org/10.1038/s41598-019-45416-4
  * Beyeler et al. (2017b) Michael Beyeler, Ariel Rokem, Geoffrey M. Boynton, and Ione Fine. 2017b.  Learning to see again: biological constraints on cortical plasticity and the implications for sight restoration technologies.  _Journal of Neural Engineering_ 14, 5 (Aug. 2017), 051003.  https://doi.org/10.1088/1741-2552/aa795e Publisher: IOP Publishing.
  * Bourne et al. (2017) Rupert R. A. Bourne, Seth R. Flaxman, Tasanee Braithwaite, Maria V. Cicinelli, Aditi Das, Jost B. Jonas, Jill Keeffe, John H. Kempen, Janet Leasher, Hans Limburg, Kovin Naidoo, Konrad Pesudovs, Serge Resnikoff, Alex Silvester, Gretchen A. Stevens, Nina Tahhan, Tien Y. Wong, Hugh R. Taylor, Rupert Bourne, Peter Ackland, Aries Arditi, Yaniv Barkana, Banu Bozkurt, Tasanee Braithwaite, Alain Bron, Donald Budenz, Feng Cai, Robert Casson, Usha Chakravarthy, Jaewan Choi, Maria Vittoria Cicinelli, Nathan Congdon, Reza Dana, Rakhi Dandona, Lalit Dandona, Aditi Das, Iva Dekaris, Monte Del Monte, Jenny Deva, Laura Dreer, Leon Ellwein, Marcela Frazier, Kevin Frick, David Friedman, Joao Furtado, Hua Gao, Gus Gazzard, Ronnie George, Stephen Gichuhi, Victor Gonzalez, Billy Hammond, Mary Elizabeth Hartnett, Minguang He, James Hejtmancik, Flavio Hirai, John Huang, April Ingram, Jonathan Javitt, Jost Jonas, Charlotte Joslin, Jill Keeffe, John Kempen, Moncef Khairallah, Rohit Khanna, Judy Kim, George Lambrou, Van Charles Lansingh, Paolo Lanzetta, Janet Leasher, Jennifer Lim, Hans Limburg, Kaweh Mansouri, Anu Mathew, Alan Morse, Beatriz Munoz, David Musch, Kovin Naidoo, Vinay Nangia, Maria Palaiou, Maurizio Battaglia Parodi, Fernando Yaacov Pena, Konrad Pesudovs, Tunde Peto, Harry Quigley, Murugesan Raju, Pradeep Ramulu, Serge Resnikoff, Alan Robin, Luca Rossetti, Jinan Saaddine, Mya Sandar, Janet Serle, Tueng Shen, Rajesh Shetty, Pamela Sieving, Juan Carlos Silva, Alex Silvester, Rita S. Sitorus, Dwight Stambolian, Gretchen Stevens, Hugh Taylor, Jaime Tejedor, James Tielsch, Miltiadis Tsilimbaris, Jan van Meurs, Rohit Varma, Gianni Virgili, Jimmy Volmink, Ya Xing Wang, Ning-Li Wang, Sheila West, Peter Wiedemann, Tien Wong, Richard Wormald, and Yingfeng Zheng. 2017\.  Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis.  _The Lancet Global Health_ 5, 9 (Sept. 2017), e888–e897.  https://doi.org/10.1016/S2214-109X(17)30293-0 Publisher: Elsevier.
  * Cao et al. (2017) Xiaofei Cao, Heng Li, Zhuofan Lu, Xinyu Chai, and Jing Wang. 2017.  Eye-hand coordination using two irregular phosphene maps in simulated prosthetic vision for retinal prostheses. In _2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI)_. 1–5.  https://doi.org/10.1109/CISP-BMEI.2017.8302244
  * Caspi and Zivotofsky (2015) Avi Caspi and Ari Z. Zivotofsky. 2015.  Assessing the utility of visual acuity measures in visual prostheses.  _Vision Research_ 108 (March 2015), 77–84.  https://doi.org/10.1016/j.visres.2015.01.006
  * Chang et al. (2012) M. H. Chang, H. S. Kim, J. H. Shin, and K. S. Park. 2012\.  Facial identification in very low-resolution images simulating prosthetic vision.  _Journal of Neural Engineering_ 9, 4 (July 2012), 046012.  https://doi.org/10.1088/1741-2560/9/4/046012 Publisher: IOP Publishing.
  * Chen et al. (2009) S. C. Chen, G. J. Suaning, J. W. Morley, and N. H. Lovell. 2009.  Simulating prosthetic vision: I. Visual models of phosphenes.  _Vision Research_ 49, 12 (June 2009), 1493–506.
  * Cruz et al. (2013) Lyndon da Cruz, Brian F. Coley, Jessy Dorn, Francesco Merlini, Eugene Filley, Punita Christopher, Fred K. Chen, Varalakshmi Wuyyuru, Jose Sahel, Paulo Stanga, Mark Humayun, Robert J. Greenberg, Gislin Dagnelie, and for the Argus II Study Group. 2013.  The Argus II epiretinal prosthesis system allows letter and word reading and long-term function in patients with profound vision loss.  _British Journal of Ophthalmology_ 97, 5 (May 2013), 632–636.  https://doi.org/10.1136/bjophthalmol-2012-301525 Publisher: BMJ Publishing Group Ltd Section: Clinical science.
  * Denis et al. (2013) Grégoire Denis, Christophe Jouffrais, Victor Vergnieux, and Marc Macé. 2013.  Human faces detection and localization with simulated prosthetic vision. In _CHI ’13 Extended Abstracts on Human Factors in Computing Systems_ _(CHI EA ’13)_. Association for Computing Machinery, New York, NY, USA, 61–66.  https://doi.org/10.1145/2468356.2468368
  * Elkin et al. (2021) Lisa A. Elkin, Matthew Kay, James J. Higgins, and Jacob O. Wobbrock. 2021.  An Aligned Rank Transform Procedure for Multifactor Contrast Tests.  In _The 34th Annual ACM Symposium on User Interface Software and Technology_. Association for Computing Machinery, New York, NY, USA, 754–768.  https://doi.org/10.1145/3472749.3474784
  * Erickson-Davis and Korzybska (2021) Cordelia Erickson-Davis and Helma Korzybska. 2021.  What do blind people “see” with retinal prostheses? Observations and qualitative reports of epiretinal implant users.  _PLOS ONE_ 16, 2 (Feb. 2021), e0229189.  https://doi.org/10.1371/journal.pone.0229189 Publisher: Public Library of Science.
  * Ferlauto et al. (2018) Laura Ferlauto, Marta Jole Ildelfonsa Airaghi Leccardi, Naïg Aurelia Ludmilla Chenais, Samuel Charles Antoine Gilliéron, Paola Vagni, Michele Bevilacqua, Thomas J. Wolfensberger, Kevin Sivula, and Diego Ghezzi. 2018.  Design and validation of a foldable and photovoltaic wide-field epiretinal prosthesis.  _Nature Communications_ 9, 1 (March 2018), 1–15.  https://doi.org/10.1038/s41467-018-03386-7
  * Fernandez (2018) Eduardo Fernandez. 2018\.  Development of visual Neuroprostheses: trends and challenges.  _Bioelectronic Medicine_ 4, 1 (Aug. 2018), 12\.  https://doi.org/10.1186/s42234-018-0013-8
  * Fine and Boynton (2015) I. Fine and G. M. Boynton. 2015.  Pulse trains to percepts: the challenge of creating a perceptually intelligible world with sight recovery technologies.  _Philos Trans R Soc Lond B Biol Sci_ 370, 1677 (Sept. 2015), 20140208.  https://doi.org/10.1098/rstb.2014.0208
  * Guo et al. (2014) Xiaoli Guo, Zheng Jin, Xinyang Feng, and Shanbao Tong. 2014\.  Enhanced Effective Connectivity in Mild Occipital Stroke Patients With Hemianopia.  _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ 22, 6 (Nov. 2014), 1210–1217.  https://doi.org/10.1109/TNSRE.2014.2325601 Conference Name: IEEE Transactions on Neural Systems and Rehabilitation Engineering.
  * Han et al. (2021) Nicole Han, Sudhanshu Srivastava, Aiwen Xu, Devi Klein, and Michael Beyeler. 2021.  Deep Learning–Based Scene Simplification for Bionic Vision. In _Augmented Humans Conference 2021_. ACM, Rovaniemi Finland, 45–54.  https://doi.org/10.1145/3458709.3458982
  * He et al. (2020) Yingchen He, Susan Y. Sun, Arup Roy, Avi Caspi, and Sandra R. Montezuma. 2020.  Improved mobility performance with an artificial vision therapy system using a thermal sensor.  _Journal of Neural Engineering_ 17, 4 (Aug. 2020), 045011.  https://doi.org/10.1088/1741-2552/aba4fb Publisher: IOP Publishing.
  * Jansonius et al. (2012) Nomdo M. Jansonius, Julia Schiefer, Jukka Nevalainen, Jens Paetzold, and Ulrich Schiefer. 2012\.  A mathematical model for describing the retinal nerve fiber bundle trajectories in the human eye: Average course, variability, and influence of refraction, optic disc size and optic disc position.  _Experimental Eye Research_ 105 (Dec. 2012), 70–78.  https://doi.org/10.1016/j.exer.2012.10.008
  * Josh et al. (2013) Horace Josh, Collette Mann, Lindsay Kleeman, and Wen Lik Dennis Lui. 2013.  Psychophysics testing of bionic vision image processing algorithms using an FPGA Hatpack. In _2013 IEEE International Conference on Image Processing_. 1550–1554.  https://doi.org/10.1109/ICIP.2013.6738319 ISSN: 2381-8549.
  * Kardong-Edgren et al. (2019) Suzan (Suzie) Kardong-Edgren, Sharon L. Farra, Guillaume Alinier, and H. Michael Young. 2019.  A Call to Unify Definitions of Virtual Reality.  _Clinical Simulation in Nursing_ 31 (June 2019), 28–34.  https://doi.org/10.1016/j.ecns.2019.02.006
  * Kasowski et al. (2021a) Justin Kasowski, Byron A. Johnson, Ryan Neydavood, Anvitha Akkaraju, and Michael Beyeler. 2021a.  Furthering Visual Accessibility with Extended Reality (XR): A Systematic Review.  _arXiv:2109.04995 [cs]_ (Sept. 2021).  http://arxiv.org/abs/2109.04995 arXiv: 2109.04995.
  * Kasowski et al. (2021b) Justin Kasowski, Nathan Wu, and Michael Beyeler. 2021b.  Towards Immersive Virtual Reality Simulations of Bionic Vision. In _Augmented Humans Conference 2021_. ACM, Rovaniemi Finland, 313–315.  https://doi.org/10.1145/3458709.3459003
  * Kvansakul et al. (2020) Jessica Kvansakul, Lachlan Hamilton, Lauren N. Ayton, Chris McCarthy, and Matthew A. Petoe. 2020\.  Sensory augmentation to aid training with retinal prostheses.  _Journal of Neural Engineering_ 17, 4 (July 2020), 045001.  https://doi.org/10.1088/1741-2552/ab9e1d
  * Lorach et al. (2015) H. Lorach, G. Goetz, R. Smith, X. Lei, Y. Mandel, T. Kamins, K. Mathieson, P. Huie, J. Harris, A. Sher, and D. Palanker. 2015.  Photovoltaic restoration of sight with high visual acuity.  _Nat Med_ 21, 5 (May 2015), 476–82.  https://doi.org/10.1038/nm.3851
  * Lu et al. (2012) Yanyu Lu, Panpan Chen, Ying Zhao, Jingru Shi, Qiushi Ren, and Xinyu Chai. 2012\.  Estimation of Simulated Phosphene Size Based on Tactile Perception.  _Artificial Organs_ 36, 1 (2012), 115–120.  https://doi.org/10.1111/j.1525-1594.2011.01288.x _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1525-1594.2011.01288.x.
  * Luo and da Cruz (2016) Yvonne Hsu-Lin Luo and Lyndon da Cruz. 2016.  The Argus® II Retinal Prosthesis System.  _Progress in Retinal and Eye Research_ 50 (Jan. 2016), 89–107.  https://doi.org/10.1016/j.preteyeres.2015.09.003
  * Luo et al. (2016) Yvonne H-L. Luo, Joe Jiangjian Zhong, Monica Clemo, and Lyndon da Cruz. 2016.  Long-term Repeatability and Reproducibility of Phosphene Characteristics in Chronically Implanted Argus II Retinal Prosthesis Subjects.  _American Journal of Ophthalmology_ 170 (Oct. 2016), 100–109.  https://doi.org/10.1016/j.ajo.2016.07.021
  * Macé et al. (2015) Marc J.-M. Macé, Valérian Guivarch, Grégoire Denis, and Christophe Jouffrais. 2015\.  Simulated Prosthetic Vision: The Benefits of Computer-Based Object Recognition and Localization.  _Artificial Organs_ 39, 7 (2015), E102–E113.  https://doi.org/10.1111/aor.12476
  * Miller and Bugnariu (2016) Haylie L. Miller and Nicoleta L. Bugnariu. 2016.  Level of Immersion in Virtual Environments Impacts the Ability to Assess and Teach Social Skills in Autism Spectrum Disorder.  _Cyberpsychology, Behavior, and Social Networking_ 19, 4 (April 2016), 246–256.  https://doi.org/10.1089/cyber.2014.0682 Publisher: Mary Ann Liebert, Inc., publishers.
  * Nanduri et al. (2012) Devyani Nanduri, Ione Fine, Alan Horsager, Geoffrey M. Boynton, Mark S. Humayun, Robert J. Greenberg, and James D. Weiland. 2012\.  Frequency and Amplitude Modulation Have Different Effects on the Percepts Elicited by Retinal Stimulation.  _Investigative Ophthalmology & Visual Science_ 53, 1 (Jan. 2012), 205–214.  https://doi.org/10.1167/iovs.11-8401
  * Pasch et al. (2009) Marco Pasch, Nadia Bianchi-Berthouze, Betsy van Dijk, and Anton Nijholt. 2009.  Immersion in Movement-Based Interaction.  In _Intelligent Technologies for Interactive Entertainment_ , Anton Nijholt, Dennis Reidsma, and Hendri Hondorp (Eds.). Vol. 9. Springer Berlin Heidelberg, Berlin, Heidelberg, 169–180.  https://doi.org/10.1007/978-3-642-02315-6_16 Series Title: Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.
  * Pedregosa et al. (2011) F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011.  Scikit-learn: Machine Learning in Python.  _Journal of Machine Learning Research_ 12 (2011), 2825–2830.
  * Perez Fornos et al. (2011) Angelica Perez Fornos, Jörg Sommerhalder, and Marco Pelizzone. 2011.  Reading with a Simulated 60-Channel Implant.  _Frontiers in Neuroscience_ 5 (2011), 57.  https://doi.org/10.3389/fnins.2011.00057
  * Perez-Yus et al. (2017) Alejandro Perez-Yus, Jesus Bermudez-Cameo, Jose J. Guerrero, and Gonzalo Lopez-Nicolas. 2017.  Depth and Motion Cues with Phosphene Patterns for Prosthetic Vision. In _2017 IEEE International Conference on Computer Vision Workshops (ICCVW)_. 1516–1525.  https://doi.org/10.1109/ICCVW.2017.179 ISSN: 2473-9944.
  * Rizzo et al. (2003) Joseph F. Rizzo, John Wyatt, John Loewenstein, Shawn Kelly, and Doug Shire. 2003.  Perceptual Efficacy of Electrical Stimulation of Human Retina with a Microelectrode Array during Short-Term Surgical Trials.  _Investigative Ophthalmology & Visual Science_ 44, 12 (Dec. 2003), 5362–5369.  https://doi.org/10.1167/iovs.02-0817 Publisher: The Association for Research in Vision and Ophthalmology.
  * Sanchez-Garcia et al. (2020) Melani Sanchez-Garcia, Ruben Martinez-Cantin, Jesus Bermudez-Cameo, and Jose J. Guerrero. 2020\.  Influence of field of view in visual prostheses design: Analysis with a VR system.  _Journal of Neural Engineering_ 17, 5 (Oct. 2020), 056002.  https://doi.org/10.1088/1741-2552/abb9be Publisher: IOP Publishing.
  * Sanchez-Garcia et al. (2019) Melani Sanchez-Garcia, Ruben Martinez-Cantin, and Josechu J. Guerrero. 2019.  Indoor Scenes Understanding for Visual Prosthesis with Fully Convolutional Networks. In _VISIGRAPP_.  https://doi.org/10.5220/0007257602180225
  * Stingl et al. (2013) K. Stingl, K. U. Bartz-Schmidt, D. Besch, A. Braun, A. Bruckmann, F. Gekeler, U. Greppmaier, S. Hipp, G. Hortdorfer, C. Kernstock, A. Koitschev, A. Kusnyerik, H. Sachs, A. Schatz, K. T. Stingl, T. Peters, B. Wilhelm, and E. Zrenner. 2013\.  Artificial vision with wirelessly powered subretinal electronic implant alpha-IMS.  _Proc Biol Sci_ 280, 1757 (April 2013), 20130077\.  https://doi.org/10.1098/rspb.2013.0077
  * Subramaniam et al. (2012) Mahadevan Subramaniam, Parvathi Chundi, Abhilash Muthuraj, Eyal Margalit, and Sylvie Sim. 2012\.  Simulating prosthetic vision with disortions for retinal prosthesis design. In _Proceedings of the 2012 international workshop on Smart health and wellbeing_ _(SHB ’12)_. Association for Computing Machinery, New York, NY, USA, 57–64.  https://doi.org/10.1145/2389707.2389719
  * Thorn et al. (2020) Jacob Thomas Thorn, Enrico Migliorini, and Diego Ghezzi. 2020\.  Virtual reality simulation of epiretinal stimulation highlights the relevance of the visual angle in prosthetic vision.  _Journal of Neural Engineering_ 17, 5 (Nov. 2020), 056019.  https://doi.org/10.1088/1741-2552/abb5bc Publisher: IOP Publishing.
  * Titchener et al. (2018) Samuel A. Titchener, Mohit N. Shivdasani, James B. Fallon, and Matthew A. Petoe. 2018.  Gaze Compensation as a Technique for Improving Hand–Eye Coordination in Prosthetic Vision.  _Translational Vision Science & Technology_ 7, 1 (Jan. 2018), 2.  https://doi.org/10.1167/tvst.7.1.2
  * Tukey (1949) John W. Tukey. 1949\.  Comparing Individual Means in the Analysis of Variance.  _Biometrics_ 5, 2 (1949), 99–114.  https://doi.org/10.2307/3001913 Publisher: [Wiley, International Biometric Society].
  * Vergnieux et al. (2014) Victor Vergnieux, Marc J.-M. Macé, and Christophe Jouffrais. 2014\.  Wayfinding with simulated prosthetic vision: Performance comparison with regular and structure-enhanced renderings. In _2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society_. 2585–2588.  https://doi.org/10.1109/EMBC.2014.6944151 ISSN: 1558-4615.
  * Vurro et al. (2014) Milena Vurro, Anne Marie Crowell, and John S. Pezaris. 2014\.  Simulation of thalamic prosthetic vision: reading accuracy, speed, and acuity in sighted humans.  _Frontiers in Human Neuroscience_ 8 (2014), 816.  https://doi.org/10.3389/fnhum.2014.00816
  * Wang et al. (2018) Lihui Wang, Fariba Sharifian, Jonathan Napp, Carola Nath, and Stefan Pollmann. 2018.  Cross-task perceptual learning of object recognition in simulated retinal implant perception.  _Journal of Vision_ 18, 13 (Dec. 2018), 22\.  https://doi.org/10.1167/18.13.22
  * Wobbrock et al. (2011) Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011.  The aligned rank transform for nonparametric factorial analyses using only anova procedures. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_. ACM, Vancouver BC Canada, 143–146.  https://doi.org/10.1145/1978942.1978963
  * Wu et al. (2014) Hao Wu, Jing Wang, Heng Li, and Xinyu Chai. 2014\.  Prosthetic vision simulating system and its application based on retinal prosthesis. In _2014 International Conference on Information Science, Electronics and Electrical Engineering_ , Vol. 1. 425–429.  https://doi.org/10.1109/InfoSEEE.2014.6948145
  * Ying et al. (2018) Zhao Ying, Geng Xiulin, Li Qi, and Jiang Guangqi. 2018\.  Recognition of virtual maze scene under simulated prosthetic vision. In _2018 Tenth International Conference on Advanced Computational Intelligence (ICACI)_. 1–5.  https://doi.org/10.1109/ICACI.2018.8377543
  * Zapf et al. (2014) Marc P. Zapf, Paul B. Matteucci, Nigel H. Lovell, Steven Zheng, and Gregg J. Suaning. 2014\.  Towards photorealistic and immersive virtual-reality environments for simulated prosthetic vision: Integrating recent breakthroughs in consumer hardware and software. In _2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society_. 2597–2600.  https://doi.org/10.1109/EMBC.2014.6944154 ISSN: 1558-4615.
  * Zhao et al. (2010) Ying Zhao, Yanyu Lu, Yukun Tian, Liming Li, Qiushi Ren, and Xinyu Chai. 2010\.  Image processing based recognition of images with a limited number of pixels using simulated prosthetic vision.  _Information Sciences_ 180, 16 (Aug. 2010), 2915–2924.  https://doi.org/10.1016/j.ins.2010.04.021
  * Zhao et al. (2011) Ying Zhao, Yanyu Lu, Ji Zhao, Kaihu Wang, Qiushi Ren, Kaijie Wu, and Xinyu Chai. 2011.  Reading Pixelized Paragraphs of Chinese Characters Using Simulated Prosthetic Vision.  _Investigative Ophthalmology & Visual Science_ 52, 8 (July 2011), 5987–5994.  https://doi.org/10.1167/iovs.10-5293

  *[SPV]: simulated prosthetic vision
  *[VR]: virtual reality
  *[HMD]: head-mounted display
  *[FOV]: field of view
  *[NFBs]: nerve fiber bundle
  *[NFB]: nerve fiber bundle
  *[HMDs]: head-mounted display
  *[AR]: augmented reality
  *[HCI]: human-computer interaction
  *[CPU]: central processing unit
  *[GPU]: graphics processing unit