# Move or Push? Studying Pseudo-Haptic Perceptions Obtained with Motion or
Force Input

Yutaro Hirao, Takuji Narumi, Ferran Argelaguet, and Anatole Lécuyer  Y. Hirao
is with Nara Institute of Science and Technology (NAIST), Nara 630-0192,
Japan. E-mail<EMAIL_ADDRESS>T. Narumi is with the University of
Tokyo, Tokyo 113-8654, Japan. E-mail<EMAIL_ADDRESS>A. Lécuyer
and F. Argelaguet work with Univ. Rennes, Inria, IRISA, CNRS, Rennes, France.

###### Abstract

Pseudo-haptics techniques are interesting alternatives for generating haptic
perceptions, which entails the manipulation of haptic perception through the
appropriate alteration of primarily visual feedback in response to body
movements. However, the use of pseudo-haptics techniques with a motion-input
system can sometimes be limited. This paper investigates a novel approach for
extending the potential of pseudo-haptics techniques in virtual reality (VR).
The proposed approach utilizes a reaction force from force-input as a
substitution of haptic cue for the pseudo-haptic perception. The paper
introduced a manipulation method in which the vertical acceleration of the
virtual hand is controlled by the extent of push-in of a force sensor. Such a
force-input manipulation of a virtual body can not only present pseudo-haptics
with less physical spaces and be used by more various users including
physically handicapped people, but also can present the reaction force
proportional to the user’s input to the user. We hypothesized that such a
haptic force cue would contribute to the pseudo-haptic perception. Therefore,
the paper endeavors to investigate the force-input pseudo-haptic perception in
a comparison with the motion-input pseudo-haptics. The paper compared force-
input and motion-input manipulation in a point of achievable range and
resolution of pseudo-haptic weight. The experimental results suggest that the
force-input manipulation successfully extends the range of perceptible pseudo-
weight by 80% in comparison to the motion-input manipulation. On the other
hand, it is revealed that the motion-input manipulation has 1 step larger
number of distinguishable weight levels and is easier to operate than the
force-input manipulation.

###### Index Terms:

Pseudo-Haptics, virtual reality, sensory substitution, cross-modal integration

##  1 Introduction

In recent years, significant advancements have been made in virtual reality
(VR) technology, allowing users to experience a sense of actual presence
through visual and audio feedback. However, compared to audio-visual
information, haptic feedback still possesses a narrower range of expression.
To address this limitation, various approaches have been proposed to present
haptic feedback. One intriguing method is the employment of pseudo-haptics
techniques, which modify haptic perception by appropriately adjusting visual
feedback in response to body movements [1, 2]. Examples of pseudo-haptics
techniques include modifying the virtual force of a spring [3] or the
perceived weight of an object [4, 5, 6] by altering the control-display gain
according to the user’s motion within a virtual environment. The main
advantage of the pseudo-haptics technique is that it primarily relies on
visual stimuli to convey haptic perceptions, eliminating the need for bulky
haptic devices.

Traditionally, the pseudo-haptics technique for VR interaction involves the
use of motion-input manipulation, where the spatial positions of the physical
body are correlated with their virtual counterparts. However, There are
several limitations with pseudo-haptics techniques with motion-input system.
For example, a conventional pseudo-haptics technique reduces the virtual
motion compared to the physical motion to present a stronger force perception
and therefore, it requires larger physical motion and spaces. Moreover,
because motion-input system in pseudo-haptics studies has morphologically
equivalent input-output body mapping, it cannot be used by users with
disabilities that prevent performing the required virtual motions. In addition
to them, even when motion gains are applied and visual cues are altered,
because there is no haptic feedback, the haptic cues remain unchanged from the
absence of motion gains. This leads to a mismatch between the haptic estimates
(e.g, width or weight of objects) derived from each sensory cue. Here, it is
said that such separation between the estimates from each sensory cue can lead
to discomfort or disrupt haptic perception when the disparities are too
significant [1, 5, 7].

To address the above problems, Hirao et al. examined pseudo-haptics with a
joystick-based avatar manipulation and confirmed that it can present pseudo-
haptic weight perception with a similar level of the one of motion-input
system [8]. Such a morphologically incongruent avatar manipulation system can
present pseudo-haptics with less physical spaces and be used by more various
users including physically handicapped people. Moreover, since the physical
and virtual bodies are not consistent at the beginning, the problem of
discomfort caused by the mismatch between them can less arise.

The purpose of this study is to further investigate a novel approach for
extending the potential of pseudo-haptics. This paper investigates the pseudo-
haptics with a force-input avatar manipulation that has morphologically
incongruent input-output mapping. Such a force-input system not only has the
merit above but also provides the user with a force feedback equivalent to the
input force applied to the sensor when manipulating the virtual body. Hence,
modifying the motion gain of the system not only changes the visual cues but
also the haptic force cue to the user. For instance, when lifting a virtual
object, pseudo-weight perception can be produced by reducing the amount of
visual movement relative to the actual input. In this scenario, the force-
input manipulation method demands greater force to lift a heavier virtual
object by the same extent as a lighter object. Then, the paper endeavors to
investigate if such a haptic force cue can be used as a substituted cue, i.e.,
be combined with a visual cues and contribute to the haptic estimates as a
haptic force cue, even though the body parts used for the input and output are
different. Fig. 1 depicts a conceptual diagram of the difference of motion-
input and force-input pseudo-haptics. In this paper, we have decided to refer
to this type of pseudo-haptics as FISpH (Force-Input Substituted Pseudo-
Haptics), pronounced the same as ”fish.”

Figure 1: The concept diagram of the motion-input and force-input pseudo-
haptics. The left indicates the motion-input and the right indicates the
force-input manipulation. In the motion-input pseudo-haptics, users estimate
haptics by combining visual posture/motion cue and haptic posture/motion cue.
On the other hand, we hypothesize that in the force-input pseudo-haptics,
users estimate haptics by combining visual posture/motion cue and the reaction
force cue with the force-input.

The paper reports two experiments which compare the pseudo-haptics utilizing
the avatar manipulation method of force-input with that of motion-input. The
paper investigates the pseudo-weight perception while virtual lifting task.
Then, we developed a manipulation system in which the vertical acceleration of
the virtual hand was controlled by the extent of push-in of a force sensor
where the reaction force proportional to the user’s input was fed back to the
user (Fig. 1). The proposed system was compared to the motion-input
manipulation system with respect to the range and resolution of presentable
pseudo-weight perception. The first experiment assessed the number of levels
of pseudo-haptic weights that could be presented by determining the range of
gains that could be applied in each method and the resolution of the pseudo-
haptic presentation within that range. The second experiment investigated the
gain at which the pseudo-weight perception with the proposed manipulation was
perceived to be equivalent to that produced at the maximum/minimum gain of the
motion-input manipulation, in order to compare the two methods on the same
axis of pseudo-weight perception.

The remainder of this paper is structured as follows. First, Section 2
introduces related work. It provides the overview of the pseudo-haptics
techniques and reviews the studies on haptic substitution. Then, Sections 3
describes the two experiments to compare the effectiveness of FISpH and
motion-input pseudo-haptics. Subsequently, Section 4 provides a comprehensive
discussion of all experimental results. Specifically, it discusses the effect
of the two manipulation methods on pseudo-weight perception and usability.
Moreover, it also introduces the discussion of the experimental methods for
measuring pseudo-haptics. Subsequently, it mentions the limitations of the
paper and future work. Finally, Section 5 concludes the paper. We would like
to note that all experiments in the paper were approved by the local ethical
committee.

##  2 Related Work

###  2.1 Pseudo-Haptics Techniques

Since the first paper on pseudo-haptics in 2000 [3], the pseudo-haptics
techniques have been studied for various haptic perceptions. Haptic properties
of objects can be classified into roughly two categories [9] and many pseudo-
haptics techniques are investigated to modify these properties: material
properties, such as texture (such as roughness and friction [10]; e.g. [11,
12, 13, 14]), compliance (e.g. [3, 15, 16, 17, 18]), and weight (e.g. [4, 5,
6]); and geometric properties, such as curvature (e.g. [19]), orientation and
angle (e.g. [20, 21, 19, 22]), and size (e.g. [23]). Moreover, the retargeting
and redirection research [20, 24] can be also considered as one part of
pseudo-haptics research because the geometric properties are explained as
haptic properties and it is mentioned that the modification of the haptic
properties using multisensory (mainly visual and haptics) integration
processing is a pseudo-haptics technique ([1]).

However, if we look at the quantitative results of to what extent these
techniques can modify haptic perception, the pseudo-haptics techniques still
have a large gap for the application. To elaborate, psychophysical experiments
suggest that the pseudo-haptics techniques can alter weight perception roughly
by 3 5% [4, 6], compliance perception by 20 35% [25, 18], friction perception
by 23% [12], and fine roughness perception by 5% [26]. Moreover, studies
including retargeting and redirection field suggest that the pseudo-haptics
techniques can alter hand motion perception up to 13.75% in the forward, and
up to 6.18% in the backward direction without being noticed [27]. In this way,
several studies suggest that the pseudo-haptics techniques can alter haptic
perception roughly by 5% to 35% depending on the target haptic perception.
However, the range of perceptual intensity that can be modified is still
narrow compared to using haptic devices, and it cannot be said enough when
considering various practical use in VR applications.

One of the possible reasons for this limitation is related to the fact that
the pseudo-haptics technique separates visual and haptic information. Here,
larger gaps are necessary to induce more intensive perception; however, gaps
that are too large result in discomfort or a failure to integrate the
information [1, 5, 7]. Based on Pusch and Lécuyer’s model [1], larger
discrepancies could result in larger complements and substitutions of sensory
information from personal memories and experiences, which would increase
individual differences. Moreover, if the gap is too large, the information is
determined to be coming from a different source and is not integrated [28].
Furthermore, forward and inverse dynamics theory can provide another
explanation in that the illusion would not arise when the brain no longer has
a model to which the large gap can be attributed [7].

Although this limitation of pseudo-haptics is well-known, there are few
studies on extending the applicable range of the pseudo-haptics techniques.
First, addition of multiple cues is suggested to enhance the pseudo-haptics
although they did not mention to what extent it can improve [29, 15]. Then,
Ban and Ujitoko proposed a method for solving the discrepancy between visual
and physical position by connecting the discrepancy between a user’s finger
and the cursor on a touchscreen with a visual string [30]. Another possible
approach involves increasing the reliability of visual information in sensory
integration, i.e., to decrease the variance of visual information in MLE so
that the final perception is more dependent on visual information by wider
field of view in VR [31] or more realistic avatar [32]. However, these
approaches do not manipulate the haptic information, thus the pseudo-haptics
is still largely restricted by the haptic cue from the body. On the other
hand, some studies endeavor to add noise signals to haptic cues so that the
final perception is less dependent on haptic cues by using noisy galvanic
vestibular stimulation (GVS) [33], bone-conducted vibration (BCV) [34, 35], or
tendon vibration [36]. For example, Hirao et al. confirmed that the tendon
vibration as noise on haptic motion cues can extend the range of
visual/physical discrepancy without being noticed by about 13%. However, these
approaches has unintended side effects that might have effect on the user
experiences or does not have large improvement. Compared with these
approaches, the proposed method, FISpH, endeavors to tackle the problem from
completely novel angle, which introduces new body schema having force-input
manipulation and newly designs the integration law of visual and haptic cues.

####  2.1.1 Pseudo-haptics using force-input system

There are several studies investigating pseudo-haptics with force-input
devices. Actually, the very first research on pseudo-haptics uses force-input
interface that mimics an interaction with a virtual spring (that looks like a
syringe) on a screen [3]. Most of these studies endeavor to simulate virtual
compliance or stiffness by changing the gain of force-input to virtual motion
with a touch display [37], a rod type device [38], or handheld devices that
sensor the grabbing or pinching force [39, 40, 41, 42]. These approaches can
be said straightforward and strong because such compliance or stiffness has
the push-in force parameter and motion parameter in their physical law and the
pseudo-haptics techniques of changing their gain are rational. Moreover, these
system use the same body schema in physical and virtual reality, i.e., the
body part that is used to the force-input and the one that moves in VR is the
same. For example, TORC simulates virtual hand’s pinch movement by a real
hand’s pinch force-input [41]. On the other hand, the force-input system in
this paper uses the different body schema in physical and virtual reality,
i.e., the height of a virtual hand is manipulated by the extent of push-in of
a force sensor with an index finger. Moreover, this paper investigates weight
perception and its physical law does not explicitly have the push-in force
parameter in its physical equation. Therefore, unlike the previous studies, it
is not obvious that the force feedback, that is, the reaction force while the
push-in, functions as a haptic cue for the weight estimates, combined with the
visual motion cue. In other words, this paper endeavors to introduce a new
body schema in virtual body manipulation and substitute the reaction force cue
of the finger for the somatosensory cue of the arm to strengthen the effect of
the pseudo-haptics techniques.

###  2.2 Sensory Substitution

As the FISpH can be said to substitute original haptic cue (haptic cues
expected from virtual body part) with a new one (haptic cues from force-
input), this section reviews the research of haptic substitution. The study of
how to substitute haptic feedback has been examined mainly in the field of the
research on sensory feedback of prostheses [43, 44]. The non-invasive methods
to present sensory feedback of prostheses are technically devided into
mechanotactile feedback (e.g. [45, 46, 47, 48, 49, 50]), vibrotactile feedback
(e.g. [51, 52]), electrotactile feedback (e.g. [53, 54, 55]), auditory
feedback (e.g. [56, 57]), and hybrid of them (e.g. [58]) [44].

Another way to classify the feedback methods is from the perspective of if the
sensory feedback is modality-matched to (i.e. is felt in the same modality as)
the target sensory input [43, 44]. For example, studies on the sensory
feedback of upper limb prostheses often try to substitute pressure exerted on
prosthetic fingers during grasping movements. In this case, the mechanotactile
feedback is modality-matched feedback, and the other sensory substitutions
such as vibrotactile, electrotactile, and auditory feedback are not. In
general, modality-matched feedback is more intuitive and requires less
cognitive effort during manipulation [43] than non-modality-matched feedback,
and can improve the sense of embodiment to the prostheses [50, 44, 59] as well
as enables closed-loop control [45, 47].

Then, which body part should the substituted feedback be presented to? In the
field of prostheses research, it is often presented at the location directly
related to the prosthesis manipulation (e.g. body-powered prostheses [43]),
the part which has similar physiological characteristics (e.g. [47]), or the
part where the amputee felt the ”phantom sensation of the lost body part” or
which has undergone ”targeted reinnervation surgery.” It is known that brain
plasticity often induces a sensory remapping of the lost body part on the
different body parts such as the amputation stump [60, 61] and face [62, 63],
where the amputee can feel a sense of touch of the lost body part. Moreover,
targeted reinnervation is a technique that ”allows severed cutaneous nerves to
reinnervate skin on a different portion of the body” [49]. Therefore, it is
natural and rational to present sensory substituted feedback to these body
parts because it is physiologically correct [43]. Meanwhile, in recent years,
sensory substitution techniques have been investigated also for non-amputee
users due to the increasing demand for haptic feedback in VR and augmented
reality research. For example, Pezent et al. proposed a wrist band shaped
device named Tasbi which can present squeeze and vibrotactile feedback to the
wrist [64]. They proposed to substitute vibrotactile feedback for fingertip
contact sensation with virtual objects or user interfaces, and squeeze
feedback for weight or stiffness associated with manipulating virtual objects.
Moreover, Kameoka et al. proposed a system named Haptopus, that transforms the
touch sensation of the hand into tactile (suction) feedback to the face [65].
Furthermore, Hiki et al. proposed a sensory substitution system that
transforms the pressure sensation of the hand to the one of the sole of the
foot [66]. Their experimental results suggest that their system can allow
participants to recognize the weight, stiffness, and shape of an object in a
similar capability to the one using a hand. These techniques are valuable
because they can present haptic information to users while allowing them
hands-free interaction. However, unlike in most cases of the prostheses
studies for amputees, these techniques do not present substituted feedback in
a physiologically correct way. Moreover, these approaches do not strictly
substitute/remove the original haptic cue but use another haptic cue in
addition to the original one. As a result, the original haptic estimates from
the body part, such as the fingers, remain present. Thus, the haptic estimates
is still restricted by the one derived from the original haptic cue. For
example, when haptic feedback of the hand is substituted with other body parts
to present the perception of the hardness of a virtual object interacting with
a virtual hand, even though the information of the hardness can be
interpreted, the perception of the hardness and/or user experience may be
interfered by the fact that the hand is physically grasping empty. This may be
solved by long-term learning with brain plasticity [67], but that is not
obvious at this moment. Then, this paper proposes to develop a new body schema
that can reduce/remove the influence of haptic estimates derived from the
original haptic cues, and that substitutes modality-matched feedback for the
original one.

##  3 Experiments

This section compares FISpH, which substitutes the original haptic cues with
the reaction force cues induced by force-input manipulation in a new body
schema, with the conventional motion-input pseudo-haptics. The main focus of
the experiment is to investigate the pseudo-haptics of the two methods from a
view of the applicable range and the resolution of the pseudo-haptic
perception. Specifically, the experiments investigate pseudo-weight perception
while lifting a virtual object.

The research questions in the experiments follow:

  * [RQ1]

Which manipulation method has wider range of pseudo-weight perception?

  * [RQ2]

Which manipulation method has more number of levels of pseudo-weight
perception?

  * [RQ3]

Is the current implementation of force-input manipulation controllable?

###  3.1 XP1: How many levels of pseudo-haptic weight can motion-input and
force-input manipulation present?

The first experiment investigated the range of applicable motion gain out of
which users start to feel the interaction is incoherent/weird or lose the
feeling of control. Moreover, this also allows to investigate the resolution
of pseudo-haptic weight perception with each manipulation. With these results,
i.e., the range and resolution, we calculated how many levels of pseudo-haptic
weight each manipulation can present.

####  3.1.1 Apparatus

Figure 2: The apparatus and virtual environment. The top figures indicate the
force-input manipulation and the bottom figures indicate the motion-input
manipulation. The middle figures indicate the virtual environment.

Figure 2 shows the experimental apparatus and virtual environment. The
experiment was conducted with participants in a sitting position. Participants
wore a VR headset (Oculus Quest 2). A force sensor (FSR 400, Interlink
Electronics Inc.) was attached to a table beside the participants. In VR,
there was a low table and a virtual dumbbell on it in front of the
participants. The virtual dumbbell was placed at 0.7 m in height from the
ground.

In this experiment, there were two manipulation methods for a virtual hand.
One manipulation method was the motion-input manipulation. Here, participants
held a VR controller with their right hand. The position of the virtual hand
tracked that of a VR controller, and the posture of the virtual hand was set
to correspond to the physical hand when participants grabbed the VR
controller. Participants could grab and release the virtual dumbbell by using
an index trigger. Once they grabbed the virtual dumbbell, the virtual hand and
dumbbell moved only on a vertical axis and their height tracks VR controller’s
height with a gain.

The other manipulation method was the force-input manipulation. Here,
participants put their right arm on the table beside them and press the force
sensor with their index finger. In this method, we calculated the acceleration
of the virtual hand and object by applying force on the sensor and Newton’s
motion equation. First, we obtained an equation for the relationship between
output voltage from the force sensor [V] ($V_{out}$) and applied force [N]
($F_{s}$) on it (equation 1) from a data sheet. Here, 0.20 in equation 1
indicates the area of the contact surface of the force sensor. Then, for the
method of simulating virtual force ($F_{v}$) induced by the virtual hand from
the applied force ($F_{sensor}$), this time, we determined to simply multiply
a constant of 10 by applied force to the force sensor and a gain ($G$) that is
used for pseudo-haptics (equation 2). Note that there would be a space to
design a better simulation of virtual force from the applied force. However,
this study is at the first step to investigating the effect of force-input
manipulation on pseudo-haptics and does not focus on the proper design of
force-input manipulation. Therefore, we determined to use the most simple
equation by using a constant of 10 that is decided by our pilot test.
Subsequently, the acceleration of virtual hand/object [$m/s^{2}$] ($A_{v}$)
was calculated by Newton’s motion equation (equation 3). Here, the weight of
the virtual object was set to 0.15 [kg] which is the standard weight of VR
controller. In the force-input manipulation, the virtual hand was set to the
grab position of the virtual dumbbell, and they moved in a vertical axis when
participants apply force to the sensor.

$F_{s}=(0.067\cdot e^{1.5\cdot V_{out}})\cdot 0.20\cdot G~{}~{}~{}~{}[N]$ (1)

$F_{v}=F_{s}\cdot 10\cdot G~{}~{}~{}~{}[N]$ (2)

$A_{v}=\frac{F_{v}}{0.15}-9.8~{}~{}~{}~{}[m/s^{2}]$ (3)

####  3.1.2 Task

There were three phases in the experiment. The first phase was the ”learning
task phase” which aimed to let participants get used to each manipulation and
investigate the task performance. Here, participants conducted reaching tasks
on the vertical axis with each manipulation method. A target height was
presented with a blue bar. Participants grabbed and lifted up a virtual object
to the blue bar. After the participants reached the target bar and released
the object, then, the bar, virtual hand, and virtual object disappeared.
Subsequently, the virtual hand and object appeared in an original position,
and the next target bar was presented. The target bar was randomly positioned
in a range from 0.2 m to 0.7 m range from the table height (0.7 m) in front of
the participants. This reaching task took 15 seconds and the participants
conducted 2 repetitions of this 15 seconds trial for each manipulation method.
The participants were asked to complete the task as fast and precisely as
possible (tried not to go over the blue bar but to stop and release at the
bar’s height).

The second phase was the ”range task phase” which aimed to investigate the
applicable gain for pseudo-haptics in each manipulation. The staircase method
was used for this aim. Here, the applicable gain for pseudo-haptics indicates
a minimum and a maximum gain where participants start to feel discomfort or
lose the feeling of control. As reported in the related work section, the most
common range of ”applicable gain” in previous work in the field of redirection
or pseudo-haptics is the one under the detection threshold of visual/physical
discrepancy. This range is the most strict range where users do not notice the
visual modification by a gain. However, we consider that range of applicable
gain in practice is wider than this range, where users can notice the
visual/physical discrepancy but feel pseudo-haptic perception without feeling
unnaturalness for the interaction. we consider that this wider definition of
the range of applicable gain is more practical. Indeed, there are many studies
on pseudo-haptics that uses a gain beyond the detection threshold of
visual/physical discrepancy (e.g. [4, 5, 6]). Moreover, because we investigate
pseudo-haptics with force-input manipulation and the force-input manipulation
does not have an obvious definition of the detection threshold of
visual/physical discrepancy, we cannot use the range using the detection
thresholds. Therefore, in this experiment, we determined the range of
applicable gain as the one where users do not feel discomfort or do not lose
the feeling of control.

In the task of the staircase method, the range of adjustable gain was from 1.0
to 3.0 for the high gain group (gain $>$ 1) and from 0.3 to 1 for the low gain
group (gain $<$ 1). Each gain group had ten equal steps i.e., a step value in
the high gain group was 0.2 and that in the low gain group was 0.07. The task
was a free interaction with the virtual dumbbell in 5 seconds in VR. After the
free interaction, participants answered the question of “It was too
heavy/light.” by ”Agree” or ”Disagree” by touching a virtual button in VR.
Here, an experimenter explained that the question indicates that “I felt it
was too heavy/light and start to feel weird or lose the feeling of control.” A
staircase method had two series of trials whose initial gain was either the
minimum or maximum gain of the range of adjustable gain. When participants
answered “Agree”, the gain moved one step closer to a gain of 1.0 and vice
versa. When each series in a staircase method reached 5 turning points, the
experiment was over.

The third phase was the ”resolution task phase” which aimed to investigate the
resolution of pseudo-haptic perception. The method of constant stimuli was
used for this aim. Participants compared the sense of the weight of reference
and comparison dumbbell within one of the manipulation and answer which
dumbbell was heaver. The reference gain was 1.0 and the comparison gains were
0.76, 0.84, 0.92, 1.08, 1.16, 1.24 (-24%, -16%, - 8%, +8%, +16%, +24%,
respectively). The task was to interact with two virtual dumbbells for 3
seconds for each and answered a question: “Which dumbbell did you feel
heavier?” by “The first” or “The second” by touching a button in VR. Each
comparison gain was compared to the reference gain ten times. Therefore,
participants conducted 120 trials in total ($6\>comparison\>gains\times
2\>manipulations\times 10\>repetitions$).

####  3.1.3 Collected data

In the learning task, the overshoot distance from the target height, each
reaching duration, and the number of reaching the target were recorded. This
data was used to check if the force-input manipulation was usable for this
lifting task. In the range task, the minimum and maximum applicable gain was
obtained by averaging adjusted gains at 10 turning points in the staircase
method. In the resolution task, JND of pseudo-haptic weight perception was
calculated by psychometric function obtained by the method of constant
stimuli.

####  3.1.4 Procedure

Twenty participants (fifteen males and five females in their twenties)
participated in the experiment. First, the objectives, methods, procedures,
and how each manipulation works were explained to them. Next, the participants
completed a consent form. Then, the participants were asked to wear the VR
headset. Here, the participants also checked the position of the force sensor
on a table. Before the main tasks, participants practiced each manipulation
method with a virtual dumbbell with a gain of 1. Subsequently, the
participants conduct 3 main tasks. The combination of the order of the
manipulation method in each task was counterbalanced between the participants.
Furthermore, other orders such as that of the high/low gain group in the
second phase and comparison gains in the third phase were randomized. The
participants could take a break whenever they wanted. After the experiment,
the participants were remunerated with an Amazon gift card of 5 euros for
their participation. The total duration of the whole procedure was around 45
minutes.

####  3.1.5 Results

TABLE I: The results of task scores in the first phase. The scores at the second trial are used. The values indicate $average\pm StandardError$. Manipulation |  | Number of Reaching
---
| Overshoot (m)
---
motion-input | $9.4\pm 0.5$ | $0.06\pm 0.01$
force-input | $9.0\pm 0.6$ | $0.23\pm 0.03$
TABLE II: The results of the applicable gain threshold. Note that the results of motion-input and force-input are not directly comparable because the meaning of gain is different. The values indicate $average\pm StandardError$ Manipulation |  | Threshold (Low)
---
| Threshold (High)
---
motion-input | $0.57\pm 0.04$ | $1.51\pm 0.08$
force-input | $0.46\pm 0.03$ | $1.88\pm 0.13$

Table I shows the results of the task scores of the reaching task, i.e.,
number of reaching the bar within the trial and overshoot distance from the
target height when the participants released the virtual object. Here, we used
the values at the second trial. In addition, the results of the overshoot of
each reaching task were averaged for each trial. Paired t-test was conducted
for each number of reaching and overshoot. As a result, a significant
difference was found in the results of overshoot ($p<0.001,d=1.76$).

Table II shows the results of the second phase. With these results, we can
calculate the range of the applicable gains in each manipulation. The range of
applicable gain for motion-input manipulation was 0.94 and that for force-
input manipulation was 1.42.

Then, for the results of the third phase, the Probit analysis [68] was
conducted for the averaged values of all participants’ results, which
calculated the parameters of the best-fitting cumulative normal function.
Subsequently, we computed the JND for each manipulation as half of the
distance between the points of 25% and 75% on the psychometric curve. These
values were 0.10 for the motion-input manipulation and 0.17 for the force-
input manipulation. Figure 3 shows the psychometric curves calculated with the
averaged values of all participants’ results. With the results of the second
and third phases, we calculated the number of pseudo-haptic weight levels that
each manipulation method can present. Then, it was 9 levels for motion-input
manipulation and 8 for force-input.

Figure 3: The results of the third phase: Psychometric curves of the averaged
results. It plots the average percentage of answers in which participants
considered the reference dumbbell heavier than the comparison dumbbell. The
error bar indicates the standard error.

###  3.2 XP2: What is the equality between the pseudo-haptic weight with the
force-input and motion-input manipulation?

The second experiment investigated the point of subjective equality (PSE) of
the pseudo-haptic weight perception between each manipulation. In the first
experiment, we obtained the range of the applicable gain for pseudo-haptic
weight and its JND with each manipulation. However, the gains cannot be
directly compared because the meaning of the gains is different in each
manipulation. Therefore, here, we investigated the PSE to enable the
comparison of the results of the first experiment between each manipulation.

####  3.2.1 Apparatus

The apparatus was the same as that in the first experiment. However, this
time, participants used both hands during the task: one hand was used for the
motion-input and the other hand for the force-input manipulation. The hand for
each manipulation was switched during the experiment.

####  3.2.2 Task

Figure 4: The conceptual diagram of an adaptive staircase method. The arrow
indicates one trial to the next trial and its length shows the distance of
each step gain. The dots and numbers indicate the turning points where the
answer of participants changed from the last one or the gain is likely to go
beyond the maximum or minimum gain. The green arrows indicate descending
series and the red indicates ascending series. The PSE was calculated as the
average of gains at each turning point.

An adaptive staircase method was used to investigate the PSE of the sense of
weight between each manipulation. Specifically, the PSE for weight perception
at a maximum and minimum gain in the motion-input manipulation was obtained by
manipulating a gain in force-input manipulation. Participants used one hand
for the motion-input manipulation and the other hand for the force-input
manipulation to lift up a virtual dumbbell in VR. Then, the participants
answered the question “which dumbbell did you feel heavier?” by “motion-input
manipulation” or “force-input manipulation.” The reference dumbbell was with
the motion-input manipulation at minimum or maximum gain obtained in the first
experiment, i.e., 0.57 or 1.51, respectively. We determined the motion-input
manipulation as the reference condition by referring to the observation of the
first experiment where most of the participants experienced that the gain
around the maximum/minimum threshold with the force-input manipulation was
heavier/lighter than that with the motion-input manipulation. The comparison
dumbbell was with the force-input manipulation at a gain in the range of
minimum and maximum gain obtained in the first experiment, i.e., 0.47 and
1.87, respectively.

Figure 4 shows the conceptual diagram of the adaptive staircase method. The
initial gain of the reference dumbbell was either maximum (0.47) or minimum
(1.87). There were three levels of distance for each step for the adaptive
staircase method. The minimum distance was 0.175 and the others were 0.35 and
0.7. These were 1/2/4 times the minimum distance. If the participants chose
the force-input manipulation as a heavier dumbbell, the force-input gain moved
one step closer to the maximum, and vice versa. Here, if the gain was likely
to move beyond the range, it remained at the same value. The steps first
changed with the maximum distance and the distance changed after two turning
points. There were also ascending and descending series and each series ended
at the sixth turning point. The participants repeated the staircase method two
times for each maximum and minimum gain of the motion-input manipulation. Note
that the combination of the participant’s hands and the manipulations changed
between the two repetitions. At the end of the experiment, participants were
asked which method was better in a point of sense of weight by a question of
“With which manipulation method did you feel a better sense of weight/effort?”

####  3.2.3 Collected data

Gains at each turning point were recorded. Then, these gains were averaged to
calculate PSE for each participant. Furthermore, the answer to the last
question ”With which manipulation method did you feel a better sense of
weight/effort?” was saved.

####  3.2.4 Procedure

Eighteen participants (16 males and 2 females in their twenties excluding two
in their 40s) participated in the experiment. The flow before the main task
was completely the same as in the first experiment. After the practice trial,
participants conducted the main task of the staircase method. When
participants changed the combination of the hand and manipulation, they
rotated their chair and VR scene was re-centered. The combination of the order
of the maximum/minimum gain of the motion-input manipulation and the order of
the hand-manipulation combination was counterbalanced between the
participants. The participants could take a break whenever they wanted. After
the experiment, the participants were remunerated with an Amazon gift card of
1000 yen for their participation. The total duration of the whole procedure
was around 30 minutes.

####  3.2.5 Results

TABLE III: The PSE of pseudo-weight perception between the motion-input and force-input manipulation. Manipulation | ($Average\pm StandardError$)
---|---
motion-input Max (1.51) | $1.57\pm 0.07$
motion-input Min (0.57) | $0.78\pm 0.06$

Gains at each turning point were averaged to calculate the PSE for each
participant. Then, paired t-test was conducted for the results between the
combinations of hand and manipulation. As a result, no significant difference
was found. Therefore, we determined to compute PSE by averaging the results of
2 repetitions (different hand-manipulation combinations). Table III shows the
results of the PSE.

As a result of the last question, 7 out of 18 participants mentioned that the
motion-input manipulation provides a better sense of weight/effort, while the
other 11 participants mentioned the opposite.

##  4 Discussion

###  4.1 The force-input can have wider pseudo-weight range while the motion-
input can have higher resolution

Figure 5: The summary of the results of the two experiments. ”T” indicates
threshold. Here, the horizontal bars indicate the range of gain which can be
used for presenting pseudo-weight perception with each manipulation method.
The bars are scaled to make them comparable in the axis of the same pseudo-
weight perception.

Figure 5 indicates the summary of the results of the two experiments. Taken
together the results of the first and second experiments, the force-input
manipulation can actually provide a wider range of pseudo-haptic weight
perceptions (about 1.8 times wider), i.e. lighter and heavier perceptions.
This results suggest that FISpH can have wider range of pseudo-weight
perception (RQ1) and support the effectiveness of the proposed concept. It is
noteworthy that these findings imply that users can experience varying degrees
of pseudo-weight perception, even when interacting with significantly
different body schema, such as controlling a virtual hand using pressure input
from an index finger, without much training effort. This would suggest that
people could integrate the substituted haptic cue with visual motion cue to
estimate virtual weight. One thing to note is that we cannot rule out the
possibility that the results would be different if different formulas or
implementation were used for the force-input manipulation. Nevertheless, this
paper’s main contribution is that it successfully suggested that the perceived
intensity range of pseudo-haptics can be extended in both heavy and light
directions with a given implementation, and confirmed the potential of this
concept as the first step.

However, at a point of a number of levels of presentable pseudo-haptic weight,
the motion-input manipulation can present 1 more level (9 levels) than the
force-input (8 levels). Moreover, the difference in weight perception between
each step is smaller in the motion-input manipulation than in the force-input.
The JND of pseudo-weight perception was gain of 0.1 for motion-input and gain
of 0.17 for force-input manipulation. These results conclude that motion-input
can have more multiple levels of pseudo-weight perception (RQ2). Still, again,
these results could be different by improving the implementation of force-
input body manipulation.

###  4.2 The motion-input is easier to operate

Regarding usability, the force-input manipulation had a lower control compared
to the motion-input manipulation as participants overshoot about 23 cm from
the point they wanted to stop although the total number of reaching the target
was not largely different. These findings imply that the current
implementation of the force-input manipulation could be manipulable, but it
may exhibit not small (approximately a 23 cm) margin of error when utilized
for tasks requiring haste (RQ3). However, this paper did not aim to design a
control law from force input to output motion but focused on the investigation
of the pseudo-haptic perception of the force-input manipulation as a first
study. Therefore, future work should consider the improvement of the
implementation of the force-input manipulation in a point of usability and
sense of embodiment. In summary, with the current implementation, it can be
concluded that the motion-input manipulation should be employed when a small
modification of weight perception is enough but better operability of
manipulation is required, while the force-input should be employed when a
larger range of weight perception is desired.

###  4.3 The difficulties of measuring pseudo-haptics

Although the paper tried to evaluate pseudo-haptic perception from a view of
the threshold (range), JND (resolution), and PSE (equality) with
psychophysical methods, how to evaluate pseudo-haptics is one of the biggest
topics to be discussed and there is no consensus on this point yet. Then, this
subsection reviews and discusses the experimental methods to evaluate pseudo-
haptics, and concludes that our experimental method could be one of the
promising solutions especially when comparing pseudo-haptics in very different
methods.

First, the Likert scale asking participants to answer their pseudo-haptic
perception from 1 to 7 can be done with a relatively simple experiment design
and the experiment requires relatively little time. However, it cannot be
clear to what extent the proposed method can extend the pseudo-haptics
quantitatively. Therefore, it would be a good method as a first step of the
research to see if the proposed method is effective or in cases where there
are many experimental conditions, tasks, or questions.

Then, the intensity of pseudo-haptics can be evaluated by directly answering
the intensity on a scale from 0 to 100 using VAS (e.g., [8]). To directly
answer the haptic sensation (VAS) may reflect the person’s pure perception.
However, the results tended to have a large variance because the evaluation
axis in a mind may vary from person to person. Moreover, the participant’s
understanding of the question may cause their answers to be biased. In most
cases, the design of proper wording in the question would be difficult. For
example, when participants evaluate pseudo-haptics in a question about the
sense of weight, the interpretation of ”weight” can be different for each
participant. Participants may evaluate the weight from their fatigue, sense of
body motion, or other similar perception such as stickiness. Therefore,
researchers should pay attention to the design of the question and explanation
for the experiment.

Next, the behavioral indicators such as exerted force (gripping force) could
capture the effect of pseudo-haptics ([29]). Although the results of the
exerted force seem to reflect the intensity of pseudo-haptics at a certain
level, the variances are relatively large and does not completely fit the
subjective results. Therefore, further study is needed to determine how
accurately the difference in pseudo-haptic perception can be captured in the
behavioral factor or how to improve it.

Moreover, a psychophysical method where pseudo-haptics were evaluated by
referring to physical quantities can also be used (e.g., [4, 6, 8]) At first
glance, this method may seem to give the most robust results because this
method does not need to suffer from wording issues. However, we need to keep
in our mind that the pseudo-haptic perception may not be qualitatively equal
to the corresponding haptic perception in our daily life [69, 2]. Moreover,
the experimental results suggest that the results would be different depending
on the experimental methods, that is, adjusting the visual gain against the
physical quantities (e.g., [8]) or vice versa (e.g., [4, 6]). This can note
that researchers need to pay attention to the experimental design not to
underestimate the capability of pseudo-haptics techniques due to the
experimental task.

Subsequently, in the paper, we tried to evaluate the pseudo-haptics from a
view of the range with resolution and PSE. This information enables us to get
the characteristics of the pseudo-haptics technique and compare the different
techniques. However, it is difficult to determine what constitutes a range in
pseudo-haptics. Moreover, it is also difficult to evaluate the thresholds
because the evaluation method also affects the results [70]. Here, there are
two possible thresholds for the spatial limitation in pseudo-haptics: a
detection threshold of visual/physical discrepancy (e.g., [36]) and an
acceptable threshold with noticing the discrepancy (e.g., [5]). Therefore, it
can be considered that there are four levels of pseudo-haptics experience: the
first level is that users do not feel pseudo-haptics nor realize the pseudo-
haptics technique (mostly the visual gain); the second level is that users
feel pseudo-haptics but unaware of the technique; the third level is that
users feel pseudo-haptics and aware of (accept) the technique; the fourth
level is that users hardly feel pseudo-haptics or have large variances for the
occurrence of pseudo-haptics and feel unnatural or discomfort for the
technique. In the conventional pseudo-haptic studies, the range was defined as
one including the first and second levels because the psychophysical
experiment can figure out relatively exact and robust threshold of the second
and third levels. This method is probably the most strict and robust method
for evaluating pseudo-haptics at present. However, because the third layer can
be and is often used in actual applications [5], this strict evaluation would
underestimate pseudo-haptics potential. On the other hand, in the present
paper, the range was defined as one including the first, second, and third
levels. we consider that this method can have the most practical results that
can be used in actual applications at present. However, it is difficult to
determine the threshold of the third and fourth levels. For example, the word
”discomfort” has a wide range of meanings and can be interpreted ambiguously
by each participant. Therefore, further study is required to investigate how
to determine a proper threshold for the third and fourth levels.

###  4.4 Limitations and Future work

First, the paper focused on the pseudo-haptics and did not investigate other
important factors of avatar manipulation such as sense of embodiment or more
detailed task performance. Sense of embodiment plays an important role in
sense of immersion/presence, i.e. sense of being there, in VR [71]. In the
future work, these factors should be considered and the implementation of the
design of the force-input manipulation should be improved referring to the
results. Specifically, the law of converting force into virtual body motion
has a space to be improved. One thing to note is that the results obtained in
the paper can be different with the different implementation. Moreover,
another possible future work is to investigate the methodology to develop a
force-input manipulation system of the whole virtual body. In the current
experiment, the implementation of force-input manipulation was limited to the
vertical motion of a virtual hand.

##  5 Conclusion

The paper investigated effectiveness of the force-input body manipulation on
pseudo-weight perception in a comparison with that of the motion-input
manipulation. Specifically, the study investigated the force-input
manipulation where the motion velocity of the virtual hand corresponds to the
force applied to a force sensor in a comparison with motion-input
manipulation. The experimental results suggest that the reaction force induced
by the force-input could substitute the original haptic cue and be integrated
with the visual motion cue, inducing pseudo-weight perception. Moreover, the
force-input manipulation can extend the range of presentable pseudo-weight
perception by 80% compared to the motion-input manipulation, while the motion-
input manipulation has 1 more steps of the presentable pseudo-weight levels.
Taken together, with the current implementation, it can be concluded that the
motion-input manipulation should be employed when a small modification of
weight perception is enough but better operability of manipulation is
required, while the force-input should be employed when a larger range of
weight perception is desired.

## Acknowledgments

This work was partially supported by the MEXT Grant-in-Aid for Scientific
Research (S) (19H05661), Grant-in-Aid for JSPS Fellows (21J12284), and Grant-
in-Aid for Research Activity Start-up (23K20007).

## References

  * [1] A. Pusch and A. Lécuyer, “Pseudo-haptics: from the theoretical foundations to practical system design guidelines,” in _Proceedings of the 13th international conference on multimodal interfaces_ , 2011, pp. 57–64.
  * [2] Y. Ujitoko and Y. Ban, “Survey of pseudo-haptics: Haptic feedback design and application proposals,” _IEEE Transactions on Haptics_ , vol. 14, no. 4, pp. 699–711, 2021.
  * [3] A. Lécuyer, S. Coquillart, A. Kheddar, P. Richard, and P. Coiffet, “Pseudo-haptic feedback: can isometric input devices simulate force feedback?” in _Proceedings IEEE Virtual Reality 2000 (Cat. No. 00CB37048)_. IEEE, 2000, pp. 83–90.
  * [4] Y. Taima, Y. Ban, T. Narumi, T. Tanikawa, and M. Hirose, “Controlling fatigue while lifting objects using pseudo-haptics in a mixed reality space,” in _2014 IEEE Haptics Symposium (HAPTICS)_. IEEE, 2014, pp. 175–180.
  * [5] M. Rietzler, F. Geiselhart, J. Gugenheimer, and E. Rukzio, “Breaking the tracking: Enabling weight perception using perceivable tracking offsets,” in _Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems_ , 2018, pp. 1–12.
  * [6] M. Samad, E. Gatti, A. Hermes, H. Benko, and C. Parise, “Pseudo-haptic weight: Changing the perceived weight of virtual objects by manipulating control-display ratio,” in _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_ , 2019, pp. 1–13.
  * [7] T. Honda, N. Hagura, T. Yoshioka, and H. Imamizu, “Imposed visual feedback delay of an action changes mass perception based on the sensory prediction error,” _Frontiers in psychology_ , vol. 4, p. 760, 2013.
  * [8] Y. Hirao, T. M. Takala, and A. Lécuyer, “Comparing motion-based versus controller-based pseudo-haptic weight sensations in vr,” in _2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)_. IEEE, 2020, pp. 305–310.
  * [9] L. A. Jones and S. J. Lederman, _Human hand function_. Oxford university press, 2006.
  * [10] S. Okamoto, H. Nagano, and Y. Yamada, “Psychophysical dimensions of tactile perception of textures,” _IEEE Transactions on Haptics_ , vol. 6, no. 1, pp. 81–93, 2012.
  * [11] T. Narumi, Y. Ujitoko, Y. Ban, T. Tanikawa, K. Hirota, and M. Hirose, “Resistive swipe: Visuo-haptic interaction during swipe gestures to scroll background images on touch interfaces,” in _2017 IEEE World Haptics Conference (WHC)_. IEEE, 2017, pp. 334–339.
  * [12] Y. Ujitoko, Y. Ban, and K. Hirota, “Presenting static friction sensation at stick-slip transition using pseudo-haptic effect,” in _2019 IEEE World Haptics Conference (WHC)_. IEEE, 2019, pp. 181–186.
  * [13] M. Fukushima, T. Takemoto, and M. Fujimura, “A study on pseudo-haptics by cursor moving with motion blur,” in _2013 Seventh International Conference on Complex, Intelligent, and Software Intensive Systems_. IEEE, 2013, pp. 751–754.
  * [14] A. Lécuyer, J.-M. Burkhardt, and L. Etienne, “Feeling bumps and holes without a haptic interface: the perception of pseudo-haptic textures,” in _Proceedings of the SIGCHI conference on Human factors in computing systems_ , 2004, pp. 239–246.
  * [15] P. Punpongsanon, D. Iwai, and K. Sato, “Softar: Visually manipulating haptic softness perception in spatial augmented reality,” _IEEE transactions on visualization and computer graphics_ , vol. 21, no. 11, pp. 1279–1288, 2015.
  * [16] F. A. Sanz, D. A. G. Jáuregui, M. Marchal, and A. Lécuyer, “Elastic images: Perceiving local elasticity of images through a novel pseudo-haptic deformation effect,” _ACM Transactions on Applied Perception_ , vol. 10, no. 3, pp. 17–1, 2013.
  * [17] A. Kumar, R. Gourishetti, and M. Manivannan, “Mechanics of pseudo-haptics with computer mouse,” in _2017 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE)_. IEEE, 2017, pp. 1–6.
  * [18] M. Tatezono, K. Sato, K. Minamizawa, H. Nii, N. Kawakami, and S. Tachi, “Effect of haptic feedback on pseudo-haptic feedback for arm display,” in _2009 ICCAS-SICE_. IEEE, 2009, pp. 4332–4337.
  * [19] Y. Ban, T. Kajinami, T. Narumi, T. Tanikawa, and M. Hirose, “Modifying an identified curved surface shape using pseudo-haptic effect,” in _2012 IEEE Haptics Symposium (HAPTICS)_. IEEE, 2012, pp. 211–216.
  * [20] L. Kohli, “Redirected touching: Warping space to remap passive haptics,” in _2010 IEEE Symposium on 3D User Interfaces (3DUI)_. IEEE, 2010, pp. 129–130.
  * [21] M. Azmandian, M. Hancock, H. Benko, E. Ofek, and A. D. Wilson, “Haptic retargeting: Dynamic repurposing of passive haptics for enhanced virtual reality experiences,” in _Proceedings of the 2016 chi conference on human factors in computing systems_ , 2016, pp. 1968–1979.
  * [22] D. T. Han, M. Suhail, and E. D. Ragan, “Evaluating remapped physical reach for hand interactions with passive haptics in virtual reality,” _IEEE transactions on visualization and computer graphics_ , vol. 24, no. 4, pp. 1467–1476, 2018.
  * [23] Y. Ban, T. Narumi, T. Tanikawa, and M. Hirose, “Modifying perceived size of a handled object through hand image deformation,” _Presence: Teleoperators and Virtual Environments_ , vol. 22, no. 3, pp. 255–270, 2013.
  * [24] S. Razzaque, _Redirected walking_. The University of North Carolina at Chapel Hill, 2005.
  * [25] A. Lécuyer, J.-M. Burkhardt, S. Coquillart, and P. Coiffet, “” boundary of illusion”: an experiment of sensory integration with a pseudo-haptic system,” in _Proceedings IEEE Virtual Reality 2001_. IEEE, 2001, pp. 115–122.
  * [26] Y. Ujitoko, Y. Ban, and K. Hirota, “Modulating fine roughness perception of vibrotactile textured surface using pseudo-haptic effect,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 25, no. 5, pp. 1981–1990, 2019.
  * [27] A. Zenner and A. Krüger, “Estimating detection thresholds for desktop-scale hand redirection in virtual reality,” in _2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)_. IEEE, 2019, pp. 47–55.
  * [28] M. O. Ernst and H. H. Bülthoff, “Merging the senses into a robust percept,” _Trends in cognitive sciences_ , vol. 8, no. 4, pp. 162–169, 2004.
  * [29] Y. Hirao and T. Kawai, “Augmented cross-modality: Translating the physiological responses, knowledge and impression to audio-visual information in virtual reality,” _Journal of Imaging Science and Technology_ , vol. 62, no. 6, pp. 60 402–1, 2018.
  * [30] Y. Ban and Y. Ujitoko, “Enhancing the pseudo-haptic effect on the touch panel using the virtual string,” in _2018 IEEE Haptics Symposium (HAPTICS)_. IEEE, 2018, pp. 278–283.
  * [31] N. L. Williams and T. C. Peck, “Estimation of rotation gain thresholds considering fov, gender, and distractors,” _IEEE transactions on visualization and computer graphics_ , vol. 25, no. 11, pp. 3158–3168, 2019.
  * [32] N. Ogawa, T. Narumi, and M. Hirose, “Effect of avatar appearance on detection thresholds for remapped hand movements,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 27, no. 7, pp. 3182–3197, 2020.
  * [33] K. Matsumoto, K. Aoyama, T. Narumi, and H. Kuzuoka, “Redirected walking using noisy galvanic vestibular stimulation,” in _2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)_. IEEE, 2021, pp. 498–507.
  * [34] S. Weech and N. F. Troje, “Vection latency is reduced by bone-conducted vibration and noisy galvanic vestibular stimulation,” _Multisensory Research_ , vol. 30, no. 1, pp. 65–90, 2017.
  * [35] T. Kondo, Y. Hirao, T. Narumi, and T. Amemiya, “Effects of bone-conducted vibration stimulation of various frequencies on the vertical vection,” _Scientific Reports_ , vol. 13, no. 1, p. 15759, 2023.
  * [36] Y. Hirao, T. Amemiya, T. Narumi, F. Argelaguet, and A. Lécuyer, “Leveraging tendon vibration to enhance pseudo-haptic perceptions in vr,” _IEEE Transactions on Visualization and Computer Graphics_ , 2023.
  * [37] M. B. Ridzuan, Y. Makino, and K. Takemura, “Direct touch haptic display using immersive illusion with interactive virtual finger,” in _International Conference on Human Haptic Sensing and Touch Enabled Computer Applications_. Springer, 2012, pp. 432–444.
  * [38] S. Heo, J. Lee, and D. Wigdor, “Pseudobend: Producing haptic illusions of stretching, bending, and twisting using grain vibrations,” in _Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology_ , 2019, pp. 803–813.
  * [39] D. Matsumoto, K. Hasegawa, Y. Makino, and H. Shinoda, “Displaying variable stiffness by passive nonlinear spring using visuo-haptic interaction,” in _2017 IEEE World Haptics Conference (WHC)_. IEEE, 2017, pp. 587–592.
  * [40] S.-i. Yabe, H. Kishino, T. Kimura, and T. Nojima, “Pseudo-haptic feedback on softness induced by squeezing action,” in _2017 IEEE World Haptics Conference (WHC)_. IEEE, 2017, pp. 557–562.
  * [41] J. Lee, M. Sinclair, M. Gonzalez-Franco, E. Ofek, and C. Holz, “Torc: A virtual reality controller for in-hand high-dexterity finger interaction,” in _Proceedings of the 2019 CHI conference on human factors in computing systems_ , 2019, pp. 1–13.
  * [42] A. Adilkhanov, A. Yelenov, R. S. Reddy, A. Terekhov, and Z. Kappassov, “Vibero: Vibrotactile stiffness perception interface for virtual reality,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 2785–2792, 2020.
  * [43] C. Antfolk, M. D’alonzo, B. Rosén, G. Lundborg, F. Sebelius, and C. Cipriani, “Sensory feedback in upper limb prosthetics,” _Expert review of medical devices_ , vol. 10, no. 1, pp. 45–54, 2013.
  * [44] P. Svensson, U. Wijk, A. Björkman, and C. Antfolk, “A review of invasive and non-invasive sensory feedback in upper limb prostheses,” _Expert review of medical devices_ , vol. 14, no. 6, pp. 439–447, 2017.
  * [45] C. Antfolk, A. Björkman, S.-O. Frank, F. Sebelius, G. Lundborg, and B. Rosen, “Sensory feedback from a prosthetic hand based on air-mediated pressure from the hand to the forearm skin,” _Journal of rehabilitation medicine_ , vol. 44, no. 8, p. 702, 2012.
  * [46] C. Antfolk, C. Cipriani, M. C. Carrozza, C. Balkenius, A. Björkman, G. Lundborg, B. Rosén, and F. Sebelius, “Transfer of tactile input from an artificial hand to the forearm: experiments in amputees and able-bodied volunteers,” _Disability and Rehabilitation: Assistive Technology_ , vol. 8, no. 3, pp. 249–254, 2013.
  * [47] A. Panarese, B. B. Edin, F. Vecchi, M. C. Carrozza, and R. S. Johansson, “Humans can integrate force feedback to toes in their sensorimotor control of a robotic hand,” _IEEE transactions on neural systems and rehabilitation engineering_ , vol. 17, no. 6, pp. 560–567, 2009.
  * [48] K. Kim, J. E. Colgate, J. J. Santos-Munné, A. Makhlin, and M. A. Peshkin, “On the design of miniature haptic devices for upper extremity prosthetics,” _IEEE/ASME Transactions On Mechatronics_ , vol. 15, no. 1, pp. 27–39, 2009.
  * [49] J. W. Sensinger, A. E. Schultz, and T. A. Kuiken, “Examination of force discrimination in human upper limb amputees with reinnervated limb sensation following peripheral nerve transfer,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 17, no. 5, pp. 438–444, 2009.
  * [50] P. D. Marasco, K. Kim, J. E. Colgate, M. A. Peshkin, and T. A. Kuiken, “Robotic touch shifts perception of embodiment to a prosthesis in targeted reinnervation amputees,” _Brain_ , vol. 134, no. 3, pp. 747–758, 2011.
  * [51] J. L. Pons, R. Ceres, E. Rocon, D. Reynaerts, B. Saro, S. Levin, and W. Van Moorleghem, “Objectives and technological approach to the development of the multifunctional manus upper limb prosthesis,” _Robotica_ , vol. 23, no. 3, pp. 301–310, 2005.
  * [52] A. Mondal and A. Kuzmanovic, “Upgrading mice to elephants: effects and end-point solutions,” _IEEE/ACM Transactions on Networking_ , vol. 18, no. 2, pp. 367–378, 2009.
  * [53] H. Xu, D. Zhang, J. C. Huegel, W. Xu, and X. Zhu, “Effects of different tactile feedback on myoelectric closed-loop control for grasping based on electrotactile stimulation,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 24, no. 8, pp. 827–836, 2015.
  * [54] M. R. Mulvey, H. J. Fawkner, H. E. Radford, and M. I. Johnson, “The use of transcutaneous electrical nerve stimulation (tens) to aid perceptual embodiment of prosthetic limbs,” _Medical hypotheses_ , vol. 72, no. 2, pp. 140–142, 2009.
  * [55] M. Franceschi, L. Seminara, S. Dosen, M. Strbac, M. Valle, and D. Farina, “A system for electrotactile feedback using electronic skin and flexible matrix electrodes: experimental evaluation,” _IEEE transactions on haptics_ , vol. 10, no. 2, pp. 162–172, 2016.
  * [56] G. Lundborg, B. Rosén, and S. Lindberg, “Hearing as substitution for sensation: a new principle for artificial sensibility,” _The Journal of hand surgery_ , vol. 24, no. 2, pp. 219–224, 1999.
  * [57] J. Gonzalez, H. Soma, M. Sekine, and W. Yu, “Psycho-physiological assessment of a prosthetic hand sensory feedback system based on an auditory display: a preliminary study,” _Journal of neuroengineering and rehabilitation_ , vol. 9, no. 1, pp. 1–14, 2012.
  * [58] F. Clemente and C. Cipriani, “A novel device for multi-modal sensory feedback in hand prosthetics: Design and preliminary prototype,” in _2014 IEEE Haptics Symposium (HAPTICS)_. IEEE, 2014, pp. 569–573.
  * [59] H. H. Ehrsson, B. Rosén, A. Stockselius, C. Ragnö, P. Köhler, and G. Lundborg, “Upper limb amputees can be induced to experience a rubber hand as their own,” _Brain_ , vol. 131, no. 12, pp. 3443–3452, 2008.
  * [60] A. Björkman, A. Weibull, J. Olsrud, H. Henrik Ehrsson, B. Rosén, and I. M. Björkman-Burtscher, “Phantom digit somatotopy: a functional magnetic resonance imaging study in forearm amputees,” _European Journal of Neuroscience_ , vol. 36, no. 1, pp. 2098–2106, 2012.
  * [61] J. P. Hunter, J. Katz, and K. D. Davis, “Dissociation of phantom limb phenomena from stump tactile spatial acuity and sensory thresholds,” _Brain_ , vol. 128, no. 2, pp. 308–320, 2005.
  * [62] H. Flor, T. Elbert, S. Knecht, C. Wienbruch, C. Pantev, N. Birbaumers, W. Larbig, and E. Taub, “Phantom-limb pain as a perceptual correlate of cortical reorganization following arm amputation,” _Nature_ , vol. 375, no. 6531, pp. 482–484, 1995.
  * [63] A. Farnè, A. C. Roy, P. Giraux, J. M. Dubernard, and A. Sirigu, “Face or hand, not both: perceptual correlates of reafferentation in a former amputee,” _Current Biology_ , vol. 12, no. 15, pp. 1342–1346, 2002.
  * [64] E. Pezent, A. Israr, M. Samad, S. Robinson, P. Agarwal, H. Benko, and N. Colonnese, “Tasbi: Multisensory squeeze and vibrotactile wrist haptics for augmented and virtual reality,” in _2019 IEEE World Haptics Conference (WHC)_. IEEE, 2019, pp. 1–6.
  * [65] T. Kameoka, Y. Kon, T. Nakamura, and H. Kajimoto, “Haptopus: haptic vr experience using suction mechanism embedded in head-mounted display,” in _The 31st Annual ACM Symposium on User Interface Software and Technology Adjunct Proceedings_ , 2018, pp. 154–156.
  * [66] K. Hiki, T. Okano, S. Sakurai, T. Nojima, M. Kitazaki, Y. Ikei, and K. Hirota, “Substitution of hand-object pressure cues with the sole of the foot for haptic presentation using a tactile pin array,” in _International Conference on Human Haptic Sensing and Touch Enabled Computer Applications_. Springer, 2018, pp. 239–251.
  * [67] P. Kieliba, D. Clode, R. O. Maimon-Mor, and T. R. Makin, “Robotic hand augmentation drives changes in neural body representation,” _Science robotics_ , vol. 6, no. 54, p. eabd7935, 2021.
  * [68] D. Finney, “Probit analysis (3rd edition),” 1971.
  * [69] A. Lécuyer, “Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback,” _Presence: Teleoperators and Virtual Environments_ , vol. 18, no. 1, pp. 39–53, 2009.
  * [70] T. Grechkin, J. Thomas, M. Azmandian, M. Bolas, and E. Suma, “Revisiting detection thresholds for redirected walking: Combining translation and curvature gains,” in _Proceedings of the ACM Symposium on Applied Perception_ , 2016, pp. 113–120.
  * [71] M. Slater, B. Spanlang, and D. Corominas, “Simulating virtual environments within virtual environments as the basis for a psychophysics of presence,” _ACM transactions on graphics (TOG)_ , vol. 29, no. 4, pp. 1–9, 2010.

|  Yutaro Hirao is an Assistant Professor at Nara Institute of Science and
Technology (NAIST), Japan (23-). His main research interests include virtual
reality (VR), cross-modal interaction, haptic perception (pseudo-haptics), and
embodiment. He received his B.S. and M.S. in engineering from Waseda
University (18-20) in Japan, and his Ph.D. in information science and
technology form the University of Tokyo (20-23).
---|---
|  Takuji Narumi is an associate professor at the Graduate School of
Information Science and Technology, the University of Tokyo. His research
interests broadly include perceptual modification and human augmentation with
virtual reality and augmented reality technologies. He received BE and ME
degree from the University of Tokyo in 2006 and 2008 respectively. He also
received his Ph.D. in Engineering from the University of Tokyo in 2011.
---|---
|  Ferran Argelaguet is an Inria research scientist at the Hybrid team
(Rennes, France) since 2016. He received his PhD degree from the Universitat
Politècnica de Catalunya (UPC), in Barcelona, Spain in 2011. His main research
interests include 3D user interfaces, virtual reality and human-computer
interaction. He was program co-chair of the IEEE Virtual Reality and 3D User
Interfaces conference track in 2019 and 2020, and the journal track in 2022.
---|---
|  Anatole Lécuyer is director of research and head of Hybrid team at Inria,
Renne, France. He is currently Associate Editor of IEEE Transactions on
Visualization and Computer Graphics, Frontiers in Virtual Reality and
Presence. He was Program Chair of IEEE VR 2015-2016 and General Chair of IEEE
ISMAR 2017. Anatole Lécuyer obtained the IEEE VGTC Technical Achievement Award
in Virtual/Augmented Reality in 2019.
---|---