# Quasi-compactons in inverted nonlinear photonic crystals

Yongyao Li1,2<EMAIL_ADDRESS>Boris A. Malomed3,4 Jianxiong Wu1 Wei Pang5
Sicong Wang2 Jianying Zhou2<EMAIL_ADDRESS>1Department of Applied
Physics, South China Agricultural University, Guangzhou 510642, China
2State Key Laboratory of Optoelectronic Materials and Technologies,
Sun Yat-sen University, Guangzhou 510275, China
3 Department of Physical Electronics, School of Electrical Engineering,
Faculty of Engineering, Tel Aviv University, Tel Aviv 69978, Israel
4 ICFO-Institut de Ciencies Fotoniques, Mediterranean Technology Park,08860
Castelldefels (Barcelona), Spain
5 Department of Experiment Teaching, Guangdong University of Technology,
Guangzhou 510006, China.

###### Abstract

We study large-amplitude one-dimensional solitary waves in photonic crystals
featuring competition between linear and nonlinear lattices, with minima of
the linear potential coinciding with maxima of the nonlinear pseudopotential,
and vice versa (inverted nonlinear photonic crystals, INPhCs), in the case of
the saturable self-focusing nonlinearity. Such crystals were recently
fabricated using a mixture of SU-8 and Rhodamine-B optical materials. By means
of numerical methods and analytical approximations, we find that large-
amplitude solitons are broad sharply localized stable pulses (quasi-
compactons, QCs). With the increase of the total power, $P$, the QC’s centroid
performs multiple switchings between minima and maxima of the linear
potential. Unlike cubic INPhCs, the large-amplitude solitons are mobile in the
medium with the saturable nonlinearity. The threshold value of the kick
necessary to set the soliton in motion is found as a function of $P$.
Collisions between moving QCs are considered too

###### pacs:

42.65.Tg; 42.70.Qs;05.45.Yv

##  I Introduction and the model

Photonic and matter waves propagating under the combined action of linear and
nonlinear lattice (LL and NL) potentials exhibit a plenty of dynamics non-
soliton . In particular, solitons in such media is a topic of considerable
current interest, see recent review RMP_Kar  and original works 1D ; Thaw
devoted to the studies of one-dimensional (1D) solitons. Other works were
dealing with solitons 2D ; Abd  and localized vortices 2Dvort  in two-
dimensional (2D) versions of such systems, which represent, in particular,
photonic crystal fibers PCF  and 2D photonic crystals PC  made of nonlinear
materials. The evolution of solitons in these systems obeys the nonlinear
Schrödinger equation (NLSE), in which the LL and NL are represented,
respectively, by a usual periodic potential and by a periodic
_pseudopotential_ LCQian ; RMP_Kar , which is induced by a spatially periodic
modulation of the local nonlinearity coefficient. In optics, the NL represents
the mismatch between the nonlinearity of the host material and the stuff
filling voids of the photonic-crystal-fibers structure, which may be air,
another solid material all-solid , or a liquid crystal LC . The same model, in
the form of the Gross-Pitaevskii equation, applies to matter waves in a Bose-
Einstein condensate (BEC) which is trapped in a combination of a linear
periodic potential, created by an optical OL  or magnetic magnetic  lattice,
and a pseudopotential lattice, that may be induced by a periodic modulation of
the local nonlinearity provided by a properly patterned RMP_Kar  external
magnetic FR-magn  or optical fields FR-magn .

Recently, a combination of _competing_ $\pi$-out-of-phase-juxtaposed LL and
NL, with maxima of the refractive index coinciding with minima of the local
strength of the self-focusing nonlinearity and vice versa, was considered in
several works Kar12 ; LYY1 ; LYY2 . This medium may be naturally called an
_inverted nonlinear photonic crystal_ (INPhC). It was reported that INPhCs
could be fabricated by means of the technique based on direct laser writing in
silica Blomer . It was also predicted that a similar setting can be created in
a virtual form, using the electromagnetically induced transparency acting on
dopant atoms periodically distributed in a passive matrix LYY2 . Due to the
competition between the LL and NL, solitons in INPhCs may feature specific
power-dependent properties, such as double symmetry breaking LYY2 . While
previous studies of INPhCs were dealing with the __ Kerr (cubic) nonlinearity,
in this work we consider solitary waves in the system with saturable
nonlinearity, which occurs in various optical media. The extension of the
analysis for this nonlinearity is natural, whilst solitons in INPhCs feature
the strong sensitivity to the power.

First, we propose an experiment setup to realize an INPhC with a saturable
nonlinearity. Recent works reported the creation of resonantly absorbing
waveguide arrays and imaginary-part photonic crystals, that feature a
spatially periodic modulation of the absorption coefficient, built on the
basis of the SU-8 polymer (a commonly used transparent negative photoresist)
doped with Rhodamine B (RhB, a dye featuring saturable absorption) MNF ; LJT ,
see Fig. 1 for a schematic setup.

Figure 1: (Color online) (a)The one-dimensional nonlinear photonic crystal,
with the blue and gray areas depicting the nonlinear and linear stripes,
respectively. (b) The wavelength dependence of the absorption coefficient and
refractive-index variation, $\alpha$ and $\Delta n$ (solid and dashed lines,
respectively) in the SU-8 - RhB mixture.

A similar pattern can be used for our purposes. In the paraxial approximation,
the light propagation in the array obeys the spatial-domain NLSE for the local
amplitude of the electromagnetic field, $u(z,x)$, where $z$ and $x$ are the
propagation distance and transverse coordinate:

$iu_{z}=-{\frac{1}{2k}}u_{xx}-{\frac{(k_{0}\delta
n+i{\alpha/2})R(x)}{1+|u|^{2}}}u.$ (1)

Here, $R(x)$ is the array’s structure function which is to be taken as per
Fig. 1, $k=k_{0}n$, where $n=1.62$ is the refractive index of pure SU-8,
$k_{0}=2\pi/\lambda$, and $\lambda$ is the wavelength of light in vacuum.
Further, $\alpha$ and $\delta n$ are the absorption coefficient and
_refractive index difference_ between the SU-8 - RhB mixture and pure SU-8,
respectively, which are coupled by the Kramers-Kronig relation, and are
plotted versus $\lambda$ in Fig. 1 LJT . According to experimental data LJT ,
in the relevant range of far blue shift from the absorption peak
($\lambda\approx 460$ nm), $\alpha/2\simeq(1/20)k_{0}\left|\delta n\right|$,
which allows one to neglect the dissipative term in Eq. (1), simplifying it to

$iu_{z}=-{\frac{1}{2}}u_{xx}+{\frac{V(x)}{1+|u|^{2}}}u,$ (2)

where $k$ was removed by rescaling of $x$, and $V(x)\equiv-\left(k_{0}\delta
n\right)R(x).$

In the system under consideration, light experiences strong saturable self-
focusing induced by RhB in the mixed material, while the nonlinearity of pure
SU-8 is negligible. On the other hand, the refractive index of the mixture is
smaller than in pure SU-8, i.e., $\delta n<0$ at $\lambda=460$ nm, hence
function $V(x)$ in Eq. (2) takes positive values, and the waveguide array
meets the condition of the $\pi$ phase shift between the spatial modulations
on the linear and nonlinear local characteristics of the medium, thus
realizing the INPhC with the saturable nonlinearity.

The objective of this work is to study the existence, stability, mobility, and
interactions of solitons in the INPhC model based on Eq. (2). In Sec. II, we
study properties of the solitons by means of numerical simulations and
analytical approximations. It will be demonstrated that they feature sharp
localization, i.e., a quasi-compacton (QC) shape. Following the increase of
the total power, $P$, the QC switches its position between maxima and minima
of the linear refractive index. In Sec. III, we study mobility of the QCs,
imposing the phase tilt onto them, i.e., suddenly multiplying the wave form by
$\exp(i\eta x)$. A critical tilt (alias kick), $\eta_{c}(P)$, beyond which the
compacton starts to move, is found. The dependence $\eta_{c}(P)$ features
variations with the same period in $P$ as the above-mentioned switching of the
quiescent solitons. Collisions between moving compactons are also studied in
Sec. III by means of direct simulations. The paper is concluded by Sec. IV.

##  II Quasi-compactons: numerical and analytical results

###  II.1 Numerical simulations

Modulation function $V(x)$ in Eq. (2) corresponding to Fig. 1 is a piecewise-
constant one, of the Kronig-Penny type. In this paper, we approximate $V(x)$
by the first term of its harmonic decomposition, assuming that contributions
of higher harmonics are negligible for sufficiently broad solitons:

$V(x)=\left(V_{0}/2\right)\left[1-\cos\left(2\pi x{/d}\right)\right],$ (3)

where $d$ and $V_{0}>0$ are the modulation period and depth. The scaling is
set by fixing $d\equiv 20$, which leaves $V_{0}$ as a free parameter. In this
section, we present numerical results for $V_{0}=0.02,~{}0.03$ and $0.04$,
which adequately illustrate the generic situation. Below, we focus on solitons
with a sufficiently large amplitude, as for small amplitudes the truncated
expansion of the saturable nonlinearity amounts to the previously studied Kerr
model Kar12 ; LYY1 ; LYY2 .

Typical numerical results for solitons solution at different values of the
total power, $P=\int_{-\infty}^{+\infty}|u(x)|^{2}dx$, are displayed in Fig.
2. The stationary profiles, presented in Figs. 2-2, were generated by dint of
the the imaginary-time-propagation method Chiofalo . It is concluded that the
profiles are strongly localized, making the solitons QCs (quasi-compactons) [a
solution fully localized within a finite (_compact_) interval of $y$ is
usually called a compacton Rosenau ; JHH ]. On the other hand, the soliton
broadens with the increase of $P$. At relatively small values of the total
power, e.g., $P=400$, the center of the soliton is located at a minimum of
$V(x)$ [Fig. 2]; then, at $P=800$ [Fig. 2], the position of the soliton is
switched to a maximum of $V(x)$, and at $P=2000$ it switches back to the
minimum [Fig. 2].

Figure 2: (Color online) Numerically found soliton profiles (blue solid lines)
for the modulation function (shown by the black solid line) with depth
$V_{0}=0.02$ (recall the modulation period is scaled to be $d\equiv 20$). (a)
At total power $P=400$, the center of the soliton is located at the minimum of
$V(x)$. (b) At $P=800$, the soliton shifts to the maximum of $V(x).$ (c) At
$P=2000$, the solitons shifts back to the minimum of $V(x)$. Panels (d), (e)
and (f) illustrate the stability of the solitons via direct simulations of
their perturbed evolution.

To quantify power-dependent properties of the solitons, we define the center-
of-mass coordinate and average width of the soliton as follows:

$\displaystyle
X_{\mathrm{mc}}(P)={P}^{-1}\int_{-\infty}^{+\infty}x|U(x,P)|^{2}dx$
$\displaystyle
W_{\mathrm{a}}(P)={P}^{-1}\int_{-\infty}^{+\infty}(x-x_{mc})^{2}|U(x,P)|^{2}dx,$
(4)

where $U(x,P)$ is the soliton solution with power $P$. Using the integration
in imaginary time, the data were collected for the solitons from $P=100$ to
$10000$, with modulation depths $V_{0}=0.02,~{}0.03$ and $0.04$. Real-time
simulations of the evolution of these solutions in the framework of Eq. (2)
demonstrate that they all are stable against perturbations.

Numerically found dependences (4) are displayed in Fig. 3. The upper panels,
Fig. 3(a-c), show that the soliton’s center of mass switches (as said above)
between the minimum ($x=0$) and maximum ($x=-10$) of the modulation function
$V(x)$. The period of the switching, $\Delta P$, increases with the increase
of modulation depth $V_{0}$. Further, Figs. 3-2 show that the average width of
the soliton grows with the increase of $P$ and decrease of $V_{0}$.

Figure 3: (Color online) The plots in panels (a1)-(c1) display the coordinate
of the soliton’s center of mass, $X_{\mathrm{mc}}$, as a function of $P$, for
$V_{0}=0.02,~{}0.03$ and $0.04$, respectively. (a2)-(c2). The critical kick
which initiates the motion of the soliton, $\eta_{\mathrm{c}}$, as a function
of $P$ for $V_{0}=0.02,~{}0.03$ and $0.04$. (d)-(f) The average width of the
soliton as a function of $P$ for $V_{0}=0.02,0.03$ and $0.04$.

###  II.2 Analytical approximations

In saturable nonlinear media, large-amplitude solitons are always broad.
Accordingly, to develop an analytical approximation, we assume a soliton with
the width much larger than $d$. Then, in the zero-order approximation, Eq. (2)
is replaced by the equation with the averaged potential, $V(x)\rightarrow
V_{0}/2$,

$iu_{z}=-\frac{1}{2}u_{xx}+\frac{V_{0}}{2\left(1+|u|^{2}\right)}u.$ (5)

Equation (5) gives rise to soliton solutions, $u(z,x)=e^{ikz}U(x-\xi)$, with
the coordinate of the soliton’s center $\xi$, and real function $U(y)$ obeying
the following equation:

$\frac{d^{2}U}{dy^{2}}=2kU+\frac{V_{0}U}{1+U^{2}}\equiv-\frac{dW}{dU},$ (6)

where $y\equiv x-\xi$. The effective potential in Eq. (6),

$W(U)=\allowbreak
kU^{2}+\frac{V_{0}}{2}\ln\left(\frac{1+U_{0}^{2}}{1+U^{2}}\right),$ (7)

formally corresponds to the Newton’s equation of motion for a particle with
coordinate $U(y)$ ($y$ plays the role of time) and unitary mass. Then, the
shape of the soliton is determined by the respective energy equation,

$\left(\frac{dU}{dy}\right)^{2}+2k\left(U_{0}^{2}-U^{2}\right)+V_{0}\ln\left(\frac{1+U_{0}^{2}}{1+U^{2}}\right)=0,$
(8)

where $U_{0}$ is the amplitude of the soliton (the largest value of $U$). As
usual, the soliton trajectory corresponds to the solution of Eq. (8) starting
from $U=0$ at $y=-\infty$, bouncing back from the potential well at $U=U_{0}$,
$y=0$, and returning to $U=0$ at $y\rightarrow+\infty$. Setting in Eq. (8)
$U=dU/dy=0$, one can find a relation between $k$ and $U_{0}$. Being interested
in solutions with large amplitudes, we assume here $U_{0}^{2}\gg 1$, which
yields

$k\approx-\left(V_{0}/2\right)U_{0}^{-2}\ln\left(U_{0}^{2}\right),$ (9)
$U_{\min}^{2}\approx U_{0}^{2}/\ln\left(U_{0}^{2}\right),$ (10)

$U_{\min}$ being the coordinate of the minimum of potential (7). Equation (10)
implies $U_{\min}^{2}\ll U_{0}^{2}$, i.e., the minimum of the potential is
located much closer to $U=0$ than the largest value $U_{0}$. This fact suggest
a possibility to use the following approximation for solving Eq. (8): at the
first stage of the approximation, we drop the logarithmic term in Eq. (8)
altogether, i.e., we replace the equation by its “primitive version”,

$\left(\frac{dU}{dy}\right)^{2}+2k\left(U_{0}^{2}-U^{2}\right)=0,$ (11)

whose obvious solution is

$U(y)=U_{0}\cos\left(\sqrt{-2k}y\right)\approx
U_{0}\cos\left(\frac{\sqrt{V_{0}\ln\left(U_{0}^{2}\right)}}{U_{0}}y\right),$
(12)

where Eq. (9) was used to replace $k$ by the $U_{0}$.

At the second stage of the approximation, we recall that the particle does not
perform periodic oscillations, as formally follows from the expression (12),
but it starts the motion from $U=0$ at $y=-\infty$, and returns to $U=0$ at
$y\rightarrow+\infty$, as the soliton solution must do, see above. This means
that approximation (12) is usable in interval

$|y|~{}<\frac{L_{\mathrm{tot}}}{2}\equiv\frac{\pi}{2\sqrt{-2k}}=\frac{\pi
U_{0}}{2\sqrt{V_{0}\ln\left(U_{0}^{2}\right)}},$ (13)

When the moving particle approaches edges of interval (13), the logarithmic
term in Eq. (8), which was neglected in Eq. (11), “suddenly” becomes
important, leading to the stoppage of the particle. Thus, the full
approximation in the large-amplitude limit, $U_{0}\gg 1$, amounts to
representing the soliton as the QC (quasi-compacton):

$U(y)\approx\left\\{\begin{array}[]{c}U_{0}\cos\left(\frac{\sqrt{V_{0}\ln\left(U_{0}^{2}\right)}}{U_{0}}y\right),~{}\mathrm{at}~{}~{}|y|~{}<L_{\mathrm{tot}}/2,\\\
0,~{}\mathrm{at}~{}~{}|y|~{}>L_{\mathrm{tot}}/2.\end{array}\right.$ (14)

where $L_{\mathrm{tot}}$, given by Eq. (13), plays the role of the total width
of the QC. Note that the large amplitude $U_{0}$ of the soliton implies that
width (13) is also large, i.e., the QC is a sharply localized but wide
localized pattern. In the present approximation, the total power of the
soliton can be calculated as follows:

$P\equiv\int_{-\infty}^{+\infty}\left|U(y)\right|^{2}dy\approx\frac{1}{2}L_{\mathrm{tot}}U_{0}^{2}=\frac{\pi
U_{0}^{3}}{2\sqrt{V_{0}\ln\left(U_{0}^{2}\right)}}.$ (15)

Further, using the fact that $P$ and $U_{0}^{2}$ are large, Eq. (15) can be
approximately inverted, to give the peak power (squared amplitude) of the QC
as a function of its total power:

$U_{0}^{2}\approx\left(\frac{2P}{\pi}\sqrt{\frac{2V_{0}}{3}\left(\ln
P\right)}\right)^{2/3}.$ (16)

Next,using Eq. (13), it is also possible to approximately express the full
width of the compacton in terms of the total power:

$L_{\mathrm{tot}}\approx\left(\frac{3\pi^{2}}{V_{0}}\frac{P}{\ln
P}\right)^{1/3}.$ (17)

Figure 4: (Color online) (a) The comparison of the numerical results for the
peak power of the quasi-compactons with the analytical prediction given by Eq.
(16). (b) The full width of the quasi-compacton, as predicted by Eq. (17). In
this figure, $V_{0}=0.02$.

Relations (16) and (17) are plotted in Fig. 4 for different values of $P$. In
particular, Fig. 4(a) demonstrates that the analytical prediction for the peak
power is consistent with the numerical results. On the other hand, the values
of the width given by Eq. (17) [see Fig. 4(b)] are larger than their
numerically found counterparts presented in Fig. 3(d). However, one should
take into regard that width (17) is “all-inclusive” (total), while the width
shown in 3(b) is the average one (definitely far smaller than the total
width). Note also the similarity in the $P$-dependence of the width in Figs.
3(d) and 4(b).

To explain the position switchings of the QC, one may use the effective energy
(potential) of the interaction of the QC with the spatial modulation,
corresponding to the term $\sim\cos\left(2\pi x/d\right)$ in expression (3).
This energy can be defined as follows, with regard to the definition of
$y\equiv x-\xi$ in Eq. (6):

$\displaystyle
E_{\mathrm{int}}(\xi)=-V_{0}\int_{-\infty}^{+\infty}dx\cos\left(\frac{2\pi
x}{d}\right)\ln\left[1+U^{2}(x)\right]$ (18) $\displaystyle\approx$
$\displaystyle-V_{0}\int_{\xi-
L_{\mathrm{tot}}/2}^{\xi+L_{\mathrm{tot}}/2}dx\cos\left(\frac{2\pi
x}{d}\right)\ln\left[1+U^{2}(x)\right].$

In this expression, it is taken into account that QC (14) occupies,
approximately, a finite region of $x$, $\xi-
L_{\mathrm{tot}}/2<x<\xi+L_{\mathrm{tot}}/2$. After three integrations by
parts, expression (18) can be approximately calculated as follows,

$E_{\mathrm{int}}(\xi)\approx
K_{d}\ln(U_{0}^{2})\sin\left(\frac{\pi^{2}U_{0}}{d\sqrt{V_{0}\ln\left(U_{0}^{2}\right)}}\right)\cos\left(\frac{2\pi}{d}\xi\right),$
(19)

where $K_{d}\equiv V_{0}^{2}d^{3}/\pi^{3}$. Expression (19) predicts that the
minimum of the interaction energy is located at points $\xi=nd$ with integer
$n$ (at minima of the modulation function) for values of $U_{0}$ such that
$\sin\left(\pi^{2}U_{0}/\left[d\sqrt{V_{0}\ln\left(U_{0}^{2}\right)}\right]\right)<0$,
and at points $\xi=\left(n+1/2\right)d$ (at maxima of the modulation) in the
opposite case,
$\sin\left(\pi^{2}U_{0}/\left[d\sqrt{V_{0}\ln\left(U_{0}^{2}\right)}\right]\right)>0$.
Because $U_{0}$ grows as the function of $P$, as per Eq. (16), this means
that, with the increase of $P$, the position of the energy minimum must indeed
switch between adjacent extrema of the modulation function. The fact that
$V_{0}$ appears in the denominator of the argument of the sinusoidal factor
explains why the period of the switching increases with $V_{0}$.

##  III Mobility of the quasi-compactons

###  III.1 Driving the soliton by the phase tilt

Mobility of solitons under the combined action of LL and NL is a problem which
was considered in a number of different settings Sakaguchi ; Zhou ; Kar3 ;
Oster ; RMP_Kar . To set the QC in motion, we follow the standard approach,
suddenly kicking a quiescent one, i.e., multiplying the respective solution,
$U(x)$, by the phase-tilt factor: $U(x,\eta)\rightarrow\exp(i\eta x)U(x)$,
where $\eta$ is the strength of kick.

The simulated propagation of the QC, induced by kicking the standing one,
$U(x,P=2000)$, is displayed in Fig. 5, for two different values of $\eta$. It
is observed that $\eta=0.020(\pi/d)$ causes only oscillations of the soliton,
without any progressive motion. However, a slightly higher kick,
$\eta=0.025(\pi/d)$, is sufficient to set the same soliton in persistent
motion. These results imply that there must be a threshold (critical) value of
the kick, $\eta_{\mathrm{c}}$, such that the kick with
$\eta>\eta_{\mathrm{c}}$ makes the given QC a traveling soliton. For $P=2000$,
it is $\eta_{\mathrm{c}}=0.021\left(\pi/d\right)$.

Figure 5: (Color online) Direct simulations of the compacton with $P=2000$,
initiated by the application of the phase tilt with $\eta=0.020(\pi/d)$ (a)
and $\eta=0.025(\pi/d)$ (b). The lattice depth is $V_{0}=0.02$.

The critical kick for the QCs is shown as a function of the total power in
Figs. 3(a2)-3(c2), for $V_{0}=0.02,0.03$, and $0.04$. The general trend to the
decrease of $\eta_{\mathrm{c}}$ with the increase of $P$ is explained by the
broadening of the soliton with the increase of the total power in the case of
the saturable nonlinearity, and ensuing enhancement of the solitons’ mobility
Oster . A specific feature of the INPhC is that oscillations of
$\eta_{\mathrm{c}}$ are superimposed on top of the gradual decay of the
critical kick, the period of the oscillations coinciding with that of the
switching of the center-of-mass position. Figures 3-3 also demonstrate that
$\eta_{\mathrm{c}}$ decreases with the decreases of modulation depth $V_{0}$.
The latter trend is correlated with the fact that, as seen in Figs. 3-3, the
width of the solitons increases, making them more mobile, with the decrease of
$V_{0}$ (as well as with the increase of $P$).

Further, Fig. 6 shows that the momentum of the moving soliton, which is
defined as

$\displaystyle\langle p(z)\rangle=-i\int dxU(x,z,\eta)^{\ast}U_{x}(x,z,\eta),$
(20)

is, naturally, a linear function of the initial kick, $\eta$. Further
simulations (not shown here) demonstrate that the QC are _not_ destroyed even
by a very strong kick, far exceeding $\eta_{\mathrm{c}}$.

Figure 6: The momentum of the soliton with $P=2000$, measured at $z=1000$, as
a function of initial kick $\eta$.

It is relevant to compare the mobility of large-amplitude solitons in the
INPhCs models with the saturable and cubic self-focusing. In the latter case,
Eq. (2) is replaced by Thaw

$iu_{z}=-{\frac{1}{2}}u_{xx}+V(x)(1-|u|^{2})u,$ (21)

where we chose the modulation function, $V(x)$, in the same form as in Eq.
(3), with $V_{0}=0.02$ and $d=20$. A typical example of simulations of the
mobility of solitons in this model is displayed in Figs. 7(a)-(c), for three
kicks, $\eta=0.15{\pi/d},0.25{\pi/d}$, and $0.35{\pi/d}$. In particular, Fig.
7 shows that $\eta=0.15{\pi/d}$ give rise to oscillations of the soliton,
without depinning. Further, it is observed in Fig. 7 that the stronger kick,
$\eta=0.25{\pi/d}$, transforms the soliton into an oscillatory beam with a
fuzzy shape, but, still, it was not set in motion. Finally, Fig. 7
demonstrates that the strongest kick, $\eta=0.35{\pi/d}$, completely destroys
the soliton (recall the destruction never happens in the model with the
saturable nonlinearity). Therefore, there must exist critical values of the
kick separating the stable solitons, fuzzy beams, and complete destruction.
These borders are plotted in the top panel of Fig. 7, in the range of powers
$50<P<100$ (the average width of the static soliton in the same region is
plotted, versus $P$, in the bottom panel). Thus, the large-amplitude solitons
in the INPhC model with the cubic self-focusing are not mobile at all. On the
other hand, for dynamics of small-amplitude solitons in the models with the
saturable and cubic nonlinearities are similar, due to the obvious expansion,
$\left(1+|u|^{2}\right)^{-1}\approx 1-|u|^{2}$ (results for this case are not
shown here in detail, as they are less interesting).

Figure 7: (Color online) (a) Direct simulations of the evolution of the
soliton with total power $P=80$ in the model with the cubic nonlinearity [Eq.
(21)], initiated by the kick with strengths $\eta=0.15(\pi/d)$ (a),
$0.25(\pi/d)$ (b), and $\eta=0.35(\pi/d)$ (c). (d) The top plot:
$\eta_{1\mathrm{c}}$ is the border between stable solitons and fuzzy beams;
$\eta_{2\mathrm{c}}$ is the border between the fuzzy beams and completely
destroyed ones. The bottom plot shows the average width of the quiescent
soliton versus its total power $P$.

###  III.2 Collisions between moving quasi-compactons

Collisions is a natural way to test interactions between solitons. Here we
present some numerical results for collisions between QCs in the INPhC with
the saturable nonlinearity. The initial state at $z=0$ is taken as a set of
two far separated kicked solitons:

$u(x)=U(x+x_{0},P)e^{i\eta_{1}(x+x_{0})}+U(x-x_{0},P)e^{-i\eta_{2}(x-x_{0})},$
(22)

where $U(x\pm x_{0},P)$ are the QC pulses with power $P$, which are centered
at $x=\mp x_{0}$, and $\eta_{1,2}$ are the kicks applied to them. Simulations
of Eq. (2) with initial conditions (22) are displayed in Fig. 8.

Figure 8: (Color online) Direct simulation of the collisions initiated by
kicks $\eta_{1}=0$, $\eta_{2}=0.1(\pi/d)$ (a); $\eta_{1}=0$,
$\eta_{2}=0.2(\pi/d)$ (b); $\eta_{1}=0.1(\pi/d)$, $\eta_{2}=0.1(\pi/d)$ (c);
$\eta_{1}=0.2(\pi/d)$, $\eta_{2}=0.2(\pi/d)$ (d). Other parameters are
$P=400$, $x_{0}=120$, and $V_{0}=0.02$. As in the rest of the paper, the
scaling is fixed by $d\equiv\allowbreak 20$.

The simulations demonstrate the repulsive interaction between the colliding
QC. In Figs. 8 and 8, they bounce back without overlapping, because the
collision velocity is small (nevertheless, one of the solitons suffers a
conspicuous perturbation in Fig. 8). In Figs. 8 and 8, the larger velocities
give rise to the overlap between the colliding QCs, which results in the
strong inelasticity of the collision in the case displayed in Fig. 8.

##  IV Conclusions

The objective of this work was to consider 1D solitons in the INPhC (inverted
nonlinear photonic crystal) with competing LL and NL (linear and nonlinear
lattices). Unlike recently studied models with the cubic nonlinearity, we here
consider the saturable self-focusing. The 1D crystal of this type was recently
fabricated, using the SU-8 polymer material periodically doped with Rhodamine
B, which lends the medium the saturable nonlinearity. Combining numerical
methods and analytical approximations, we have demonstrated that broad
solitons are sharply localized in this setting, thus taking the shape of the
QCs (quasi-compactons). With the increase of the total power, $P$, the QC
remains a stable object, which switches its position between the linear and
nonlinear layers, which form the INPhC. The width of the soliton increases
with $P$, which is a manifestation of the saturable nonlinearity. The large
width of the QC makes it a mobile object, unlike solitons in the INPhC with
the cubic nonlinearity. The threshold value of the transverse kick (phase
tilt), which is necessary to set the QC in motion, was found as a function of
$P$. The threshold value gradually decays with the increase of $P$. Collisions
between two moving QCs were are also studied, by means of direct simulations.

The results reported in this work may be applied to the design of all-optical
data-processing schemes. In particular, the power-controlled switch of the
spatial soliton between adjacent layers, as well as the high mobility of these
solitons, may be quite relevant properties, in this context.

This work may be extended in other directions. It particular, 2D waveguide
arrays, based on the RhB-SU-8 mixtures, can be fabricated MNF ; LJT , which
suggests to consider spatial solitons in two-dimensional INPhC with the
saturable nonlinearity.

Y.L. appreciates useful discussions with Dr. M. Feng. This work was supported
by the Chinese agencies NKBRSF (grant No. G2010CB923204), NSFC(grant No.
11104083) and CNNSF(grant No. 10934011).

## References

  * (1) Q. Li, C. T. Chan, K. M. Ho, and C. M. Soukoulis, Phys. Rev. B 53, 15577 (1996); E. Lidorikis, Q. Li, and C. M. Soukoulis, ibid. 54, 10249 (1996); D. Hennig and G. P. Tsironis, Phys. Rep. 307, 333 (1999); B. Maes, P. Bienstman, and R. Baets, J. Opt. Soc. Am. B 22, 613 (2005).
  * (2) Y. V. Kartashov, B. A. Malomed, and L. Torner, Rev. Mod. Phys. 83, 247 (2011).
  * (3) A. A. Sukhorukov and Y. S. Kivshar, Phys. Rev. Lett. 87, 083901 (2001); Y. Kominis and K. Hizanidis, Opt. Lett. 31, 2888 (2006); Y. Kominis, Phys. Rev. E. 73, 06619 (2006); Y. Kominis and K. Hizanidis, Opt. Exp. 16, 12124 (2008); Z. Rapti, P. G. Kevrekidis, V. V. Konotop, and C. K. R. T. Jones, J. Phys. A., 40, 14151 (2007).
  * (4) T. Mayteevarunyoo and B. A. Malomed, J. Opt. Soc. Am. B 25, 1854 (2008).
  * (5) P. Xie, Z.-Q. Zhang, and X. Zhang, Phys. Rev. E 67, 026607 (2003); A. Ferrando, M. Zacarés, P. F. de Córdoba, D. Binosi, and J. A. Monsoriu, Opt. Exp. 11, 452 (2003).
  * (6) H. L. F. da Luz, F. Kh. Abdullaev, A. Gammal, M. Salerno, and L. Tomio, Phys. Rev. A 82, 043618 (2010).
  * (7) A. Ferrando, M. Zacarés, P. F. de Córdoba, D. Binosi, and J. A. Monsoriu, Opt. Exp. 12, 817 (2004); Y. V. Kartashov, A. Ferrando, A. A. Egorov, and L. Torner, Phys. Rev. Lett. 95, 123902 (2005).
  * (8) P. St. J. Russell, J. Lightwave Tech. 24, 4729 (2006); S. Arismar Cerqueira, Jr., Rep. Prog. Phys. 73, 024401 (2010).
  * (9) J. D. Joannopoulos, S. G. Johnson, J. N. Winn, and R. D. Meade, Photonic Crystals: Molding the Flow of Light (Princeton University Press: Princeton, 2008); M. Skorobogatiy and J. Yang, Fundamentals of Photonic Crystals Guiding (Cambridge University Press: Cambridge, 2009).
  * (10) L. C. Qian, M. L. Wall, S. Zhang, Z. Zhou, and H. Pu, Phys. Rev. A 77, 013611 (2008); T. Mayteevarunyoo, B. A. Malomed, and G. Dong, ibid. 78, 053601 (2008).
  * (11) F. Luan, A. K. George, T. D. Hedley, G. J. Pearce, D. M. Bird, J. C. Knight, and P. St. J. Russell, Opt. Lett. 29, 2369 (2004); G. Bouwmans, L. Bigot, Y. Quiquempois, F. Lopez, L. Provino, and M. Douay, Opt. Exp. 13, 8452 (2005); A. Fuerbach, P. Steinvurzel, J. A. Bolger, A. Nulsen, and B. J. Eggleton, Opt. Lett. 30, 830 (2005).
  * (12) T. T. Larsen, A. Bjarklev, D. S. Hermann, and J. Broeng, Opt. Exp. 11, 2589 (2003); F. Du , Y. Q. Lu, and S. T. Wu, Appl. Phys. Lett. 85, 2181 (2004); M. W. Haakestad, T. T. Alkeskjold, M. D. Nielsen, L. Scolari, J. Riishede, H. E. Engan, and A. Bjarklev, IEEE Phot. Tech. Lett. 17, 819 (2005); C. R. Rosberg, F. H. Bennet, D. N. Neshev, P. D. Rasmussen, O. Bang, W. Królikowski, A. Bjarklev, and Y. S. Kivshar, Opt. Exp. 15, 12145 (2007).
  * (13) D. Jaksch, C. Bruder, J. I. Cirac, C. W. Gardiner, and P. Zoller, Phys. Rev. Lett. 81, 3108 (1998); M. Greiner, O. Mandel, T. Esslinger, T. W. Hansch, and I. Bloch, Nature 415, 39 (2002).
  * (14) S. Ghanbari, T. D. Kieu, A. Sidorov, and P. Hannaford, J. Phys. B 39, 847 (2006).
  * (15) S. Inouye, M. R. Andrews, J. Stenger, H. J. Miesner, D. M. Stamper-Kurn, and W. Ketterle, Nature 392, 151 (1998); S. L. Cornish, N. R. Claussen, J. L. Roberts, E. A. Cornell, and C. E. Wieman, Phys. Rev. Lett. 85, 1795 (2000); E. Timmermans, P. Tommasini, M. S. Hussein, and A. Kerman, Phys. Rep. 315, 199 (1999).
  * (16) P. O. Fedichev, Y. Kagan, G. V. Shlyapnikov, and J. T. M. Walraven, Phys. Rev. Lett. 77, 2913 (1996); M. Theis, M., G. Thalhammer, K. Winkler, M. Hellwig, G. Ruff, R. Grimm, and J. H. Denschlag, Phys. Rev. Lett. 93, 123001 (2004).
  * (17) Y. V. Kartashov, V. A. Vysloukh, and L. Torner, Opt. Lett., 33, 1747 (2008); ibid. 33, 2173 (2008).
  * (18) Y. Li, B. A. Malomed, M. Feng, and J. Zhou, Phys. Rev. A, 82, 063813 (2010); C. Hang and V. V. Konotop, _ibid._ 81, 053849 (2010). J. Wu, M. Feng, W. Pang, S. Fu and Y. Li, J. Nonlinear Opt. Phys. (JNOP), 22, 193(2011).
  * (19) Y. Li, B. A. Malomed, M. Feng, and J. Zhou, Phys. Rev. A, 83, 053823 (2011). J. Wu, M. Feng, W. Pang, S. Fu, and Y. Li, J. Nonlinear Opt. Phys. (JNOP), 20, 193 (2011).
  * (20) A. Szameit,J. Burghoff, T. Pertsch, S. Nolte, A. Tünnermann, and F. Lederer, Opt. Exp., 14, 6055 (2006). D. Blömer, A. Szameit, F. Dreisow, T. Pertsch, S. Nolte, and A. Tünnermann, Opt. Exp., 14, 2151 (2006).
  * (21) M. Feng, Y. Liu, Y. Li, X. Xie, and J. Zhou, Opt. Exp. 19, 7222 (2011).
  * (22) J. Li, B. Liang, Y. Liu, P. Zhang, J. Zhou, S. O. Klimonsky, A. S. Slesarev, Y. D. Tretyakov, L. O’Faolain, and T. F. Krauss, Adv. Mater. 22, 1 (2010).
  * (23) M. L. Chiofalo, S. Succi, and M. P. Tosi, Phys. Rev. E, 62, 7438 (2000). J. Yang, and T. I. Lakoba, Stud. Appl. Math., 120, 265 (2008). J. Yang, and T. I. Lakoba, Stud. Appl. Math., 118, 153 (2007). Y. Li, W. Pang, Y. Chen, Z. Yu, J. Zhou, and H. Zhang, Phys. Rev. A., 80, 043824 (2009).
  * (24) P. Rosenau and J. M. Hyman, Phys. Rev. Lett. 70, 564 (1993).
  * (25) J. He and X. Wu, Chaos, Sol. Fractals 29, 108 (2006).
  * (26) H. Sakaguchi, and B. A. Malomed, Phys. Rev. E., 70, 066613 (2005).
  * (27) J. Zhou, C. Xue, Y. Qi, and S. Lou, Phys. Lett. A. 372, 4395 (2008).
  * (28) Y. V. Kartashov, V. A. Vysloukh, and L. Torner, Prog. Opt., 52, 63 (2009).
  * (29) M. Öster, M. Johansson and A. Eriksson, Phys. Rev. E. 67, 056606 (2003); R. Vicencio and M. Johansson, ibid. 73, 046602 (2006); U. Naether, R. A. Vicencio, and M. Johansson, ibid. 83, 036602 (2011).


# Trust-Aware Motion Planning for Human-Robot Collaboration under Distribution
Temporal Logic Specifications

Pian Yu1, Shuyang Dong2, Shili Sheng2, Lu Feng2, and Marta Kwiatkowska1 *This
work was supported in part by the ERC ADG FUN2MODEL (Grant agreement ID:
834115), NSF grant CCF-1942836, and AFOSR grant FA9550-21-1-0164. 1Pian Yu and
Marta Kwiatkowska are with the Department of Computer Science, University of
Oxford, Oxford, United Kingdom pian.yu<EMAIL_ADDRESS>Dong, Shili Sheng, and Lu Feng are with School of Engineering, University of
Virginia, United States sd3mn, ss7dr<EMAIL_ADDRESS>
###### Abstract

Recent work has considered trust-aware decision making for human-robot
collaboration (HRC) with a focus on model learning. In this paper, we are
interested in enabling the HRC system to complete complex tasks specified
using temporal logic that involve human trust. Since human trust in robots is
not observable, we adopt the widely used partially observable Markov decision
process (POMDP) framework for modelling the interactions between humans and
robots. To specify the desired behaviour, we propose to use syntactically co-
safe linear distribution temporal logic (scLDTL), a logic that is defined over
predicates of states as well as belief states of partially observable systems.
The incorporation of belief predicates in scLDTL enhances its expressiveness
while simultaneously introducing added complexity. This also presents a new
challenge as the belief predicates must be evaluated over the continuous
(infinite) belief space. To address this challenge, we present an algorithm
for solving the optimal policy synthesis problem. First, we enhance the belief
MDP (derived by reformulating the POMDP) with a probabilistic labelling
function. Then a product belief MDP is constructed between the
probabilistically labelled belief MDP and the automaton translation of the
scLDTL formula. Finally, we show that the optimal policy can be obtained by
leveraging existing point-based value iteration algorithms with essential
modifications. Human subject experiments with 21 participants on a driving
simulator demonstrate the effectiveness of the proposed approach.

##  I Introduction

Autonomous robots are rapidly evolving into an essential component of our
society, to mention home assistive robots [1] and automated vehicles (AV) [2].
Despite significant advances made in automation in recent years, attaining
full autonomy, which would enable robots to successfully deal with complicated
and unpredictable events or situations, remains stubbornly out of reach. For
instance, the AV industry has had to reset expectations as it shifts its focus
from level 5 to 4 autonomy [3]. In many applications where robots work with or
alongside humans, it is customary to have robots that are operated or
supervised by a human operator. Collaborative human-robot partnerships often
hinge on the foundation of trust. Therefore, recognizing human trust and
incorporating it into the decision-making process is essential for achieving
the full potential of human-robot interactive systems.

The subject of trust has been actively studied in multiple contexts such as
psychology [4] and automation [5]. It is a multifaceted concept that can be
influenced by a large number of factors. In the context of human-robot
collaboration (HRC), studies have shown that the level of human trust in
robots evolves over their interaction, affected by factors such as the
automation’s reliability, predictability, and transparency [6]. While earlier
work has focused on studying the measurement [7], modelling [8], and
calibration [9] of human trust in robots, recent work has gravitated towards
devising strategies that enable robots to proactively infer and influence the
human collaborator’s trust [10, 11].

Various methods exist for modeling the interaction between humans and robots.
Among these, the game-theoretic approaches [12, 13] and the partially
observable Markov decision process (POMDP) framework [14, 15] have been
extensively explored. Since trust is not fully observable, in this work we
adopt the POMDP, where human trust can be modelled as a hidden variable. While
the POMDP formulation allows the robot to act according to its beliefs about
the human collaborator’s trust based on observations, finding solutions to
POMDPs of a realistic size is computationally challenging and existing work
often relies on approximation algorithms [16, 17, 18]. Due to the inherent
complexity of solving a POMDP, prior work devoted to trust-based decision
making for HRC often focused on relatively simple specifications (e.g.,
accumulated reward maximisation) [19, 20]. Moreover, in all these studies,
trust was treated as an implicit factor that impacts the performance of
collaboration. None of these works has considered trust as part of the
specification, where explicit requirements can be imposed. Real-world case
studies have shown that an inappropriate level of trust may result in the
misuse or disuse of automation [5]. Therefore, in practical scenarios, it
might be advantageous to stipulate conditions such as “the trust level must
not fall below a certain threshold” and “the trust level at a particular
juncture must surpass a certain threshold”.

In this work, we investigate trust-aware motion planning for HRC with complex
temporal logic specifications applied to both the state of the robot and the
trust (belief) of human. The trust-based human-robot interaction is modelled
by a trust POMDP and syntactically co-safe linear distribution temporal logic
(scLDTL) [21] is utilised to specify the desired behaviour of the system.
Syntactically co-safe linear temporal logic (scLTL) is a commonly employed
logic in robotics to specify the intended behaviour of a robot [22, 23]. In
[21], scLDTL was introduced as an extension of scLTL to leverage the richness
of information contained within belief states of partially observable systems.
It was shown that scLDTL is capable of expressing properties involving
uncertainty and likelihood that cannot be described by existing logic.
Nevertheless, the increased complexity introduced by the inclusion of belief
predicates in scLDTL, which must be evaluated over the continuous (infinite)
belief space, renders verification and synthesis from scLDTL a more demanding
task. In [21], a feasibility checking algorithm was proposed for POMDPs with
scLDTL specifications. However, to the best of our knowledge, the more
challenging synthesis problem remains unresolved. Our contributions are
summarised as follows: i) We demonstrate the suitability of scLDTL for
specifying the desired behaviour of trust-aware HRC systems that involve
requirements in the robot workspace as well as the trust (belief) space. ii)
We propose an efficient algorithm to solve the scLDTL optimal policy synthesis
for trust POMDPs, which overcomes the aforementioned complexity of scLDTL
specifications. iii) We design and conduct human subject experiments with 21
participants on a driving simulator to evaluate the proposed approach, with
encouraging results.

##  II Motivating example

Figure 1: Route planning for AVs: a) A map with three types of road incidents
(pedestrian, obstacle, and oncoming truck). b) A schematic view of the
decision making process.

We describe a route planning problem for AVs. A human is driving an AV in a
town, whose map is shown in Fig. 1(a). Within the town, we consider that there
are 3 types of typical incidents that may occur on the road: (1) a pedestrian
crossing the road, (2) an obstacle (e.g., a broken bicycle) ahead of the lane,
and (3) an oncoming truck in the neighbouring lane. For simplicity, here we
assume that there is at most one incident at a time for each road segment.

Fig. 1(b) shows a schematic view of the AV traveling from one location to
another. Imagine the AV is approaching an incident on a road segment while in
autopilot mode. For safety considerations, the driver might choose to take
over control of the AV and switch to manual driving. The level of trust that
the driver has in the AV’s ability to handle various types of incidents can
influence their takeover decision; a driver with lower trust is more inclined
to do so. Furthermore, the driver’s level of trust changes over time and
depends on the takeover decision and the vehicle’s ability to handle an
incident. In our previous work [20], human subject experiments have shown
that, by proactively inferring human trust and taking it into account during
decision making, the AV can achieve higher cumulative rewards.

The research focus of [20] was on optimal route planning (e.g., navigating
from one location to the other) for AVs. In contrast, in this work we are
interested in trust-aware HRC in a boarder context. Our goal is to develop a
trust-aware motion planning approach for HRC systems, which is capable of
completing complex tasks specified in temporal logic that involve requirements
on human trust levels.

##  III Preliminaries

Before formulating our problem, we provide preliminary background on POMDPs
and scLDTL.

###  III-A Partially observable Markov decision processes

This section introduces POMDPs, which are well suited to the modelling of HRC
systems under investigation. The human internal states (e.g., trust), which
are not fully observable to robots, can be modelled as hidden states in
POMDPs. In order to accurately represent the interactions between humans and
robots, modifications to the conventional definition of a POMDP [24] are
incorporated.

######  Definition 1 (POMDPs)

A POMDP is defined as a tuple $\mathcal{M}=(S,A,O,\mathcal{Z},\mathcal{T})$,
where

  * •

$S$ is a finite set of states;

  * •

$A$ is a finite set of actions;

  * •

$O$ is a finite set of observations;

  * •

$\mathcal{Z}:S\times A\times O\to[0,1]$ is the probabilistic observation
function, which gives the probability of observing $o$ after taking action $a$
in state $s$, i.e., $\mathcal{Z}(s,a,o)=p(o|s,a)$;

  * •

$\mathcal{T}:S\times A\times O\times S\to[0,1]$ is the probabilistic
transition function, which gives the probability that the state has value
$s^{\prime}$, after taking action $a$ and receiving observations $o$ in state
$s$, i.e., $\mathcal{T}(s,a,o,s^{\prime})=p(s^{\prime}|s,a,o)$.

Firstly, we note that, in Definition 1, the probability of receiving
observation $o\in O$ is determined by the previous state $s$ (instead of the
resulting state $s^{\prime}$) and the action $a$ that was just taken.
Secondly, the transition function $\mathcal{T}$ is dependent on the
observations. For the purpose of this work, a reward function is redundant and
has been omitted.

Since a POMDP state is partially observable, we rely on the concept of a
_belief state_ 111A belief state is a probability distribution over all
possible states in the POMDP. It represents the agent’s subjective probability
distribution of being in each state given its past observations and actions..
Let $\mathcal{B}$ be the belief space of $S$. A POMDP policy
$\pi:\mathcal{B}\to A$ maps a belief state $b\in\mathcal{B}$, which is a
probability distribution over $S$, to a prescribed action $a\in A$. Given a
policy $\pi$, the control of the agent’s actions is performed online. First,
the agent takes an action $a=\pi(b)$ according to the given policy $\pi$ and
the current belief is $b$. Second, after taking an action $a$ and receiving an
observation $o$, the agent updates its belief:

$b^{\prime}(s^{\prime})=\sum_{s\in
S}b(s){\sum_{o\in\mathbb{F}_{O}(s,a)}\mathcal{Z}(s,a,o)\cdot\mathcal{T}(s,a,o,s^{\prime})},$
(1)

where $\mathbb{F}_{O}(s,a)=\\{o\in O|\mathcal{Z}(s,a,o)>0\\}$. The process
then repeats. An interesting property to note about the POMDP described in
Definition 1 is that the belief update (1) is linear222The belief update of a
conventional POMDP is often represented using the Bayes’ filter.. An
_execution_ $\bm{\rho}$ of a POMDP is a possibly infinite alternating sequence
of belief states, actions, and observations, i.e.,
$\bm{\rho}=b_{0}a_{0}o_{0}b_{1}a_{1}o_{1}\cdots$.

###  III-B Syntactically Co-Safe Linear Distribution Temporal Logic

This section introduces scLDTL for concisely specifying the desired behaviour
of the HRC systems. It will become clear later that scLDTL is capable of
expressing requirements in both the robot workspace and the trust (belief)
space.

scLDTL consists of two types of predicates: i) state predicates $\nu\in 2^{S}$
and ii) belief predicates $\mu$, which are obtained after evaluation of a
predicate function $g_{\mu}:\mathcal{B}\to\mathbb{R}$ on the belief space
$\mathcal{B}$ as follows

$\mu:=\left\\{\begin{aligned} \top,&\quad\text{if }\quad g_{\mu}(b)<0\\\
\bot,&\quad\text{if }\quad g_{\mu}(b)\geq 0.\end{aligned}\right.$

An scLDTL formula is defined inductively according to the following syntax
[21]:

$\varphi::=\top|\nu|\mu|\neg\nu|\neg\mu|\varphi_{1}\wedge\varphi_{2}|\varphi_{1}\vee\varphi_{2}|\varphi_{1}\mathsf{U}\varphi_{2}|\bigcirc\varphi|\lozenge\varphi,$
(2)

where $\nu\in 2^{S}$ is a set of states, $\mu$ is a belief predicate, $\neg$
(negation), $\wedge$ (conjunction), and $\vee$ (disjunction) are logic
connectives, and $\mathsf{U}$ (until), $\bigcirc$ (next) and $\lozenge$
(eventually) are temporal operators. We omit the scLDTL semantics due to page
limit and refer the reader to [21].

Let ${\rm AP}$ be a set of state predicates and ${\rm BP}$ be a set of belief
predicates. The satisfaction of an scLDTL formula $\varphi$ over ${\rm
AP}\cup{\rm BP}$ can be captured through a deterministic finite automaton
(DFA) $\mathcal{A}=(Q,q_{0},2^{\rm AP\cup BP},\delta,{\rm Acc})$, where

  * •

$Q$ is a finite set of states,

  * •

$q_{0}\in Q$ is the initial state,

  * •

$\delta:Q\times 2^{\rm AP\cup BP}\to Q$ is the transition function, and

  * •

${\rm Acc}\subseteq Q$ is the set of accepting states.

A finite _run_ $q=q_{0}q_{1}\ldots q_{k}$ of $\mathcal{A}$ is called
_accepting_ if $q_{k}\in{\rm Acc}$.

Now we define the notion of probabilistic satisfaction with respect to an
execution $\bm{\rho}$ of a POMDP.

###### Definition 2

[scLDTL satisfaction with respect to a POMDP execution] Given an execution
$\bm{\rho}=b_{0}a_{0}o_{0}b_{1}a_{1}o_{1}\ldots$ of a POMDP $\mathcal{M}$, the
probability that the execution $\bm{\rho}$ satisfies the scLDTL formula
$\varphi$ is given by

${\rm Pr}_{\mathcal{M}}(\\{\text{$s_{0}s_{1}\cdots$ such that
$(s_{0},b_{0})(s_{1},b_{1})\cdots\models\varphi$}\\}\mid\bm{\rho}).$

For simplicity, it is denoted in shorthand as ${\rm
Pr}_{\mathcal{M}}(\varphi\mid\bm{\rho}).$

##  IV POMDPs for HRC

In this work, we consider a human and a robot working collaboratively in the
workspace $X$. The human (H) adopts a supervisory role and the robot (R) is
charged with performing tasks. The human can intervene in the task execution
due to, for instance, low trust.

###  IV-A HRC modelling

Within the workspace $X$, one can identify a set of incidents $I$, i.e., a set
of events that can affect human trust and/or takeover decision. The likelihood
of observing an incident $id\in I$ is determined by the current state and the
action of the robot. Denote by $\Theta$ the state space of human trust in the
robot, which is not observable by the robot.

The human-robot interaction can be modelled as a POMDP $\mathcal{M}$ as per
Definition 1, where the state space $S$ of $\mathcal{M}$ is factored as the
observable state space $X$ and the non-observable state space $\Theta$, i.e.,
$S=X\times\Theta$. Accordingly, the probabilistic transition function
$\mathcal{T}$ is factored as the world state and the human trust probabilistic
transition functions $\mathcal{T}_{X}$ and $\mathcal{T}_{\Theta}$,
respectively. It has been shown in [19] that the human trust affects human
behaviour (e.g., takeover decision) and the human trust is affected by factors
such as the robot performance (i.e., success/fail in handling an incident).
Therefore, the state evolution of the trust POMDP $\mathcal{M}$ is determined
not only by the robot action $a^{r}$, but also posterior observations,
including the incident $id$ encountered during execution, the human takeover
decision $a^{h}$, and the robot performance $e^{r}$. Formally, the trust POMDP
is specified as a tuple
$\mathcal{M}=(X,\Theta,A^{r},O,\mathcal{Z},\mathcal{T}_{X},\mathcal{T}_{\Theta})$,
where

  * •

$A^{r}$ is the finite action space of the robot;

  * •

$O=I\times A^{h}\times\mathcal{E}^{r}$ is the observation set, where

    * –

$I$ is the set of incidents;

    * –

$A^{h}=\\{\rm{tk,st}\\}$ is the action space of the human, where ${\rm tk}$
and ${\rm st}$ stand for “takeover” and “standstill”, respectively; similarly
to [19], we assume that the human first observes the robot’s action $a^{r}$
and then decides his or her own action $a^{h}$;

    * –

$\mathcal{E}^{r}=\\{\rm{succ,fail}\\}$ represents the performance of the
robot, where ${\rm succ}$ and ${\rm fail}$ stand for “success” and “failure”,
respectively.

  * •

$\mathcal{Z}:X\times\Theta\times A^{r}\times O\to[0,1]$ is the probabilistic
observation function, which is given by
$\mathcal{Z}(x,\theta,a^{r},o)=p(id,a^{h},e^{r}|x,\theta,a^{r});$

  * •

$\mathcal{T}_{X}:X\times A^{r}\times O\times X\to[0,1]$ is the world state
probabilistic transition function, i.e.,
$\mathcal{T}_{X}(x,a^{r},o,x^{\prime})=p(x^{\prime}|x,a^{r},o);$

  * •

$\mathcal{T}_{\Theta}:\Theta\times O\times\Theta\to[0,1]$ is the human trust
probabilistic transition function, i.e.,
$\mathcal{T}_{\Theta}(\theta,o,\theta^{\prime})=p(\theta^{\prime}|\theta,o)$.

A graphical model of the trust POMDP is shown in Fig. 1. It contains two key
components: (i) a trust dynamics model, which captures the evolution of human
trust in the robot, and (ii) a human decision model, which connects trust with
human actions.

Let $\mathcal{B}_{\Theta}$ be the belief space of $\Theta$. We associate with
each value $x\in X$ a belief space for $\theta$:
$\mathcal{B}_{\Theta}(x):=\\{(x,b_{\Theta})|b_{\Theta}\in\mathcal{B}_{\Theta}\\}$.
Then the belief space of the trust POMDP $\mathcal{M}$ can be defined as

$\mathcal{B}=\cup_{x\in X}\mathcal{B}_{\Theta}(x).$ (3)

The POMDP policy $\pi:\mathcal{B}\to A^{r}$ maps a belief $b\in\mathcal{B}$ to
a prescribed robot action $a^{r}\in A^{r}(b)$, where $A^{r}(b)$ is the set of
actions available at belief $b$.

Figure 2: The graphical model for the trust POMDP $\mathcal{M}$.

We illustrate our results with the motivating example.

######  Example 1 (continued)

The town map shown in Fig. 1(a) has 12 road intersections
$\\{\rm{A,\cdots,L}\\}$. Depending on the driving direction, each intersection
can be factored into 3 different states (for instance, intersection $A$
contains states ${\rm EA}$, ${\rm BA}$, and ${\rm FA}$). We use Muir’s
questionnaire [25] with a 7-point Likert scale as a human trust metric (i.e.,
trust ranges from 1 to 7). Therefore, one has that $X=\\{{\rm EA},{\rm
BA},{\rm FA}\cdots,{\rm EL},{\rm KL},{\rm HL}\\}$, $\Theta=\\{1,\cdots,7\\}$,
and $I=\\{\rm{`pedestrian^{\prime},`obstacle^{\prime},`truck^{\prime}}\\}$.
The robot action is route choices and one can define an indicator function
$\mathcal{I}$ for incidents. For instance, $\mathcal{I}({\rm EA},{\rm
AB},{\rm`pedestrian^{\prime}})=1$.

###  IV-B Problem formulation

In this work, we consider that the HRC system is required to complete complex
tasks in the robot workspace $X$. Moreover, there are requirements on the
human trust $\Theta$ and/or trust belief $b_{\Theta}$. We formulate the tasks
in the workspace as well as the requirements on human trust and trust belief
in the form of an scLDTL formula $\varphi$.

######  Example 2 (continued)

Consider now that the AV is required to complete the following tasks: i)
visits the target intersections $G,J$ and $L$ (in this order) from the initial
location ${\rm EA}$ (see Fig. 1(a)) and ii) the human trust level can not be
too low (lower than 2) at all times, and iii) when the vehicle reaches the
final intersection $L$, the likelihood that the human trust level is high
(higher than or equal to 6) is no less than 0.5. In scLDTL, this specification
can be written as

$\varphi=(\neg\textsc{lowtrust})\mathsf{U}(\diamondsuit(\nu_{1}\wedge\diamondsuit(\nu_{2}\wedge\diamondsuit(\nu_{3}\wedge\textsc{hightrust}))),$

where there are 3 state predicates $\nu_{1}=\\{{\rm BG},{\rm FG},{\rm
IG}\\},\nu_{2}=\\{{\rm IJ},{\rm CJ},{\rm KJ}\\},\nu_{3}=\\{{\rm KL},{\rm
EL},{\rm HL}\\}$ and 2 belief predicates
$\textsc{lowtrust},\textsc{hightrust}$. The predicate functions are given by
$g_{\textsc{lowtrust}}=1-A_{1}b_{\Theta},g_{\textsc{hightrust}}=0.5-A_{2}b_{\Theta}$,
where $A_{1}=[1,0,0,0,0,0,0]$ encodes trust lower than 2 and
$A_{2}=[0,0,0,0,0,1,1]$ encodes trust higher than or equal to 6.

Given the initial belief state $b_{0}$ and a policy ${\pi}$, denote by
${\bm{\rho}}^{\pi}(b_{0})$ the set of all possible executions generated by
${\pi}$. We consider the optimal policy synthesis problem for HRC under scLDTL
specifications, i.e., find an optimal policy $\pi$ such that the probability
of the set of all executions that satisfy an scLDTL formula $\varphi$ under
$\pi$ is maximised. Mathematically, this problem can be formulated as follows.

###### Problem 1

Given the trust POMDP ${\mathcal{M}}$ and the scLDTL specification $\varphi$,
find a policy $\pi\in\Pi$ such that

$\displaystyle\max_{\pi\in\Pi}\quad{\rm
Pr}_{{\mathcal{M}}}^{\pi}(\varphi)\triangleq\sum_{\bm{\rho}\in\bm{\bm{\rho}}^{\pi}(b_{0})}{\rm
Pr}(\bm{\rho}|b_{0},\pi){\rm Pr}_{{\mathcal{M}}}(\varphi\mid\bm{\rho}),$ (4)

where $\Pi$ is the set of all policies for the trust POMDP ${\mathcal{M}}$ and
${\rm Pr}_{{\mathcal{M}}}(\varphi\mid\bm{\rho})$ is given in Definition 2.

###### Remark 1

Point-based value iteration (PBVI) algorithm has been proposed for POMDPs
under LTL specifications. In [26], the atomic propositions of an LTL formula
are evaluated on the state space of the POMDP, which is finite. Therefore, the
construction of the product POMDP and the computation of maximal end
components are similar to finite MDPs, for which the existing graph-based
methods [27] can be utilised. In this work we consider scLDTL specifications,
in which the belief predicates are evaluated over the belief space
$\mathcal{B}$ of the trust POMDP ${\mathcal{M}}$, which is infinite.
Therefore, the approach proposed in [26] is not applicable here.

##  V Proposed Approach

This section presents our approach to solve the optimal policy synthesis
problem (Problem 1), which falls outside the purview of existing policy
synthesis algorithms designed for POMDPs. It is divided into two parts: 1) the
construction of the product POMDP with the DFA of the scLDTL formula $\varphi$
and 2) an algorithm to approximately compute a policy that maximises the
probability of satisfying the given scLDTL formula $\varphi$.

###  V-A Product POMDP

To begin with, we define the deterministic state and belief predicate
labelling functions $L_{s}$ and $L_{b}$ as

  * •

$L_{s}:X\times\Theta\to 2^{\rm AP}$, which contains the set of state
predicates that can be true at state $(x,\theta)$;

  * •

$L_{b}:X\times\mathcal{B}_{\Theta}\to 2^{\rm BP}$, which contains the set of
belief predicates that can be true at belief state $(x,b_{\Theta})$.

Then the associated probabilities are given by $p_{L_{s}}:X\times\Theta\times
2^{\rm AP}\to\\{0,1\\}$ and $p_{L_{b}}:X\times\mathcal{B}_{\Theta}\times
2^{\rm BP}\to\\{0,1\\}$, respectively.

Now we propose to reformulate the trust POMDP $\mathcal{M}$ (equivalently) as
a belief MDP and further expand it by including probabilistic labels, which
gives a _probabilistically labelled_ belief MDP
$\hat{\mathcal{M}}=(\mathcal{B},A^{r},O,\hat{\mathcal{Z}},\hat{\mathcal{T}}_{\mathcal{B}},\hat{L},p_{\hat{L}})$,
where $\mathcal{B}$ is defined in (3), $A^{r},O$ are given in $\mathcal{M}$,
and

  * •

$\hat{\mathcal{Z}}:\mathcal{B}\times A^{r}\times O\to[0,1]$ is the
probabilistic observation function;

  * •

$\hat{\mathcal{T}}_{\mathcal{B}}=(\mathcal{T}_{X},\mathcal{T}_{\mathcal{B}_{\Theta}})$
is the probabilistic transition function, where $\mathcal{T}_{X}$ is defined
in $\mathcal{M}$ and
$\mathcal{T}_{\mathcal{B}_{\Theta}}:\mathcal{B}_{\Theta}\times
O\times\mathcal{B}_{\Theta}\to[0,1]$;

  * •

$\hat{L}:\mathcal{B}\to 2^{2^{\rm AP\cup BP}}$ is the belief state labelling
function, where $\hat{L}(x,b_{\Theta})$ contains the set of state and belief
predicate subsets that can be true at $(x,b_{\Theta})$;

  * •

$p_{\hat{L}}:\mathcal{B}\times 2^{\rm AP\cup BP}\to[0,1]$ specifies the
associated probability.

The probabilistic labelling function $p_{\hat{L}}$ provides a unified way of
assigning belief states with both state and belief predicates in an scLDTL
formula. For instance, given a belief state $(x,b_{\Theta})$, a state
predicate $\nu$, and a belief predicate $\mu$, one has that
$p_{\hat{L}}(x,b_{\Theta},\nu)=b_{\Theta}(\theta)p_{L_{s}}(x,\theta,\nu),\forall\theta\in\Theta$
and $p_{\hat{L}}(x,b_{\Theta},\mu)=p_{L_{b}}(x,b_{\Theta},\mu)$.

The product belief MDP $\mathcal{M}^{\times}$ is constructed between the
probabilistically labelled belief MDP $\hat{\mathcal{M}}$ and the DFA
$\mathcal{A}=(Q,q_{0},2^{\rm AP\cup BP},\delta,{\rm Acc})$ of the scLDTL
formula $\varphi$.

######  Definition 3 (Product belief MDP)

Denote by $\mathcal{M}^{\times}$ the product
$\hat{\mathcal{M}}\times\mathcal{A}$ as a tuple
$\mathcal{M}^{\times}=(S^{\times},A^{r},O,{\mathcal{Z}}^{\times},\mathcal{T}^{\times},{\rm
Acc}^{\times})$, where

$S^{\times}=\mathcal{B}\times 2^{\rm AP\times BP}\times Q$

is so that $(x,b_{\Theta},l,q)\in
S^{\times},\forall(x,b_{\Theta})\in\mathcal{B},\forall
l\in\hat{L}(x,b_{\Theta})$, and $\forall q\in Q$,
$\mathcal{Z}^{\times}:S^{\times}\times A^{r}\times O\to[0,1]$ is the
probabilistic observation function, and ${\rm
Acc}^{\times}=\\{(x,b_{\Theta},l,q)\in S^{\times}|q\in{\rm Acc}\\}$. The
probabilistic transition function $\mathcal{T}^{\times}:S^{\times}\times
A^{r}\times O\times S^{\times}\to[0,1]$ is defined as

$\displaystyle\mathcal{T}^{\times}((x,b_{\Theta},l,q),a^{r},o,(x^{\prime},b^{\prime}_{\Theta},l^{\prime},q^{\prime}))=$
$\displaystyle\begin{cases}\hat{\mathcal{T}}_{\mathcal{B}}((x,b_{\Theta}),a^{r},o,(x^{\prime},b^{\prime}_{\Theta}))\cdot
p_{\hat{L}}(x^{\prime},b^{\prime}_{\Theta},l^{\prime}),&\text{if
$q^{\prime}\in\delta(q,l),$}\\\ 0,&\text{otherwise};\end{cases}$

where $\hat{\mathcal{Z}},\hat{\mathcal{T}}_{\mathcal{B}},$ and $p_{\hat{L}}$
are defined in $\hat{\mathcal{M}}$.

Let $\Pi^{\times}$ be the set of all policies for the product belief MDP
$\mathcal{M}^{\times}$. The set of _accepting states_ of
$\mathcal{M}^{\times}$ is given by ${\rm Acc}^{\times}$. We have the following
result.

###### Theorem 1

Given the trust POMDP ${\mathcal{M}}$ and the scLDTL formula $\varphi$, the
maximal probability of satisfying $\varphi$ is:

$\displaystyle\max_{\pi\in\Pi}\\{{\rm
Pr}_{{\mathcal{M}}}^{\pi}(\varphi)\\}=\max_{{\hat{\pi}}\in{\Pi^{\times}}}\\{{\rm
Pr}_{{\mathcal{M}}^{\times}}^{{\hat{\pi}}}(\diamondsuit{\rm
Acc}^{\times})\\}.$

Theorem 1 shows that, with the set of accepting states ${\rm Acc}^{\times}$,
the original optimal policy synthesis problem reduces to a reachability
problem.

###  V-B Optimal policy synthesis

PBVI algorithms have been widely used for solving POMDP synthesis problems
[16, 17, 18]. They often offer convergence guarantees specified as upper and
lower bounds on the value function. However, these PBVI algorithms are not
directly applicable for solving our problem (Problem 1). This is because
solving a reachability problem for POMDPs necessitates the presence of a
clearly defined reward function, which assigns value 1 to states in the goal
set and 0 otherwise. However, in our case, capturing the satisfaction of an
scLDTL specification through a state-based reward function is not feasible due
to the presence of belief predicates.

In the following, we show that, with essential modifications, the existing
PBVI algorithms can be leveraged for solving Problem 1 with the set of
accepting states ${\rm Acc}^{\times}$.

Given a state $s$ of the product belief MDP $\mathcal{M}^{\times}$, we first
define a value function $V:S^{\times}\to\mathbb{R}_{\geq 0}$ as

$V(s)=\max_{{\pi}\in{\Pi^{\times}}}\\{{\rm
Pr}_{{\mathcal{M}}^{\times}}^{{\pi}}(\diamondsuit{\rm Acc}^{\times})\\},$

which represents the maximal probability of reaching ${\rm Acc}^{\times}$ from
initial state $s$. Then one can get that $V(s)=1,\forall s\in{\rm
Acc}^{\times}$. For $s\notin{\rm Acc}^{\times}$, we further define the dynamic
programming operator $T$ as

$\displaystyle T({V})(s)$ $\displaystyle=$ $\displaystyle\max\limits_{a^{r}\in
A^{r}}\left\\{\sum\limits_{o\in\mathbb{F}_{O}(s,a^{r})}{\mathcal{Z}}^{\times}(s,a^{r},o)\sum\limits_{s^{\prime}\in
S^{\times}}\mathcal{T}^{\times}(s,a^{r},o,s^{\prime}){V}(s^{\prime})\right\\}.$

Before running a PBVI algorithm, first we initialize the upper- and lower-
bounds of the value function $V$ as follows:

$\overline{V}^{0}(s)=1,\forall s\in
S^{\times},\quad\underline{V}^{0}(s)=\begin{cases}1,&\text{if $s\in{\rm
Acc}^{\times},$}\\\ 0,&\text{otherwise.}\end{cases}$ (5)

Then a precision parameter $\tau$ is provided that controls the tightness of
the convergence (for example, by controlling the depth of the tree in SARSOP
[18]), which yields $|\overline{V}(s_{0})-\underline{V}(s_{0})|\leq\tau,$
where $s_{0}=(b_{0},q_{0})$ is the initial state of the product belief MDP
$\mathcal{M}^{\times}$.

Denote by ${\rm Pr}_{{\mathcal{M}}}^{\max}(\varphi):=\max_{\pi\in\Pi}\\{{\rm
Pr}_{{\mathcal{M}}}^{\pi}(\varphi)\\}$ the maximal probability of satisfying
the scLDTL formula $\varphi$. We have the following result.

###### Theorem 2

Let $\overline{V}(s_{0})$ and $\underline{V}(s_{0})$ be the upper- and lower-
bounds of $V(s_{0})$ obtained using PBVI with the initialization function (5).
One has that

$\underline{V}(s_{0})\leq{\rm
Pr}_{{\mathcal{M}}}^{\max}(\varphi)\leq\overline{V}(s_{0}).$

Finally, the optimal policy $\hat{\pi}^{*}$ for state $s\in S^{\times}$ can be
derived using the value function.

##  VI Implementation

We have implemented the proposed approach to obtain the optimal policy for
each scLDTL specification under consideration for the motivating example. To
construct the trust POMDP, we utilise the trust dynamics model and the human
takeover decision model, which were derived through an online user study
involving 100 anonymous participants on Amazon Mechanical Turk (AMT) platform
[20]. Then two scLDTL specifications $\varphi_{1}$ and $\varphi_{2}$ are
considered for the AV,

$\displaystyle\varphi_{1}=(\neg\textsc{lowtrust})\mathsf{U}(\diamondsuit(\nu_{1}\wedge\diamondsuit(\nu_{2}\wedge\diamondsuit(\nu_{3}\wedge\textsc{hightrust}))),$
$\displaystyle\varphi_{2}=(\neg\textsc{lowtrust})\mathsf{U}(\diamondsuit(\nu_{4}\wedge\diamondsuit(\nu_{5}\wedge\diamondsuit(\nu_{6}\wedge\textsc{hightrust}))),$

where $\nu_{1},\nu_{2},\nu_{3},\textsc{lowtrust},\textsc{hightrust}$ are
defined in the Example and $\nu_{4}=\\{{\rm AB},{\rm GB},{\rm
CB}\\},\nu_{5}=\\{{\rm DK},{\rm JK},{\rm LK}\\},\nu_{6}=\\{{\rm AE},{\rm
DE},{\rm LE}\\}$. The DFA for each scLDTL specification
$\varphi_{i},i\in\\{1,2\\}$ is derived using [28]. Then the corresponding
product belief MDP $\mathcal{M}_{i}^{\times}$ is constructed with the DFA of
$\varphi_{i}$. Finally, the upper- and lower-bounds of the value function
$V_{i}$ are computed using the POMDP toolkit “pomdp_py” [29] (with essential
modifications describled in Section V.B). The precision parameter is set as
$\tau=0.01$. All simulations are carried out on a Macbook Pro (2.6 GHz 6-Core
Intel Core i7 and 16 GB of RAM).

##  VII Driving Simulator Study

We evaluate the effectiveness of obtained policies via a driving simulator
study. 333This study was approved by the University of Virginia Institutional
Review Boards under IRB-SBS protocol #6045.

###  VII-A Study design

Apparatus. The study was conducted in a fixed-based driving simulator from
SimXperience, consisting of a 55-inch display, a racing car seat, a Logitech
G29 steering wheel, and sport pedals, see Fig. 3. In our study, four buttons
on the steering wheel were programmed to let the drivers increase/decrease
trust, switch driving mode between manual and autopilot, switch gear between
drive and reverse.

Driving Scenario. The experiments were run on a machine with 3.5GHz CPU,
NVIDIA GeForce RTX 3080 Ti, 62GB memory, and Ubuntu 20.04.6 LTS operating
system. The virtual driving environment was created using CARLA 0.9.13. An
autopilot controller was programmed for several driving tasks such as lane
keeping, taking turns at intersections, and handling incidents.

Subject allocation. We recruited 21 participants from the university
community. All participants had a valid driver’s license and regular or
corrected-to-normal vision. Each participant was compensated with a $10 gift
card. We adopted a within-subject study design: each participant took 4 unique
routes, i.e., trust-aware and trust-free routes for both scLDTL
specifications. The start and destination is either from A to L
($\varphi_{1}$) or from H to E ($\varphi_{2}$). The route is either the trust-
aware route (obtained using the computed optimal policy) or the trust-free
route (which is the shortest-distance route from the start to the
destination). However, if the vehicle has not reached the destination after
travelling 20 intersections, it reschedules a shortest-distance route. The
order of trials are randomized and counter-balanced.

###  VII-B Study Procedure

Upon arrival, a participant was instructed to read and sign a consent form
approved by the Institutional Review Board. We conducted a five-minute
training session to familiarize the participant with the driving simulator
setup.

The vehicle started driving in autopilot mode by default. Participants can
decide whether to take over the vehicle to handle the incidents on the road.
If the participants did takeover, they were required to switch back to
autopilot mode before arriving at the next intersection so that the vehicle
can choose the next direction to go. We asked the participants to periodically
record their trust in the AV using the buttons on the steering wheel. It took
about 40 minutes for each participant to complete the entire experiment.

Figure 3: Driving simulator setup.

###  VII-C Results

The computed maximal probabilities for satisfying the scLDTL specifications
$\varphi_{1}$ and $\varphi_{2}$ are ${\rm
Pr}_{{\mathcal{M}}}^{\max}(\varphi_{1})=0.8357$ and ${\rm
Pr}_{{\mathcal{M}}}^{\max}(\varphi_{2})=0.7288$, respectively.

For each participant, we evaluate the satisfaction of each scLDTL
specification $\varphi_{i}$ using the robot states $x$ and human trust levels
$\Theta$ recorded in the experiment, and the induced trust beliefs
$b_{\Theta}$. In total, 18 out of 21 ($18/21=0.8571>0.8357$) and 17 out of 21
($17/21=0.8095>0.7288$) participants successfully complete the specifications
$\varphi_{1}$ and $\varphi_{2}$ respectively, which validates the
effectiveness of the proposed approach. Moreover, we further compare the
trust-aware and trust-free routes for all drivers and both trails. The
percentage of scLDTL satisfaction is 0.85 (trust-aware) vs 0.775 (trust-free).
The average human trust is 4.6018 (trust-aware) vs 4.3933 (trust-free). One
can see that the trust-aware policy outperforms the trust-free one. A video
demonstration of the human experiment can be found at: link.

We summarise three key observations gained from the experiments. First,
observing the AV successfully handle the same incidents multiple times does
not necessarily guarantee an increase in human trust. Second, if a human’s
trust remains consistently low for an extended period, it becomes challenging
for them to re-establish trust in the AV. Third, having the capability to
effectively address an incident beforehand can contribute to boosting human
trust. Based on the feedback received after the experiment, participants have
indicated that, had they noticed the car braking earlier in situations
involving pedestrians, they might have considered the AV more trustworthy.

##  VIII Conclusions

In this work, we presented a trust-aware motion planning approach for HRC. We
demonstrated the suitability of scLDTL for describing the desired behaviours
of HRC systems and an algorithm was proposed for solving the optimal policy
synthesis problem. Human subject experiments were conducted on a driving
simulator, validating the effectiveness of the proposed approach and providing
valuable new insights. Additionally, we observed variations in trust dynamics
among individuals, which will be further investigated in future research.

## References

  * [1] J. Pineau, M. Montemerlo, M. Pollack, N. Roy, and S. Thrun, “Towards robotic assistants in nursing homes: Challenges and results,” _Robotics and autonomous systems_ , vol. 42, no. 3-4, pp. 271–281, 2003.
  * [2] W. Schwarting, J. Alonso-Mora, and D. Rus, “Planning and decision-making for autonomous vehicles,” _Annual Review of Control, Robotics, and Autonomous Systems_ , vol. 1, pp. 187–210, 2018.
  * [3] M. Anderson, “Surprise! 2020 is not the year for self-driving cars,” _IEEE Spectrum_ , 2020.
  * [4] D. Dunning and D. Fetchenhauer, “Understanding the psychology of trust,” in _Social motivation_. Psychology Press, 2011, pp. 147–169.
  * [5] J. D. Lee and K. A. See, “Trust in automation: Designing for appropriate reliance,” _Human factors_ , vol. 46, no. 1, pp. 50–80, 2004.
  * [6] K. E. Schaefer, J. Y. Chen, J. L. Szalma, and P. A. Hancock, “A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems,” _Human factors_ , vol. 58, no. 3, pp. 377–400, 2016.
  * [7] A. Freedy, E. DeVisser, G. Weltman, and N. Coeyman, “Measurement of trust in human-robot collaboration,” in _2007 International symposium on collaborative technologies and systems_. Ieee, 2007, pp. 106–114.
  * [8] A. Xu and G. Dudek, “Optimo: Online probabilistic trust inference model for asymmetric human-robot collaborations,” in _Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction_ , 2015, pp. 221–228.
  * [9] N. Wang, D. V. Pynadath, and S. G. Hill, “Trust calibration within a human-robot team: Comparing automatically generated explanations,” in _2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI)_. IEEE, 2016, pp. 109–116.
  * [10] M. Chen, S. Nikolaidis, H. Soh, D. Hsu, and S. Srinivasa, “Planning with trust for human-robot collaboration,” in _Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction_ , 2018, pp. 307–315.
  * [11] S. Sheng, E. Pakdamanian, K. Han, Z. Wang, J. Lenneman, and L. Feng, “Trust-based route planning for automated vehicles,” in _Proceedings of the ACM/IEEE 12th International Conference on Cyber-Physical Systems_ , 2021, pp. 1–10.
  * [12] D. Sadigh, S. Sastry, S. A. Seshia, and A. D. Dragan, “Planning for autonomous cars that leverage effects on human actions.” in _Robotics: Science and systems_ , vol. 2. Ann Arbor, MI, USA, 2016, pp. 1–9.
  * [13] X. Huang, M. Kwiatkowska, and M. Olejnik, “Reasoning about cognitive trust in stochastic multiagent systems,” _ACM Transactions on Computational Logic (TOCL)_ , vol. 20, no. 4, pp. 1–64, 2019.
  * [14] F. Broz, I. R. Nourbakhsh, and R. G. Simmons, “Planning for human-robot interaction using time-state aggregated pomdps.” in _AAAI_ , vol. 8, 2008, pp. 1339–1344.
  * [15] M. Lauri, D. Hsu, and J. Pajarinen, “Partially observable markov decision processes in robotics: A survey,” _IEEE Transactions on Robotics_ , vol. 39, no. 1, pp. 21–40, 2022.
  * [16] D. Silver and J. Veness, “Monte-carlo planning in large pomdps,” _Advances in neural information processing systems_ , vol. 23, 2010.
  * [17] T. Smith and R. Simmons, “Point-based pomdp algorithms: Improved analysis and implementation,” _arXiv preprint arXiv:1207.1412_ , 2012.
  * [18] H. Kurniawati, D. Hsu, and W. S. Lee, “Sarsop: Efficient point-based pomdp planning by approximating optimally reachable belief spaces.” in _Robotics: Science and systems_ , vol. 2008. Citeseer, 2008.
  * [19] M. Chen, S. Nikolaidis, H. Soh, D. Hsu, and S. Srinivasa, “Trust-aware decision making for human-robot collaboration: Model learning and planning,” _ACM Transactions on Human-Robot Interaction (THRI)_ , vol. 9, no. 2, pp. 1–23, 2020.
  * [20] S. Sheng, E. Pakdamanian, K. Han, Z. Wang, J. Lenneman, D. Parker, and L. Feng, “Planning for automated vehicles with human trust,” _ACM Transactions on Cyber-Physical Systems_ , 2022.
  * [21] A. Jones, M. Schwager, and C. Belta, “Distribution temporal logic: Combining correctness with quality of estimation,” in _52nd IEEE Conference on Decision and Control_. IEEE, 2013, pp. 4719–4724.
  * [22] A. M. Ayala, S. B. Andersson, and C. Belta, “Temporal logic motion planning in unknown environments,” in _2013 IEEE/RSJ International Conference on Intelligent Robots and Systems_. IEEE, 2013, pp. 5279–5284.
  * [23] K. Cho, J. Suh, C. J. Tomlin, and S. Oh, “Cost-aware path planning under co-safe temporal logic specifications,” _IEEE Robotics and Automation Letters_ , vol. 2, no. 4, pp. 2308–2315, 2017.
  * [24] L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Partially observable markov decision processes for artificial intelligence,” in _KI-95: Advances in Artificial Intelligence: 19th Annual German Conference on Artificial Intelligence Bielefeld, Germany, September 11–13, 1995 Proceedings 19_. Springer, 1995, pp. 1–17.
  * [25] B. M. Muir, _Operators’ Trust in and Use of Automatic Controllers in a Supervisory Process Control Task_. University of Toronto, 1990.
  * [26] M. Bouton, J. Tumova, and M. J. Kochenderfer, “Point-based methods for model checking in partially observable markov decision processes,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 06, 2020, pp. 10 061–10 068.
  * [27] C. Baier and J.-P. Katoen, _Principles of model checking_. MIT press, 2008.
  * [28] J. Klein, “ltl2dstar-ltl to deterministic streett and rabin automata,” 2007.
  * [29] K. Zheng and S. Tellex, “pomdp_py: A framework to build and solve pomdp problems,” in _ICAPS 2020 Workshop on Planning and Robotics (PlanRob)_ , 2020\.