# Network quantum steering enables randomness certification without seed
randomness

Shubhayan Sarkar  Laboratoire d’Information Quantique, Université libre de
Bruxelles (ULB), Av. F. D. Roosevelt 50, 1050 Bruxelles, Belgium

###### Abstract

Quantum networks with independent sources allow the observation of quantum
nonlocality without inputs. Consequently, the incompatibility of measurements
is not a necessity for observing quantum nonlocality when one has access to
independent sources. Here we investigate the minimal scenario without inputs
where one can observe any form of quantum nonlocality. We show that even two
parties with two sources that might be classically correlated can witness a
form of quantum nonlocality, in particular quantum steering, in networks
without inputs if one of the parties is trusted, that is, performs a fixed
known measurement. We term this effect as swap-steering. The scenario
presented in this work is minimal to observe such an effect. Consequently, a
scenario exists where one can observe quantum steering but not Bell non-
locality. We further construct a linear witness to observe swap-steering.
Interestingly, this witness enables self-testing of the quantum states
generated by the sources and the local measurement of the untrusted party.
This in turn allows certifying two bits of randomness that can be obtained
from the measurement outcomes of the untrusted device without the requirement
of initially feeding the device with randomness.

Introduction— Quantum nonlocality is one of the most remarkable features of
quantum mechanics that defy our classical intuitions about the world. It
refers to the property of quantum particles to exhibit correlations that seem
to occur instantaneously even when they are separated by large distances. This
quantum property was first conceptualized in the celebrated work of Einstein,
Podolsky and Rosen [1]. Based on it, Bell in 1964 [2, 3] proposed a
theoretical test, known as Bell’s inequality, that could distinguish between
classical and quantum correlations. It was then experimentally verified [4, 5,
6, 7] and is now recognized as a fundamental aspect of quantum mechanics. The
implications of quantum nonlocality are far-reaching, with potential
applications in fields such as cryptography, quantum teleportation, quantum
communication, and quantum computing (refer to [8] for a review).

Another form of quantum nonlocality, known as quantum steering, allows for one
observer to remotely influence the state of another observer’s quantum system,
even if the two observers are separated by large distances. Quantum steering
was first conceptualized by Schrodinger [9] and was then rigorously introduced
in [10]. The major difference between the scenarios to observe Bell
nonlocality and quantum steering is that one of the parties is assumed to be
trusted in the latter one, that is, known to perform fixed measurements.

To observe quantum nonlocality or quantum steering, any party involved in the
experiment must have at least two inputs as incompatible measurements are
necessary to witness any of these phenomena. Interestingly, quantum networks
allow for witnessing such non-classical features without the requirement of
incompatibility of measurements. The framework to witness quantum nonlocality
in networks was introduced in [11, 12, 13, 14]. However, it was first noted in
[12] and then in [14] that considering independent sources shared between non-
communicating parties allows one to observe quantum nonlocality with a single
fixed measurement for every party. Recently, the authors in [15, 16, 17, 18]
explore this phenomenon to construct scenarios where one can observe genuine
quantum network nonlocality.

One of the intriguing problems in this regard concerns the minimal scenario in
which any form of quantum nonlocality can be observed without any inputs. It
was shown in [15], that genuine network nonlocality can be observed without
inputs if there are three parties with three independent sources. Inspired by
entanglement swapping [19], we show here that if one of the parties is assumed
to be trusted then one can observe a form of quantum nonlocality, which we
term as swap-steering, using only two parties and two sources that might be
classically correlated. This is in fact the minimal scenario where one can
observe a form of quantum nonlocality without inputs. Further on, there is a
lack of witnesses when observing quantum nonlocality without inputs in
networks. This restricts the possibility of testing these phenomena at the
operational level. Interestingly, we find a linear witness to observe swap-
steering thus, making our notion of nonlocality experimentally testable. We
further show that states that are unsteerable in the standard quantum steering
scenario are swap-steerable, which can be interpreted as an entanglement-
assisted activation of quantum steering.

As an application of our work, we utilize the above result for one-sided
device-independent (DI) certification. The strongest DI certification is
termed self-testing where one can completely characterize the states and
measurements up to some degrees of freedom which can be then used to securely
generate genuine randomness from the outcomes of the measurement devices even
when an intruder might have access to them. This is extremely important for
any cryptographic scheme as the security of these schemes relies on access to
random number generators. Any of the known schemes for DI certification of
randomness requires access to seed randomness, that is, the measurement
devices whose outcomes will be used to generate random numbers, have inputs
that have to be chosen randomly in order for the protocol to be secure [for
instance see Refs. [20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]].
Furthermore, DI certification of quantum states and measurements in quantum
networks was recently explored in Refs. [32, 33, 34, 35, 18, 36, 37]. However,
all of these certification schemes require at least two inputs for most of the
measurement devices. A partial certification scheme was proposed in [36] that
utilizes the genuine network nonlocality without inputs in a triangle network
[15]. However, using the proposed scheme [36], one can only conclude that the
sources need to prepare entangled states with at least $2.5\ \%$ of
entanglement of formation and one can securely extract randomness of .04 bits.
We utilize the maximal violation of the proposed swap-steering inequality for
self-testing the singlet state along with the Bell basis which is then used
for generating secured randomness without the requirement to initially feed
the devices with random numbers. This is the first instance where the exact
certification of quantum states, measurements, and randomness could be
achieved without inputs.

The scenario— In this work, we consider the simplest scenario consisting of
two parties namely, Alice and Bob in two different labs far away from each
other. Both of them receive two subsystems from two different sources
$S_{1},S_{2}$ that might be classically correlated to each other. Now they
perform a single four-outcome measurement on their respective subsystems where
the outcomes are denoted as $a,b=0,1,2,3$ respectively for Alice and Bob.
Alice is trusted here implying that the measurement performed by her on her
subsystems is known (see Fig. 1). We consider here that she performs the
measurement corresponding to the Bell basis given by
$M_{A}=\\{|\phi_{+}\rangle\\!\\!\langle\phi_{+}|,|\phi_{-}\rangle\\!\\!\langle\phi_{-}|,|\psi_{+}\rangle\\!\\!\langle\psi_{+}|,|\psi_{-}\rangle\\!\\!\langle\psi_{-}|\\}_{A_{1}A_{2}}$
where

$\displaystyle|\phi_{\pm}\rangle_{A_{1}A_{2}}$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{2}}\left(|0\rangle_{A_{1}}|0\rangle_{A_{2}}\pm|1\rangle_{A_{1}}|1\rangle_{A_{2}}\right)$
$\displaystyle|\psi_{\pm}\rangle_{A_{1}A_{2}}$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{2}}\left(|0\rangle_{A_{1}}|1\rangle_{A_{2}}\pm|1\rangle_{A_{1}}|0\rangle_{A_{2}}\right).$
(1)

Here $A_{1}/A_{2},B_{1}/B_{2}$ denote the two different subsystems of Alice
and Bob respectively.

Now, Alice and Bob repeat the experiment enough times to construct the joint
probability distribution (correlations) $\vec{p}=\\{p(a,b)\\}$ where $p(a,b)$
denotes the probability of obtaining outcome $a,b$ with Alice and Bob
respectively. These probabilities can be computed in quantum theory as

$\displaystyle p(a,b)=\sum_{j}p_{j}\mathrm{Tr}\left[(M^{a}\otimes
N^{b})\rho_{A_{1}B_{1}}^{j}\otimes\rho_{A_{2}B_{2}}^{j}\right]$ (2)

where $M^{a},N^{b}$ denote the measurement elements of Alice and Bob which are
positive and $\sum_{a}M^{a}=\sum_{b}N^{b}=1$ and $\sum_{j}p_{j}=1$. It is
important to recall here that Alice and Bob can not communicate with each
other during the experiment.

Figure 1: Swap-steering scenario. Alice and Bob are spatially separated and
each of them receives two subsystems from the sources $S_{1},S_{2}$. On the
received subsystem they perform a single four-outcome measurement. Alice is
trusted here, meaning that she is known to perform the Bell-basis measurement.
They are not allowed to communicate during the experiment. Once it is
complete, they construct the joint probability distribution $\\{p(a,b)\\}$.

Swap-steering— Let us now suppose that there are some variables $\lambda_{i}$
that are being sent by the sources $S_{i}$. These variables in general might
be hidden and have non-classical features with the parties not having access
to them. Further on, as Alice is known to perform quantum measurements, the
variable she receives is some quantum state $\rho_{\lambda_{1},\lambda_{2}}$,
however, there is no such restriction on Bob.

Let us now state the two assumptions, namely outcome-independence and
separable quantum sources, that must be satisfied if Bob is classical, or
equivalently if the correlations are not swap-steerable from Bob to Alice.

######  Assumption 1 (Outcome-independence).

The outcomes of two parties are independent of each other if one has access to
the hidden variables $\lambda_{i}$.

In the scenario considered in this work, Bob’s outcome $b$ being independent
of Alice’s outcome $a$ means that for any $a,b,\lambda_{1},\lambda_{2}$,

$\displaystyle p(b|\lambda_{1},\lambda_{2},a)=p(b|\lambda_{1},\lambda_{2}).$
(3)

This is a weaker definition of locality when compared to Bell’s assumption of
locality, or the notion of locality in the standard quantum steering scenario.
Notice that $p(a,b)$ and $p(b|\lambda_{1},\lambda_{2},a)$ denote different
probability distributions, however, in order to alleviate the notation we use
the same symbol $p$ to denote different probability distributions wherever it
is clear from the context.

######  Assumption 2 (Separable quantum sources).

Two sources $S_{i}$ $(i=1,2)$ generating a joint quantum state
$\rho_{\lambda_{1},\lambda_{2}}$ are separable if the state
$\rho_{\lambda_{1},\lambda_{2}}$ is separable for any
$\lambda_{1},\lambda_{2}$.

Notice that the above assumption 1 that we impose on the sources is weaker
when compared to independent quantum sources. As a matter of fact, the above
assumption allows the sources to communicate classically with each other or
equivalently the sources might generate classically correlated states. As
shown in Appendix A of [38], assuming outcome-independence 1 and separable
quantum sources 2, we arrive at the following expression of $p(a,b)$

$\displaystyle\sum_{\lambda_{1},\lambda_{2}}p(\lambda_{1},\lambda_{2})\sum_{p^{j}_{\lambda_{1},\lambda_{2}}}p^{j}_{\lambda_{1},\lambda_{2}}p(a|\
|\psi^{j}_{\lambda_{1}}\rangle|\psi^{j}_{\lambda_{2}}\rangle)p(b|\lambda_{1},\lambda_{2}).$
(4)

If correlations $\vec{p}$ admit the form (21), then they are describable using
a separable outcome-independent hidden state (SOHS) model.

Figure 2: Difference between SOHS and NLHV model in the minimal scenario.
(left) Alice and Bob can explain the observed correlations $p(a,b)$ using a
SOHS model. Alice is trusted and thus receives quantum states from the sources
but there is no restriction over Bob. The grey box denotes an unknown source
of classical random variables that might correlate the sources $S_{1},S_{2}$.
(right) Alice and Bob can explain the observed correlations $p(a,b)$ using a
NLHV model.

To witness swap-steering, a functional $W$ can be constructed which depends on
$\vec{p}$ as

$\displaystyle W(\vec{p})=\sum_{a,b}c_{a,b}p(a,b)\leq\beta_{SOHS}$ (5)

where $c_{a,b}$ are real coefficients and $\beta_{SOHS}$ denotes the maximum
value attainable using assemblages admitting a SOHS model (21). For the
purpose of this article, we consider only functionals that are linear over
$\vec{p}$.

Now, consider the following functional

$\displaystyle W=p(0,0)+p(1,1)+p(2,2)+p(3,3)\leq\beta_{SOHS}$ (6)

Recall here that Alice is trusted and performs the measurements with elements
given in (A). Let us now find the maximum value that can be achieved using
correlations that admit a SOHS model (21) the proof of which is given in [38].

######  Fact 1.

Consider the swap-steering functional $W$ (16). The maximum value
$\beta_{SOHS}$ that can be achieved using correlations that admit a SOHS model
(21) is $\beta_{SOHS}=\frac{1}{2}$.

Now, consider that the sources prepare the state
$|\psi_{i}\rangle=|\phi_{+}\rangle_{A_{i}B_{i}}$ and Bob performs the same
measurement as Alice, that is,
$M_{B}=\\{|\phi_{+}\rangle\\!\\!\langle\phi_{+}|,|\phi_{-}\rangle\\!\\!\langle\phi_{-}|,|\psi_{+}\rangle\\!\\!\langle\psi_{+}|,|\psi_{-}\rangle\\!\\!\langle\psi_{-}|\\}_{B_{0}B_{1}}$
where the corresponding states are given in (A). Using these states and Bob’s
measurement one can simply evaluate the steering functional $W$ in (16) to get
the value $1$, which is the quantum bound of $W$. Notice that this is also the
algebraic value of $W$.

Let us also show here that one can not observe Bell type non-locality with
only two parties without inputs. Without loss of generality, we consider here
the scenario similar to one depicted in Fig. 1 such that Alice and Bob perform
a measurement with arbitrary number of outcomes on subsystems sent by two
independent or classically correlated sources. However unlike the previous
scenario Alice is untrusted.

If the correlations $\vec{p}=\\{p(a,b)\\}$ admit a network-local hidden
variable (NLHV) model [15, 18], then they can be represented as

$\displaystyle
p(a,b)=\sum_{\lambda_{1},\lambda_{2}}p(\lambda_{1})p(\lambda_{2})p(a|\lambda_{1},\lambda_{2})p(b|\lambda_{1},\lambda_{2})$
(7)

for any $a,b$. Let us state the following fact which is simple to prove.

######  Fact 2.

Consider the scenario depicted in Fig. 1 with untrusted Alice. The
correlations $\vec{p}=\\{p(a,b)\\}$ obtained by Alice and Bob can always be
described by an NLHV model (7).

The above fact can be straightforwardly generalized to the scenario with
arbitrary number of sources between Alice and Bob. It is then well-known that
one can not observe any non-locality without inputs when there is a single
source distributing subsystems to Alice and Bob. Thus, to observe any form of
quantum non-locality in the minimal possible scenario, in the sense that there
are no inputs and only two parties, one has to trust either of the parties.
Consequently, quantum steering can also be observed in scenarios where one can
not observe Bell non-locality. Let us now show that states that are
unsteerable in the standard quantum steering scenario are swap-steerable.

Entanglement assisted activation of steerability— Let us now consider the
Werner state given by

$\displaystyle\rho^{W}(\alpha)=\alpha|\phi_{+}\rangle\\!\\!\langle\phi_{+}|+(1-\alpha)\frac{\mathbbm{1}}{4}.$
(8)

The above state is separable iff $\alpha\leq\frac{1}{3}$ [39]. As proven in
[10, 40], the above state is steerable in the standard quantum steering
scenario iff $\alpha>\frac{1}{2}$. Thus, in the range of
$\frac{1}{3}<\alpha\leq\frac{1}{2}$, the Werner state is unsteerable but
entangled. We show here that the Werner state when coupled with the maximally
entangled state is swap-steerable. Thus when assisted with entanglement,
unsteerable states can be activated to display steerability without inputs.

######  Fact 3.

The Werner state $\rho^{W}(\alpha)$ (25) with the maximally entangled state is
swap-steerable for any $\alpha>\frac{1}{3}$.

Thus, states that are unsteerable in the standard quantum steering scenario
can be activated using the maximally entangled state and shown to be swap-
steerable. However, we also notice that to observe swap-steering, the states
generated by both sources can not be unsteerable simultaneously. Let us now
find some necessary conditions to observe swap-steering.

Necessary conditions for swap-steering— Consider again the scenario depicted
in Fig. 1. Notice that one of the trivial necessary conditions to observe
swap-steering is that the trusted party, here Alice, needs to perform an
entangled measurement. Let us now restrict to the case when the number of
outcomes on Bob’s side is a composite number, that is, $b=b_{0}b_{1}$ where
$b_{0},b_{1}$ are positive integers. Now, Bob’s measurement $\\{N^{b}\\}$ with
$b=b_{0}b_{1}$ prepares a set of positive operators on the trusted Alice’s
side, known as assemblage, denoted as $\\{\sigma_{b}\\}$ where
$\sigma_{b}=\sum_{j}p_{j}\mathrm{Tr}_{B}(\mathbbm{1}_{A}\otimes
N^{b}\rho_{A_{1}B_{1}}^{j}\otimes\rho_{A_{2}B_{2}}^{j})$. Now, we show that if
the assemblage is of a particular form, one can never observe swap-steering.

######  Fact 4.

Consider the swap-steering scenario depicted in Fig. 1 where Alice and Bob
share the states $\rho_{A_{1}B_{1}},\rho_{A_{2}B_{2}}$. Let us assume that Bob
performs a $n-$outcome measurement which prepares the assemblage
$\\{\sigma_{b_{0}b_{1}}\\}$ on the trusted Alice’s side. If
$\sigma_{b_{0}b_{1}}$ is separable for $b_{0}=0,1,\ldots,n_{1}-1,\
b_{1}=0,1,\ldots,n_{2}-1$,then there exists a SOHS model for both the states
$\rho_{A_{1}B_{1}},\rho_{A_{2}B_{2}}$.

The proof of the above fact is given in Appendix C of [38]. Consequently, one
can observe from Fact 3 that if Bob performs a product measurement, then the
states are not swap-steerable from Bob to Alice. Further on, both states
prepared from the sources are needed to be entangled to observe swap-steering.
Thus, to observe swap-steering both the states and measurements must be
entangled.

Self-testing and randomness certification— Let us now utilise the above swap-
steering inequality (16) for self-testing the quantum realisations suggested
after Fact 1. For our purpose, we generalize the definition of self-testing in
one-sided DI introduced in [41, 42, 43]. Before proceeding, let us define
Alice’s observable corresponding to the Bell basis as

$\displaystyle
A_{0}=\sum_{k=1}^{4}\mathbbm{i}^{k}|\phi_{k}\rangle\\!\\!\langle\phi_{k}|.$
(9)

where
$|\phi_{1}\rangle=|\phi^{+}\rangle,|\phi_{2}\rangle=|\psi^{+}\rangle,|\phi_{3}\rangle=|\phi^{-}\rangle,|\phi_{4}\rangle=|\psi^{-}\rangle$.

######  Fact 5.

Assume that the steering inequality (16), with trusted Alice choosing the
observable $A_{0}$ (33), is maximally violated by a separable state
$\rho_{AB}$ acting on
$\mathbbm{C}^{2}\otimes\mathbbm{C}^{2}\otimes\mathcal{H}_{B}$ and Bob’s
observable $B_{0}$. Then, the following statements hold true:

1\. Bob’s measurement is projective with his Hilbert space decomposing as
$\mathcal{H}_{B}=(\mathbbm{C}^{2})_{B_{1}^{\prime}}\otimes(\mathbbm{C}^{2})_{B_{2}^{\prime}}\otimes\mathcal{H}_{B_{12}^{\prime\prime}}$
for some auxiliary Hilbert space
$\mathcal{H}_{B^{\prime\prime}_{12}}=\mathcal{H}_{B^{\prime\prime}_{1}}\otimes\mathcal{H}_{B^{\prime\prime}_{2}}$.

2\. There exist unitary transformations,
$U_{i}:\mathcal{H}_{B}\rightarrow\mathcal{H}_{B}$, such that

$\displaystyle(\mathbbm{1}_{A}\otimes U_{B})\rho_{AB}(\mathbbm{1}_{A}\otimes
U_{B}^{\dagger})\qquad\qquad\qquad\qquad$
$\displaystyle=|\phi^{+}\rangle\\!\\!\langle\phi^{+}|_{A_{1}B_{1}^{\prime}}\otimes|\phi^{+}\rangle\\!\\!\langle\phi^{+}|_{A_{2}B_{2}^{\prime}}\otimes\rho_{B_{1}^{\prime\prime}B_{2}^{\prime\prime}},$
(10)

where $B_{i}^{\prime\prime}$ denotes Bob’s auxiliary system, and

$\displaystyle\quad
U_{B}\,B_{0}\,U_{B}^{\dagger}=A_{0}\otimes\mathbbm{1}_{B_{1}^{\prime\prime}B_{2}^{\prime\prime}}$
(11)

where $U_{B}=U_{1}\otimes U_{2}$.

The proof of the above fact is given in Appendix D of [38]. An interesting
application of the above self-testing statement is that the untrusted Bob’s
measurement device can generate true randomness that is secure against any
adversary. For this purpose, we consider an eavesdropper Eve who has access to
Bob’s laboratory. Consequently, we consider a state $\rho_{ABE}$ which is
shared among Alice, Bob and Eve. As Eve’s dimension is unrestricted, we can
purify the state as $|\psi_{ABE}\rangle$ such that
$\mathrm{Tr}_{E}\psi_{ABE}=\rho_{AB}$ where $\rho_{AB}$ is separable.

Now, to certify whether the measurement outcomes as observed by Bob is truly
random, we consider that Eve wants to guess the outcome of Bob’s measurement.
In order to do so, she performs a measurement $Z=\\{E_{e}\\}$ on her part of
the shared states. Here the outcome $e$ is Eve’s best guess of Bob’s outcome.
However, any operation by Eve should not alter the statistics
$\vec{p}=\\{p(a,b)\\}$ observed by Alice and Bob, that is,

$\displaystyle p(a,b)=\langle\psi|M_{a}\otimes
N_{b}\otimes\mathbbm{1}_{E}|\psi\rangle.$ (12)

This is extremely important as the adversary Eve would like to remain
invisible to Alice and Bob.

The number of random bits that can be securely generated from Bob’s
measurement is quantified as $H_{\min}=-\log_{2}G(y,\vec{p})$ [20], where
$G(y,\vec{p})$ is known as the local guessing probability which can be
computed as,

$G(\vec{p})=\sup_{S\in S_{\vec{p}}}\sum_{b}\langle\psi|\mathbbm{1}_{A}\otimes
N_{b}\otimes E_{b}|\psi\rangle,$ (13)

where $S_{\vec{p}}$ is the set of all Eve’s strategies comprising of the
shared states and her measurement that reproduce the probability distribution
$\vec{p}$ as expected by Alice and Bob.

Let us now suppose that the swap-steering inequality (16) is maximally
violated by $\vec{p}$. As proven above in fact 4, this implies that the state
shared by Alice, Bob, and Eve up to local unitary operations is,
$|\psi_{{ABE}}\rangle=|\phi^{+}_{A_{1}B_{1}^{\prime}}\rangle|\phi^{+}_{A_{2}B_{2}^{\prime}}\rangle|\mathrm{\mathrm{aux}}_{\mathrm{B_{12}^{\prime\prime}E}}\rangle$
as well as
$N_{b}=|\phi_{b}\rangle\\!\\!\langle\phi_{b}|\otimes\mathbbm{1}_{\mathrm{B^{\prime\prime}_{12}}}$
where $|\phi_{b}\rangle$ are given above Eq. (33). Putting these states and
measurement in the formula (13) we obtain

$\displaystyle
G(\vec{p})=\sum_{b}\langle\phi^{+}|\langle\phi^{+}|(\mathbbm{1}_{A}\otimes|\phi_{b}\rangle\\!\\!\langle\phi_{b}|)|\phi^{+}\rangle|\phi^{+}\rangle$
$\displaystyle\langle\mathrm{aux}|\mathbbm{1}_{B_{12^{\prime\prime}}}\otimes
E_{b}|\mathrm{aux}\rangle.$ (14)

Now for all $b$,
$\langle\phi^{+}|\langle\phi^{+}|(\mathbbm{1}_{A}\otimes|\phi_{b}\rangle\\!\\!\langle\phi_{b}|)|\phi^{+}\rangle|\phi^{+}\rangle=1/4$
which allows us to conclude from (Network quantum steering enables randomness
certification without seed randomness) that

$\displaystyle
G(\vec{p})=\frac{1}{4}\sum_{b}\langle\mathrm{aux}|\mathbbm{1}_{B_{12^{\prime\prime}}}\otimes
E_{b}|\mathrm{aux}\rangle=\frac{1}{4}.$ (15)

Consequently, $-\log_{2}G(\vec{p})=2$ bits of randomness can be certified from
Bob’s measurement outcomes using our self-testing scheme.

It is important here to note here that the generation of secure randomness is
based on the assumption that the sources can only be correlated in a classical
way. However, the adversary can always guess the outcomes of Bob if she
manages to entangle the sources. For instance, (i) she can prepare both
devices beforehand or (ii) she herself could perform an entangled measurement
on the systems arriving on Bob’s side and then send the outcome to Bob. This
problem would persist in any security protocols involving two different
constrained sources. However, the second type of attack (ii) can be avoided if
Bob randomly chooses not to perform a measurement in some runs of the
experiment. Since Eve is unaware of this fact, she would still entangle both
sources and can be detected by Alice. It will be extremely interesting if
Alice and Bob can perform some local operations on their subsystems to figure
out whether the received subsystems are generated from separable sources or
not.

Discussions— The idea of quantum steering in networks was introduced recently
in [44]. However, the scenario considered in this work was not dealt with in
Ref. [44]. Further on, the notion of quantum steering in networks [44]
required the trusted party to perform a full tomography which implied that the
trusted party has input. Contrary to this, the notion of swap-steering even
the trusted party performs a single fixed measurement. This also makes our
scheme experimentally friendly as one has to consider less number of
correlations in order to witness quantum steering in networks. However, the
measurement elements of the trusted party are maximally entangled and thus it
would be beneficial to explore the possibilities of observing swap-steering
with less entangled measurmenents.

Constructing witnesses to observe quantum nonlocality in networks has been
extremely difficult mainly due to the fact that the network-local polytope
might not be convex as shown in [11, 13]. In this work, we find that assuming
one of the parties to be trusted allows constructing linear witnesses to
observe a form of quantum nonlocality in networks. One of the interesting
follow-up directions would be to explore the structure of the set of
correlations admitting the SOHS model. We showed in this work that any
entangled Werner state can be used to witness swap-steering. An interesting
follow-up question is whether every entangled state violates the notion of
swap-steering. Furthermore, we used swap-steering for the certification of
randomness without seed randomness. It will be highly desirable to generalize
the above scheme to the DI regime where no party is trusted. Another direction
would be to generalize the scheme presented in this work to certify an
unbounded amount of randomness. Moreover, it would be interesting to
investigate whether the randomness certification can be made robust to
experimental imperfections.

###### Acknowledgements.

We would like to thank Stefano Pironio for reviewing the manuscript and
providing critical comments that considerably improved the manuscript. This
project was funded within the QuantERA II Programme (VERIqTAS project) that
has received funding from the European Union’s Horizon 2020 research and
innovation programme under Grant Agreement No 101017733.

## References

  * Einstein et al. [1935] A. Einstein, B. Podolsky, and N. Rosen, Can quantum-mechanical description of physical reality be considered complete?, Phys. Rev. 47, 777 (1935).
  * Bell [1964] J. S. Bell, On the einstein podolsky rosen paradox, Physics Physique Fizika 1, 195 (1964).
  * Bell [1966] J. S. Bell, On the problem of hidden variables in quantum mechanics, Rev. Mod. Phys. 38, 447 (1966).
  * Aspect et al. [1982] A. Aspect, J. Dalibard, and G. Roger, Experimental test of bell’s inequalities using time-varying analyzers, Phys. Rev. Lett. 49, 1804 (1982).
  * Aspect et al. [1981] A. Aspect, P. Grangier, and G. Roger, Experimental tests of realistic local theories via bell’s theorem, Phys. Rev. Lett. 47, 460 (1981).
  * Giustina et al. [2015] M. Giustina, M. A. M. Versteegh, S. Wengerowsky, J. Handsteiner, A. Hochrainer, K. Phelan, F. Steinlechner, J. Kofler, J.-A. Larsson, C. Abellán, W. Amaya, V. Pruneri, M. W. Mitchell, J. Beyer, T. Gerrits, A. E. Lita, L. K. Shalm, S. W. Nam, T. Scheidl, R. Ursin, B. Wittmann, and A. Zeilinger, Significant-loophole-free test of bell’s theorem with entangled photons, Phys. Rev. Lett. 115, 250401 (2015).
  * Shalm et al. [2015] L. K. Shalm, E. Meyer-Scott, B. G. Christensen, P. Bierhorst, M. A. Wayne, M. J. Stevens, T. Gerrits, S. Glancy, D. R. Hamel, M. S. Allman, K. J. Coakley, S. D. Dyer, C. Hodge, A. E. Lita, V. B. Verma, C. Lambrocco, E. Tortorici, A. L. Migdall, Y. Zhang, D. R. Kumor, W. H. Farr, F. Marsili, M. D. Shaw, J. A. Stern, C. Abellán, W. Amaya, V. Pruneri, T. Jennewein, M. W. Mitchell, P. G. Kwiat, J. C. Bienfang, R. P. Mirin, E. Knill, and S. W. Nam, Strong loophole-free test of local realism, Phys. Rev. Lett. 115, 250402 (2015).
  * Brunner et al. [2014] N. Brunner, D. Cavalcanti, S. Pironio, V. Scarani, and S. Wehner, Bell nonlocality, Rev. Mod. Phys. 86, 419 (2014).
  * Schrödinger [1935] E. Schrödinger, Discussion of probability relations between separated systems, Mathematical Proceedings of the Cambridge Philosophical Society 31, 555–563 (1935).
  * Wiseman et al. [2007] H. M. Wiseman, S. J. Jones, and A. C. Doherty, Steering, entanglement, nonlocality, and the einstein-podolsky-rosen paradox, Phys. Rev. Lett. 98, 140402 (2007).
  * Branciard et al. [2010] C. Branciard, N. Gisin, and S. Pironio, Characterizing the nonlocal correlations created via entanglement swapping, Phys. Rev. Lett. 104, 170401 (2010).
  * Branciard et al. [2012] C. Branciard, D. Rosset, N. Gisin, and S. Pironio, Bilocal versus nonbilocal correlations in entanglement-swapping experiments, Phys. Rev. A 85, 032119 (2012).
  * Pironio and Massar [2013] S. Pironio and S. Massar, Security of practical private randomness generation, Phys. Rev. A 87, 012336 (2013).
  * Fritz [2012] T. Fritz, Beyond bell’s theorem: correlation scenarios, New Journal of Physics 14, 103001 (2012).
  * Renou et al. [2019] M.-O. Renou, E. Bäumer, S. Boreiri, N. Brunner, N. Gisin, and S. Beigi, Genuine quantum nonlocality in the triangle network, Phys. Rev. Lett. 123, 140401 (2019).
  * Renou and Beigi [2022] M.-O. Renou and S. Beigi, Nonlocality for generic networks, Phys. Rev. Lett. 128, 060401 (2022).
  * Pozas-Kerstjens et al. [2023] A. Pozas-Kerstjens, N. Gisin, and M.-O. Renou, Proofs of network quantum nonlocality in continuous families of distributions, Phys. Rev. Lett. 130, 090201 (2023).
  * Šupić et al. [2022] I. Šupić, J.-D. Bancal, Y. Cai, and N. Brunner, Genuine network quantum nonlocality and self-testing, Phys. Rev. A 105, 022206 (2022).
  * Żukowski et al. [1993] M. Żukowski, A. Zeilinger, M. A. Horne, and A. K. Ekert, “event-ready-detectors” bell experiment via entanglement swapping, Phys. Rev. Lett. 71, 4287 (1993).
  * Pironio et al. [2010] S. Pironio, A. Acín, S. Massar, A. B. de la Giroday, D. N. Matsukevich, P. Maunz, S. Olmschenk, D. Hayes, L. Luo, T. A. Manning, and C. Monroe, Random numbers certified by bell’s theorem, Nature 464, 1021 (2010).
  * Acín et al. [2016] A. Acín, S. Pironio, T. Vértesi, and P. Wittek, Optimal randomness certification from one entangled bit, Phys. Rev. A 93, 040102 (2016).
  * Woodhead et al. [2020] E. Woodhead, J. Kaniewski, B. Bourdoncle, A. Salavrakos, J. Bowles, A. Acín, and R. Augusiak, Maximal randomness from partially entangled states, Phys. Rev. Research 2, 042028 (2020).
  * Gómez et al. [2019] S. Gómez, A. Mattar, I. Machuca, E. S. Gómez, D. Cavalcanti, O. J. Farías, A. Acín, and G. Lima, Experimental investigation of partially entangled states for device-independent randomness generation and self-testing protocols, Phys. Rev. A 99, 032108 (2019).
  * Andersson et al. [2018] O. Andersson, P. Badziąg, I. Dumitru, and A. Cabello, Device-independent certification of two bits of randomness from one entangled bit and gisin’s elegant bell inequality, Phys. Rev. A 97, 012314 (2018).
  * Curchod et al. [2017] F. J. Curchod, M. Johansson, R. Augusiak, M. J. Hoban, P. Wittek, and A. Acín, Unbounded randomness certification using sequences of measurements, Phys. Rev. A 95, 020102 (2017).
  * Fehr et al. [2013] S. Fehr, R. Gelles, and C. Schaffner, Security and composability of randomness expansion from Bell inequalities, Phys. Rev. A 87, 012335 (2013).
  * Nieto-Silleras et al. [2014] O. Nieto-Silleras, S. Pironio, and J. Silman, Using complete measurement statistics for optimal device-independent randomness evaluation, New Journal of Physics 16, 013035 (2014).
  * Tavakoli et al. [2021] A. Tavakoli, M. Farkas, D. Rosset, J.-D. Bancal, and J. Kaniewski, Mutually unbiased bases and symmetric informationally complete measurements in bell experiments, Science Advances 7, eabc3847 (2021).
  * Šupić et al. [2016] I. Šupić, R. Augusiak, A. Salavrakos, and A. Acín, Self-testing protocols based on the chained bell inequalities, New J. Phys. 18, 035013 (2016).
  * Sarkar et al. [2021] S. Sarkar, D. Saha, J. Kaniewski, and R. Augusiak, npj Quantum Information 7, 151 (2021).
  * Borkała et al. [2022] J. J. Borkała, C. Jebarathinam, S. Sarkar, and R. Augusiak, Device-independent certification of maximal randomness from pure entangled two-qutrit states using non-projective measurements, Entropy 24, 10.3390/e24030350 (2022).
  * Renou et al. [2018] M.-O. Renou, J. Kaniewski, and N. Brunner, Self-testing entangled measurements in quantum networks, Phys. Rev. Lett. 121, 250507 (2018).
  * Šupić and Brunner [2022] I. Šupić and N. Brunner, Self-testing nonlocality without entanglement, arXiv:2203.13171 (2022).
  * Zhou et al. [2022] Q. Zhou, X.-Y. Xu, S. Zhao, Y.-Z. Zhen, L. Li, N.-L. Liu, and K. Chen, Robust self-testing of multipartite Greenberger-Horne-Zeilinger-state measurements in quantum networks, Phys. Rev. A 106, 042608 (2022).
  * Šupić et al. [2022] I. Šupić, J. Bowles, M.-O. Renou, A. Acín, and M. J. Hoban, Quantum networks self-test all entangled states (2022), arXiv:2201.05032 .
  * Sekatski et al. [2022] P. Sekatski, S. Boreiri, and N. Brunner, Partial self-testing and randomness certification in the triangle network (2022), arXiv:2209.09921 [quant-ph] .
  * Sarkar et al. [2023a] S. Sarkar, C. Datta, S. Halder, and R. Augusiak, Self-testing composite measurements and bound entangled state in a unified framework (2023a), arXiv:2301.11409 [quant-ph] .
  * [38] See Supplemental Material.
  * Werner [1989] R. F. Werner, Quantum states with einstein-podolsky-rosen correlations admitting a hidden-variable model, Phys. Rev. A 40, 4277 (1989).
  * Bowles et al. [2016] J. Bowles, F. Hirsch, M. T. Quintino, and N. Brunner, Sufficient criterion for guaranteeing that a two-qubit state is unsteerable, Phys. Rev. A 93, 022121 (2016).
  * Sarkar et al. [2022] S. Sarkar, D. Saha, and R. Augusiak, Certification of incompatible measurements using quantum steering, Phys. Rev. A 106, L040402 (2022).
  * Sarkar et al. [2023b] S. Sarkar, J. J. Borkała, C. Jebarathinam, O. Makuta, D. Saha, and R. Augusiak, Self-testing of any pure entangled state with the minimal number of measurements and optimal randomness certification in a one-sided device-independent scenario, Phys. Rev. Appl. 19, 034038 (2023b).
  * Sarkar [2023] S. Sarkar, Certification of the maximally entangled state using nonprojective measurements, Phys. Rev. A 107, 032408 (2023).
  * Jones et al. [2021] B. D. M. Jones, I. Šupić, R. Uola, N. Brunner, and P. Skrzypczyk, Network quantum steering, Phys. Rev. Lett. 127, 170405 (2021).

##  Appendix A Classical bound

The swap-steering inequality is given as

$\displaystyle W=p(0,0)+p(1,1)+p(2,2)+p(3,3)$ (16)

with Alice being trusted and her measurement corresponding to the Bell basis
is given by
$M_{A}=\\{|\phi_{+}\rangle\\!\\!\langle\phi_{+}|,|\phi_{-}\rangle\\!\\!\langle\phi_{-}|,|\psi_{+}\rangle\\!\\!\langle\psi_{+}|,|\psi_{-}\rangle\\!\\!\langle\psi_{-}|\\}_{A_{1}A_{2}}$
where

$\displaystyle|\phi_{\pm}\rangle_{A_{1}A_{2}}$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{2}}\left(|0\rangle_{A_{1}}|0\rangle_{A_{2}}\pm|1\rangle_{A_{1}}|1\rangle_{A_{2}}\right)$
$\displaystyle|\psi_{\pm}\rangle_{A_{1}A_{2}}$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{2}}\left(|0\rangle_{A_{1}}|1\rangle_{A_{2}}\pm|1\rangle_{A_{1}}|0\rangle_{A_{2}}\right).$
(17)

Now, given two sources $S_{i}$ for $i=1,2$ that generate some (for now hidden)
states $\lambda_{i}$, we can always express the probability $p(a,b)$ as

$\displaystyle
p(a,b)=\sum_{\lambda_{1},\lambda_{2}}p(\lambda_{1},\lambda_{2})p(a,b|\lambda_{1},\lambda_{2}).$
(18)

Using Bayes rule and the fact that Alice is known to be performing quantum
measurements, we can express the above expression as

$\displaystyle
p(a,b)=\sum_{\lambda_{1},\lambda_{2}}p(\lambda_{1},\lambda_{2})p(a|\rho_{\lambda_{1},\lambda_{2}})p(b|\lambda_{1},\lambda_{2},a).$
(19)

Assuming outcome-independence, we arrive at

$\displaystyle
p(a,b)=\sum_{\lambda_{1},\lambda_{2}}p(\lambda_{1},\lambda_{2})p(a|\rho_{\lambda_{1},\lambda_{2}})p(b|\lambda_{1},\lambda_{2}).$
(20)

Now, assuming uncorrelated sources we express $\rho_{\lambda_{1},\lambda_{2}}$
using pure state decompositions to arrive at the following expression of
$p(a,b)$

$\displaystyle
p(a,b)=\sum_{\lambda_{1},\lambda_{2}}p(\lambda_{1},\lambda_{2})\sum_{p^{j}_{\lambda_{1},\lambda_{2}}}p^{j}_{\lambda_{1},\lambda_{2}}p(a|\
|\psi^{j}_{\lambda_{1}}\rangle|\psi^{j}_{\lambda_{2}}\rangle)p(b|\lambda_{1},\lambda_{2}).$
(21)

If correlations $\vec{p}$ admit the form (21), then they are describable using
a separable outcome-independent hidden state (SOHS) model.

######  Fact 1.

Consider the swap-steering functional $W$ (16). The maximum value
$\beta_{SOHS}$ that can be achieved using correlations that admit a SOHS model
of $W$ is $\beta_{SOHS}=\frac{1}{2}$.

###### Proof.

The proof follows the exact same lines as presented in [41, 43, 42]. Let us
now consider the steering functional $W$ in Eq. (16) and express it in terms
of the SOHS model as

$\displaystyle\sum_{a=0}^{3}\sum_{\lambda_{1},\lambda_{2}}\
p(\lambda_{1},\lambda_{2})p(a|\rho_{\lambda_{1},\lambda_{2}})p(a|\lambda_{1},\lambda_{2})\leq\sum_{\lambda_{1},\lambda_{2}}\
p(\lambda_{1},\lambda_{2})\max_{a}\\{p(a|\rho_{\lambda_{1},\lambda_{2}})\\}$
(22)

where we used the fact that $\sum_{a}p(a|\lambda_{1},\lambda_{2})=1$ for any
$\lambda_{1},\lambda_{2}$. Now, maximising over
$\rho_{\lambda_{1},\lambda_{2}}$ gives us

$\displaystyle\sum_{\lambda_{1},\lambda_{2}}\
p(\lambda_{1},\lambda_{2})\max_{a}\\{p(a|\rho_{\lambda_{1},\lambda_{2}})\\}\leq\sum_{\lambda_{1},\lambda_{2}}\
p(\lambda_{1},\lambda_{2})\max_{\rho_{\lambda_{1},\lambda_{2}}}\max_{a}\\{p(a|\rho_{\lambda_{1},\lambda_{2}})\\}.$
(23)

Now, using the fact that $\sum_{\lambda_{i}}p(\lambda_{i})=1$ for $i=1,2$
allows us to conclude that

$\displaystyle\beta_{SOHS}\leq\max_{|\psi\rangle_{A_{1}},|\psi\rangle_{A_{2}}}\max_{a}\\{p(a|\
|\psi\rangle_{A_{1}},|\psi\rangle_{A_{2}})\\}.$ (24)

As the steering functional $W$ is linear, without loss of generality we
consider the maximization only over pure states. Now, putting in the
measurement of the trusted Alice (A), which locally acts on qubit Hilbert
spaces, and the optimizing over pure states
$|\psi\rangle_{A_{1}},|\psi\rangle_{A_{2}}\in\mathbbm{C}^{2}$ gives us
$\beta_{SOHS}\leq\frac{1}{2}$. This bound can be saturated when the sources
prepare the maximally mixed $\rho_{i}=\frac{1}{2}\left(|00\rangle\\!\\!\langle
00|+|11\rangle\\!\\!\langle 11|\right)_{A_{i}B_{i}}$ and the measurement with
Bob is $M_{B}=\\{|00\rangle\\!\\!\langle 00|,|01\rangle\\!\\!\langle
01|,|10\rangle\\!\\!\langle 10|,|11\rangle\\!\\!\langle 11|\\}_{B_{0}B_{1}}$.
This state clearly has a SOHS model and thus we get the desired SOHS bound. ∎

##  Appendix B Activation of quantum steering

The Werner state is given by

$\displaystyle\rho^{W}(\alpha)=\alpha|\phi_{+}\rangle\\!\\!\langle\phi_{+}|+(1-\alpha)\frac{\mathbbm{1}}{4}.$
(25)

The above state is separable iff $\alpha\leq\frac{1}{3}$ [39]. As proven in
[10, 40], the above state is steerable in the standard quantum steering
scenario iff $\alpha>\frac{1}{2}$.

######  Fact 2.

The Werner state $\rho^{W}(\alpha)$ (25) with the maximally entangled state is
swap-steerable for any $\alpha>\frac{1}{3}$.

###### Proof.

Consider the scenario presented in Fig. 1 of the manuscript. Now, suppose that
the source $S_{i}$ generates the state $\rho^{W}_{A_{i}B_{i}}(\alpha_{i})$ for
$i=1,2$. Bob again performs the Bell basis measurement $M_{B}$. Given these
states and measurements, let us again evaluate the steering functional $W$ in
(16) to obtain

$\displaystyle W=\frac{3\alpha_{1}\alpha_{2}+1}{4}.$ (26)

As proven above in Fact 1, if $W>\frac{1}{2}$ then the state is swap-steerable
from Bob to Alice. Thus, we have from (26) that the Werner state (25) is
steerable if $\frac{3\alpha_{1}\alpha_{2}+1}{4}>\frac{1}{2}$. Consequently,
for any value of $\alpha_{1}\alpha_{2}>\frac{1}{3}$, the Werner states are
swap-steerable. Let us now observe that if $\alpha_{1}=1$, that is, the source
$S_{1}$ generates maximally entangled state, then for any
$\alpha_{2}>\frac{1}{3}$ the Werner state becomes swap-steerable. ∎

##  Appendix C Necessary conditions for swap-steering

######  Fact 3.

Consider the swap-steering scenario depicted in Fig. 1 of the manuscript where
Alice and Bob share the states $\rho_{A_{1}B_{1}},\rho_{A_{2}B_{2}}$. Let us
assume that Bob performs a $n-$outcome measurement which prepares the
assemblage $\\{\sigma_{b_{0}b_{1}}\\}$ on the trusted Alice’s side. If
$\sigma_{b_{0}b_{1}}$ is separable for $b_{0}=0,1,\ldots,n_{1}-1,\
b_{1}=0,1,\ldots,n_{2}-1$,then there exists a SOHS model for both the states
$\rho_{A_{1}B_{1}},\rho_{A_{2}B_{2}}$.

###### Proof.

Let us first notice that

$\displaystyle\sum_{b_{0},b_{1}}\sigma_{b_{0}b_{1}}$ $\displaystyle=$
$\displaystyle\sum_{b_{0},b_{1},j}p_{j}\mathrm{Tr}_{B}(\mathbbm{1}_{A}\otimes
N^{b}\rho_{A_{1}B_{1}}^{j}\otimes\rho^{j}_{A_{2}B_{2}})$ (27) $\displaystyle=$
$\displaystyle\sum_{j}p_{j}\rho_{A_{1}}^{j}\otimes\rho_{A_{2}}^{j}$

which also allows us to conclude that
$\sum_{b_{0},b_{1}}\mathrm{Tr}(\sigma_{b_{0}b_{1}})=1$. Consider now the
assemblage $\\{\sigma_{b_{0}b_{1}}\\}$ is separable, that is, the operators
$\sigma_{b_{0}b_{1}}=\sum_{j}\sigma_{b_{0}}^{j}\otimes\sigma_{b_{1}}^{j}$.
Notice that the following states

$\displaystyle\tilde{\rho}_{A_{i}B_{i}}^{j}=\frac{1}{\mathcal{N}_{i,j}}\sum_{b_{i}=0}^{n_{i}-1}\sigma_{b_{i},A_{i}}^{j}\otimes|b_{i}\rangle\\!\\!\langle
b_{i}|_{B_{i}}$ (28)

where $\mathcal{N}_{i,j}=\sum_{b_{i}}\mathrm{Tr}(\sigma_{b_{i}}^{j})$ and Bob
performing a measurement of the form

$\displaystyle\tilde{M}_{b_{0}b_{1}}=|b_{0}\rangle\\!\\!\langle
b_{0}|_{B_{1}}\otimes|b_{1}\rangle\\!\\!\langle b_{1}|_{B_{2}}\qquad
b_{i}=0,1\ldots,n_{i}-1$ (29)

give the same assemblage on Alice’s side as the states
$\sum_{j}p_{j}\rho_{A_{1}B_{1}}^{j}\rho^{j}_{A_{2}B_{2}}$ and the measurement
$M_{b}=\\{N^{b}\\}$. It is straightforward to observe that the states
$\tilde{\rho}_{A_{i}B_{i}}$ are separable and thus the $\rho_{A_{i}B_{i}}$
admit a SOHS model. ∎

##  Appendix D Self-testing

In quantum theory, it is advantageous to express the correlations
$\\{p(a,b)\\}$in terms of expectation values rather than probability
distributions. When dealing with $d$-outcome measurements, a useful technique
is to utilize the two-dimensional Fourier transform of the conditional
probabilities $p(a,b)$ as

$\langle
A^{(k)}_{0}B^{(l)}_{0}\rangle=\sum^{d-1}_{a,b=0}\omega^{ak+bl}p(a,b),$ (30)

where $\omega$ is the $d$-th root of unity $\omega=\exp(2\pi\mathbbm{i}/d)$
and $k,l=0,\ldots,d-1$ and $A^{(k)}_{0},B^{(l)}_{0}$ are known as observables.
Using the inverse Fourier transform of (30), we obtain that

$\displaystyle
p(a,b)=\frac{1}{d^{2}}\sum_{k,l=0}^{d-1}\omega^{-(ak+bl)}\langle
A^{(k)}_{0}B^{(l)}_{0}\rangle.$ (31)

The expectation value appearing on the left-hand side of Eq. (30) can be
simply represented as $\langle
A^{(k)}_{0}B^{(l)}_{0}\rangle=\mathrm{Tr}(A^{(k)}_{0}\otimes
B^{(l)}_{0}\rho_{AB})$ for some state $\rho_{AB}$ with $\\{A^{(k)}_{0}\\}$ and
$\\{B^{(l)}_{0}\\}$ are operators defined as

$A^{(k)}_{0}=\sum^{d-1}_{a=0}\omega^{ak}P^{(a)},\qquad
B^{(l)}_{0}=\sum^{d-1}_{b=0}\omega^{bl}Q^{(b)}.$ (32)

where $P^{(a)},Q^{(b)}$ represent the measurement elements of Alice, Bob
respectively. As proven in [Jed1], the observables $A^{(k)}_{0}$ have the
following properties (same for $B^{(l)}_{0}$):
$A^{(d-k)}_{0}=(A^{(k)}_{0})^{\dagger}$ and
$A^{(k)}_{0}(A^{(k)}_{0})^{\dagger}\leq\mathbbm{1}$. For the special case of
projective measurements, the observables $A^{(k)}_{0}$ are unitary and
$A^{(k)}_{0}=(A^{(1)}_{0})^{k}=A^{k}_{0}$. As Alice performs the Bell-basis
measurement whose corresponding measurement elements for the rest of the
manuscript will be denoted as
$|\phi_{1}\rangle=|\phi^{+}\rangle,|\phi_{2}\rangle=|\psi^{+}\rangle,|\phi_{3}\rangle=|\phi^{-}\rangle,|\phi_{4}\rangle=|\psi^{-}\rangle$
and the corresponding observable using (32) is given as

$\displaystyle
A_{0}=\sum_{k=1}^{4}\mathbbm{i}^{k}|\phi_{k}\rangle\\!\\!\langle\phi_{k}|.$
(33)

Let us first revisit the swap-steering inequality (16) and then using (31),
the above steering inequality can be simply represented as

$\displaystyle W=\frac{1}{4}\sum_{k=0}^{3}\langle A_{0}^{k}\otimes
B_{0}^{(4-k)}\rangle\leq\beta_{LHS}$ (34)

As shown in [sarkar15], the quantum bound of the above steering inequality is
$1$ which is also the maximum algebraic value of $W$. Consequently, we observe
from (34) that the maximum value can be attained iff each term is $1$, that
is, for $k=0,1,2,3$

$\displaystyle\langle A_{0}^{k}\otimes B_{0}^{(4-k)}\rangle=1.$ (35)

Now, using Cauchy-Schwarz inequality we get that

$\displaystyle A_{0}^{k}\otimes B_{0}^{(4-k)}\rho_{AB}=\rho_{AB}.$ (36)

Recalling that $\rho_{AB}$ is separable, we can express it as
$\rho_{AB}=\sum_{j}p_{j}\ \rho^{j}_{A_{1}B_{1}}\otimes\rho^{j}_{A_{2}B_{2}}$
which using its eigendecomposition can be expressed as
$\rho_{AB}=\sum_{s,s^{\prime}}p_{s,s^{\prime}}|\psi_{s,A_{1}B_{1}}\rangle\\!\\!\langle\psi_{s,A_{1}B_{1}}|\otimes|\psi_{s^{\prime},A_{2}B_{2}}\rangle\\!\\!\langle\psi_{s^{\prime},A_{2}B_{2}}|$.
Consequently, we get from the above expression Eq. (38) that

$\sum_{s,s^{\prime}}p_{s,s^{\prime}}A_{0}^{k}\otimes B_{0}^{(4-k)}\
\psi^{1}_{s}\otimes\psi_{s^{\prime}}^{2}=\sum_{s,s^{\prime}}p_{s,s^{\prime}}\
\psi^{1}_{s}\otimes\psi_{s^{\prime}}^{2}$ (37)

where for simplicity, we represent the states
$|\psi_{s,A_{i}B_{i}}\rangle\\!\\!\langle\psi_{s,A_{i}B_{i}}|$ as
$\psi^{i}_{s}$. It is now straightforward to observe from the above relation
that for all $s,s^{\prime}$

$\displaystyle A_{0}^{k}\otimes\overline{B}_{0,ss^{\prime}}^{4-k}\
|\psi^{1}_{s}\rangle|\psi^{2}_{s^{\prime}}\rangle=|\psi^{1}_{s}\rangle|\psi^{2}_{s^{\prime}}\rangle$
(38)

Here $\overline{B}_{0,ss^{\prime}}$ is the projection of $B_{0}$ on the
support of
$\mathrm{Tr}_{A}\psi^{1}_{s}\otimes\mathrm{Tr}_{A}\psi^{2}_{s^{\prime}}$. The
above relations are sufficient to self-test the state $\rho_{AB}$ and Bob’s
measurement $B_{0}$. Before proceeding toward the self-testing result, it is
important to recall the assumption that the local states are full-rank as the
measurements can only be characterized on the local support of the states. For
a note, we closely follow the techniques introduced in [41].

######  Fact 4.

Assume that the steering inequality (34), with trusted Alice choosing the
observable $A_{0}$ (33), is maximally violated by a separable state
$\rho_{AB}$ acting on
$\mathbbm{C}^{2}\otimes\mathbbm{C}^{2}\otimes\mathcal{H}_{B}$ and Bob’s
observable $B_{0}$. Then, the following statements hold true:

1\. Bob’s measurement is projective with his Hilbert space decomposing as
$\mathcal{H}_{B}=(\mathbbm{C}^{2})_{B_{1}^{\prime}}\otimes(\mathbbm{C}^{2})_{B_{2}^{\prime}}\otimes\mathcal{H}_{B_{12}^{\prime\prime}}$
for some auxiliary Hilbert space
$\mathcal{H}_{B^{\prime\prime}_{12}}=\mathcal{H}_{B^{\prime\prime}_{1}}\otimes\mathcal{H}_{B^{\prime\prime}_{2}}$.

2\. There exist unitary transformations,
$U_{i}:\mathcal{H}_{B}\rightarrow\mathcal{H}_{B}$, such that

$\displaystyle(\mathbbm{1}_{A}\otimes U_{B})\rho_{AB}(\mathbbm{1}_{A}\otimes
U_{B}^{\dagger})\qquad\qquad\qquad\qquad$
$\displaystyle=|\phi^{+}\rangle\\!\\!\langle\phi^{+}|_{A_{1}B_{1}^{\prime}}\otimes|\phi^{+}\rangle\\!\\!\langle\phi^{+}|_{A_{2}B_{2}^{\prime}}\otimes\rho_{B_{1}^{\prime\prime}B_{2}^{\prime\prime}},$
(39)

where $B_{i}^{\prime\prime}$ denotes Bob’s auxiliary system, and

$\displaystyle\quad
U_{B}\,B_{0}\,U_{B}^{\dagger}=A_{0}\otimes\mathbbm{1}_{B_{1}^{\prime\prime}B_{2}^{\prime\prime}}$
(40)

where $U_{B}=U_{1}\otimes U_{2}$.

###### Proof.

Let us first show that Bob’s measurement is projective. For this purpose, we
consider the relations (36) for $k=1$ and then multiply it with
$A_{0}^{3}\otimes B_{0}$ to obtain

$\displaystyle\mathbbm{1}_{A}\otimes B_{0}B_{0}^{(3)}\
\rho_{AB}=A_{0}^{3}\otimes B_{0}\ \rho_{AB}$ (41)

where we used the fact that $A_{0}^{4}=\mathbbm{1}_{A}$. Notice that the
right-hand side of the above expression (41) can be simplified using the
relation (36) for $k=3$ to obtain

$\displaystyle\mathbbm{1}_{A}\otimes B_{0}B_{0}^{(3)}\ \rho_{AB}=\rho_{AB}.$
(42)

Thus, taking a partial trace over Alice’s subsystem and recalling that
$B_{0}^{(3)}=B_{0}^{\dagger}$ gives us

$\displaystyle B_{0}B_{0}^{\dagger}\ \rho_{B}=\rho_{B}$ (43)

where $\rho_{B}=\mathrm{Tr}_{B}\ \rho_{AB}$. As the local states are full-
rank, they are invertible too and consequently one can arrive at

$\displaystyle B_{0}B_{0}^{\dagger}=\mathbbm{1}_{B}.$ (44)

Similarly, one can also find that $B_{0}^{\dagger}B_{0}=\mathbbm{1}_{B}$. Both
these relations of Bob’s observable suggest that the observable $B_{0}$ and
unitary, and thus Bob’s measurement is projective. In a similar manner,
considering the relation (38) one can observe that
$\overline{B}_{0,ss^{\prime}}$ for all $s,s^{\prime}$ are unitary.

Let us now consider the relation Eq. (38) and characterize the states
$|\psi^{1}_{s}\rangle,|\psi^{2}_{s^{\prime}}\rangle$ that satisfy the relation
(38). For simplicity, we drop the indices $s,s^{\prime}$ for now. As the local
states on Alice’s side belong to $\mathbb{C}^{2}$, using Schmidt decomposition
we represent $|\psi^{1}\rangle,|\psi^{2}\rangle$ as

$\displaystyle|\psi^{i}\rangle=\sum_{j=0,1}\lambda_{j,i}|e_{j,i}\rangle|f_{j,i}\rangle$
(45)

where $\lambda_{j,i}\geq 0$ and $\\{|e_{j,i}\rangle\\},\\{|f_{j,i}\rangle\\}$
form an orthonormal basis for each $i$. Now applying a unitary $U_{i}$ on
these states such that $U_{i}|f_{j,i}\rangle=|e^{*}_{j,i}\rangle$ gives us

$\displaystyle|\tilde{\psi}^{i}\rangle=U_{i}|\psi^{i}\rangle=\sum_{j=0,1}\lambda_{j,i}|e_{j,i}\rangle|e^{*}_{j,i}\rangle.$
(46)

Now, notice that the state on the right-hand side can be represented as

$\displaystyle|\tilde{\psi}^{i}\rangle=P_{i}\otimes\mathbbm{1}_{B_{i}}|\phi^{+}\rangle$
(47)

where

$\displaystyle
P_{i}=\sqrt{2}\sum_{j=0,1}\lambda_{j,i}|e_{j,i}\rangle\\!\\!\langle e_{j,i}|.$
(48)

Notice that $P_{i}$ is full-rank as states that are separable between Alice
and Bob can not violate the swap-steering inequality (16). Putting the state
(47) in the relation (38) gives us

$\displaystyle A_{0}(P_{1}\otimes
P_{2})\otimes\tilde{B}_{0}^{\dagger}|\phi^{+}\rangle|\phi^{+}\rangle=P_{1}\otimes
P_{2}|\phi^{+}\rangle|\phi^{+}\rangle$ (49)

where $\tilde{B}_{0}=U_{1}^{\dagger}\otimes U_{2}^{\dagger}\ \overline{B}_{0}\
U_{1}\otimes U_{2}$. Now, using the fact that

$\displaystyle|\phi^{+}\rangle_{A_{1}B_{1}}|\phi^{+}\rangle_{A_{2}B_{2}}=|\phi^{+}_{4}\rangle_{A_{1}A_{2}|B_{1}B_{2}}$
(50)

where $|\phi^{+}_{4}\rangle$ is the maximally entangled state of local
dimension four. This allows us to conclude from (49) that

$\displaystyle(P_{1}^{-1}\otimes P_{2}^{-1})A_{0}(P_{1}\otimes
P_{2})\otimes\tilde{B}_{0}^{\dagger}\
|\phi^{+}_{4}\rangle=|\phi^{+}_{4}\rangle.$ (51)

Now, using the fact that $R\otimes
Q|\phi^{+}\rangle=RQ^{T}\otimes\mathbbm{1}|\phi^{+}\rangle$, where $T$ denotes
the transpose in the computational basis, gives us

$\displaystyle(P_{1}^{-1}\otimes P_{2}^{-1})A_{0}(P_{1}\otimes
P_{2})\tilde{B}_{0}^{*}\otimes\mathbbm{1}_{B}|\phi^{+}_{4}\rangle=|\phi^{+}_{4}\rangle.$
(52)

Taking the partial trace over $B^{\prime}s$ subsystem allows us to conclude
that

$\displaystyle(P_{1}^{-1}\otimes P_{2}^{-1})A_{0}(P_{1}\otimes
P_{2})\tilde{B}_{0}^{*}=\mathbbm{1}_{A}$ (53)

which eventually leads us to Bob’s measurement being

$\displaystyle\tilde{B}^{T}_{0}=(P_{1}^{-1}\otimes
P_{2}^{-1})A_{0}(P_{1}\otimes P_{2}).$ (54)

As $\tilde{B}_{0}$ is unitary and $P_{1},P_{2}$ are Hermitian, we get from the
above condition that

$\displaystyle(P_{1}^{-1}\otimes P_{2}^{-1})A_{0}(P_{1}\otimes
P_{2})^{2}A_{0}^{\dagger}(P_{1}^{-1}\otimes P_{2}^{-1})=\mathbbm{1}_{A}.$ (55)

Rearranging the terms we obtain that

$\displaystyle A_{0}(P_{1}\otimes P_{2})^{2}=(P_{1}\otimes P_{2})^{2}A_{0}$
(56)

which is equivalent to

$\displaystyle[A_{0},(P_{1}\otimes P_{2})^{2}]=0.$ (57)

Now, notice that if two matrices commute then they share the same basis.
However, the matrix $A_{0}$ has an entangled basis and the matrix
$P_{1}\otimes P_{2}$ have a product basis. Thus, the only instance for these
two matrices to commute is when $P_{1}\otimes P_{2}=\mathbbm{1}$ which imposes
that $P_{1}=P_{2}=\mathbbm{1}$. Going back to Eq. (47) allows us to conclude
that the states $|\psi^{1}\rangle,|\psi^{2}\rangle$ are the maximally
entangled state, that is,

$\displaystyle\mathbbm{1}_{A}\otimes
U_{i}|\psi^{i}\rangle=|\phi^{+}\rangle\quad i=1,2$ (58)

and Bob’s measurement using (54) is

$\displaystyle U_{1}^{\dagger}\otimes U_{2}^{\dagger}\ \overline{B}_{0}\
U_{1}\otimes U_{2}=A_{0}^{T}=A_{0}.$ (59)

Let us now bring back the indices $s,s^{\prime}$ and rewrite the states and
measurements as

$\displaystyle|\psi^{i}_{s}\rangle=\frac{1}{\sqrt{2}}\sum_{j=0,1}|j\rangle|f_{j,i,s}\rangle$
(60)

where $U_{s,i}^{\dagger}|j\rangle=|f_{j,i,s}\rangle$ and

$\displaystyle\overline{B}_{0,ss^{\prime}}=U_{s,1}\otimes U_{s^{\prime},2}\
A_{0}\ U_{s,1}^{\dagger}\otimes U_{s^{\prime},2}^{\dagger}$ (61)

for all $s,s^{\prime}$. From Theorem 1.1 of [41], we can express $B_{0}$ as

$\displaystyle B_{0}=\overline{B}_{0,ss^{\prime}}\oplus E_{ss^{\prime}}$ (62)

where $E_{ss^{\prime}}$ are unitary matrices.

Let us now denote Bob’s local support of the states $|\psi^{i}_{s}\rangle$ as
$V_{i,s}=\ $span$\\{|f_{0,i,s}\rangle\\!\\!\langle
f_{0,i,s}|,|f_{1,i,s}\rangle\\!\\!\langle f_{1,i,s}|\\}$ for all $i,s$.
Further on, we will show that the supports $V_{i,l},V_{i,l^{\prime}}$ are
orthogonal for any $l,l^{\prime}$. For this purpose, we first express the
product of the states $|\psi^{1}_{s}\rangle|\psi^{2}_{s^{\prime}}\rangle$ as

$\displaystyle|\psi^{1}_{s}\rangle|\psi^{2}_{s^{\prime}}\rangle=\frac{1}{2}\sum_{i,j=0,1}|ij\rangle|f_{i,1,s}\rangle|f_{j,2,s^{\prime}}\rangle$
(63)

which can equivalently be expressed using the Bell basis as

$\displaystyle|\psi^{1}_{s}\rangle|\psi^{2}_{s^{\prime}}\rangle=\frac{1}{2}\sum_{i=1}^{4}|\phi_{i}\rangle|g^{i}_{ss^{\prime}}\rangle$
(64)

where $|\phi_{i}\rangle$ are given just above Eq. (33) and

$\displaystyle|g^{1}_{ss^{\prime}}\rangle=\frac{1}{\sqrt{2}}\left(|f_{0,1,s}\rangle|f_{0,2,s^{\prime}}\rangle+|f_{1,1,s}\rangle|f_{1,2,s^{\prime}}\rangle\right)$
$\displaystyle|g^{2}_{ss^{\prime}}\rangle=\frac{1}{\sqrt{2}}\left(|f_{0,1,s}\rangle|f_{1,2,s^{\prime}}\rangle+|f_{1,1,s}\rangle|f_{0,2,s^{\prime}}\rangle\right)$
$\displaystyle|g^{3}_{ss^{\prime}}\rangle=\frac{1}{\sqrt{2}}\left(|f_{0,1,s}\rangle|f_{0,2,s^{\prime}}\rangle-|f_{1,1,s}\rangle|f_{1,2,s^{\prime}}\rangle\right)$
$\displaystyle|g^{4}_{ss^{\prime}}\rangle=\frac{1}{\sqrt{2}}\left(|f_{0,1,s}\rangle|f_{1,2,s^{\prime}}\rangle-|f_{1,1,s}\rangle|f_{0,2,s^{\prime}}\rangle\right)$
(65)

Let us again utilize the relation (38) and apply the state (64) to it to
observe that

$\displaystyle\sum_{i=1}^{4}\omega^{i}|\phi_{i}\rangle
B_{0}^{3}|g^{i}_{ss^{\prime}}\rangle=\sum_{i=1}^{4}|\phi_{i}\rangle|g^{i}_{ss^{\prime}}\rangle.$
(66)

Multiplying with $\langle\phi_{i}|$ on both sides of the above expression
gives us

$\displaystyle\omega^{i}B_{0}^{3}|g^{i}_{ss^{\prime}}\rangle=|g^{i}_{ss^{\prime}}\rangle\qquad\forall
i.$ (67)

As $B_{0}$ is unitary, we can conclude from the above formula (67) that

$\displaystyle\langle g^{j}_{ll^{\prime}}|g^{i}_{ss^{\prime}}\rangle=0\qquad
i\neq j$ (68)

for any $i,j,l,l^{\prime},s,s^{\prime}$. Let us now consider Eq. (68) with
$l=s,j=1$ and expand it using (D) to obtain the following conditions for
$i=2,3,4$ as

$\langle{f_{0,2,l^{\prime}}}|f_{0,2,s^{\prime}}\rangle-\langle{f_{1,2,l^{\prime}}}|f_{1,2,s^{\prime}}\rangle=0$
(69a)
$\langle{f_{0,2,l^{\prime}}}|f_{1,2,s^{\prime}}\rangle+\langle{f_{1,2,l^{\prime}}}|f_{0,2,s^{\prime}}\rangle=0$
(69b)
$\langle{f_{0,2,l^{\prime}}}|f_{1,2,s^{\prime}}\rangle-\langle{f_{1,2,l^{\prime}}}|f_{0,2,s^{\prime}}\rangle=0.$
(69c)

From Eqs. (69b) and (69c), it is straightforward to observe that
$\langle{f_{0,2,l^{\prime}}}|f_{1,2,s^{\prime}}\rangle=\langle{f_{1,2,l^{\prime}}}|f_{0,2,s^{\prime}}\rangle=0$.
Let us now recall that $|\psi^{2}_{l^{\prime}}\rangle$ and
$|\psi^{2}_{s^{\prime}}\rangle$ are orthogonal as they correspond to two
different eigenvectors of $\rho_{AB}$ which gives us an additional condition

$\displaystyle\langle{f_{0,2,l^{\prime}}}|f_{0,2,s^{\prime}}\rangle+\langle{f_{1,2,l^{\prime}}}|f_{1,2,s^{\prime}}\rangle=0.$
(70)

It is again straightforward to observe from (69a) and (70) that
$\langle{f_{0,2,l^{\prime}}}|f_{0,2,s^{\prime}}\rangle=\langle{f_{1,2,l^{\prime}}}|f_{1,2,s^{\prime}}\rangle=0$.
Thus, the local supports $V_{2,s^{\prime}}$ and $V_{2,l^{\prime}}$ are
orthogonal for any $s^{\prime},l^{\prime}$ such that $s^{\prime}\neq
l^{\prime}$. Proceeding the same way as above, we can also conclude that the
local supports $V_{1,s}$ and $V_{1,l}$ are orthogonal for any $s,l$ such that
$s\neq l$. Consequently, the local supports $V_{ss^{\prime}}=V_{1,s}\otimes
V_{2,s^{\prime}}$ are mutually orthogonal for any $s,s^{\prime}$.

The local supports $V_{ss^{\prime}}$ being mutually orthogonal imply that
Bob’s Hilbert space admits the following decomposition

$\mathcal{H}_{B}=\bigoplus_{s}\bigoplus_{s^{\prime}}V_{ss^{\prime}}=\bigoplus_{s}V_{1,s}\otimes\bigoplus_{s^{\prime}}V_{2,s^{\prime}}.$
(71)

As $\dim V_{1,s}=\dim V_{2,s^{\prime}}=2$ for any $s,s^{\prime}$, we can
straightforwardly conclude that
$\mathcal{H}_{B}=(\mathbbm{C}^{2})_{B^{\prime}_{1}}\otimes(\mathbbm{C}^{2})_{B^{\prime}_{1}}\otimes\mathcal{H}_{B^{\prime\prime}_{12}}$
where
$\mathcal{H}_{B^{\prime\prime}_{1}}\otimes\mathcal{H}_{B^{\prime\prime}_{2}}$
for some Hilbert spaces $\mathcal{H}_{B^{\prime\prime}_{i}}$.

The rest of the proof is exactly the same as step 3 in Theorem 1.2 of [41],
which allows us conclude that there exist unitary transformations,
$U_{i}:\mathcal{H}_{B}\rightarrow\mathcal{H}_{B}$, such that

$\displaystyle(\mathbbm{1}_{A}\otimes U_{1}\otimes
U_{2})\rho_{AB}(\mathbbm{1}_{A}\otimes U_{1}^{\dagger}\otimes
U_{2}^{\dagger})\qquad\qquad\qquad$
$\displaystyle=|\phi^{+}\rangle\\!\\!\langle\phi^{+}|_{A_{1}B_{1}^{\prime}}\otimes|\phi^{+}\rangle\\!\\!\langle\phi^{+}|_{A_{2}B_{2}^{\prime}}\otimes\rho_{B_{1}^{\prime\prime}B_{2}^{\prime\prime}},$
(72)

where $\rho_{B_{1}^{\prime\prime}B_{2}^{\prime\prime}}$ denotes Bob’s
auxiliary state which is separable with

$\displaystyle U_{i}=\bigoplus_{s}U_{s,i}\qquad i=1,2$ (73)

and

$\displaystyle U_{1}\otimes U_{2}\,B_{0}\,U_{1}^{\dagger}\otimes
U_{2}^{\dagger}=A_{0}\otimes\mathbbm{1}_{B_{1}^{\prime\prime}B_{2}^{\prime\prime}}.$
(74)

This completes the proof. ∎


# Multiverse Transformer: 1st Place Solution for Waymo Open Sim Agents
Challenge 2023

Yu Wang Tiebiao Zhao Fan Yi
Pegasus
<EMAIL_ADDRESS>
###### Abstract

This technical report presents our 1st place solution for the Waymo Open Sim
Agents Challenge (WOSAC) 2023. Our proposed MultiVerse Transformer for Agent
simulation (MVTA) effectively leverages transformer-based motion prediction
approaches, and is tailored for closed-loop simulation of agents. In order to
produce simulations with a high degree of realism, we design novel training
and sampling methods, and implement a receding horizon prediction mechanism.
In addition, we introduce a variable-length history aggregation method to
mitigate the compounding error that can arise during closed-loop
autoregressive execution. On the WOSAC, our MVTA and its enhanced version MVTE
reach a realism meta-metric of 0.5091 and 0.5168, respectively, outperforming
all the other methods on the leaderboard. †† Project page: https://multiverse-
transformer.github.io/sim-agents/

Figure 1: Simulations of agents produced by MVTA. 4 parallel universes are
visualized for each of the two scenes, in which the autonomous driving vehicle
(ADV) uniquely represented by a car icon for easy identification is
undertaking a U-turn maneuver (top) or making a left turn (bottom). For
illustrative purposes, eight timesteps are shown for each agent, and time
progression is encoded in the opacity of the boxes. Figure 2: Main
Architecture of the MVTA. The world agents, ADV and road graph inputs are
processed by transformer-based encoding steps to generate enhanced scene
context features. In the decoding step, rollout is executed in an
autoregressive manner. The ADV operates at 0.1s intervals, and concurrently,
each world agent decoder also simulates the forthcoming states at the same
0.1s interval. Q denotes the query content feature, while K and V stand for
the keys and values, respectively. The coin flip icon indicates sampling at
each timestep.

##  1 Introduction

The simulation of traffic agents is an integral element for evaluating self-
driving systems, facilitating rapid development and ensuring safety [17].
WOSAC [11] is the first public benchmark for the evaluation of simulation
agents in the domain of autonomous driving, introducing new evaluation metrics
and leveraging large-scale real-world logged data [4] with a diverse set of
scenarios and agent behaviors.

Recent advancements in traffic agent simulators [17, 2, 5, 20, 8, 15, 18] have
shown a notable shift towards learning from logged real-world driving data and
data-driven generative models conditioned on the scene context, rather than
relying on traditional heuristic-based models encoding traffic rules. Our
proposed simulator also falls within the learning-based generative model
paradigm. Specifically, we leverage state-of-the-art motion prediction models
[16, 12] and adapt them to an autoregressive closed-loop agent simulator.

However, for autoregressive models, the error can accumulate over time, as
future state of agents is heavily dependent on the immediate past. This can
potentially lead to drift, where the predictions become increasingly
inaccurate over time.

Inspired by TrafficSim [17] and MTR [16], we propose a novel closed-loop
simulation framework based on transformer encoder and decoder, and further
enhance the realism of the simulations through the development of novel
training and sampling strategies, as well as the receding horizon prediction
and variable-length history aggregation methods. Several example simulations
generated by MVTA are depicted in Figure Multiverse Transformer: 1st Place
Solution for Waymo Open Sim Agents Challenge 2023.

##  2 Related Work

Multi-modal motion prediction. In the motion prediction literature, there are
agent-centric prediction algorithms [12, 16], as well as scene-centric and
joint multi-agent [7, 13, 19, 10] prediction methods. Most recent prediction
methods adopt encoder-decoder Transformer networks [12, 16, 1]. There is also
a trend of employing diffusion in the literature for predicting trajectories
through the denoising process [14, 9].

Most motion prediction methods are open-loop, in the sense that the whole
trajectory is produced in one-shot independently. However, in this challenge,
traffic agent simulation needs to be simulated in a closed-loop autoregressive
manner at 0.1s intervals. Additionally, the method must differentiate the
world agents and the ADV and factorize their joint distribution for
conditional independence.

Traffic agent simulation. TrafficSim [17] utilizes real-world logged data to
mimic a broad range of human driving behaviors, and leverages an implicit
latent variable model [3] to generate socially-consistent plans for all
traffic actors jointly. TrafficGen [5] places agents in the scene based on the
learned distribution and simulates their future states. CTG [20] developed a
conditional diffusion model that allows user to control over trajectory
properties while maintaining realism.

The WOSAC requires the simulator to be agnostic to the choice of ADV policy,
therefore it can be swapped with arbitrary ADV policy or planner. Both ADV and
environment agent models need to obtain multiple modes in order to perform
well [11].

##  3 MultiVerse Transformer Agent Simulator (MVTA)

Problem formulation. Given the scene context, including map and past positions
of the agents (_i.e_., world agents and ADV), the goal is to simulate new
states of the agents at 0.1s intervals for the upcoming $T=80$ timesteps
(_i.e_., a 8s episode). There are two constraints: 1) the simulator must be
closed-loop and run in autoregressive manner; 2) the joint distribution
involving the world agents and ADV must be factorized into two conditionally
independent components, to ensure that the ADV component can be replaced with
any arbitrary policy or planner.

Figure 3: Illustrating the receding horizon. Even though predictions are made
for the next 1s, only the waypoint of the initial 0.1s is utilized, with the
remaining prediction being discarded.

###  3.1 Network Architecture

Main architecture. The main architecture of MVTA is illustrated in Figure 2.
Scene context features are obtained by processing the world agents, ADV, and
map data through polyline and transformer-based encoding steps. The
transformer decoder layer takes the scene context features and queries as
input and unrolls the agent states for the next timestep. This architecture is
implemented to fulfill the requirement of executing closed-loop simulations at
0.1s intervals. Current state of the ADV is also used as input to the decoder
layer so the environment agents can react to it. Query content feature output
by each decoder layer is used as the input for the subsequent decoder layer.
The motion prediction head, based on Gaussian Mixture Model (GMM), outputs
multi-modal trajectory predictions. To sample the state from the multi-modal
prediction, we either pick the maximum-likelihood trajectory or randomly
sample from the top-k trajectories with the highest likelihood. In our
implementation, we leverage the same architecture for the ADV policy, but it
can be swapped with any policy or planner.

Transformer-based scene encoder. Given the agent-centric scene inputs (_i.e_.,
agents, ADV, and road graph), we utilize their vector representation [6], and
adopt polyline encoders consisting of a multi-layer perceptron network (MLP)
followed by maxpooling [16]. As in [16], the agent input is represented as the
agent history motion state (_i.e_., agent position, size, heading and
velocity) with a one-hot category mask, while the map input consists of the
position and direction of each polyline point and the polyline type. The
polyline encoders produce agent features $A_{past}\in\mathbb{R}^{N_{a}\times
D}$ and map features $M_{past}\in\mathbb{R}^{N_{m}\times D}$, where $N_{a}$
and $N_{m}$ are the number of agents and map polylines, respectively, and $D$
is the feature dimension. A scene context transformer encoder leverages local
self-attention [16] to produce enhanced scene context features that serve as
inputs of the subsequent decoder network.

Autoregressive transformer decoder. The autoregressive decoding consists of a
group of transformer decoder layers. Each decoder layer has a self-attention
component, and a cross-attention component that attends to the scene context
features, and a GMM prediction module that produces multi-modal predictions.
Each Gaussian component is represented as
$(\mu_{x},\mu_{y},\sigma_{x},\sigma_{y},\rho)$ and predicted with a
probability $p$. The motion prediction head also predicts the velocity
$(v_{x},v_{y})$ and heading angle $(sin(\theta),cos(\theta))$ of each agent
for the next timestep. We adopt the motion query pair design in [16]. There
are a total of 64 queries, corresponding to the 64 motion modes, each
associated with an intention point.

Receding horizon. The next 0.1s state is simulated by sampling the multi-modal
predictions output by the decoder layer. However, each decoder layer is
trained to predict a 1s trajectory, and we adopt a receding horizon solution
in which only the initial 0.1s is utilized, as illustrated in Figure 3. The
benefits of longer prediction horizon include the promotion of multi-modal
diversity, reduction of compounding error and also more flexibility in
inference setup.

Figure 4: Training loss is calculated at each timestep. Each decoder layer
produces multi-modal predictions for one timestep.

Scene context update. The states output by the decoder layer at each timestep
are used to update the scene context features. However, the design of our
decoder allows running the scene encoding at specified periodic intervals,
rather than at every simulation step. Alternatively, to keep the scene context
features updated when predicting the next timestep, we implement two
modifications to the network. Firstly, current position of the ADV is used as
input to the decoder layer. Similar to the static intention query in [16],
sinusoidal position encoding and MLP are applied, and the resulting position
embedding is added to the query content. Secondly, we update the agent
features with additional features encoding the current positions of the other
agents $A=MLP([A_{past},A_{current}])$.

###  3.2 Training

Our simulation model is trained end-to-end in a closed-loop manner similar to
that used in [17]. As shown in Figure 4, supervision is provided at each
decoder layer, and losses are computed for each timestep.

Training samples. The training samples are generated to accommodate variable
lengths of past history, as opposed to adhering to a fixed length of 1.1s.
Specifically, for each 9.1s training trajectory, we randomly pick a point to
separate the trajectory to history and future components. This way, more
training samples can be generated from each ground-truth trajectory. Moreover,
this facilities the trajectory history aggregating mechanism in our inference
step.

Training losses. We use L1 loss for regressing the agent velocity and heading
angles, and the Gaussian regression loss implemented based on the negative
log-likelihood loss to maximize the likelihood of ground-truth trajectory
[16].

At each timestep the loss can be formulated as:

$\mathcal{L}_{NLL}=-{\rm
log}\mathcal{N}(S_{x}-\mu_{x},\sigma_{x};S_{y}-\mu_{y},\sigma_{y};\rho)$ (1)

where $S_{x},S_{y}$ is the waypoint of the ground-truth trajectory at this
timestep, and $(\mu_{x},\mu_{y},\sigma_{x},\sigma_{y},\rho)$ represent the
parameters of the selected positive Gaussian component.

We calculate the final loss based on the weighted sum of the
$\mathcal{L}_{NLL}$ loss and the L1 losses of velocity and heading angles.

$\mathcal{L}_{total}=\sum_{t}\lambda_{1}\mathcal{L}_{NLL}^{t}+\lambda_{2}\mathcal{L}_{Vel}^{t}+\lambda_{3}\mathcal{L}_{\theta}^{t}$
(2)

During the challenge, we also made an attempt to implement a simplified
version of the collision avoidance loss [17], however the preliminary result
did not indicate any improvement in terms of realism metric, and we leave it
to future studies.

###  3.3 Inference

Top-k sampling. During model inference, each simulation step produces multi-
modal trajectories. There are two alternatives available for trajectory
sampling. The first approach is to select the maximum-likelihood trajectory,
while the second is to randomly choose among the top-k (_e.g_., k=3)
trajectories (Figure 5a) with the highest likelihood. The first one, while
producing accurate trajectory, tends to yield less varied trajectories. On the
other hand, opting for the top-k trajectories encourages diversity but is
susceptible to the compounding error and could generate trajectories with
unrealistic kinematic motions or even drift, As a result, we employ the top-k
sampling at periodic intervals during the simulation steps to strike a balance
between realism and diversity.

Variable-length history aggregation. Instead of using fixed 1.1s history, we
continuously aggregate the past history as the agent state unrolls overtime
(Figure 5b), and use the aggregated history trajectory for the scene context
encoding in the next simulation step. The motivation is two-fold, firstly, our
training process also uses variable-length history. Secondly, we aim to
enhance the stability of the trajectory simulation, thereby reducing the
potential for compounding error. One example showcasing the autoregressive
rollout is shown in Figure 6.

(a)

(b)

Figure 5: (a) Top-k sampling. (b) Aggregating new waypoint to the past
trajectory. Figure 6: An example of the autoregressive rollout. The context
history, current state (blue-edged circle) and next simulated waypoint (green-
edged circle) are visualized for each agent.

##  4 Experimental Evaluation

WAYMO |  [gray]0.95META METRIC | KINEMATIC | INTERACTIVE | MAP |
---|---|---|---|---|---
LEADERBOARD |  [gray]0.95REALISM | LINEAR | LINEAR | ANG. | ANG. | DIST. | COLLISION | TTC | DIST. | OFFROAD | minADE
|  [gray]0.95 | SPEED | ACCEL. | SPEED | ACCEL. | TO OBJ. |  |  | TO ROAD |  | ($\downarrow$)
MVTE (ours) |  [gray]0.950.5168 | 0.4426 | 0.2218 | 0.5353 | 0.481 | 0.382 | 0.4509 | 0.832 | 0.6641 | 0.6409 | 1.677
MVTA (ours) |  [gray]0.950.5091 | 0.4365 | 0.22 | 0.533 | 0.4805 | 0.3729 | 0.4359 | 0.8298 | 0.6545 | 0.6288 | 1.8698
MTR+++ |  [gray]0.950.4697 | 0.4119 | 0.1066 | 0.4838 | 0.4365 | 0.3457 | 0.4144 | 0.7969 | 0.6545 | 0.577 | 1.6817
CAD |  [gray]0.950.4321 | 0.3464 | 0.2526 | 0.4327 | 0.311 | 0.33 | 0.3114 | 0.7893 | 0.6376 | 0.5397 | 2.3146
multipath |  [gray]0.950.424 | 0.4318 | 0.2304 | 0.0193 | 0.0355 | 0.3493 | 0.4854 | 0.8111 | 0.6372 | 0.613 | 2.0517
sim_agents_tutorial |  [gray]0.950.3941 | 0.3143 | 0.1738 | 0.4785 | 0.4631 | 0.2641 | 0.2671 | 0.7709 | 0.5575 | 0.4111 | 3.6198
QCNeXt |  [gray]0.950.392 | 0.4773 | 0.2424 | 0.3252 | 0.1987 | 0.3759 | 0.3244 | 0.7569 | 0.6099 | 0.36 | 1.083
sim_agents_tutorial |  [gray]0.950.3201 | 0.3826 | 0.0999 | 0.0318 | 0.0391 | 0.2909 | 0.336 | 0.7549 | 0.5251 | 0.3804 | 3.108
linear_extrapolation_baseline_tutorial |  [gray]0.950.2576 | 0.0745 | 0.1659 | 0.0187 | 0.0348 | 0.2221 | 0.2211 | 0.7551 | 0.479 | 0.3352 | 7.5148
|  |  |  |  |  |  |  |  |  |  |

Table 1: WOSAC Leaderboard. Realism meta-metric is the primary metric for
ranking the methods. Our simulator reached the highest meta-metric of 0.5168
among all the methods on the leaderboard. (a)
Figure 7: Three simulations of a scene, in which a vehicle waiting to get onto
the main road. The vehicle turns left or right, or keeps waiting for the right
time to go. Additionally, the ADV also demonstrates multi-modal behavior,
either proceeding straight or making a left turn. Five timesteps are rendered
for each simulation.

###  4.1 Dataset and Metrics

We use the Waymo Open Motion Dataset (WOMD) [4] v1.2.0 release in our
experiments. There are a total of 486,995, 44,097, and 44,920 scenarios in the
training, validation, and test set, respectively. Each scenario in the
training and validation sets comprises of 11 observations for history and 80
observations from 8 seconds of future data, therefore the total duration of
each scenario is 9.1 seconds.

The task is to simulate up to 128 agents including the ADV, and generate 80
simulation steps (8s) for each agent in a 0.1s sampling interval, and in an
autoregressive and reactive manner [11]. 32 simulations are required for each
agent to be simulated.

There are three object types (vehicles, cyclists, and pedestrians), and their
$x/y/z$ centroid coordinates and heading need to be simulated. There is no
need to simulate the size of each agent since it stays constant. In our
experiments, we keep the $z$ value the same as the starting state. The
challenge does not enforce any motion model, and therefore there are no
kinematic constraints.

The main evaluation metric is the realism meta-metric, aggregating a group of
component metrics including kinematic, interactive and map-based metrics. For
more details, please refer to [11].

###  4.2 Implementation and Simulation Setup

Table 2 summarizes the hyperparameters of different modules used in our
implementation.

Training details. The simulation model is trained end-to-end for all three
agent types, using AdamW optimizer for 30 epochs. The learning rate is set to
0.0001. We set the loss weights $\lambda_{1},\lambda_{2},\lambda_{3}$ in
Equation (2) to 1.0, 0.5, 0.5, respectively. Similar to [16], we use 64 motion
query pairs based on 64 intention points learned by running $k$-means
clustering algorithm on the future 1s waypoints of the training trajectories.
A set of 64 intention points is obtained for each object category.

Batch inference and optimization. There are a total of 44,920 scenes in the
test set, and each scene requires running the model inference for $32\times T$
times to generate the 32 simulations for a group of agents. As such, we
implemented batch inference to speed up the simulation process. Given that our
model design supports periodic updates of the scene context features, the
inference speed can be further optimized by running the scene context encoder
every few timesteps (_e.g_., 0.5s) and running several decoder layers (_e.g_.,
the first 5).

gray!20 Module | Hyperparameters | Values
---|---|---
Scene MLP | No. Channels-Agent | 256
| No. Layers-Agent | 3
| No. Channels-Map | 64
| No. Layers-Map | 5
Encoder | Hidden Feature Dim. | 256/384
| No. Encoder Layers | 6
| No. Attention Head | 8
Decoder | Hidden Feature Dim.-Agent | 512/768
| Hidden Feature Dim.-Map | 256/384
| No. Decoder Layers | 10
| No. Attention Head | 8
| No. Motion Modes | 64
Training | Learning rate | 0.0001
| No. Epochs | 30
| Loss weights | 1.0, 0.5, 0.5
|  |

Table 2: Hyperparameters of different modules in MVTA.

MVTE. We explore the design space of the MVTA model and trained 3 variants of
the model by increasing the number of hidden feature dimension in the encoder
(_e.g_., 384 as opposed to 256) and decoder (_e.g_., 768 as opposed to 512).
In the enhanced MVTE solution, model is randomly sampled to generate each
simulation, encouraging more diversity in the resulting simulations.

###  4.3 Experimental Results

WOSAC 2023 leaderboard. On the WOSAC
leaderboard111https://waymo.com/open/challenges/2023/sim-agents/, the realism
meta-metric is the official primary metric used for ranking the methods. The
minADE metric is also calculated but it is primarily used for evaluating
motion prediction methods. The official baseline extrapolates the trajectory
of an agent using the last heading and speed logged in the provided history
[11]. For more baselines based on Wayformer [12] on the validation set, please
refer to [11].

The leaderboard is shown in Table 1. On the test set, our MVTA reaches a
realism meta-metric of 0.5091 and our MVTE further improves the meta-metric to
0.5168, ranking the 1st place in the challenge. Notably, it also has the
highest scores in component metrics except the linear and collision metrics.

Note in Table 1 that minADE does not always correlate with the ranking, as the
method achieving the lowest minADE has lower realism meta metric compared to
other methods.

(a)  Figure 8: Five simulated scenarios featuring reactive environment agents,
each with five timesteps rendered. Scenario 1 depicts a congested right lane
where an agent, indicated by the arrow, attempts to overtake the ADV. Scenario
2 and 3 showcase vehicles at intersections, waiting for the oncoming traffic
to clear before making an unprotected left turn. Scenario 4 shows the ADV
slows down to yield to a car merging onto the main road from a driveway. The
car behind the ADV then changes to the left lane and overtakes the autonomous
vehicle. Lastly, scenario 5 demonstrates the ADV executing a slow right-turn,
resulting in the agent behind it having to slow down or stop.

Qualitative results. In Figure 7, we present a scenario demonstrating the
multi-modal behavior of an agent. Figure 8 features five simulated scenarios
showcasing reactive environment agents. These agents exhibit a wide variety of
behaviors including yielding, overtaking, pausing for unprotected left turns,
and engaging with the ADV. Qualitative simulation results of several
intersection scenes with agents undertaking a wide variety of maneuvers are
provided in Figure 9. Due to the complexity of these scenes, it is impossible
for heuristic-based models that encode traffic rules to simulate these
realistic agents.

(a)
Figure 9: Qualitative simulation results of several complex intersection
scenarios. Each row shows 4 out of the 32 simulations for a scenario.

##  5 Conclusion

In this technical report, we have presented the Multiverse Transformer (MVTA)
framework which produces parallel universes for the application of traffic
agents simulation. It achieved state-of-the-art performance and ranks the 1st
place in the Waymo Open Sim Agents Challenge 2023. We hope our work inspires
further research in the area of simulation agents. In our upcoming research,
we intend to investigate scene-centric simulation approaches for improving the
degree of realism of simulations, and also explore the possibility of using
diffusion/denoising-based approaches.

## References

  * [1] Lina Achaji, Thierno Barry, Thibault Fouqueray, Julien Moreau, Francois Aioun, and Francois Charpillet.  Pretr: Spatio-temporal non-autoregressive trajectory prediction transformer.  arXiv:2203.09293, 2022.
  * [2] Luca Bergamini, Yawei Ye, Oliver Scheel, Long Chen, Chih Hu, Luca Del Pero, Blazej Osinski, Hugo Grimmett, and Peter Ondruska.  Simnet: Learning reactive self-driving simulations from real-world observations.  In ICRA, 2021.
  * [3] Sergio Casas, Cole Gulino, Simon Suo, Katie Luo, Renjie Liao, and Raquel Urtasun.  Implicit latent variable model for scene-consistent motion forecasting.  In ECCV, 2020.
  * [4] Scott Ettinger, Shuyang Cheng, Benjamin Caine, Chenxi Liu, Hang Zhao, Sabeek Pradhan, Yuning Chai, Ben Sapp, Charles Qi, Yin Zhou, Zoey Yang, Pei Sun, and Dragomir Anguelov et al.  Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset.  In ICCV, 2021.
  * [5] Lan Feng, Quanyi Li, Zhenghao Peng, Shuhan Tan, and Bolei Zhou.  Trafficgen: Learning to generate diverse and realistic traffic scenarios.  In ICRA, 2023.
  * [6] Jiyang Gao, Chen Sun, Hang Zhao, Yi Shen, Dragomir Anguelov, Congcong Li, and Cordelia Schmid.  Vectornet: Encoding hd maps and agent dynamics from vectorized representation, 2020.
  * [7] Roger Girgis, Florian Golemo, Felipe Codevilla, Martin Weiss, Jim Aldon D’Souza, Samira Ebrahimi Kahou, Felix Heide, and Christopher Pal.  Latent variable sequential set transformers for joint multi-agent motion prediction.  In ICLR, 2022.
  * [8] Maximilian Igl, Daewoo Kim, Alex Kuefler, Paul Mougin, Punit Shah, Kyriacos Shiarlis, Dragomir Anguelov, Mark Palatucci, Brandyn White, and Shimon Whiteson.  Symphony: Learning realistic and diverse agents for autonomous driving simulation.  In ICRA, 2022.
  * [9] Chiyu Max Jiang, Andre Cornman, Cheolho Park, Ben Sapp, Yin Zhou, and Dragomir Anguelov.  Motiondiffuser: Controllable multi-agent motion prediction using diffusion.  In CVPR, 2023.
  * [10] Wenjie Luo, Cheolho Park, Andre Cornman, Benjamin Sapp, and Dragomir Anguelov.  Jfp: Joint future prediction with interactive multi-agent modeling for autonomous driving.  In PMLR, 2022.
  * [11] Nico Montali, John Lambert, Paul Mougin, Alex Kuefler, Nick Rhinehart, Michelle Li, Cole Gulino, Tristan Emrich, Zoey Yang, Shimon Whiteson, Brandyn White, and Dragomir Anguelov.  The waymo open sim agents challenge.  arXiv:2305.12032, 2023.
  * [12] Nigamaa Nayakanti, Rami Al-Rfou, Aurick Zhou, Kratarth Goel, Khaled S. Refaat, and Benjamin Sapp.  Wayformer: Motion forecasting via simple and efficient attention networks.  arXiv:2207.05844, 2022.
  * [13] Jiquan Ngiam, Benjamin Caine, Vijay Vasudevan, Zhengdong Zhang, Hao-Tien Lewis Chiang, Jeffrey Ling, Rebecca Roelofs, Alex Bewley, and et al. Chenxi Liu.  Scene transformer: A unified architecture for predicting multiple agent trajectories.  In ICLR, 2022.
  * [14] Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, and Or Litany.  Trace and pace: Controllable pedestrian animation via guided trajectory diffusion.  In CVPR, 2023.
  * [15] Davis Rempe, Jonah Philion, Leonidas J. Guibas, Sanja Fidler, and Or Litany.  Generating useful accident-prone driving scenarios via a learned traffic prior.  In CVPR, 2022.
  * [16] Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele.  Motion transformer with global intention localization and local movement refinement.  In NeurIPS, 2022.
  * [17] Simon Suo, Sebastian Regalado, Sergio Casas, and Raquel Urtasun.  Trafficsim: Learning to simulate realistic multi-agent behaviors.  In CVPR, 2021.
  * [18] Danfei Xu, Yuxiao Chen, Boris Ivanovic, and Marco Pavone.  Bits: Bi-level imitation for traffic simulation.  arXiv:2208.12403, 2022.
  * [19] Ye Yuan, Xinshuo Weng, Yanglan Ou, and Kris Kitani.  Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting.  In ICCV, 2021.
  * [20] Ziyuan Zhong, Davis Rempe, Danfei Xu, Yuxiao Chen, Sushant Veer, Tong Che, Baishakhi Ray, and Marco Pavone.  Guided conditional diffusion for controllable traffic simulation.  In ICRA, 2023.