# Towards Robots That Know When They Need Help:
Affordance-Based Uncertainty for Large Language Model Planners

James F. Mullen Jr1, and Dinesh Manocha1
Supplemental version including Code, Video, and Appendix is available at
https://gamma.umd.edu/LAP/ 1The authors are associated with the University of
Maryland, College Park, USA<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Large language models (LLMs) showcase many desirable traits for intelligent
and helpful robots. However, they are also known to hallucinate predictions.
This issue is exacerbated in consumer robotics where LLM hallucinations may
result in robots confidently executing plans that are contrary to user goals,
relying more frequently on human assistance, or preventing the robot from
asking for help at all. In this work, we present LAP, a novel approach for
utilizing off-the-shelf LLM’s, alongside scene and object Affordances, in
robotic Planners that minimize harmful hallucinations and know when to ask for
help. Our key finding is that calculating and leveraging a scene affordance
score, a measure of whether a given action is possible in the provided scene,
helps to mitigate hallucinations in LLM predictions and better align the LLM’s
confidence measure with the probability of success. We specifically propose
and test three different affordance scores, which can be used independently or
in tandem to improve performance across different use cases. The most
successful of these individual scores involves prompting an LLM to determine
if a given action is possible and safe in the given scene and uses the LLM’s
response to compute the score. Through experiments in both simulation and the
real world, on tasks with a variety of ambiguities, we show that LAP
significantly increases success rate and decreases the amount of human
intervention required relative to prior art. For example, in our real-world
testing paradigm, LAP decreases the human help rate of previous methods by
over 33% at a success rate of 70%.

##  I Introduction

Figure 1: LLMs are not grounded in the real world and will frequently
hallucinate. Additionally, when provided with an ambiguous instruction, the
LLM must know when to ask users for help. We present LAP, a novel approach for
calibrating LLM confidence with scene affordances to better detect
hallucinations and resolve ambiguities, before asking a user for help when
necessary. We integrate affordances by calculating a score for each possible
action and combining it with the next token probability of that action to
determine both which option is most likely and if help is needed. One method
of calculating the affordance score uses LLM prompts to score each possible
action on safety and grounding in the scene. Note, in the provided example,
‘pick up coffee cup’ is a hallucination as no such specific object exists. In
LAP, our affordance scores, seen in orange, mitigate this case.

Imagine you have a home robot and you want it to bring you your coffee cup. As
you tell the robot your instruction, it must comprehend your goal, no matter
how you phrase the instruction, and ideally complete the task without further
clarification. Now imagine there are multiple coffee cups on the counter, the
robot may need to ask you for help determining which one is yours. Finally,
imagine that there is one coffee cup, but many other objects on the counter
that may distract the robot. You would expect the robot to get the coffee cup,
without asking for your assistance. If the robot asked you for help, brought
you the wrong item, or failed due to some hallucination, you would be inclined
to question its ability and potentially be less likely to trust it in the
future. Overcoming these problems is core to a successful home robot. There
will always be uncertainty in the unstructured and novel environments these
types of robots operate in, but they must operate reliably and intelligently
nonetheless.

Recent approaches that leverage large language models (LLMs) for planning [1,
2, 3] have demonstrated an ability to navigate these types of complex
environments with a higher success rate than prior methods, while also
responding properly to natural and unstructured language instructions.
Furthermore, each new generation of the language model, such as GPT-3 to
GPT-4, greatly improves model capabilities and intelligence, and subsequently
performance on these robotics tasks. However, a significant challenge with all
LLMs, new or old, is their tendency to present an incorrect answer
confidently, or hallucinate [4, 5].

Additional challenges arise when the LLM is embodied and must interact with
users directly. Significant uncertainty is present as human-provided
instructions can be ambiguous, in some cases causing further hallucinations.
Completing the wrong action or hallucinating and failing could hurt user trust
or even be potentially dangerous. One method of mitigating these issues is to
ask users for help or clarifications when needed [3, 6, 7, 8]. However, most
prior work in language-based planning often does not ask for clarification
from users in uncertain or ambiguous situations, or does so via extensive
manual programming, oftentimes excessively relying on seeking assistance [6].
More recently, KnowNo [3] frames the task of when a robot should ask for help
as uncertainty alignment, and outlines the main challenges for robots that ask
for help: (i) calibrated confidence – the robot should seek sufficient help to
ensure a probability of task success, and (ii) minimal help – the robot should
minimize the amount of help it seeks by narrowing down valid possibilities on
its own. However, KnowNo primarily focuses on creating a means of evaluation
for the uncertainty alignment task and provides a basic baseline. In contrast,
we aim to improve the confidence measure, increase the success rate, and
minimize human intervention, moving towards more helpful and less annoying
robot partners.

Main Contributions: Our key finding is that scene affordances, which can
intuitively be thought of as the potential for a scene to allow a given
action, can minimize hallucinations and ground possible actions in the scene
and safety standards. By handling these issues, the affordances can be used to
increase the task success rate and minimize the frequency of human
intervention. An additional advantage of our affordance methods is that they
do not require extensive training, which is difficult to do without
overfitting for extremely diverse and data-limited tasks like those found in
home robotics.

The main contributions of our work include:

  1. 1.

We introduce LAP, a novel approach for uncertainty alignment that uses Large
Language Models and affordances to better align model confidence with task
success for high-level robotic planners.

  2. 2.

We evaluate our approach in both simulation and on hardware in the real world
using a suite of language-instructed manipulation tasks from the KnowNo
Simulation and Mobile Manipulator Datasets [3]. These datasets contain many
different potential ambiguities that the agent must handle. We show that LAP
significantly increases the success rate and reduces the amount of help needed
as compared to our baselines across different environments and LLMs.
Specifically in our real-world testing, we see a decrease in the human help
rate of previous methods by over 33% at a success rate of 70%, with a similar
decrease at most success rates.

  3. 3.

We explore and evaluate different ways of extracting an affordance value for
use in LAP, including a novel prompt-based method. Our prompt-based method
allows added flexibility in the affordance score to encapsulate safety and
other important behavior.

  4. 4.

We show that using LAP on the newest, most powerful language models out-of-
the-box outperforms fine-tuning LAP and prior art [3] on the best models
available for fine-tuning on this task.

Figure 2: We present LAP, a novel approach for uncertainty alignment that
leverages affordances found in the scene. After generating a set of multiple-
choice candidate actions, each prepended with ‘A’, ‘B’, ‘C’, etc., we
calculate an affordance score for each option (shown in orange). One version
of our method uses the scene context from earlier prompts (shown in blue),
while another sends a new query to the LLM to score a given option. This
prompt-based method allows us to include factors like safety into each
option’s affordance score, and both options allow us to mitigate
hallucinations. After computing each option’s affordance score, we multiply
the next-token likelihood and construct a prediction set of options with a
total value above a threshold, $t$. Note that we eliminate option A, ”Put
metal bowl in the microwave,” as a valid option as it is dangerous. If the
size of the prediction set were greater than one, the agent would ask for
help.

##  II Related Work

###  II-A Hallucinations and Uncertainty in LLMs

As LLMs have risen to the fore, an increasing amount of work has been aimed at
addressing their hallucinations, times when the LLM generates content not
grounded in reality. Some works aim to quantify the frequency or severity of
hallucinations [4] in a given LLM, while many aim to decrease their frequency
[5], or better calibrate uncertainty [9, 10, 11]. Some recent works attempt to
handle hallucinations by using conformal prediction-based methods to provide
coverage guarantees[12]. Ren et al. [3] introduce KnowNo, which uses conformal
prediction to create coverage guarantees for robotics tasks, asking users for
help when the agent is unsure. In contrast to these works, we mitigate
hallucinations through the introduction of an affordance score, minimizing the
amount of help needed.

###  II-B Scene Affordances and Robotics

Affordances, or what is possible and plausible in an environment, is a well-
studied topic in computer vision and graphics [13, 14, 15]. Most of these
works focus on placing virtual humans or animations into 3D scenes such that
the resultant placement generates visually plausible results [14, 16, 17].
Recent works in robotics use similar techniques to determine what actions are
possible and likely for a robot agent given the environmental context,
typically for manipulators[18]. SayCan [1] applied these techniques to robot
planners by adding an affordance value, obtained from trained value functions
and indicating the value of an action in the environment, into the planning
scheme. We take inspiration from this method and explore ways of extracting an
affordance value without the need for training or LLM fine-tuning. We choose
to do this as training capable value functions for every possible action in
the real world (as SayCan does) is extremely difficult due to a lack of domain
data and vast task diversity. We also show that using affordances can help
mitigate hallucinations in an LLM’s generations by grounding the generations
in the real world.

###  II-C LLMs for Robot Planning and Human-Robot Interaction

Robot motion planning is a well-studied problem in robotics [19, 20, 21].
Recently, Large Language Models (LLMs) have shown an ever-increasing set of
capabilities from reasoning and logic [22, 23, 24] to math and physics [25].
This has extended to robotics, with LLMs improving the state of the art in
tasks from high-level planning [1, 26, 27, 28], to object goal navigation[29,
2].

Natural language is one of the most popular methods of producing effective
human-robot interaction [30, 7, 31, 8] and the advent of LLMs has made
dialogue an even more natural medium of communication between a user and their
robotic agent. Recent LLM-based robotic works have extended their tasks,
incorporating user interaction [6, 32, 3] to receive user instructions or to
resolve uncertainty.

A key issue in LLM-based robotics is coming up with a set of candidate actions
that respond to the user’s instruction. Prompting LLMs directly for each valid
step in a robot plan has proven to perform poorly on uncertain tasks [33].
This becomes a greater issue when we would like to characterize or measure the
model’s uncertainty. Furthermore, prior art has shown that the LLM itself is a
poor scoring function [33], especially when the plan is expressed in natural
language. As such, we choose to build off of prior work, specifically KnowNo,
and formulate LLM planning as multiple-choice Q&A (MCQA).

##  III Decreasing Uncertainty with Affordances

In this work, we consider the following problem statement: Given a robotic
agent and a user-provided instruction, execute the instruction completely and
accurately while resolving any ambiguities as needed through dialogue with the
user. Minimize the amount of dialogue. Specifically, our robot will begin in a
set environment, awaiting a user’s instruction. The user will provide an
instruction, which may be under-specified, at which point our method will need
to evaluate whether or not it requires additional information from the user.
This problem space was initially explored by [3]. Our key finding is that
leveraging scene affordances can greatly decrease the frequency of help needed
by the robot, as it can resolve many ambiguities on its own, and handle LLM
hallucinations. For example, if the user asks the robot to put a bowl in the
microwave, and there is a plastic and metal bowl on the counter, prior art
[3], would need to ask the user for clarification as to which bowl they were
referring to. However, by leveraging affordances, the robot should know to use
the plastic bowl as putting the metal bowl in the microwave would be
dangerous.

To transform the problem into an MCQA task, as in KnowNo [3], the LLM is
provided a few-shot prompt that includes possible next steps in a few example
scenarios. The LLM in turn generates a set, $\\{y^{i}\\}$, of candidate
actions that are semantically different for the current scenario. Then, the
task of choosing among the candidate actions is formatted as a multiple choice
question to the LLM, with each option preceded with ‘A’, ‘B’, ‘C’, ‘D’, or ‘E’
as shown in Figure 2. This allows us to define the problem as next-token
prediction – aligning well with LLM loss functions and training data. The
probabilities of each possible token (i.e. the multiple choice options ‘A’,
‘B’, ‘C’, …) can then serve as normalized scores that can be utilized by
uncertainty quantification methods, like LAP. In this work, we utilize these
normalized scores alongside an affordance score to generate a prediction set
of plans from $\\{y^{i}\\}$. Please see [3] for further rationale on why MCQA
should be used for uncertain robot planning.

Method Overview. We present an overview of our method, LAP, in Figure 2. LAP
begins with the MCQA paradigm described above, outputting the model’s
uncalibrated confidence score for each generated action option,
$p^{*}(y^{i})$. We then compute an affordance score, $a^{i}$, for each option,
corresponding to the plausibility of that action given the scene. We describe
three different methods of obtaining this score below, context-based,
perception-based, and prompt-based. Each of these scores can be used
independently or combined into a superscore as shown in Figure 2. We choose to
present all three as each could be advantageous in different use cases and
their combination can create a more performative final score. We now define
our final probability of each option being the user’s intent as:

$p(y^{i})=p^{*}(y^{i})\cdot a^{i}.$ (1)

With each option scored, we generate the prediction set of plans,
$\mathcal{P}$ as plans with a score above a threshold, $t$

$y^{i}\in\mathcal{P}\text{ if }p(y^{i})>t.$ (2)

The LLM is certain if this set is a singleton, and queries the user for help
if not. Calculating an optimal $t$ is the main subject of KnowNo [3], which
uses conformal prediction to provide statistical guarantees of a certain
success rate. Our work differs significantly from KnowNo as we improve
$p(y^{i})$ by integrating affordances, with their $p(y^{i})$ including just
the next-token likelihood from the LLM ($p^{*}(y^{i})$ in Figure 2). Our
experiments are instead agnostic to the threshold value used, primarily
focusing on improving the relationship between success rate and help rate
across all arbitrary thresholds and subsequent success rates.

Context-Based Affordance Score. The first method of calculating the affordance
score that we explore is the most efficient, requiring the least amount of
time, compute, and control over the robot agent. This method uses the context
of the scene that the robot already found and provided to the LLM from its
perception algorithms. We label this affordance value $a^{i}_{c}$.

To create the prompts for both MCQA and the scoring of the MCQA options,
context about the scene had to be found. For example, in the simulation
environment, there are three blocks and three bowls, one of each is blue,
green, and yellow, respectively. The robot’s perception algorithms found these
objects and conveyed their existence to the agent as part of its initial
prompts. However, this does not always prevent the LLM from hallucinating new
objects into existence, like a ‘gold bowl.’ These types of hallucinations
become even more prominent when the hallucinated object is in the user
command. An example of this can be seen in Figure 1 with the ‘coffee mug’
object.

To calculate $a^{i}_{c}$, first, the scene context is parsed into a set of
found objects, $\mathcal{O}$. Each generated action candidate from the LLM’s
MCQA creation response, $y^{i}$, is parsed for any objects, $o^{i}$, present.
If all of the objects in $y^{i}$ are in the set of found objects,
$a^{i}_{c}=1$. Otherwise, $a^{i}_{c}=0$. To continue our example from the
previous paragraph, if $y^{i}$ included a ‘blue bowl’ and a ‘yellow block,’
then $a^{i}_{c}=1$. If $y^{i}$ included a ‘gold bowl,’ then $a^{i}_{c}=0$. To
formalize this:

$a^{i}_{c}=\begin{cases}1&\text{if }\forall o^{i}\in
y^{i},\,o^{i}\in\mathcal{O}\\\ 0&\text{otherwise}.\end{cases}$ (3)

Perception-Based Affordance Score. Building off of the context-based
affordance score, our next affordance score takes into account any uncertainty
in the perception algorithms themselves. To do this, we run an open-vocabulary
vision-language model, like CLIP [34] or ViLD [35], on the scene observations
with each object, $o^{i}$ (from the option generated by the LLM, $y^{i}$) as
input.

Specifically, we take the camera images of the scene, $s$, and feed them
through ViLD with the list of objects $\\{o^{i}\\}$ as input. We then take the
probabilities of each object $p(o^{i}|s)$ as outputted by the model and find
the average for a given option, $y^{i}$. Formally, we define $a^{i}_{perc}$ as

$a^{i}_{perc}=\frac{1}{n}\sum_{i=1}^{n}p(o^{i}|s),$ (4)

for $n$ objects in $y^{i}$.

We believe that in practice, this score can be applied using information
directly from the initial perception algorithms, with objects not initially
perceived running through the above process.

Prompt-Based Affordance Score. Recent works like [5] and [22] have shown that
additional prompts to the LLM requiring it to reason on its prior outputs can
isolate harmful hallucinations. We leverage this idea and propose a prompt-
based affordance scoring of each action option, $y^{i}$, generated by the
model. This additionally allows us to generate a more flexible affordance
score, which can account for factors in affordance beyond perception. Our
prompt, given our real-world evaluation environment, begins with a series of
few-shot samples, before serving the following text:

> {Header}
> …
>
> We: On the counter, there is a metal bowl, a plastic bowl, and a microwave.
> We: pick up the metal bowl and put it in the microwave
> We: Is this possible and safe given the provided knowledge of the scene?
> You: False
>
> We: On the counter, there is an orange, a bag of rice chips, and an apple.
> We: pick up the orange
> We: Is this possible and safe given the provided knowledge of the scene?
> You: True
>
> We: On the counter, there is {$\mathcal{O}$}.
> We: {$y^{i}$}
> We: Is this possible and safe given the provided knowledge of the scene?
> You:
>

In the prompt, the items inside the brace notation are replaced with their
respective values. The header includes the same description of items found in
the scene from the MCQA generation and scoring prompts and describes the task
for the LLM: determining whether the given action is ‘possible and safe’ in
the current scene. The header also tells the LLM to reply in either ‘True’ or
‘False’ and nothing else. Once we receive the response from the LLM, we
extract the normalized probability of the ‘True’ token and utilize this as our
affordance score, $a^{i}_{pr}$, for the option $y^{i}$.

Figure 3: Comparison of task success rate vs human help rate (left) and
average prediction set size (right) in our Simulation experiments. Note LAP
improving upon KnowNo for almost all success rates. Including a reduction in
the human help rate of over 70% at a success rate of 60% No Help and Binary do
not create prediction sets and are not included in the plot.

One benefit of this approach is the ability to tailor the affordance score to
new criteria. For example, in this prompt, we add safety as a major component
of the affordance, allowing the LLM to lower the affordance of, putting the
metal bowl in the microwave. Similarly, if an important characteristic of the
robot agent is that it does not leave a certain area, one could specify this
in the task description and the affordance score should adapt accordingly. The
perception-based methods above cannot apply logic to the generated options,
$y^{i}$, in this way when calculating their affordance scores.

Fine-Tuning. We also hypothesize that utilizing the newest, most powerful, LLM
available off-the-shelf will generally outperform fine-tuning the older models
available for fine-tuning. We tested this hypothesis by fine-tuning
gpt-3.5-turbo-1106 [36], the most powerful model available from OpenAI for
fine-tuning. For the simulation environment, we fine-tuned on each of the
three LLM phases of LAP, the MCQA generation, MCQA scoring, and Affordance
scoring. In the real-world environment, however, we did not fine-tune the MCQA
generation as we could not create a large enough dataset.

##  IV Experiments and Results

We evaluate LAP in a diverse set of language-instructed tasks and environments
as described below. Each of these environments is outlined in [3]. We chose to
use these environments for easy comparison to [3] and to continue to move
towards a standard set of evaluations for the uncertainty alignment task. We
utilize GPT-4 [37] as the LLM in all experiments and ablate against it
separately.

Scenario Distribution and Data. LAP can be applied to any unknown scenario
distribution. Ideally, if a set of i.i.d. samples can be collected from the
distribution, they can be utilized for calibration alongside the KnowNo [3]
framework to select a threshold, $t$, that provides statistical guarantees of
task success. However, collecting a sufficient amount of calibration data is
difficult, especially in a consumer environment, so we primarily focus on
improving the relationship between success rate and help rate across any
threshold value. We envision that users can provide feedback on the robot’s
task successes and failures to tune the threshold value over time to fit their
expectations. For our testing, we utilized the scenario distribution outlined
in the KnowNo paper. Each environment has a parameterized scenario
distribution with a pre-defined set of possible ambiguities in the human’s
instruction. We utilize the truth labels that they provide.

Baselines. Our primary baseline is KnowNo [3], which uses the probability of
each option directly from the LLM, and builds a prediction set of options with
a probability greater than a threshold value. KnowNo uses conformal prediction
to set the threshold value in such a way that a statistical guarantee of a
certain success rate is possible. They do nothing to improve the confidence
measure itself outside of the task framing as MCQA. All of the additional
baselines use this MCQA framing. Other baselines include:

  * •

Prompt: Prompts the LLM to output the prediction set (e.g. “Prediction set:
[B, D]”)

  * •

Binary: Prompts the LLM to directly output a binary indicator of uncertainty
(e.g. “Certain/Uncertain: Certain”) similar to [6].

  * •

No Help: Always uses the highest score directly from the LLM without creating
a prediction set or asking for human intervention.

Metrics. We primarily use human-help rate and prediction set size versus
success rate to compare LAP to baselines. Intuitively, when given a set
success rate, these metrics provide information about the help rate and
average prediction set size for that success rate, with lower help rates and
lower average prediction set sizes better. For a given data sample, the human-
help rate is 1 if the prediction set size is greater than 1. Prompt and Binary
produce a single success rate and thus a singular help rate and average
prediction set size. With KnowNo and LAP, we have a tunable threshold value
that is used to confine the prediction set. For these methods, we apply
threshold values, $t$, ranging from 0.0000001 to 0.7 for full coverage. The
success rate and human help rate or average prediction set size for each
threshold value are then plotted, and the area under the resultant curve (AuC)
becomes a valuable metric to quantify an improved relationship between task
success rate and human help rate.

Figure 4: Comparison of task success rate vs human help rate (left) and
prediction set size (right) in our real-world experiments on a ClearPath
TurtleBot. Note the improvement over KnowNo across all success rates. LAP with
the prompt and context-based affordance scores outperforms using each score
separately, while all improve upon KnowNo.

###  IV-A Simulation: Tabletop Rearrangement

In this scenario, a robot arm is asked to rearrange objects on a table in the
PyBullet simulator. Each scenario is initialized with three bowls and three
blocks, with one each of blue, green, and yellow, respectively. The task
requires the robot to move a certain number of blocks or bowls toward a
specific location relative to a different object. For example, “Move the Green
Block to the left of the Blue Bowl,” would be a valid task.

As introduced in the KnowNow paper, three different ambiguities are injected
into the user instruction: (1) attribute, where ‘block’ or ‘bowl’ is replaced
with a potentially ambiguous term like ‘thing’ or ‘cube,’ (2) numeric, where
the number of blocks to move is underspecified by using a term like ‘some’ in
place of ‘two,’ and (3) spatial, where the exact location like ‘left’ or ‘
front’ is replaced with an ambiguous term like ‘near.’ The scenario
distribution is uniform over all possible ambiguities and a testing set is
generated for use. The full outline of every possible ambiguity is included in
the supplementary material.

We show in Figure 3 that LAP with the context-based affordance score ($a_{c}$)
greatly outperforms KnowNo, as well as all other baselines, with a higher
success-to-help ratio across threshold values. Quantitatively, LAP+$a_{c}$
reduces the amount of help needed by over 70%, from 30.1% with KnowNo to 8.9%
at a success rate of 60%. Similarly, at a success rate of 80%, we improve upon
the 63.8% help rate of KnowNo with a 41.0% help rate. Similarly, prediction
sets created by LAP are significantly smaller than those from KnowNo. Using
the AuC metric for the Task Success Rate $\times$ Human Help Rate curve, we
find that LAP+$a_{c}$ has an Auc of $0.780$ compared to KnowNo with an AuC of
$0.699$.

Of Prompt and Binary, both produce a singular success rate and help rate.
Binary performed the best for this environment, getting close to the
performance of LAP+$a_{c}$ at its success level of 77%. Prompt performs
relatively poorly, with a success rate below 60%, illustrating the challenges
present when attempting to retrieve the prediction sets directly from the LLM.

LAP with the perception-based affordance score, LAP+$a_{perc}$ also under-
performs LAP with the context-based score. We believe this is caused by MCQA
options, $y^{i}$, becoming more difficult to separate relative to LAP+$a_{c}$.
For example, if $y^{i}$ includes a ‘gold block,’ and none exists, LAP+$a_{c}$
will provide an affordance score of 0. Meanwhile, LAP+$a_{perc}$ will still
provide some value, likely picking up on the yellow block. This relatively
increases the likelihood of this option when it is not possible. The
perception score for the ‘gold block’ should still be lower than that for a
‘yellow block,’ enabling improved performance over KnowNo in most cases. While
we found poorer results in our scenarios with ground true perception data
available, we believe that $a_{perc}$ may still be valuable in deployed
scenarios without this information and choose to report it here accordingly.

We chose not to run LAP with prompting-based affordance, LAP+$a_{pr}$, on the
simulation environment, as it has little to add over the previous methods.
There are no safety concerns or otherwise in this environment, and all actions
with a valid object are possible. This also allowed us to save on API costs.
We view these explorations as a proof-of-concept for affordance-based
uncertainty alignment.

Figure 5: Example of our real-world experimentation setup. The ClearPath
TurtleBot must navigate towards the correct object, at which point a human
moves the object into the basket before the robot continues towards a goal
location. We chose this setup as no viable mobile manipulator was available to
us.

###  IV-B Real World: Mobile Manipulator in a Kitchen

For this environment, we continue to use the task specifications laid out in
KnowNo [3]. Shown in Figure 5, Each scenario involves a mobile robot in front
of a countertop in an office kitchen, next to a set of
recycling/compost/landfill bins. The tasks include picking up objects from the
countertop and possibly putting it into a bin, or somewhere else on the
countertop. New ambiguities are added relative to the simulator environment,
including some involving unsafe actions (e.g. task: “place the bowl in the
microwave.” In the scene there is a plastic bowl and a metal bowl, but only
the plastic one is safe for the microwave). The full outline of every possible
ambiguity is included in the supplementary material. In our lab, no viable
mobile manipulator exists, so we run our testing on a ClearPath TurtleBot.
During the on-hardware testing, we manually pick up the object selected by the
agent and put it in a basket on the robot, simulating the use of the
manipulator. We mark a task as successful when the robot indicates the correct
actions, and navigates to within 1 meter of the target objects or
destinations. In Figure 4 we utilize the planning success rate, defined by
taking the correct actions without accounting for any failures executing the
plan, to focus more specifically on the method’s ability over error injected
while executing them. Qualitatively, the real-world deployment hinders all
methods relatively evenly, with only transient factors affecting one method
more than another.

Similar to the simulation environment, we find that LAP outperforms KnowNo and
our other baselines. We show these results in Figure 4. Our prompt-based score
and context-based score combined into a superscore, LAP+$a_{pr}\times a_{c}$
produced our best results, with a human help rate of only 42% relative to
KnowNo’s 63% at a 70% planning success rate. Similarly, at a planning success
rate of 85%, LAP has a help rate of 72% compared to 87% with KnowNo. These
findings are corroborated with LAP having a lower average prediction set size
than KnowNo, and an improved AuC of $0.72$ relative to KnowNo’s $0.67$.
Qualitatively, we notice that LAP+$a_{pr}\times a_{c}$ improves performance on
scenarios with a safety component, likely due to $a_{pr}$ scoring each option
partially on its safety.

We can identify a similar trend for the Binary and Prompt baselines as in the
simulation environment, with Binary outperforming Prompt, but with neither
getting particularly close to LAP. Prompt performs worse relative to the other
methods in the real-world environment, exhibiting how the added complexity of
the environment further hampers the ability of the LLM to produce an accurate
prediction set directly.

In addition to LAP+$a_{pr}\times a_{c}$, we test LAP+$a_{pr}$ and LAP+$a_{c}$
separately and find that each improves performance over KnowNo individually.
However, combining them allows for the greatest effect. We believe this is due
to each term contributing somewhat different things, with $a_{pr}$ measuring
not just possibility, but also safety, for a given action. Meanwhile, $a_{c}$
does a very good job at removing any hallucinated objects or destinations from
the list of available options entirely. We withheld LAP+$a_{perc}$ from this
exploration due to its underperformance in the simulation environment and as a
time-saving measure. We plan on including those results, as well as further
breakdowns on full-task success in our supplemental materials.

Figure 6: Comparison of LAP+$a_{pr}\times a_{c}$ and KnowNo accross different
LLMs. Note that LAP improves upon KnowNo with every model, and that
performance overall generally increases with each model. GPT-4-Turbo exhibited
poor performance in the MCQA generation phase relative to GPT-4, frequently
asking for clarifications instead of producing a set of candidate options,
capping the possible success rate.

###  IV-C LLM Ablations

To further test LAP, we ablated our model choice, GPT-4 for both our method,
LAP+$a_{pr}\times a_{c}$, and KnowNo in our real-world paradigm. Figure 6
shows these results. We show a large improvement over KnowNo across all
models. The performance gap between LAP and KnowNo is even wider for
GPT-3.5-Turbo and GPT-4-Turbo than it is for GPT-4. With hit maximum
performance on GPT-4-Turbo at around half the human help rate of KnowNo.

Additionally, we show that the performance of LAP improves as the model
improves. This encouraging behavior implies that when subsequent models get
released in the future, LAP will continue to apply and increase their
performance for these tasks.

One finding in this ablation is that GPT-4-Turbo followed instructions less
frequently in the MCQA action generation phase of LAP and KnowNo. On about 30%
of examples, the model would reply with some form of “More information is
needed” instead of producing a set of possible actions. This greatly hindered
overall performance as the remainder of the framework could not be run. Future
work should investigate ways to improve the MCQA action generation phase if
future models continue to exhibit this behavior.

As an additional ablation, we also tested LAP using the given model for the
MCQA generation and scoring phases, but only GPT-3.5-Turbo for finding
$a_{pr}$. Performance was largely similar with an increase in help rate of
under 4% on average. We envision this as a possible cost-saving measure for
users of LAP who want to minimize API or compute costs. Visually, the
difference is similar to that between LAP+$a_{pr}\times a_{c}$ and LAP+$a_{c}$
in Figure 4. These plots can be seen in our supplementary materials.

###  IV-D Why Not Just Fine-Tune?

In simulation, where we can generate a large set of possible tasks, we
attempted to fine-tune gpt-3.5-turbo-1106, the best model available at the
time, on each part of LAP. For training on the first stage, the MCQA action
option proposal, we attempted to separate certain ambiguities of each type, as
well as just one type, for use exclusively in the testing set, which has a
distribution identical to our dataset utilized above.

We find that overall help rates increase for the same model by on average 14%.
Much of this difference is due to severely impacted performance on ambiguities
that were outside of the training data, showing a tendency to overfit seen by
others including [3]. Similar issues occurred when fine-tuning on the option
scoring component of LAP, albeit with less of an impact, decreasing help rates
by less than 5% on average, but requiring help more frequently on the withheld
ambiguities. This method still underperforms LAP on GPT-4 across most success
rates. However, prediction set sizes were decreased more than LAP on average,
showing the model improving its rejection of poor options.

In the real-world environment, with even more limited data, fine-tuning the
MCQA proposal phase of LAP and KnowNo produced an overfitted model that could
not generalize to the withheld testing set of tasks, no matter what
hyperparameters or settings were used while training. When fine-tuning on the
scoring task, we saw a similar phenomenon to in the simulation paradigm with
improved performance on samples inside the training distribution, but poorer
performance on those outside. Overall, help rates decreased on average while
still underperforming LAP on GPT-4 just as in the simulation scheme.

These results show that fine-tuning the LLM is possible, especially for better
aligning the MCQA scoring and next-token probabilities with reality. However,
we can draw two separate conclusions: (1) Generating a large and diverse
enough dataset of scenarios in a robotic paradigm is exceptionally difficult,
and (2) The newest, most powerful, models available outperform a fine-tuned
older model.

##  V Conclusions, Limitations, and Future Work

Consumer robots must complete user-provided instructions accurately, with
minimal additional feedback from the user. In this work, we’ve moved towards
robots with these abilities by applying affordances to a robot planning
scheme. We specifically introduce affordance scores, the most individually
performative of which involves prompting an LLM to produce a score
corresponding to feasibility and safety. We find that our method, which we
call LAP significantly reduces the amount of human feedback needed to complete
tasks successfully, including a reduction in human help over prior art KnowNo
[3] of 33% at a success rate of 70% in our real-world experiments.

Limitations and Future Work. The primary limitation of the methods we propose
is perception. In all of our scenarios, the LLM is provided with ground truth
perception information. While this is following prior art, the uncertainty in
perception found in a deployed environment is important to properly handle. An
additional limitation is the limited task diversity of the dataset we tested
on, and the assumption that humans will always faithfully provide correct
answers to queries. Future work should address these elements to make sure the
robot completes only safe actions. An additional direction for future work is
integrating user preferences and using them to further limit the amount of
human help needed. Finally, work on more diverse datasets with additional
tasks and instructions would be valuable to continue working towards more
helpful home robots.

## References

  * [1] B. Ichter, A. Brohan _et al._ , “Do As I Can, Not As I Say: Grounding Language in Robotic Affordances,” in _Proceedings of The 6th Conference on Robot Learning_. PMLR, Mar. 2023, pp. 287–318.
  * [2] V. S. Dorbala, J. F. M. Jr, and D. Manocha, “Can an Embodied Agent Find Your “Cat-shaped Mug”? LLM-Based Zero-Shot Object Navigation,” _IEEE Robotics and Automation Letters_ , pp. 1–8, 2023.
  * [3] A. Z. Ren, A. Dixit _et al._ , “Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners,” Sep. 2023.
  * [4] T. Guan, F. Liu _et al._ , “HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination & Visual Illusion in Large Vision-Language Models,” Nov. 2023.
  * [5] S. Dhuliawala, M. Komeili _et al._ , “Chain-of-Verification Reduces Hallucination in Large Language Models,” Sep. 2023.
  * [6] W. Huang, F. Xia _et al._ , “Inner Monologue: Embodied Reasoning through Planning with Language Models,” Jul. 2022.
  * [7] X. Gao, Q. Gao _et al._ , “Dialfred: Dialogue-enabled agents for embodied instruction following,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 4, pp. 10 049–10 056, 2022.
  * [8] J. Thomason, M. Murray _et al._ , “Vision-and-dialog navigation,” in _Conference on Robot Learning_. PMLR, 2020, pp. 394–406.
  * [9] K. Zhou, D. Jurafsky, and T. Hashimoto, “Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models,” in _EMNLP_ , 2023, pp. 5506–5524.
  * [10] S. Lin, J. Hilton, and O. Evans, “Teaching Models to Express Their Uncertainty in Words,” Jun. 2022.
  * [11] Y. Xiao and W. Y. Wang, “Quantifying Uncertainties in Natural Language Processing Tasks,” _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 33, no. 01, pp. 7322–7329, Jul. 2019.
  * [12] B. Kumar, C. Lu _et al._ , “Conformal Prediction with Large Language Models for Multi-Choice Question Answering,” Jul. 2023.
  * [13] T.-T. Do, A. Nguyen, and I. Reid, “Affordancenet: An end-to-end deep learning approach for object affordance detection,” in _ICRA_ , 2018.
  * [14] M. Hassan, P. Ghosh _et al._ , “Populating 3D Scenes by Learning Human-Scene Interaction,” in _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, 2021.
  * [15] X. Li, S. Liu _et al._ , “Putting humans in a scene: Learning affordance in 3d indoor environments,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * [16] J. F. Mullen and D. Manocha, “PACE: Data-Driven Virtual Agent Interaction in Dense and Cluttered Environments,” _IEEE Transactions on Visualization and Computer Graphics_ , May 2023.
  * [17] J. F. Mullen, D. Kothandaraman _et al._ , “Placing Human Animations Into 3D Scenes by Learning Interaction- and Geometry-Driven Keyframes,” in _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , 2023, pp. 300–310.
  * [18] N. Yamanobe, W. Wan _et al._ , “A brief review of affordance in robotic manipulation research,” _Advanced Robotics_ , vol. 31, no. 19-20, pp. 1086–1101, Oct. 2017.
  * [19] S. LaValle, “Planning algorithms,” _Cambridge University Press google schola_ , vol. 2, pp. 3671–3678, 2006.
  * [20] J. Canny, _The complexity of robot motion planning_. MIT press, 1988.
  * [21] D. Manocha, _Algebraic and numeric techniques in modeling and robotics_. University of California, Berkeley, 1992.
  * [22] J. Wei, X. Wang _et al._ , “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” Jan. 2023.
  * [23] T. Kojima, S. S. Gu _et al._ , “Large Language Models are Zero-Shot Reasoners,” Jan. 2023.
  * [24] A. Creswell, M. Shanahan, and I. Higgins, “Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning,” in _The Eleventh International Conference on Learning Representations_ , Sep. 2022.
  * [25] A. Lewkowycz, A. Andreassen _et al._ , “Solving Quantitative Reasoning Problems with Language Models,” Jun. 2022.
  * [26] W. Huang, P. Abbeel _et al._ , “Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents,” in _Proceedings of the 39th International Conference on Machine Learning_. PMLR, Jun. 2022, pp. 9118–9147.
  * [27] J. Wu, R. Antonova _et al._ , “TidyBot: Personalized robot assistance with large language models,” _Autonomous Robots_ , vol. 47, no. 8, pp. 1087–1102, Nov. 2023.
  * [28] B. Liu, Y. Jiang _et al._ , “LLM+P: Empowering Large Language Models with Optimal Planning Proficiency,” Sep. 2023.
  * [29] S. Y. Gadre, M. Wortsman _et al._ , “CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigation,” Dec. 2022.
  * [30] A. Padmakumar, J. Thomason _et al._ , “TEACh: Task-Driven Embodied Agents That Chat,” _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 36, no. 2, pp. 2017–2025, Jun. 2022.
  * [31] J. S. Park, B. Jia _et al._ , “Efficient generation of motion plans from attribute-based natural language instructions using dynamic constraint mapping,” in _ICRA_. IEEE, 2019.
  * [32] V. S. Dorbala, P. Goyal _et al._ , “S-eqa: Tackling subjective queries in embodied question answering,” https://gamma.umd.edu/researchdirections/embodied/seqa/, [Accessed 26-02-2024].
  * [33] K. Murray and D. Chiang, “Correcting Length Bias in Neural Machine Translation,” in _Proceedings of the Third Conference on Machine Translation: Research Papers_ , Oct. 2018, pp. 212–223.
  * [34] A. Radford, J. W. Kim _et al._ , “Learning transferable visual models from natural language supervision,” in _International conference on machine learning_. PMLR, 2021, pp. 8748–8763.
  * [35] X. Gu, T.-Y. Lin _et al._ , “Open-vocabulary Object Detection via Vision and Language Knowledge Distillation,” May 2022.
  * [36] T. Brown, B. Mann _et al._ , “Language models are few-shot learners,” _Advances in neural information processing systems_ , 2020.
  * [37] OpenAI, J. Achiam _et al._ , “Gpt-4 technical report,” 2024.

##  Appendix A Additional Experiment Details

Simulation setting. This information on the simulated task is directly from
[3], the source of the task.

  * •

Environment: there are always three blocks and bowls of color red, yellow, and
green with random locations on the table.

  * •

Goal: we use the following template: put, place, move a, one, a single of,
two, a pair of, three, all, red, yellow, green block(s), bowl(s) on, to the
left of, to the right of, to the front of, at the back of the red, green,
yellow block(s), bowl(s). The scenario distribution is uniform over these
possibilities.

  * •

Instruction: for the language instructions, we modify the goal (from the
template) according to the ambiguity type. The scenario distribution is
uniform over the listed ambiguous cases in each ambiguity type.

    * –

Attribute ambiguities: refer to the block as one of “cube”, “cuboid”, “box”,
“square object”, to the bowl as one of “container”, “round object”,
“receptacle”, or to either block or bowl as one of “object”, “item”, “thing”
(“move the blue object in yellow bowl”); refer to “blue” as one of “cyan”,
“navy”, to “green” as one of “greenish”, “grass-colored”, and to “yellow” as
“orange” or “gold”. This setting is the least ambiguous one among the three
ambiguity types.

    * –

Numeric ambiguities: refer to either two or three numerically with one of “a
few”, “a couple of”, “some”, “a handful of” (“put some blocks in the green
bowl”).

    * –

Spatial ambiguities: refer to any of the four possible directions with “near”,
“close to”,“beside”, “next to”, refer to either left to right with “lateral
to”, and refer to either front or behind with “along the line of sight”. This
setting is the most ambiguous one among the three ambiguity types.

Outsize of [3]’s task description, we exchanged communications with them that
provided further clarification for setting up this task. Our additional
findings which should help with setting up this task are below.

  * •

For goals with numeric ambiguities, the ‘block’ is always the object to move,
and all the ‘blocks’ are the same color to avoid multiple ambiguities.

  * •

The authors of [3] used different few shot prompts for each ambiguity, so that
the examples provided matched the ambiguity accordingly.

We chose not to conform to the second point, instead using one few shot prompt
for all ambiguities. We believe that this is more faithful to the task as the
agent should have no knowledge of which ambiguity is being provided to it. We
did test multiple prompts with varying examples and chose the one with the
best results, although results did not change much when using reasonable
examples of all ambiguity types. Our code, which will be release after
acceptance, will show how we produced the full dataset listing from [3] and
should help resolve any additional questions.

Hardware Mobile Manipulator setting. This information on the mobile
manipulator task is directly from [3], the source of the task.

  * •

Environment: the full list of possible objects include: bottled water, bottled
tea, orange soda, RedBull, Coke, Pepsi, Sprite, rice chips, jalapeno chips,
kettle chips, multigrain chips, apple, orange, energy bar, clean sponge, dirty
sponge, metal bowl, plastic bowl. Depending on the ambiguity listed below,
there is three objects placed on the top of the counter (including randomly
sampled distractors from the list). There is also a set of landfill, compost,
and recycling bins, a microwave, and a portable stove.

  * •

Instruction: for convenience, we introduce the possible instructions first in
different ambiguous scenarios; they each correspond to possible goals. Please
refer to https://robot-help.github.io/prompts/mobile_tasks.txt for the full
list. The possible instructions are a uniform distribution over different
types: (1) single-label, e.g., ‘Bring me a Coke’ (unambiguous); (2) creative-
single-label, e.g., ‘I want a healthy fruit to munch on.’ which means the
apple (unambiguous); (3) multi-label, e.g., ‘Bring me a cola.’ and either Coke
or Pepsi is acceptable; (4) creative-multi-label, e.g., ‘Bring me something
with a kick.’ and either RedBull or jalapeno chips are acceptable; (5)
spatially-ambiguous, e.g., ‘Put the Coke in the drawer’ or ‘Put the Coke near
the fruit’ which under-specifies the drawer or fruit; (6) unsafe, e.g., ‘Can
you dispose of the bottle drink? It should have expired.’ or ‘Place the bowl
on the stove, please.’; (7) Winograd, e.g., ’There is a sponge and a bag of
rice chips…I don’t want to use it for cleaning any more. Can you please
dispose of it?” We use the GPT-4 model for generating the creative tasks.

  * •

Goal: the corresponding goal for the ambiguous instructions above. For
example, the instruction is “Put the Coke in the drawer”, and the goal is
uniform over the two possibilities: put the Coke in the top drawer, and put
the Coke in the bottom drawer.

We faithfully implement [3]’s task description exactly as described, using
their code provided here: https://github.com/google-research/google-
research/tree/master/language_model_uncertainty. They specifically provide two
files with their 300 tasks and corresponding prompts in the code. While this
task is challenging and diverse, we believe that more can be done to expand
the task to a much larger set of commands and situations. This is very limited
in scope with very few object classes, and most commands boiling down to
picking up and moving an object. Future work must be conducted to expand this
task for more general future robots.

##  Appendix B Additional LLM Ablations

As described in our experimentation section, we tested LAP with one model
(GPT-4, GPT-4-Turbo, and GPT-3.5-Turbo) for the MCQA generation and scoring,
but GPT-3.5-Turbo for finding our prompt-based affordance score, $a_{pr}$.
This is a possible cost saving measure for users of LAP who need to minimize
their API or compute costs. This plot is found in Figure 7. We find that
performance slightly decreases when using GPT-3.5-Turbo for our prompt-based
affordance score. We suggest that users run some testing on their own use case
to determine the right trade off.

Figure 7: Comparison of LAP+$a_{pr}\times a_{c}$ and KnowNo accross different
LLMs. Note that LAP improves upon KnowNo with every model, and that
performance overall generally increases with each model. GPT-4-Turbo exhibited
poor performance in the MCQA generation phase relative to GPT-4, frequently
asking for clarifications instead of producing a set of candidate options,
capping the possible success rate.


# No-signaling in Nonlinear Extensions of Quantum Mechanics

Rohit Kishan Ray<EMAIL_ADDRESS>Department of Physics, Indian
Institute of Technology Kharagpur, Kharagpur, West Bengal - 721302, India
Gian Paolo Beretta<EMAIL_ADDRESS>Department of Mechanical and
Industrial Engineering, University of Brescia, 25123 Brescia, Italy

###### Abstract

Devising a nonlinear extension of quantum mechanics is nontrivial because
unphysical features such as supraluminal communication (signaling) are to be
excluded. In this Letter, we show that the steepest entropy ascent formalism
is a viable no-signaling extension belonging to a broader class of no-
signaling nonlinear evolution equations for which the local evolution of a
subsystem is not necessarily bound to depend only on its reduced state. We
prove that, in addition to the local reduced density operator, there is a
broad class of local operators called ‘local perceptions’, which are
insensitive to unitary operations localized within other non-interacting
systems.

Quantum mechanics (QM) in its most common form (Schrödinger–von Neumann
formalism) is linear in state space, where linear operators operate upon state
vectors, and the time evolution is linear. Linearity implies no-cloning, as
first introduced by Park [1] and later explicitly shown by Wootters, Zurek,
and Dieks [2, *dieks_1982_communication], which is sufficient for no-
signaling. However, Weinberg felt the necessity to test the said linearity of
QM [4]. He introduced nonlinearity in the operator formalism of the
Schrödinger equation while maintaining order one homogeneity. Gisin [5] showed
that such formalism attracts the possibility of faster-than-light
communication, _viz.,_ signaling. Polchinski [6] proposed an alternate
approach, where to have no-signaling one must constrain the local operators to
be functions of the corresponding reduced density matrix only. Despite this
condition, Polchinski formalism allowed for signaling between different
branches of the wavefunction (Everett telephone) [6]. Wódkiewicz and Scully
[7] calculated Weinberg-type nonlinear evolution for two-level atoms, which
later proved to be an interpretation-dependent result [8]. Czachor showed that
a nonlinear operator induces mobility phenomena (nonconservation of the inner-
product of two pure states)[9]. Nonlinearity, when introduced via stochastic
QM through Lindblad operator formalism [10] for open quantum systems, has been
shown to respect no-signaling. The Weinberg-inspired nonlinearities can be
used to design faster algorithms that solve $NP$ complete problems in
polynomial time [11]. Ensuing work by Ferrero  _et al._ [12] has shown that
nonlinearity in QM can be accommodated without compromising no-signaling if we
consider the time evolution to be nonlinear albeit maintaining linearity in
the state space and operators. In a more recent work [13,
*rembielinski_2021_nonlinear], it has been shown that convex quasilinear maps
can contribute to the nonlinear dynamics of QM without invoking signaling, and
this formalism retains much of QM as it is. Thusly, Rembieliński and Caban
[13] have found a minimal permissible deviation from the linear structure of
QM dynamics.

As mentioned in the above-mentioned literature [9, 7, 8, 11], experimental
verification of nonlinearity has been inconclusive and reinstated linear
quantum mechanics. However, these experiments have mostly focused on operator
nonlinearity. One could wonder if there was a more fundamental approach to
device a nonlinear model of quantum mechanics devoid of signaling-related
peculiarities. A nonlinear theory that will not allow oscillation of entropy
[9], or stochastic jump-induced mixing of pure states [10] causing the rank of
the density matrix to arbitrarily increase. For about four decades, such a
nonlinear extension of quantum mechanics initiated by Hatsopoulos and
Gyftopoulos [15, *hatsopoulos_1976_unifieda, *hatsopoulos_1976_unifiedb,
*hatsopoulos_1976_unifiedc], and developed by Beretta [19,
*beretta_1985_quantuma, *maddox1985uniting, *beretta_2005_nonlinear,
*beretta_2009_nonlinear, *beretta_2010_maximum] has existed. This formalism
originally sought to establish the second law of thermodynamics as a
foundational principle at par with other conservation laws, embedded in a
nonlinear law of evolution for mixed states, thus conceptualizing stability of
canonical states and spontaneous decoherence. Later, the steepest entropy
ascent (SEA) formalism was developed as a powerful modeling tool, applied to
various quantum systems [25, *vonspakovsky_2014_trends, 27], and even elevated
to the status of fourth law of thermodynamics [28, 29] because of its
equivalence with most of the more recent far-nonequilibrium modeling
formalisms. Nonetheless, while discussing possible nonlinear theories of QM in
the context of no-signaling, SEA formalism is hardly ever mentioned in the
literature cited in the preceding paragraph. The authors in Refs. [22, 23, 24,
26] claim that the SEA formalism abides by the no-signaling criteria as
discussed by Gisin and others. Yet, an explicit focus on no-signaling is
lacking in their works. As a result, a lacuna remains to be breached to answer
Weinberg’s original question. This Letter provides definitive proof that the
SEA nonlinear extension of QM respects the no-signaling principle.

Philosophically, SEA evolution was designed as part of a theory whereby
spontaneous decoherence could be conceived as a fundamental dynamical feature,
in contrast to the coarse-graining approach. Entropy, the second law, and
irreversibility could acquire a more fundamental stature, by emerging as
deterministic consequences of the law of evolution, without contradicting
standard QM [30]. Nonlinearity was known to be essential for this purpose [31,
*ParkSimmons1983_knots]. While the prevalent notion is that the second law is
statistical, the pioneers of SEA believed that the many knots of
thermodynamics [32, 33] could be untied by elevating it to a more fundamental
stature [15, *hatsopoulos_1976_unifieda, *hatsopoulos_1976_unifiedb,
*hatsopoulos_1976_unifiedc]. But later, the strength and generality of its
mathematical formalism made SEA a suitable tool for thermodynamically
consistent nonequilibrium modeling even beyond and outside of its original
context, including the current quest for fundamental or phenomenological
nonlinear extensions of QM for quantum computing applications, and attempts to
assign ontological status to mixed density operators, albeit motivated
differently from Refs. [15, *hatsopoulos_1976_unifieda,
*hatsopoulos_1976_unifiedb, *hatsopoulos_1976_unifiedc, 31, 30, 4, 34, 33].
The ontological hypothesis amounts to assuming, accepting, and interpreting
the elevation of the mixed density operators to the same status of ontic
physical states that traditional von Neumann QM reserves to pure states. To
see that it is a conceptual prerequisite to adopt a nonlinear law of evolution
for mixed states, let $\mathcal{W}_{t}$ with
$\rho^{\prime}=\mathcal{W}_{t}(\rho)$ denote a nonlinear map representing the
evolution $\rho\rightarrow\rho^{\prime}$ after some time $t$. Consider the
evolution of three states $\rho_{1}$, $\rho_{2}$, $\rho_{3}$ such that
$\rho_{1}\rightarrow\rho_{1}^{\prime}$,
$\rho_{2}\rightarrow\rho_{2}^{\prime}$,
$\rho_{3}\rightarrow\rho_{3}^{\prime}$, respectively. Further, assume that
$\rho_{3}=w\rho_{1}+(1{-}w)\rho_{2}$ with $0\leq w\leq 1$. Due to the
nonlinearity of $\mathcal{W}_{t}$, in general $\rho_{3}^{\prime}\neq
w\rho_{1}^{\prime}+(1{-}w)\rho_{2}^{\prime}$. This is no issue only if the
ontological hypothesis is accepted, since $w$ and $1{-}w$ do not represent
epistemic ignorance anymore. A sufficient condition [13] for the nonlinear map
to be no-signaling is that it be convex quasilinear, i.e., it always admits a
$w^{\prime}$, $0\leq w^{\prime}\leq 1$, such that
$\rho_{3}^{\prime}=w^{\prime}\rho_{1}^{\prime}+(1{-}w^{\prime})\rho_{2}^{\prime}$.
In this Letter, we introduce a much broader class of convex-nonlinear maps
that are no-signaling. The SEA map, despite its logarithmic nonlinearity and
structured construction, belongs to this class and has general thermodynamic
compatibility features.

The no-signaling condition, as noted in [12], is usually imposed by asking
that in the absence of mutual interactions between subsystems A and B, the
evolution of the local observables of A should only depend on its own reduced
state. The SEA formalism, however, demonstrates that we can take a less
restrictive view [22]: we only require that, if A and B are non-interacting,
the law of evolution must not allow that a local unitary operation within B
could affect the time evolution of local (reduced, marginal) state of A. Thus,
the condition $\rho_{A}=\rho_{A}^{\prime}$, such as for the two different
states $\rho\neq\rho_{A}\otimes\rho_{B}$ and
$\rho^{\prime}=\rho_{A}\otimes\rho_{B}$, does not require that
$\rm{d}\rho_{A}/\rm{d}t=\rm{d}\rho_{A}^{\prime}/\rm{d}t$, because local memory
of past interactions, i.e., existing entanglement and/or correlations, may
well influence the local evolutions without violating no-signaling. This
incorporates the idea that (1) by studying the local evolutions we can
disclose the existence of correlations, but only of the type that can be
classically communicated between the subsystems, and (2) in the absence of
interactions the nonlinear dynamics may produce the fading away of
correlations (spontaneous decoherence) but cannot create new correlations.

In linear QM, the system’s composition is specified by declaring: (1) the
Hilbert space structure as direct product
${\mathcal{H}}=\bigotimes_{{J\vphantom{\overline{J}}}=1}^{M}{\mathcal{H}}_{J\vphantom{\overline{J}}}$
of the subspaces of the $M$ component subsystems, and (2) the overall
Hamiltonian operator $H=\sum_{J=1}^{M}H_{J\vphantom{\overline{J}}}\otimes
I_{\overline{J}}+V$ where $H_{J\vphantom{\overline{J}}}$ (on
${\mathcal{H}}_{J\vphantom{\overline{J}}}$) is the local Hamiltonian of the
$J$-th subsystem, $I_{\overline{J}}$ the identity on the direct product
${\mathcal{H}}_{\overline{J}}=\bigotimes_{K\neq J}{\mathcal{H}}_{K}$ of all
the other subspaces, and $V$ (on ${\mathcal{H}}$) is the interaction
Hamiltonian. The linear law of evolution,
$\dot{\rho}=-\mathrm{i}[H,\rho]/\hbar$, has a universal structure and entails
the local evolutions through partial tracing,
$\dot{\rho}_{J\vphantom{\overline{J}}}=-\mathrm{i}[H_{J\vphantom{\overline{J}}},\rho_{J\vphantom{\overline{J}}}]/\hbar-\mathrm{i}\Tr_{\overline{J}}([V,\rho])/\hbar$.
Thus, we recover the universal law
$\dot{\rho}_{J\vphantom{\overline{J}}}=-\mathrm{i}[H_{J\vphantom{\overline{J}}},\rho_{J\vphantom{\overline{J}}}]/\hbar$
for the local density operator $\rho_{J}=\Tr_{\overline{J}}(\rho)$ if
subsystem ${J\vphantom{\overline{J}}}$ does not interact with the others
(i.e., if $V=I_{J\vphantom{\overline{J}}}\otimes V_{\overline{J}}$).

Instead, a fully nonlinear QM cannot have a universal structure, because the
subdivision into subsystems must explicitly concur to the structure of the
dynamical law (see [24] for more on this). A different subdivision requires a
different equation of motion. This high price for abandoning linearity is
clearly reflected in the nontrivial structure of the SEA law of evolution. But
this renders it compatible with the compelling constraint that correlations
should not build up and signaling between subsystems should not occur other
than per effect of the interaction Hamiltonian $V$ through the standard
Schrödinger term $-i[H,\rho]/\hbar$ in the evolution law.

Seldom used in composite quantum dynamics analysis, but crucial in our
opinion, are the physical observables first introduced in [20] and later
(starting with [35]) referred to as ‘local perception’ operators (on
${\mathcal{H}}_{J\vphantom{\overline{J}}}$). Together with their ‘deviation
from the local mean value’ operators and covariance functionals, they are
defined as follows

$\displaystyle(X)^{J}_{\rho}$
$\displaystyle=\Tr_{\overline{J}}[(I_{J\vphantom{\overline{J}}}\otimes\rho_{\overline{J}})X]\,,$
(1) $\displaystyle\Delta(X)^{J\vphantom{\overline{J}}}_{\rho}$
$\displaystyle=(X)^{J\vphantom{\overline{J}}}_{\rho}-I_{J\vphantom{\overline{J}}}\Tr[\rho_{J\vphantom{\overline{J}}}(X)^{J\vphantom{\overline{J}}}_{\rho}]\,,$
(2) $\displaystyle(X,Y)^{J\vphantom{\overline{J}}}_{\rho}$
$\displaystyle=\frac{1}{2}\Tr[\rho_{J\vphantom{\overline{J}}}\anticommutator{\Delta(X)^{J\vphantom{\overline{J}}}_{\rho}}{\Delta(Y)^{J\vphantom{\overline{J}}}_{\rho}}]\,,$
(3)

where $\rho_{\overline{J}}=\Tr_{J\vphantom{\overline{J}}}(\rho)$. For a
bipartite system AB, the local perception operators $(X)^{A}_{\rho}$ (on
${\mathcal{H}}_{A}$) and $(X)^{B}_{\rho}$ (on ${\mathcal{H}}_{B}$) are the
unique operators that for a given $X$ on ${\mathcal{H}}_{AB}$ satisfy for all
states $\rho$ the identity

$\Tr[\rho_{A}(X)^{A}_{\rho}]=\Tr[(\rho_{A}\otimes\rho_{B})X]=\Tr[\rho_{B}(X)^{B}_{\rho}]\,,\\\
$ (4)

which shows that they represent all that A and B can say about the overall
observable X by classically sharing their local states. Operator
$(X)^{A}_{\rho}$ can be viewed as the projection onto ${\mathcal{H}}_{A}$ of
the operator $X$ weighted by the local state $\rho_{B}$ of subsystem $B$. It
is a local observable for subsystem A which, however, depends on the overall
state $\rho$ and overall observable $X$. Its local mean value
$\Tr_{A}[\rho_{A}(X)^{A}_{\rho}]$ differs from the mean value $\Tr(\rho X)$
for the overall system AB, except when A and B are uncorrelated
($\rho=\rho_{A}\otimes\rho_{B}$). It was dubbed ‘local perception’ because
even if B performs a local tomography and sends the measured $\rho_{B}$ to A
by classical communication, the most that A can measure locally about the
overall observable $X$ is $(X)^{A}_{\rho}$.

The overall energy and entropy of the composite system are locally perceived
within subsystem $J$ through the operators
$(H)^{J\vphantom{\overline{J}}}_{\rho}$ and
$(S(\rho))^{J\vphantom{\overline{J}}}_{\rho}$ defined on
${\mathcal{H}}_{J\vphantom{\overline{J}}}$ by Eq. (1), respectively with
$X=H$, the overall Hamiltonian, and $X=S(\rho)=-k_{\rm\scriptscriptstyle
B}\rm{Bln}(\rho)$, that we call the overall entropy operator, where
$\rm{Bln}(x)$ denotes the discontinuous function $\rm{Bln}(x)=\ln(x)$ for
$0<x\leq 1$ and $\rm{Bln}(0)=0$. Note that the ‘locally perceived overall
entropy’ operator $(S(\rho))^{J\vphantom{\overline{J}}}_{\rho}$ is different
from the ‘local entropy’ operator
$S(\rho_{J\vphantom{\overline{J}}})=-k_{\rm\scriptscriptstyle B}{\rm
B}_{J}{\rm ln}(\rho_{J\vphantom{\overline{J}}})$. Their mean values
$\Tr[\rho_{J\vphantom{\overline{J}}}(S(\rho))^{J\vphantom{\overline{J}}}_{\rho}]=-k_{\rm\scriptscriptstyle
B}\Tr[(\rho_{J\vphantom{\overline{J}}}\otimes\rho_{\overline{J}}){\rm
Bln}(\rho)]$ and
$\Tr[\rho_{J\vphantom{\overline{J}}}S(\rho_{J\vphantom{\overline{J}}})]=-k_{\rm\scriptscriptstyle
B}\Tr[\rho_{J\vphantom{\overline{J}}}\ln(\rho_{J\vphantom{\overline{J}}})]$
are different. Only when
$\rho=\rho_{J\vphantom{\overline{J}}}\otimes\rho_{\overline{J}}$ they are
related by
$\Tr[\rho_{J\vphantom{\overline{J}}}(S(\rho))^{J\vphantom{\overline{J}}}_{\rho}]=\Tr[\rho_{J\vphantom{\overline{J}}}S(\rho_{J\vphantom{\overline{J}}})]+\Tr[\rho_{\overline{J}}S(\rho_{\overline{J}})]=-k_{\rm\scriptscriptstyle
B}\Tr[\rho\ln(\rho)]$. Likewise, the ‘locally perceived overall Hamiltonian’
operator $(H)^{J\vphantom{\overline{J}}}_{\rho}$ is different from the ‘local
Hamiltonian’ operator $H_{J\vphantom{\overline{J}}}$. Their mean values
$\Tr[\rho_{J\vphantom{\overline{J}}}(H)^{J\vphantom{\overline{J}}}_{\rho}]=\Tr[(\rho_{J\vphantom{\overline{J}}}\otimes\rho_{\overline{J}})H]$
and $\Tr(\rho_{J\vphantom{\overline{J}}}H_{J\vphantom{\overline{J}}})$ are
different, and only when $V=I_{J\vphantom{\overline{J}}}\otimes
V_{\overline{J}}$ they are related by
$\Tr[\rho_{J\vphantom{\overline{J}}}(H)^{J\vphantom{\overline{J}}}_{\rho}]=\Tr(\rho_{J\vphantom{\overline{J}}}H_{J\vphantom{\overline{J}}})+\Tr(\rho_{\overline{J}}H_{\overline{J}})=\Tr(\rho
H)$. However, it is noteworthy that when the overall observable $X$ is
‘separable for subsystem J’, in the sense that
$X=X_{J\vphantom{\overline{J}}}\otimes
I_{\overline{J}}+I_{J\vphantom{\overline{J}}}\otimes X_{\overline{J}}$ then,
even if $\rho\neq\rho_{J\vphantom{\overline{J}}}\otimes\rho_{\overline{J}}$,
the deviations and covariances reduce to their local versions,

$\displaystyle\Delta(X)^{J\vphantom{\overline{J}}}_{\rho}$
$\displaystyle=\Delta
X_{J\vphantom{\overline{J}}}=X_{J\vphantom{\overline{J}}}-I_{J\vphantom{\overline{J}}}\Tr[\rho_{J\vphantom{\overline{J}}}X_{J\vphantom{\overline{J}}}]\,,$
(5) $\displaystyle(X,Y)^{J\vphantom{\overline{J}}}_{\rho}$
$\displaystyle=\Tr[\rho_{J\vphantom{\overline{J}}}\anticommutator{\Delta
X_{J\vphantom{\overline{J}}}}{\Delta Y_{J\vphantom{\overline{J}}}}]/2\,.$ (6)

Now, to formalize the no-signaling definition following [22] as discussed
above, we impose that if A and B are non-interacting, a local unitary
operation on B should not affect the evolution of A. So, assume that with AB
in the state $\rho$, a local operation on B changes the state to

$\rho^{\prime}=(I_{A}\otimes U_{B})\,\rho\,(I_{A}\otimes U_{B}^{\dagger})\,,$
(7)

where $U_{B}$ is an arbitrary unitary operator ($U_{B}^{\dagger}U_{B}=I_{B}$).
Using the properties of the partial trace, in particular,

$\displaystyle\Tr_{B}[(I_{A}\otimes X_{B})Z_{AB}]=\Tr_{B}[Z_{AB}(I_{A}\otimes
X_{B})]\,,$ $\displaystyle\Tr_{A}[(I_{A}\otimes X_{B})Z_{AB}(I_{A}\otimes
Y_{B})]=X_{B}\Tr_{A}(Z_{AB})Y_{B}\,,$

we obtain the identities

$\displaystyle\rho_{B}$ $\displaystyle=\Tr_{A}[(I_{A}\otimes
U_{B}^{\dagger})\,\rho^{\prime}\,(I_{A}\otimes
U_{B})]=U_{B}^{\dagger}\rho^{\prime}_{B}U_{B},$ (8)
$\displaystyle\rho^{\prime}_{A}$ $\displaystyle=\Tr_{B}[(I_{A}\otimes
U_{B})\,\rho\,(I_{A}\otimes U_{B}^{\dagger})]=\Tr_{B}[(I_{A}\otimes
U_{B}^{\dagger}U_{B})\,\rho]$ $\displaystyle=\Tr_{B}[(I_{A}\otimes
I_{B})\,\rho]=\rho_{A},$ (9)

which confirms that a local operation on B does not affect the local state
$\rho_{A}$ of A, hence the usual idea [12] that for no-signaling it is
sufficient that the dynamical model implies evolutions of local observables
that depend only on $\rho_{A}$. But it is seldom noted that this is not a
necessary condition. In fact, we prove next that not only the local reduced
state $\rho_{A}$ but also the local perception operators $(F(\rho))^{A}$ of
any well-defined nonlinear function $F(\rho)$ of the overall state (such as
the function $S(\rho)$ defined above for entropy) are not affected by local
operations on B according to Eq. (7). And since the SEA formalism is based on
such local perception operators, this is an important lemma in the proof that
SEA is no-signaling.

So, let us apply Eq. (7) to a function of $F(\rho)$ as locally perceived by A
represented, according to definition Eq. (1), by its partial trace weighted
with respect to $\rho_{B}$,

$(F(\rho))^{A}=\Tr_{B}[(I_{A}\otimes\rho_{B})F(\rho)].$ (10)

A function of $\rho$ is defined from its eigenvalue decomposition by
$F(\rho)=VF(D)V^{\dagger}=\sum_{j}F(\lambda_{j})\outerproduct{\lambda_{j}}{\lambda_{j}}$,
where $\rho=VDV^{\dagger}$, $D=\sum_{j}\lambda_{j}\outerproduct{j}{j}$, and
$V=\sum_{j}\outerproduct{\lambda_{j}}{j}$. Since unitary transformations do
not alter the eigenvalues,

$F(\rho^{\prime})=V^{\prime}F(D)V^{\prime\dagger}\mbox{ where
}V^{\prime}=(I_{A}\otimes U_{B})V\,,$ (11)

and therefore, using Eq. (8) in the last step, we obtain

$\displaystyle(F(\rho^{\prime}))^{A}=\Tr_{B}[(I_{A}\otimes\rho^{\prime}_{B})F(\rho^{\prime})]$
$\displaystyle=\Tr_{B}[(I_{A}\otimes\rho^{\prime}_{B})\,(I_{A}\otimes
U_{B})VF(D)V^{\dagger}(I_{A}\otimes U_{B}^{\dagger})]$
$\displaystyle=\Tr_{B}[(I_{A}\otimes
U_{B}^{\dagger}\rho^{\prime}_{B}U_{B})\,VF(D)V^{\dagger}]$
$\displaystyle=\Tr_{B}[(I_{A}\otimes\rho_{B})\,F(\rho)]=(F(\rho))^{A}\,.$ (12)

This confirms that local operations on B do not affect the local perception
operators of A and, therefore, their proper use in nonlinear QM does not cause
signaling issues.

We are now ready to introduce the last but not least essential ingredient of a
general composite-system nonlinear QM, namely, the system’s structure-
dependent expressions of the separate contribution of each subsystem to the
dissipative term of the equation of motion for the overall state $\rho$. As
discussed above (and clearly recognized in the early SEA literature [20, 22,
24]), the composite-system nonlinear evolution should reflect explicitly the
internal structure of the system, essentially by declaring which subsystems
are to be prevented from nonphysical effects such as signaling, exchange of
energy, or build-up of correlations between non-interacting subsystems. In
terms of the notation introduced above, the structure proposed in [20, 24] for
the dissipative term of the dynamics to be added to the usual Hamiltonian term
is as follows

$\derivative{\rho}{t}=-\frac{\mathrm{i}}{\hbar}\commutator{\mathit{H}}{\rho}-\sum_{J=1}^{M}\anticommutator*{\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}}{\rho_{J\vphantom{\overline{J}}}}\otimes\rho_{\overline{J}}\,,$
(13)

where the ‘local dissipation operators’
$\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$ (on
${\mathcal{H}}_{J\vphantom{\overline{J}}}$) may be nonlinear functions of the
local observables of J, the reduced state $\rho_{J\vphantom{\overline{J}}}$,
and the local perception operators of overall observables. For the dissipative
term to preserve $\Tr(\rho)$, operators
$\anticommutator*{\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}}{\rho_{J\vphantom{\overline{J}}}}$
must be traceless. To preserve $\Tr(\rho H)$ [and possibly other conserved
properties $\Tr(\rho C_{k})$], operators
$\anticommutator*{\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}}{\rho_{J\vphantom{\overline{J}}}}(H)^{J\vphantom{\overline{J}}}_{\rho}$
[and
$\anticommutator*{\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}}{\rho_{J\vphantom{\overline{J}}}}(C_{k})^{J\vphantom{\overline{J}}}_{\rho}$]
must also be traceless. The rate of change of the overall system entropy
$s(\rho)=-k_{\rm\scriptscriptstyle B}\Tr[\rho\ln(\rho)]$ is

$\derivative{s(\rho)}{t}=-\sum_{J=1}^{M}\Tr[\anticommutator*{\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}}{\rho_{J\vphantom{\overline{J}}}}(S(\rho))^{J\vphantom{\overline{J}}}_{\rho}]\,,$
(14)

and the local nonlinear evolution of subsystem J is obtained by partial
tracing over ${\mathcal{H}}_{\overline{J}}$, i.e., in general,

$\derivative{\rho_{J\vphantom{\overline{J}}}}{t}=-\frac{\mathrm{i}}{\hbar}\commutator*{H_{J\vphantom{\overline{J}}}}{\rho_{J\vphantom{\overline{J}}}}-\frac{\mathrm{i}}{\hbar}\Tr_{\overline{J}}(\commutator*{V}{\rho})-\anticommutator*{\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}}{\rho_{J\vphantom{\overline{J}}}}\,,$
(15)

where we recall that the second term in the rhs can be put, for weak
interactions and under well-known assumptions, in Kossakowski-Lindblad form.

Before introducing the SEA assumption, we emphasize that the construction
obtained so far, Eq. (13), opens up and paves the way for a class of no-
signaling nonlinear evolution equations that is much broader, through all the
possible compatible choices of the operators
$\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$, than nonlinear laws
restricted by the sufficient but not necessary condition that ${\rm
d}\rho_{J\vphantom{\overline{J}}}/{\rm d}t$ be a function of
$\rho_{J\vphantom{\overline{J}}}$ only. We can formally state this no-
signaling condition using the following statement,

$\derivative{\rho_{J}}{t}=f(\rho_{J},(C_{k})^{J}).$ (16)

Finally, to introduce the SEA assumption in the spirit of the fourth law of
thermodynamics [29, 28], one way is employing a variational principle. We
first observe from Eq. (14) that the rate of entropy change contributed by
subsystem J is directly proportional to the norm of the operator
$\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$, so there is no maximum
entropy production rate because we can trivially increase it indefinitely by
simple multiplication of $\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$ by a
positive scalar. But we can fix that norm, and maximize against the direction
in operator space, to identify, for each given state $\rho$, the operators
$\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$ that point in the direction of
steepest entropy ascent. To this end, to recover the original SEA formulation
[20] let us maximize Eq. (14) subject to the conservation constraints
$\Tr[\anticommutator*{\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}}{\rho_{J\vphantom{\overline{J}}}}(C_{k})^{J\vphantom{\overline{J}}}_{\rho}]=0$
where $C_{1}=I$, $C_{2}=H$, and $C_{k}$ are other conserved properties (if
any), together with the fixed weighted norm constraints
$\Tr[\rho_{J\vphantom{\overline{J}}}(\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho})^{2}]=\rm{const}$
(for more general SEA formulations in terms of a different metric as necessary
to incorporate Onsager reciprocity see [28, 29]). Introducing Lagrange
multipliers $\beta^{J\vphantom{\overline{J}}}_{k}$ and
$\tau_{J\vphantom{\overline{J}}}$ for the conservation and norm constraints,
respectively, and imposing vanishing variational derivatives with respect to
operators $\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$ at fixed $\rho$ and
$\rho_{J\vphantom{\overline{J}}}$’s (derivation details in [24, 28]) yields

$2\tau_{J\vphantom{\overline{J}}}\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}=({\rm
Bln}(\rho))^{J\vphantom{\overline{J}}}_{\rho}+{\textstyle\sum_{\ell}}\beta^{J\vphantom{\overline{J}}}_{\ell}(C_{\ell})^{J\vphantom{\overline{J}}}_{\rho}.$
(17)

where the multipliers $\beta^{J\vphantom{\overline{J}}}_{\ell}$ must solve the
system of equations obtained by substituting these maximizing expressions of
the $\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$’s into the conservation
constraints,

$\displaystyle\sum_{\ell}\beta^{J\vphantom{\overline{J}}}_{\ell}\Tr[\rho_{J\vphantom{\overline{J}}}\anticommutator{(C_{\ell})^{J\vphantom{\overline{J}}}_{\rho}}{(C_{k})^{J\vphantom{\overline{J}}}_{\rho}}]$
$\displaystyle=-\Tr[\rho_{J\vphantom{\overline{J}}}\anticommutator{({\rm
Bln}(\rho))^{J\vphantom{\overline{J}}}_{\rho}}{(C_{k})^{J\vphantom{\overline{J}}}_{\rho}}]\,.$
(18)

When $C_{1}=I$ and $C_{2}=H$ determine the conserved properties and Eqs. (No-
signaling in Nonlinear Extensions of Quantum Mechanics) are linearly
independent, using Cramers’ rule, properties of determinants, and definitions
(2) and (3), the SEA dissipators can be cast as

$\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}=\dfrac{1}{4\tau_{J\vphantom{\overline{J}}}}\tfrac{\vmqty{undef}}{\displaystyle(H,H)^{J\vphantom{\overline{J}}}_{\rho}}\,.$
(19)

The rate of entropy production may be expressed as

$\derivative{s(\rho)}{t}=\sum_{J=1}^{M}\dfrac{1}{2\tau_{J\vphantom{\overline{J}}}}\tfrac{\vmqty{undef}}{\displaystyle(H,H)^{J\vphantom{\overline{J}}}_{\rho}}\,,$
(20)

showing clearly that it is nonnegative since the numerators in the summation
are Gram determinants.

Regarding no-signaling, we note that: (1) if subsystem J is noninteracting,
$H=H_{J\vphantom{\overline{J}}}\otimes
I_{\overline{J}}+I_{J\vphantom{\overline{J}}}\otimes H_{\overline{J}}$, then
$\Delta(H)^{J\vphantom{\overline{J}}}_{\rho}=H_{J\vphantom{\overline{J}}}-I_{J\vphantom{\overline{J}}}\Tr(\rho_{J\vphantom{\overline{J}}}H_{J\vphantom{\overline{J}}})$
and $(H,H)^{J\vphantom{\overline{J}}}_{\rho}=\Tr[\rho_{J}(\Delta
H_{J\vphantom{\overline{J}}})^{2}]$ depend only on the local
$H_{J\vphantom{\overline{J}}}$ and $\rho_{J\vphantom{\overline{J}}}$; and (2)
if J is uncorrelated, ${\rm Bln}(\rho)={\rm
Bln}(\rho_{J\vphantom{\overline{J}}})\otimes
I_{\overline{J}}+I_{J\vphantom{\overline{J}}}\otimes{\rm
Bln}(\rho_{\overline{J}})$, then $\Delta({\rm
Bln}(\rho))^{J\vphantom{\overline{J}}}_{\rho}={\rm
Bln}(\rho_{J\vphantom{\overline{J}}})-I_{J\vphantom{\overline{J}}}\Tr(\rho_{J\vphantom{\overline{J}}}\ln(\rho_{J\vphantom{\overline{J}}}))$
and $({\rm Bln}(\rho),{\rm
Bln}(\rho))^{J\vphantom{\overline{J}}}_{\rho}=\Tr[\rho_{J}(\ln(\rho_{J\vphantom{\overline{J}}}))^{2}]$
depend only on the local $\rho_{J\vphantom{\overline{J}}}$. Therefore, it is
only when J is both noninteracting and uncorrelated that its local dissipation
operator $\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$ depends only on the
local $H_{J\vphantom{\overline{J}}}$ and $\rho_{J\vphantom{\overline{J}}}$,
and the local equation of motion Eq. (15) reduces exactly to the non-composite
system version of SEA evolution [19]. Instead, if J is either interacting or
correlated, $\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$ and, therefore,
the local nonlinear SEA evolution according to Eq. (15), is determined not
only by the local $H_{J\vphantom{\overline{J}}}$ and
$\rho_{J\vphantom{\overline{J}}}$, but also by the local perceptions of the
overall interaction Hamiltonian and/or the overall entropy operator ${\rm
Bln}(\rho)$, nonetheless without violating the no-signaling condition.

To prove no-signaling, assume that subsystem J is correlated but not
interacting with any of the subsystems in $\rm{\overline{J}}$. Now switch on
an arbitrary interaction that may involve the subsystems in
$\rm{\overline{J}}$, but not subsystem J, so that the only change in the
overall Hamiltonian $H$ is the term $H_{\overline{J}}$ which changes to
$H^{\prime}_{\overline{J}}$. Within J, the locally perceived deviation
operators are $\Delta(H)^{J\vphantom{\overline{J}}}_{\rho}$ and $\Delta({\rm
Bln}(\rho))^{J\vphantom{\overline{J}}}_{\rho}$, and hence the SEA dissipator
$\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$ as well as all the terms in
the rhs of Eq. (15) are not modified by such change. Therefore, by acting
within $\rm{\overline{J}}$ it is impossible to affect the time evolution of
$\rho_{J}$ and hence of any local observable of J.

In extremal cases, it is known [19, 36, 23] that even if the subsystems are
entangled and therefore the local states $\rho_{J\vphantom{\overline{J}}}$ are
mixed, operators $\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}$ vanish and
Eqs. (13) and (15) reduce to the standard Schrödinger equation. E.g., if the
overall system is in a pure state, ${\rm Bln}(\rho)=0$, standard unitary
evolutions of pure states emerge as limit cycles of the nonlinear SEA
dynamics.

Consider the example of a two-qubit composite AB. The mixed and correlated
states

$\rho=\dfrac{1}{4}\Big{[}\mathrm{I}_{4}+\\!\\!\\!\\!\sum_{j=\\{x,y,z\\}}\\!\\!\\!(a_{j}\,\sigma_{j}\otimes\mathrm{I}_{2}+b_{j}\,\mathrm{I}_{2}\otimes\sigma_{j}+c_{j}\,\sigma_{j}\otimes\sigma_{j})\Big{]},$
(21)

are Bell diagonal states if $a_{j}=b_{j}=0$ for all $j$’s (and Werner states
if in addition $c_{j}=4w/3-1$ for all $j$’s) with eigenvalues
$4\lambda_{1}=1-c_{x}-c_{y}-c_{z}$, $4\lambda_{2}=1-c_{x}+c_{y}+c_{z}$,
$4\lambda_{3}=1+c_{x}-c_{y}+c_{z}$, $4\lambda_{4}=1+c_{x}+c_{y}-c_{z}$.
Somewhat surprisingly, Bell diagonal states are nondissipative limit cycles
within nonlinear SEA dynamics, under any Hamiltonian. Indeed, we find $({\rm
Bln}(\rho^{\mbox{\tiny
Bell}}))^{J\vphantom{\overline{J}}}_{\rho}=\mathrm{I}_{2}\sum_{k}{\rm
Bln}(\lambda_{k})/2$, so that $\Delta({\rm
Bln}(\rho))^{J\vphantom{\overline{J}}}_{\rho}=0$ and
$\mathcal{D}^{J\vphantom{\overline{J}}}_{\rho}=0$, for both $J=A,B$. But most
neighboring and other states in this class are dissipative.

For a simple example of correlated but separable mixed states, assume
$a_{x}=a$, $b_{x}=b$, and $a_{y}=a_{z}=b_{y}=b_{z}=c_{x}=c_{y}=c_{z}=0$, so
that $\rho_{A}\otimes\rho_{B}-\rho=(ab/4)\sigma_{x}\otimes\sigma_{x}$ and the
eigenvalues are $4\lambda_{1}=1-a-b$, $4\lambda_{2}=1-a+b$,
$4\lambda_{3}=1+a-b$, $4\lambda_{4}=1+a+b$. If the two noninteracting qubits A
and B have local Hamiltonians $H_{A}=H_{B}=\sigma_{z}$, we find

$\displaystyle\anticommutator*{\mathcal{D}^{A}_{\rho}}{\rho_{A}}$
$\displaystyle=\dfrac{(1-a^{2})}{16}(bf_{a,b}-g_{a,b})\,\sigma_{x},$ (22)
$\displaystyle\anticommutator*{\mathcal{D}^{B}_{\rho}}{\rho_{B}}$
$\displaystyle=\dfrac{(1-b^{2})}{16}(af_{a,b}-h_{a,b})\,\sigma_{x},$ (23)

where $f_{a,b}={\rm Bln}(\lambda_{1})-{\rm Bln}(\lambda_{2})-{\rm
Bln}(\lambda_{3})+{\rm Bln}(\lambda_{4})$, $g_{a,b}={\rm
Bln}(\lambda_{1})+{\rm Bln}(\lambda_{2})-{\rm Bln}(\lambda_{3})-{\rm
Bln}(\lambda_{4})$, $h_{a,b}={\rm Bln}(\lambda_{1})-{\rm
Bln}(\lambda_{2})+{\rm Bln}(\lambda_{3})-{\rm Bln}(\lambda_{4})$ so that the
nonlinear evolution is clearly nontrivial. But it preserves the zero mean
energies of both qubits, and while the overall entropy increases and mutual
information partially fades away, it drives the overall state towards a
nondissipative correlated state with maximally mixed marginals. We proved
above that signaling is impossible, even though $\mathcal{D}^{A}_{\rho}$
depends not only on $a$ but also on $b$, and $\mathcal{D}^{B}_{\rho}$ on $a$
which agrees with our no-signaling condition in Eq. (16).

For a slightly more elaborate example that includes entangled mixed states,
assume $a_{x}=a_{z}=a/\sqrt{2}$, $b_{x}=b_{z}=b/\sqrt{2}$, and
$c_{x}=c_{y}=c_{z}=2(a-b)/3$, so that the eigenvalues are
$4\lambda_{1}=1+a-b$, $12\lambda_{2}=3-a-5b$, $12\lambda_{3}=3+5a+b$,
$12\lambda_{4}=3-7a+7b$, and those of the partial tranpose
$12\lambda^{PT}_{1}=3+a-b$, $12\lambda^{PT}_{2}=3-5a+5b$,
$12\lambda^{PT}_{3}=3+2a-2b+\sqrt{d}$, $12\lambda^{PT}_{4}=3+2a-2b-\sqrt{d}$
with $d=25a^{2}-14ab+25b^{2}$. For $a=-b$ these states are separable for
$-3/14\leq b\leq 1/4$ and entangled for $1/4<b\leq 1/2$. If the two
noninteracting qubits A and B have local Hamiltonians
$H_{A}=H_{B}=\sigma_{z}$, we find

$\displaystyle\anticommutator*{\mathcal{D}^{A}_{\rho}}{\rho_{A}}$
$\displaystyle=\dfrac{\sqrt{2}(1-a^{2})}{80(2-a^{2})}(f_{a,b}-5bh_{a,b})\,\sigma_{x},$
(24) $\displaystyle\anticommutator*{\mathcal{D}^{B}_{\rho}}{\rho_{B}}$
$\displaystyle=-\dfrac{\sqrt{2}(1-b^{2})}{80(2-b^{2})}(g_{a,b}+5ah_{a,b})\,\sigma_{x},$
(25)

where here $f_{a,b}=3{\rm Bln}(\lambda_{1})-5{\rm Bln}(\lambda_{2})+5{\rm
Bln}(\lambda_{3})-3{\rm Bln}(\lambda_{4})$, $g_{a,b}=3{\rm
Bln}(\lambda_{1})+5{\rm Bln}(\lambda_{2})-5{\rm Bln}(\lambda_{3})-3{\rm
Bln}(\lambda_{4})$, $h_{a,b}={\rm Bln}(\lambda_{1})-{\rm
Bln}(\lambda_{2})-{\rm Bln}(\lambda_{3})+{\rm Bln}(\lambda_{4})$ so that again
the nonlinear evolution is clearly nontrivial in the sense that the local
nonlinear evolution of A (B) does not depend only on $\rho_{A}$ ($\rho_{B}$),
despite being no-signaling.

To summarize, in this Letter we prove that the SEA formalism provides a valid
non-linear extension of QM. To show this, we explore the definition of no-
signaling for composite systems and provide generalized necessary criteria in
terms of locally perceived operators, less restrictive than the traditional
criterion in terms of local density operators. Furthermore, we build on that
definition and show how, by construction, SEA is no-signaling. For non-
interacting subsystems, the traditional criterion is met for uncorrelated
states, but we provide nontrivial examples of correlated states for which it
is not met.

RKR is grateful to the INSPIRE Fellowship program by the Department of Science
and Technology, Govt. of India for funding his Ph.D., to Prof. Alok Pan of the
Indian Institute of Technology Hyderabad for many useful discussions, and to
the Wolfram Publication Watch Team for providing full access to online
Mathematica [37].

## References

  * Park [1970] J. L. Park, Found Phys 1, 23 (1970).
  * Wootters and Zurek [1982] W. K. Wootters and W. H. Zurek, Nature 299, 802 (1982).
  * Dieks [1982] D. Dieks, Physics Letters A 92, 271 (1982).
  * Weinberg [1989] S. Weinberg, Annals of Physics 194, 336 (1989).
  * Gisin [1990] N. Gisin, Physics Letters A 143, 1 (1990).
  * Polchinski [1991] J. Polchinski, Phys. Rev. Lett. 66, 397 (1991).
  * Wódkiewicz and Scully [1990] K. Wódkiewicz and M. O. Scully, Phys. Rev. A 42, 5111 (1990).
  * Czachor [1996] M. Czachor, Phys. Rev. A 53, 1310 (1996).
  * Czachor [1991] M. Czachor, Found. Phys. Lett. 4, 351 (1991).
  * Gisin and Rigo [1995] N. Gisin and M. Rigo, J. Phys. A: Math. Gen. 28, 7375 (1995).
  * Abrams and Lloyd [1998] D. S. Abrams and S. Lloyd, Phys. Rev. Lett. 81, 3992 (1998).
  * Ferrero  _et al._ [2004] M. Ferrero, D. Salgado, and J. L. Sánchez-Gómez, Phys. Rev. A 70, 014101 (2004).
  * Rembieliński and Caban [2020] J. Rembieliński and P. Caban, Phys. Rev. Research 2, 012027 (2020).
  * Rembieliński and Caban [2021] J. Rembieliński and P. Caban, Quantum 5, 420 (2021), arXiv:2003.09170 [quant-ph] .
  * Hatsopoulos and Gyftopoulos [1976a] G. N. Hatsopoulos and E. P. Gyftopoulos, Foundations of Physics 6, 15 (1976a).
  * Hatsopoulos and Gyftopoulos [1976b] G. N. Hatsopoulos and E. P. Gyftopoulos, Foundations of Physics 6, 127 (1976b).
  * Hatsopoulos and Gyftopoulos [1976c] G. N. Hatsopoulos and E. P. Gyftopoulos, Foundations of Physics 6, 439 (1976c).
  * Hatsopoulos and Gyftopoulos [1976d] G. N. Hatsopoulos and E. P. Gyftopoulos, Foundations of Physics 6, 561 (1976d).
  * Beretta  _et al._ [1984] G. P. Beretta, E. P. Gyftopoulos, J. L. Park, and G. N. Hatsopoulos, Nuovo Cimento B 82, 169 (1984).
  * Beretta  _et al._ [1985] G. P. Beretta, E. P. Gyftopoulos, and J. L. Park, Nuovo Cimento B 87, 77 (1985).
  * Maddox [1985] J. Maddox, Nature 316, 11 (1985).
  * Beretta [2005] G. P. Beretta, Modern Physics Letters A 20, 977 (2005).
  * Beretta [2009] G. P. Beretta, Reports on Mathematical Physics 64, 139 (2009).
  * Beretta [2010] G. P. Beretta, Journal of Physics: Conference Series 237, 012004 (2010).
  * Cano-Andrade  _et al._ [2015] S. Cano-Andrade, G. P. Beretta, and M. R. von Spakovsky, Phys. Rev. A 91, 013848 (2015).
  * Von Spakovsky and Gemmer [2014] M. R. Von Spakovsky and J. Gemmer, Entropy 16, 3434 (2014).
  * Ray [2022] R. K. Ray, Phys. Rev. E 106, 024115 (2022).
  * Beretta [2014] G. P. Beretta, Physical Review E 90, 042113 (2014).
  * Beretta [2020] G. P. Beretta, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 378, 20190168 (2020).
  * Beretta [1986] G. P. Beretta, in  _Frontiers of Nonequilibrium Statistical Physics_, edited by G. T. Moore and M. O. Scully (Springer US, Boston, MA, 1986) pp. 205–212.
  * Simmons and Park [1981] R. F. Simmons and J. L. Park, Foundations of Physics 11, 297 (1981).
  * Park and Simmons [1983] J. L. Park and R. F. Simmons, The knots of quantum thermodynamics, in  _Essays in Honor of Wolfgang Yourgrau_, edited by A. van der Merwe (Springer US, Boston, MA, 1983) pp. 289–308.
  * Hatsopoulos and Beretta [2008] G. N. Hatsopoulos and G. P. Beretta, AIP Conference Proceedings 1033, 34 (2008).
  * Beretta [2006a] G. P. Beretta, Modern Physics Letters A 21, 2799 (2006a).
  * Beretta [1987] G. P. Beretta, in  _International Symposium on Stability_ (ONERA, Paris, France, 3-5 November 1987) pp. 87–104.
  * Beretta [2006b] G. P. Beretta, Phys. Rev. E 73, 026113 (2006b).
  * Wolfram Research Inc. [2022] Wolfram Research Inc., Mathematica Online (2022).