# Non-Trivial Query Sampling For Efficient Learning To Plan

Sagar Suhas Joshi1 Panagiotis Tsiotras2 1,2 Institute for Robotics and
Intelligent Machines, Georgia Institute of Technology, USA. Email:
<EMAIL_ADDRESS>
###### Abstract

In recent years, learning-based approaches have revolutionized motion
planning. The data generation process for these methods involves caching a
large number of high quality paths for different queries (start, goal pairs)
in various environments. Conventionally, a uniform random strategy is used for
sampling these queries. However, this leads to inclusion of “trivial paths” in
the dataset (e.g.,, straight line paths in case of length-optimal planning),
which can be solved efficiently if the planner has access to a steering
function. This work proposes a “non-trivial” query sampling procedure to add
more complex paths in the dataset. Numerical experiments show that a higher
success rate can be attained for neural planners trained on such a non-trivial
dataset.

##  I INTRODUCTION

Motion planning, a core problem in artificial intelligence and robotics, is
one of finding a collision free, low cost path connecting a start and goal
state in a search-space. Popular discrete-space planners such as A* [1] and
LPA* [2] conduct a prioritized search using heuristics and guarantee
resolution-optimal paths. On the other hand, single-query sampling-based
motion planning (SBMP) algorithms, such as RRT [3], solve this problem by
constructing a connectivity graph online using a set of probing samples. The
RRT algorithm is probabilistically complete, while it’s asymptotically optimal
variants such as RRT* [4], RRT# [5], BIT* [6], FMT* [7] converge to the
optimal solution almost surely, as the number to samples tends to infinity.
However, these algorithms may suffer from a slow convergence rate, especially
in higher dimensional settings. In order to address this issue, several
techniques that leverage heuristics and collision information have been
suggested to improve the performance of these planners. These include [8],
[9], [10], [11], [12] among others. The recently proposed Informed Set [13],
[14], [15] and Relevant Region [16], [17] family of algorithms utilize current
solution information and heuristics to focus the search onto a subset of the
search-space, while still maintaining the theoretical guarantees of asymptotic
optimality.

Although the above methods can improve the performance of motion planning
algorithms, they require handcrafted heuristics for efficacy. Also, these
methods do not leverage prior experience or data gathered from expert
demonstrations. Deep learning based approaches for motion planning address
these two limitations by generating a dataset of high quality paths in various
environments. In the offline phase, this dataset is used to train a deep
neural network (DNN) model to predict quantities of interest, such as cost-to-
go [18], next point along the optimal path [19], or a sampling distribution
[20]. The DNN model can then be used online to focus search and dramatically
increase the efficiency of planning algorithms.

Figure 1: Schematic for the proposed data generation method. Instead of
trivial queries that can be solved by a greedy connection, more non-trivial
queries are added. In addition, the data pruning procedure filters out the
trivial part $(\pi_{3},\textbf{x}_{\mathrm{g}})$ of the path.

The data generation process for DNN-based motion planning methods involves
sampling and solving a set of queries (start, goal pairs) in a given
environment. For many applications, the map/environment in which the robot
needs to operate may be fixed or given apriori. However, the query sampling
distribution can be modified in order to extract more informative paths
beneficial to the learning process. Conventionally, uniform random sampling is
used to generate the start and goal states. Many queries in this uniformly
sampled dataset can be solved by greedily connecting the start and goal state
using a steering function (if available). Such a steering function provides
the optimal path between any two states after relaxing the collision
constraint. This work proposes adding more “non-trivial” queries to the
dataset, which cannot be solved by a simple greedy connection. The efficiency
of the neural planners can be boosted by training deep models on this dataset
comprising of relatively more complex paths. This is demonstrated by creating
datasets with different degrees of non-triviality and benchmarking the
performance of the neural planner on various robotic planning tasks.

##  II RELATED WORK

Exciting progress has been made at the intersection of learning and motion
planning in recent years. To improve the performance of discrete space
planners, methods that leverage reinforcement learning [21] and imitation
learning [22], [23] have been proposed. Neural A* [24] presents a data-driven
approach that reformulates conventional A* as a differentiable module. Huh et
al [25] present a higher-order function network that can predict cost-to-go
values to guide the A* search. For sampling-based planning, techniques such as
[26] and [27] learn a local sampling strategy for global planning. Zhang et al
[28] present a deep learning based rejection sampling scheme for SBMP. Ichter
et al [20] train a conditional variational autoencoder (CVAE) model to learn
sampling distributions and generate samples along the solution paths. NEXT
[18] learns a local sampling policy and cost-to-go function using “meta self-
improving learning”. Kuo et al [29] input the planner state and local
environment information into a deep sequential model to guide search during
planning. Approaches such as [30], [31] focus on identifying critical or
bottleneck states in an environment to generate a sparse graph while planning.
While the above techniques may differ in terms of their model architectures or
outputs, they do not tackle the problem of improving the data-generation
process via query sampling for increasing the efficacy of neural planners.

1 NeuralPlanner  _(_
$\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}},\mathcal{X}_{\mathrm{obs}}$
_)_:

2 $\pi\leftarrow\\{\textbf{x}_{\mathrm{s}}\\}$;

3 for  _$i=1:N_{\mathrm{plan}}$_ do

4 if
_$\mathsf{steerTo}(\pi_{\mathrm{end}},\textbf{x}_{\mathrm{g}},\mathcal{X}_{\mathrm{obs}})$_
then

5 $\pi\leftarrow\pi\cup\\{\textbf{x}_{\mathrm{g}}\\}$;

6 $\mathsf{break}$;

7

8 else

9
$\textbf{x}_{\mathrm{new}}\leftarrow\mathsf{PNet}(\pi_{\mathrm{end}},\textbf{x}_{\mathrm{g}})$;

10 $\pi\leftarrow\pi\cup\\{\textbf{x}_{\mathrm{new}}\\}$;

11

12

13 if  _$\textbf{x}_{\mathrm{g}}\notin\pi$_ then

14 return $\emptyset$ ;

15

16 if  _$\mathsf{Feasible}(\pi)$_ then

17 return $\pi$;

18

19 else

20 return $\mathsf{Replan}(\pi)$;

21

22

23

Algorithm 1 Neural Planner

In [32], Huh et al extend their previous work to present a query sampling
technique for non-holonomic robots. However, this technique is purely based on
the dynamics of the robot, does not utilize obstacle information and is only
applicable for car-like robots. The recently proposed OracleNet [33] and
Motion Planning Networks (MPNet) algorithm [19] learn a deep model to
recursively predict the next state along the solution path, given a query. The
authors in [19] use “Active Continual Learning (ACL)” to improve the data-
efficiency of the training process. Similar to the DAGGER algorithm [34] the
ACL process involves training a MPNet deep model on a set of expert
demonstrations for a $N_{c}>0$ number of initial iterations. The ACL algorithm
then finds the cases where the MPNet planner fails and invokes the expert
planner only to solve them. The solutions generated by the expert planner are
stored in a replay buffer to be used during training. ACL also leverages the
“Gradient Episodic Memory (GEM)” technique [35] to alleviate the problem of
catastrophic forgetting during the learning process. While more data-
efficient, the ACL process can be tricky to implement and computationally more
expensive, as it involves interleaving the training process with running the
neural planner multiple times. The performance of ACL models can also be worse
than that of batch-offline models in many cases [19]. In contrast, this work
proposes a modified query sampling procedure for data-generation and trains
all models in a batch-offline manner.

##  III PROBLEM DEFINITION

###  III-A Path Planning Problem

Let $\mathcal{X}\subset\mathbb{R}^{d}$ denote the search-space for the
planning problem with dimension $d\geq 2$. Let
$\mathcal{X}_{\mathrm{obs}}\subset\mathcal{X}$ denote the obstacle space and
$\mathcal{X}_{\mathrm{free}}=\mathcal{X}\setminus\mathcal{X}_{\mathrm{obs}}$
denote the free space. Let
$\mathrm{c}_{\pi}(\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}})$ denote the
cost of moving from a point $\textbf{x}_{\mathrm{s}}\in\mathcal{X}$ to
$\textbf{x}_{\mathrm{g}}\in\mathcal{X}$ along a path $\pi$. This path $\pi$
can be represented as an ordered list of length $L\geq 2$,
$\pi=\\{\pi_{1},\pi_{2}\dots\pi_{L}\\}$. Let $\pi_{\mathrm{end}}$ denote the
last point on the path $\pi$. Then, the optimal path planning problem is one
of finding the minimum cost, feasible path $\pi^{*}$ connecting a start
$\textbf{x}_{\mathrm{s}}$ and goal $\textbf{x}_{\mathrm{g}}$ state.

$\displaystyle\pi^{*}=\arg\min_{\pi\in\Pi}$ $\displaystyle\
\mathrm{c}_{\pi}(\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}}),$ (1)
subject to: $\displaystyle\
\pi_{1}=\textbf{x}_{\mathrm{s}},\pi_{\mathrm{end}}=\textbf{x}_{\mathrm{g}}$
$\displaystyle\
\pi_{\mathrm{i}}\in\mathcal{X}_{\mathrm{free}},\leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ i=1,2\dots L.$

Classical planners discretize $\mathcal{X}_{\mathrm{free}}$ or build a
connectivity graph and then perform a search over this graph to solve the
above planning problem (1).

###  III-B Supervised Learning for Planning

Learning-based methods use the data gathered from successful plans to train
models in the offline phase. In the online phase, this learned model can be
used to solve (1) or assist the classical planners. The data generation
process involves creating an environment and sampling a set of
$K_{\mathrm{train}}>0$ queries or (start, goal) pairs in
$\mathcal{X}_{\mathrm{free}}$. Let
$\mathcal{Q}=\mathcal{X}_{\mathrm{free}}\times\mathcal{X}_{\mathrm{free}}$
denote the “query-space”. A classical planner is then used to solve the path
planning problem (1) for each of the $K_{\mathrm{train}}$ queries and obtain a
good quality solution. A quantity of interest to be learned as output (such as
cost-to-go, next state on the optimal path etc) is extracted from these
solution paths. The objective of the training process is to learn a function
$f_{\theta}$ (usually a deep neural network), on the dataset $\mathcal{D}$ by
minimizing an empirical loss with respect to weight parameters $\theta$

###  III-C Neural Planner

The MPNet procedure involves learning a planning network $\mathsf{PNet}$, that
predicts the next state on the optimal path, given a current state, a goal
state and environment information as the input. A typical neural planning
algorithm, in line with the one described in [19], is illustrated in Algorithm
1. Given a query $\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}}$, the path
$\pi$ to be returned is initialized with the start-state. At each iteration,
the neural planner attempts a greedy connection to the goal using the
$\mathsf{steerTo}$ function. For length-optimal or geometric planning case,
the $\mathsf{steerTo}$ connects any two points using a straight line. For car-
like robots, this steering function can use the Dubins curves for dynamically
feasible connections [36]. To probe the feasibility of this greedy connection,
the $\mathsf{steerTo}$ procedure discretizes the path and collision-checks the
points on it. If the greedy connection $\pi_{\mathrm{end}}$ to
$\textbf{x}_{\mathrm{g}}$ is valid, the goal-state is appended to the path and
the planning loop terminates. Else, the learned planning network
$\mathsf{PNet}$, is used to predict the next state on the optimal path. If the
path $\pi$ is infeasible, a neural replanning procedure is performed on this
coarse path in an attempt to repair it. Please see [19] for more details about
these procedures.

##  IV NON-TRIVIAL QUERY SAMPLING

1 $\mathcal{D}\leftarrow\emptyset$;

2
$\mathcal{X},\mathcal{X}_{\mathrm{obs}}\leftarrow\mathsf{createEnvironment}()$;

3 for  _$j=1:K_{\mathrm{train}}$_ do

4 $u_{\mathrm{rand}}\sim U[0,1]$;

5 if  _$u_{\mathrm{rand}} <p_{\mathrm{nt}}$_ then

6
$\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}}\leftarrow\mathsf{nonTrivialQuerySampling}(\mathcal{X}_{\mathrm{free}})$;

7

8 else

9
$\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}}\leftarrow\mathsf{uniformSampling}(\mathcal{X}_{\mathrm{free}})$;

10

11
$\pi\leftarrow\mathsf{solveQuery}(\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}},\mathcal{X}_{\mathrm{obs}})$;

12 $\mathcal{D}\leftarrow\mathsf{includeData}(\mathcal{D},\pi)$;

13

return $\mathcal{D}$

Algorithm 2 Data Generation Algorithm

###  IV-A Non-trivial Queries

Conventionally, uniform random sampling is used in the data generation process
to obtain a query
$\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}}\in\mathcal{Q}$. However, this
may result in the inclusion of “trivial” queries in the dataset, for which
$\mathsf{steerTo}(\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}})=\mathsf{True}$.
In case of such trivial queries, the neural planning Algorithm 1 terminates in
the first iteration after processing lines 4-6. Thus, a key observation is
that $\mathsf{steerTo}$ procedure in Algorithm 1, line 4 performs an implicit
classification of queries, so that only “non-trivial” queries are passed over
to the $\mathsf{PNet}$. Concretely, the set of non-trivial queries can be
defined as

$\mathcal{Q}_{\mathrm{nt}}=\\{\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}}\in\mathcal{Q}\
|\
\mathsf{steerTo}(\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}})=\mathsf{False}\\}.$
(2)

This motivates the proposed data generation Algorithm 2, which aims to
increase the number of non-trivial data samples in $\mathcal{D}$. After an
environment $\mathcal{X},\mathcal{X}_{\mathrm{obs}}$ is created, data is
generated by solving a total of $K_{\mathrm{train}}$ queries. With probability
$p_{\mathrm{nt}}$, the proposed $\mathsf{nonTrivialQuerySampling}$ procedure
is used to obtain
$\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}}\in\mathcal{Q}_{\mathrm{nt}}$.
Else, conventional $\mathsf{uniformSampling}$ returns a query in
$\mathcal{Q}$. A classical planner such as A* or BIT* then solves this query
and outputs a good quality solution path $\pi$. Samples from this path $\pi$
are appended to the dataset $\mathcal{D}$ with the proposed data inclusion
procedure $\mathsf{includeData}$.

###  IV-B Non-trivial Query Sampling

A rejection sampling algorithm to generate new queries in
$\mathcal{Q}_{\mathrm{nt}}$ is given in Algorithm 3. For $N_{\mathrm{nt}}$
number of attempts, uniform sampling is used to first generate a valid query
$\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}}\in\mathcal{Q}$. The
$\mathsf{steerTo}$ module then validates the connection between start and goal
state. If found invalid, the corresponding non-trivial query is returned.
Thus, this procedure intends to filter out trivial paths while maintaining the
exploratory/coverage property of uniform sampling. Please see Fig. 2 and Fig.
4 for a visualization of queries generated using the proposed non-trivial
sampling procedure.

###  IV-C Data Inclusion

The data inclusion Algorithm 4 iterates over the segments of path $\pi$ and
logs the current state, goal state $(\pi_{i},\pi_{\mathrm{end}})$ as the input
and the next state $(\pi_{i+1})$ as the output/label. However, if the flag
$\mathsf{pruneData}=\mathsf{True}$, the algorithm skips including the data-
sample $\\{(\pi_{i},\pi_{\mathrm{end}}),\pi_{i+1}\\}$ if
$\pi_{i},\pi_{\mathrm{end}}\not\in\mathcal{Q}_{\mathrm{nt}}$. Thus,
$\mathsf{pruneData}=\mathsf{True}$ ensures that only the non-trival segments
of $\pi$ are incorporated in $\mathcal{D}$. Please see Fig. 1 for an
illustration of this step.

Depending on the topology of $\mathcal{X}_{\mathrm{obs}}$, the neural planner
may find it relatively harder to predict feasible paths in certain
environments. The notion of non-trivial queries can be used to define a metric
that captures this level of difficulty. Consider a “non-triviality ratio”,
which can be defined as,

$\gamma_{\mathrm{nt}}=\frac{\text{\\# Non-trivial queries}}{\text{\\#
Uniformly sampled queries}}.$ (3)

Thus, $\gamma_{\mathrm{nt}}$ is the ratio of number of non-trivial queries
found in a (large enough) set of uniformly sampled queries. This ratio will be
high for complex, cluttered and narrow-passage type environments and low for
relatively simpler, single-obstacle type environments.

1 nonTrivialQuerySampling  _(_ $\mathcal{X}_{\mathrm{obs}}$ _)_:

2 for  _$i=1:N_{\mathrm{max}}$_ do

3
$(\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}})\leftarrow\mathsf{uniformSampling}(\mathcal{X}_{\mathrm{free}})$;

4
$\mathsf{valid}\leftarrow\mathsf{steerTo}(\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}})$;

5 if  _$\mathsf{not\leavevmode\nobreak\ valid}$_ then

6 return $(\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}})$;

7

8

9

10return $(\textbf{x}_{\mathrm{s}},\textbf{x}_{\mathrm{g}})$;

Algorithm 3 Non-trivial Query Sampling

1 includeData  _(_ $\mathcal{D},\pi$ _)_:

2 for  _$i=1:L-1$_ do

3 if  _$\mathsf{pruneData}$_ then

4 $\mathsf{valid}\leftarrow\mathsf{steerTo}(\pi_{i},\pi_{\mathrm{end}})$;

5 if  _$\mathsf{valid}$_ then

6 $\mathsf{continue}$;

7

8

9
$\mathcal{D}\leftarrow\mathcal{D}\cup\\{(\pi_{i},\pi_{\mathrm{end}}),\pi_{i+1}\\}$

10

11return $\mathcal{D}$;

Algorithm 4 Data Inclusion Procedure

##  V NUMERICAL EXPERIMENTS

In order to benchmark the proposed data generation algorithm, the following
procedure was implemented for each planning environment. First, four datasets
with different parameter settings were created. These were as follows:
$\mathcal{D}_{0}\leavevmode\nobreak\ (p_{\mathrm{nt}}=0,\leavevmode\nobreak\
\mathsf{pruneData}=\mathsf{False})$, $\mathcal{D}_{1}\leavevmode\nobreak\
(p_{\mathrm{nt}}=0.5,\leavevmode\nobreak\ \mathsf{pruneData}=\mathsf{False})$,
$\mathcal{D}_{2}\leavevmode\nobreak\ (p_{\mathrm{nt}}=1.0,\leavevmode\nobreak\
\mathsf{pruneData}=\mathsf{False})$, $\mathcal{D}_{3}\leavevmode\nobreak\
(p_{\mathrm{nt}}=1.0,\leavevmode\nobreak\ \mathsf{pruneData}=\mathsf{True})$.
Thus, $\mathcal{D}_{0}$ represents the dataset generated using the
conventional uniform query sampling, whereas $\mathcal{D}_{3}$ is created
using the proposed non-trivial query sampling and data pruning procedure. Four
deep models,
$\mathsf{PNet}_{0},\mathsf{PNet}_{1},\mathsf{PNet}_{2},\mathsf{PNet}_{3}$ were
then trained on their respective datasets. The neural network shape, size and
the training parameters were held constant while learning all four models.
Performance of the neural planner 1 using these four models was evaluated on
1) $K_{\mathrm{test}}$ number of new uniform queries and 2)
$K_{\mathrm{test}}$ number of new non-trivial queries. Two performance
metrics, namely, success ratio and cost ratio were considered. Success ratio
gives the number of times (out of $K_{\mathrm{test}}$ in total) the neural
planner was successful in finding a feasible (collision-free) solution. Cost
ratio denotes the ratio of the neural planner’s solution cost to that of
classical planner, averaged over $K_{\mathrm{test}}$ trials. Model training
and evaluation was performed using the Python PyTorch API on a 64 bit, 16 GB
RAM laptop with Intel i7 processor and a NVIDIA GeForce RTX 2060 GPU. A
description of robotic planning tasks along with a discussion of results is
given below.

Figure 2: Four different environments for the point robot planning task. Data
generated using the proposed non-trivial query sampling results in following
(start, goal) distribution. Some of the length-optimal paths solved using a
classical planner are illustrated in magenta

Point Robot: Four different $20\times 20$ environments, illustrated in Fig. 2,
were considered for the case of point robot planning. Four datasets
$\\{\mathcal{D}_{i}\\}^{3}_{i=0}$, as described above, were generated for each
environment. A total of $K_{\mathrm{train}}=3000$ number of queries were
sampled for each dataset. An A* planner, followed by post-
processing/smoothening, was used to solve these queries and obtain length-
optimal paths. A small padding of $0.8$ units around the obstacles was
propagated during the data generation step, and was relaxed during the final
performance evaluation step. This was found to greatly boost the success ratio
of the neural planner, while making slight compromise in the cost ratio
metric. The performance metrics were logged by solving $K_{\mathrm{test}}=500$
number of unseen uniform and non-trivial queries with the four learned
$\mathsf{PNet}$ models.

Rigid Body Planning: Fig. 3 shows the instance of planning for a rigid robot
in four $10\times 10$ environments. A total of $K_{\mathrm{train}}=5000$
queries were considered to create each of the four datasets
$\\{\mathcal{D}_{i}\\}^{3}_{i=0}$. All the queries were solved in the
$\mathrm{SE}(2)$ space using OMPL’s [37] implementation of the BIT* planner.
An obstacle-padding of $0.4$ units was propagated during the data-generation
phase. The learned $\mathsf{PNet}$ models predicted a three dimensional
$[x,y,\theta]$ vector representing the robot’s pose. These models were
evaluated on $K_{\mathrm{test}}=500$ unseen uniform and non-trivial queries.
The BIT* planner was allowed a run-time of $3$ seconds during the data-
generation phase and $2$ seconds during the evaluation phase.

Figure 3: Four different environments for the rigid body planning task.

$n$-link Manipulator Planning: To observe performance of the neural planner in
higher dimensions, a planning problem for $2,4$ and $6$-link manipulator robot
was considered. Please see Fig. 4. The joint angles were constrained to lie
between $-\pi$ and $\pi$. Four datasets $\\{\mathcal{D}_{i}\\}^{3}_{i=0}$ were
created for each of the $2,4$ and $6$-link case by considering a total of
$3000,4000$ and $5000$ queries respectively. These queries were solved using
OMPL’s BIT* planner with an padding of $0.8$ units around the workspace
obstacles. The final performance evaluation was done by solving
$K_{\mathrm{test}}=500$ new queries with the neural planner. The BIT* planner
was run for $2,4,6$ seconds during the data-generation stage and $1,2,4$
seconds during the evaluation stage for the $2,4$ and $6$-link planning
respectively.

Figure 4: A schematic for $n$-link manipulator planning. Left figure shows the
obstacles and solution path in the work-space of the robot. The corresponding
obstacles and solution path in the configuration-space (middle). The (start,
goal) distribution produced by the proposed non-trivial query sampling in the
configuration-space (right). Table I: Point robot planning

Environment $0$, $\gamma_{\mathrm{nt}}=0.311$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.970$ $1.004$ $0.894$ $1.006$
$\mathsf{PNet}_{1}$ $0.980$ $1.002$ $0.954$ $1.007$ $\mathsf{PNet}_{2}$
$0.974$ $1.003$ $0.918$ $1.010$ $\mathsf{PNet}_{3}$ $0.980$ $1.002$ $0.930$
$1.007$
Environment $1$, $\gamma_{\mathrm{nt}}=0.402$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.992$ $1.031$ $0.986$ $1.075$
$\mathsf{PNet}_{1}$ $0.976$ $1.029$ $0.940$ $1.064$ $\mathsf{PNet}_{2}$
$0.986$ $1.027$ $0.966$ $1.063$ $\mathsf{PNet}_{3}$ $0.992$ $1.029$ $0.988$
$1.067$
Environment $2$, $\gamma_{\mathrm{nt}}=0.414$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.966$ $1.051$ $0.880$ $1.117$
$\mathsf{PNet}_{1}$ $0.960$ $1.051$ $0.906$ $1.125$ $\mathsf{PNet}_{2}$
$0.950$ $1.044$ $0.916$ $1.120$ $\mathsf{PNet}_{3}$ $0.962$ $1.056$ $0.900$
$1.113$
Environment $3$, $\gamma_{\mathrm{nt}}=0.628$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.944$ $1.056$ $0.864$ $1.156$
$\mathsf{PNet}_{1}$ $0.958$ $1.069$ $0.896$ $1.167$ $\mathsf{PNet}_{2}$
$0.956$ $1.063$ $0.898$ $1.163$ $\mathsf{PNet}_{3}$ $0.974$ $1.095$ $0.912$
$1.191$

Table II: Rigid body planning

Environment $0$, $\gamma_{\mathrm{nt}}=0.503$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.828$ $0.985$ $0.656$ $0.972$
$\mathsf{PNet}_{1}$ $0.842$ $0.985$ $0.622$ $0.972$ $\mathsf{PNet}_{2}$
$0.834$ $0.979$ $0.698$ $0.960$ $\mathsf{PNet}_{3}$ $0.856$ $0.978$ $0.784$
$0.958$
Environment $1$, $\gamma_{\mathrm{nt}}=0.659$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.788$ $0.948$ $0.714$ $0.874$
$\mathsf{PNet}_{1}$ $0.784$ $0.948$ $0.686$ $0.878$ $\mathsf{PNet}_{2}$
$0.792$ $0.971$ $0.638$ $0.889$ $\mathsf{PNet}_{3}$ $0.856$ $0.950$ $0.776$
$0.865$
Environment $2$, $\gamma_{\mathrm{nt}}=0.556$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.862$ $0.965$ $0.770$ $0.900$
$\mathsf{PNet}_{1}$ $0.898$ $0.963$ $0.776$ $0.907$ $\mathsf{PNet}_{2}$
$0.902$ $0.963$ $0.806$ $0.908$ $\mathsf{PNet}_{3}$ $0.922$ $0.970$ $0.876$
$0.912$
Environment $3$, $\gamma_{\mathrm{nt}}=0.690$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.722$ $0.985$ $0.606$ $1.000$
$\mathsf{PNet}_{1}$ $0.714$ $0.991$ $0.660$ $1.042$ $\mathsf{PNet}_{2}$
$0.756$ $0.994$ $0.650$ $0.996$ $\mathsf{PNet}_{3}$ $0.772$ $0.981$ $0.652$
$1.006$

General trends seen from the results in Table I, II, IV are as follows. In
most cases, $\mathsf{PNet}_{2}$ and $\mathsf{PNet}_{3}$ outperform
$\mathsf{PNet}_{0}$ in terms of success ratio, where as the performance of
$\mathsf{PNet}_{1}$ is more sporadic. The cost ratio is relatively lower for
the rigid body and $6$-link case compared to others, as the BIT* planner may
not find a good quality solution in the given planning time for these
challenging cases. The success ratio for all $\mathsf{PNet}$ models is
naturally higher over uniform test queries rather than non-trivial test
queries. The success ratio also generally has an inverse relation with
$\gamma_{\mathrm{nt}}$, as seen strongly in the case of rigid body and
$n$-link manipulator planning. For the case of point robot planning, all
models perform well with a success rate of over $90\%$ (see Table I). However,
slight performance gains due to the proposed method can be seen for
Environments $0,2,3$. These gains are much more noticeable for the rigid body
planning case (see Table II). The $\mathsf{PNet}_{3}$ model has the highest
success ratio in all cases except one (Environment $3$, Non-trivial Query),
where its performance is comparable to $\mathsf{PNet}_{1}$. The gradation in
performance due to dimensionality and $\gamma_{\mathrm{nt}}$ can be seen
clearly in the $n$-link planning case (see Table IV). The success ratio over
uniform queries is in the range of $0.9,0.8$ and $0.7$ for the case of $2,4$
and $6$-link planning case respectively. For the relatively simpler $2$-link
planning case with $\gamma_{\mathrm{nt}}=0.225$, only small gains in the
success ratio over non-trivial queries can be seen. However, the improvement
in performance is much more evident for the higher dimensional $4$ and
$6$-link cases. The $\mathsf{PNet}_{3}$ model shows about a $25\%$ increase in
the success ratio over $\mathsf{PNet}_{0}$ for the $6$-link (non-trivial
queries) case.

The neural planning Algorithm 1 and the corresponding results discussed above
assume the availability of a steering function. While this is readily
available for cases such as geometric or non-holonomic (car-like) planning
[36], it may not be computationally tractable for others. To analyze the
performance of $\mathsf{PNet}$ models without the $\mathsf{steerTo}$ function,
simulations were performed by only executing the lines $8$ and $9$ of the
neural planner 1 for maximum $N_{\mathrm{plan}}$ iterations. Instead of lines
4-6 in Algorithm 1, the following termination condition was implemented,
$\|\pi_{\mathrm{end}}-\textbf{x}_{\mathrm{g}}\|_{2}\leq\delta$, with a small
$\delta>0$. As illustrated in Fig. 5, the $\mathsf{PNet}_{3}$ model, which has
no trivial sample in its training dataset, naturally cannot solve a trivial
query. Numerical results for the rigid body planning without the
$\mathsf{steerTo}$ function and $\delta=1.0$ are tabulated in Table III. The
success ratio of all $\mathsf{PNet}$ models is adversely affected in this
case. The $\mathsf{PNet}_{0}$ model performs best on uniform queries in all
environments, whereas the performance of $\mathsf{PNet}_{3}$ is the worst.
However, $\mathsf{PNet}_{1}$ or $\mathsf{PNet}_{2}$ show better performance
over non-trivial queries in some cases. Thus, without a steering function, a
uniformly sampled training dataset might be the best choice if the test
queries are uniformly distributed too. However, a model trained over a dataset
with an appropriate value of $p_{\mathrm{nt}}$ may perform better over non-
trivial test queries. This makes a case for an ensemble model.

Table III: Rigid Body without $\mathsf{steerTo}$

Environment $0$, $\gamma_{\mathrm{nt}}=0.503$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.624$ $0.982$ $0.414$ $0.957$
$\mathsf{PNet}_{1}$ $0.592$ $0.983$ $0.372$ $0.979$ $\mathsf{PNet}_{2}$
$0.564$ $0.977$ $0.392$ $0.960$ $\mathsf{PNet}_{3}$ $0.228$ $0.985$ $0.212$
$0.978$
Environment $1$, $\gamma_{\mathrm{nt}}=0.659$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.498$ $0.971$ $0.380$ $0.936$
$\mathsf{PNet}_{1}$ $0.476$ $0.965$ $0.430$ $0.923$ $\mathsf{PNet}_{2}$
$0.456$ $0.974$ $0.372$ $0.946$ $\mathsf{PNet}_{3}$ $0.266$ $0.970$ $0.208$
$0.968$
Environment $2$, $\gamma_{\mathrm{nt}}=0.556$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.684$ $0.943$ $0.514$ $0.872$
$\mathsf{PNet}_{1}$ $0.684$ $0.944$ $0.506$ $0.889$ $\mathsf{PNet}_{2}$
$0.630$ $0.929$ $0.550$ $0.890$ $\mathsf{PNet}_{3}$ $0.284$ $0.937$ $0.270$
$0.931$
Environment $3$, $\gamma_{\mathrm{nt}}=0.690$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.514$ $0.986$ $0.374$ $0.990$
$\mathsf{PNet}_{1}$ $0.504$ $0.996$ $0.426$ $1.005$ $\mathsf{PNet}_{2}$
$0.502$ $1.004$ $0.396$ $1.004$ $\mathsf{PNet}_{3}$ $0.302$ $1.001$ $0.196$
$0.979$

Table IV: $n$-link manipulator planning

$2$-link planning, $\gamma_{\mathrm{nt}}=0.225$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.984$ $1.002$ $0.924$ $1.061$
$\mathsf{PNet}_{1}$ $0.990$ $1.004$ $0.956$ $1.060$ $\mathsf{PNet}_{2}$
$0.984$ $1.003$ $0.962$ $1.062$ $\mathsf{PNet}_{3}$ $0.988$ $1.004$ $0.972$
$1.068$
$4$-link planning, $\gamma_{\mathrm{nt}}=0.366$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.862$ $1.005$ $0.722$ $1.003$
$\mathsf{PNet}_{1}$ $0.876$ $0.997$ $0.718$ $0.991$ $\mathsf{PNet}_{2}$
$0.890$ $0.994$ $0.772$ $1.003$ $\mathsf{PNet}_{3}$ $0.928$ $0.860$ $0.82$
$0.992$
$6$-link planning, $\gamma_{\mathrm{nt}}=0.608$
Model Uniform Query Non-trivial Query success ratio cost ratio success ratio
cost ratio $\mathsf{PNet}_{0}$ $0.716$ $0.982$ $0.512$ $0.972$
$\mathsf{PNet}_{1}$ $0.650$ $0.980$ $0.454$ $0.966$ $\mathsf{PNet}_{2}$
$0.712$ $0.984$ $0.528$ $0.971$ $\mathsf{PNet}_{3}$ $0.776$ $0.975$ $0.644$
$0.958$

Figure 5: Solving a trivial query with the four learned models without the
$\mathsf{steerTo}$ function. Models
$\mathsf{PNet}_{0}$,$\mathsf{PNet}_{1}$,$\mathsf{PNet}_{2}$ can successfully
solve the query. However, the $\mathsf{PNet}_{3}$ model, which does not have
any trivial sample in its training dataset, is unable to solve it.

##  VI CONCLUSION

Many of the previous techniques in the literature have focused on exploring
different deep architectures for planning, while using a uniformly sampled
dataset for training. This work, on the other hand, investigates the problem
of improving the data-generation process while holding the model architecture
and planning algorithm constant. The proposed query sampling and data pruning
procedures add more complicated paths in the dataset. Numerical experiments
show that the success rate of the neural planner can be boosted using the deep
models trained on such non-trivial datasets.

This work presents many opportunities for future research. An ensemble model
can be constructed by combining predictions from different models trained on
datasets with varying degrees of non-triviality. Instead of a Boolean
$\mathsf{pruneData}$ flag, calling the pruning procedure with a probability of
$\gamma_{\mathrm{nt}}$ can be explored. This can prevent excessive pruning and
result in a drastic reduction in the size of the dataset for relatively less
cluttered environments.

Acknowledgements: Authors would like to thank Prof. Le Song, Binghong Chen and
Ethan Wang for insightful discussions on this topic.

## References

  * [1] P. E. Hart, N. J. Nilsson, and B. Raphael, “A formal basis for the heuristic determination of minimum cost paths,” _IEEE Transactions on Systems Science and Cybernetics_ , vol. 4, no. 2, pp. 100–107, 7 1968.
  * [2] S. Koenig, M. Likhachev, and D. Furcy, “Lifelong planning astar,” _Artificial Intelligence_ , vol. 155, no. 1-2, pp. 93–146, 2004.
  * [3] S. M. LaValle and J. J. Kuffner Jr, “Randomized kinodynamic planning,” _The International Journal of Robotics Research_ , vol. 20, no. 5, pp. 378–400, 2001.
  * [4] S. Karaman and E. Frazzoli, “Sampling-based algorithms for optimal motion planning,” _The International Journal of Robotics Research_ , vol. 30, no. 7, pp. 846–894, June 2011.
  * [5] O. Arslan and P. Tsiotras, “Use of relaxation methods in sampling-based algorithms for optimal motion planning,” in _IEEE International Conference on Robotics and Automation_ , Karlsruhe, Germany, May 6–10 2013, pp. 2421–2428.
  * [6] J. D. Gammell, S. S. Srinivasa, and T. D. Barfoot, “Batch informed trees (BIT*): Sampling-based optimal planning via the heuristically guided search of implicit random geometric graphs,” in _IEEE International Conference on Robotics and Automation_ , Seattle, WA, May, 25–30 2015, pp. 3067–3074.
  * [7] L. Janson, E. Schmerling, A. Clark, and M. Pavone, “Fast marching tree: A fast marching sampling-based method for optimal motion planning in many dimensions,” _The International Journal of Robotics Research_ , vol. 34, no. 7, pp. 883–921, May 2015.
  * [8] J. M. Phillips, N. Bedrossian, and L. E. Kavraki, “Guided expansive spaces trees: a search strategy for motion-and cost-constrained state spaces,” in _IEEE International Conference on Robotics and Automation_ , New Orleans, LA, April 26–30 2004, pp. 3968–3973.
  * [9] S. M. Persson and I. Sharf, “Sampling-based A* algorithm for robot path-planning,” _The International Journal of Robotics Research_ , vol. 33, no. 13, pp. 1683–1708, 10 2014.
  * [10] B. Akgun and M. Stilman, “Sampling heuristics for optimal motion planning in high dimensions,” in _IEEE/RSJ International Conference on Intelligent Robots and Systems_ , San Francisco, CA, Sept. 25–30 2011, pp. 2640–2645.
  * [11] S. Rodriguez, X. Tang, J.-M. Lien, and N. M. Amato, “An obstacle-based rapidly-exploring random tree,” in _IEEE International Conference on Robotics and Automation_ , Orlando, FL, May 15–19 2006, pp. 895–900.
  * [12] C. Urmson and R. Simmons, “Approaches for heuristically biasing RRT growth,” in _IEEE/RSJ International Conference on Intelligent Robots and Systems._ , vol. 2, Las Vegas, NV, Oct. 27–31 2003, pp. 1178–1183.
  * [13] J. D. Gammell, T. D. Barfoot, and S. S. Srinivasa, “Informed sampling for asymptotically optimal path planning,” _IEEE Transactions on Robotics_ , vol. 34, no. 4, pp. 966–984, Aug. 2018.
  * [14] S. S. Joshi, S. Hutchinson, and P. Tsiotras, “Time-informed exploration for robot motion planning,” 2020.
  * [15] A. Mandalika, R. Scalise, B. Hou, S. Choudhury, and S. S. Srinivasa, “Guided incremental local densification for accelerated sampling-based motion planning,” _arXiv preprint arXiv:2104.05037_ , 2021.
  * [16] S. S. Joshi and P. Tsiotras, “Relevant region exploration on general cost-maps for sampling-based motion planning,” in _International Conference on Intelligent Robots and Systems (IROS)_. Las Vegas, NV: IEEE/RSJ, Oct. 25–29 2020.
  * [17] O. Arslan and P. Tsiotras, “Dynamic programming guided exploration for sampling-based motion planning algorithms,” in _IEEE International Conference on Robotics and Automation_ , Seattle, WA, May 26–30 2015, pp. 4819–4826.
  * [18] B. Chen, B. Dai, Q. Lin, G. Ye, H. Liu, and L. Song, “Learning to plan in high dimensions via neural exploration-exploitation trees,” in _International Conference on Learning Representations_ , 2020. [Online]. Available: https://openreview.net/forum?id=rJgJDAVKvB
  * [19] A. H. Qureshi, Y. Miao, A. Simeonov, and M. C. Yip, “Motion planning networks: Bridging the gap between learning-based and classical motion planners,” _IEEE Transactions on Robotics_ , vol. 37, no. 1, pp. 48–66, 2020.
  * [20] B. Ichter, J. Harrison, and M. Pavone, “Learning sampling distributions for robot motion planning,” in _2018 IEEE International Conference on Robotics and Automation (ICRA)_. Brisbane, Australia: IEEE, May, 21–25 2018, pp. 7087–7094.
  * [21] M. Zucker, J. Kuffner, and J. A. Bagnell, “Adaptive workspace biasing for sampling-based planners,” in _2008 IEEE International Conference on Robotics and Automation_. IEEE, 2008, pp. 3757–3762.
  * [22] M. Bhardwaj, S. Choudhury, and S. Scherer, “Learning heuristic search via imitation,” in _Conference on Robot Learning_. PMLR, 2017, pp. 271–280.
  * [23] S. Choudhury, M. Bhardwaj, S. Arora, A. Kapoor, G. Ranade, S. Scherer, and D. Dey, “Data-driven planning via imitation learning,” _The International Journal of Robotics Research_ , vol. 37, no. 13-14, pp. 1632–1672, 2018.
  * [24] R. Yonetani, T. Taniai, M. Barekatain, M. Nishimura, and A. Kanezaki, “Path planning using neural a* search,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 12 029–12 039.
  * [25] J. Huh, G. Xing, Z. Wang, V. Isler, and D. D. Lee, “Learning to generate cost-to-go functions for efficient motion planning,” _arXiv preprint arXiv:2010.14597_ , 2020.
  * [26] C. Chamzas, A. Shrivastava, and L. E. Kavraki, “Using local experiences for global motion planning,” in _International Conference on Robotics and Automation (ICRA)_. IEEE, 2019, pp. 8606–8612.
  * [27] T. Lai, P. Morere, F. Ramos, and G. Francis, “Bayesian local sampling-based planning,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 1954–1961, 2020.
  * [28] C. Zhang, J. Huh, and D. D. Lee, “Learning implicit sampling distributions for motion planning,” in _International Conference on Intelligent Robots and Systems (IROS)_. IEEE/RSJ, 2018, pp. 3654–3661.
  * [29] Y.-L. Kuo, A. Barbu, and B. Katz, “Deep sequential models for sampling-based planning,” in _International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2018, pp. 6490–6497.
  * [30] B. Ichter, E. Schmerling, T.-W. E. Lee, and A. Faust, “Learned critical probabilistic roadmaps for robotic motion planning,” in _IEEE International Conference on Robotics and Automation (ICRA)_ , 2020, pp. 9535–9541.
  * [31] R. Kumar, A. Mandalika, S. Choudhury, and S. Srinivasa, “Lego: Leveraging experience in roadmap generation for sampling-based planning,” _International Conference on Intelligent Robots and Systems (IROS)_ , pp. 1488–1495, 2019.
  * [32] J. Huh, D. D. Lee, and V. Isler, “Learning continuous cost-to-go functions for non-holonomic systems,” _arXiv preprint arXiv:2103.11168_ , 2021.
  * [33] M. J. Bency, A. H. Qureshi, and M. C. Yip, “Neural path planning: Fixed time, near-optimal path generation via oracle imitation,” in _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2019, pp. 3965–3972.
  * [34] S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” in _Proceedings of the fourteenth international conference on artificial intelligence and statistics_. JMLR Workshop and Conference Proceedings, 2011, pp. 627–635.
  * [35] D. Lopez-Paz and M. Ranzato, “Gradient episodic memory for continual learning,” _Advances in neural information processing systems_ , vol. 30, pp. 6467–6476, 2017.
  * [36] J. J. Johnson, L. Li, F. Liu, A. H. Qureshi, and M. C. Yip, “Dynamically constrained motion planning networks for non-holonomic robots,” in _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2020, pp. 6937–6943.
  * [37] I. A. Sucan, M. Moll, and L. E. Kavraki, “The open motion planning library,” _IEEE Robotics & Automation Magazine_, vol. 19, no. 4, pp. 72–82, Dec. 2012\.


# Through the Theory of Mind’s Eye: Reading Minds with Multimodal Video Large
Language Models

Zhawnen Chen1, Tianchun Wang2, Yizhou Wang3, Michal Kosinski4, Xiang Zhang2,
Yun Fu3, Sheng Li1†
1School of Data Science, University of Virginia, 2The Pennsylvania State
University,
3Northeastern University, 4Stanford University
<EMAIL_ADDRESS>{tkw5356<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Can large multimodal models have a human-like ability for emotional and social
reasoning, and if so, how does it work? Recent research has discovered
emergent theory-of-mind (ToM) reasoning capabilities in large language models
(LLMs). LLMs can reason about people’s mental states by solving various text-
based ToM tasks that ask questions about the actors’ ToM (e.g., human belief,
desire, intention). However, human reasoning in the wild is often grounded in
dynamic scenes across time. Thus, we consider videos a new medium for
examining spatio-temporal ToM reasoning ability. Specifically, we ask explicit
probing questions about videos with abundant social and emotional reasoning
content. We develop a pipeline for multimodal LLM for ToM reasoning using
video and text. We also enable explicit ToM reasoning by retrieving key frames
for answering a ToM question, which reveals how multimodal LLMs reason about
ToM.

##  1 Introduction

Recent research has discovered emergent theory-of-mind (ToM) reasoning
abilities in text-based large language models (LLMs) such as ChatGPT-4
(Kosinski, 2024). For the first time, an artificial intelligence model seem to
be able to read minds. For example, John pranks Kelly by placing marbles
inside a chocolate box that he gives her for birthday. John expects her to be
surprised when she opens the box. Current research suggests that LLMs suppose
that Kelly should have a false belief that the box is filled with chocolates.
However, follow-up research has highlighted limitations to such emergent ToM
reasoning skills. For example, when a visual caveat is introduced where the
chocolate box is transparent, the model fails to infer the additional layer of
ToM reasoning and reaches the same conclusion as before, thereby failing the
task. We argue that such challenges can be better solved with additional
visual and temporal features. The goal of this project is to examine and
improve ToM understanding in dynamic scenes. Meanwhile, recent work by Shain
et al. (2022)

This project addresses the research challenge of improving theory-of-mind
(ToM) reasoning for video-based large language models (LLMs). Current
investigations of the ToM capabilities of LLMs have focused exclusively on
text-based tasks (Kosinski, 2024). However, we argue that the nature of ToM,
i.e., the reasoning of mental states over time, necessitates the introduction
of a temporal dimension and thus can be more meaningfully modeled by video-
based tasks. To this end, we propose a novel video frame retrieval task to
localize ToM understanding to video segments. We thus treat ToM as a multitask
problem that requires explicit temporal ToM reasoning in addition to global
question answering. To solve this new problem, we propose a new architecture,
Video Theory of Mind (VToM) to model the evolution of mental states over time.
We build our novel evolutionary mental state graph module with textual
information from video transcripts and video information from available video
captioning models. We use video and text features extracted by state-of-the-
art video LLM, Video-ChatGPT (Muhammad Maaz & Khan, 2023). We fine-tune the
LLMs on the ToM reasoning task. Figure 1 illustrates our proposed
architecture. We will evaluate our approach on a ToM video dataset, Social-IQ
2.0 (Wilf et al., 2023). We also evaluate our method on non-ToM video datasets
such as TVQA.

Figure 1: An overview of our proposed Video Theory-of-Mind (VToM) architecture
for solving Theory-of-Mind reasoning tasks in videos..

Our project aims to study and address the social and psychological effects of
large language models (LLMs) on human behavior, cognition, and emotion. For
this goal, our project will explore whether LLMs manifest an emergent
understanding of the Theory of Mind (ToM) in dynamic scenes. This study will
not only improve the ToM reasoning ability of multimodal LLMs. Overall, it
would provide insights into how humans interact with AI systems and may help
researchers identify how AI impacts human decision making, mood, and
engagement.

The expected contributions of this project are as follows.

  1. 1.

We design a frame retrieval task on video theory-of-mind (ToM) question
answering, aimed to facilitate the temporal reasoning ToM reasoning. We create
and release the corresponding video frame retrieval dataset, Theory of Mind
Localization (ToMLoc).

  2. 2.

We evaluate foundational video LLMs on their ability for video-based
multimodal ToM reasoning on our new TToM task, the Social-IQ 2.0 ToM video
question answering task (Wilf et al., 2023).

  3. 3.

We release the GPT-4V labels as an additional benchmark dataset for evaluating
the video theory of mind capabilities of foundational video large multimodal
models.

We organize this paper as follows. §2 discusses the theoretical background for
ToM and related works in video large multimodal modals. §3 introduces the
ToMLoc task and our pre-training process. §4 provides an overview of our
experiments and §5 analyzes the results of our experiments.

##  2 Background and Related Work

In this section, we first introduce the psychological background of theory-of-
mind (ToM) reasoning. We then contextualize our work in terms of a long line
of studies on computational ToM modeling. Because of our focus on videos as a
medium, we

###  2.1 Theory of Mind Reasoning

Human Theory-of-Mind Reasoning on Text Traditional Theory of Mind (ToM)
Reasoning tasks and assessments revolve around text. In addition to text, we
engage in ToM reasoning when we watch movies or television. In fact, we
sometimes do so explicitly when we discuss films or TV shows with one another,
such as discussing character intentions or the morality of certain actions in
dynamic scenes. Indeed, video-based ToM reasoning can serve as an important
developmental tool. Valkenburg et al. (1999) found that parents often use
videos as a form of instructive mediation (also known as active mediation),
where they watch a video and ask their children about the motives that
characters might have, or why certain actions are considered good or bad. The
children’s answers are accompanied by instructional feedback from the parents.
Video-based instructive mediation is especially useful, considering that
children can spend between 2.5 and 3 hours watching videos a day (Rideout &
Robb, 2019). In the context of developmental psychology, the intervention for
autism spectrum disorder (ASD) incorporates many ToM tasks in formal
assessments (Heavey et al., 2000; Dziobek et al., 2006), neuroimaging stimulus
materials (Jacoby et al., 2016), behavioral studies (Black & Barnes, 2015),
and interventions (Muller & Kane, 2017). A notable example is the “Movie Time
Social Learning” program (Vagin, 2012), which uses movie discussions to assist
children with autism in grasping social contexts and understanding others’
points of view.

###  2.2 Computational ToM Reasoning

Computational Theory-of-Mind Reasoning on Text Efforts on computational theory
of mind encompass a diverse range of datasets aimed at enhancing AI’s
understanding and prediction of human mental states and social interactions.
The Theory of Mind Task dataset provides brief text stories paired with ToM-
focused questions, facilitating the evaluation of AI’s ability to reason about
mental states (Nematzadeh et al., 2018). The Visual Beliefs dataset offers
short comic-style image sequences that challenge models to identify mistaken
beliefs, a key aspect of theory of mind (Eysenbach et al., 2016).
Additionally, the Motivations dataset includes images of individuals annotated
with probable motivations, allowing models to infer underlying intentions
(Vondrick et al., 2016).

Narrative descriptions of prevalent social scripts and cultural norms further
contribute to this domain by providing context-specific social knowledge (Li
et al., 2012). Social Stories (Sap et al., 2019) are specifically designed to
help individuals with autism improve their interpersonal skills via better
understanding and navigating social situations. The Story Commonsense and
GLUCOSE datasets contain stories annotated with commonsense information,
enabling models to grasp everyday logic and causality in narratives (Chen et
al., 2019; Mostafazadeh et al., 2020). Finally, the Neural Theory-of-Mind
(TOMI) dataset focuses on assessing the ability of large language models to
reason about mental states and social interactions, highlighting current
limitations and future directions for research (Sap et al., 2022).

These combined efforts are crucial for advancing the field of AI, making
machines better at understanding and predicting human behavior and
interactions.

Theory-of-Mind Reasoning with Large Language Models Kosinski (2024) examined
the performance of large language models (LLMs) on text-based false-belief
tasks and unexpected transfer tasks and found that the latest model,
ChatGPT-4, performs at a level comparable to that of seven-year-old children
on these tasks. The study suggests that LLMs’ performance on these tasks is
driven by a single factor, possibly an ability to detect false belief. The
high correlation between LLMs’ performance on both types of tasks suggests
that a single factor, such as the ability to detect false belief, is driving
their performance rather than separate task-specific abilities.

Computational Theory-of-Mind Reasoning on Videos Current research in theory-
of-mind reasoning typically involves training a supervised machine learning
model. The training data comprises (1) a video segment with related features
or annotations, (2) a question in natural language, sometimes accompanied by
multiple-choice answers, and (3) the accurate answer (s). The Social-IQ
dataset, for example, features video snippets from real-life YouTube
interactions like seminars and interviews, and poses multiple choice questions
centered on social perceptions, intentions, mental states, and attitudes Zadeh
et al. (2019). There are other data sets such as TVQA, which sources videos
from popular TV series (Lei et al., 2018); PororoQA, derived from a kids’
animated show (Kim et al., 2017); and MovieQA, which revolves around movie-
related questions (Tapaswi et al., 2016).

###  2.3 Video Large Multimodal Models

Recent advancements in video understanding have been marked by the development
of several notable models. VideoLLaMA (Zhang et al., 2023) is a large
multimodal model that has shown significant promise in comprehending video
content through an integrative approach. Similarly, VideoChat (Ask Anything)
(Li et al., 2023) leverages extensive video data to enable interactive
question-answering, enhancing the model’s ability to provide detailed and
context-aware responses. Another significant contribution is Video-ChatGPT
(Muhammad Maaz & Khan, 2023), which builds on the capabilities of ChatGPT by
incorporating video understanding, allowing it to generate more informed and
contextually relevant interactions. Additionally, MovieChat Li et al. (2024)
introduces a memory consolidation mechanism to handle ultra-long videos
efficiently, using both short- and long-term memory modules to reduce
redundancy and improve performance. Together, these models represent the
forefront of research in video understanding, each contributing unique
methodologies and capabilities to the field.

##  3 Methods

In this section, we introduce the ToM Localization (ToMLoc) task and a
pipeline to pre-train and evaluate baseline video LLMs models on ToMLoc.

###  3.1 The ToM Localization (ToMLoc) Task

Given a video $X_{V}$ and a question $X_{Q}$, the ToM Localization (ToMLoc)
task retrieves the most relevant input frame $X_{I}$ to the user text query.
Although both ToMLoc and the traditional video-text retrieval task learn the
function $R_{\phi}(X_{V},X_{Q})\rightarrow X_{I}$, ToMLoc requires a global
understanding of neurotypical ToM modeling and the ability to model the mental
states of each character in a video over time. This additional challenge posed
by temporal ToM reasoning stems from the base task, ToM-based video question
answering (QA). As illustrated in Example 1, ToM-rich questions need higher-
order mental state modeling for each character for the duration of the video.
The MASC video ToM assessment paused the ToM-rich video at 46 key moments as a
means of ablation Dziobek et al. (2006). ToMLoc can be seen as a temporally
ablative understanding of the ToM content in video.

###  3.2 Example Illustrating the ToM Video Frame Retrieval

We show an example of ToM video frame retrieval to draw a distinction from
that of traditional video frame retrieval.

###### Example 1

Consider the scenario from the TinySocial dataset where Harry Potter is at the
Weasley’s home. In this scene, the father begins to scold the boys, and the
task is to predict the boys’ reaction. The question posed is: "How did the
boys react when the father began to scold them?" The possible answers are: (A)
The boys looked understandingly at each other and looked down in pretended
embarrassment, (B) They got defensive, (C) They started crying and begging for
forgiveness, and (D) They jumped up from the table and ran outside. The
correct answer is (A).

To correctly answer the ToM-probing question in Example 1, a video LLM first
needs to understand the first-order actions and attributes such the boys’
looking at each other and looking down. It is also necessary to recognize
facial expressions such as laughing and body language such as looking down
indicating embarrassment. Furthermore, a seemingly trivial question belies a
challenging two-hop ToM reasoning task. To arrive at "understandingly" and
"pretended embarrassment," a model would need to understand the mental states
of both Harry Potter and Ron Weaseley. In turn, the two boys had themselves
understood the mental states of Arthur Weaseley, who is pretending to scold
Ron for driving the flying car without permission. Without the key frame of
the boys witnessing Molly hitting Arthur on the arm and causing him to change
his question and demeanor, it would be impossible for humans and AI alike to
answer this question.

This example illustrates the necessity of the ToM localization task to being
able to answer ToM-rich questions about a video. In contrast, traditional
video retrieval is primarily focused on the alignment between video and text
modalities is not concerned with modeling human mental states.

To determine the most influential frame for answering this question, we apply
our proposed frame relevance scoring method. Let
$F=\\{f_{1},f_{2},\ldots,f_{n}\\}$ be the sequence of frames in the video, and
$Q=\\{q_{1}\\}$ be the set of ToM questions pertaining to the scene. The
relevance score function $R(f_{i},q_{1})$ is defined to measure the alignment
between the visual features in frame $f_{i}$ and the mental state question
$q_{1}$.

######  Definition 1 (Frame Relevance Score)

We define the relevance score function $R(f_{i},q_{1})$ as:

$R(f_{i},q_{1})=\text{SIM}(\phi(f_{i}),\psi(q_{1}))$

where $\phi(f_{i})$ represents the feature vector of frame $f_{i}$, and
$\psi(q_{1})$ represents the feature vector of the question $q_{1}$. The
similarity measure, $\text{SIM}(\cdot,\cdot)$, can be computed using cosine
similarity.

The subset of frames $F^{*}\subseteq F$ that maximizes the relevance score for
answering $q_{1}$ is given by:

$F^{*}=\arg\max_{F^{\prime}\subseteq F}\sum_{q_{1}\in Q}\max_{f_{i}\in
F^{\prime}}R(f_{i},q_{1})$

By optimizing this function, we identify the frames $F^{*}$ that are crucial
for understanding the mental states of the characters in the scene. This
allows us to determine the boys’ reaction based on the visual and contextual
cues present in the most relevant frames. The optimization process helps in
pinpointing these critical moments, providing a robust basis for answering the
ToM question accurately. The methodology demonstrates the effectiveness of our
frame selection approach in capturing essential social and emotional dynamics
in video-based ToM tasks.

###  3.3 ToMLoc Pre-training

In order to evaluate the spatio-temporal theory-of-mind understanding of video
MLLM models, we need to first pre-train a linear projector before a
foundational model to map a textual query to a specific segment within a
video. Given a video and a textual query, the objective is to retrieve the
frame most relevant to the query. The Theory of Mind localization task is
designed to identify specific segments within a video based on a textual
query. Essentially, given a video and a textual query, the aim is to determine
the start and end frames relevant to the question. For this task to be
effective, each query should ideally be associated with specific frame
annotations, marking the beginning and end of the relevant segment.

As illustrated in Figure 1, we first extract the visual features, $Z_{V}$,
from the input video frames $X_{V}$ using a visual encoder $g_{\phi}$. We use
a linear neural network, $P$, to project visual features $Z_{V}$ to large
multimodal model input tokens $H_{V}$. In terms of the textual query, we
simply use tokenizer $T$ to generate text tokens $H_{Q}=T(X_{Q})$. Finally, we
pass both the visually-project tokens $H_{V}$ and text tokens $H_{Q}$ to a
foundational video large multimodal model, $f$, (e.g., Video-ChatGPT (Muhammad
Maaz & Khan, 2023)) to output the answer $X_{I}$. In sum, our pipeline shown
in Figure 1 can be described by Equation 1.

$\begin{split}X_{I}&=f(H_{V}\oplus H_{Q})\\\ &=f(P(Z_{V})\oplus T(X_{Q}))\\\
&=f(P(g(X_{V})\oplus T(X_{Q}))\\\ \end{split}$ (1)

Figure 2: The Median Frame Approximation Task for Most Influential Frame
Retrieval

###  3.4 Approximating Most Influential Frames with Median Frame

However, a significant challenge arises with the Social-IQ dataset, which
lacks these vital timestamps. Owing to the absence of temporal annotations in
the Social-IQ dataset and budget constraints precluding the acquisition of
human labels, we propose an approximation task for most influential frame
retrieval using median frames. We combine multiple videos into a single
composite video, the entire duration of each individual video can serve as the
ground-truth timestamp. For example, a new video $V_{123}$, might comprise
clips from $V_{1}$ (e.g.,“Mrs. Doubtfire”), $V_{2}$ (“Big Bang Theory”), and
$V_{3}$ (“Harry Potter”). The task then evolves into localizing a query to a
specific video within this composite video. For example, a query regarding why
a man says “I used to” would be localized to the segment from “Mrs.
Doubtfire”.

##  4 Empirical Studies

###  4.1 Experimental Setup

We evaluated the Theory of Mind (ToM) reasoning capabilities of video-based
large language models (LLMs) through a series of experiments. The experiments
were designed to assess the models’ performance on frame localization and
question answering (QA) tasks using the Social-IQ 2.0 dataset. We compared
baseline models with our proposed Video Theory of Mind (VToM) pipeline.

##### Dataset for QA:

We used the Social-IQ 2.0 dataset for fine-tuning Video-ChatGPT. The dataset
consists of 1,403 video clips, with a total of 8,076 questions and 32,304
answers (4 answers per question, 3 incorrect, 1 correct). Prior to fine-
tuning, we performed preprocessing steps including normalization, resizing,
and frame extraction at a uniform sampling rate of 100 frames per video.

###  4.2 Frame Localization Task

The frame localization task involves identifying the most relevant frame in a
video for answering a given question. This task is crucial for ToM reasoning
as it requires understanding the context and the mental states of characters
in the video. The results are shown in Table 1.

Model | Trained/Finetuned on | Accuracy
---|---|---
Video-ChatGPT | ActivityNet | N/A%
Video-ChatGPT | ActivityNet/ToMLoc | 69.06%
Leave-Frame-Out | Video-ChatGPT | 77.97%
Table 1: Frame Localization Task Results

As observed, the baseline model Video-ChatGPT trained on ActivityNet alone
generated nonsensical responses, indicating its inability to perform the frame
localization task without specific ToM-related training. This is unsurprising.
Current VLMMs typically use ActivityNet and derivative datasets, which don’t .
However, fine-tuning the model on the ToMLoc dataset significantly improved
the accuracy to 69.06%, demonstrating the importance of ToM-specific data. Our
proposed Leave-Frame-Out method further improved the accuracy to 77.97%,
suggesting its effectiveness in enhancing ToM reasoning.

###  4.3 QA Performance on Social-IQ 2.0

The QA performance was evaluated to understand the models’ ability to answer
questions based on videos, which directly tests their ToM reasoning
capabilities. The results are shown in Table 2.

Model | Trained/Finetuned on | QA Accuracy
---|---|---
Random Guesses | N/A | 25.00%
Video-ChatGPT | ActivityNet/Social-IQ 2.0 QA | 39.61%
Video-ChatGPT | ActivityNet/ToMLoc | 47.15%
Table 2: QA Performance on Social-IQ 2.0

The baseline model with random guesses achieved 25.00% accuracy, which serves
as the lower bound. Video-ChatGPT trained on ActivityNet/Social-IQ 2.0 QA
improved the accuracy to 39.61%, indicating some level of ToM reasoning. Fine-
tuning on ToMLoc further increased the accuracy to 47.15%, confirming that
ToM-specific training enhances the models’ ToM reasoning abilities.

###  4.4 Semi-Supervised Labels

We also investigated the effectiveness of using semi-supervised labels
generated by GPT-4V for training our models. The results are shown in Table 3.

Model | Trained/Finetuned on | Accuracy (Strict)
---|---|---
Random Guesses | N/A | 1.00%
Video-ChatGPT | ActivityNet | N/A
Video-ChatGPT | ActivityNet/ToMLoc | 0.71%
Table 3: Can We Use Semi-Supervised Labels?

The results indicate that using semi-supervised labels generated by GPT-4V led
to a modest improvement in accuracy, achieving 0.71% compared to the baseline.
This suggests that while semi-supervised labels can provide some benefit,
high-quality human annotations are crucial for significant performance gains.

##  5 Discussions

The experimental results provide insights into the Theory of Mind (ToM)
capabilities of video-based large language models (LLMs). The significant
improvement in frame localization and QA accuracy after fine-tuning on ToM-
specific datasets (ToMLoc) highlights the importance of targeted training
data.

###  5.1 Fine-tuning and Model Performance

The fine-tuning process substantially improved the models’ ability to
understand and reason about complex actions and interactions within videos,
crucial for ToM reasoning. However, challenges such as handling occlusions and
varying lighting conditions remain. Future work should focus on augmenting the
dataset with more diverse samples and exploring alternative model
architectures to further enhance performance.

###  5.2 Limitations and Future Work

Our study highlights significant challenges in modeling and assessing
computational ToM reasoning. One major limitation is the scarcity of high-
quality, diverse video datasets, which constrains the models’ ability to
generalize across different contexts and scenarios. Additionally, the current
video datasets often lack high-quality human-annotated labels, making accurate
training and evaluation challenging. Addressing these limitations is crucial
for advancing the field of computational ToM reasoning. Future work should
focus on creating and curating richer datasets and exploring alternative model
architectures to improve performance and generalizability.

##  6 Discussions

##### Fine-tuning

The fine-tuning process significantly improved the model’s performance,
particularly in recognizing complex actions and interactions within the
videos. However, some challenges remained, such as handling occlusions and
varying lighting conditions. Future work will focus on augmenting the dataset
with more diverse samples and exploring alternative model architectures to
further enhance performance.

##### Limitations

Our work is an initial study on the hypothesis that computational theory-of-
mind reasoning can be effectively modeled and assessed. However, this area
poses significant challenges, mirroring the difficulties in assessing theory
of mind in humans. One major limitation is the scarcity of high-quality,
diverse video datasets essential for training these models. The availability
of such datasets is limited, which constrains the model’s ability to
generalize across different contexts and scenarios. Furthermore, the current
video datasets often lack high-quality human-annotated labels, making it
challenging to train and evaluate the models accurately. These limitations
highlight the need for more comprehensive and well-annotated datasets to
advance the field of computational theory-of-mind reasoning.

##  7 Conclusion

Our work uncovers emergent theory-of-mind (ToM) reasoning in large multimodal
models for video understanding. We introduced the Video Theory of Mind (VToM)
architecture to model the evolution of mental states over time. By integrating
textual and visual features from state-of-the-art video captioning models, our
approach aims to enhance the ToM reasoning capabilities of these models.
Through rigorous evaluation on datasets such as Social-IQ 2.0 and TVQA, we
demonstrated the potential of our method in capturing complex mental state
transitions within dynamic video contexts. However, our study also highlights
significant challenges, including the scarcity of high-quality, diverse video
datasets and the need for comprehensive human annotations. Addressing these
limitations is crucial for further advancements in computational ToM
reasoning. Future work should focus on creating and curating richer datasets
and exploring alternative model architectures to improve performance and
generalizability. In summary, this research provides a foundational step
towards developing AI systems capable of human-like ToM reasoning, with
implications for improving human-computer interactions and enhancing the
social intelligence of AI agents.

#### Acknowledgments

This research project has benefitted from the Microsoft Accelerate Foundation
Models Research (AFMR) grant program through which leading foundation models
hosted by Microsoft Azure along with access to Azure credits were provided to
conduct the research.

## References

  * Black & Barnes (2015) Jessica Black and Jennifer L Barnes.  Fiction and social cognition: The effect of viewing award-winning television dramas on theory of mind.  _Psychology of Aesthetics, Creativity, and the Arts_ , 9(4):423, 2015.
  * Chen et al. (2019) Jiaao Chen, Jianshu Chen, and Zhou Yu.  Incorporating structured commonsense knowledge in story completion.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 33(01):6244–6251, 2019.  doi: 10.1609/aaai.v33i01.33016244.  URL https://doi.org/10.1609/aaai.v33i01.33016244.
  * Dziobek et al. (2006) Isabel Dziobek, Alexandra Rogers, Jana Fleck, Axel Bahnemann, Kirsten Heekeren, Oliver T. Wolf, and Simon Baron-Cohen.  Introducing masc: A movie for the assessment of social cognition.  _Journal of Autism and Developmental Disorders_ , 36(5):623–636, 2006.  doi: 10.1007/s10803-006-0107-0.  URL https://link.springer.com/article/10.1007/s10803-006-0107-0.
  * Eysenbach et al. (2016) Benjamin Eysenbach, Carl Vondrick, and Antonio Torralba.  Who is mistaken?  _arXiv:1612.01175_ , 2016.
  * Heavey et al. (2000) Lisa Heavey, Wendy Phillips, Simon Baron-Cohen, and Michael Rutter.  The awkward moments test: A naturalistic measure of social understanding in autism.  _J. autism and developmental disorders_ , 30(3):225–236, 2000.
  * Jacoby et al. (2016) Nir Jacoby, Emile Bruneau, Jorie Koster-Hale, and Rebecca Saxe.  Localizing pain matrix and theory of mind networks with both verbal and non-verbal stimuli.  _Neuroimage_ , 126:39–48, 2016.
  * Kim et al. (2017) Kyung-Min Kim, Min-Oh Heo, Seong-Ho Choi, and Byoung-Tak Zhang.  Deepstory: Video story qa by deep embedded memory networks.  _arXiv:1707.00836_ , 2017.
  * Kosinski (2024) Michal Kosinski.  Evaluating large language models in theory of mind tasks, 2024.
  * Lei et al. (2018) Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L Berg.  Tvqa: Localized, compositional video question answering.  _arXiv:1809.01696_ , 2018.
  * Li et al. (2012) Boyang Li, Darren Scott Appling, Stephen Lee-Urban, and Mark Riedl.  Learning sociocultural knowledge via crowdsourced examples.  In _AAAI Workshop on Human Computation_ , 2012.
  * Li et al. (2023) Kunchang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao.  Videochat: Chat-centric video understanding.  _arXiv preprint arXiv:2305.06355_ , 2023.
  * Li et al. (2024) Kunchang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao.  Moviechat: From dense token to sparse memory for long video understanding.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2024.  URL https://arxiv.org/abs/2307.16449.
  * Mostafazadeh et al. (2020) Nasrin Mostafazadeh, Chris Brockett, Michel Galley, and Bill Dolan.  Glucose: Generalized and contextualized story explanations.  _arXiv preprint arXiv:2009.07758_ , 2020.  URL https://arxiv.org/abs/2009.07758.
  * Muhammad Maaz & Khan (2023) Salman Khan Muhammad Maaz, Hanoona Rasheed and Fahad Khan.  Video-chatgpt: Towards detailed video understanding via large vision and language models.  _ArXiv 2306.05424_ , 2023.
  * Muller & Kane (2017) Eve Muller and Steven R. Kane.  Using film study to teach perspective taking in high school students with autism and other social cognition challenges.  _J. Child and Adolescent Behavior_ , 5:1–8, 2017.
  * Nematzadeh et al. (2018) Aida Nematzadeh, Kaylee Burns, Erin Grant, Alison Gopnik, and Tom Griffiths.  Evaluating theory of mind in question answering.  In _EMNLP_ , pp. 2392–2400, 2018.
  * Rideout & Robb (2019) Victoria J Rideout and Michael B Robb.  _The common sense census: Media use by tweens and teens_.  Common Sense Media, 2019.
  * Sap et al. (2019) Maarten Sap, Ronan LeBras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Noah A. Smith, and Yejin Choi.  Social iqa: Commonsense reasoning about social interactions.  _arXiv preprint arXiv:1904.09728_ , 2019.  URL https://arxiv.org/abs/1904.09728.
  * Sap et al. (2022) Maarten Sap, Ronan Le Bras, Daniel Fried, and Yejin Choi.  Neural theory-of-mind? on the limits of social intelligence in large LMs.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pp. 3762–3780, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.  doi: 10.18653/v1/2022.emnlp-main.248.  URL https://aclanthology.org/2022.emnlp-main.248.
  * Shain et al. (2022) Cory Shain, Alexander Paunov, Xuanyi Chen, Benjamin Lipkin, and Evelina Fedorenko.  No evidence of theory of mind reasoning in the human language network.  _Cerebral Cortex_ , 33(10):6299–6319, 12 2022.  ISSN 1047-3211.  doi: 10.1093/cercor/bhac505.  URL https://doi.org/10.1093/cercor/bhac505.
  * Tapaswi et al. (2016) Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, and Sanja Fidler.  Movieqa: Understanding stories in movies through question-answering.  In _CVPR_ , pp. 4631–4640, 2016.
  * Vagin (2012) Anna Vagin.  _Movie time social learning: Using movies to teach social thinking and social understanding_.  Social Thinking Publishing, 2012.
  * Valkenburg et al. (1999) Patti M Valkenburg, Marina Krcmar, Allerd L Peeters, and Nies M Marseille.  Developing a scale to assess three styles of television mediation:“instructive mediation,”“restrictive mediation,” and “social coviewing”.  _J. broadcasting & electronic media_, 43(1):52–66, 1999.
  * Vondrick et al. (2016) Carl Vondrick, Deniz Oktay, Hamed Pirsiavash, and Antonio Torralba.  Predicting motivations of actions by leveraging text.  In _CVPR_ , pp. 2997–3005, 2016.
  * Wilf et al. (2023) Alex Wilf, Leena Mathur, Sheryl Mathew, Claire Ko, Youssouf Kebe, Paul Pu Liang, and Louis-Philippe Morency.  Social-iq 2.0 challenge: Benchmarking multimodal social understanding.  https://github.com/abwilf/Social-IQ-2.0-Challenge, 2023.
  * Zadeh et al. (2019) Amir Zadeh, Michael Chan, Paul Pu Liang, Edmund Tong, and Louis-Philippe Morency.  Social-iq: A question answering benchmark for artificial social intelligence.  In _CVPR_ , pp. 8807–8817, 2019.
  * Zhang et al. (2023) Hang Zhang, Xin Li, and Lidong Bing.  Video-llama: An instruction-tuned audio-visual language model for video understanding, 2023.