# LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe
Diffusion-based Planning

Zeyu Feng†1, Hao Luan†1, Pranav Goyal1, and Harold Soh1,2 †Equal
contribution.1All authors are with the Department of Computer Science, School
of Computing, National University of Singapore, Singapore. {zeyu, haoluan,
pgoyal<EMAIL_ADDRESS>S. is also with the Smart Systems
Institute, National University of Singapore.

###### Abstract

Operating effectively in complex environments while complying with specified
constraints is crucial for the safe and successful deployment of robots that
interact with and operate around people. In this work, we focus on generating
long-horizon trajectories that adhere to novel static and temporally-extended
constraints/instructions at test time. We propose a data-driven diffusion-
based framework, LTLDoG, that modifies the inference steps of the reverse
process given an instruction specified using finite linear temporal logic
($\textnormal{LTL}_{f}$). LTLDoG leverages a satisfaction value function on
$\textnormal{LTL}_{f}$ and guides the sampling steps using its gradient field.
This value function can also be trained to generalize to new instructions not
observed during training, enabling flexible test-time adaptability.
Experiments in robot navigation and manipulation illustrate that the method is
able to generate trajectories that satisfy formulae that specify obstacle
avoidance and visitation sequences.

##  I Introduction

Recent methodologies [1, 2, 3] utilizing data-driven diffusion models [4, 5,
6] have shown remarkable performance in generating robot behaviors across a
wide range of tasks. Thanks to their ability to model complex distributions,
these methods have surpassed several leading offline reinforcement learning
techniques and classical model-based trajectory optimization methods,
especially in long-horizon decision-making tasks [1, 3]. However, while
conventional diffusion models excel at learning from training datasets, they
lack the ability to adapt to new objectives or comply with new constraints
during deployment. This shortcoming can lead to unsafe behaviors, posing risks
to humans, robots, and their surrounding environment.

In view of this limitation, there has been very recent work on diffusing
_safe_ trajectories. Xiao et al. [7] integrated a dynamics model into the
denoising diffusion process and incorporated a class of Control Barrier
Functions (CBF) to meet safety criteria. Botteghi et al. [8] approached the
issue by embedding both safety and reward considerations into a constrained
optimization framework, employing CBF constraints as labels for classifier
guidance. However, these approaches primarily address _static_ environmental
constraints. For example, while they can maneuver around obstacles on a
_local_ scale, they fail to comply with more complex temporally-extended
directives such as “ _avoid the kitchen until you are clean_ ”.

Figure 1: We present LTLDoG, a diffusion-based planning framework for
generating trajectories that comply with specified $\textnormal{LTL}_{f}$
formulae. In the example above, a robot dog is tasked to arrive at the goal
position (A), but first has to visit B and avoid obstacles (crosses).

In this work, we propose an alternative approach to flexible trajectory
planning with diffusion models, designed to satisfy both static safety
requirements and temporal constraints. The core idea is to plan with diffusion
models to satisfy finite linear temporal logic ($\textnormal{LTL}_{f}$)
formulae [9]. $\textnormal{LTL}_{f}$ offers the ability to define a broad
spectrum of instructions/constraints that might emerge during deployment. For
example, $\textnormal{LTL}_{f}$ can describe a visitation order of different
objects and locations. The use of propositional logic operators, such as not,
facilitates the delineation of safe regions within the state space.

We develop LTLDoG (LTL Diffusion-orienting Guidance, pronounced “Little Dog”),
a posterior-sampling based diffusion framework that accommodates finite LTL
formulae at test time. We present two variants of LTLDoG: our main method,
LTLDoG-S, can be applied to the generation of finite-length trajectory in
robot tasks where the labeling function for propositional events is
differentiable. LTLDoG-S employs a differentiable formula checker in
conditional sampling — specifically, we modify the reverse process to
condition upon the criteria that the final (predicted) trajectory satisfies a
given $\textnormal{LTL}_{f}$ formula. For when a differentiable labeling
function is unavailable, we propose LTLDoG-R, which uses a trained
$\textnormal{LTL}_{f}$ neural-symbolic evaluator for posterior sampling.
Notably, both variants do not require collecting expert demonstrations for
every potential $\textnormal{LTL}_{f}$ instruction. They retain the temporal
compositionality and local consistency properties associated with diffusion
models [1] — as long as the dataset contains a diverse set of paths, they can
potentially “stitch together” snippets of trajectories from the training data
to generate plans for unseen $\textnormal{LTL}_{f}$ formulae.

Experiments on two benchmark environments (long-horizon planning for
navigation and policy learning for manipulation) demonstrate that LTLDoG is
able to generate trajectories that satisfy feasible safety and temporal
constraints. We find that our methods possess the ability to re-plan
alternative paths at a high-level based on given instruction. Moreover, real
robot experiments show that the generated trajectories can be successfully
transferred to a quadruped robot. In summary, this paper makes three key
contributions:

  * •

A conditional trajectory sampling approach designed for $\textnormal{LTL}_{f}$
instructions that leverages pre-trained diffusion models;

  * •

A regressor-guidance neural network for diffusion that generalizes to novel
trajectories and $\textnormal{LTL}_{f}$ formulae;

  * •

Experimental results on benchmark problems and real world demonstrations that
validate the effectiveness of planning with safety and temporal constraints.

From a broader perspective, LTLDoG is the first method that fuses symbolic
model checking (using $\textnormal{LTL}_{f}$) with expressive diffusion-based
generative models. We hope our results lays the groundwork towards performant,
yet safer and more trustworthy robots.

##  II Preliminaries and Notation

In this work, our focus is to extend diffusion-based planning methods towards
generating trajectories that comply with specified $\textnormal{LTL}_{f}$
formulae. Here, we provide a concise introduction to diffusion methods in the
context of planning and finite linear temporal logic.

###  II-A Planning with Diffusion

Many tasks in planning, reinforcement learning, and imitation learning require
generating trajectories under some specific objective. Let $\mathcal{S}$ and
$\mathcal{A}$ denote the state and action space, respectively. We use
$\bm{\tau}=\left(\mathbf{s}_{0},\mathbf{a}_{0},\mathbf{s}_{1},\mathbf{a}_{1},\ldots,\mathbf{s}_{T},\mathbf{a}_{T}\right)$
to refer to a trajectory, where $T$ is the planning horizon. The environment
transitions to a new state $\mathbf{s}_{t+1}$ when an agent executes action
$\mathbf{a}_{t}$ at state $\mathbf{s}_{t}$. Let the abbreviation
$\mathcal{J}(\bm{\tau}|g)$ denote the objective value function conditioned on
a goal state where the trajectory must terminate at, for example, (discounted)
cumulative rewards in reinforcement learning, cumulative error of actions in
imitation learning, or cost for safety constraints.

Diffusion-based planning methods directly generate partial or entire
trajectories by using diffusion models pre-trained on a dataset of
trajectories. Let $p_{0}\left(\bm{\tau}^{0}\right)$ denote the distribution of
trajectories in dataset, where $\bm{\tau}^{0}$ represents a noiseless
trajectory. Given an $N$-step discrete approximation of forward diffusion
process $p\left(\bm{\tau}^{i}|\bm{\tau}^{i-1}\right)$ that slowly corrupts
data by adding prespecified noise, diffusion models learn an iterative
denoising procedure by approximating the score function
$\nabla_{\bm{\tau}^{i}}\log p_{i}\left(\bm{\tau}^{i}\right)$ using a step-
dependent neural network $s_{\theta}$ trained with denoising score matching
[10]:

$\theta^{*}=\operatorname*{arg\,min}_{\theta}\mathbb{E}_{i,\bm{\tau}^{i},\bm{\tau}^{0}}\left[\left\lVert
s_{\theta}\left(\bm{\tau}^{i},i\right)-\nabla_{\bm{\tau}^{i}}\log
p\left(\bm{\tau}^{i}|\bm{\tau}^{0}\right)\right\rVert^{2}\right],$ (1)

in which $i\sim\mathcal{U}\\{1,2,\ldots,N\\}$ is the diffusion timestep, and
$\bm{\tau}^{i}\sim p\left(\bm{\tau}^{i}|\bm{\tau}^{0}\right)$ is the
trajectory $\bm{\tau}^{0}$ corrupted with noise. Throughout the paper, we
adopt Denoising Diffusion Probabilistic Models (DDPM) [5] as the sampling
method, where
$p\left(\bm{\tau}^{i}|\bm{\tau}^{0}\right)=\mathcal{N}\left(\sqrt{\bar{\alpha}_{i}}\bm{\tau}^{0},\left(1-\bar{\alpha}_{i}\right)\bm{I}\right)$,
$\alpha_{i}\mathrel{\mathop{:}}=1-\beta_{i}$, and
$\bar{\alpha}_{i}\mathrel{\mathop{:}}=\prod_{j=1}^{i}\alpha_{i}$ following
[5].

###  II-B Linear Temporal Logic (LTL)

Given a finite set of propositional symbols $\mathcal{P}$, the formula set
$\Psi$ of $\textnormal{LTL}_{f}$ contains formulas recursively defined in
Backus-Naur form as follows [11, 12]:

$\varphi\mathrel{\mathop{:}}=p\;|\;\neg\varphi\;|\;\varphi\wedge\psi\;|\;\bigcirc\varphi\;|\;\varphi\operatorname{\mathsf{U}}{}\psi,$

where $p\in\mathcal{P}$ and $\varphi,\psi\in\Psi$. Intuitively, the formula
$\bigcirc\varphi$ (next $\varphi$) is satisfied if $\varphi$ is satisfied at
the next time step. $\varphi\operatorname{\mathsf{U}}{}\psi$ ($\varphi$ until
$\psi$) is satisfied if $\varphi$ is satisfied until $\psi$ is satisfied, and
$\psi$ is satisfied by the end of the sequence. From these, other commonly
used logical connectives and temporal operators can be defined according to
the following equivalences:
$\varphi\vee\psi=\neg\left(\neg\varphi\wedge\neg\psi\right)$,
$\Diamond\varphi=\mathsf{true}{}\operatorname{\mathsf{U}}{}\varphi$
(eventually $\varphi$) and $\Box\varphi=\neg\Diamond\left(\neg\varphi\right)$
(always $\varphi$). The symbols $\mathsf{true}{}$ and $\mathsf{false}{}$ can
also be in the formula set defined by $\mathsf{true}{}=\varphi\vee\neg\varphi$
and $\mathsf{false}{}=\neg\mathsf{true}{}$.

In contrast to propositional logic, these formulas are evaluated over finite
sequences of observations
$\bm{\sigma}=\langle\sigma_{0},\sigma_{1},\sigma_{2},\ldots\sigma_{T}\rangle$
(i.e.,  _truth assignments_ to the propositional symbols in $\mathcal{P}$),
where $\sigma_{t}\in\\{0,1\\}^{|\mathcal{P}\rvert}$ and $\sigma_{t,p}=1$ iff
proposition $p\in\mathcal{P}$ is satisfied at time step $t$. $\mathsf{true}{}$
($\mathsf{false}{}$) is always satisfied (not satisfied) by any assignment.
Formally, $\bm{\sigma}$ _satisfies_ $\varphi$ at time $t\geq 0$, denoted by
$\langle{\bm{\sigma},t}\rangle\models\varphi$, as follows:

  * •

$\langle{\bm{\sigma},t}\rangle\models p$ iff $\sigma_{t,p}=1$, where
$p\in\mathcal{P}$

  * •

$\langle{\bm{\sigma},t}\rangle\models\neg\varphi$ iff
$\langle{\bm{\sigma},t}\rangle\not\models\varphi$

  * •

$\langle{\bm{\sigma},t}\rangle\models(\varphi\wedge\psi)$ iff
$\langle{\bm{\sigma},t}\rangle\models\varphi$ and
$\langle{\bm{\sigma},t}\rangle\models\psi$

  * •

$\langle{\bm{\sigma},t}\rangle\models\bigcirc\varphi$ iff
$\langle{\bm{\sigma},t+1}\rangle\models\varphi$

  * •

$\langle{\bm{\sigma},t}\rangle\models\varphi\operatorname{\mathsf{U}}{}\psi$
iff $\exists t_{2}\in[t,T]$ s.t.
$\langle{\bm{\sigma},t_{2}}\rangle\models\psi$ and $\forall
t_{1}\in[t,t_{2})$, $\langle{\bm{\sigma},t_{1}}\rangle\models\varphi$

A sequence $\bm{\sigma}$ is then said to _satisfy_ $\varphi$, i.e.,
$\bm{\sigma}\models\varphi$, iff
$\langle{\bm{\sigma},0}\rangle\models\varphi$.

##  III Method

In this section, we describe our primary contribution, LTLDoG, a diffusion-
based framework for generating trajectories that satisfy
$\textnormal{LTL}_{f}$ formulae. We first discuss how to conditionally sample
using diffusion models, followed by how $\textnormal{LTL}_{f}$ formulae can be
used to guide the diffusion process.

###  III-A Conditional Sampling in Diffusion Models

Given a trained score function from (1) such that
$s_{\theta^{*}}\approx\nabla_{\bm{\tau}^{i}}\log
p_{i}\left(\bm{\tau}^{i}\right)$, a diffusion model denoises examples
according to the distribution
$p_{\theta^{*}}\left(\bm{\tau}^{i-1}|\bm{\tau}^{i}\right)=\mathcal{N}\left(\bm{\tau}^{i-1};\frac{1}{\sqrt{\alpha_{i}}}\left(\bm{\tau}^{i}+\left(1-\alpha_{i}\right)s_{\theta^{*}}\left(\bm{\tau}^{i},i\right),\left(1-\alpha_{i}\right)\bm{I}\right)\right)$
starting from a Gaussian prior
$\bm{\tau}^{N}\sim\mathcal{N}\left(\bm{0},\bm{I}\right)$. For example,
Diffuser [1] samples a trajectory $\bm{\tau}^{0}$ from a diffusion model,
which an agent then executes. However, this original sampling process is
unable to control detailed properties of generated context.

Here, we are interested in sampling trajectories that satisfy both the final
goal and the specified instructions encoded as an $\textnormal{LTL}_{f}$
formula $\varphi$ that is provided during deployment. In other words, we aim
to sample trajectories under an objective function
$\mathcal{J}_{\varphi}\left(\bm{\tau}^{0}|g\right)$. For example,
$\mathcal{J}$ can have a high value if the events induced by $\bm{\tau}^{0}$
satisfy $\varphi$ and have a low value otherwise.

Formally, given a set of atomic propositions $\mathcal{P}$, the assignments
for $\bm{\tau}$ are given by a labeling function
$L:\mathcal{S}\times\mathcal{A}\rightarrow 2^{\lvert\mathcal{P}\rvert}$, where
each timestep of $\bm{\tau}$ induces an assignment to the propositional
symbols in $\mathcal{P}$. For example, in navigation task, $\mathcal{P}$ can
represent multiple regions to avoid and $L$ is a function indicating whether
$(\mathbf{s}_{t},\mathbf{a}_{t})$ in $\bm{\tau}$ are inside these regions or
not. We will slightly abuse notation and write $\bm{\tau}\models\varphi$ to
indicate that $\bm{\tau}$’s assignments $\bm{\sigma}$ satisfy a
$\textnormal{LTL}_{f}$ formula.

We aim to sample from the posterior
$p_{0}\left(\bm{\tau}^{0}|\bm{\tau}^{0}\models\varphi,g\right)$ with the
diffusion model as the prior. In this work, the constraint of goal state
conditioning $g$ can be either achieved by inpainting (similarly on the start
state $\mathbf{s}_{0}$) or implicitly encoded in the dataset, which does not
require separate modeling in conditional sampling. Therefore, we mainly target
the posterior $p_{0}\left(\bm{\tau}^{0}|\bm{\tau}^{0}\models\varphi\right)$.
Let $y_{0}$ denote a binary random variable indicating the likelihood of
$\bm{\tau}^{0}$ satisfying $\varphi$. Hence, the denoising process requires a
score function conditioned on $y_{0}=1$ and by Bayes’ rule:
$\nabla_{\bm{\tau}^{i}}\log
p_{i}\left(\bm{\tau}^{i}|y_{0}=1\right)=\nabla_{\bm{\tau}^{i}}\log
p_{i}\left(\bm{\tau}^{i}\right)+\nabla_{\bm{\tau}^{i}}\log
p_{i}\left(y_{0}=1|\bm{\tau}^{i}\right)$. The first term on the right-hand
side has been learned by a neural network in diffusion. However, the latter
term requires an integration over all possible values of $\bm{\tau}^{0}$:
$p_{i}\left(y_{0}|\bm{\tau}^{i}\right)=\int
p\left(y_{0}|\bm{\tau}^{0}\right)p_{i}\left(\bm{\tau}^{0}|\bm{\tau}^{i}\right)\mathop{}\\!\mathrm{d}\bm{\tau}^{0}$.
We consider the plug-and-play conditional generation setting and approximating
this integration with sample estimation [13], e.g., point estimation with
$p_{i}\left(y_{0}|\bm{\tau}^{i}\right)\approx
p\left(y_{0}|\hat{\bm{\tau}}^{0}\right)$ where the noiseless trajectory
$\hat{\bm{\tau}}^{0}$ is estimated via Tweedie’s formula [14]
$\hat{\bm{\tau}}^{0}=\frac{1}{\sqrt{\bar{\alpha}_{i}}}\left(\bm{\tau}^{i}+\left(1-\bar{\alpha}_{i}\right)\nabla_{\bm{\tau}^{i}}\log
p_{i}\left(\bm{\tau}^{i}\right)\right)$. We model the likelihood term as
$p\left(y|\bm{\tau}\right)=Ze^{\mathds{1}\left[y=\mathds{1}\left[\bm{\tau}\models\varphi\right]\right]}$,
where $Z$ is a normalizing constant and $\mathds{1}\left[\cdot\right]$ is the
indicator function. Putting the above elements together, the conditional score
function can be computed as follows,

$\begin{split}\nabla_{\bm{\tau}^{i}}\log
p_{i}\left(y_{0}|\bm{\tau}^{i}\right)&\approx\nabla_{\bm{\tau}^{i}}\log
p\left(y_{0}|\hat{\bm{\tau}}^{0}\right)\\\
&=\nabla_{\bm{\tau}^{i}}\log\left(Ze^{\mathds{1}\left[y=\mathds{1}\left[\hat{\bm{\tau}}^{0}\models\varphi\right]\right]}\right)\\\
&=\nabla_{\bm{\tau}^{i}}\mathds{1}\left[y=\mathds{1}\left[\hat{\bm{\tau}}^{0}\models\varphi\right]\right].\end{split}$
(2)

Unfortunately, both the indicator function and the satisfaction evaluation
(performed by techniques like model checking with finite automata) are non-
differentiable — this prohibits application in the gradient-based sampling
process of diffusion models. We address this problem in the next subsection.

###  III-B Differentiable Evaluation of $\textnormal{LTL}_{f}$

Our key approach is to “soften” the satisfaction evaluation. To make the
evaluation differentiable, we modify our formula evaluator to output positive
real values if a trajectory satisfies the $\textnormal{LTL}_{f}$ formula, and
negative real values otherwise. Instead of using a binary labeling functions,
we assume real valued assignments of atomic propositions can be obtained
through computation on the generated trajectories. As a specific example,
consider a navigation task where a robot has to avoid obstacles; one can
determine the assignment for proposition $p$ at timestep $t$ using the
Euclidean distance between $\mathbf{s}_{t}$ and the centers of the region
$\mathbf{c}_{p}$ (assuming a circular shape). A positive value of
$\ell(p,t)=r_{p}-\lVert\mathbf{s}_{t}-\mathbf{c}_{p}\rVert_{2}$, where $r_{p}$
is the radius of the circle, indicates a true assignment. Consequently, the
labeling function for the entire trajectory
$L:(\mathcal{S}\times\mathcal{A})^{T+1}\rightarrow\mathbb{R}^{\lvert\mathcal{P}\rvert\times(T+1)}$
is differentiable and the assignments are $\bm{\sigma}=L(\bm{\tau})$.

A binary version of $\bm{\sigma}$ through a sign function
$\text{sgn}(\bm{\sigma})$ satisfies the definition of  _satisfy_ defined in
Section II-B, but the sign operation breaks differentiability. As such, we
employ a formula evaluator
$f_{t}\left(\varphi,\bm{\sigma}_{t:T}\right):\Psi\times\mathbb{R}^{\lvert\mathcal{P}\rvert\times(T-t+1)}\rightarrow\mathbb{R}$
to check satisfaction [15], with positive values implying
$\langle{\bm{\sigma},t}\rangle\models\varphi$, similar to signal temporal
logic [16]. As such, $f$ is differentiable with the evaluation process defined
as follows:

  * •

$f_{t}(\mathsf{true}{},\bm{\sigma})=+\infty$

  * •

$f_{t}(\mathsf{false}{},\bm{\sigma})=-\infty$

  * •

$f_{t}(p,\bm{\sigma})=\bm{\sigma}_{t,p}$

  * •

$f_{t}(\neg\varphi,\bm{\sigma})=-f_{t}(\varphi,\bm{\sigma})$

  * •

$f_{t}(\varphi\wedge\psi,\bm{\sigma})=\min^{\gamma}\left\\{f_{t}(\varphi,\bm{\sigma}),f_{t}(\psi,\bm{\sigma})\right\\}$

  * •

$f_{t}(\varphi\vee\psi,\bm{\sigma})=\max^{\gamma}\\{f_{t}(\varphi,\bm{\sigma}),f_{t}(\psi,\bm{\sigma})\\}$

  * •

$f_{t}(\bigcirc\varphi,\bm{\sigma})=f_{t+1}(\varphi,\bm{\sigma})$

  * •

$f_{t}(\Box\varphi,\bm{\sigma})=\min^{\gamma}\\{f_{t:T}(\varphi,\bm{\sigma})\\}$

  * •

$f_{t}(\Diamond\varphi,\bm{\sigma})=\max^{\gamma}\\{f_{t:T}(\varphi,\bm{\sigma})\\}$

  * •

$f_{t}(\varphi\operatorname{\mathsf{U}}{}\psi,\bm{\sigma})=\min^{\gamma}\\{f_{t:k}(\varphi,\bm{\sigma}),f_{t}\left(\Diamond\psi,\bm{\sigma}\right)\\}$,
where $k\geq t$ is the smallest integer s.t. $f_{k}(\psi,\bm{\sigma})>0$

Note that the $\min$ and $\max$ functions are likewise “soft” to maintain
differentiability. To reduce clutter, we have omitted the subscript $t$ when
$t=0$. With these operations, $f$ maintains _quantitative semantics_ , which
preserves the relative values between different $\bm{\sigma}$ such that
trajectories with larger margin to a satisfying assignment have larger values.

Given $L$ and $f$, we can obtain a differentiable score function by replacing
the likelihood term in (2) with
$p\left(y|\bm{\tau}\right)=Ze^{(2y-1)f\left(\varphi,L\left(\bm{\tau}\right)\right)}$.
The conditional score is then

$\begin{split}\nabla_{\bm{\tau}^{i}}\log
p_{i}\left(y_{0}=1|\bm{\tau}^{i}\right)&\approx\nabla_{\bm{\tau}^{i}}\log\left(Ze^{f\left(\varphi,L\left(\hat{\bm{\tau}}^{0}\right)\right)}\right)\\\
&=\nabla_{\bm{\tau}^{i}}f\left(\varphi,L\left(\hat{\bm{\tau}}^{0}\right)\right).\end{split}$
(3)

This approximation of the conditional score function can be directly used with
a pre-trained diffusion model to sample trajectories conditioned on an
$\textnormal{LTL}_{f}$ formula. We call this method LTLDoG-S, since it
performs the above posterior sampling in the reverse process (Algorithm 1).
The gradient ascent step is controlled by a stepsize
$\\{\zeta_{i}\\}_{i=1}^{N}$; in practice, the stepsize for each denoise step
can be adaptive such that $\bm{\tau}^{i-1}$ remains valid according to the
formula evaluator $f\left(\varphi,L\left(\hat{\bm{\tau}}^{0}\right)\right)$.

Algorithm 1 $\textnormal{LTL}_{f}$ Planning with Posterior Sampling

1:$\varphi$, $N$, $s_{\theta^{*}}$, $\\{\zeta_{i}\\}_{i=1}^{N}$

2:$\bm{\tau}^{N}\sim\mathcal{N}\left(\bm{0},\bm{I}\right)$

3:for $i=N-1$ to $0$ do

4: $\hat{s}\leftarrow s_{\theta^{*}}\left(\bm{\tau}^{i},i\right)$

5:
$\hat{\bm{\tau}}^{0}\leftarrow\frac{1}{\sqrt{\bar{\alpha}_{i}}}\left(\bm{\tau}^{i}+\left(1-\bar{\alpha}_{i}\right)\hat{s}\right)$

6: $\bm{\epsilon}\sim\mathcal{N}\left(\bm{0},\bm{I}\right)$

7:
$\bm{\tau}^{i-1}\leftarrow\frac{\sqrt{\alpha_{i}}\left(1-\bar{\alpha}_{i-1}\right)}{1-\bar{\alpha}_{i}}\bm{\tau}^{i}+\frac{\sqrt{\bar{\alpha}_{i-1}}(1-\alpha_{i})}{1-\bar{\alpha}_{i}}\hat{\bm{\tau}}^{0}+\sqrt{1-\alpha_{i}}\bm{\epsilon}$

8:
$\bm{\tau}^{i-1}\leftarrow\bm{\tau}^{i-1}+{\zeta_{i}}\nabla_{\bm{\tau}^{i}}f\left(\varphi,L\left(\hat{\bm{\tau}}^{0}\right)\right)$

9:end for

10:return $\hat{\bm{\tau}}^{0}$

###  III-C Classifier Guidance over $\textnormal{LTL}_{f}$

One limitation of LTLDoG-S is that it necessitates a known (and
differentiable) expression for the formula evaluator
$f\left(\varphi,L\left(\hat{\bm{\tau}}^{0}\right)\right)$. This requirement
can be challenging to meet in scenarios where the truth assignments of
propositions are uncertain, for example, when the ground-truth physical
dynamics are unknown. Here, we circumvent this problem by employing classifier
guidance using a trained formula evaluator.

We propose a variant of LTLDoG with _regressor guidance_ , which we abbreviate
as LTLDoG-R. Using an $\ell_{2}$ loss, we train a neural network to predict
the satisfaction values from noisy trajectories conditioned on
$\textnormal{LTL}_{f}$ instructions. In other words, our neural network
approximates the conditional score function $\nabla_{\bm{\tau}^{i}}\log
p_{i}\left(y_{0}=1|\bm{\tau}^{i}\right)$. Note that in contrast to learning a
binary classifier for trajectory satisfaction, we apply the labeling function
in Section III-B on the dataset and associate with each trajectory its
objective value $\mathcal{J}_{\varphi}\left(\bm{\tau}^{0}\right)$. We conduct
ablation study in Section V-D to show that using real values performs better
than using binary labels.

To generalize over different $\textnormal{LTL}_{f}$, the neural network takes
both noisy trajectory and $\textnormal{LTL}_{f}$ formula embedding as input.
Multiple methods exist for embedding $\textnormal{LTL}_{f}$ formulae. For
instance, one can use Graph Neural Networks (GNNs) [17, 18] to embed the tree
representation of an $\textnormal{LTL}_{f}$ formula directly. Alternatively,
the deterministic finite-state automaton (DFA) [19, 20] associated with the
formulae can be embedded [21]. In this work we employ $\textnormal{LTL}_{f}$
tasks from [22] and embed the directed graph of an $\textnormal{LTL}_{f}$
formula using the Relational Graph Convolutional Network (R-GCN) [23], which
can generalize to $\textnormal{LTL}_{f}$ formulae with same template
structure. The model that approximates the score function
$s_{\phi^{*}}(\phi,\bm{\tau}^{i},i)\approx\nabla_{\bm{\tau}^{i}}\log
p_{i}\left(y_{0}|\bm{\tau}^{i}\right)$ after training can be plugged into the
conditional reverse process using regressor guidance as summarized in
Algorithm 2.

Algorithm 2 $\textnormal{LTL}_{f}$ Planning with Regressor Guidance

1:$\varphi$, $N$, $s_{\theta^{*}}$, $s_{\phi^{*}}$,
$\\{\zeta_{i}\\}_{i=1}^{N}$

2:$\bm{\tau}^{N}\sim\mathcal{N}\left(\bm{0},\bm{I}\right)$

3:for $i=N-1$ to $0$ do

4: $\bm{\epsilon}\sim\mathcal{N}\left(\bm{0},\bm{I}\right)$

5:
$\bm{\tau}^{i-1}\leftarrow\frac{1}{\sqrt{\alpha_{i}}}\left(\bm{\tau}^{i}+(1-\alpha_{i})s_{\theta^{*}}\left(\bm{\tau}^{i},i\right)\right)+\sqrt{1-\alpha_{i}}\bm{\epsilon}$

6:
$\bm{\tau}^{i-1}\leftarrow\bm{\tau}^{i-1}+\zeta_{i}s_{\phi^{*}}(\phi,\bm{\tau}^{i},i)$

7:end for

8:return $\hat{\bm{\tau}}^{0}$

##  IV Related Work

LTLDoG builds upon prior work in diffusion-based planning and symbolic
reasoning using $\textnormal{LTL}_{f}$ for robotics. In the following, we give
a brief overview of related work.

Learning and Planning under $\textnormal{LTL}_{f}$. As an expressive language
for specifying high-level planning requirements [24, 25, 26],
$\textnormal{LTL}_{f}$ has been extensively used in various robotic tasks to
express temporally extended goals [27, 28]. These methods usually require the
information about the environment’s dynamics, e.g., a model or an abstraction,
to effectively plan under a given formula. Reinforcement learning agent learn
in a model-free way under $\textnormal{LTL}_{f}$ objectives or constraints
[29, 30, 31] with the ability to generalize over different formulae [22].
However, these methods operate agent in an online manner via trial and error,
which can lead to expensive or even unsafe interactions.

Planning and Policy Learning with Diffusion. Recent diffusion-based planning
methods are flexible that only rely on offline datasets without access or
interaction to environments. They have been successfully applied to long-
horizon planning problems by generating states or actions for control [1, 3,
2], but not tasks with test-time temporal requirements. Recent work has looked
into safety critical tasks e.g., the aforementioned CBF-based methods [7, 8].
As discussed above, these methods were designed for static safety criteria,
and the lack the ability to satisfy temporally extended behaviors. Our work
inherits the advantages of diffusion based methods and can fulfill
$\textnormal{LTL}_{f}$ requirements.

Inverse Problems in Diffusion. Our proposed method formulates conditional
measurement under $\textnormal{LTL}_{f}$ with differentiable loss function
using the unnormalized likelihood [13, 32] for posterior sampling and can
guide the sampling process in a plug-and-play fashion. The most popular
methods to guide diffusion models during inference is classifier guidance [33]
and classifier-free guidance [34]. However, these methods cannot be applied in
a plug-and-play fashion for new conditioning factors. Our work is also related
to inverse task that infers a posterior distribution over data given a
measurement. Inverse methods [35, 36, 37, 38] do not require training a
conditional model in diffusion and can directly leverage pre-trained neural
networks.

(a) Studio-like Lab.

(b) Office room.

(c) Unitree Go2.

Figure 2: Real world environments for quadruped robot navigation.

##  V Experiments

TABLE I: Performance on Avoidance Tasks in Maze2d

Method\Perf. | Satisfaction rate1 ($\%$) $\uparrow$ |  Reward (UnCon) 2 $\uparrow$
---|---|---
Planning | Rollout
Diffuser | 9.5$\pm$3.1 | 11.0$\pm$1.7 | 142.2$\pm$5.1
SafeDiffuser | 99.4$\pm$0.9 | 12.3$\pm$3.5 | 135.8$\pm$5.2
LTLDoG-S | 99.0$\pm$0.8 | 73.0$\pm$3.0 | 97.3$\pm$2.8
LTLDoG-R | 98.8$\pm$0.8 | 92.0$\pm$1.4 | 127.1$\pm$5.1

  * 1

Calculated from $10$ groups of $100$ test trials. Best result during rollout
is highlighted. Same for other tables of Maze2d.

  * 2

Unconstrained rewards do _not_ take unsafe penalties into account.

TABLE II: Results of Avoidance Task in PushT Method\Perf. | Satisfaction rate($\%$) $\uparrow$ | Score$\uparrow$
---|---|---
Diffusion Policy | 34.8$\pm$18.0 | 0.941$\pm$0.0584
LTLDoG-S | 85.6$\pm$13.1 | 0.890$\pm$0.0647
LTLDoG-R | 85.6$\pm$12.5 | 0.842$\pm$0.0985

(a) Diffuser.

(b) SafeDiffuser.

(c) LTLDoG-S.

Figure 3: Examples of safe planning in Maze2d-Large. There are three unsafe
blocks (red squares, labeled $p_{L}$, $p_{M}$, $p_{R}$ from left to right)
that need to be avoided during navigation to the goal (shaded circle). The
$\textnormal{LTL}_{f}$ constraint for this task is
$\varphi=\Box\neg(p_{L}\wedge p_{M}\wedge p_{R})$. (a) Trajectories from
Diffuser ignore safety and can violate the specified constraints. (b)
SafeDiffuser produces discontinuous trajectories. (c) Our LTLDoG is able to
plan trajectories that detours around the obstacles to successfully arrive at
the goal.

(a) PushT environment.

(b) Diffusion Policy.

(c) LTLDoG-S.

Figure 4:  Results of safe control in PushT. (a) a robot arm’s end effector
(circles filled in blue) should move the T block (gray) to a goal pose
(green), and avoid entering unsafe regions (hollow circles marked with $pX$),
specified by an $\textnormal{LTL}_{f}$ formula (text in black). In this
example, the $\textnormal{LTL}_{f}$ specifies the end effector should never
enter regions $p1$ (purple) and $p3$ (cyan). (b) The actions generated and
executed by Diffusion Policy fails to satisfy the $\textnormal{LTL}_{f}$
formula. (c) In contrast, LTLDoG-S guides the diffusion to avoid $p1$ and
$p3$, yet still completes the manipulation task.

Our experiments focus on testing LTLDoG’s ability to handle static and
temporal safety constraints. We first briefly describe the simulated and real
environments, and baseline data-driven methods. Then we report empirical
results on benchmark environments and demonstrate LTLDoG’s applicability in
real world tasks through a case study on a quadruped robot (Fig. 2). We
conclude with a brief ablation study and analysis. Due to space restrictions,
we focus on conveying our main results. More details of the environments,
implementations, analysis and videos can be found in the appendix and online
supplementary material111https://github.com/clear-nus/ltldog.

###  V-A Experimental Setup

Environments. We evaluate methods in two simulation benchmark environments
(Maze2d [1] and PushT [3]) and demonstrate in two real indoor rooms. Maze2d
(Fig. 3 and 5) presents challenging long-horizon navigation tasks, where
state-of-the-art offline algorithms fail to learn a reliable goal-reaching
policy. The atomic propositions are determined by the occurrence of events
when the agent is inside key regions in the maze (for avoidance and
visitation). The PushT task (Fig. 4) requires manipulation of a T block
through interaction with a controllable mover. In our experiments, the mover
is constrained to visit specific regions and avoid others. Our real-world
experiments involve two indoor environments: a lab designed to mimic a studio
apartment, and an office environment (Fig. 2).

Compared methods. Our work involves trajectory generation by learning from an
offline dataset and as such, we compare against data-driven planning methods.
Diffuser and Diffusion Policy are state-of-the-art methods for sampling viable
plans but without any guarantees over external constraints. To evaluate how
well LTLDoG enforces safety specifications, we compare with SafeDiffuser, a
safe planning diffusion model using CBFs. However, note that SafeDiffuser
cannot handle temporal constraints or instructions; to our knowledge, our work
is the first data-driven method to handle both static and temporal
constraints. As such, there is no direct comparison baseline. We analyze the
differences between the two variants of our method, LTLDoG-S and LTLDoG-R.

###  V-B Comparative Analysis of Methods

Can LTLDoG achieve safe planning for static constraints? Our results indicate
that yes, LTLDoG is better able to generate trajectories that satisfy given
region-avoidance constraints relative to existing methods. In both the Maze2D
and PushT benchmarks, LTLDoG achieves high success rates (in both planning and
rollout), without severely compromising reward accumulation (Tbls. IX and II).
Lower total rewards are expected since safe paths are typically longer and
rollout scores are generally lower as the low-level controller may not exactly
follow the diffused trajectory. Qualitatively, Fig. 3 shows that LTLDoG is
able to generate safe trajectories in Maze2D, whilst SafeDiffuser is limited
to “local” deviations and fails to find paths that detour around unsafe
regions. Results in PushT are consistent with Maze2D; Fig. 4 shows that LTLDoG
performed the orientation task without entering unsafe regions.

(a) Regions for evaluation of atomic propositions in $\mathcal{P}$.

(b) Diffuser.

(c) Ours (LTLDoG-S).

Figure 5: Temporal Constraints in Maze2D. (a) Each maze has 6 non-overlapping regions. Agents are requested to visit some of these blocks under different temporally-extended orders. (b) and (c) show generated trajectories under $\varphi=\neg p_{3}\operatorname{\mathsf{U}}{}(p_{5}\wedge(\neg p_{2}\operatorname{\mathsf{U}}{}p_{0}))$. Our method can satisfy $\neg$ propositions (red zones) before reaching the green regions. TABLE III: Performance on Different $\textnormal{LTL}_{f}$s in Maze2d.  Environment | Method\Performance |  | Training $\textnormal{LTL}_{f}$s | Testing $\textnormal{LTL}_{f}$s
---|---|---|---|---
| Satisfaction rate ($\%$) $\uparrow$ | Reward (UnCon) $\uparrow$ | Satisfaction rate ($\%$) $\uparrow$ | Reward (UnCon) $\uparrow$
| Planning | Rollout | Planning | Rollout
U-Maze (Horizon 256) | Diffuser |  | 31.1$\pm$0.5 | 31.0$\pm$0.5 | 33.5$\pm$2.7 | 33.9$\pm$0.5 | 34.1$\pm$0.6 | 35.6$\pm$0.3
LTLDoG-S |  | 83.8$\pm$0.2 | 57.6$\pm$1.3 | 31.3$\pm$1.2 | 82.7$\pm$0.3 | 56.6$\pm$0.9 | 32.8$\pm$0.5
LTLDoG-R |  | 56.3$\pm$0.4 | 51.3$\pm$0.9 | 31.5$\pm$0.2 | 57.7$\pm$0.4 | 52.1$\pm$0.3 | 32.3$\pm$0.7
Medium (Horizon 384) | Diffuser |  | 15.0$\pm$0.7 | 13.4$\pm$0.6 | 84.8$\pm$0.3 | 11.6$\pm$1.4 | 10.1$\pm$1.2 | 84.8$\pm$0.5
LTLDoG-S |  | 77.9$\pm$5.7 | 31.8$\pm$2.6 | 53.1$\pm$5.2 | 68.4$\pm$6.7 | 28.7$\pm$3.5 | 50.5$\pm$4.7
LTLDoG-R |  | 51.8$\pm$1.8 | 39.5$\pm$1.6 | 57.3$\pm$0.2 | 43.3$\pm$4.4 | 30.6$\pm$1.9 | 57.7$\pm$0.1
Large (Horizon 512) | Diffuser |  | 13.5$\pm$0.4 | 12.8$\pm$0.1 | 76.3$\pm$0.1 | 11.6$\pm$2.3 | 11.5$\pm$1.7 | 77.8$\pm$3.9
LTLDoG-S |  | 73.8$\pm$2.4 | 32.6$\pm$1.4 | 42.3$\pm$5.0 | 66.6$\pm$2.7 | 24.9$\pm$1.7 | 40.9$\pm$4.4
LTLDoG-R |  | 66.9$\pm$0.6 | 47.4$\pm$0.8 | 54.6$\pm$1.3 | 57.5$\pm$2.3 | 39.0$\pm$2.9 | 54.5$\pm$3.9
TABLE IV: Generalization to Different $\textnormal{LTL}_{f}$ formulae in PushT Method\Performance |  $\textnormal{LTL}_{f}$ Set | Satisf. rate ($\%$) $\uparrow$ | Score $\uparrow$
---|---|---|---
Diffusion Policy | Training | 22.9$\pm$8.0 | 0.354$\pm$0.153
Test | 30.7$\pm$13.9 | 0.371$\pm$0.177
LTLDoG-S | Training | 28.2$\pm$8.33 | 0.290$\pm$0.115
Test | 43.0$\pm$17.0 | 0.299$\pm$0.145
LTLDoG-R | Training | 69.3$\pm$9.90 | 0.292$\pm$0.121
Test | 66.0$\pm$20.8 | 0.340$\pm$0.168

Can LTLDoG satisfy static and temporal constraints, and generalize to novel
$\textnormal{LTL}_{f}$ formulae? Table III and IV show the performance of the
compared methods on both training and test LTLs in Maze2d and PushT
environments. To elaborate, we follow the $\textnormal{LTL}_{f}$
specifications in [22] and adopt the Until sampler to generate random
$\textnormal{LTL}_{f}$s ($200$ for Maze2d and $36$ for PushT) that contain
different visitation sequences and unsafe regions. The training set has $80\%$
of all $\textnormal{LTL}_{f}$s and the rests are used as test set. Atomic
propositions consist of 6 regions in Maze2d (Fig. 5(a), i.e.,
$p_{0},p_{1},\ldots,p_{5}$) and 4 regions in PushT (Fig. 4(a)).

Results reveal that both LTLDoG-S and LTLDoG-R achieve significantly higher
success rates than Diffuser/Diffusion Policy. The baselines have a non-zero
performance as some generated $\textnormal{LTL}_{f}$ formulae are trivial to
satisfy at some start locations. Also note that some specifications can be
impossible to satisfy. Fig. 5 shows examples of trajectories generated in
Maze2d.

(a) Lab map and dataset coverage.

(b) Diffuser.

(c) Ours.

(d) Office map and dataset coverage.

(e) Diffuser.

(f) Ours.

Figure 6: Results in real world rooms. The instructed $\textnormal{LTL}_{f}$ is $\varphi=\neg\text{Passageway}\operatorname{\mathsf{U}}{}\text{Kitchen}$ for lab (first row) and $\varphi=\Diamond\left(\text{Door}\wedge\Diamond\text{Seat}\right)$ for office (second row). In the lab task the robot has to first unload in the kitchen area after entering the door before being allowed to go to the goal. When loaded, the robot is prohibited from going through the narrow passageway where people sit on the side. In the office task, the robot has to first visit the seat, followed by the door, before the goal. TABLE V: Results of achieving goals and $\textnormal{LTL}_{f}$ in real-world tasks. Environment | Method\Performance |  | Satisfaction rate ($\%$) $\uparrow$
---|---|---|---
| Goal | $\textnormal{LTL}_{f}$
Lab | Diffuser |  | 100.00 | 0.00
LTLDoG-S |  | 91.67 | 91.67
Office | Diffuser |  | 100.00 | 0.00
LTLDoG-S |  | 95.83 | 95.83

###  V-C Real Robot Case Study.

We show that LTLDoG can plan for a robot dog (Unitree Go2 Edu) given
$\textnormal{LTL}_{f}$ instructions in two real-world navigation environments
— a lab that mimics a studio apartment and an office room. Training of
diffusion models was performed in simulation using Gazebo and ROS1 using LIDAR
scanned maps (Fig. 6(a) and 6(d)). Note that the training trajectories do not
require running an oracle policy to satisfy many different
$\textnormal{LTL}_{f}$ formulae; we simply made the robot navigate to randomly
sampled goals using the global planner and TEB Local Planner [39] from the
ROS1 Navigation stack (Fig. 6(a) and 6(d)).

To test on potential constraints, we queried GPT-4 for $\textnormal{LTL}_{f}$
formulae representing meaningful robotic tasks including obstacle avoidance
and sequential navigation. We used 4 kinds of $\textnormal{LTL}_{f}$s from the
generated results, i.e., $\Box\neg p_{0}$, $\Diamond p_{0}\wedge\Diamond
p_{1}$, $\Diamond\left(p_{0}\wedge\Diamond p_{1}\right)$ and $\neg
p_{1}\operatorname{\mathsf{U}}{}p_{0}$. The first $\textnormal{LTL}_{f}$
corresponds to an obstacle avoidance task where the robot should never visit a
specific region. For example, $\Box\neg\text{Seat}$ means the robot should
never enter the Seat region (Fig 6(d)). The remaining three
$\textnormal{LTL}_{f}$ formulae represent: 1) visiting all regions at least
once; 2) visiting regions in a specific sequence; 3) avoiding a specific
region until another has been visited. Some example regions designed in our
real environments and start/goal locations are shown in Fig 6.

In total, $96$ trials were executed on the real robot; $12$ trajectories ($6$
for baseline Diffuser and $6$ for our method LTLDoG-S) with varying regions
and different $\textnormal{LTL}_{f}$ formulae in each room. The overall
satisfaction rate of all raw generated trajectories in _simulation_ is
$85.8\pm 14.0\%$ (c.f. baseline $2.9\pm 5.4\%$). For each specific formula, a
sample was selected for real-world execution based on their feasibility. The
results in TABLE V show that LTLDoG has a high satisfaction rate compared to
Diffuser; Fig. 6(c) and 6(f) illustrate trajectories for qualitative
comparison.

###  V-D Ablation Study and Analysis

TABLE VI: Ablation Study – Binary Classifier Guidance

Method\Performance | LTL Set | Satisfaction rate ($\%$) $\uparrow$ |  Reward (UnCon) $\uparrow$
---|---|---|---
Planning | Rollout
Diffuser | Training | 31.1$\pm$0.5 | 31.0$\pm$0.5 | 32.5$\pm$2.7
Test | 33.9$\pm$0.5 | 34.1$\pm$0.6 | 35.6$\pm$0.3
Classifier guidance1 | Training | 41.1$\pm$0.6 | 40.6$\pm$0.7 | 33.9$\pm$0.9
Test | 40.0$\pm$0.8 | 41.8$\pm$1.1 | 35.3$\pm$0.3
LTLDoG-R | Training | 56.3$\pm$0.4 | 51.3$\pm$0.9 | 31.5$\pm$0.2
Test | 57.7$\pm$0.4 | 52.1$\pm$0.3 | 32.3$\pm$0.7

  * 1

The classifier guidance method only leverages _binary labels_ for LTL
satisfaction checking in Maze2d U-Maze.

Ablation study. Unlike classifier guidance, where each trajectory is labelled
as _satisfy_ or _not satisfy_ , we leverage the continuous values from our
formula evaluator (as described in Section III-C) and train a regressor
guidance network. Although classifier guidance improves over the vanilla
Diffuser, it achieves lower performance than LTLDoG-R ( Table VI). This
comparison supports the notion that soft labels improves the guidance neural
network; we posit using real values provides richer information in terms of
how _well_ the trajectory satisfies a given $\textnormal{LTL}_{f}$ formula.

Analysis on horizon. Fig. 7 demonstrates that planning with a longer horizon
leads to improved performance in terms of $\textnormal{LTL}_{f}$ satisfaction.
This improvement is attributed to the fact that $\textnormal{LTL}_{f}$
instructions often require a longer sequence of steps in a path compared to
mere goal navigation, e.g., visiting a specific region before reaching the
goal.

(a) Planning.

(b) Rollout.

Figure 7: Performance in Maze2d U-Maze with different lengths of trajectory.
DM stands for Diffuser and DoG-S/DoG-R are our methods.

##  VI Conclusion, Discussion and Future Work

In this work, we presented LTLDoG, an approach towards generating safe
trajectories that comply with $\textnormal{LTL}_{f}$ specifications at test
time. Within our overall scheme, we presented two methods: LTLDoG-S guides the
sampling process under any $\textnormal{LTL}_{f}$ formula while LTLDoG-R uses
a trained model that we show generalizes to new formulae with similar
structure. To our knowledge, this work is the first that successfully
incorporates model checking using a formal language with diffusion models for
safe planning. Notably, LTLDoG does not require data collection for each
potential $\textnormal{LTL}_{f}$ instruction; rather, we control the sampling
process during diffusion using “soft” model checking to generate new
trajectories using existing information provided by the training dataset.

Limitations and Future Work. LTLDoG is a step towards trustworthy trajectory
planning using generative models. There are several areas where LTLDoG can be
improved. Similar to other diffusion models, LTLDoG is generally unable to
generate realistic trajectories when the context (environment/goal) is far
from the training distribution. As such, the dataset should preferably contain
trajectories with a variety of complex behaviors that can potentially satisfy
different test-time $\textnormal{LTL}_{f}$ formulae. It would be interesting
to develop methods to ensure the sampling process of LTLDoG-S adheres to the
data manifold. Additionally, LTLDoG plans in an open-loop fashion, where the
entire trajectory is generated conditioned on the constraints. This is mainly
because the evaluation of an $\textnormal{LTL}_{f}$ formula depends on the
entire trajectory. We plan to further explore planning using receding horizon
control using partial evaluations on an incomplete trajectory.

## References

  * [1] M. Janner, Y. Du, J. B. Tenenbaum, and S. Levine, “Planning with diffusion for flexible behavior synthesis,” in _Int. Conf. Mach. Learn._ , vol. 162, 2022, pp. 9902–9915.
  * [2] A. Ajay, Y. Du, A. Gupta, J. B. Tenenbaum, T. S. Jaakkola, and P. Agrawal, “Is conditional generative modeling all you need for decision making?” in _Int. Conf. Learn. Representations_ , 2023.
  * [3] C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” in _Proc. Robot.: Sci. and Syst. (RSS)_ , 2023.
  * [4] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in _Proc. 32nd Int. Conf. Mach. Learn._ , 2015, pp. 2256–2265.
  * [5] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in _Advances in Neural Inf. Process. Syst._ , 2020, pp. 6840–6851.
  * [6] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” in _Int. Conf. Learn. Representations_ , 2021.
  * [7] W. Xiao, T.-H. Wang, C. Gan, and D. Rus, “SafeDiffuser: Safe planning with diffusion probabilistic models,” _arXiv preprint arXiv:2306.00148_ , 2023.
  * [8] N. Botteghi, F. Califano, M. Poel, and C. Brune, “Trajectory generation, control, and safety with denoising diffusion probabilistic models,” _arXiv preprint arXiv:2306.15512_ , 2023.
  * [9] A. Pnueli, “The temporal logic of programs,” in _18th Annu. Symp. Found. Comput. Sci._ , 1977, pp. 46–57.
  * [10] P. Vincent, “A connection between score matching and denoising autoencoders,” _Neural Comput._ , vol. 23, no. 7, pp. 1661–1674, 2011.
  * [11] C. Baier and J. Katoen, _Principles of Model Checking_. MIT Press, 2008.
  * [12] C. Belta, B. Yordanov, and E. A. Gol, _Formal Methods for Discrete-Time Dynamical Systems_. Springer, 2017, vol. 89.
  * [13] H. Chung, J. Kim, M. T. Mccann, M. L. Klasky, and J. C. Ye, “Diffusion posterior sampling for general noisy inverse problems,” in _Int. Conf. Learn. Representations_ , 2023.
  * [14] B. Efron, “Tweedie’s formula and selection bias,” _J. Amer. Statistical Assoc._ , vol. 106, no. 496, pp. 1602–1614, 2011.
  * [15] Z. Xu, Y. S. Rawat, Y. Wong, M. Kankanhalli, and M. Shah, “Don’t pour cereal into coffee: Differentiable temporal logic for temporal action segmentation,” in _Advances in Neural Inf. Process. Syst._ , 2022.
  * [16] K. Leung, N. Aréchiga, and M. Pavone, “Backpropagation through signal temporal logic specifications: Infusing logical structure into gradient-based methods,” _Int. J. Robot. Res._ , vol. 42, no. 6, pp. 356–370, 2023.
  * [17] M. Gori, G. Monfardini, and F. Scarselli, “A new model for learning in graph domains,” in _Proc. IEEE Int. Joint Conf. Neural Netw._ , vol. 2, 2005, pp. 729–734.
  * [18] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” _IEEE Trans. Neural Networks_ , vol. 20, no. 1, pp. 61–80, 2009.
  * [19] S. Zhu, L. M. Tabajara, J. Li, G. Pu, and M. Y. Vardi, “Symbolic LTLf synthesis,” in _Int. Joint Conf. Artif. Intell._ , 2017, pp. 1362–1369.
  * [20] A. Camacho, J. Baier, C. Muise, and S. McIlraith, “Finite LTL synthesis as planning,” in _Proc. Int. Conf. Automated Planning and Scheduling_ , vol. 28, 2018, pp. 29–38.
  * [21] Y. Xie, F. Zhou, and H. Soh, “Embedding symbolic temporal knowledge into deep sequential models,” in _IEEE Int. Conf. Robot. Automat._ , 2021, pp. 4267–4273.
  * [22] P. Vaezipoor, A. C. Li, R. A. T. Icarte, and S. A. Mcilraith, “LTL2Action: Generalizing LTL instructions for multi-task RL,” in _Int. Conf. Mach. Learn._ , 2021, pp. 10 497–10 508.
  * [23] M. Schlichtkrull, T. N. Kipf, P. Bloem, R. Van Den Berg, I. Titov, and M. Welling, “Modeling relational data with graph convolutional networks,” in _The Semantic Web_ , 2018, pp. 593–607.
  * [24] F. Bacchus and F. Kabanza, “Using temporal logics to express search control knowledge for planning,” _Artif. Intell._ , vol. 116, no. 1, pp. 123–191, 2000.
  * [25] J. A. Baier and S. A. McIlraith, “Planning with temporally extended goals using heuristic search,” in _Proc. Int. Conf. Automated Planning and Scheduling_ , 2006, p. 342–345.
  * [26] A. Camacho, E. Triantafillou, C. Muise, J. Baier, and S. McIlraith, “Non-deterministic planning with temporally extended goals: LTL over finite and infinite traces,” in _Proc. AAAI Conf. Artif. Intell._ , vol. 31, no. 1, 2017.
  * [27] G. Fainekos, H. Kress-Gazit, and G. Pappas, “Temporal logic motion planning for mobile robots,” in _IEEE Int. Conf. Robot. Automat._ , 2005, pp. 2020–2025.
  * [28] V. Kurtz and H. Lin, “Temporal logic motion planning with convex optimization via graphs of convex sets,” _IEEE Trans. Robot._ , vol. 39, no. 5, pp. 3791–3804, 2023.
  * [29] C. Yang, M. L. Littman, and M. Carbin, “On the (in)tractability of reinforcement learning for LTL objectives,” in _Int. Joint Conf. Artif. Intell._ , 2022, pp. 3650–3658.
  * [30] R. Toro Icarte, T. Q. Klassen, R. Valenzano, and S. A. McIlraith, “Teaching multiple tasks to an RL agent using LTL,” in _Proc. Int. Conf. Autonomous Agents Multiagent Syst._ , 2018, pp. 452–461.
  * [31] C. Voloshin, H. M. Le, S. Chaudhuri, and Y. Yue, “Policy optimization with linear temporal logic constraints,” in _Advances in Neural Inf. Process. Syst._ , 2022, pp. 17 690–17 702.
  * [32] J. Song, Q. Zhang, H. Yin, M. Mardani, M.-Y. Liu, J. Kautz, Y. Chen, and A. Vahdat, “Loss-guided diffusion models for plug-and-play controllable generation,” in _Int. Conf. Mach. Learn._ , vol. 202, 2023, pp. 32 483–32 498.
  * [33] P. Dhariwal and A. Q. Nichol, “Diffusion models beat GANs on image synthesis,” in _Advances in Neural Inf. Process. Syst._ , vol. 34, 2021, pp. 8780–8794.
  * [34] J. Ho and T. Salimans, “Classifier-free diffusion guidance,” in _NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications_ , 2021.
  * [35] A. Jalal, M. Arvinte, G. Daras, E. Price, A. G. Dimakis, and J. Tamir, “Robust compressed sensing MRI with deep generative priors,” in _Advances in Neural Inf. Process. Syst._ , vol. 34, 2021, pp. 14 938–14 954.
  * [36] H. Chung and J. C. Ye, “Score-based diffusion models for accelerated MRI,” _Med. Image Anal._ , p. 102479, 2022.
  * [37] H. Chung, B. Sim, and J. C. Ye, “Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction,” in _IEEE/CVF Conf. Comput. Vis. Pattern Recognit._ , 2022, pp. 12 413–12 422.
  * [38] H. Chung, B. Sim, D. Ryu, and J. C. Ye, “Improving diffusion models for inverse problems using manifold constraints,” in _Advances in Neural Inf. Process. Syst._ , vol. 35, 2022, pp. 25 683–25 696.
  * [39] C. Rösmann, F. Hoffmann, and T. Bertram, “Integrated online trajectory planning and optimization in distinctive topologies,” _Robot. Auton. Syst._ , vol. 88, pp. 142–153, 2017.
  * [40] J. Fu, A. Kumar, O. Nachum, G. Tucker, and S. Levine, “D4RL: Datasets for deep data-driven reinforcement learning,” _arXiv preprint arXiv:2004.07219_ , 2020.
  * [41] A. Camacho, R. Toro Icarte, T. Q. Klassen, R. Valenzano, and S. A. McIlraith, “LTL and beyond: Formal languages for reward function specification in reinforcement learning,” in _Int. Joint Conf. Artif. Intell._ , 2019, pp. 6065–6073.

###  -A Data

####  -A1 Maze2d

The Maze2d environment is part of a well-known open-source benchmark for
offline reinforcement learning — D4RL [40]. Datasets of trajectories are
publicly available in the D4RL library. Each of the trajectories within is one
continuous trajectory of an agent navigating to random goals without
interruption. See Fig. 8 for an overview of the environments. We utilize the
original dataset in D4RL as is to train the baseline Diffuser, which also
serves as the diffusion model of LTLDoG.

(a) U-Maze.

(b) Medium.

(c) Large.

Figure 8: Maze2d environments. The agent is tasked to move force-actuated ball
to goals (red circle) from random start locations (green ball). There are
three kinds of maze configurations with increasing difficulties and all of
them are long-horizon path planning tasks with continuous states and actions.

####  -A2 PushT

PushT is a robot manipulation benchmark adopted in Diffusion Policy [3] to
showcase diffusion models’ planning capabilities. A dataset of expert policies
for pushing a T block to the same goal position starting from random poses are
publicly available at [3]. Though Diffusion Policy achieves good performance
on the original manipulation task, trained with around only 200 expert
demonstrations (i.e., 206 episodes), we find that these trajectories lack the
modalities or abundant behaviors that potentially or partially satisfy
different $\textnormal{LTL}_{f}$ constraints. As such, expanding the original
dataset is necessary for our method to work.

Data Augmentation Given that obtaining extra expert demonstrations is
inefficient, we performed common data augmentation techniques to expand the
original dataset. Concretely, we first employed simple geometric augmentation,
only rotation and flipping, for each trajectory in the original dataset,
assuming basic physical symmetry in this simulation environment. Then, we used
a pretrained Diffusion Policy to push the T block to the goal pose, aligning
with the original dataset in which the goal of the manipulation task is
implicitly encoded in trajectories. In this way, we augmented the original set
from 206 trajectories to 6686 trajectories, and elongated the average episode
length from 125 to 289 steps. The augmented dataset is publicly available
online222https://github.com/clear-nus/ltldog.

In this task, the baseline (Diffusion Policy) and our methods share the same
diffusion model backbone, which is trained over the augmented dataset.

####  -A3 Real Robot Scenarios

Data for both of the real-world scenarios are collected in simulated
environments. For the office we collected 4 long continuous trajectories in
the simulation of step size of 11,000, 107,000, 60,000 and 130,000. Similarly
for the lab map we collected 4 trajectories of steps 11,000, 19,000, 110,000
and 120,000. First, we built by SLAM the maps of the two environments, a
studio-like lab and an office room, and the maps are used to create two
simulation scenarios. For creating the maps, we teleoperated a quadruped robot
(Unitree Go2) mounted with LiDAR in each of these environments. We used the
slam_toolbox along with Nav2 for generating the maps in both environments. We
created simple 3D world for each of the maps for simulation in gazebo. After
that, we enabled a Go2 robot simulated in Gazebo and driven by ROS Navigation
Stack to navigate to random goal positions sampled in the maps for data
collection, creating long trajectories in a similar fashion to Maze2d.

####  -A4 LTL Satisfaction Values

The soft labels (values indicating how well the $\textnormal{LTL}_{f}$s are
satisfied) for training the regressor guide (LTLDoG-R) are calculated with an
adapted version of the DTL toolbox [15] and saved before each training
session, and can be reused if possible, since each training session may sample
and cut different segments of the trajectories in datasets based on different
training configurations. Values calculation for benchmarking test results are
performed online simultaneously with all tests.

###  -B $\textnormal{LTL}_{f}$ Generation

To generate practical $\textnormal{LTL}_{f}$s meaningful for real robotic
tasks, we query GPT-4 via prompting with some constraints on
$\textnormal{LTL}_{f}$s. The prompt and the response from ChatGPT is listed in
TABLE VII. In the real experiments, we tested the first four returned
$\textnormal{LTL}_{f}$s as these are also commonly used in literature [30, 41,
22]. Some other generated $\textnormal{LTL}_{f}$s shown in this table are not
logically correct or meaningful for robotic tasks. For instance, the
$\textnormal{LTL}_{f}$s with imply operator do not enforce any constraints if
the condition is not met (e.g., the condition in the $6$-th
$\textnormal{LTL}_{f}$ can never be satisfied for disjoint regions).

TABLE VII: Prompts and Responses from ChatGPT (GPT-4)1 for
$\textnormal{LTL}_{f}$ Generation.

Prompt |  |  In a robot navigation task, the robot is required to visit four key regions (referred to as p_0, p_1, p_2 and p_3) in specific temporal orders. The constraints on the robot’s trajectory concerning the visitation (or non-visitation) of these regions can be specified using finite Linear Temporal Logic (LTLf). Please generate several LTLf formulas (e.g., 10) that are realistic in real-world environments and tasks involving robot navigation. The details of the task and additional requirements are as follows: 1\. The LTLf operators that can be used are: $\neg$ (Negate), $\wedge$ (And), $\vee$ (Or), F (Eventually), G (Always), U (Until). Avoid using other LTL operators. Also, avoid using X (Next) since the number of timesteps needed to move from one region to another is unknown. 2\. The atomic propositions are determined by whether the robot is inside p_0, p_1, p_2 and p_3. Therefore, the labelling function returns assignments with a 4-dimensional 0-1 vector at each timestep of the robot’s trajectory. 3\. Only one proposition can be true at each timestep because these regions are disjoint. The robot cannot be inside any two regions simultaneously. An assignment at a timestep can be a zero vector (0, 0, 0, 0), indicating that the robot is not currently in any of these regions. This occurs when the robot is initially not in these regions and all atomic propositions are False. 4\. The robot cannot move directly from one region to another as the regions might not be connected. Therefore, LTLf formulas such as p_0 U p_1 are not valid since both propositions will be False when the robot is not inside these two regions. 5\. The robot’s trajectory is finite, so the robot cannot repeatedly visit different regions. 6\. The LTLf formulas are supposed to apply from the first timestep of the assignment. 7\. When generating an LTLf formula, please analyze the generated formula first to ensure that it is valid and that there exist assignments that can satisfy it.
---|---|---
ChatGPT |  |  To create realistic LTLf formulas for a robot navigation task considering the specified constraints and operators, we need to structure these formulas in a way that reflects possible navigation scenarios. Here are ten LTLf formulas that align with the given requirements: 1. Visit all regions at least once: • $F(p_{0})\wedge F(p_{1})\wedge F(p_{2})\wedge F(p_{3})$ This formula states that eventually, the robot will visit each of the four regions at least once during its trajectory. 2. Visit regions in a specific sequence: • $F(p_{0}\wedge F(p_{1}\wedge F(p_{2}\wedge F(p_{3}))))$ This formula requires the robot to visit $p_{0}$, then $p_{1}$, followed by $p_{2}$, and finally $p_{3}$, in that order. 3. Avoid a specific region until another has been visited: • $\neg p_{1}\operatorname{\mathsf{U}}{}p_{0}$ Here, the robot must avoid $p_{1}$ until $p_{0}$ has been visited. 4. Never visit a specific region: • $G(\neg p_{3})$ This formula specifies that the robot should never enter region $p_{3}$. 5. Visit one region only after visiting two others: • $F(p_{0}\wedge F(p_{1}))\operatorname{\Rightarrow}{}F(p_{2})$ The robot can visit $p_{2}$ only after it has visited $p_{0}$ and then $p_{1}$. 6. Ensure two regions are visited before a third: • $F(p_{0}\wedge p_{1})\operatorname{\Rightarrow}{}F(p_{2})$ This formula ensures that $p_{2}$ and $p_{1}$ are visited (in no particular order) before visiting $p_{2}$: 7. Visit at least three regions: • $F(p_{0})\vee F(p_{1})\vee F(p_{2})\vee F(p_{3})$ This ensures the robot visits at least three out of the four regions. It is a less strict requirement, allowing for flexibility in navigation. 8. Visit a specific region last: • $F(p_{0}\wedge F(p_{1}\wedge F(p_{2})))\wedge G(\neg p_{3}\operatorname{\mathsf{U}}{}(p_{0}\wedge F(p_{1}\wedge F(p_{2}))))$ The robot must visit $p_{3}$ last, after visiting $p_{0}$, $p_{1}$, and $p_{2}$ in any order. 9. Avoid a region until all other regions have been visited: • $\neg p_{3}\operatorname{\mathsf{U}}{}(F(p_{0})\wedge F(p_{1})\wedge F(p_{2}))$ The robot should avoid $p_{3}$ until it has visited $p_{0}$, $p_{1}$, and $p_{2}$ in any order. 10. Visit a pair of regions, avoiding others until then: • $\neg(p_{2}\vee p_{3})\operatorname{\mathsf{U}}{}(p_{0}\wedge F(p_{1}))$ The robot must avoid $p_{2}$ and $p_{3}$ until it has visited $p_{0}$ and then $p_{1}$: Each of these formulas considers the finite nature of the robot’s trajectory, the impossibility of being in multiple regions simultaneously, and the requirement that the robot may need to travel through unspecified areas (not within $p_{0}$, $p_{1}$, $p_{2}$, or $p_{3}$) to move from one region to another. The formulas are structured to reflect realistic navigation tasks that could be encountered in real-world robot navigation scenarios, such as search and rescue, surveillance, or delivery tasks within a specified area.

  * 1

https://chat.openai.com/share/43670685-3cb9-4496-ad3e-1ee70cd358b8.

###  -C Additional Illustrations

More illustrations and comparisons against the baseline for the obstacle
avoidance task in Maze2d environment are in Fig. 9.

(a) Trajectories from Diffuser.

(b) Trajectories from SafeDiffuser.

(c) Trajectories from LTLDoG-S.

(d) Trajectories from LTLDoG-R.

Figure 9: Results of safe planning in Maze2d-Large with unsafe regions. There are three unsafe blocks (red squares, labeled $p_{L}$, $p_{M}$, $p_{R}$ from left to right) that need to be avoided during navigation to the goal (shaded circle). The $\textnormal{LTL}_{f}$ constraint for this task is $\varphi=\Box\neg(p_{L}\wedge p_{M}\wedge p_{R})$. (a) Trajectories from Diffuser ignore safety and can violate the specified constraints. (b) SafeDiffuser produces discontinuous trajectories (the constraints here are slightly reshaped to accommodate the CBF function). (c, d) Our LTLDoG is able to plan trajectories that detours around the obstacles to successfully arrive at the goal. TABLE VIII: Results of generated $\textnormal{LTL}_{f}$s in simulation rooms. Task | Method\Performance | Success rate ($\%$) $\uparrow$
---|---|---
Lab | Office
Obstacle avoidance | Diffuser | 11.7$\pm$3.7 | 0.0$\pm$0.0
LTLDoG-S | 100.0$\pm$0.0 | 80.0$\pm$8.1
Temporal behaviors | Diffuser | 0.0$\pm$0.0 | 0.0$\pm$0.0
LTLDoG-S | 96.5$\pm$1.5 | 66.7$\pm$1.8

(a) Planning.

(b) Rollout C1.

(c) Rollout C2.

Figure 10: Example of failure cases from LTLDoG-S. TABLE IX: Comparison of
Controllers1 in Avoidance Tasks in Maze2d

Method\Perf. | Success rate ($\%$) $\uparrow$ | Unconstrained score2 $\uparrow$
---|---|---
Planning | Rollout | C1 | C2
C1 | C2
Diffuser | 9.5$\pm$3.1 | 11.0$\pm$1.7 | 15.0$\pm$2.6 | 142.2$\pm$5.1 | 92.1$\pm$3.0
SafeDiffuser | 99.4$\pm$0.9 | 12.3$\pm$3.5 | 14.2$\pm$2.8 | 135.8$\pm$5.2 | 131.9$\pm$5.4
LTLDoG-S | 99.0$\pm$0.8 | 73.0$\pm$3.0 | 90.6$\pm$0.6 | 97.3$\pm$2.8 | 36.2$\pm$11.7
LTLDoG-R | 98.8$\pm$0.8 | 92.0$\pm$1.4 | 99.6$\pm$0.6 | 127.1$\pm$5.1 | 76.4$\pm$13.4

  * 1

Planned trajectories are rolled out using two different low level control
strategies, denoted as C1 and C2.

  * 2

Unconstrained scores _do not_ take unsafe penalties into account.

Fig. 11–Fig. 13 delineate two more examples of the comparisons between our
method LTLDoG against the baseline Diffusion Policy on the obstacle avoidance
and temporal behavior tasks. These results demonstrate our method’s
capabilities in guiding diffusion models to follow not only _purely spatial_
but also _spatial-temporal_ constraints.

(a) Regions.

(b) Diffusion Policy.

(c) Ours (LTLDoG-S).

(d) Ours (LTLDoG-R).

Figure 11: PushT task. Diffusion policy generates next locations of the mover
to push the T block by following these locations. Atomic propositions for LTLs
are defined by constraints over actions.

(a) LTLDoG-S

(b) LTLDoG-R

Figure 12:  Snapshots of two successful rollouts of the two variants of our
method accomplishing the same task in Fig. 11. The mover (circle filled in
blue) should avoid region $p1$ and $p3$ at all time, as designated in the
$\textnormal{LTL}_{f}$ “$\Box(\neg p_{1}\wedge\neg p_{3})$.” Fig. 12(a) and
Fig. 12(b) showcase that LTLDoG-S and LTLDoG-R exploit different modalities to
complete the original manipulation task while respecting the constraint
specified by the $\textnormal{LTL}_{f}$.

(a) Diffusion Policy (baseline)

(b) LTLDoG-S (ours)

(c) LTLDoG-R (ours)

Figure 13:  Snapshots of comparisons between the baseline and LTLDoG given an
$\textnormal{LTL}_{f}$ with intricate _temporal_ constraints. The
$\textnormal{LTL}_{f}$ “$\neg
p_{1}\operatorname{\mathsf{U}}{}(p_{2}\wedge(\neg
p_{0}\operatorname{\mathsf{U}}{}p_{3}))$” demands the mover never enter $p1$
until (or, unless by the time of entering) the two conditions are both met: 1)
it has already been in $p2$, and 2) $p0$ is never reached until $p3$ is
visited. Clearly, the baseline breaches this complex constraint when
proceeding with the original manipulation task, shown from the third to the
fifth snapshot (the mover enters $p0$ _without_ reaching $p3$ beforehand and
then gets into $p0$). In contrast, our method, both LTLDoG-S and LTLDoG-R,
manage to satisfy the constraint and complete the manipulation task by two
different ways (the common ground is to first visit $p3$ and $p2$ during or on
the way to maneuvers).

We also show in Fig. 14 more snapshots of a quadruped robot running in the
simulated Lab environment with different policies (baseline against ours). The
overall satisfaction rate of all raw generated trajectories in different tasks
and environments are summarized in Table VIII. The overall performance is
$85.8\pm 14.0\%$ (_c.f._ baseline $2.9\pm 5.4\%$) as reported in the main
paper.

(a) Frames of trajectories from Diffuser in simulation.

(b) Frames of trajectories from LTLDoG-S in simulation.

(c) Frames of trajectories from LTLDoG-R in simulation.

Figure 14: Results of safe planning and execution in simulation and indoor
room on a quadruped robot.

###  -D Failure case study

Fig. 10(a) shows an infeasible trajectory generated from LTLDoG-S. Our
hypothesis is that posterior sampling in LTLDoG-S calculate gradients based on
Euclidean distance on the trajectory space due to the LTL evaluation, which
may push the trajectory away from the data manifold. On the other hand, the
neural network in regressor guidance is trained on noised intermediate
samples, which are linked to the data manifold. Gradients ascent on noise
space will likely push the noised example to a place where both the
satisfaction value and likelihood are high. Visualizations of the difference
between two controllers as shown in Fig. 10(b) Fig. 10(c) suggest that
selecting next locations sequentially without distance checking is a less safe
option.

Low-Level Control Policy The success rate during rollout also depends on the
specific controller used to visit each location of the generated trajectory.
TABLE IX shows the comparative results between the two low-level control
policies. A more conservative controller (denoted as C2) that visits every
location exactly with distance checking can enjoy higher success rate than
greedily move (C1, originally used in [1]).


# vTTS: visual-text to speech

###### Abstract

This paper proposes visual-text to speech (vTTS), a method for synthesizing
speech from visual text (i.e., text as an image). Conventional TTS converts
phonemes or characters into discrete symbols and synthesizes a speech waveform
from them, thus losing the visual features that the characters essentially
have. Therefore, our method synthesizes speech not from discrete symbols but
from visual text. The proposed vTTS extracts visual features with a
convolutional neural network and then generates acoustic features with a non-
autoregressive model inspired by FastSpeech2. Experimental results show that
1) vTTS is capable of generating speech with naturalness comparable to or
better than a conventional TTS, 2) it can transfer emphasis and emotion
attributes in visual text to speech without additional labels and
architectures, and 3) it can synthesize more natural and intelligible speech
from unseen and rare characters than conventional TTS.

Index Terms: speech synthesis, visual text, visual-text to speech, visual
feature

##  1 Introduction

Text to speech (TTS) is a method for synthesizing speech from arbitrary text.
Along with the development of deep neural networks (DNNs), neural TTS has
succeeded in synthesizing speech with the same naturalness as human speech [1,
2, 3, 4]. Generally, TTS first converts each character (or each phoneme based
on linguistic knowledge) into a discrete symbol and synthesizes a speech
waveform from it (Figure 1(a)).

However, when we read a sentence out loud, we do not view each character as a
discrete symbol but rather use the visual information of the character. For
example, when we read a phonogram (a character representing a speech sound),
the character or the combination of sub-characters (components of one
character) determines the reading. In addition, we can recognize attributes in
visual text and reflect them in speech. For example, when reading a textbook,
we can recognize an underlined (or bold, italic) word as important and read
the word emphatically [5]. Another example is typeface, which sometimes evokes
a certain emotion [6, 7]. Advertisements and comics utilize this to convey
desired emotions to readers [8]. From these facts, it is more appropriate to
use the visual information of text to synthesize speech.

In this paper, we propose visual-text to speech (vTTS), which generates a
speech waveform from visual text (Figure 1(b)). Our vTTS consists of a visual
feature extractor and FastSpeech2-inspired model. The visual feature extractor
extracts visual features from visual text using a convolutional neural network
(CNN). The FastSpeech2-inspired model predicts acoustic features from the
extracted visual features.

Our vTTS has three advantages over conventional TTS. First, vTTS can reflect
emphasis and emotion attributes in speech by recognizing the difference of the
typeface of visual text itself whereas the basic TTS requires additional
labels and architectures [9, 10]. Second, our vTTS does not require a
vocabulary (lookup table) of characters, unlike the basic TTS. A pre-
determined fixed-size vocabulary cannot be used to embed out-of-vocabulary
(OOV) characters (typically, “unknown” symbol is used). Our vTTS avoids this
and obtains visual features from OOV characters. Last, for some languages, our
model can efficiently predict the reading from character compositionality (the
reading or meaning is determined by the structure of the character). Even if
rare and OOV characters emerge, vTTS can synthesize more natural and
intelligible speech utilizing character compositionality than the basic TTS.

Figure 1: Comparison of speech synthesis methods. Whereas the basic TTS uses
discrete text features, our vTTS uses visual features.

We compared our vTTS with the basic TTS in three phonetic languages: Japanese
(Hiragana), Korean (Hangul), and English (Roman Alphabet) and then revealed
the following properties.

  * •

Our vTTS is comparable to or better than the basic TTS in terms of naturalness
of synthetic speech.

  * •

It can transfer emphasis and emotion attributes in visual text to speech
without additional labels and architectures.

  * •

It can synthesize more natural and intelligible speech from OOV and rare
phonograms than the basic TTS.

Our implementation111https://github.com/Yoshifumi-Nakano/visual-text-to-speech
and corpus222https://sites.google.com/site/shinnosuketakamichi/research-
topics/jecs_corpus are open-sourced on the project pages for reproducibility
and future work.

##  2 Related Work

###  2.1 Text to speech (TTS)

As the basic TTS model, we use FastSpeech2 [11], a well-known non-
autoregressive model. FastSpeech2 consists of three components: an encoder,
variance adapter, and decoder. The input of the original FastSpeech2 is a
phoneme sequence, but we use a character sequence instead for fair comparison
with the proposed method. Section 4.1 notes the results of preliminary
experiments on this change.

Some studies use information other than text such as reference voices [12],
face images [13], and articulation [14]. These methods have an additional
encoder for such information. However, similar architecture is not suitable
for vTTS. Since visual text is not discrete as described in Section 1, text
and visual information should not be treated separately. Therefore, we propose
a model that jointly treats text and visual information in Section 3.

Although a character is a input unit of speech synthesis that does not rely on
language knowledge, some studies use other units, e.g., phrase [15], word
[16], subword [17], and byte [18] in descending order of length. In this
paper, we use a pixel, which is a smaller input unit than ever before. We can
observe a similar trend in natural language processing as described below.

###  2.2 Visual text for natural language processing (NLP)

In NLP, the input unit of language models and machine translation models has
been examined. Starting with phrases [19] and words [20] and moving through
subwords [21], characters [22], and sub-characters [23], some research
proposes the use of pixels (i.e., visual text).

There is one study that extracted character embeddings from visual text [24].
This study utilized logogram compositionality, allowing for better processing
of rare words. Similarly, there are studies that have used character
compositionality in a word segmentation task [25] or a sentiment
classification task [26]. Other than that, the use of sliced visual text,
which is analogous to subword text symbols, leads to there being more
resistance to various types of noise [27]. These studies mainly used a CNN to
extract the visual information from visual text.

These studies are based on the compositionality in logograms and acquire
meaning-related features. In contrast, our study aims to acquire sound-related
ones contained in characters, especially in phonograms.

##  3 Method

Figure 2: Overall architecture of the proposed method. Figure 3: Example of
visual text. Structure and combination of sub-characters determine the overall
reading (top). Emphatic (middle) and typeface-decorated (bottom) text evoke
emphasis and emotion, respectively.

###  3.1 Architecture comparison between vTTS and TTS

As for the architecture of the proposed vTTS, we replace the character
embedding layer (i.e., text feature extractor) of the original FastSpeech2
with a visual feature extractor (Figure 2(a)). This replacement allows for
extracting visual features from visual text. The subsequent encoder, variance
adapter, and decoder are the same as the original FastSpeech2. To investigate
the ideal performance of vTTS, in this paper, we use artificially generated
visual text rather than that in in-the-wild images (e.g., advertisements and
comics).

###  3.2 Generating visual text

The first step is to transform text to grayscale sliced images. The overall
process is illustrated in Figure 2(c). Since the original FastSpeech2 is a
non-autoregressive TTS utilizing a duration-based upsampler, we must take the
temporal alignment between visual text and a speech feature sequence.
Therefore, we use visual text with monospace fonts in this work. Each
character is of a specified width $w$, height $h$, and font size $fs$.
Therefore, characters of length $n$ are transformed into an image whose width
is $nw$ and height is $h$. We extract sliced images from it using sliding
windows similar to a previous study [27]. The window is of a specified width
$wc$ and height $h$ extracted at intervals $s(=w)$. Here, $c$ represents the
number of characters in one sliced image, and adjusting $c$ leads to a change
in the preceding and following characters to be considered when obtaining
visual features from each piece of visual text. For example, if $c=3$,
convolution is performed including one character before and after. By
considering the number of preceding and following characters, reading and
prosody can be acquired that depend on the neighboring characters. Blank
images are padded to the left and right edges to create sliced images for the
number of characters. Finally, we generate $n$ sliced images of height $h$ and
width $wc$ from text.

###  3.3 Visual features

We extract visual features from the sliced images generated in Section 3.2.
The basic TTS uses the output from a text feature extractor. In comparison,
vTTS uses the output of a visual feature extractor instead. The architecture
of the extractor is shown in Figure 2(b) and is inspired by a machine
translation study using visual text [27]. The extractor mainly comprises $2$D
convolution, $2$D batch normalization [28], and the activation function ReLU
[29]. Finally, we apply a linear layer to the reshaped output to obtain a
visual feature for each character. The visual features are fed to the
FastSpeech2-inspired encoder.

The visual feature extractor utilizes the following information. Figure 3
shows examples of visual text with the following information.

  * •

Compositionality: In phonetic languages (e.g., Korean), the character or
combination of sub-characters determines the overall reading. The visual
feature extractor acquires the correspondence between a character (or sub-
character) and the reading. Therefore, even if OOV and rare characters emerge,
we expect our vTTS to be able to accurately predict the readings using the
visual features of the characters or sub-characters.

  * •

Emphasis attribute: The visual feature extractor distinguishes emphasized
typefaces (underline, bold, italic) from normal typefaces. This enables vTTS
to synthesize word-emphasized speech only from emphasized typefaces.

  * •

Emotional attribute: The visual feature extractor acquires the correspondence
between typeface and emotion, and it reflects the emotion in speech in
accordance with the typeface.

##  4 Experiments and Results

###  4.1 Experimental Setup

Language. We evaluated the performance of the proposed method on three
phonetic languages: Japanese (Hiragana), Korean (Hangul), and English (Roman
Alphabet). For Japanese, we converted Katakana and Kanji into $80$ Hiragana-
characters in advance. In Hiragana, one character corresponds to the specific
sound. For Korean, we used $1226$ Hangul-characters. In Hungul, the
combination of jamo determines the sound. For English, we applied lowercasing
to facilitate training.

Dataset. We used the JSUT [30] (Japanese), KSS [31] (Korean), and LJSpeech
[32] (English) corpora for naturalness comparison of vTTS and TTS as described
in Section 4.2.1. The training (validation) data sizes for these languages
were $8.3$ ($0.61$), $9.0$ ($0.38$), and $19$ ($0.89$) hours, respectively.
The test data sizes were $100$ samples for all languages. We used the JECS
corpus (see our project page), which contains noun-emphasized Japanese speech,
for the emphasis experiments described in Section 4.2.2. The training,
validation, and test data sizes were $0.412$ hours, $0.03$ hours, and $50$
samples, respectively. We used the manga2voice corpus [33], which contains
emotional speech, for the emotion experiment described in Section 4.2.3. The
training, validation, and test data sizes were $0.062$ hours, $0.003$ hours,
and $50$ samples, respectively. Since the training data sizes of the JECS and
manga2voice corpora were small, we fine-tuned a vTTS model pretrained using
the JSUT corpus. We used the KSS corpus in Section 4.2.4. The data sizes were
almost the same but slightly different from those in Section 4.2.1. See
Section 4.2.4 for details. All audio was downsampled to $22.05$ kHz. The
alignment between visual text and acoustic features was obtained through
forced alignment of visual text and a character sequence, and a character
sequence and a acoustic feature sequence.

Visual text. We used pygame333https://www.pygame.org/news to generate visual
text from text. For Japanese and English, we used the IPA typeface; for
Korean, the Gowun Batang typeface. Visual text of $w=30$ and $h=30$ was used
for all languages. $fs$ was set to $15$ for Japanese and Korean and $20$ for
English. This is the result of setting the text to be the appropriate size for
an image.

Model configuration. The model size and hyperparameters of the original
FastSpeech2 and FastSpeech2-inspired architecture in vTTS followed the open-
source implementation [34]444https://github.com/Wataru-
Nakata/FastSpeech2-JSUT. In the visual feature extractor, the $2$D convolution
used only one channel, and there was a padding of $1$, kernel size of $3$, and
stride of $1$ [27]. The dimension of the output of the visual feature
extractor was set to $256$, the same as the encoder hidden dimension of
FastSpeech2. We generated speech waveforms from mel spectrograms using the
pretrained HiFi-GAN [35]555https://github.com/jik876/hifi-gan.

Preliminary experiment. As described in Section 2.1, we used characters
instead of phonemes as input unit for FastSpeech2. We conducted a subjective
evaluation of this substitution under the same conditions as in Section 4.2.1.
The results showed that there was no significant difference in naturalness in
Japanese and English. On the other hand, the use of phonemes was better than
that of characters and visual text in Korean. This comparison is not the main
scope of this paper, but deep investigation will be our future work.

###  4.2 Results

####  4.2.1 Naturalness comparison between vTTS and TTS

Table 1: Results of MOS and $95\%$ confidence interval for naturalness. Bold means significantly better score than TTS. For all languages, vTTS was comparable to or better than TTS in naturalness. Lang. | TTS | vTTS
---|---|---
$c=1$ | $c=3$ | $c=5$
ja | $3.45\pm 0.09$ | $3.41\pm 0.09$ | $3.46\pm 0.09$ | $3.49\pm 0.10$
ko | $3.04\pm 0.16$ | $\mathbf{3.55\pm 0.15}$ | $3.18\pm 0.15$ | $3.01\pm 0.15$
en | $3.72\pm 0.10$ | $3.69\pm 0.10$ | $3.70\pm 0.11$ | $3.71\pm 0.10$

To compare our vTTS with conventional TTS, we conducted a mean opinion score
(MOS) evaluation on the naturalness of the synthetic speech. We used three
settings for vTTS, $c=1,3,5$, to investigate the impact of the window size.
Native speakers of each language participated in the evaluation. $150$ and
$200$ speakers recruited using our crowdsourcing system listened to $20$
speech utterances in Japanese and English, respectively. In contrast, $10$
speakers recruited by snow-ball sampling666We could not collect enough number
of native listeners in our crowdsourcing system listened to $160$ speech
utterances in Korean. The text content was kept consistent among different
methods so that all listeners examined only naturalness without other
interfering factors.

The results are shown in Table 1. For all languages, the naturalness of vTTS
was comparable to or better than that of TTS. Comparing the highest score of
vTTS ($c=5$ for Japanese and English, $c=1$ for Korean) with TTS, although
there was no significant difference for Japanese and English, there was a
significant difference for Korean ($p<0.05$). This means that the extraction
of visual features was effective for Hangul, whose readings are determined by
the combination of sub-characters.

It can also be seen that the appropriate window size differed depending on the
language. For English, there was no significant difference between $c=1,3,5$.
However, for Japanese, the naturalness with a sliding window of $c=5$ was
slightly better than that with $c=1$ ($p<0.10$), while for Korean, $c=1$ was
significantly better than $c=5$ ($p<0.001$). One possible reason is that the
number of phonemes expressed by one character differs depending on the
language. The following experiments used the best $c$ value for each language.

####  4.2.2 Emphasis attribute

We evaluated whether our vTTS can transfer the emphasis attribute in visual
text to speech. We trained the vTTS model using pairs of word-emphasized
speech and visual text. We used three types of word emphasis methods
(underline, bold, and italic) as shown in Figure 3(b). We trained the vTTS
model using each type, i.e., three vTTS models were trained. The trained
models try to synthesize speech with emphasis placed on specified words. We
prepared synthetic speech of not only emphasis groups (“Emphasis (*)”) but
also control groups (“Control (*)”). Their difference was in the input visual
text: word-emphasized visual text in the emphasis groups and non-emphasized
text in the control groups.

For the listening test, listeners listened to speech randomly selected from
seven kinds (three emphasis groups, three control groups, and ground-truth
natural speech) of speech utterances. The listeners answered with which word
was emphasized, from word (specifically, noun) candidates. $150$ native
speakers listened to $20$ speech utterances. The average number of candidates
per sentence was $3.6$; the chance rate was $0.299$.

The results are shown in Table 2. For all emphasis methods, the accuracy of
the emphasis groups was significantly better than that of the control groups
(${p<0.01}$). This shows that our vTTS properly recognized the emphasis
attribute in visual text. In addition, the accuracy of the emphasis groups was
close to that of the ground truth. There was no significant difference between
two emphasis methods (underline, bold) and the ground truth. These results
show that these two emphasis methods can accurately transfer the emphasis
attribute in visual text to speech, without additional emphasis labels and
architectures. One possible reason for the lower accuracy of the italic input
compared with the two other emphasis methods is that the shape of italic is
not as similar to that of normal typeface because of geometric transformation.

Table 2: Accuracy of perceived emphasized words. Our method can properly reflect emphasis attribute in visual text in speech. Speech | Accuracy
---|---
Ground truth | $0.960$
Emphasis (underline) | $0.933$
Emphasis (bold) | $0.898$
Emphasis (italic) | $0.877$
Control (italic) | $0.505$
Control (bold) | $0.400$
Control (underline) | $0.381$

####  4.2.3 Emotion attribute

Table 3: Confusion matrix between reference (true_*) and perceived (pred_*) emotion. Our method can properly reflect emotion attributes in visual text in synthetic speech. | pred_happy | pred_sad
---|---|---
true_happy | $0.795$ | $0.205$
true_sad | $0.114$ | $0.886$

We evaluated whether our method can transfer emotions evoked by certain
typefaces to speech. We trained the vTTS model using pairs of emotional speech
and typeface-decorated visual text. Before the training, we conducted
preliminary experiments to find typefaces that evoke emotions. As a result, we
decided to use the Koruri
typeface777https://github.com/Koruri/Koruri/releases/tag/20210720 and
Aiharahudemozikaisyo typeface888https://faraway.work/font.html, which evoke
joy and sadness, respectively. These are typefaces that more than $70$ percent
of evaluators thought of as corresponding emotion.

We trained one vTTS model using the above two typefaces. In the listening
test, listeners listened to the synthetic speech and answered the perceived
emotion. To avoid the listener from perceiving emotion from the linguistic
content, the content was common between emotions, and it was selected from the
“neutral” subset of the manga2voice corpus. $120$ listeners participated, and
each listener listened to $20$ speech utterances.

The results are shown in Table 3. When listening to speech synthesized from
the Koruri typeface, $79$% of the listeners answered that it sounded joyful,
and $89$% of the listeners answered that it sounded sad when listening to
speech synthesized from the Aiharahudemozikaisyo typeface. These scores are
comparable to TTS with emotion labels [9]. These results show that vTTS can
accurately transfer the emotion attribute in visual text to speech, without
additional emotion labels and architectures.

####  4.2.4 Speech synthesis from rare and OOV characters

Table 4: Results of MOS and $95\%$ confidence interval on naturalness. $\Delta$ denotes MOS decrease from “in-vocab.” Our method can synthesize more natural speech than TTS. | in-vocab |  rare ($\Delta$) |  OOV ($\Delta$)
---|---|---|---
TTS | $3.29\pm 0.16$ |  $2.32\pm 0.16$ ($-0.97$) |  $2.31\pm 0.20$ ($-0.98$)
vTTS | $3.58\pm 0.13$ |  $3.12\pm 0.16$ ($-0.46$) |  $2.95\pm 0.21$ ($-0.63$)
Table 5: Results on CER. $\Delta$ denotes CER increase from “in-vocab.” Our method can synthesize more intelligible speech than TTS. | in-vocab |  rare ($\Delta$) |  OOV ($\Delta$)
---|---|---|---
TTS | $0.120$ |  $0.194$ ($+0.074$) |  $0.255$ ($+0.135$)
vTTS | $0.080$ |  $0.114$ ($+0.034$) |  $0.163$ ($+0.083$)

Finally, we investigated the ability of vTTS to synthesize speech from OOV and
rare characters. A good example for this purpose is Korean because the
character compositionality determines the reading. We designed three test
sets: in-vocab, rare, and OOV. The “in-vocab” set consisted of only characters
appearing more than three times in the training data. Sentences in the “rare”
set included characters appearing less than three times in the training data,
and those in the “OOV” set included OOV characters. Each test set consisted of
$50$ sentences. For comparison, we trained the TTS model under the same
conditions as for the vTTS. The TTS model had an “unknown” symbol, and an OOV
character was encoded in the symbol. We conducted a MOS test on naturalness
and a dictation test. The listener listened to speech utterances randomly
selected from all test sets and then answered the speech naturalness in the
MOS test and dictated the text in the dictation test. The character error rate
(CER) was used in the dictation test. $10$ listeners participated in each
test. Each listener listened to $160$ and $64$ speech utterances in the MOS
test and dictation test, respectively.

The MOS results on naturalness are shown in Table 4. There was a significant
difference between vTTS and TTS in the “in-vocab,” “rare,” and “OOV” sets
($p<0.001$). For both TTS and vTTS, the scores decreased from “in-vocab” to
“rare” and “OOV,” but the changes in vTTS were smaller than those in TTS. The
dictation results for intelligibility are shown in Table 5. The overall
tendency was the same as the MOS test. There was a significant difference
between TTS and vTTS in the “in-vocab,” “rare,” and “OOV” sets ($p<0.001$).
Also, the CER increase for vTTS was also smaller than that for TTS.

From these results, we can say that our vTTS can capture character
compositionality for speech synthesis and synthesize intelligible speech even
from OOV and rare characters.

##  5 Conclusions

In this paper, we proposed vTTS, which generates speech waveforms from visual
text. Experimental results showed that 1) the basic performance of vTTS is
comparable to or better than that of TTS, 2) it can reflect several attributes
in visual text in speech, and 3) it can generate more natural speech from OOV
and rare characters. In future work, we plan to apply our method to a variety
of media such as manga or posters. We hope that this research will serve as a
first benchmark in the development of speech synthesis from visual text.

Acknowledgements: This work was supported by JST, Moonshot R&D Grant Number
JPMJPS2011 (for multi-language experiments), and JSPS KAKENHI 21H05054,
19H01116, 21H04900 (for basic technique).

## References

  * [1] A. v. d. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “WaveNet: A generative model for raw audio,” _arXiv preprint arXiv:1609.03499_ , Sep. 2016.
  * [2] Y. Wang, R. J. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. V. Le, Y. Agiomyrgiannakis, R. A. J. Clark, and R. A. Saurous, “Tacotron: Towards end-to-end speech synthesis,” in _Proc. INTERSPEECH_ , Aug. 2017, pp. 4006–4010.
  * [3] J. Shen, R. Pang, R. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. Skerrv-Ryan, R. Saurous, Y. Agiomvrgiannakis, and Y. Wu, “Natural TTS synthesis by conditioning WaveNet on mel spectrogram predictions,” in _Proc. ICASSP_ , Apr. 2018, pp. 4779–4783.
  * [4] Y. Ren, Y. Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y. Liu, “Fastspeech: Fast, robust and controllable text to speech,” in _Proc. NeurIPS_ , May. 2019, pp. 3165–3174.
  * [5] H. Strobelt, D. Oelke, B. C. Kwon, T. Schreck, and H. Pfister, “Guidelines for effective usage of text highlighting techniques,” _IEEE Transactions on Visualization and Computer Graphics_ , pp. 489–498, May. 2016.
  * [6] S. Choi, T. Yamasaki, and K. Aizawa, “Typeface emotion analysis for communication on mobile messengers,” in _Proc. the 1st International Workshop on Multimedia Alternate Realities_ , Oct. 2016, pp. 37–40.
  * [7] J. Caldwell, “Japanese typeface personalities: Are typeface personalities consistent across culture?” in _Proc. IEEE International Professonal Communication 2013 Conference_ , Jul. 2013, pp. 1–8.
  * [8] K. Nonaka, J. Saito, and S. Nakamura, “Music video clip impression emphasis method by font fusion synchronized with music,” in _Proc. Entertainment Computing and Serious Games_ , Nov. 2019, pp. 146–157.
  * [9] R. Liu, B. Sisman, and H. Li, “Reinforcement learning for emotional text-to-speech synthesis with improved emotion discriminability,” in _Proc. INTERSPEECH_ , Aug. 2021, pp. 4648–4652.
  * [10] C. Cui, Y. Ren, J. Liu, F. Chen, R. Huang, M. Lei, and Z. Zhao, “EMOVIE: A mandarin emotion speech dataset with a simple emotional text-to-speech model,” in _Proc. INTERSPEECH_ , Aug. 2021, pp. 2766–2770.
  * [11] Y. Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y. Liu, “FastSpeech 2: Fast and high-quality end-to-end text to speech,” in _Proc. ICLR_ , Sep. 2021.
  * [12] Y. Wang, D. Stanton, Y. Zhang, R. J. Skerry-Ryan, E. Battenberg, J. Shor, Y. Xiao, F. Ren, Y. Jia, and R. A. Saurous, “Style tokens: Unsupervised style modeling, control and transfer in end-to-end speech synthesis,” in _Proc. ICML_ , Mar. 2018, p. 5167–5176.
  * [13] S. Goto, K. Onishi, Y. Saito, K. Tachibana, and K. Mori, “Face2speech: Towards multi-speaker text-to-speech synthesis using an embedding vector predicted from a face image,” in _Proc. INTERSPEECH_ , Oct. 2020.
  * [14] T. G. Csap’o, L. V. T’oth, G. Gosztolya, and A. Mark’o, “Speech synthesis from text and ultrasound tongue image-based articulatory input,” _arXiv preprint arXiv:2107.02003_ , Jul. 2021.
  * [15] Y. Hono, K. Tsuboi, K. Sawada, K. Hashimoto, K. Oura, Y. Nankaku, and K. Tokuda, “Hierarchical Multi-Grained Generative Model for Expressive Speech Synthesis,” in _Proc. INTERSPEECH_ , Oct. 2020, pp. 3441–3445.
  * [16] G. Sun, Y. Zhang, R. J. Weiss, Y. Cao, H. Zen, and Y. Wu, “Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis,” in _Proc. ICASSP_ , May. 2020, pp. 6264–6268.
  * [17] M. Aso, S. Takamichi, N. Takamune, and H. Saruwatari, “Acoustic model-based subword tokenization and prosodic-context extraction without language knowledge for text-to-speech synthesis,” _Speech Communication_ , vol. 125, pp. 53–60, Dec. 2020.
  * [18] B. Li, Y. Zhang, T. Sainath, Y. Wu, and W. Chan, “Bytes are all you need: End-to-end multilingual speech recognition and synthesis with bytes,” in _Proc. ICASSP_ , May. 2019, pp. 5621–5625.
  * [19] R. Zens, F. J. Och, and H. Ney, “Phrase-based statistical machine translation,” in _Proc. Annual Conference on Artificial Intelligence_ , Sep. 2002, pp. 18–32.
  * [20] P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” _arXiv preprint arXiv:1607.04606_ , Jul. 2016\.
  * [21] R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” in _Proc. Association for Computational Linguistics_ , Aug. 2016, pp. 1715–1725.
  * [22] H. El Boukkouri, O. Ferret, T. Lavergne, H. Noji, P. Zweigenbaum, and J. Tsujii, “CharacterBERT: Reconciling ELMo and BERT for word-level open-vocabulary representations from characters,” in _Proc. COLING_ , Dec. 2020.
  * [23] L. Zhang and M. Komachi, “Neural machine translation of logographic language using sub-character level information,” in _Proc. the Third Conference on Machine Translation_ , Oct. 2018, pp. 17–25.
  * [24] F. Liu, H. Lu, C. Lo, and G. Neubig, “Learning character-level compositionality with visual features,” in _Proc. ACL_ , Jul. 2017, pp. 2059–2068.
  * [25] F. Dai and Z. Cai, “Glyph-aware embedding of Chinese characters,” in _Proc. EMNLP_ , Sep 2017, pp. 64–69.
  * [26] B. Sun, L. Yang, P. Dong, W. Zhang, J. Dong, and C. Young, “Super characters: A conversion from sentiment classification to image classification,” in _Proc. EMNLP_ , Oct. 2018, p. 309–315.
  * [27] E. Salesky, D. Etter, and M. Post, “Robust open-vocabulary translation from visual text representations,” in _Proc. EMNLP_ , Nov. 2021, pp. 7235–7252.
  * [28] S. Ioffe and C. Szegedy, “Batch Normalization: Accelerating deep network training by reducing internal covariate shift,” in _Proc. ICML_ , Jul. 2015, pp. 448–456.
  * [29] V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in _Proc. ICML_ , Jun. 2010, pp. 807–814.
  * [30] R. Sonobe, S. Takamichi, and H. Saruwatari, “JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis,” _arXiv preprint arXiv:1711.00354_ , Oct. 2017.
  * [31] K. Park, “KSS dataset: Korean single speaker speech dataset,” https://kaggle.com/bryanpark/korean-single-speaker-speech-dataset, 2018.
  * [32] K. Ito and L. Johnson, “The LJSpeech dataset,” https://keithito.com/LJ-Speech-Dataset/, 2017.
  * [33] S. Takamichi, Y. Saito, T. Nakamura, T. Kooriyama, and H. Saruwatari, “manga2voice: speech analysis towards audio synthesis from comic image,” in _Proc. 2020 Spring meeting of the Acoustical Society of Japan_ , Mar. 2020\.
  * [34] C.-M. Chien, J.-H. Lin, C.-y. Huang, P.-c. Hsu, and H.-y. Lee, “Investigating on incorporating pretrained and learnable speaker representations for multi-speaker multi-style text-to-speech,” in _Proc. ICASSP_ , Jun. 2021, pp. 8588–8592.
  * [35] J. Kong, J. Kim, and J. Bae, “HiFi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” in _Proc. NeurIPS_ , Oct. 2020, pp. 17 022–17 033.