# Hybrid Belief Pruning with Guarantees for Viewpoint-Dependent Semantic SLAM

Tuvy Lemberg and Vadim Indelman  The authors are with the Department of
Aerospace Engineering, Technion - Israel Institute of Technology, Haifa 32000,
Israel<EMAIL_ADDRESS><EMAIL_ADDRESS>This work
was partially supported by US NSF/US- Israel BSF, and by the Israeli Smart
Transportation Research Center (ISTRC).

###### Abstract

Semantic simultaneous localization and mapping is a subject of increasing
interest in robotics and AI that directly influences the autonomous vehicles
industry, the army industries, and more. One of the challenges in this field
is to obtain object classification jointly with robot trajectory estimation.
Considering view-dependent semantic measurements, there is a coupling between
different classes, resulting in a combinatorial number of hypotheses. A common
solution is to prune hypotheses that have a sufficiently low probability and
to retain only a limited number of hypotheses. However, after pruning and
renormalization, the updated probability is overconfident with respect to the
original probability. This is especially problematic for systems that require
high accuracy. If the prior probability of the classes is independent, the
original normalization factor can be computed efficiently without pruning
hypotheses. To the best of our knowledge, this is the first work to present
these results. If the prior probability of the classes is dependent, we
propose a lower bound on the normalization factor that ensures cautious
results. The bound is calculated incrementally and with similar efficiency as
in the independent case. After pruning and updating based on the bound, this
belief is shown empirically to be close to the original belief.

##  I Introduction

In robotics, the problem of simultaneous localization and mapping (SLAM)
concerned with estimating robot poses and, concurrently, reconstructing the
environment observed thus far by the robot [1, 2, 3, 4]. The SLAM problem is
essential for the robot to be able to navigate autonomously in uncertain or
unknown environments.

Recent advances in object recognition and classification enabled to
incorporate object classification and semantic features within the SLAM
framework, in order to improve both localization estimations and
classification [5, 6, 7]. In particular, semantic features are considered to
be more robust and indicative than geometric features and can be used in
ambiguous and repetitive environments, see e.g. [8]. In addition, semantic
SLAM provides a high level understanding that is essential for autonomous
decision making and advanced robotic capabilities.

Many semantic SLAM works make use of only the maximum likelihood of a
classifier output [9, 10, 11]. This approach reduces the classifier output to
one discrete variable and cannot represent the uncertainty of the classifier
given a specific image input. More sophisticated approaches consider that
classes are not fixed to the maximum likelihood. Yet, many of these approaches
assume that object’s class and object’s poses are independent, and as such,
solve two separable problems (e.g. [5, 6]). However, this corresponds to not
taking into account the inherent coupling between classifier output and the
viewpoint from which the object is seen.

In recent years, several works considered this coupling by using a viewpoint-
dependent semantic observation model. In particular, it has been shown that
utilizing such a model enhances disambiguation and localization in challenging
scenarios [7]. The resulting belief is hybrid, combining continuous variables
such as robot and object poses with discrete variables such as object classes.
Moreover, the statistical dependency between the classifier output and the
viewpoint creates a statistical dependency between classes of different
objects. Another source for this coupling is a dependent prior probability,
i.e. when the prior over classification variables of all objects is _not_
equal to the product of priors on each of the objects. Such a case is more
general as it allows to incorporate readily available statistical knowledge
(e.g. high chances to find a keyboard next to a monitor). Therefore, the
number of hypotheses is the number of class combinations, which increases
_exponentially_ with the number of objects. With a few dozen objects and a few
hundred classes, the computational complexity becomes prohibitively expensive.

A common way to handle this prohibitive computational complexity is by pruning
hypotheses that are considered with the lowest probability. Yet, after pruning
and renormalization, the resulting belief is overconfident in its hypotheses
with respect to the original belief. Overconfident probabilities may lead to
reckless and dangerous behavior. An autonomous vehicle that considers to
continue driving according to the overconfident classification may cause fatal
errors. Additionally, there is no indication of the significance of pruned
hypotheses after they have been pruned. Yet, such an indication can be of
prime importance for safe autonomy and the decision making process;
furthermore, it may be used to trigger a change in maintained hypotheses (e.g.
resurrection of a previously pruned hypothesis).

In this paper, we suggest two methods for maintaining probabilities after
pruning in a more realistic and conservative manner. We show that the in case
of an a-priori independent class probability, the original normalization
factor without pruning can be calculated very efficiently. By obtaining the
original normalization factor, we can retrieve the _exact_ probability of each
hypothesis separately. If the a-priori probability is dependent, we propose a
lower bound on the probability of the retained hypotheses. Applying both
methods, provides an indication of the probability of the pruned hypotheses.
Using the first method, the probability of the entire pruned set can be
computed. Using the second method, the probability of the entire pruned set
can be bounded from above. In terms of computational complexity, both methods
are similar to pruning.

##  II Related Work

It appears that most semantic SLAM methods can be divided into two types. In
the first type, the semantic observations are considered class-dependent and
viewpoint-independent. In the second type, the semantic observations are
considered to be class independent and viewpoint dependent. There are very few
studies which consider the semantic observations to be both class-dependent
and viewpoint-dependent. Part of the reason for this is the high computational
complexity involved in using this model.

Omidshafiei et al. [12] proposed a Dirichlet distribution as the noisy
semantic observation model for the classifier output. They developed a
Bayesian filter and showed that the resulting classification is more robust to
ambiguity. The semantic observations are assumed to be viewpoint-independent.
Nicholson et al. [13] used two-dimensional object detection and represented
objects as three-dimensional oriented quadratic surfaces. The semantic
observations are the object shape detections which are considered to be
viewpoint-dependent and class-independent; Classification took place
externally. Yang and Scherer, [10] proposed object SLAM fed by a single image
3D cuboid object detector - the semantic model, which is viewpoint-dependent
and class-independent. In Doherty et al. [5] for each new measurement, the
marginal posterior of the data association was computed. The posterior of the
data association was then used to compute the posterior of the remaining
parameters. As a result, they did not have to directly maintain a hybrid
belief. Moreover, they implemented this method within a factor graph in [14].
In both works, the semantic model is assumed to be view independent, thereby
avoiding the mentioned high computational complexity.

Both viewpoint-dependent and class-dependent models were recently studied. A
neural network was used by Kopitkov and Indelman to learn a viewpoint-
dependent measurement model of CNN classifier output features [15]. That study
focused on the learning procedure of both viewpoint-dependent and class-
dependent semantic model, and how to incorporate it as a factor in a factor
graph framework. Feldman and Indelman [8] learned a viewpoint-dependent model
of semantic measurements, and represent the model as a Gaussian process.
Tchuiev and Indelman [16] developed a method for sequential reasoning about
the posterior uncertainty of a semantic model. This method was later applied
to autonomous planning scenarios [17]. Feldman and Indelman [18] replaced
categorical class variables with latent continuous object representations.
This representation provides the potential to represent semantic information
in a more meaningful way than only the object’s class.

Bowman et al. [6] showed that a joint hybrid belief of trajectory, data
association, landmark locations, and landmark class can be maximized through
expectation-maximization (EM). In the case of scenarios containing many
classes and objects, the number of hypotheses through which the expectation
must pass is too large and therefore is still infeasible. Tchuiev et al. [7]
used a hybrid belief to infer also data association. Data association
hypotheses with small probabilities were pruned. They used a semantic model
within the factor graph, improving both classification and localization.
However, there is no guarantee that the updated belief after pruning will be
close to the original belief. Furthermore, due to the high computational
complexity of using such a model, they were limited to scenarios with very few
classes and objects.

##  III Problem Formulation and Notations

Suppose a robot travels in an unknown environment and receives observations
from various objects as it moves. Let $N$ denote the number of objects the
robot observes and $M$ the number of possible classes. Denote the robot’s pose
at time-step $k$ by $x_{k}$, and its trajectory from the beginning to time-
step $k$ by $x_{1:k}=\left\\{x_{1},\ldots,x_{k}\right\\}$. Let $x^{o}_{n}$
represent the pose of the $n$th object, and
$X^{o}=\left\\{x^{o}_{1},\ldots,x^{o}_{N}\right\\}$ the poses of all $N$
objects. The class of the $n$th object is $c_{n}$, and the concatenation of
all objects’ classes is $C=\left\\{c_{1},\dots,c_{N}\right\\}$.

The robot receives observations
$z_{k}=\left\\{z_{k,1},\ldots,z_{k,N_{k}}\right\\}$ at the $k$th time step,
where $N_{k}$ is the number of objects observed at time-step $k$. In general,
it is unknown which observation $z_{k,j}$ corresponds to which of the objects.
Let the data association (DA) random variable $\beta_{k,j}=n$ represent the
event that observation $z_{k,j}$ corresponds to the $n$th object, and,
$\beta_{k}=\\{\beta_{k,1},\ldots,\beta_{k,N_{k}}\\}$ is the concatenation of
all DAs at time-step $k$. In this paper, we assume that DA is known and will
extend to unknown DA in the future. Each observation $z_{k,j}$ consists of a
geometric part $z_{k,j}^{g}$ and a semantic part $z_{k,j}^{s}$, that are
assumed to be independent on each other. The observations model is given by

$\mathbb{P}\left(z_{k,j}\mid
x^{o}_{\beta_{k,j}},x_{i},c_{\beta_{k,j}}\right)=\\\
\mathbb{P}\left(z^{s}_{k,j}\mid
x^{o}_{\beta_{k,j}},x_{k},c_{\beta_{k,j}}\right)\mathbb{P}\left(z^{g}_{k,j}\mid
x^{o}_{\beta_{k,j}},x_{k}\right),$ (1)

where $\mathbb{P}\left(z^{s}_{k,j}\mid
x^{o}_{\beta_{k,j}},x_{k},c_{\beta_{k,j}}\right)$ is the viewpoint-dependent
semantic model, and $\mathbb{P}\left(z^{g}_{k,j}\mid
x^{o}_{\beta_{k,j}},x_{k}\right)$ is the geometric model, and both are assumed
to be given. Additionally, the actions $a_{0:k-1}$ and the motion model
$\mathbb{P}\left(x_{k}\mid x_{k-1},a_{k-1}\right)$ are also assumed to be
given.

Define $\mathcal{X}_{k}=\\{X^{o},x_{1:k}\\}$ as the concatenation of all
unknown continuous random variables; we shall refer to it as the state. Define
history at time-step $k$ as $H_{k}=\\{z_{1:k},a_{1:k-1}\\}$. The joint hybrid
belief over $\mathcal{X}_{k}$ and $C$ is defined as follows

$b\left[\mathcal{X}_{k},C\right]\triangleq\mathbb{P}\left(\mathcal{X}_{k},C\mid
H_{k}\right).$ (2)

We present a recursive derivation of the belief. Based on Bayes’ theorem, the
last observation can be pulled from the history

$\mathbb{P}\left(\mathcal{X}_{k},C\mid
H_{k}\right)\\!\\!=\\!\eta_{k}\mathbb{P}\\!\left(z_{k}\mid\mathcal{X}_{k},C,H_{k}^{-}\right)\\!\mathbb{P}\\!\left(\mathcal{X}_{k},C\mid
H_{k}^{-}\right)\\!,$ (3)

where $\eta_{k}\triangleq\mathbb{P}\left(z_{k}\mid H_{k}^{-}\right)^{-1}$ is
the normalization factor and $H_{k}^{-}$ is the the history without the last
observations, $z_{k}$. The observations are dependent only on the state at the
current time, therefore

$\mathbb{P}\left(z_{k}\mid\mathcal{X}_{k},C,H_{k}^{-}\right)=\prod_{j=1}^{N_{k}}\mathbb{P}\left(z_{k,j}\mid
x_{k},c_{\beta_{k,j}},x^{o}_{\beta_{k,j}}\right)$ (4)

Applying chain rule, the last term in (3) can be written as

$\mathbb{P}\left(\mathcal{X}_{k},C\mid
H_{k}^{-}\right)=\mathbb{P}\left(x_{k}\mid
a_{k-1},x_{k-1}\right)\mathbb{P}\left(\mathcal{X}_{k-1},C\mid H_{k-1}\right).$
(5)

This process can be repeated recursively, resulting in the following
formulation

$\tilde{b}_{k}\left[\mathcal{X}_{k},C\right]=\mathbb{P}_{0}\left(C\right)\mathbb{P}_{0}\left(\mathcal{X}_{0}\right)\prod_{i=1}^{k}\mathbb{P}\left(x_{i}\mid
a_{i-1},x_{i-1}\right)\\\ \times\prod_{j=1}^{N_{k}}\mathbb{P}\left(z_{i,j}\mid
x_{i},c_{\beta_{i,j}},x^{o}_{\beta_{i,j}}\right),$ (6)

where $c_{\beta_{i,j}}$ is the element of $C$ corresponding to index
$\beta_{i,j}$. By normalizing (6) we get the belief which is the joint
probability of $\mathcal{X}_{k}$ and $C$

$b_{k}\left[\mathcal{X}_{k},C\right]=\eta_{1:k}\tilde{b}_{k}\left[\mathcal{X}_{k},C\right],$
(7)

where the normalization factor $\eta_{1:k}$ is given by

$\eta_{1:k}^{-1}=\sum_{C\in\mathcal{C}}\intop_{\Omega_{\mathcal{X}_{k}}}\tilde{b}_{k}\left[\mathcal{X}_{k},C\right]d\mathcal{X}_{k}.$
(8)

The belief can be rearranged so that all observations related to the same
object are grouped together. Consider $I\left(n,k\right)$ as the set of
indices $\left<i,j\right>$ of observations associated with the $n$th object
that were observed up to time-step $k$. $I\left(n,k\right)$ can be defined
mathematically as follows

$I\left(n,k\right)=\left\\{\left<i,j\right>:\beta_{i,j}=n,1\leq i\leq
k,\>1\leq j\leq N_{i}\right\\}.$ (9)

The belief can then be reordered as follows

$b\left[\mathcal{X}_{k},C\right]=\eta_{1:k}\mathbb{P}_{0}\left(C\right)\mathbb{P}_{0}\left(X^{0}\right)\left(\prod_{i=1}^{k}\mathbb{P}\left(x_{i}\mid
a_{i-1},x_{i-1}\right)\right)\\\ \times\prod_{n=1}^{N}\prod_{i,j\in
I\left(n,k\right)}\mathbb{P}\left(z_{i,j}\mid x_{n}^{o},x_{i},c_{n}\right).$
(10)

Using a viewpoint-dependent semantic model, it is evident that classes of
different objects are dependent. There are two reasons for this. First, the
prior probability of the classes may be dependent, i.e.,
$\mathbb{P}_{0}\left(C\right)\neq\prod_{n=1}^{N}\mathbb{P}_{0}\left(c_{n}\right)$,
for example, near crosswalks we expect to see pedestrians. Second, because of
the viewpoint-dependent semantic model, the classes are dependent on the
poses, which are in turn dependent on each other, so the classes are
dependent.

In general, to infer $\mathcal{X}_{k}$ and $C$, it is necessary to go through
all possible combinations of classes, even when using an efficient algorithm
such as expectation maximization. The number of combinations, or hypotheses,
is $M^{N}$. As a result, all algorithms that consider all of the hypotheses
run at least at $O\left(M^{N}\right)$. Because of this, pruning the vast
majority of hypotheses is essential. Using the naive pruning approach, after
the pruning and renormalization, it is impossible to determine whether the
pruned hypotheses maintain a high probability. In naive pruning, the
probabilities of the remaining hypotheses are assumed to sum to one, meaning
that the probabilities of the pruned hypotheses are assumed to be zero.

Yet, it is irresponsible to assume that the probabilities of the pruned set
are zero. Consider that a robot operates in a human environment, and the robot
can do some potentially dangerous things only if it is confident that no human
is nearby. By using naive pruning, the robot will be overconfident in the
retained hypotheses. In the event that the correct hypothesis was pruned,
because there is no indication that the correct hypothesis was pruned, the
robot may assume that there is no human nearby and proceed to take the
dangerous action. In contrast, if the robot obtains the exact probability that
the correct hypothesis is pruned, or at least a lower bound on this
probability, the robot will know that it lacks sufficient confidence to take
the dangerous action and that the correct hypothesis was pruned.

Figure 1: Illustration of the pruned set $\Omega^{out}_{k}$ in white and the
retained set $\Omega^{in}_{k}$, in gray in a simple case of two objects. A
hypothesis consists of a class for each object. It is considered that
$|\Omega^{in}_{k}|\ll|\Omega^{out}_{k}|$.

##  IV Approach

We propose two alternative approaches to evaluating the normalization factor
after pruning. We consider two cases. In the first case, the prior probability
of the classes is independent, i.e.
$\mathbb{P}_{0}\left(C\right)=\prod_{n=1}^{N}\mathbb{P}_{0}\left(c_{n}\right)$.
Our study will demonstrate that in this case the original normalization factor
can be calculated with the same computational efficiency as the pruning
version of the normalization factor. Because we have the original
normalization factor, we can query the original probability of each
hypothesis. In this case, we know the real probabilities of the remaining
hypotheses, and if they are low, we should replace them.

In the second case, the prior probability of the classes is dependent, i.e.
$\mathbb{P}_{0}\left(C\right)\neq\prod_{n=1}^{N}\mathbb{P}_{0}\left(c_{n}\right)$.
In this case, we propose an upper bound on the probabilities of the pruned
hypotheses. Using this bound, we derive a lower bound on the normalization
factor. Using an upper bound on the probability of the pruned hypotheses, we
can determine whether there is a high probability hypothesis within the pruned
set. This bound is also computed with the same computational efficiency.

In both cases, we denote the set of maintained hypotheses by $\Omega^{in}_{k}$
and the rest of the (pruned) hypotheses by $\Omega^{out}_{k}$ (see
illustration in Fig. 1).

###  IV.I Independent Class Prior

In order to simplify notation, the unnormalized belief (6) can be divided into
two parts, one containing the semantic observations, and the other containing
the rest.

$\tilde{b}_{k}\left[\mathcal{X}_{k},C\right]=\\\
\left(\mathbb{P}_{0}\left(X^{0}\right)\prod_{i=1}^{k}\mathbb{P}\left(x_{i}\mid
a_{i-1},x_{i-1}\right)\prod_{j=1}^{N_{i}}\mathbb{P}\left(z^{g}_{i,j}\mid
x^{o}_{\beta_{i,j}},x_{i}\right)\right)\\\
\times\left(\mathbb{P}_{0}\left(C\right)\prod_{i=1}^{k}\prod_{j=1}^{N_{i}}\mathbb{P}\left(z^{s}_{i,j}\mid
x^{o}_{\beta_{i,j}},x_{i},c_{\beta_{i,j}}\right)\right).$ (11)

Define the geometric unnormalized belief as follows

$\tilde{b}^{g}_{k}\left[\mathcal{X}_{k}\right]\triangleq\mathbb{P}_{0}\left(\mathcal{X}_{0}\right)\prod_{i=1}^{k}\mathbb{P}\left(x_{i}\mid
a_{i-1},x_{i-1}\right)\prod_{j=1}^{N_{i}}\mathbb{P}\left(z_{i,j}^{g}\mid
x_{\beta_{i,j}}^{o},x_{i}\right),$ (12)

where $H_{k}^{g}$ is the history without the semantic observations. The
unnormalized belief is equal to the probability of $\mathcal{X}_{k}\mid
H_{k}^{g}$ multiplied by the normalization factor $\eta^{g}_{1:k}$. By
substituting (12) into (11), and reordering the equation as in (10), one
obtains

$\tilde{b}_{k}\left[\mathcal{X}_{k},C\right]=\tilde{b}^{g}_{k}\left[\mathcal{X}_{k}\right]\mathbb{P}_{0}\left(C\right)\prod_{n=1}^{N}\prod_{i,j\in
I\left(n,k\right)}\mathbb{P}\left(z^{s}_{i,j}\mid
x_{n}^{o},x_{i},c_{n}\right).$ (13)

Further simplifying the belief, we define $\psi_{k}$ as follows

$\psi_{k}(n,c,\mathcal{X}_{k})\triangleq\prod_{i,j\in
I\left(n,k\right)}\mathbb{P}\left(z^{s}_{i,j}\mid
x_{i},x_{n}^{o},c_{n}=c\right).$ (14)

It is the likelihood of the object’s class $c_{n}=c$ and the state, given the
semantic observations associated with the $n$th object. Substituting
$\psi_{k}$ into (13) results in

$\tilde{b}_{k}\left[\mathcal{X}_{k},C\right]=\tilde{b}^{g}_{k}\left[\mathcal{X}_{k}\right]\mathbb{P}_{0}\left(C\right)\prod_{n=1}^{N}\psi_{k}(n,c_{n},\mathcal{X}_{k}).$
(15)

The normalization factor is given in (8), and by substituting (15), we obtain

$\eta_{1:k}^{-1}=\sum_{C\in\mathcal{C}}\intop_{\Omega_{\mathcal{X}_{k}}}\tilde{b}^{g}_{k}\left[\mathcal{X}_{k}\right]\mathbb{P}_{0}\left(C\right)\prod_{n=1}^{N}\psi_{k}(n,c_{n},\mathcal{X}_{k})d\mathcal{X}_{k}.$
(16)

The above integration, (16), can be represented as expectation over
$\mathcal{X}_{k}\mid H_{k}^{g}$ up to the scale of the normalization factor
$\eta^{g}_{1:k}$ of the geometric belief, results in the following

$\eta_{1:k}^{-1}=(\eta^{g}_{1:k})^{-1}\sum_{C\in\mathcal{C}}\mathbb{P}_{0}\left(C\right)\mathbb{E}_{\mathcal{X}_{k}\mid
H_{k}^{g}}\left[\prod_{n=1}^{N}\psi_{k}(n,c_{n},\mathcal{X}_{k})\right].$ (17)

Since the normalized belief is required only for the probabilities of classes,
the state will be marginalized later. Thus, we could apply the same
expectation as in (17), and the normalization factor $\tilde{b}^{g}_{k}$ will
be eliminated. Until now, the derivation has been general and does not assume
independent priors and therefore applies also to the dependent case.

By using the independent property of the prior of $C$, and reorganizing (16),
we obtain

$\eta_{1:k}^{-1}=\mathbb{E}_{\mathcal{X}_{k}\mid
H_{k}^{g}}\left[\sum_{c_{1}=1}^{M}\dots\sum_{c_{N}=1}^{M}\prod_{n=1}^{N}\mathbb{P}_{0}\left(c_{n}\right)\psi_{k}(n,c_{n},\mathcal{X}_{k})\right].$
(18)

Our _key observation_ is that it is possible to substitute the sum and the
product, rewriting (18) as

$\eta_{1:k}^{-1}=\mathbb{E}_{\mathcal{X}_{k}\mid
H_{k}^{g}}\left[\prod_{n=1}^{N}\mathbb{P}_{0}\left(c_{n}=c\right)\sum_{c=1}^{M}\psi_{k}(n,c,\mathcal{X}_{k})\right],$
(19)

since each $c_{n}$ is independent of the other, inside the expectation. Using
(19), the computing complexity of the normalization factor is significantly
reduced.

Let us compare the running time of the naive approach in (17) and the
efficient approach in (19). In both cases, integration of the state and
calculation of $\psi_{k}(n,c,\mathcal{X}_{k})$ is required. Since this
calculation is common, it is not included in the analysis to follow. Consider
that the expectation over $\mathcal{X}_{k}$ is approximated by $N_{s}$ samples
drawn from $b_{k}^{g}[\mathcal{X}_{k}]$. Using the naive approach, one must
sum the unnormalized probabilities for all hypotheses, as shown in (17).
Therefore, the naive approach runs in $O(M^{N}\cdot N_{s})$. In contrast, the
computational complexity of the efficient approach (19) is $O\left(N\cdot
M\cdot N_{s}\right)$.

To summarize, using the exact normalization factor (17), the _exact_
probability can be computed for each of the maintained hypotheses in
$\Omega^{in}_{k}$. Moreover, the exact probabilities of hypotheses in
$\Omega^{in}_{k}$, provide the exact probability of
$\mathbb{P}({C\in\Omega^{out}_{k}})$, which can indicate if the correct
hypothesis was pruned. In Fig. 1, and illustration of $\Omega^{in}_{k}$ and
$\Omega^{out}_{k}$ is provided.

###  IV.II Dependent Class Prior

(a) Unnormalized belief

(b) Unnormalized sums and bound

(c) Normalized sums and bound

(d) Normalized belief

Figure 2:  The bound process is illustrated in the following figures. (2(a))
Unnormalized belief separated to $\Omega^{in}_{~{}}$ in the left (before the
dashed line) and $\Omega^{out}_{~{}}$ in the right. In (2(b)),
$\sum_{C\in\Omega^{in}_{~{}}}\tilde{b}[C]$ and $\tilde{U}$ are computed.
$\sum_{C\in\Omega^{out}_{~{}}}\tilde{b}[C]$ is not computed. In (2(c)), the
sums bound are normalized using $\underline{\eta}$. A comparison is shown
(2(d)) between the original normalized probabilities, the bound and the naive
pruning approach. Using naive pruning can lead to absurd overconfident
probabilities. Furthermore, it cannot indicate that a better hypothesis is in
the pruned set $\Omega^{out}_{~{}}$. In contrast, if there is a high
probability that the correct hypothesis was pruned, then using the upper bound
$U$ will provide an indication of this and will allow for a more conservative
and realistic normalization of the retained hypotheses. Suppose a hypothesis
in $\Omega^{in}_{~{}}$ has probability $\eta\tilde{b}[C]$ higher than $U$,
then we are guaranteed that this hypothesis has higher probability than any
hypothesis in $\Omega^{out}_{~{}}$.

In this case we consider the prior of the classes is dependent. Suppose we
have an upper bound $\tilde{U}_{k}$ that fulfills the following inequality

$\sum_{C\in\Omega^{out}_{k}}\tilde{b}_{k}\left[C\right]\leq\tilde{U}_{k}.$
(20)

The normalization factor can be bounded from below as

$\eta_{1:k}=\left(\sum_{C\in\mathcal{C}}\tilde{b}_{k}\left[C\right]\right)^{-1}\geq\left(\sum_{C\in\Omega^{in}_{k}}\tilde{b}_{k}\left[C\right]+\tilde{U}_{k}\right)^{-1}\triangleq\underline{\eta}_{1:k},$
(21)

where $\underline{\eta}_{1:k}$ is the normalization factor induced by the
bound. The probabilities of $C$ can be bounded from below

$b\left[C\right]=\eta_{1:k}\tilde{b}\left[C\right]\geq\underline{\eta}_{1:k}\tilde{b}\left[C\right],$
(22)

A lower bound for the class’s probability is a significant result. It enables
an autonomous robot to take cautious actions, and indicates whether
$\mathbb{P}({C\in\Omega^{out}_{k}})$ is high.

Furthermore, we can also bound the probability
$\mathbb{P}\left(C\in\Omega^{out}_{k}\mid H_{k}\right)$ from above.

$\mathbb{P}\left(C\in\Omega^{out}_{k}\mid
H_{k}\right)=\sum_{C\in\Omega^{out}_{k}}b\left[C\right]\\\
=1-\sum_{C\in\Omega^{in}_{k}}b\left[C\right]\leq
1-\sum_{C\in\Omega^{in}_{k}}\underline{\eta}_{1:k}\tilde{b}\left[C\right].$
(23)

This bound is also useful, since it indicates if it is likely that the correct
hypothesis was pruned. Fig. 2 illustrates the derivation of the bound clearly.

Next, we will derive the bound and present a method for updating it
efficiently. We begin with the simpler version, where the state is assumed to
be known, and then we generalize it to the case where it is unknown.

####  IV.II1 Derivation of $\tilde{U}_{k}$

We will begin by considering that the $\mathcal{X}_{k}$ is known and then
extend it to an unknown $\mathcal{X}_{k}$. The unnormalized belief of $C$ can
be derived from degenerating (6)

$\tilde{b}_{k}\left[C\right]=\mathbb{P}_{0}(C)\prod_{i=1}^{k}\prod_{j=1}^{N_{k}}\mathbb{P}\left(z^{s}_{i,j}\mid
x_{i},c_{\beta_{i,j}},x^{o}_{\beta_{i,j}}\right).$ (24)

We obtain the following belief by reorganizing (24) as was done in (10), and
plugging in $\psi_{k}(n,c,\mathcal{X}_{k})$

$\tilde{b}_{k}\left[C\right]=\mathbb{P}_{0}\left(C\right)\prod_{n=1}^{N}\psi_{k}(n,c_{n},\mathcal{X}_{k}).$
(25)

According to (20), we need to bound (26) from above

$\sum_{C\in\Omega^{out}_{k}}\tilde{b}_{k}\left[C\right]=\sum_{C\in\Omega^{out}_{k}}\mathbb{P}_{0}\left(C\right)\prod_{n=1}^{N}\psi_{k}(n,c,\mathcal{X}_{k}).$
(26)

Using the Cauchy-Schwarz inequality, (26) can be bounded from above as follows

$\sum_{C\in\Omega^{out}_{k}}\tilde{b}_{k}\left[C\right]\leq\left(\sum_{C\in\Omega^{out}_{k}}\mathbb{P}_{0}^{2}\left(C\right)\right)^{1/2}\left(\sum_{C\in\Omega^{out}_{k}}\prod_{n=1}^{N}\psi_{k}^{2}(n,c,\mathcal{X}_{k})\right)^{1/2}.$
(27)

Furthermore, we can use the Hölder’s inequality, which is an extension of the
Cauchy–Schwarz inequality. The Hölder’s inequality states that for any two
vectors $u,v$ in some inner product space, the following inequality holds

$\left|\left<u,v\right>\right|\leq\left\|u\right\|_{q_{1}}\cdot\left\|v\right\|_{q_{2}},$
(28)

for any $q_{1},q_{2}\geq 1$ satisfy $\dfrac{1}{q_{1}}+\dfrac{1}{q_{2}}=1$,
where $\left\|\cdot\right\|_{q}$ is the $L(q)$ norm. Therefore,

$\tilde{U}_{k}\left(\mathcal{X}_{k}\right)=\left(\sum_{C\in\Omega^{out}_{k}}\mathbb{P}_{0}^{q_{1}}\left(C\right)\right)^{1/q_{1}}\left(\sum_{C\in\Omega^{out}_{k}}\prod_{n=1}^{N}\psi_{k}^{q_{2}}(n,c,\mathcal{X}_{k})\right)^{1/q_{2}}.$
(29)

####  IV.II2 Efficient update of $\tilde{U}_{k}$

Two events can cause an update: a new observation is received, and a change in
the composition of the $\Omega^{in}_{k},\Omega^{out}_{k}$. In the former case,
the bound should be updated incrementally like the belief. In the latter case,
we avoid starting from scratch and re-use previous calculations.

Suppose that the number of elements in $\Omega^{in}_{k}$ is limited to
$N_{in}=\left|\Omega^{in}_{k}\right|\ll\left|\Omega^{out}_{k}\right|$. A sum
over $\Omega^{out}_{k}$ can be replaced with sum over $\mathcal{C}$ minus sum
over $\Omega^{in}_{k}$. Using this attribute, the bound can be rewritten as
follows

$\tilde{U}_{k}\left(\mathcal{X}_{k}\right)=\left(\sum_{C\in\mathcal{C}}\mathbb{P}_{0}^{q_{1}}\left(C\right)-\sum_{C\in\Omega^{in}_{k}}\mathbb{P}_{0}^{q_{1}}\left(C\right)\right)^{1/q_{1}}\\\
\times\left(\sum_{C\in\mathcal{C}}\prod_{n=1}^{N}\psi_{k}^{q_{2}}(n,c,\mathcal{X}_{k})-\sum_{C\in\Omega^{in}_{k}}\prod_{n=1}^{N}\psi_{k}^{q_{2}}(n,c,\mathcal{X}_{k})\right)^{1/q_{2}}.$
(30)

In order to simplify $\tilde{U}_{k}$ , the sums inside (30) can be defined as
following

$\displaystyle S^{0}_{\mathcal{C}}$
$\displaystyle\triangleq\sum_{C\in\mathcal{C}}\mathbb{P}_{0}^{q_{1}}\left(C\right)$
(31) $\displaystyle S^{0}_{\Omega^{in}_{k}}$
$\displaystyle\triangleq\sum_{C\in\Omega^{in}_{k}}\mathbb{P}_{0}^{q_{1}}\left(C\right)$
(32) $\displaystyle
S^{\psi_{k}}_{\Omega^{in}_{k}}\left(\mathcal{X}_{k}\right)$
$\displaystyle\triangleq\sum_{C\in\Omega^{in}_{k}}\prod_{n=1}^{N}\psi_{k}^{q_{2}}(n,c_{n},\mathcal{X}_{k})$
(33) $\displaystyle S^{\psi_{k}}_{\mathcal{C}}\left(\mathcal{X}_{k}\right)$
$\displaystyle\triangleq\sum_{C\in\mathcal{C}}\prod_{n=1}^{N}\psi_{k}^{q_{2}}(n,c,\mathcal{X}_{k}).$
(34)

In (33), the class of the $n$th object is the $n$th element of the vector $C$.
In light of definitions (31-34), the bound (30) can be rewritten as follows

$\tilde{U}_{k}\left(\mathcal{X}_{k}\right)=\left(S_{\mathcal{C}}^{0}-S_{\Omega^{in}_{k}}^{0}\right)^{1/q_{1}}\left(S_{\mathcal{C}}^{\psi_{k}}\left(\mathcal{X}_{k}\right)-S_{\Omega^{in}_{k}}^{\psi_{k}}\left(\mathcal{X}_{k}\right)\right)^{1/q_{2}}.$
(35)

#### Update due to new observation

Observing new observation $z_{k}$ will change $S^{\psi_{k}}_{\mathcal{C}}$,
$S^{\psi_{k}}_{\Omega^{in}_{k}}$ and $\psi_{k}$. As before, we exclude the
time complexity of computing $\psi_{k}$. The dependency on $\mathcal{X}_{k}$
is omitted here for simplicity of notation, keeping in mind that $\psi_{k}$ is
dependents on $\mathcal{X}_{k}$. The computation time for the update of
$\psi_{k}$ is $O(M\cdot N_{k})$. For the update of
$S_{\Omega^{in}_{k}}^{\psi_{k}}$, define

$\varphi_{k}(C)\triangleq\prod_{n=1}^{N}\psi_{k}(n,c=c_{n},\mathcal{X}_{k})\>\>\>C\in\Omega^{in}_{k}.$
(36)

It is clear that

$S_{\Omega^{in}_{k}}^{\psi_{k}}=\sum_{C\in\Omega^{in}_{k}}\varphi_{k}(C).$
(37)

$\varphi_{k}(C)$ updates as follows

$\varphi_{k}(C)=\varphi_{k-1}(C)\prod_{j=1}^{N_{k}}\mathbb{P}^{q_{2}}\left(z^{s}_{k,j}\mid
x_{\beta_{k,j}}^{o},x_{k},c=c_{\beta_{k,j}}\right).$ (38)

The computational time for updating $\varphi_{k}(C)$ is $O(N_{k})$, it should
be done for all $C\in\Omega^{in}_{k}$, therefore, the computation time for
updating $S_{\Omega^{in}_{k}}^{\psi_{k}}$ is $O(N_{k}N_{in})$.

The explicit formulation of $S^{\psi_{k}}_{\mathcal{C}}$ is given by

$S^{\psi_{k}}_{\mathcal{C}}=\sum_{C\in\mathcal{C}}\prod_{n=1}^{N}\psi_{k}(n,c,\mathcal{X}_{k})\\\
=\sum_{c_{1}=1}^{M}\cdots\sum_{c_{N}=1}^{M}\prod_{n=1}^{N}\psi_{k}(n,c=c_{n},\mathcal{X}_{k}).$
(39)

The sum and the product can be substituted for each other in this case

$S^{\psi_{k}}_{\mathcal{C}}=\prod_{n=1}^{N}\sum_{c_{n}=1}^{M}\psi_{k}(n,c=c_{n},\mathcal{X}_{k}).$
(40)

Define

$s^{\psi_{k}}_{n}\triangleq\sum_{c=1}^{M}\psi_{k}(n,c,\mathcal{X}_{k}).$ (41)

(41) can be inserted to $S^{\psi_{k}}_{\mathcal{C}}$

$S_{\mathcal{C}}^{\psi_{k}}=\prod_{n=1}^{N}s^{\psi_{k}}_{n}.$ (42)

The update affects only $s^{\psi_{k}}_{n}$ for $n\in\beta_{k}$. The
computation time for updating $s^{\psi_{k}}_{n}$ is $O(M)$ and for
$S_{\mathcal{C}}^{\psi_{k}}$ is $O\left(M\cdot N_{k}\right)$. The final
computation time for an observation-based update is $O(N_{k}\max(M,N_{in}))$.

#### Update due to change in $\Omega^{in}_{k},\Omega^{out}_{k}$

The second type of update is caused by a change in the composition of
$\Omega^{in}_{k},\Omega^{out}_{k}$, and affects only
$S_{\Omega^{in}_{k}}^{0},S_{\Omega^{in}_{k}}^{\psi_{k}}$. Consider adding a
new hypothesis, $C^{a}$, and removing existing hypothesis $C^{r}$ from
$\Omega^{in}_{k}$. The update will be performed as follows

$\displaystyle S_{\Omega^{in}_{k}}^{0}$ $\displaystyle\leftarrow
S_{\Omega^{in}_{k}}^{0}+\mathbb{P}_{0}^{q_{1}}(C^{a})$ (43)
$\displaystyle\varphi(C^{a})$
$\displaystyle=\prod_{n=1}^{N}\psi_{k}(n,c=c_{n}^{a})$ (44) $\displaystyle
S_{\Omega^{in}_{k}}^{\psi_{k}}$ $\displaystyle\leftarrow
S_{\Omega^{in}_{k}}^{\psi_{k}}+\varphi(C^{a}).$ (45)

The update involves computing $\varphi(C^{a})$, which is $O(N)$. Now the
removal of $C^{r}$ is done by

$\displaystyle S_{\Omega^{in}_{k}}^{0}$ $\displaystyle\leftarrow
S_{\Omega^{in}_{k}}^{0}-\mathbb{P}_{0}^{q_{1}}(C^{r})$ (46) $\displaystyle
S_{\Omega^{in}_{k}}^{\psi_{k}}$ $\displaystyle\leftarrow
S_{\Omega^{in}_{k}}^{\psi_{k}}-\varphi(C^{r}),$ (47)

which is done in $O(1)$.

###  IV.III Belief over unknown classes and state

The unnormalized belief over the classes and the state can be taken from (15).
According to (20), $\tilde{U}_{k}$ should satisfy

$\tilde{U}_{k}\geq\sum_{C\in\Omega^{out}_{k}}\intop_{\Omega_{\mathcal{X}_{k}}}\tilde{b}^{g}_{k}\left[\mathcal{X}_{k}\right]p_{0}\left(C\right)\prod_{n=1}^{N}\psi_{k}(n,c_{n},\mathcal{X}_{k})d\mathcal{X}_{k}.$
(48)

Integration can be replaced by expectation over $\mathcal{X}_{k}\mid
H_{k}^{g}$, as done in (18),

$\tilde{U}_{k}\geq\mathbb{E}_{\mathcal{X}_{k}\mid
H_{k}^{g}}\left[\sum_{C\in\Omega^{out}_{k}}p_{0}\left(C\right)\prod_{i=1}^{k}\prod_{j=1}^{n_{i}}\psi_{k}(n,c_{n},\mathcal{X}_{k})\right].$
(49)

The term inside the expectation of (49) is the unnormalized belief of
$C\mid\mathcal{X}_{k}$ given in (25). Therefore, it can also be bounded using
the bound in (35), thus

$\tilde{U}_{k}=\mathbb{E}_{\mathcal{X}_{k}\mid
H_{k}^{g}}\left[\left(S_{\mathcal{C}}^{0}-S_{\Omega^{in}_{k}}^{0}\right)^{1/q_{1}}\left(S_{\mathcal{C}}^{\psi_{k}}\left(\mathcal{X}_{k}\right)-S_{\Omega^{in}_{k}}^{\psi_{k}}\left(\mathcal{X}_{k}\right)\right)^{1/q_{2}}\right].$
(50)

$\left(S_{\mathcal{C}}^{0}-S_{\Omega^{in}_{k}}^{0}\right)^{1/q_{1}}$ is not
dependent on $\mathcal{X}_{k}$ so it can be taken out of the expectation. The
time complexity of computing the bound is the same as before multiplied by the
number of samples $N_{s}$.

##  V Experiments

In this section, we evaluate our methods using synthetic simulations. It is
important to note that we do not propose a new viewpoint-dependent model, but
only suggest that given such a model is utilized, our method can reduce
computational complexity substantially

A 2D environment is assumed with a robot that moves around, observing the
objects that it encounters along the way. Each object has orientation and
class type. Objects are randomly positioned throughout the environment, and
their classes are randomly selected. Initially, the robot does not know where
the objects are placed or to which class they belong. The geometric
observations are bearing measurements and the semantic observations are
synthetic classifier output. Based on the geometric measurements and the
motion model, the geometric belief is evaluated. The geometric belief is
assumed to be Gaussian. To calculate the full SLAM graph, we used the GTSAM
library with Python wrapper, resulting in a mean and a covariance of the
geometric belief. Throughout all the simulations, 100 samples are drawn from
the geometric belief.

For the robot’s prior pose, process noise, and geometric observation noise,
the covariances were $\Sigma_{0}=\text{diag}\left(0.01,0.01,0.001\right)$,
$\Sigma_{p}=\text{diag}\left(0.3,0.3,0.03\right)$, and
$\Sigma_{m}=\text{diag}\left(0.03\right)$, respectively. The dependent prior
probability of $C$ is randomly selected using the following procedure. A
matrix $A$ of size $\underset{N}{\underbrace{M\times M\dots\times M}}$ was
allocated. The elements of $A$ was sampled from the uniform distribution
$U[0.001,1]$. Then $A$ was divided by the sum of its elements. Thus, the prior
probability of $C$ is given by $\mathbb{P}_{0}(C)=A[C]$. In the case of an
independent prior, this procedure was used for each marginal.

In Fig. 3 we can see the scheme of the scene.

Figure 3:  Visualization of the scene. The triangles in red represent objects
that are oriented. The angles and classes were chosen at random. The robot
moves counterclockwise, starting from the lower left side.

The distribution of the semantic observations is logit-normal distribution,
thus,

$z_{k,j}\sim
LN\left(\mu\left(x^{o}_{\beta_{k,j}},x_{k},c\right),\Sigma\right),$ (51)

where $\Sigma=I_{M}\sigma_{s}^{2}$ and $\sigma^{s}=0.015$. The same
distribution is used in equation [17] as well. It is possible to interpret
this distribution as the output of a classifier since the samples are vectors
of positive elements that sum to 1. For simplicity of notation, consider that
$z_{k,j}$ is associated with the $n$th object, thus $\beta_{k,j}=n$. The
semantic measurement model is defined as follows

$\mu\left(x^{o}_{n},x_{k},c\right)=\begin{cases}e_{c}\cdot
h\left(x^{o}_{n},x_{k}\right)&c\in[1,M-1]\\\ 0_{M-1}&c=M\end{cases},$ (52)
$h\left(x^{o}_{n},x_{k}\right)=\left(1-\cos\left(\theta\right)\right)\min\left\\{\frac{1}{dist(x_{k},x_{n}^{o})},\frac{1}{2}\right\\},$
(53)

where $\mathbf{e}_{c}\in\mathbb{R}^{M-1}$is a vector of zeros with a single
element at the $c$th position equal to one, and $dist(\cdot,\cdot)$ is the
euclidean distance. $\theta$ is the relative angle, calculated from the
relative pose $x^{rel}=x_{k}\ominus x_{n}^{o}$. The mean
$\mu\left(x^{o}_{n},x_{k},c\right)$ is characterized by the following
features: under the the hypothesis $c=M$, the mean of $z_{k,j}$ is
$\frac{1}{M}$. For the other hypotheses, as the camera gets closer to the
object and becomes oriented towards the front of the object, the probability
that the right element of $z_{k,n}$ will respond to object’s class is
increased.

We compare between the original probability without pruning, the naive pruning
approach, the bound approach, and pruning using the original normalization in
the independent prior case.

The sets $\Omega^{in}_{k},\Omega^{out}_{k}$ are predefined and are constant.
The size of $\Omega^{in}_{k}$ is $N_{in}=8$. A true hypothesis
$C^{(t)}=(c_{1}^{(t)},\ldots,c_{N}^{(t)})$ is predefined as well. The semantic
observations are sampled from the probability $\mathbb{P}\left(z_{k,n}\mid
x_{n}^{o},x_{k},c_{n}^{(t)}\right)$ described in (51), (52), and (53).

The results are divided into two cases: when $C^{(t)}\in\Omega^{in}_{k}$ and
where $C^{(t)}\in\Omega^{out}_{k}$. In order to determine whether the system
is confident in its best hypothesis, we look at the maximum probability in
$\Omega^{in}_{k}$,
$\underset{C\in\Omega^{in}_{k}}{\text{max}}\left\\{b[C]\right\\}$, over time.
The most likely hypothesis is the same across all methods because the
difference between them is only the normalization factors, but different
probability is provided by each approach. The same sets
$\Omega^{in}_{k},\Omega^{out}_{k}$ used for all methods.

Both Fig. 5(a) and Fig. 5(b) illustrate the scenario of classes with dependent
priors. The naive exact belief, naive pruning, and bound are compared. The
number of objects is 5 and number of classes is 3, the total number of
hypotheses is $3^{5}=243$. Even when considering such a small number of
classes and objects, the number of hypotheses is not so small.

Fig. 5(a) shows the case for $C^{(t)}\in\Omega^{in}_{k}$. The naive pruning
approach is very confident in its hypothesis from the beginning to the end.
The naive exact approach, which provides the exact probability of this
hypothesis, is less confident. The bound, which bounds the exact probability
from below (22), following the exact approach throughout the simulation.
Although the bound is a lower bound on the exact belief, it remains close to
the exact belief even when the latter reaches a high probability.

Fig. 5(b), shows the case for $C^{(t)}\in\Omega^{out}_{k}$. Throughout the
simulation, the exact probability is close to zero and therefore also the
bound. By contrast, the naive pruning approach is too confident, assuming that
the correct hypothesis is in $\Omega^{in}_{k}$.

The independent prior case is considered in figures Fig. 4(a) and Fig. 4(b).
For the independent prior case, we used $N=5$ and $M=100$. In this scenario
the number of hypotheses is $10^{10}$. In this case, we compare the efficient
exact belief with naive pruning and the bound approach. Despite the fact that
the bound is not required in this case, we wish to evaluate how it will work
when many classes are involved.

In this case, the results appear to be the same as in the dependent case. In
Fig. 4(a), the naive approach is overconfident. In the exact belief we see
that only after several time-steps the belief gain confidence, and the bound
is follows the exact belief. According to Fig. 4(b), the naive approach is
absurdly overconfident, while the exact belief and the bound approach indicate
that the probability is zero.

In figures Fig. 6(a) and Fig. 6(b) we compare the runtime of naive pruning,
naive exact, the efficient exact and the bound approach. For each simulation,
one hundred trails were performed and the mean value of the runtime was taken.

In Fig. 6(a), show the runtime versus the number of objects $N$. The number of
classes is $M=2$. It appears that the efficient exact and the bound approach
have the same runtime as the naive pruning approach. In Fig. 6(b), the runtime
is plotted against the number of classes $M$. The number of objects is $N=3$.
In comparison to a naive pruning approach, there is a slight increase in
runtime for the efficient exact and bound approaches. Given the significant
improvement in accuracy and reliability, we consider the efficient exact and
bound approaches to be worthwhile.

(a)

(b)

Figure 4: The maximal probability of the maintained hypotheses. Considering an
_independent_ prior, where (a) $C^{(t)}\in\Omega^{in}_{k}$, and (b)
$C^{(t)}\in\Omega^{out}_{k}$.

(a)

(b)

Figure 5: The maximal probability of the maintained hypotheses. Considering an
_dependent_ prior, where (a) $C^{(t)}\in\Omega^{in}_{k}$, and (b)
$C^{(t)}\in\Omega^{out}_{k}$.

(a)

(b)

Figure 6: (a) Running time versus number of objects $N$. The number of classes
is $M=2$. The computational complexity of both the bound and the efficient
method for the independent case original are the same as pruning. (b) Running
time versus number of classes $M$. The number of object is $N=3$. Here, the
naive pruning running time is slightly better.

##  VI Conclusions

In the context of semantic SLAM, we explore the pruning procedure often used
when the size of the alphabet of the hybrid belief is exponentially large, in
the case where the semantic observation model is dependent on the relative
position of the object. In this scenario, the number of hypotheses is
exponential in the number of objects, and therefore, not feasible in a real-
time framework. Due to this reason, the pruning procedure is used, but
following pruning and renormalization, the resulting belief is overconfident,
disregards the possibility that the true hypothesis is pruned. When the prior
over classification variables of different objects is independent, we
demonstrated that the normalization factor can be calculated efficiently,
allowing us to query the exact probability of each hypothesis individually. In
the dependent case, using our method, we bound the probability that the true
hypothesis was pruned, and, as a result, we obtain a more accurate and
conservative estimate of the original belief, with the guarantee that the
original belief over classification variables without pruning is equal or
higher than our belief after pruning. Future research might explore the
minimization of the upper bound $U_{k}$ with respect to $\Omega^{in}_{k}$ as a
method to decide which hypotheses to maintain and which to prune.

## References

  * [1] R. Smith, M. Self, and P. Cheeseman, “Estimating uncertain spatial relationships in Robotics,” in _Autonomous Robot Vehicles_ , I. Cox and G. Wilfong, Eds. Springer-Verlag, 1990, pp. 167–193.
  * [2] J. Leonard and H. Durrant-Whyte, “Mobile robot localization by tracking geometric beacons,” _IEEE Trans. Robot. Automat._ , vol. 7, no. 3, pp. 376–382, 1991.
  * [3] H. Durrant-Whyte, S. Majunder, S. Thrun, M. de Battista, and S. Scheding, “A Bayesian algorithm for simultaneous localization and map building,” in _Proceedings of the 10th International Symposium of Robotics Research_ , 2001\.
  * [4] S. Thrun, _Simultaneous Localization and Mapping_. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 13–41.
  * [5] K. Doherty, D. Fourie, and J. Leonard, “Multimodal semantic slam with probabilistic data association,” in _2019 international conference on robotics and automation (ICRA)_. IEEE, 2019, pp. 2419–2425.
  * [6] S. Bowman, N. Atanasov, K. Daniilidis, and G. Pappas, “Probabilistic data association for semantic slam,” in _IEEE Intl. Conf. on Robotics and Automation (ICRA)_. IEEE, 2017, pp. 1722–1729.
  * [7] V. Tchuiev, Y. Feldman, and V. Indelman, “Data association aware semantic mapping and localization via a viewpoint-dependent classifier model,” in _IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS)_ , 2019.
  * [8] Y. Feldman and V. Indelman, “Bayesian viewpoint-dependent robust classification under model and localization uncertainty,” in _IEEE Intl. Conf. on Robotics and Automation (ICRA)_ , 2018.
  * [9] S. Pillai and J. Leonard, “Monocular slam supported object recognition,” in _Robotics: Science and Systems (RSS)_ , 2015.
  * [10] S. Yang and S. Scherer, “Cubeslam: Monocular 3-d object slam,” _IEEE Transactions on Robotics_ , vol. 35, no. 4, pp. 925–938, 2019.
  * [11] R. F. Salas-Moreno, R. A. Newcombe, H. Strasdat, P. H. Kelly, and A. J. Davison, “Slam++: Simultaneous localisation and mapping at the level of objects,” in _IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)_ , 2013, pp. 1352–1359.
  * [12] S. Omidshafiei, B. T. Lopez, J. P. How, and J. Vian, “Hierarchical bayesian noise inference for robust real-time probabilistic object classification,” _arXiv preprint arXiv:1605.01042_ , 2016.
  * [13] L. Nicholson, M. Milford, and N. Sünderhauf, “Quadricslam: Dual quadrics from object detections as landmarks in object-oriented slam,” _IEEE Robotics and Automation Letters (RA-L)_ , vol. 4, no. 1, pp. 1–8, 2018.
  * [14] K. J. Doherty, D. P. Baxter, E. Schneeweiss, and J. J. Leonard, “Probabilistic data association via mixture models for robust semantic slam,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 1098–1104.
  * [15] D. Kopitkov and V. Indelman, “Robot localization through information recovered from cnn classificators,” in _IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS)_. IEEE, October 2018.
  * [16] V. Tchuiev and V. Indelman, “Inference over distribution of posterior class probabilities for reliable bayesian classification and object-level perception,” _IEEE Robotics and Automation Letters (RA-L)_ , vol. 3, no. 4, pp. 4329–4336, 2018.
  * [17] ——, “Epistemic uncertainty aware semantic localization and mapping for inference and belief space planning,” _arXiv preprint arXiv:2105.12359_ , 2021.
  * [18] Y. Feldman and V. Indelman, “Towards self-supervised semantic representation with a viewpoint-dependent observation model,” in _Workshop on Self-Supervised Robot Learning, in conjunction with Robotics: Science and Systems (RSS)_ , July 2020.


[1]Yoshiyuki Tagawa

[1]Department of Mechanical Systems Engineering, Graduate School of
Engineering, Tokyo University of Agriculture and Technology, 2-24-16 Naka-cho,
Koganei, 184-8588, Tokyo, Japan

2] Department of Mechanical Engineering, Nippon Institute of Technology, 4-1
Gakuendai, Miyashiro-machi, Minamisaitama-gun, 345-8501, Saitama, Japan

# A phase diagram of the pinch-off-type behavior of impulsively-induced
viscoelastic liquid jets

Asuka Hosokawa  Kyota Kamamoto  Hiroya Watanabe  Hiroaki Kusuno  Kazuya U.
Kobayashi<EMAIL_ADDRESS>*  [

###### Abstract

In this study, we systematically investigate the jet behaviors of viscoelastic
liquids, focusing on the region of high velocity and high viscoelasticity,
which has not been investigated in previous studies. We generate viscoelastic
jets using an impulsive force and categorize the resulting jets into two
types: pinch-off jets (jets that break up during elongation after ejection)
and no-pinch-off jets (jets that pull back to the nozzle after maximum
elongation or jets that return without elongation after ejection). We then
propose criteria to characterize these regions using the Weissenberg number
and the Reynolds number, which are dimensionless numbers composed of the
rheological properties of the solution and the initial conditions of the jet
injection. It is found that pinch-off jets occur at $Re\gtrsim 23.4Wi$ in
regions of high elasticity $Wi\gtrsim 10$, and at $Re\gtrsim 250$ in regions
of low elasticity $Wi\lesssim 10$. We also show that the phase diagram can be
explained by modeling a focused jet using the finitely extensible non-linear
elastic dumbbell model with the Chilcott–Rallison closure approximation (FENE-
CR).

###### keywords:

Viscoelasticity, Liquid jet, Jet Breakup, FENE dumbbell fluids

##  1 Introduction

Liquid jets have been studied in the past from a variety of perspectives due
to their significance in fundamental research and their widespread occurrence
[1]. Practical applications of liquid jets have also been demonstrated in
inkjet printing [2], pharmaceutical sprays [3], and agricultural irrigation
[4].

In order to develop these applications, there is a need to elucidate the
breakup process of liquid jets. Previous studies have shown that the addition
of polymers to liquid jets can significantly change this process [5]. In
particular, in inkjet printing, the addition of polymers has a significant
effect on the splitting of liquid filaments generated from the nozzle [6]. The
addition of small amounts of high molecular weight polymers can inhibit the
formation of undesirable satellite droplets, thus improving print clarity. On
the other hand, a ‘bungee-jumper jet’ is observed when the polymer
concentration is high [7]. This refers to a phenomenon in which a jet is
generated, maximally elongated, and then pulled back to the initial interface
due to the elasticity of the liquid. Such a jet interferes with the detachment
of droplets produced by the nozzle and with the coating of the liquid.
Therefore, various studies have been conducted to clarify and control the
bungee-jumper jet phenomenon using numerical simulations [7, 8] and
experiments [9, 6].

However, these studies have been limited to the injection of low-velocity
($<O(10^{0})$ m/s) and low-viscosity ($<20$ mPa$\cdot$s) solutions used in
conventional inkjet printing [10]. In recent years, next-generation printing
technologies such as 3D printing [11] and artificial cell printing [12] have
been developed. Viscoelastic liquids are among the non-Newtonian fluids used
in these printing techniques, and there is a need to understand the behavior
of jets with a wide range of viscoelastic properties to further develop next-
generation printing technologies. Therefore, it is essential to investigate
jet behavior at high velocities ($\geq O(10^{1})$ m/s) and in the highly
viscoelastic regime ($De>O(10^{1})$). Here, $De$ is the Deborah number, a
dimensionless number that indicates the ratio of the elastic and inertia-
capillary time scales.

One technique for producing high-velocity jets is the focused liquid jet
injection technique [13, 14]. A focused jet is a jet in which the jet tip
diameter is smaller than the jet bottom diameter [15]. A well-known example is
the Worthington jet, which is generated when a solid sphere or droplet impacts
the surface of a liquid bath [16]. [17] developed a method to generate focused
liquid jets by having a liquid-filled container free fall and collide with the
floor, providing a rapid acceleration at the concave gas-liquid interface.
This method can generate a focused jet easily and is highly controllable [18].
[19] developed a method of jet generation by inserting a small tube inside a
container. This method can eject a liquid with a viscosity 50 times higher
than that from conventional methods (1000 mPa$\cdot$s).

The aim of this study is to elucidate jet behaviors at high velocity and high
viscoelasticity. For this purpose, a high-velocity focused liquid jet is
generated using an impulsive force and observed using a high-speed camera. We
also use polymers of different molecular weights and concentrations to help
the viscoelastic liquids achieve high viscoelasticity. By analyzing the
observed jets in detail and classifying them using two dimensionless numbers,
we attempt to understand the jet behavior of viscoelastic liquids.

##  2 Focused jet generation

###  2.1 Jet generator using a test tube

A schematic diagram of the experimental setup is shown in Fig. 1. A test tube
(A-16.5, Maruemu, inner diameter $2r=14.3$ mm) is filled with liquid (liquid
height $L=32$ mm) and suspended by an electromagnet. When the electromagnet is
switched off, the test tube falls freely. During this time, the gas-liquid
interface inside the test tube develops a concave shape due to the capillary
force. When the test tube collides with the floor, the liquid is accelerated
over a very short time, and as a result, a focused liquid jet is formed from
the gas-liquid interface [18].

In this experiment, the initial velocity $U_{0}$ of the fluid is controlled by
varying the drop height of the test tube ($H$ = 3–300 mm) and the floor
material (SS400, nitrile rubber). The initial velocity $U_{0}$ is the sum of
the impact velocity $V_{0}$ of the test tube to the floor and the bounce
velocity $V^{\prime}$ of the test tube from the floor, expressed as in Eq. 1
[20]. In this case, the impact velocity $V_{0}$ of the test tube on the floor
is given by Eq. 2, which is derived from the energy conservation law, where
$g$ is the gravitational acceleration. The bounce velocity $V^{\prime}$ of the
test tube is given by Eq. 3, where $H^{\prime}$ is the bounce velocity of the
test tube 10 ms after impact ($t$ = 0 ms).

$U_{0}=V_{0}+V^{\prime}.$ (1)

$V_{0}=\sqrt{2gH}.$ (2)

$V^{\prime}=\frac{H^{\prime}}{t}.$ (3)

The jet formation is captured using a high-speed camera (FASTCAM SA-X,
Photron) and a backlight (White Led Backlight, Phlox) at 30,000 fps.

Figure 1: The experimental setup for generating a focused jet using a falling
test tube

###  2.2 Jet generator using a double-layer narrow tube

We use an injection device with a capillary tube to generate jets using the
impulsive force [19]. A schematic diagram of the experimental setup is shown
in Fig. 2. In this experiment, a glass capillary tube (Fuji Science Industry,
inner diameter $2r=2$ mm) is inserted into a container partially filled with
liquid and sealed outside the tube. We control the pressure of the sealed air
outside the tube using a syringe in order to push the liquid interface inside
the capillary near to the bottom of the container and thus change the position
of the liquid surface inside the capillary $L_{cap}$. The liquid level inside
the tube is kept deeper than that outside the tube. An electromechanical
device (ShinDengen solenoids, KGK power supply) is used to apply a vertical
upward impulsive force at a velocity $V_{0}$. The liquid inside the tube is
rapidly accelerated and a focused jet is generated from the inside of the
tube. The initial velocity $U_{0}$ of the generated jet varies depending on
the ratio of the distance between the gas-liquid interface outside the tube
$L_{top}$ and the liquid surface position inside the tube $L_{cap}$. The
initial velocity $U_{0}$ is calculated using the following equation [19]:

$U_{0}=\frac{L_{top}}{L_{cap}}V_{0}.$ (4)

The jet formation is captured using a high-speed camera and a backlight at
30,000–60,000 fps.

Figure 2: The experimental setup for generating a focused jet using the
impact-induced jet generator. The capillary tube is inserted into the liquid,
with the liquid level inside the tube set deeper than that outside the tube

##  3 Properties of the liquid solutions

Polymer solutions were used as the viscoelastic liquids in the experiments
(Table 1). We used two types of polymers: polyethylene oxide (Sigma-Aldrich,
molecular weight $M_{w}$ = 0.4, 2, 5, and 8 M (= million), polymer
concentration $c$ = 0.5, 1, and 2 wt$\%$) and polyacrylamide (Sigma-Aldrich,
$M_{w}$ = 5–6 M, $c$ = 0.2, 0.4, and 0.8 wt$\%$). The solutions were prepared
by adding polyethylene oxide (PEO) and polyacrylamide (PAM) to pure water and
aqueous-glycerin solution (glycerol: 70 wt%, water: 30 wt%, G70W30) as
solvents and stirring at 650 rpm, 60 ∘C for 24 to 48 hours.

Table 1: The physical properties of the solutions | Solutions | Density $\rho$ | Surface tension | Relaxation time
---|---|---|---|---
| [kg/m3] |  $\sigma$ [mN/m] |  $\lambda$ [ms]
PEO | 0.4M 1wt% | 999.7 | 64.0 | 1.9
2M 1wt% | 998.3 | 64.1 | 12.4
5M 0.5wt% | 996.4 | 62.8 | 35.4
5M 1wt% | 998.0 | 62.0 | 57.2
8M 1wt% | 981.4 | 60.6 | 207.6
8M 2wt% | 999.1 | 62.1 | 301.4
PAM | 5-6M 0.2wt% | 995.2 | 72.5 | 3.3
5-6M 0.4wt% | 1002.8 | 63.0 | 4.6
5-6M 0.8wt% | 1000.0 | 71.0 | 16.5
5-6M 0.4wt% (G70W30) | 1176.9 | 66.4 | 91.0
|  |  |  |

Polymer solutions generally exhibit shear-thinning behavior, in which the
viscosity decreases with an increasing strain rate. The shear viscosities of
the PEO and PAM solutions as measured with a shear rheometer (MCR-302, Anton
Paar) are shown in Fig. 3. The dashed line in Fig. 3 represents the confidence
limit of the measured shear viscosity due to the torque sensitivity of the
shear rheometer [21]. From Fig. 3, the shear viscosity $\mu_{s}$ of both the
PEO and PAM solutions decreases with an increasing shear rate $\dot{\gamma}$,
indicating shear-thinning properties. In both solutions, the higher the
molecular weight and concentration, the greater the shear viscosity and the
greater the slope of the shear curve. The higher the molecular weight and
concentration, the greater the viscoelasticity and thus, the stronger the
shear-thinning property [22].

Figure 3: The shear viscosity for the PEO and PAM solutions

Viscoelastic liquids also have spring-like elastic properties. We use the
relaxation time $\lambda$ as a physical property value to express the
magnitude of the elasticity. $\lambda$ is calculated using dripping-onto-
substrate capillary breakup elongation rheometry (DoS-CaBER), which is a
method for evaluating the extensional rheology of viscoelastic liquids. This
method enables the measurement of the relaxation times of low-viscoelastic
liquids, which are difficult to measure with rheometers [23, 24]. Figure 4
shows the experimental DoS-CaBER setup. First, the sample is pushed out of a
syringe to form a droplet at the tip of the nozzle (outer diameter
$2R_{0}=1.27$ mm, inner diameter $2R_{i}=0.97$ mm). When the droplet adheres
to the glass plate with its own weight, the liquid spreads on the glass plate
and forms a liquid thread due to the surface tension. The liquid thread
becomes thin and shreds gradually. The force that resists the surface tension
acting on the liquid thread differs depending on the solution, and the elastic
force is dominant in viscoelastic liquids (in the elasto-capillary regime)
[25]. The time evolution equation of the liquid thread radius $R$ formed in
this case is expressed as [26]:

$\frac{R(t)}{R_{0}}{\propto}\exp(-\frac{t}{3\lambda}),$ (5)

where $R_{0}$ is the nozzle radius and $t$ is the time since the liquid thread
was formed. Therefore, the relaxation time $\lambda$ of the sample can be
calculated from the time evolution of the liquid thread radius $R$.

An example analysis of the liquid relaxation time $\lambda$ using DoS-CaBER is
shown in Fig. 5. The horizontal axis is the time $t$ and the vertical axis is
the dimensionless liquid thread radius $R/R_{0}$, which is the liquid thread
radius $R$ divided by the nozzle radius $R_{0}$. The plot shows a 5M 1wt$\%$
PEO solution. Since the elongation rate is constant in the elasto-capillary
regime, the elongation rate is calculated from the DoS-CaBER experimental
results and the elasto-capillary regime is determined. An exponential
approximation is made in this region, and the relaxation time $\lambda$ is
derived using Eq. 5.

Figure 4: The experimental setup for the dripping-onto-substrate capillary
breakup extensional rheometry: DoS-CaBER Figure 5: The non-dimensional radius
(the ratio of the neck radius to the nozzle radius) as a function of time for
5M 1wt% PEO shown on a semi-log plot. The dashed lines show the fitted region
obtained using Eq. 5

The relaxation times $\lambda$ for the PEO and PAM solutions are calculated
using the same procedure and the results are shown in Table 1. The relaxation
time $\lambda$ increases with an increasing concentration $c$ and molecular
weight $M_{w}$ for both the PEO and PAM solutions. In a previous study, it was
reported that the relaxation time $\lambda$ increases with increasing polymer
concentration $c$ and molecular weight $M_{w}$, and we believe that this
result is consistent [22]. In addition, when comparing 2M 1wt% and 5M 0.5wt%
PEO solutions, $\lambda$ is larger for 5M 0.5wt%. This is because the
molecular weight $M_{w}$ makes a larger contribution to the relaxation time
$\lambda$ than the concentration $c$ [6].

##  4 Results and Discussions

In this experiment, we have succeeded in identifying two different types of
jets through systematic experiments by varying the magnitude of the
viscoelasticity and initial velocity.

The typical behaviors of the generated viscoelastic liquid jets are shown in
Fig. 6. Note that the $t=0$ ms time of the test tube impacting the floor is
used as the reference time, and the time evolution from that point is shown in
Fig. 6. Figure 6(a) shows images of a jet generated by the ejection of a 0.4M
1wt% PEO solution at an initial velocity of $U_{0}=0.84$ m/s. The jet ejected
from the interface has a shape in which the tip diameter is smaller than the
tail diameter, indicating that it is a focused jet. After ejection, the
focused jet is elongated vertically upward, and the tip is pinched off at
$t=63$ ms. Thus, we define this type of jet as a ‘pinch-off jet’. Figure 6(b)
shows images of a jet generated by ejecting an 5M 1 wt% PEO solution at an
initial velocity of $U_{0}=2.9$ m/s. The jet is elongated after ejection from
the interface and reaches its maximum length at $t=23$ ms. After that, the jet
returns to the initial interface. It should be noted that the jet is prevented
from being pinched off despite the large impact force applied in Fig. 6(b)
compared to that in Fig. 6(a). This phenomenon was also observed in a previous
study as a ‘bungee-jumper jet’ when the elasticity of the solution is large
[10], and it is believed that the elastic force contributes to the pulling
back of the jet. We define such a jet that pulls back to the initial interface
after maximum elongation as a ‘no-pinch-off jet’. Finally, Fig. 6(c) shows
images of a jet generated by ejecting a 8M 1 wt% PEO solution at an initial
velocity of $U_{0}=1.2$ m/s. The jet is ejected from the interface but returns
to the initial interface without elongation. The reason for this is presumed
to be that the initial velocity is smaller in Fig. 6(c) than in Fig. 6(b),
i.e., the jet is not given a sufficient impulsive force to generate a jet.
Such jets are also defined as ‘no-pinch-off jets’.

Figure 6: Viscoelastic liquid jets can be divided into two typical types: (a)
pinch-off jets and (b)(c) no-pinch-off jets. (a) Images of a jet generated by
the ejection of a 0.4M 1wt% PEO solution at an initial velocity of
$U_{0}=0.84$ m/s. The jet is ejected from the interface and extends vertically
upward, and the tip is pinched off after a certain time. (b) Images of a jet
generated by the ejection of an 5M 1wt% PEO solution at an initial velocity of
$U_{0}=2.9$ m/s. The jet is elongated after ejection from the interface and,
after maximum elongation, pulled back to the initial interface. (c) Images of
a jet generated by the ejection of a 8M 1wt% PEO solution at an initial
velocity of $U_{0}=1.2$ m/s. The jet is ejected from the interface but returns
to the initial interface without elongation

These changes in jet behavior are considered to be caused by the balance
between the viscoelasticity of the liquid and the inertial force that
dominates during jet extension. We organize our results in terms of two
dimensionless numbers. One is the Weisenberg number $Wi$, which represents the
ratio of the elastic force to the viscous force, and is defined as

$Wi=\frac{\lambda U_{0}}{r}.$ (6)

The other is the Reynolds number $Re$, which represents the ratio of the
inertial force to the viscous force, and is defined as

$Re=\frac{\rho U_{0}r}{\mu_{s}},$ (7)

where $\rho$ is the density of the solution. Since the polymer solution used
in this experiment has shear-thinning properties, we calculate the strain rate
$\dot{\gamma}$ from the elongation behavior of the jet
($\dot{\gamma}=O(10^{2}$-$10^{3})$) and assume the shear viscosity $\mu_{s}$
at that strain rate as the representative viscosity.

The results of organizing the jet behaviors using the Weisenberg number $Wi$
and the Reynolds number $Re$ are shown in Fig. 7. The error bars in Fig. 7
indicate the error range associated with the estimation of the shear viscosity
$\mu_{s}$. The higher the viscoelasticity of the liquid, the greater the
shear-thinning property, resulting in a larger error in the region of high
$Wi$.

Figure 7 shows that the jet behavior can be divided into two regions, pinch-
off and no-pinch-off jets, depending on $Wi$ and $Re$. The fact that the
results can be organized by parameters related to the physical properties of
the solution and the initial conditions of jet injection suggests that these
parameters have a significant effect on the behavior of the viscoelastic
liquid jets. Furthermore, the experimental results are well-organized in a
unified manner regardless of the type of polymer (PEO or PAM). Therefore, the
effect of the type of polymer on the jet behavior is considered to be small in
this study. In the following, we will discuss the jet behavior in detail in
two regions: 1) the region with $Wi\gtrsim 10$ and 2) the region with
$Wi\lesssim 10$.

Figure 7: A phase diagram of the impulsively-induced jet behavior of
viscoelastic liquids with a Weissenberg number $Wi$ and Reynolds number $Re$.
The outer frame of the plot indicates the jet type: red indicates a pinch-off
jet and blue indicates a no-pinch-off jet. The markers in the plot indicate
the type of solution: a lack of markers indicates PEO solutions, markers
indicate PAM solutions, and gray markers indicate an aqueous-glycerin
solution, which is a Newtonian fluid. The inner figure shows the region with
low $Wi$ ($Wi\leq 0.6$) in detail

###  4.1 The region with $Wi\gtrsim 10$

Figure 7 shows that in the region with $Wi\gtrsim 10$, a no-pinch-off jet
occurs in the region with high $Wi$ and a pinch-off jet occurs in the region
with low $Wi$. It should be noted that even for a sufficiently high $Re$,
$Re\gtrsim O(10^{3})$, the pinching off of the jet is inhibited in the region
with high $Wi$ ($Wi\gtrsim 0(10^{3})$). Since the elasticity increases with
increasing $Wi$, this phenomenon is considered to be caused by the elasticity
of the viscoelastic liquids.

To understand this phenomenon, we model a jet using a viscoelastic model
following previous studies [27, 28]. The effect of elasticity on the jet
behavior can be evaluated by introducing a constitutive equation that accounts
for elastic stress [29]. There are various models, but here we use the
finitely extensible non-linear elastic dumbbell model with the
Chilcott–Rallison closure approximation, known as the FENE-CR model [30]. The
FENE-CR model is one of the simple constitutive equations used for for the
rheological properties of polymer solutions.

Assuming an incompressible fluid, the continuity equation and Navier–Stokes
equation can be expressed using the fluid velocity $\mathbf{u}$, pressure $p$,
and stress tensor $\boldsymbol{\sigma}$ as

$\nabla\cdot\mathbf{u}=0,$ (8)

$\rho\frac{D\mathbf{u}}{Dt}=-\nabla p+\nabla\boldsymbol{\sigma}.$ (9)

In the FENE-CR model, the stress is given by

$\boldsymbol{\sigma}=2\mu_{s}\mathbf{E}+Gf(\mathbf{A-I}),$ (10)

$\frac{D\mathbf{A}}{Dt}=\nabla\mathbf{u}\cdot\mathbf{A}+\mathbf{A}\cdot(\nabla\mathbf{u})\rm{{}^{T}}-\mathit{\frac{f}{\lambda}}(\mathbf{A-I}),$
(11)

where $\mathbf{E}=(\nabla\mathbf{u}+\nabla\mathbf{u}\rm{{}^{T}}$)/2 is the
strain rate tensor, $\mathbf{G}$ is the elastic modulus, $\mathbf{I}$ is the
unit matrix, $\mathbf{A}$ is the conformation tensor related to the
deformation of the polymer chains, and $f$ is a nonlinear function that
enforces finite extensibility. Using the elongation limit of the polymer chain
$L$, $f$ is expressed as

$f=(L^{2}-tr(\mathbf{A})/L^{2})^{-1}.$ (12)

Since the tip of the jet is faster than the tail, we assume that the inertia
at the tip of the jet affects the jet behavior, and propose a simple model as
shown in Fig. 8. We define the initial condition to be $t=t_{r}$, which is the
time when the jet tip is ejected by a distance $r$ (=representative length)
from the initial interface in Fig. 8(a). The jet velocity at this time is
$U_{r}$. The jet then decelerates to a velocity $U$, and the jet tail deforms
uniformly with time from an initial length $r$ and then increases to $z$ at
time $t$ (Fig. 8(b)). The volume of fluid at the tip $\Omega_{tip}$ and at the
tail $\Omega_{tail}$ are assumed constant.

Figure 8: A simplified model of the viscoelastic jet

When viscous and elastic forces act on the jet, the equation of motion of the
tip is given by

$\rho\Omega_{tip}\frac{dU}{dt}=-\frac{\Omega_{tail}}{z}\bigg{(}\frac{3\mu_{s}U}{z}+Gf(A_{zz}-A_{rr})\bigg{)}.$
(13)

Assuming an axisymmetric flow, Eq. 11 can be expanded into an axial component
$A_{zz}$ and a radial component $A_{rr}$, which can be expressed as follows:

$\frac{dA_{zz}}{dt}=\bigg{(}\frac{2U}{z}-\frac{f}{\lambda}\bigg{)}A_{zz}+\frac{f}{\lambda},$
(14)

$\frac{dA_{rr}}{dt}=-\bigg{(}\frac{U}{z}+\frac{f}{\lambda}\bigg{)}A_{rr}+\frac{f}{\lambda}.$
(15)

Furthermore, assuming that the axial elongation of the polymer chain is
sufficiently large ($A_{zz}\gg 1$, $A_{zz}\gg A_{rr}$), we can write
$A_{zz}\sim(z/r)^{2}e^{-t/\lambda}$. Assuming that the elastic deformation is
smaller than the elongation limit ($A_{zz}\ll L^{2}$ → $f\sim 1$), it can be
expressed as $A_{rr}\sim 0$. Therefore, the deceleration $\Delta U$ at
position $z$ ($z>r$) is obtained by integrating Eq. 13 at time $t$ (Fig.
8(b)).

$\rho\Omega_{tip}\mathit{\Delta}U(z)=-\Omega_{tail}\bigg{(}3\mu_{s}\bigg{(}\frac{1}{r}-\frac{1}{z}\bigg{)}+\frac{G}{r^{2}}\int_{r}^{z}ze^{-\frac{t}{\lambda}}dt\bigg{)}.$
(16)

Assuming $U=0$ when $z\rightarrow\infty$ (Fig. 8(c)),

$\rho\Omega_{tip}U_{r}=\Omega_{tail}{}\bigg{(}\frac{3\mu_{s}}{r}+\frac{U_{r}}{r^{2}}G\lambda^{2}\bigg{)}.$
(17)

When the elastic term is sufficiently large compared with the viscous term,
Eq. 17 can be transformed into

$\rho\Omega_{tip}U_{r}=\Omega_{tail}\frac{U_{r}}{r^{2}}G\lambda^{2}.$ (18)

Assuming that the jet tip diameter $d_{tip}$ and the tail diameter $d_{tail}$
satisfy $d_{tip}\sim\alpha r$ and $d_{tail}\sim r$ in the initial state (Fig.
8(a)), then $\Omega_{tip}\sim 4\pi(\alpha r/2)^{3}/3$, $\Omega_{tail}\sim\pi
r^{3}/4$. Note that $\alpha$ is a constant. Furthermore, assuming
$G\sim\mu_{s}\lambda$ and $U_{r}\propto U_{0}$, and substituting these into
Eq. 18, we obtain

$Re\sim\frac{3}{2\alpha^{3}}Wi.$ (19)

For $Re\lesssim 3/(2\alpha^{3})Wi$, the jet is pulled back by the elastic
force, resulting in a no-pinch-off jet, and for $Re\gtrsim 3/(2\alpha^{3})Wi$,
a pinch-off jet occurs due to the lack of elastic force. The experimental
value of $\alpha$ is $\alpha\sim 0.4$. From the above, it is suggested that
the thresholds of the no pinch-off jet and pinch-off jet in Fig. 7 can be
expressed by the following equation.

$Re\sim 23.4Wi.$ (20)

Figure 9: The jet behavior of aqueous-glycerin solution (G70W30). (a) A jet
ejected from the interface and extended vertically upward, with the tip
pinched off after a certain time when $Re\sim 370$. (b) A jet ejected from the
interface but returned to the initial interface without being pinched off when
$Re\sim 230$

###  4.2 The region with $Wi\lesssim 10$

From Fig. 7, no-pinch-off jets are observed for $Re\lesssim 250$ and pinch-off
jets are observed for $Re\gtrsim 250$ when $Wi\lesssim 10$. This is considered
to be because the elasticity of the solution has little effect on the jet
behavior in the low $Re$ region, and the jet behaves like a Newtonian fluid.
In a previous study, the effect of viscosity on the jet velocity of a
Newtonian fluid was investigated [19]. It was reported that a viscous boundary
layer develops on the wall surface at $Re\lesssim 200$, which prevents the
focusing effect of the flow. Therefore, the lower $Re$ is in the present
study, the more the viscous boundary layer develops on the wall surface and
near the free surface, which prevents jet extension.

Therefore, in Eq. 17, when the viscous term is sufficiently large compared to
the elastic term, the equation can be transformed into

$\rho\Omega_{tip}U_{r}=\Omega_{tail}\frac{3\mu_{s}}{r}.$ (21)

Assuming $d_{tip}\propto r$ and $d_{tail}\sim r$ as before, $\Omega_{tip}$ and
$\Omega_{tail}$ can be calculated and substituted into Eq. 21. Also, assuming
$U_{r}\propto U_{0}$, Eq. 21 can be transformed into

$Re\sim\beta,$ (22)

where $\beta$ is a constant. For $Re\lesssim\beta$, a no-pinch-off jet is
expected to occur because the viscosity prevents the jet extension, and for
$Re\gtrsim\beta$, a pinch-off jet is expected to occur because the inertial
force is sufficient.

To confirm that the elasticity of the solution does not affect the jet
behavior in this region, we investigated the jet behavior of a Newtonian
fluid, aqueous-glycerin solution (G70W30) where the density $\rho$, viscosity
$\mu_{s}$, and surface tension $\sigma$ of the glycerin solution are
$\rho=1175.4$ [kg/m3], $\mu_{s}=0.0169$ [Pa $\cdot$ s], and $\sigma=68.4$
[mN/m], respectively. Here, the range of the surface tension for the
viscoelastic liquids and aqueous-glycerin solution used in this experiment is
$\sigma$ = 62.0–72.5 [mN/m]. Therefore, the effects of surface tension
(Rayleigh instability) on the jet behavior are considered to be almost
equivalent. Figure 9 shows a typical image of a G70W30 jet. As shown in Fig.
9(a), when $Re\sim 370$, the jet is pinched off at the tip after extension and
a pinch-off jet is generated. On the other hand, when $Re\sim 230$, as shown
in Fig. 9(b), the jet returns to the initial interface without being pinched
off after ejection, so a no-pinch-off jet is generated. We determined from the
experiments that the jet behaviors of the Newtonian fluid transitioned from
no-pinch-off to pinch-off at $Re\sim 250$ (Fig. 7). Therefore, in Eq. 22, the
$\beta$ obtained from the experiments is $\beta\sim 250$, and the threshold in
the region of low $Wi$ is

$Re\sim 250.$ (23)

In Fig. 7, the no-pinch-off and pinch-off jets can be generally distinguished
using Eq. 20 (solid line) and Eq. 23 (dashed line). Therefore, it is concluded
that the jet behavior can be classified through modeling with the FENE-CR
model.

##  5 Conclusion

In this study, we have conducted systematic experiments on jet behaviors in
the high-viscoelasticity and high-velocity regime, which have not been
investigated before. We injected a polymer solution with a wide range of
viscoelasticities as a focused jet and observed two characteristic behaviors.
These behaviors are: 1) After injection in the vertically upward direction,
the jet is elongated and after a certain amount of time, the tip is pinched
off (pinch-off jet). 2) After injection, the jet is elongated and after
maximum elongation, it is pulled back to the initial interface, or after
injection, it returns to the interface without elongation (no-pinch-off jet).
It is suggested that these jet behaviors can be predicted using the Weisenberg
number $Wi$ and the Reynolds number $Re$, which are dimensionless numbers
composed of the rheological properties of the solution and the initial
conditions of the jet injection. Furthermore, we modeled this phenomenon using
a viscoelastic model, the FENE-CR model, and obtained good agreement with the
experimental results.

We believe that this study provides significant insight into the
viscoelasticity of jets with a focused geometry. Furthermore, it will be
useful for clarifying the effects of viscoelasticity on the breakup processes
of various liquid jets.

## Declarations

Conflict of interest The authors declare that they have no conflict of
interest.

Ethics approval Not applicable, because this article does not contain any
studies with human or animal subjects.

Informed consent Not applicable.

Funding This work was supported by JSPS KAKENHI Grant Number JP20H00223.

Author Contributions AH: investigation, experiments, writing original draft,
and editing. KK: discussion of organizing experimental results. HW: discussion
of modeling. HK: discussion of modeling. KUK: design of study, supervision,
and supporting for original draft. YT: design of study and supervision.

## References

  *   * Eggers and Villermaux [2008] Eggers, J., Villermaux, E.: Physics of liquid jets. Rep. Prog. Phys. 71(3), 036601 (2008) https://doi.org/10.1088/0034-4885/71/3/036601
  * Derby [2010] Derby, B.: Inkjet Printing of Functional and Structural Materials: Fluid Property Requirements, Feature Stability, and Resolution. Annu. Rev. Mater. Res. 40(1), 395–414 (2010) https://doi.org/10.1146/annurev-matsci-070909-104502
  * Villermaux et al. [2004] Villermaux, E., Marmottant, P., Duplat, J.: Ligament-Mediated Spray Formation. Phys. Rev. Lett. 92, 074501 (2004) https://doi.org/10.1103/PhysRevLett.92.074501
  * Kincaid et al. [1996] Kincaid, D.C., Solomon, K.H., Oliphant, J.C.: Drop Size Distributions for Irrigation Sprinklers. Trans. ASABE 39(3), 839–845 (1996) https://doi.org/%****␣sn-article.bbl␣Line␣100␣****10.13031/2013.27568
  * Clasen et al. [2006] Clasen, C., Eggers, J., Fontelos, M.A., Li, J., McKinley, G.H.: The beads-on-string structure of viscoelastic threads. J. Fluid Mech. 556, 283–308 (2006) https://doi.org/10.1017/S0022112006009633
  * Bazilevskii et al. [2005] Bazilevskii, A.V., Meyer, J.D., Rozhkov, A.N.: Dynamics and Breakup of Pulse Microjets of Polymeric Liquids. Fluid Dyn. 40(3), 376–392 (2005) https://doi.org/10.1007/s10697-005-0078-4
  * Morrison and Harlen [2010] Morrison, N.F., Harlen, O.G.: Viscoelasticity in inkjet printing. Rheol. Acta 49, 619–632 (2010) https://doi.org/10.1007/S00397-009-0419-Z
  * Torres and Fonte [2022] Torres, P., Fonte, C.: Flow regimes of a single pulse of inelastic and elastic liquids from a round capillary tube. Chem. Eng. Sci. 248(B) (2022) https://doi.org/10.1016/j.ces.2021.117245
  * Turkoz et al. [2018] Turkoz, E., Perazzo, A., Kim, H., Stone, H.A., Arnold, C.B.: Impulsively Induced Jets from Viscoelastic Films for High-Resolution Printing. Phys. Rev. Lett. 120, 074501 (2018) https://doi.org/10.1103/PhysRevLett.120.074501
  * Franco-Gómez et al. [2021] Franco-Gómez, A., Onuki, H., Yokoyama, Y., Nagatsu, Y., Tagawa, Y.: Effect of liquid elasticity on the behaviour of high-speed focused jets. Exp. Fluids 62, 1–15 (2021) https://doi.org/10.1007/s00348-020-03128-w
  * Truby and Lewis [2016] Truby, R.L., Lewis, J.A.: Printing soft matter in three dimensions. Nature 540(7633), 371–378 (2016) https://doi.org/10.1038/nature21003
  * Noor et al. [2019] Noor, N., Shapira, A., Edri, R., Gal, I., Wertheim, L., Dvir, T.: 3D Printing of Personalized Thick and Perfusable Cardiac Patches and Hearts. Adv. Sci. 6(11), 1900344 (2019) https://doi.org/10.1002/advs.201900344
  * Tagawa et al. [2012] Tagawa, Y., Oudalov, N., Visser, C.W., Peters, I.R., Meer, D., Sun, C., Prosperetti, A., Lohse, D.: Highly Focused Supersonic Microjets. Phys. Rev. X 2, 031002 (2012) https://doi.org/%****␣sn-article.bbl␣Line␣250␣****10.1103/PhysRevX.2.031002
  * Peters et al. [2013] Peters, I.R., Tagawa, Y., Oudalov, N., Sun, C., Prosperetti, A., Lohse, D., Meer, D.: Highly focused supersonic microjets: numerical simulations. J. Fluid Mech. 719, 587–605 (2013) https://doi.org/10.1017/jfm.2013.26
  * Gordillo et al. [2020] Gordillo, J.M., Onuki, H., Tagawa, Y.: Impulsive generation of jets by flow focusing. J. Fluid Mech. 894, 3 (2020) https://doi.org/10.1017/jfm.2020.270
  * Worthington [1908] Worthington, A.M.: A Study of Splashes. Longmans, Green, and Company, London (1908)
  * Antkowiak et al. [2007] Antkowiak, A., Brémond, N., Dizes, S.L., Villermaux, E.: Short-term dynamics of a density interface following an impact. J. Fluid Mech, 577, 241–250 (2007) https://doi.org/10.1017/S0022112007005058
  * Kiyama et al. [2016] Kiyama, A., Tagawa, Y., Ando, K., Kameda, M.: Effects of a water hammer and cavitation on jet formation in a test tube. J. Fluid Mech. 787, 224–236 (2016) https://doi.org/10.1017/jfm.2015.690
  * Onuki et al. [2018] Onuki, H., Oi, Y., Tagawa, Y.: Microjet Generator for Highly Viscous Fluids. Phys. Rev. Appl. 9, 014035 (2018) https://doi.org/10.1103/PhysRevApplied.9.014035
  * Kiyama et al. [2014] Kiyama, A., Yuto, N., Tagawa, Y.: The generation of a liquid jet induced by a pressure impulse (experimental study on the jet velocity). Trans. JSME (in Japanese) 80(814), 0151–0151 (2014) https://doi.org/10.1299/transjsme.2014fe0151
  * Gaillard et al. [2019] Gaillard, A., Roché, M., Lerouge, S., Gay, C., Lebon, L., Limat, L.: Viscoelastic liquid curtains: experimental results on the flow of a falling sheet of polymer solution. J. Fluid Mech. 873, 358–409 (2019) https://doi.org/10.1017/jfm.2019.389
  * Dinic et al. [2017] Dinic, J., Biagioli, M., Sharma, V.: Pinch-off dynamics and extensional relaxation times of intrinsically semi-dilute polymer solutions characterized by dripping-onto-substrate rheometry. J. Polym. Sci. B: Polym. Phys. 55(22), 1692–1704 (2017) https://doi.org/10.1002/polb.24388
  * Dinic et al. [2015] Dinic, J., Zhang, Y., Jimenez, L.N., Sharma, V.: Extensional Relaxation Times of Dilute, Aqueous Polymer Solutions. ACS Macro Lett. 4(7), 804–808 (2015) https://doi.org/10.1021/acsmacrolett.5b00393
  * Sur and Rothstein [2018] Sur, S., Rothstein, J.: Drop breakup dynamics of dilute polymer solutions: Effect of molecular weight, concentration, and viscosity. J. Rheol. 62(5), 1245–1259 (2018) https://doi.org/10.1122/1.5038000
  * Anna and McKinley [2001] Anna, S.L., McKinley, G.H.: Elasto-capillary thinning and breakup of model elastic liquids. J. Rheol. 45(1), 115–138 (2001) https://doi.org/10.1122/1.1332389
  * Mathues et al. [2018] Mathues, W., Formenti, S., McIlroy, C., Harlen, O.G., Clasen, C.: Caber vs ROJER—Different time scales for the thinning of a weakly elastic jet. J. Rheol. 62(5), 1135–1153 (2018) https://doi.org/10.1122/1.5021834
  * McIlroy et al. [2013] McIlroy, C., Harlen, O.G., Morrison, N.F.: Modelling the jetting of dilute polymer solutions in drop-on-demand inkjet printing. J. Nonnewton Fluid Mech. 201, 17–28 (2013) https://doi.org/%****␣sn-article.bbl␣Line␣475␣****10.1016/j.jnnfm.2013.05.007
  * Hoath et al. [2012] Hoath, S.D., Harlen, O.G., Hutchings, I.M.: Jetting behavior of polymer solutions in drop-on-demand inkjet printing. J. Rheol. 56(5), 1109–1127 (2012) https://doi.org/10.1122/1.4724331
  * Turkoz et al. [2021] Turkoz, E., Stone, H.A., Arnold, C.B., Deike, L.: Simulation of impulsively induced viscoelastic jets using the oldroyd-b model. J. Fluid Mech. 911, 14 (2021) https://doi.org/10.1017/jfm.2020.1053
  * Chilcott and Rallison [1988] Chilcott, M.D., Rallison, J.M.: Creeping flow of dilute polymer solutions past cylinders and spheres. J. Nonnewton Fluid Mech. 29, 381–432 (1988) https://doi.org/10.1016/0377-0257(88)85062-6