# Deformable Linear Object Prediction Using Locally Linear Latent Dynamics

Wenbo Zhang1, Karl Schmeckpeper2, Pratik Chaudhari3, and Kostas Daniilidis4
1Wenbo Zhang did the work when he was a student at the GRASP Lab, University
of Pennsylvania, USA<EMAIL_ADDRESS>Schmeckpeper is with the GRASP
Lab, University of Pennsylvania, USA<EMAIL_ADDRESS>Chaudhari is
with the GRASP Lab, University of Pennsylvania, USA
<EMAIL_ADDRESS>Daniilidis is with the GRASP Lab, University of
Pennsylvania, USA<EMAIL_ADDRESS>https://github.com/zwbgood6/deform

###### Abstract

We propose a framework for deformable linear object prediction. Prediction of
deformable objects (e.g., rope) is challenging due to their non-linear
dynamics and infinite-dimensional configuration spaces. By mapping the
dynamics from a non-linear space to a linear space, we can use the good
properties of linear dynamics for easier learning and more efficient
prediction. We learn a locally linear, action-conditioned dynamics model that
can be used to predict future latent states. Then, we decode the predicted
latent state into the predicted state. We also apply a sampling-based
optimization algorithm to select the optimal control action. We empirically
demonstrate that our approach can predict the rope state accurately up to ten
steps into the future and that our algorithm can find the optimal action given
an initial state and a goal state.

##  I INTRODUCTION

Robotic manipulation can be applied to a variety of scenarios, such as
surgery, housekeeping, manufacturing, and agriculture [1][2]. Deformable
object manipulation is one of the important robotic manipulation problems. We
face deformable objects frequently in our everyday lives, such as when folding
the clothes [3], knot-typing [4][5], and moving a cable [6]. However,
deformable object manipulation is still a challenging task due to the
following reasons. (i) High model uncertainty. It is hard to simulate an
accurate deformable object model because of environmental noises. (ii)
Infinite-dimensional configuration spaces. We need to get lots of parameters
for modeling a good deformable object. (iii) High cost of simulation. It is
expensive to get all the simulation model parameters for deformable objects,
and sometimes it is infeasible to have all the parameters. (iv) Hyper-
underactuated systems. The robotic manipulator cannot fully control deformable
objects with arbitrary trajectories. (v) Non-linear dynamics. Non-linearity
makes it hard to predict future states when executing actions on deformable
objects. Our paper explicitly addresses the problems caused by non-linear
dynamics and infinite-dimensional configuration spaces through mapping
dynamics from a non-linear space to a linear space and shrinking the state to
a lower-dimensional latent state.

In this paper, we consider the problems on how to predict the future state of
deformable linear objects (i.e., rope) given an initial state and a sequence
of actions, and on how to find optimal control actions to move the rope from
the initial state to a goal state given a prediction model.

To solve the prediction problem, we propose a prediction framework that
consists of a state autoencoder model, an action autoencoder model, and a
dynamics model. The state autoencoder model is based on convolutional neural
networks. Rope dynamics are non-linear and are very complicated to learn
directly. Unlike non-linear dynamics, linear dynamics are easier to learn and
more efficient for state predictions. Thus, we consider mapping the non-linear
space into a linear space, predicting future states in the linear space, and
decoding back to the non-linear space from the linear space. To do so, we
first use learned encoders to encode both the state and the action into the
latent state and the latent action. Then, we run the dynamics model to get the
state matrix and control matrix for the linear dynamics in the latent space.
After that, we get a predicted next latent state by using locally linear
latent dynamics based on the current latent state, latent action, state
matrix, and control matrix. Finally, we decode the predicted next latent state
into a predicted next state. Our experiments show that the prediction
framework can accurately predict states up to ten timesteps in the future.

We also want to find the optimal control actions to manipulate the rope from
an initial position to a goal position. To solve this control problem, we use
a sampling-based optimization algorithm to select the optimal action sequence
based on an initial state, a goal state, and the trained prediction framework.
This algorithm is an extension of the cross-entropy method. The algorithm
works by repeating the processes of (i) sampling action sequences from a
probability distribution, (ii) predicting the goal state, (iii) calculating
and sorting the loss between predicted state and ground truth, and (iv)
fitting a new distribution to selected action sequences. We stop this process
when the distribution converges and select the optimal action sequence with
the lowest loss.

Contributions. Our key contributions are: (i) a novel prediction framework for
predicting future states of deformable linear objects under a sequence of
actions (Section III), (ii) a sampling-based optimization algorithm for
selecting optimal control actions to move the deformable linear objects
(Section III), (iii) experiments showing multi-step prediction results and
comparing our method with a baseline qualitatively and quantitatively (Section
IV), and (iv) ablation studies regarding locally and globally latent dynamics,
model training methods, and how small latent size we can use to achieve good
performance (Section IV).

##  II RELATED WORK

Learning Models for Non-linear Systems. There are active studies about
learning a model for non-linear dynamical systems. Kingma et al. [7] introduce
the variational autoencoder (VAE) using approximate Bayesian inference. Based
on VAE, Watter et al. [8] propose an Embed to Control (E2C) method to learn a
non-linear dynamical system model. They use the VAE to learn latent parameters
and set locally linear constraints on dynamics using an optimal control
concept. We are inspired by this method. We do experiments using locally
linear latent dynamics and compare it with globally linear latent dynamics.
However, instead of applying the optimal control formulation, we train an
action-conditioned dynamics model to enable system identification in the
latent space. Our dynamics model can generate the state matrix and the control
matrix at each time step. Li et al. [9] apply the Koopman operator theory to
provide nonlinear-to-linear transformation. Based on this approach, the
infinite-dimensional state space changes to finite-dimensional state space. We
learn from this method to encode high-dimensional state space into a smaller
latent state in the linear space. There are also many researchers [10][11][12]
working on learning high-dimensional non-linear state space and system
identification.

State Estimation and Prediction. For state estimation, Yan et al. [13] apply
self-supervised learning for state estimation. Schulman et al. [14]
incorporate the point cloud in the probabilistic generative model for tracking
deformable objects. Petit et al. [15] track objects with point cloud data from
an RGB-D camera. Unlike those approaches to getting the object’s points for
state estimation, we use the binary mask of rope image as an input state to a
state autoencoder. Then, the encoder can output a latent state. For the
prediction of state, Finn et al. [16][17] present deep action-conditioned
video prediction models. Unlike [16][17], we predict the future states in the
latent space based on the estimation of the latent states. Besides, we predict
the next state using the current state information without including previous
states’.

Planning and Control. The work [13] uses Model Predictive Path Integral (MPPI)
controller for choosing the best grasping point with the least moving length
to the goal. Wang et al. [18] apply the data-driven approach to generate a
sequence of planned images and then use an inverse dynamics model to execute
the plan. Sundaresan et al. [19] use dense depth object descriptors to learn a
policy for manipulating a rope into different configuration settings. Pathak
et al. [20] learn a goal-conditional control policy for the rope manipulation
based on a sequence of images. Moll et al. [21] compute the minimal-energy
curves for path planning in a wire manipulation task. Bayazit et al. [22]
introduce a Probabilistic Roadmap (PRM) method. There are also some other
sampling-based methods which have been introduced in [4][17][23][24][25].
Similar to [17], we have a similar control setup and we also use a sampling-
based optimization algorithm to select the optimal control action. Besides,
researchers apply imitation learning to learn control policies. Nair et al.
[26] use human demonstrations to teach the robot how to move a rope. The robot
will learn the policy from the video sequence and apply this policy for
execution. Huang et al. [27] approach the problem by trajectory transfer
through non-rigid registration.

Deformable Objects Manipulation. There are mainly two types of deformable
objects. One is about linear objects [25] (e.g., rope, cable) and the other is
about non-linear objects [3][28] (e.g., fabric, cloth, bed sheet, garment).
This paper is focusing on a simpler scenario on deformable linear objects,
such as the ropes. Besides, our attentions are more on how to predict the
future states and how to sample actions to reach a goal state, and less on how
to manipulate the deformable objects using robots.

##  III METHOD

Deformable objects modeling [15][28][29][30] is a good way to model the
objects in a simulation. However, existing methods are not practical for real-
time robotic tasks due to slow speed, high simulation expense, high-
dimensional state space, high model uncertainty, and non-linear dynamics.
These reasons result in a challenging task of predicting deformable objects in
future states under specific actions. Thus, we want to simplify the non-linear
dynamics and reduce high-dimensional state space for deformable linear
objects.

Instead of using a physics-based simulation to model the objects, we design an
encoding network to extract the latent states from rope images. Then we learn
the dynamical system in the latent space. Using the dynamics in the latent
space, we can predict the future latent states. After that, the model decodes
the future latent states into rope images. We elaborate on each step as
following.

###  III-A Preliminary

First, we introduce some fundamental concepts.

Rope Dynamics. We consider nonlinear rope dynamics

$x(t+1)=f(x(t),u(t)),$

where $t\in\\{0,1,...,T\\}$ is the time step,
$x(t)\in\mathcal{X}\subseteq\mathbb{R}^{m}$ is the state,
$u(t)\in\mathcal{U}\subseteq\mathbb{R}^{n}$ is the control input. Since we
cannot get the rope state easily, we denote the rope image as the state
$x(t)$. In our case, $u=(x,y,l,\theta)$, where $x$ and $y$ are positions in
the image space, moving length $l$ and moving angle $\theta$ are in the world
space.

Latent Dynamics. The rope dynamics are non-linear, very complicated to learn,
and hard to generalize, while linear dynamics are generally simpler to learn
and can help make state predictions more easily. Thus, we consider a linear
dynamics model

$g(t+1)=K(t)g(t)+L(t)a(t)$ (1)

in the latent space. Here, $g(t)\in\mathcal{G}\subseteq\mathbb{R}^{p}$ is the
latent state, $a(t)\in\mathcal{A}\subseteq\mathbb{R}^{q}$ is the latent
action, $K(t)\in\mathcal{K}\subseteq\mathbb{R}^{p\times p}$ is the state
matrix, and $L(t)\in\mathcal{L}\subseteq\mathbb{R}^{p\times q}$ is the control
matrix. The state matrix $K(t)$ is a transition matrix from the current latent
state $g(t)$ to the next latent state $g(t+1)$. The control matrix $L(t)$
learns the impact of the latent action $a(t)$ on the state transition.

For globally linear latent dynamics, $K(t)=K$ and $L(t)=L$ for all
$t\in\\{0,1,...,T\\}$, where $K$ and $L$ are fixed matrices. For locally
linear latent dynamics, $K(t)$ depends on the state $x(t)$ and $L(t)$ depends
on both state $x(t)$ and action $u(t)$.

Encoder Models. The state encoder model is a map
$\phi_{e}:\mathcal{X}\to\mathcal{G}$, the action encoder model is a map
$\varphi_{e}:\mathcal{U}\to\mathcal{A}$. We then have

$g(t)=\phi_{e}(x(t)),~{}a(t)=\varphi_{e}(u(t)).$

Encoder models encode the state $x$ into the latent state $g$ and the action
$u$ into the latent action $a$. For the state $x$ (i.e., raw image), it is
high-dimensional and redundant, so we want to reduce the state size and use
the smaller latent state as a representation. For the action $u$ (i.e.,
grasping positions, gripper moving length, and gripper moving angle), the raw
action usually has a very non-linear effect on the state, so we lift the
actions to a higher dimension and make them easier to linearize in the latent
space.

Decoder Model. The state decoder model is a map
$\phi_{d}:\mathcal{G}\to\mathcal{X}$, the action decoder model is a map
$\varphi_{d}:\mathcal{A}\to\mathcal{U}$. We then have

$\hat{x}(t)=\phi_{d}(g(t)),~{}\hat{u}(t)=\varphi_{d}(a(t)).$

Paired with encoder models, decoder models decode the latent state $g$ into
the reconstructed state $\hat{x}$ and the latent action $a$ into the
reconstructed action $\hat{u}$.

Dynamics Model. The dynamics model contains one map
$\psi_{s}:\mathcal{X}\to\mathcal{K}$ for the state matrix and the other map
$\psi_{c}:\mathcal{X},\mathcal{U}\to\mathcal{L}$ for the control matrix. We
then have

$K(t)=\psi_{s}(x(t)),~{}L(t)=\psi_{c}(x(t),u(t)).$

###  III-B Prediction Framework

We consider the prediction framework in Figure 1. It consists of the state
autoencoder model, action autoencoder model, and dynamics model.

When predicting, given a new state $x(t)$ and a new action $u(t)$, we can get
latent state $g(t)$, latent action $a(t)$, state matrix $K(t)$, control matrix
$L(t)$. Using locally linear latent dynamics, we can have predicted next
latent state $g^{pred}(t+1)=K(t)g(t)+L(t)a(t)$. By decoding the predicted next
latent state $g^{pred}(t+1)$, we will have the predicted next state
$x^{pred}(t+1)$.

Figure 1: Diagram for prediction framework. We take a part of the diagram from
time step 0 to 1 as an example. The red arrows show the state autoencoder
model. We use a state encoder $\phi_{e}$ to encode a state $x(0)$ into a
latent state $g(0)$ and then use a state decoder $\phi_{d}$ to decode the
latent state $g(0)$ into a reconstructed state $\hat{x}(0)$. The orange arrows
show the action autoencoder model. We use an action encoder $\varphi_{e}$ to
encode an action $u(0)$ into a latent action $a(0)$ and then use an action
decoder $\varphi_{d}$ to decode the latent action $u(0)$ into a reconstructed
action $\hat{u}(0)$. The green arrows show the rope dynamics. Given the state
$x(0)$ and the action $u(0)$, we can have next state $x(1)$. The blue arrows
show the latent dynamics. Given the latent state $g(0)$, the latent action
$a(0)$, state matrix $K(0)$, and control matrix $L(0)$, we can get the next
latent state $g(1)$ based on locally linear latent dynamics.

Loss Function. We consider minimizing the losses for state autoencoder model,
action autoencoder model, and dynamics model. The loss of state autoencoder
model is

$\mathcal{L}_{state}=\sum_{t=0}^{T}||\phi_{d}(\phi_{e}(x(t)))-x(t)||_{2}^{2}.$
(2)

The state loss $\mathcal{L}_{state}$ makes sure the reconstructed state
$\hat{x}$ is similar to ground truth state $x$. The loss of action autoencoder
model is

$\mathcal{L}_{action}=\sum_{t=0}^{T}||\varphi_{d}(\varphi_{e}(u(t)))-u(t)||_{2}^{2}.$
(3)

The action loss $\mathcal{L}_{action}$ makes sure the reconstructed action
$\hat{u}$ is similar to ground truth action $u$. The loss of dynamics model is

$\displaystyle\mathcal{L}_{dyn}=$
$\displaystyle\sum_{t=0}^{T}||K(t)g(t)+L(t)a(t)-g(t+1)||_{2}^{2}.$ (4)

The dynamics loss $\mathcal{L}_{dyn}$ sets a locally linear constraint on
latent dynamics. The loss of prediction model is

$\displaystyle\mathcal{L}_{pred}=$
$\displaystyle\sum_{t=0}^{T-1}||x^{pred}(t+1)-x(t+1)||_{2}^{2}.$ (5)

The prediction loss $\mathcal{L}_{pred}$ makes sure the predicted state
$x^{pred}$ is similar to ground truth state $x$. The expansions of Equation
(4) and Equation (5) are in Appendix V-C. The overall training loss is

$\mathcal{L}=\mathcal{L}_{state}+\lambda_{1}*\mathcal{L}_{action}+\lambda_{2}*\mathcal{L}_{dyn}+\lambda_{3}*\mathcal{L}_{pred},$

where $\lambda_{1}$, $\lambda_{2}$, and $\lambda_{3}$ are coefficients.

Training Prediction Framework. Figure 2 shows 4 steps on how to train the
prediction framework $\mathcal{M}$.

Figure 2: Prediction framework in four steps. (i) Step 1: we do color space
segmentation to get a binary mask from the raw rope image. We use this mask as
the input $x(t)$ for the state encoder and the output is the latent state
$g(t)$ (orange vector). We also use action encoder to encode the action $u(t)$
into the latent action $a(t)$ (blur vector). (ii) Step 2: we train the
dynamics model using the binary mask $x(t)$ and the action $u(t)$. The output
is state matrix $K(t)$ (yellow matrix) and control matrix $L(t)$ (green
matrix). (iii) Step 3: by using locally linear latent dynamics, we have
predicted next latent state $g^{pred}(t+1)=K(t)g(t)+L(t)a(t)$. (iv) Step 4: we
use the state decoder to decode the latent state $g(t)$ into the reconstructed
state $\hat{x}(t)$, run the action decoder to decode the latent action $a(t)$
into reconstructed action $\hat{u}(t)$, and apply the same state decoder to
decode the predicted next latent state $g^{pred}(t+1)$ into predicted next
state $x^{pred}(t+1)$.

Training Autoencoders. In Figure 2 Step 1, we use Equation (2) and Equation
(3) to train the autoencoders to get both latent states $g$ and latent actions
$a$. When state loss $\mathcal{L}_{state}<\epsilon_{state}$ and action loss
$\mathcal{L}_{action}<\epsilon_{action}$, we save latent state $g$ and latent
action $a$ in each time step. Here, $\epsilon_{state}$ and $\epsilon_{action}$
are very small values. We make sure state $x$ and action $u$ are similar to
reconstructed state $\hat{x}$ and reconstructed action $\hat{u}$,
respectively. We also do early stopping to prevent the autoencoders from
overfitting to the training set.

Training Dynamics Model. The state matrix $K$ depends on state $x$ and the
control matrix $L$ depends on both state $x$ and action $u$. In Figure 2 Step
2, we train dynamics model to minimize the dynamics loss $\mathcal{L}_{dyn}$
in Equation (4) and prediction loss $\mathcal{L}_{pred}$ in Equation (5).

There are two methods for model training: (i) training autoencoder models
first and then training dynamics model, and (ii) training autoencoder models
and dynamics model together. We discuss the model training order and our
preference in Section IV ablation study.

To wrap up, after training the autoencoders and the dynamics model using the
training set, we start to make predictions for new states and actions in the
test set. The detailed architectures and the hyperparameters for the
autoencoders and the dynamics model are in the Appendix V-D and Appendix V-A.

###  III-C Sampling-based Model Predictive Control

We consider using cross-entropy method (CEM) [31] to select optimal action
given trained prediction framework $\mathcal{M}$. CEM is an optimization
algorithm. It works by repeating two phases: (i) providing a probability
distribution and sampling from this distribution and (ii) minimizing the
cross-entropy between the current distribution and a target distribution for
better sampling in the next iteration. Our Algorithm 1 is an extension of CEM.
For each time step, given trained prediction framework $\mathcal{M}$, initial
image $x_{init}$, and goal image $x_{goal}$, we first initialize a uniform
distribution $Q_{pre}$. Then, we sample $m$ action sequences from distribution
$Q_{pre}$. Each action sequence has length $h$, meaning that there are $h$
actions in this sequence. We use the prediction framework $\mathcal{M}$ to
predict the future states $x^{pred}$ given initial image $x_{init}$ and each
action sequence. Assuming we can get a predicted goal image $x_{goal}^{pred}$
after taking $h$ actions, we then calculate the Binary Cross Entropy (BCE)
loss between the predicted goal images $x_{goal}^{pred}$ and goal image
$x_{goal}$ we want. By sorting $m$ BCE losses, we choose $n~{}(n<m)$ action
sequences that have lowest losses. We apply multivariate normal distribution
$Q_{post}$ to fit $n$ samples. If the KL divergence between $Q_{pre}$ and
$Q_{post}$ is not less than a specified small value, we will repeat the same
process until the distribution $Q$ converges. After distribution convergence,
we execute the optimal action with the lowest loss.

Algorithm 1 Sampling-based optimization algorithm

0: prediction framework $\mathcal{M}$, initial image $x_{init}$, goal image
$x_{goal}$

for $t=1,...,T$ do

Initialize $Q$ with uniform distribution

while $Q$ does not converge do

Sample $m$ action sequences with length $h$ from $Q$

Use prediction framework $\mathcal{M}$ to predict goal images
$x^{pred}_{goal}$ using $m$ action sequences

Calculate loss between predicted goal image $x^{pred}_{goal}$ and goal image
$x_{goal}$

Select $n$ action sequences with lowest sorted losses

Fit multi-variate normal distribution $Q$ to $n$ samples

end while

Execute action with lowest loss

Observe new next state

end for

##  IV EXPERIMENTS AND RESULTS

We demonstrate that our prediction framework can predict well, and our model
can be used for control. We also show that training methods and locally linear
latent dynamics are critical for good performance.

###  IV-A Experimental Setup

We use Rope Manipulation Dataset from paper [26]. This dataset is challenging
because (i) the rope is deformable and we do not know the rope dynamics
beforehand, (ii) the images are not taken straight down and there is a less-
than-ninety-degree angle between the camera and the table, and (iii) the
consecutive actions are not applied on the same grasping position. These
challenges combine to make it difficult to predict the rope state accurately
at each time step.

State. Different from [26] which uses the whole image as an input, we merely
want to get the rope-related information in the state. Thus, we do color space
segmentation and add a black mask (Figure 2, Step 1) to replace the noisy
background. We resize the image from $240\times 240$ to $50\times 50$ and
apply data augmentation consisting of translation, vertical flips, and
horizontal flips.

Action. The original action $u=(x,y,l,\theta,\mathbbm{1})$ includes $x$ and
$y$ positions in the image space, moving length $l$ and moving angle $\theta$
in the world space, and an indicator $\mathbbm{1}$ that decides whether this
action validly moves the rope or not. Situations for the invalid actions
include the robotic gripper not contacting the rope and not moving the rope
according to the command. We re-calculate the action $x$ and $y$ positions
according to the change of image size and filter the invalid actions when
training and testing.

###  IV-B Multi-step Prediction Results And Comparison

One-step Prediction. In Figure 2 Step 3, we use Equation (1) to get predicted
next latent state $g_{pred}$ based on the current state matrix $K$, the
current latent state $g$, the current control matrix $L$, and the current
latent action $a$. In Figure 2 Step 4, we use the same decoder in the state
autoencoder and decode the predicted next latent state $g_{pred}$ into
predicted next state $x_{pred}$.

Multi-step Prediction. In order to test our framework’s prediction horizon, we
use the trained prediction model for multi-step prediction. Given state
$x(0)$, action $u(0)$, and trained model $\mathcal{M}$, we can predict next
state $x^{pred}(1)$. Then using predicted state $x^{pred}(1)$ and ground truth
action $u(1)$, we predict the next state $x^{pred}(2)$. By iterating the same
process, we get ten predicted states shown in Figure 3.

Figure 3: Comparison between Ground Truth (GT), our method (Our), and E2C++
from time step 0 to 10. (a), (b) and (c) represent three cases starting at
different initial states. In each case, it includes GT, Our, and E2C++. The
first row is the initial state at time step 0. The second row (time step 1) to
the last row (time step 10) are the predicted states. We can conclude that our
method can predict the rope state accurately up to ten steps into the future,
while the prediction results using baseline E2C++ are blurry.

Comparison. We compare our results with Ground Truth and results using
Embedded to Control (E2C) [8]. E2C is a method for learning and control of
non-linear dynamics from images. Unlike E2C using optimal control formulation
in the latent space, we use the dynamics model to get the state matrix and the
control matrix for state prediction in the latent space. To make a fair
comparison, we use the same neural network architecture for autoencoders and
the same latent size for the state. When running the E2C on the Rope
Manipulation Dataset, we find the reconstruction and prediction results are
blurry, and we cannot observe a clear rope shape from the image. Thus, we add
a prediction loss $\mathcal{L}_{pred}$ and a state loss $\mathcal{L}_{state}$
on the original loss of variational bound and KL divergence. We name this new
approach E2C++. Figure 3 (a) and (b) show that our method can predict the
future states well in ten steps, and the rope shape is similar to the ground
truth. For Figure 3 (c), there is a clear distinction between GT and our
method at time step 8 because the previous states (time step 1 to 7) gradually
accumulate the state prediction error. E2C++ can only predict a blurry rope
shape within two-time steps. The remaining state predictions using E2C++ do
not make sense.

###  IV-C Action Planning and Control Results

Qualitative Results. We use the sampling-based optimization method, shown in
Algorithm 1, to plan rope manipulation. The objective is to generate an
optimal action sequence. Similar to the standard cross-entropy method (CEM),
our algorithm stops repeating in the inner loop when the probability
distribution $Q$ converges. However, unlike repeating two phases for the
standard CEM, our algorithm needs to do more steps of sampling action
sequences, predicting the goal state, calculating and sorting the loss, and
fitting a new distribution. In the experiments, we first sample the optimal
action by applying CEM. Then we use the trained prediction framework to
predict the next state $x^{pred}(t+1)$ given the current state $x(t)$ and the
sampled optimal action.

Figure 4 shows the experimental results. We overlay sampled action on the rope
and compare it with the ground truth action. By comparing the Ground Truth
(GT) action with sampled action using CEM, we can find sampled action is close
to GT action concerning grasping positions, moving length, and moving angle.
However, sometimes there are many possible actions to move the rope from one
state to the next state. For example, Figure 4 (c) shows the different
grasping positions for sampled action, but the rope ends up with a similar
predicted next state as the GT state.

Figure 4: Comparison between Ground Truth (GT) actions and sampled actions
using the Cross-Entropy Method (CEM). (a), (b) and (c) are three cases with
different initial rope positions. In each case, the first row shows GT action
(blue arrow) at state $x(t)$ (left) and the GT next state $x(t+1)$ (right).
The second row shows sampled action (red arrow) at state $x(t)$ (left) and the
predicted next state $x^{pred}(t+1)$ (right). We can conclude that the sampled
actions using CEM are close to GT actions concerning the grasping positions,
the moving length, and the moving angle. For case (c), sometimes there are
multiple options to move the rope. Besides, our prediction model can
accurately predict the next state. The predicted state $x^{pred}(t+1)$ is
similar to GT state $x(t+1)$.

Quantitative Results. We calculate the Mean Square Error (MSE) between sampled
actions and ground truth actions. We also apply MSE between the predicted next
state and ground truth next state. After running 100 sampled one-step actions
using our method and E2C++, we get the mean and standard deviation in Table I.
By comparing Action Score, we can observe that our score is lower. It means
the sampled action is closer to the ground truth action when using our
prediction model. From State Score, we can find our method has a lower score.
It means our method is better at predicting the next state after applying the
sampled action. Detailed methods to get the quantitative results are mentioned
in Appendix V-B.

Method | Action Score (MSE) | State Score (MSE)
---|---|---
Our | $\textbf{18.20}\pm\textbf{9.46}$ | $\textbf{0.59}\pm\textbf{0.27}$
E2C++ | $29.35\pm 5.86$ | $1.12\pm 0.18$
TABLE I: Action score and state score comparison between our method and E2C++.

###  IV-D Ablation Study

Locally Latent Dynamics v.s. Globally Latent Dynamics. For locally latent
dynamics, state matrix $K(t)$ and control matrix $L(t)$ are different at each
time step. However, $K$ and $L$ are constant matrices for globally latent
dynamics. If $K$ and $L$ are constant matrices, it will set a more strict
constraint in the latent space. It is difficult to train a good-performance
model with globally linear latent dynamics since we need to guarantee the
dynamics are linear for all latent states. In our experiments, we use locally
linear latent dynamics.

Order for Training Models. We have autoencoder models for state and action and
the dynamics model. The order to train models is important to get good
performance. We use two methods: (i) training autoencoder models first and
then training dynamics model, (ii) training autoencoder models and dynamics
model together. We demonstrate that method (i) can provide better prediction
and control results, as shown in Figure 3, while method (ii) reconstructs
blurry rope images, and we cannot find a clear rope shape. The reason is that
we can get a good latent state estimation when training autoencoder models.
After having a good latent state estimation, we then train the dynamics model
to get good system dynamics. For method (ii), training all models together to
optimize all losses is more difficult to get a good prediction result. It is
because we need to find the optimal neural network weights to get accurate
latent state, latent action, state matrix, and control matrix at the same
time.

Latent State and Latent Action Size. Since the dimension of state $x$ is
$50\times 50$, we need to shrink its size to a smaller dimension for latent
state $g$. Different from the state, we use the autoencoder model to expand
the action dimension from $(x,y,l,\theta)$ to a larger size. We do experiments
to find how small we can get for the latent size to achieve good performance.
Experiments show that the latent state size cannot be less than 50, and latent
action size cannot be less than 10. When the latent state and latent action
size are too small, there is no enough information about the real state and
action. Besides, their sizes do not need to be too large (e.g., latent state
size is 300+ and latent action size is 100+). One reason is that it increases
more time to train since we have state matrix $K$ and control matrix $L$ as
well. Another reason is that the larger size does not improve the prediction
performance much. Thus, we choose 80 and 80 for latent state size and latent
action size, respectively.

Whether Using Action Autoencoder. In the prediction framework, we use the
autoencoder to lift the action space to a higher dimensional space so it can
work linearly with the latent state in the latent space. We also try not to
use the autoencoder in the prediction framework, and we cannot achieve good
prediction results due to the non-linearity of the impact of the four-element
action.

##  V CONCLUSIONS

This paper presents a prediction framework for rope prediction. Our method
first maps a non-linear space into a locally linear space using the encoders.
Then, we get the state matrix and control matrix for linear dynamics by
running the dynamics model. In the next step, we make a prediction based on
the locally linear latent dynamics. Finally, we decode the predicted latent
state into the predicted state in the non-linear space. We also propose a
sampling-based optimization algorithm to select the optimal control action to
move the rope from an initial state to a goal state. The experimental results
demonstrate that our method can accurately predict the rope images to ten
steps in the future. The experiments also show that our sampling-based
optimization algorithm can find the optimal actions to relocate the rope. By
comparing sampled actions using our method and actions using the baseline, we
demonstrate that our method achieves better performance.

Our approach has not demonstrated success on more complex rope configurations
for linear rope prediction, such as loops or knots. Future work includes
predicting more complex rope configurations, generalizing the rope prediction
framework across different rope materials, and deploying our framework on a
robotic manipulator.

## ACKNOWLEDGMENT

We thank Bernadette Bucher for her contributions to this project. The research
was sponsored by the Army Research Office and was accomplished under grants
ARO MURI W911NF-20-1-0080, NSF CPS 2038873, ARL DCIST CRA W911NF-17-2-0181,
ONR N00014-17-1-2093, and by the Honda Research Institute.

## References

  * [1] J. Sanchez, J.-A. Corrales, B.-C. Bouzgarrou, and Y. Mezouar, “Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey,” _The International Journal of Robotics Research_ , vol. 37, no. 7, pp. 688–716, 2018.
  * [2] F. F. Khalil and P. Payeur, “Dexterous robotic manipulation of deformable objects with multi-sensory feedback-a review,” in _Robot Manipulators Trends and Development_. IntechOpen, 2010\.
  * [3] A. Doumanoglou, J. Stria, G. Peleka, I. Mariolis, V. Petrik, A. Kargakos, L. Wagner, V. Hlaváč, T.-K. Kim, and S. Malassiotis, “Folding clothes autonomously: A complete pipeline,” _IEEE Transactions on Robotics_ , vol. 32, no. 6, pp. 1461–1478, 2016.
  * [4] M. Saha and P. Isto, “Motion planning for robotic manipulation of deformable linear objects,” in _Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006._ IEEE, 2006, pp. 2478–2484.
  * [5] W. Wang and D. Balkcom, “Tying knot precisely,” in _2016 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2016, pp. 3639–3646.
  * [6] Y. She, S. Wang, S. Dong, N. Sunil, A. Rodriguez, and E. Adelson, “Cable manipulation with a tactile-reactive gripper,” _arXiv preprint arXiv:1910.02860_ , 2019.
  * [7] D. P. Kingma and M. Welling, “Stochastic gradient vb and the variational auto-encoder,” in _Second International Conference on Learning Representations, ICLR_ , vol. 19, 2014.
  * [8] M. Watter, J. Springenberg, J. Boedecker, and M. Riedmiller, “Embed to control: A locally linear latent dynamics model for control from raw images,” in _Advances in neural information processing systems_ , 2015, pp. 2746–2754.
  * [9] Y. Li, H. He, J. Wu, D. Katabi, and A. Torralba, “Learning compositional koopman operators for model-based control,” _arXiv preprint arXiv:1910.08264_ , 2019.
  * [10] R. G. Krishnan, U. Shalit, and D. Sontag, “Structured inference networks for nonlinear state space models,” in _Thirty-first aaai conference on artificial intelligence_ , 2017.
  * [11] M. Karl, M. Soelch, J. Bayer, and P. Van der Smagt, “Deep variational bayes filters: Unsupervised learning of state space models from raw data,” _arXiv preprint arXiv:1605.06432_ , 2016.
  * [12] E. Yeung, S. Kundu, and N. Hodas, “Learning deep neural network representations for koopman operators of nonlinear dynamical systems,” in _2019 American Control Conference (ACC)_. IEEE, 2019, pp. 4832–4839.
  * [13] M. Yan, Y. Zhu, N. Jin, and J. Bohg, “Self-supervised learning of state estimation for manipulating deformable linear objects,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 2372–2379, 2020.
  * [14] J. Schulman, A. Lee, J. Ho, and P. Abbeel, “Tracking deformable objects with point clouds,” in _2013 IEEE International Conference on Robotics and Automation_. IEEE, 2013, pp. 1130–1137.
  * [15] A. Petit, V. Lippiello, and B. Siciliano, “Real-time tracking of 3d elastic objects with an rgb-d sensor,” in _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2015, pp. 3914–3921.
  * [16] C. Finn, I. Goodfellow, and S. Levine, “Unsupervised learning for physical interaction through video prediction,” in _Advances in neural information processing systems_ , 2016, pp. 64–72.
  * [17] C. Finn and S. Levine, “Deep visual foresight for planning robot motion,” in _2017 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2017, pp. 2786–2793.
  * [18] A. Wang, T. Kurutach, K. Liu, P. Abbeel, and A. Tamar, “Learning robotic manipulation through visual planning and acting,” _arXiv preprint arXiv:1905.04411_ , 2019.
  * [19] P. Sundaresan, J. Grannen, B. Thananjeyan, A. Balakrishna, M. Laskey, K. Stone, J. E. Gonzalez, and K. Goldberg, “Learning rope manipulation policies using dense object descriptors trained on synthetic depth data,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 9411–9418.
  * [20] D. Pathak, P. Mahmoudieh, G. Luo, P. Agrawal, D. Chen, Y. Shentu, E. Shelhamer, J. Malik, A. A. Efros, and T. Darrell, “Zero-shot visual imitation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition workshops_ , 2018, pp. 2050–2053.
  * [21] M. Moll and L. E. Kavraki, “Path planning for deformable linear objects,” _IEEE Transactions on Robotics_ , vol. 22, no. 4, pp. 625–636, 2006.
  * [22] O. B. Bayazit, J.-M. Lien, and N. M. Amato, “Probabilistic roadmap motion planning for deformable objects,” in _Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292)_ , vol. 2. IEEE, 2002, pp. 2126–2133.
  * [23] E. Anshelevich, S. Owens, F. Lamiraux, and L. E. Kavraki, “Deformable volumes in path planning applications,” in _Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065)_ , vol. 3. IEEE, 2000, pp. 2290–2295.
  * [24] F. Lamiraux and L. E. Kavraki, “Planning paths for elastic objects under manipulation constraints,” _The International Journal of Robotics Research_ , vol. 20, no. 3, pp. 188–208, 2001.
  * [25] O. Roussel, A. Borum, M. Taix, and T. Bretl, “Manipulation planning with contacts for an extensible elastic rod by sampling on the submanifold of static equilibrium configurations,” in _2015 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2015, pp. 3116–3121.
  * [26] A. Nair, D. Chen, P. Agrawal, P. Isola, P. Abbeel, J. Malik, and S. Levine, “Combining self-supervised learning and imitation for vision-based rope manipulation,” in _2017 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2017, pp. 2146–2153.
  * [27] S. H. Huang, J. Pan, G. Mulcaire, and P. Abbeel, “Leveraging appearance priors in non-rigid registration, with application to manipulation of deformable objects,” in _2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2015, pp. 878–885.
  * [28] Y. Li, Y. Wang, Y. Yue, D. Xu, M. Case, S.-F. Chang, E. Grinspun, and P. K. Allen, “Model-driven feedforward prediction for manipulation of deformable objects,” _IEEE Transactions on Automation Science and Engineering_ , vol. 15, no. 4, pp. 1621–1638, 2018.
  * [29] S. F. Gibson and B. Mirtich, “A survey of deformable modeling in computer graphics,” Citeseer, Tech. Rep., 1997.
  * [30] N. Essahbi, B. C. Bouzgarrou, and G. Gogu, “Soft material modeling for robotic manipulation,” in _Applied Mechanics and Materials_ , vol. 162. Trans Tech Publ, 2012, pp. 184–193.
  * [31] R. Y. Rubinstein and D. P. Kroese, _The cross-entropy method: a unified approach to combinatorial optimization, Monte-Carlo simulation and machine learning_. Springer Science & Business Media, 2013.

## APPENDIX

###  V-A Hyperparameters

Our experimental results are based on the hyper-parameters in Table II.

TABLE II: Hyperparameters for our experimental results hyperparameters | value
---|---
epochs (overall) | $1000$
epochs (state and action encoder-decoder) | $500$
epochs (dynamics model) | $500$
learning rate | $1\times 10^{-3}$
batch size | $32$
latent state size | $80$
latent action size | $80$
$\lambda_{1}~{}(\text{action coefficient in the loss function})$ | $450$
$\lambda_{2}~{}(\text{dynamics coefficient in the loss function})$ | $900$
$\lambda_{3}~{}(\text{prediction coefficient in the loss function})$ | $10$

###  V-B Action Score And State Score

We calculate the Mean Square Error (MSE) between the sampled actions and the
ground truth actions.

$\text{MSE}_{\text{action}}=\frac{1}{n}\sum_{i=1}^{n}(u_{\text{sample}}-u_{\text{GT}})^{2},$

where $u_{\text{sample}}$ is the sampled action and $u_{\text{GT}}$ is the
ground truth action. $n=4$ since there are four elements $(x,y,l,\theta)$ in
the action $u$.

We also apply MSE between predicted next state and ground truth next state.

$\text{MSE}_{\text{state}}=\frac{1}{n}\sum_{i=1}^{n}(x_{\text{pred}}-x_{\text{GT}})^{2},$

where $x_{\text{pred}}$ is the predicted next state and $x_{\text{GT}}$ is the
ground truth state. $n=2500$ since the state is a $50\times 50$ image. To get
the MSE for the state, we first do pixel-wise subtraction between
$x_{\text{sample}}$ and $x_{\text{GT}}$. Then we square each pixel value in
the image matrix. After that, we sum up all the pixel values and divide it by
the image size $n$.

After applying different sampled one-step actions on different states and
running it 100 times based on our method and E2C++, we get the mean and
standard deviation for the State Score and the Action Score.

###  V-C Loss Function Expansion

The expansion of the dynamic loss is

$\displaystyle\mathcal{L}_{dyn}=$
$\displaystyle\sum_{t=0}^{T}||K(t)g(t)+L(t)a(t)-g(t+1)||_{2}^{2},$
$\displaystyle=$
$\displaystyle\sum_{t=0}^{T}||K(t)\phi_{e}(x(t))+L(t)\varphi_{e}(u(t))-$
$\displaystyle~{}~{}~{}~{}~{}~{}\phi_{e}(x(t+1))||_{2}^{2}.$

The expansion of the prediction loss is

$\displaystyle\mathcal{L}_{pred}=$
$\displaystyle\sum_{t=0}^{T-1}||x^{pred}(t+1)-x(t+1)||_{2}^{2},$
$\displaystyle=$
$\displaystyle\sum_{t=0}^{T-1}||\phi_{d}\Big{(}K(t)g(t)+L(t)a(t)\Big{)}-x(t+1)||_{2}^{2},$
$\displaystyle=$
$\displaystyle\sum_{t=0}^{T-1}||\phi_{d}\Big{(}K(t)\phi_{e}(x(t))+L(t)\varphi_{e}(u(t))\Big{)}-$
$\displaystyle~{}~{}~{}~{}~{}~{}x(t+1)||_{2}^{2}.$

###  V-D Neural Network Architecture

Our experimental results are based on the following neural networks. Figure 5
includes the input, output, and details of each layer for both the state
encoder and the state decoder. Figure 6 introduces the action encoder and the
action decoder. In the action decoder, the Multiplication and Addition layers
make sure that each element (position $x$, position $y$, moving length $l$,
moving angle $\theta$) in the action are within certain range. We multiply
tensor $[50,50,0.14,2\pi]$ to the output from the Sigmoid layer and then add
another tensor $[0,0,0.01,0]$. The position $x\in[0,50]$, the position
$y\in[0,50]$, the moving length $l\in[0.01,0.15]$, and the moving angle
$\theta\in[0,2\pi]$. Figure 7 includes two mappings. One mapping is from the
state $x$ to the state matrix $K$. The other mapping is from the state $x$ and
the action $u$ to the control matrix $L$.

Figure 5: Neural network architectures for the state encoder and the state
decoder. For the state encoder, the input state $x$ is a $50\times 50$ binary
mask and the output is an $80\times 1$ vector for the latent state $g$. For
the state decoder, the input is an $80\times 1$ vector for the latent state
$g$ and the output is a $50\times 50$ matrix for the reconstructed state
$\hat{x}$. $3\times 3$ conv means the convolutional layer with the kernel size
$3\times 3$. The transConv means the transposed convolutional layer. In some
layers, s is for the stride, p is for the padding, in means the number of
input channels, out means the number of output channels, and FC means the
fully connected layer.

Figure 6: Neural network architectures for the action encoder and the action
decoder. For the action encoder, the input action $u=(x,y,l,\theta)$ is a
$4\times 1$ vector, where $x$ and $y$ are the grasping positions in the image
space, $l$ is the moving length, and $\theta$ is the moving angle. The output
is an $80\times 1$ vector for the latent action. In some layers, FC means the
fully connected layer, in means the number of input features, and out means
the number of output features. ReLU and Sigmoid are the activation functions.
For the Multiplication layer, we do element-wise multiplication between the
input and the tensor $[50,50,0.14,2\pi]$. For the Addition layer, we do
element-wise addition between the input and the tensor $[0,0,0.01,0]$. Figure
7: Neural network architecture for the dynamics model. The inputs are the
$50\times 50$ binary mask for the state $x$ and the $4\times 1$ vector for the
action $u$. In the dynamics model, there are two mappings. In the blue layers,
we apply the convolutional (conv) layer followed by the activation function
ReLU and max pooling for several times. After flattening the output from the
blue layers, we use the flattened vector as the input for the orange layers.
In the orange layers, we apply fully connected (FC) layers and activation
function ReLU. After that, we reshape the vector into an $80\times 80$ state
matrix $K$. For the input action $u$, we first use FC and activation function
ReLU to lift the action into an $80\times 1$ vector in the high-dimensional
space. Then we concatenate this vector with the flattened vector from the blue
layers. Finally, we use the concatenated vector as an input to the green
layers and reshape the output into an $80\times 80$ control matrix. In the
convolutional layers, s is for the stride, p is for the padding, in is for the
number of input channels, and out is for the number of output channels. In the
FC layers, in is for the number of input features and out is for the number of
output features.


* [109] A. N. Morozov, W. van Saarloos, An introductory essay on subcritical instabilities and the transition to turbulence in viscoelastic parallel shear flows, Phys. Rep. 447 (2007) 112–143.
  * [110] J. Azaiez, G. M. Homsy, Linear stability of free shear flow of viscoelastic liquids, Journal of Fluid Mechanics 268 (1994) 37–69.  doi:10.1017/S0022112094001254.
  * [111] C. S. Yih, Instability due to viscosity stratification, Journal of Fluid Mechanics 27 (2) (1967) 337–352.
  * [112] E. J. Hinch, A note on the mechanism of the instability at the interface between two shearing fluids, Journal of Fluid Mechanics 144 (1984) 463–465.  doi:10.1017/S0022112084001695.
  * [113] L. Preziosi, K. P. Chen, D. D. Joseph, Lubricated pipelining: stability of core-annular flow, Journal of Fluid Mechanics 201 (1989) 323–356.
  * [114] N. D. Waters, A. M. Keeley, The stability of two stratified non-Newtonian liquids in Couette flow, Journal of Non-Newtonian Fluid Mechanics 24 (2) (1987) 161–181.
  * [115] K. P. Chen, Elastic instability of the interface in Couette flow of viscoelastic liquids, Journal of Non-Newtonian Fluid Mechanics 40 (2) (1991) 261–267.
  * [116] O. Bonhomme, A. Morozov, J. Leng, A. Colin, Elastic instability in stratified core annular flow, Phys. Rev. E 83 (2011) 065301.
  * [117] E. J. Hinch, O. J. Harris, J. M. Rallison, The instability mechanism for two elastic liquids being co-extruded, Journal of Non-Newtonian Fluid Mechanics 43 (2-3) (1992) 311–324.
  * [118] Y. Y. Renardy, Stability of the interface in two-layer Couette flow of upper convected Maxwell liquids, Journal of Non-Newtonian Fluid Mechanics 28 (1) (1988) 99–115.
  * [119] J. C. Miller, J. M. Rallison, Interfacial instability between sheared elastic liquids in a channel, Journal of Non-Newtonian Fluid Mechanics 143 (2-3) (2007) 71–87.
  * [120] H. J. Wilson, J. M. Rallison, Short wave instability of co-extruded elastic liquids with matched viscosities, Journal of Non-Newtonian Fluid Mechanics 72 (2-3) (1997) 237–251.
  * [121] J. C. Miller, J. M. Rallison, Instability of coextruded elastic liquids at high Weissenberg number, Journal of Non-Newtonian Fluid Mechanics 143 (2-3) (2007) 88–106.
  * [122] Y. Y. Su, B. Khomami, Purely elastic interfacial instabilities in superposed flow of polymeric fluids, Rheologica Acta 31 (5) (1992) 413–420.
  * [123] P. Chokshi, S. Gupta, S. Yadav, A. Agrawal, Interfacial instability in two-layer Couette–Poiseuille flow of viscoelastic fluids, Journal of Non-Newtonian Fluid Mechanics 224 (2015) 17–29.
  * [124] J. Peng, K.-Q. Zhu, Instability of the interface in co-extrusion flow of two UCM fluids in the presence of surfactant, Journal of Non-Newtonian Fluid Mechanics 166 (1-2) (2011) 152–163.
  * [125] V. Shankar, Stability of two-layer viscoelastic plane Couette flow past a deformable solid layer, Journal of Non-Newtonian Fluid Mechanics 117 (2) (2004) 163–182.
  * [126] V. Shankar, Stability of two-layer viscoelastic plane Couette flow past a deformable solid layer: implications of fluid viscosity stratification, Journal of Non-Newtonian Fluid Mechanics 125 (2) (2005) 143–158.
  * [127] M. W. Johnson, Jr., D. Segalman, Model for viscoelastic fluid behavior which allows non-affine deformation, Journal of Non-Newtonian Fluid Mechanics 2 (3) (1977) 225–270.
  * [128] J. Yerushalmi, S. Katz, R. Shinnar, The stability of steady shear flows of some viscoelastic fluids, Chemical Engineering Science 25 (12) (1970) 1891–1902.
  * [129] M. E. Cates, S. M. Fielding, Rheology of giant micelles, Advances in Physics 55 (7-8) (2006) 799–879.
  * [130] H. Rehage, H. Hoffmann, Viscoelastic surfactant solutions: model systems for rheological research, Molecular Physics 74 (5) (1991) 933–973.
  * [131] P. T. Callaghan, Rheo-NMR: nuclear magnetic resonance and the rheology of complex fluids, Reports on Progress in Physics 62 (4) (1999) 599–670.
  * [132] S. Lerouge, J.-F. Berret, Shear-induced transitions and instabilities in surfactant wormlike micelles, in: K. Dusek, J.-F. Joanny (Eds.), Advances in Polymer Science, Polymer Characterization, Vol. 230 of Advances in Polymer Science, Springer Berlin / Heidelberg, 2010, pp. 1–71.
  * [133] M.-A. Fardin, S. Lerouge, Instabilities in wormlike micelle systems, The Eur. Phys. J. E 35 (9) (2012) 1–29.
  * [134] M. A. Fardin, D. Lopez, J. Croso, G. Grégoire, O. Cardoso, G. H. McKinley, S. Lerouge, Elastic turbulence in shear banding wormlike micelles, Phys. Rev. Lett. 104 (2010) 178303.
  * [135] S. Lerouge, M. Argentina, J. P. Decruppe, Interface instability in shear-banding flow, Phys. Rev. Lett. 96 (2006) 088301.
  * [136] S. Lerouge, M. A. Fardin, M. Argentina, G. Grégoire, O. Cardoso, Interface dynamics in shear-banding flow of giant micelles, Soft Matter 4 (9) (2008) 1808–1819.  doi:10.1039/b804915g.
  * [137] P. Nghe, S. M. Fielding, P. Tabeling, A. Ajdari, Interfacially driven instability in the microchannel flow of a shear-banding fluid, Physical Review Letters 104 (24) (2010) 248303.
  * [138] T. Yamamoto, T. Hashimoto, A. Yamashita, Flow analysis for wormlike micellar solutions in an axisymmetric capillary channel, Rheologica Acta 47 (9) (2008) 963–974.
  * [139] M. Cromer, L. P. Cook, G. H. McKinley, Interfacial instability of pressure-driven channel flow for a two-species model of entangled wormlike micellar solutions, Journal of Non-Newtonian Fluid Mechanics 166 (11) (2011) 566–577.
  * [140] S. M. Fielding, Linear instability of planar shear banded flow, Physical Review Letters 95 (13) (2005) 134501.
  * [141] S. M. Fielding, H. J. Wilson, Shear banding and interfacial instability in planar Poiseuille flow, J. of Non-Newt. Fluid Mech. 165 (5-6) (2010) 196–202.  doi:10.1016/j.jnnfm.2009.12.001.
  * [142] H. A. Castillo, H. J. Wilson, Bulk and interfacial modes of instability in channel flow of thixotropic-viscoelasto-plastic fluids with shear-banding, Journal of Non-Newtonian Fluid Mechanics 284 (2020) 104357.
  * [143] G. I. Taylor, Viii. stability of a viscous liquid contained between two rotating cylinders, Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character 223 (605-615) (1923) 289–343.
  * [144] C. D. Andereck, S. S. Liu, H. L. Swinney, Flow regimes in a circular couette system with independently rotating cylinders, Journal of Fluid Mechanics 164 (1986) 155–183.  doi:10.1017/S0022112086002513.
  * [145] W. R. Dean, S. Chapman, Fluid motion in a curved channel, Proceedings of the Royal Society of London. Series A 121 (787) (1928) 402–420.
  * [146] Y. L. Joo, E. S. G. Shaqfeh, Observations of purely elastic instabilities in the taylor–dean flow of a boger fluid, Journal of Fluid Mechanics 262 (1994) 27–73.  doi:10.1017/S002211209400042X.
  * [147] R. F. Ginn, M. M. Denn, Rotational stability in viscoelastic liquids: Theory, AIChE Journal 15 (3) (1969) 450–454.
  * [148] O. Maklad, R. Poole, A review of the second normal-stress difference; its importance in various flows, measurement techniques, results for various complex fluids and theoretical predictions, Journal of Non-Newtonian Fluid Mechanics 292 (2021) 104522.
  * [149] R. I. Tanner, Engineering Rheology, Oxford University Press, Oxford, 2000.
  * [150] R. H. Thomas, K. Walters, The stability of elastico-viscous flow between rotating cylinders. part 1, Journal of Fluid Mechanics 18 (1964) 33–43.
  * [151] R. H. Thomas, K. Walters, The stability of elastico-viscous flow between rotating cylinders. part 2, Journal of Fluid Mechanics 19 (1964) 557–560.
  * [152] D. W. Beard, D. M. H., K. Walters, The stability of elastic-viscous flow between ro- tating cylinders. part 3. overstability in vis- cous and maxwell fluids, J. Fluid Mech. 24 (1966) 321.
  * [153] Y. L. Joo, E. S. G. Shaqfeh, The effects of inertia on the viscoelastic dean and Taylor–Couette flow instabilities with application to coating flows, Physics of Fluids A: Fluid Dynamics 4 (11) (1992) 2415–2431.  doi:10.1063/1.858483.
  * [154] S. J. Muller, R. G. Larson, E. S. Shaqfeh, A purely elastic transition in Taylor-Couette flow, Rheol. Acta 28 (1989) 499–503.
  * [155] R. G. Larson, E. S. G. Shaqfeh, S. J. Muller, A purely elastic instability in Taylor–Couette flow, J. Fluid Mech. 218 (1990) 573–600.
  * [156] E. S. G. Shaqfeh, S. J. Muller, R. G. Larson, The effects of gap width and dilute solution properties on the viscoelastic Taylor-Couette instability, Journal of Fluid Mechanics 235 (1992) 285–317.  doi:10.1017/S0022112092001113.
  * [157] D. F. James, Boger fluids, Annual Review of Fluid Mechanics 41 (1) (2009) 129–142.
  * [158] L. M. Quinzani, G. H. Mckinley, R. A. Brown, R. C. Armstrong, Modeling the rheology of polyisobutylene solutions, Journal of Rheology 34 (5) (1990) 705–748.
  * [159] Y. L. Joo, E. S. Shaqfeh, Viscoelastic Poiseuille flow through a curved channel: A new elastic instability, Phys. Fluids A: Fluid Dyn. 3 (1991) 2043–2046.
  * [160] M. Avgousti, A. N. Beris, Nonaxisymmetric modes in viscoelastic Taylor-Couette flow, Journal of Non-Newtonian Fluid Mechanics 50 (2) (1993) 225 – 251.
  * [161] M. Avgousti, A. N. Beris, Viscoelastic Taylor-Couette flow: bifurcation analysis in the presence of symmetries, Proceedings of the Royal Society of London. Series A 443 (1917) (1993) 17–37.
  * [162] G. I. Ogilvie, M. R. E. Proctor, On the relation between viscoelastic and magnetohydrodynamic flows and their instabilities, Journal of Fluid Mechanics 476 (2003) 389–409.  doi:https://doi.org/10.1017/S0022112002003051.
  * [163] Velikhov, On the high weissenberg number problem, J. Exp. Theor. Phys. 36 (1959) 1398.
  * [164] S. Chandrasekhar, Hydrodynamic and Hydromagnetic Stability, Oxford, Clarendon, 1961\.
  * [165] S. A. Balbus, J. F. Hawley, Powerful local shear instability in weakly magnetized disks. i. linear analysis, Astrophysical Journal 376 (1991) 214–233.
  * [166] J. M. Rallison, E. J. Hinch, Instability of a high-speed submerged elastic jet, Journal of Fluid Mechanics 288 (1995) 311–324.  doi:10.1017/S0022112095001157.
  * [167] B. Stanislav, H. Don, P. Vladimir, Analog of astrophysical magnetorotational instability in a couette-taylor flow of polymer fluids, Phys. Rev. E 80 (2009) 066310.
  * [168] G. I. Ogilvie, A. T. Potter, Magnetorotational-type instability in couette-taylor flow in a viscoelastic polymer liquid, Phys. Rev. Lett. 100 (2008) 074503.
  * [169] J. Magda, R. Larson, A transition occurring in ideal elastic liquids during shear flow, Journal of Non-Newtonian Fluid Mechanics 30 (1) (1988) 1–19.
  * [170] K. Jackson, K. Walters, R. Williams, A rheometrical study of Boger fluids, Journal of Non-Newtonian Fluid Mechanics 14 (1984) 173–188.
  * [171] R. J. Binnington, D. V. Boger, Remarks on non-shear thinning elastic fluids, Polymer Engineering and Science 26 (2) (1986) 133–138.
  * [172] G. H. McKinley, J. A. Byars, R. A. Brown, R. C. Armstrong, Observations on the elastic instability in cone-and-plate and parallel-plate flows of a polyisobutylene boger fluid, Journal of Non-Newtonian Fluid Mechanics 40 (2) (1991) 201 – 229.
  * [173] N. Phan-Thien, Coaxial-disk flow of an Oldroyd-b fluid: exact solution and stability, Journal of Non-Newtonian Fluid Mechanics 13 (3) (1983) 325–340.
  * [174] N. Phan-Thien, Cone-and-plate flow of the Oldroyd-b fluid is unstable, Journal of Non-Newtonian Fluid Mechanics 17 (1) (1985) 37–44.
  * [175] A. Oztekin, R. A. Brown, Instability of a fluid between rotating parallel disks: analysis of the Oldroyd-b fluid, J. Fluid Mech. 225 (1993) 473–502.
  * [176] J. A. Byars, A. Oztekin, R. A. Brown, G. H. McKinley, Spiral instabilities in the flow of highly elastic fluids between rotating parallel disc, J.Fluid Mech. 271 (1994) 173–218.
  * [177] G. H. McKinley, A. Oztekin, J. A. Byars, R. A. Brown, Self-similar spiral instabilities in elastic flows between a cone and a plate, J. Fluid Mech. 285 (1995) 123–164.
  * [178] A. Oztekin, R. A. Brown, G. H. McKinley, Quantitative prediction of the viscoelastic instability in cone-and-plate flow of a boger fluid using a multi-mode giesekus model, Journal of Non-Newtonian Fluid Mechanics 54 (1994) 351–377.
  * [179] P. J. Northey, R. C. Armstrong, R. A. Brown, Finite-amplitude time-periodic states in viscoelastic taylor-couette flow described by the ucm model, Journal of Non-Newtonian Fluid Mechanics 42 (1) (1992) 117–139.
  * [180] R. Sureshkumar, A. N. Beris, M. Avgousti, Non-axisymmetric subcritical bifurcations in viscoelastic taylor-couette flow, Proceedings: Mathematical and Physical Sciences 447 (1929) (1994) 135–153.
  * [181] J. M. White, S. J. Muller, Viscous heating and the stability of newtonian and viscoelastic taylor-couette flows, Phys. Rev. Lett. 84 (2000) 5130–5133.
  * [182] J. M. White, S. J. Muller, Experimental studies on the effect of viscous heating on the hydrodynamic stability of viscoelastic Taylor–Couette flow, Journal of Rheology 47 (6) (2003) 1467–1492.
  * [183] B. M. Baumert, S. J. Muller, Flow regimes in model viscoelastic fluids in a circular couette system with independently rotating cylinders, Phys. Fluids 9 (3) (1997) 566–586.
  * [184] A. Groisman, V. Steinberg, Mechanism of elastic instability in couette flow of polymer solutions: Experiment, Physics of Fluids 10 (10) (1998) 2451–2463.
  * [185] U. A. Al-Mubaiyedh, R. Sureshkumar, B. Khomami, Influence of energetics on the stability of viscoelastic Taylor–Couette flow, Physics of Fluids 11 (11) (1999) 3217–3226.  doi:10.1063/1.870183.
  * [186] U. A. Al-Mubaiyedh, R. Sureshkumar, B. Khomami, Linear stability of viscoelastic Taylor–Couette flow: Influence of fluid rheology and energetics, Journal of Rheology 44 (5) (2000) 1121–1138.  doi:10.1122/1.1289279.
  * [187] J. P. Rothstein, G. H. McKinley, Non-isothermal modification of purely elastic flow instabilities in torsional flows of polymeric fluids, Physics of Fluids 13 (2) (2001) 382–396.  doi:10.1063/1.1338540.
  * [188] D. O. Olagunju, L. P. Cook, G. H. McKinley, Effect of viscous heating on linear stability of viscoelastic cone-and-plate flow: axisymmetric case, Journal of Non-Newtonian Fluid Mechanics 102 (2) (2002) 321–342.
  * [189] T. Mullin, Experimental studies of transition to turbulence in a pipe, Ann. Rev. Fluid Mech. 43 (2011) 1–24.
  * [190] S. Grossmann, D. Lohse, C. Sun, High–reynolds number taylor-couette turbulence, Annual Review of Fluid Mechanics 48 (1) (2016) 53–80.
  * [191] C. S. DUTCHER, S. J. MULLER, Spatio-temporal mode dynamics and higher order transitions in high aspect ratio newtonian taylor–couette flows, Journal of Fluid Mechanics 641 (2009) 85–113.  doi:10.1017/S0022112009991431.
  * [192] B. A. Schiamberg, L. T. Shereda, H. hu, R. G. Larson, Transitional pathway to elastic turbulence in torsional, parallel-plate flow of a polymer solution, Journal of Fluid Mechanics 554 (2006) 191–216.  doi:10.1017/S0022112006009426.
  * [193] A. Fouxon, V. Lebedev, Spectra of turbulence in dilute polymer solutions, Physics of Fluids 15 (7) (2003) 2060–2072.
  * [194] A. Groisman, V. Steinberg, Elastic turbulence in curvilinear flows of polymer solutions, New J. Phys. 6 (2004) 29.
  * [195] A. Groisman, V. Steinberg, Couette-taylor flow in a dilute polymer solution, Phys. Rev. Lett. 77 (1996) 1480–1483.
  * [196] A. Groisman, V. Steinberg, Solitary vortex pairs in viscoelastic couette flow, Phys. Rev. Lett. 78 (1997) 1460–1463.
  * [197] A. Groisman, V. Steinberg, Elastic vs. inertial instability in a polymer solution flow, Europhysics Letters (EPL) 43 (2) (1998) 165–170.
  * [198] K. A. Kumar, M. D. Graham, Solitary coherent structures in viscoelastic shear flow: Computation and mechanism, Phys. Rev. Lett. 85 (2000) 4056–4059.  doi:10.1103/PhysRevLett.85.4056.
URL https://link.aps.org/doi/10.1103/PhysRevLett.85.4056

  * [199] D. G. THOMAS, R. SURESHKUMAR, B. KHOMAMI, Thermo-mechanical instabilities in dean and taylor–couette flows: mechanisms and scaling laws, Journal of Fluid Mechanics 517 (2004) 251–279.  doi:10.1017/S002211200400093X.
  * [200] B. M. Baumert, S. J. Muller, Axisymmetric and non-axisymmetric elastic and inertio-elastic instabilities in taylor–couette flow, Journal of Non-Newtonian Fluid Mechanics 83 (1) (1999) 33–69.
  * [201] C. S. Dutcher, S. J. Muller, Effects of weak elasticity on the stability of high reynolds number co- and counter-rotating taylor-couette flows, Journal of Rheology 55 (6) (2011) 1271–1295.
  * [202] C. S. Dutcher, S. J. Muller, Effects of moderate elasticity on the stability of co- and counter-rotating taylor–couette flows, Journal of Rheology 57 (3) (2013) 791–812.
  * [203] A. N. Beris, M. Avgousti, A. Souvaliotis, Spectral calculations of viscoelastic flows: evaluation of the giesekus constitutive equation in model flow problems, Journal of Non-Newtonian Fluid Mechanics 44 (1992) 197–228.
  * [204] R. Larson, S. Muller, E. Shaqfeh, The effect of fluid rheology on the elastic Taylor-Couette instability, Journal of Non-Newtonian Fluid Mechanics 51 (1994) 195–225.
  * [205] M. Somasi, B. Khomami, Linear stability and dynamics of viscoelastic flows using time-dependent stochastic simulation techniques, Journal of Non-Newtonian Fluid Mechanics 93 (2) (2000) 339–362.
  * [206] M. Somasi, B. Khomami, A new approach for studying the hydrodynamic stability of fluids with microstructure, Physics of Fluids 13 (6) (2001) 1811–1814.
  * [207] R. Prabhakar, S. Gadkari, T. Gopesh, M. J. Shaw, Influence of stretching induced self-concentration and self-dilution on coil-stretch hysteresis and capillary thinning of unentangled polymer solutions, Journal of Rheology 60 (3) (2016) 345–366.
  * [208] A. V. Bazilevsky, V. M. Entov, A. N. Rozhkov, Liquid filament microrheometer and some of its applications, in: D. R. Oliver (Ed.), Third European Rheology Conference, Elsevier, 1990.
  * [209] G. H. McKinley, A. Tripathi, How to extract the newtonian viscosity from capillary breakup measurements in a filament rheometer, Journal of Rheology 44 (3) (2000) 653–670.
  * [210] G. H. McKinley, P. Pakdel, A. Oztekin, Rheological and geometric scaling of purely elastic flow instabilities, Journal of Non-Newtonian Fluid Mechanics 67 (1996) 19–47.
  * [211] S. W. Saric, Gortler vortices, Ann. Rev. Fluid Mech. 26 (1994) 379–409.
  * [212] G. Subramanian, D. Koch, Inertial effects on the transfer of heat or mass from neutrally buoyant spheres in a steady linear velocity field, Phys. Fluids 18 (7) (2006) 073302.
  * [213] R. J. Poole, Three-dimensional viscoelastic instabilities in microchannels, Journal of Fluid Mechanics 870 (2019) 1–4.  doi:10.1017/jfm.2019.260.
  * [214] M. T. Arigo, D. Rajagopalan, N. Shapley, G. H. Mckinley, Sedimentation of a sphere through an elastic fluid. part 1. steady motion, Journal of Non-Newtonian Fluid Mechanics 60 (1995) 225–257.
  * [215] D. Rajagopalan, M. T. Arigo, G. H. Mckinley, Sedimentation of a sphere through an elastic fluid. part 2. transient motion, Journal of Non-Newtonian Fluid Mechanics 65 (1996) 17–46.
  * [216] O. Hassager, Working group on numerical techniques, Journal of Non-Newtonian Fluid Mechanics 29 (1988) 2–5.
  * [217] R. A. Brown, G. H. McKinley, Report on the viiith international workshop on numerical methods in viscoelastic flows, Journal of Non-Newtonian Fluid Mechanics 52 (1994) 407–413.
  * [218] M. Alves, P. Oliveira, F. Pinho, Numerical methods for viscoelastic fluid flows, Annu. Rev. Fluid Mech. 53 (1) (2021) 509–541.
  * [219] R. Keunings, On the high weissenberg number problem, Journal of Non-Newtonian Fluid Mechanics 20 (1986) 209–226.
  * [220] O. Harlen, J. Rallison, M. Chilcott, High-deborah-number flows of dilute polymer solutions, Journal of Non-Newtonian Fluid Mechanics 34 (3) (1990) 319–349.
  * [221] M. Chilcott, J. Rallison, Creeping flow of dilute polymer solutions past cylinders and spheres, Journal of Non-Newtonian Fluid Mechanics 29 (C) (1988) 381–432.
  * [222] P. J. Oliveira, A. I. Miranda, A numerical study of steady and unsteady viscoelastic flow past bounded cylinders, Journal of Non-Newtonian Fluid Mechanics 127 (1) (2005) 51–66.
  * [223] M. Bajaj, M. Pasquali, J. R. Prakash, Coil-stretch transition and the breakdown of computations for viscoelastic fluid flow around a confined cylinder, Journal of Rheology 52 (1) (2008) 197–223.
  * [224] M. A. Hulsen, R. Fattal, R. Kupferman, Flow of viscoelastic fluids past a cylinder at high weissenberg number: Stabilized simulations using matrix logarithms, Journal of Non-Newtonian Fluid Mechanics 127 (1) (2005) 27–39.
  * [225] K. Gardner, E. R. Pike, M. J. Miles, A. Keller, K. Tanaka, Photon-correlation velocimetry of polystyrene solutions in extensional flow fields, Polymer 23 (1982) 1435–1442.
  * [226] J. Feng, L. Leal, Numerical simulations of the flow of dilute polymer solutions in a four-roll mill, Journal of Non-Newtonian Fluid Mechanics 72 (2) (1997) 187–218.
  * [227] O. Harlen, E. Hinch, J. Rallison, Birefringent pipes: the steady flow of a dilute polymer solution near a stagnation point, Journal of Non-Newtonian Fluid Mechanics 44 (1992) 229–265.
  * [228] D. Cruz, F. Pinho, Analytical solution of steady 2D wall-free extensional flows of UCM fluids, Journal of Non-Newtonian Fluid Mechanics 223 (2015) 157–164.
  * [229] P. Becherer, A. N. Morozov, W. van Saarloos, Probing a subcritical instability with an amplitude expansion: An exploration of how far one can get, Physica D 238 (18) (2009) 1827–1840.
  * [230] S. J. Haward, M. S. N. Oliveira, M. A. Alves, G. H. McKinley, Optimized cross-slot flow geometry for microfluidic extensional rheometry, Phys. Rev. Lett. 109 (2012) 128301.
  * [231] A. Muller, J. Odell, J. Tatham, Stagnation-point extensional flow behaviour of m1, Journal of Non-Newtonian Fluid Mechanics 35 (2) (1990) 231–250.
  * [232] J. Broadbent, D. Pountney, K. Walters, Experimental and theoretical aspects of the two-roll mill problem, Journal of Non-Newtonian Fluid Mechanics 3 (4) (1978) 359–378.
  * [233] R. C. Ng, L. G. Leal, Concentration effects on birefringence and flow modification of semidilute polymer solutions in extensional flows, Journal of Rheology 37 (3) (1993) 443–468.
  * [234] P. E. Arratia, C. C. Thomas, J. Diorio, J. P. Gollub, Elastic instabilities of polymer solutions in cross-channel flow, Physical Review Letters 96 (2006) 144502\.
  * [235] P. C. Sousa, F. T. Pinho, M. S. N. Oliveira, M. A. Alves, Purely elastic flow instabilities in microscale cross-slot devices, Soft Matter 11 (2015) 8856–8862.
  * [236] R. J. Poole, M. A. Alves, P. J. Oliveira, Purely elastic flow asymmetries, Phys. Rev. Lett. 99 (2007) 164503.
  * [237] G. N. Rocha, R. J. Poole, M. A. Alves, P. J. Oliveira, On extensibility effects in the cross-slot flow bifurcation, J. Non-Newt. Fluid Mech. 156 (2009) 58–69.
  * [238] A. Afonso, M. Alves, F. Pinho, Purely elastic instabilities in three-dimensional cross-slot geometries, Journal of Non-Newtonian Fluid Mechanics 165 (13) (2010) 743–751.
  * [239] S. J. Haward, T. J. Ober, M. S. Oliveira, M. A. Alves, G. H. McKinley, Extensional rheology and elastic instabilities of a wormlike micellar solution in a microfluidic cross-slot device, Soft Matter 8 (2012) 536–555.
  * [240] F. Cruz, R. Poole, A. Afonso, F. Pinho, P. Oliveira, M. Alves, A new viscoelastic benchmark flow: Stationary bifurcation in a cross-slot, Journal of Non-Newtonian Fluid Mechanics 214 (2014) 57–68.
  * [241] H. J. Wilson, Open mathematical problems regarding non-newtonian fluids, Nonlinearity 25 (3) (2012) R45–R51.
  * [242] R. Lagnado, N. Phan-Thien, L. Leal, The stability of two-dimensional linear flows of an Oldroyd-type fluid, Journal of Non-Newtonian Fluid Mechanics 18 (1) (1985) 25–59.
  * [243] R. R. Lagnado, N. Phan-Thien, L. G. Leal, The stability of two‐dimensional linear flows, Phys. Fluids 27 (1984) 1094.
  * [244] E. Hinch, The flow of an oldroyd fluid around a sharp corner, Journal of Non-Newtonian Fluid Mechanics 50 (2) (1993) 161–171.
  * [245] M. Renardy, The stresses of an upper convected maxwell fluid in a newtonian velocity field near a re-entrant corner, Journal of Non-Newtonian Fluid Mechanics 50 (2) (1993) 127–134.
  * [246] M. Davoodi, A. F. Domingues, R. J. Poole, Control of a purely elastic symmetry-breaking flow instability in cross-slot geometries, Journal of Fluid Mechanics 881 (2019) 1123–1157.  doi:10.1017/jfm.2019.781.
  * [247] S. J. Haward, G. H. McKinley, Instabilities in stagnation point flows of polymer solutions, Physics of Fluids 25 (8) (2013) 083104.  doi:10.1063/1.4818151.
  * [248] S. J. Haward, G. H. McKinley, A. Q. Shen, Elastic instabilities in planar elongational flow of monodisperse polymer solutions, Scientific Reports 6 (2016) 33029.
  * [249] A. Oztekin, B. Alakus, G. H. McKinley, Stability of planar stagnation flow of a highly viscoelastic fluid, Journal of Non-Newtonian Fluid Mechanics 72 (1) (1997) 1–29.
  * [250] O. Harris, J. Rallison, Instabilities of a stagnation point flow of a dilute polymer solution, Journal of Non-Newtonian Fluid Mechanics 55 (1) (1994) 59–90.
  * [251] L. Xi, M. D. Graham, A mechanism for oscillatory instability in viscoelastic cross-slot flow, Journal of Fluid Mechanics 622 (2009) 145–165.  doi:10.1017/S0022112008005119.
  * [252] J. P. Rothstein, G. H. McKinley, Extensional flow of a polystyrene boger fluid through a 4:1:4 axisymmetric contraction/expansion, Journal of Non-Newtonian Fluid Mechanics 86 (1) (1999) 61–88.
  * [253] D. M. Binding, Further considerations of axisymmetric contraction flows, Journal of Non-Newtonian Fluid Mechanics 42 (1991) 27–42.
  * [254] G. P. Samsal, A finite volume approach for calculation of viscoelastic flow through an abrupt axisymmetric contraction, Journal of Non-Newtonian Fluid Mechanics 56 (1995) 15–47.
  * [255] G. H. McKinley, W. P. Raiford, R. A. Brown, R. C. Armstrong, Nonlinear dynamics of viscoelastic flow in axisymmetric abrupt contractions, Journal of Fluid Mechanics 223 (1991) 411–456.  doi:10.1017/S0022112091001489.
  * [256] J. P. Rothstein, G. H. McKinley, The axisymmetric contraction–expansion: the role of extensional rheology on vortex growth dynamics and the enhanced pressure drop, Journal of Non-Newtonian Fluid Mechanics 98 (1) (2001) 33–63.  doi:https://doi.org/10.1016/S0377-0257(01)00094-5.
URL https://www.sciencedirect.com/science/article/pii/S0377025701000945

  * [257] L. E. Rodd, T. P. Scott, D. V. Boger, J. J. Cooper-White, G. H. McKinley, The inertio-elastic planar entry flow of low-viscosity elastic fluids in micro-fabricated geometries, J. Non-Newtonian Fluid Mech. 129 (2005) 1–22.
  * [258] L. E. Rodd, J. J. Cooper-White, G. H. Boger, D V amd McKinley, Role of the elasticity number in the entry flow of dilute polymer solutions in micro-fabricated contraction geometries, J. Non-Newtonian Fluid Mech. 143 (2007) 170–191.
  * [259] D. A. Boger, Viscoelastic flows through contractions, Annual Reviews of Fluid Mechanics 19 (1987) 157–182.
  * [260] R. Keiller, Entry-flow calculations for the oldroyd-b and fene equations, Journal of Non-Newtonian Fluid Mechanics 46 (2) (1993) 143–178.
  * [261] P. Szabo, J. Rallison, E. Hinch, Start-up of flow of a fene-fluid through a 4:1:4 constriction in a tube, Journal of Non-Newtonian Fluid Mechanics 72 (1) (1997) 73–86.
  * [262] M. A. Alves, P. J. Oliveira, F. T. Pinho, On the effect of contraction ratio in viscoelastic flow through abrupt contractions, Journal of Non-Newtonian Fluid Mechanics 122 (1) (2004) 117–130, xIIIth International Workshop on Numerical Methods for Non-Newtonian Flows.
  * [263] M. S. Oliveira, P. J. Oliveira, F. T. Pinho, M. A. Alves, Effect of contraction ratio upon viscoelastic flow in contractions: The axisymmetric case, Journal of Non-Newtonian Fluid Mechanics 147 (1) (2007) 92–108.
  * [264] D. G. Hassell, M. R. Mackley, M. Sahin, H. J. Wilson, O. G. Harlen, T. C. B. McLeish, Molecular physics of a polymer engineering instability: experiments and computation, Physical Review E 77 (5) (2008) 050801.
  * [265] A. E. Likhtman, G. R. S, Simple constitutive equation for linear polymer melts derived from molecular theory: Rolie–poly equation, J. Non-Newt. Fluid Mech. 114 (2003) 1–12.
  * [266] S. J. Haward, K. Toda-Peters, A. Q. Shen, Steady viscoelastic flow around high-aspect-ratio, low-blockage-ratio microfluidic cylinders, Journal of Non-Newtonian Fluid Mechanics 254 (2018) 23–35.
  * [267] S. J. Haward, N. Kitajima, K. Toda-Peters, T. Takahashi, A. Q. Shen, Flow of wormlike micellar solutions around microfluidic cylinders with high aspect ratio and low blockage ratio, Soft Matter 15 (2019) 1927–1941.
  * [268] D. Fabris, S. J. Muller, D. Liepmann, Wake measurements for flow around a sphere in a viscoelastic fluid, Physics of Fluids 11 (12) (1999) 3599–3612.
  * [269] S. Kenney, K. Poper, G. Chapagain, G. F. Christopher, Large deborah number flows around confined microfluidic cylinders, Rheologica Acta 52 (2013) 485–497.
  * [270] G. H. McKinley, R. C. Armstrong, R. A. Brown, The wake instability in viscoelastic flow past confined circular cylinders, Philos. Trans. R. Soc. A 344 (1993) 265–304.  doi:10.1098/rsta.1993.0091.
  * [271] S. Varchanis, C. C. Hopkins, A. Q. Shen, J. Tsamopoulos, S. J. Haward, Asymmetric flows of complex fluids past confined cylinders: A comprehensive numerical study with experimental validation, Physics of Fluids 32 (5) (2020) 053103\.
  * [272] S. J. Haward, C. C. Hopkins, A. Q. Shen, Asymmetric flow of polymer solutions around microfluidic cylinders: Interaction between shear-thinning and viscoelasticity, Journal of Non-Newtonian Fluid Mechanics 278 (2020) 104250.
  * [273] P. S. Doyle, E. S. Shaqfeh, Dynamic simulation of freely-draining, flexible bead-rod chains: Start-up of extensional and shear flow, Journal of Non-Newtonian Fluid Mechanics 76 (1) (1998) 43–78.
  * [274] P. S. Doyle, E. S. Shaqfeh, G. H. McKinley, S. H. Spiegelberg, Relaxation of dilute polymer solutions following extensional flow, Journal of Non-Newtonian Fluid Mechanics 76 (1) (1998) 79–110.  doi:https://doi.org/10.1016/S0377-0257(97)00113-4.
URL https://www.sciencedirect.com/science/article/pii/S0377025797001134

  * [275] G. H. McKinley, Steady and transient motion of spherical particles in viscoelastic liquids, in: D. DeKee, R.-P. Chhabra (Eds.), Transport Processes in Bubbles, Drops and Particles, Taylor and Francis, 2002.
  * [276] J. Rallison, Dissipative stresses in dilute polymer solutions, Journal of Non-Newtonian Fluid Mechanics 68 (1) (1997) 61–83.
  * [277] M. Verhoef, B. van den Brule, M. Hulsen, On the modelling of a pib/pb boger fluid in extensional flow, Journal of Non-Newtonian Fluid Mechanics 80 (2) (1999) 155–182.
  * [278] B. Yang, B. Khomami, Simulations of sedimentation of a sphere in a viscoelastic fluid using molecular based constitutive models, Journal of Non-Newtonian Fluid Mechanics 82 (2) (1999) 429–452.  doi:https://doi.org/10.1016/S0377-0257(98)00174-8.
URL https://www.sciencedirect.com/science/article/pii/S0377025798001748

  * [279] A. P. KOPPOL, R. SURESHKUMAR, A. ABEDIJABERI, B. KHOMAMI, Anomalous pressure drop behaviour of mixed kinematics flows of viscoelastic polymer solutions: a multiscale simulation approach, Journal of Fluid Mechanics 631 (2009) 231–253.  doi:10.1017/S0022112009006922.
  * [280] A. Abedijaberi, B. Khomami, Sedimentation of a sphere in a viscoelastic fluid: a multiscale simulation approach, Journal of Fluid Mechanics 694 (2012) 78–99.  doi:10.1017/jfm.2011.504.
  * [281] R. L. Panton, Incompressible Flow, 4th Edition, John Wiley & Sons, 2013.
  * [282] P. G. Drazin, Introduction to Hydrodynamic Stability, Cambridge University Press, 2002.
  * [283] F. Charru, Hydrodynamic Instabilities, Cambridge University Press, 2011.
  * [284] P. J. Schmid, Nonmodal stability theory, Annu. Rev. Fluid Mech. 39 (2007) 129–162.  doi:10.1146/annurev.fluid.38.050304.092139.
  * [285] M. R. Jovanović, From bypass transition to flow control and data-driven turbulence modeling: An input–output viewpoint, Annu. Rev. Fluid Mech. 53 (2021) 311–345.
  * [286] C. C. T. Pringle, R. R. Kerswell, Using nonlinear transient growth to construct the minimal seed for shear flow turbulence, Phys. Rev. Lett. 105 (2010) 154502\.
  * [287] R. Kerswell, Nonlinear nonmodal stability theory, Ann. Rev. Fluid Mech. 50 (2018) 319–345.
  * [288] S. Grossmann, The onset of shear flow turbulence, Rev. Mod. Phys. 72 (2000) 603–618.
  * [289] B. K. Lieu, M. R. Jovanović, S. Kumar, Worst-case amplification of disturbances in inertialess Couette flow of viscoelastic fluids, J. Fluid Mech. 723 (2013) 232–263.
  * [290] A. W. Naylor, G. R. Sell, Linear Operator Theory in Engineering and Sceince, Holt, Reinhart and Winston, 1971.
  * [291] G. Strang, Linear Algebra and its Applications, 3rd Edition, Harcourt Brace Jovanovich, 1988.
  * [292] M. R. Jovanović, B. Bamieh, Componentwise energy amplification in channel flows, J. Fluid Mech. 534 (2005) 145–183.
  * [293] L. Brandt, The lift-up effect: The linear mechanism behind transition and turbulence in shear flows, Eur. J. Mech. B/Fluids 47 (2014) 80–96.
  * [294] T. Ellingsen, E. Palm, Stability of linear flow, Phys. Fluids 18 (1975) 487–488.
  * [295] L. N. Trefethen, A. E. Trefethen, S. C. Reddy, T. A. Driscoll, Hydrodynamic stability without eigenvalues, Science 261 (1993) 578–584.
  * [296] K. M. Butler, B. F. Farrell, Three-dimensional optimal perturbations in viscous shear flow, Phys. Fluids A 4 (1992) 1637–1650.
  * [297] L. H. Gustavsson, Energy growth of three-dimensional disturbances in plane Poiseuille flow, J. Fluid Mech. 224 (1991) 241–260.
  * [298] D. S. Henningson, A. Lunbladh, A. V. Johansson, A mechanism for bypass transition from localized disturbances in wall-bounded shear flows, J. Fluid Mech. 250 (1993) 169–207.
  * [299] S. C. Reddy, D. S. Henningson, Energy growth in viscous channel flows, J. Fluid Mech. 252 (1993) 209–238.
  * [300] B. Bamieh, M. Dahleh, Energy amplification in channel flows with stochastic excitations, Phys. Fluids 13 (2001) 3258–3269.
  * [301] B. F. Farrell, P. J. Ioannou, Stochastic forcing of the linearized Navier-Stokes equations, Phys. Fluids A 5 (1993) 2600–2609.
  * [302] N. Hoda, M. R. Jovanović, S. Kumar, Energy amplification in channel flows of viscoelastic fluids, J. Fluid Mech. 601 (2008) 407–424.
  * [303] N. Hoda, M. R. Jovanović, S. Kumar, Frequency responses of streamwise-constant perturbations in channel flows of Oldroyd-B fluids, J. Fluid Mech. 625 (2009) 411–434.
  * [304] R. Sureshkumar, M. D. Smith, R. C. Armstrong, R. A. Brown, Linear stability and dynamics of viscoelastic flows using time-dependent numerical simulations, J. Non-Newtonian Fluid Mech. 82 (1999) 57–104.
  * [305] K. Atalik, R. Keunings, Non-linear temporal stability analysis of viscoelastic plane channel flows using a fully spectral method, J. Non-Newtonian Fluid Mech. 102 (2002) 299–319.
  * [306] R. Kupferman, On the linear stability of plane Couette flow for an Oldroyd-B fluid and its numerical approximation, J. Non-Newtonian Fluid Mech. 127 (2005) 169–190.
  * [307] C. R. Doering, B. Eckhardt, J. Schumacher, Failure of energy stability in Oldroyd-B fluids at arbitrarily low Reynolds numbers, J. Non-Newtonian Fluid Mech. 135 (2006) 92–96.
  * [308] M. Renardy, Stress modes in linear stability of viscoelastic flows, J. Non-Newtonian Fluid Mech. 159 (2009) 137–140.
  * [309] G. Hariharan, M. R. Jovanović, S. Kumar, Amplification of localized body forces in channel flows of viscoelastic fluids, J. Non-Newtonian Fluid Mech. 260 (2018) 40–53.
  * [310] K. P. Nolan, A. Agarwal, S. Lei, R. Shields, Viscoelastic flow in an obstructed microchannel at high Weissenberg number, Microfluid. Nanofluidics 20 (2016) 1–12.
  * [311] A. Agarwal, L. Brandt, T. A. Zaki, Linear and nonlinear evolution of a localized disturbance in polymeric channel flow, J. Fluid Mech. 760 (2014) 278–303.
  * [312] G. Hariharan, M. R. Jovanović, S. Kumar, Localized stress amplification in inertialess channel flows of viscoelastic fluids, J. Non-Newtonian Fluid Mech. 291 (2021) 104514.
  * [313] A. Morozov, W. van Sarloos, Subcritical instabilities in plane Poiseuille flow of an Oldroyd-b fluid, J. Stat. Phys. 175 (2019) 554–577.
  * [314] I. Hameduddin, D. F. Gayme, T. A. Zaki, Perturbative expansions of the conformation tensor in viscoelastic flows, J. Fluid Mech. 858 (2019) 377–406.
  * [315] L. Biancofiore, L. Brandt, T. A. Zaki, Streak instability in viscoelastic Couette flow, Phys. Rev. Fluids 2 (2017) 043304.
  * [316] G. Hariharan, S. Kumar, M. R. Jovanović, Well-conditioned ultraspherical and spectral integration methods for resolvent analysis of channel flows of Newtonian and viscoelastic fluids, J. Comput. Phys. 439 (2021) 110241.
  * [317] B. Meulenbroek, C. Storm, V. Bertola, C. Wagner, D. Bonn, W. van Saarloos, Intrinsic route to melt fracture in polymer extrusion: A weakly nonlinear subcritical instability of viscoelastic Poiseuille flow, Phys. Rev. Lett. 90 (2003) 024502.
  * [318] B. Meulenbroek, C. Storm, A. N. Morozov, W. van Saarloos, Weakly nonlinear subcritical instability of visco-elastic Poiseuille flow, J. Non-Newtonian Fluid Mech. 116 (2004) 235–268.
  * [319] M. C. Cross, P. C. Hohenberg, Pattern-formation outside of equilibrium, Rev. Mod. Phys. 65 (3, 2) (1993) 851–1112.  doi:10.1103/RevModPhys.65.851.
  * [320] M. van Hecke, P. C. Hohenberg, W. van Saarloos, Amplitude equations for pattern forming systems, in: H. van Beijeren, M. H. Ernst (Eds.), Fundamental Problems in Statistical Mechanics VIII, North-Holland, Amsterdam, 1994, pp. 245–278.
  * [321] R. B. Bird, C. F. Curtiss, R. C. Armstrong, O. Hassager, Dynamics of polymeric liquids, Vol. 2, kinetic theory, John Wiley and Sons, 1987.
  * [322] D. Bonn, F. Ingremeau, Y. Amarouchene, H. Kellay, Large velocity fluctuations in small-Reynolds-number pipe flow of polymer solutions, Phys. Rev. E 84 (2011) 045301.  doi:10.1103/PhysRevE.84.045301.
  * [323] M. D. Graham, D. Floryan, Exact coherent states and the nonlinear dynamics of wall-bounded turbulent flows, Annu. Rev. Fluid Mech. 53 (1) (2021) 227–253.
  * [324] R. Owens, T. Phillips, Computational Rheology, Computational Rheology, Imperial College Press, 2002.
  * [325] R. Fattal, R. Kupferman, Time-dependent simulation of viscoelastic flows at high weissenberg number using the log-conformation representation, J. Non-Newton. Fluid Mech. 126 (1) (2005) 23 – 37.
  * [326] F. Pimenta, M. Alves, Stabilization of an open-source finite-volume solver for viscoelastic fluid flows, J. Non-Newton. Fluid Mech. 239 (2017) 85 – 104.
  * [327] B. Eckhardt, T. Schneider, How does flow in a pipe become turbulent?, Eur. Phys. J. B 64 (2008) 457–462.
  * [328] D. Barkley, Theoretical perspective on the route to turbulence in a pipe, J. Fluid Mech. 803.
  * [329] M. Nagata, Three-dimensional finite-amplitude solutions in plane Couette flow: bifurcation from infinity, Journal of Fluid Mechanics 217 (1990) 519–527.  doi:10.1017/S0022112090000829.
  * [330] J. Hamilton, J. Kim, F. Waleffe, Regeneration mechanisms of near-wall turbulence structures, J. of Fluid Mech. 287 (1995) 317–348.
  * [331] F. Waleffe, On a self-sustaining process in shear flows, Phys. Fluids 9 (4) (1997) 883–900.
  * [332] B. Hof, C. W. H. van Doorne, J. Westerweel, F. T. M. Nieuwstadt, H. Faisst, B. Eckhardt, H. Wedin, R. R. Kerswell, F. Waleffe, Experimental observation of nonlinear traveling waves in turbulent pipe flow, Science 305 (5690) (2004) 1594–1598.
  * [333] L. S. Tuckerman, M. Chantry, D. Barkley, Patterns in wall-bounded shear flows, Annu. Rev. Fluid Mech. 52 (1) (2020) 343–367.
  * [334] T. Burghelea, E. Segre, I. Bar-Joseph, A. Groisman, V. Steinberg, Chaotic flow and efficient mixing in a microchannel with a polymer solution, Phys. Rev. E 69 (2004) 066305.  doi:10.1103/PhysRevE.69.066305.
  * [335] Y. Dubief, J. Page, R. R. Kerswell, V. E. Terrapon, V. Steinberg, A first coherent structure in elasto-inertial turbulence (2020).  arXiv:2006.06770.
  * [336] A. N. Morozov, W. van Saarloos, Subcritical instabilities in plane Poiseuille flow of an Oldroyd-b fluid, J. Stat. Phys. 175 (2019) 554–577.
  * [337] R. Prabhakar, J. R. Prakash, T. Sridhar, Effect of configuration-dependent intramolecular hydrodynamic interaction on elastocapillary thinning and breakup of filaments of dilute polymer solutions, Journal of Rheology 50 (6) (2006) 925–947.
  * [338] A. Varshney, V. Steinberg, Elastic alfven waves in elastic turbulence, Nature Communications 10 (2019) 652.
  * [339] H. Bodiguel, H. Beaumont, A. Machado, L. Martinie, H. Kellay, A. Colin, Flow enhancement due to elastic turbulence in channel flows of shear thinning fluids, Phys. Rev. Lett. 114 (2015) 028302(5).
  * [340] R. J. Poole, Elastic instabilities in parallel shear flows of a viscoelastic shear-thinning liquid, Phys. Rev. Fluids 1 (2016) 041301.
  * [341] C. Wen, R. J. Poole, A. P. Willis, D. J. C. Dennis, Experimental evidence of symmetry-breaking supercritical transition in pipe flow of shear-thinning fluids, Phys. Rev. Fluids 2 (2017) 031901.
  * [342] L. Picaut, O. Ronsin, C. Caroli, T. Baumberger, Experimental evidence of a helical, supercritical instability in pipe flow of shear thinning fluids, Phys. Rev. Fluids 2 (2017) 083303.
  * [343] B. Chandra, R. Mangal, D. Das, V. Shankar, Instability driven by shear thinning and elasticity in the flow of concentrated polymer solutions through microtubes, Phys. Rev. Fluids 4 (2019) 083301.
  * [344] H. Wilson, J. Rallison, Instability of channel flow of a shear-thinning White-Metzner fluid, J. Non-Newtonian Fluid Mech. 87 (1999) 75–96.
  * [345] H. Wilson, V. Loridan, Linear instability of a highly shear-thinning fluid in channel flow, J. Non-Newtonian Fluid Mech. 223 (2015) 200–208.
  * [346] H. Castillo, H. Wilson, Towards a mechanism for instability in channel flow of highly shear-thinning viscoelastic fluids, J. Non-Newtonian Fluid Mech. 247 (2017) 15–21.
  * [347] E. J. Hemingway, S. M. Fielding, Interplay of edge fracture and shear banding in complex fluids, Journal of Rheology 64 (5) (2020) 1147–1159.
  * [348] J. F. Brady, I. C. Carpen, Second normal stress jump instability in non-newtonian fluids, Journal of Non-Newtonian Fluid Mechanics 102 (2) (2002) 219–232, a Collection of Papers Dedicated to Professor ANDREAS ACRIVOS on the Occasion of his Retirement from the Benjamin Levich Institute for Physiochemical Hydrodynamics and the City College of the CUNY.
  * [349] S. M. Fielding, P. D. Olmsted, Early stage kinetics in a unified model of shear-induced demixing and mechanical shear banding instabilities, Physical review letters 90 (22) (2003) 224501.
  * [350] S. M. Fielding, P. D. Olmsted, Flow phase diagrams for concentration-coupled shear banding, The European Physical Journal E 11 (1) (2003) 65–83.
  * [351] M. Cromer, M. C. Villet, G. H. Fredrickson, L. G. Leal, Shear banding in polymer solutions, Physics of Fluids 25 (5) (2013) 051703.
  * [352] M. Cromer, G. H. Fredrickson, L. G. Leal, A study of shear banding in polymer solutions, Physics of Fluids 26 (6) (2014) 063101.
  * [353] J. Eggers, Instability of a polymeric thread, Physics of Fluids 26 (3) (2014) 033106\.
  * [354] J. D. Peterson, G. H. Fredrickson, L. G. Leal, Does shear induced demixing resemble a thermodynamically driven instability?, Journal of Rheology 63 (2) (2019) 335–359.
  * [355] R. G. Larson, Flow-induced mixing, demixing, and phase transitions in polymeric fluids, Rheol. Acta 31 (1992) 497–520.