Dependence of long GRBs duration histogram on the cosmological model
Sec:GRB_Histogram In this section, we consider two cosmological models for
$\omega(z)$ first $\Lambda CDM$ model with $\omega(z)=-1$ and second a model
with $\omega(z)=-1+\alpha z$, where we assume $\alpha=0.9$. we want to compare
the histogram of LGRBs duration resulting from this $\omega CDM$ model and
$\Lambda CDM$ model. First, from Eq.(Eq:Gamma_Def) we calculate $\Gamma(z)$
for $\Lambda CDM$ and $\omega CDM$ with $\omega(z)=-1+0.9z$ models. The
results for $\Gamma(z)/R_{0}$ are shown in Fig.fig:Gamma.

[width=0.7]GRB_Histogram/Gamma_Rz.png

Figure :  The orange line shows the theoretical curves of $\Gamma(z)/R_{0}$ of
Eq.(LABEL:Eq:Gamma_Def) for $\Lambda CDM$ model and the blue line depicts this
parameter for a cosmological model with dark energy model of
$\omega(z)=-1+0.9z$. Both $\Gamma(z)/R_{0}$ are normalized to one.

Then the detection rate per redshift for two cosmological models ($\Lambda
CDM$ and $\omega CDM$ with $\omega(z)=-1+0.9z$) are calculated by
Eq.(LABEL:Eq:effi) that are shown in Fig..

[width=0.7]GRB_Histogram/efficiency.png

Figure :  Swift/BAT detection rate of long GRBs for $\Lambda CDM$ model and
blue line depicted this parameter for a cosmological model with dark energy
model of $\omega(z)=-1+0.9z$.

From EQ.(LABEL:Eq:Gamma_effi_final) we find normalized $\Gamma_{Obs}(z)$ as
$\int_{0}^{z_{max}}\Gamma_{Obs}(z)=1$ where $z_{max}$ is the maximum redshift
of observed GRBs is taken place at $z_{max}=9.4$. The normalized
$\Gamma_{Obs}(z)$ for two cosmological models are depicted in Fig.. The
cosmological redshift of 385 Swift LGRBs was reported in Swift
archive111https://swift.gsfc.nasa.gov/archive/grb_table/
https://swift.gsfc.nasa.gov/archive/grb_table/ (up to 23/04/2024). We showed
the histogram of the redshift of these LGRBs in Fig. that has a shape similar
to $\Gamma_{Obs}(z)$ in Fig.. We use these probability density functions for
generating the redshift of LGRBs in our Monte-Carlo simulation.

[width=0.7]GRB_Histogram/Gamma_Rz_effi_normalized.png

Figure :  The orange line shows the normalized $\Gamma_{Obs}(z)/R_{0}$ for
$\Lambda CDM$ model and the blue line depicts this parameter for a
cosmological model with dark energy model of $\omega(z)=-1+0.9z$

For the second step of Monte-Carlo simulation, we need a histogram of LGRBs
duration in their comoving frame. The cosmological redshift and $T_{90}$ of
385 Swift LGRBs were reported in this archive. If $\tau$ is taken as the
observed duration of burst ($\tau=T_{90}$), we can detect the duration of the
burst at the source frame ($\tau_{0}$), as $\tau_{0}=\tau/(1+z)$. We find the
histogram of $\tau_{0}$ of these 385 LGRBs that is shown in Fig.. Two
parameters of $\tau_{0}=\tau/(1+z)$ and redshift are independent (see Fig  and
Horvath2022,Lien3thCatalog). We examine this point from the normalized
correlation coefficient definition,

$c=\frac{\sum_{1}^{385}\tau_{0}(i)z(i)}{\sum_{1}^{385}(\tau_{0}(i)-\Bar{\tau_{0}})^{2}\sum_{1}^{385}(z(i)-\Bar{z})^{2}}$
(1)

we find $c=-0.09$ for redshift and the burst at the source frame, which means
$\tau_{0}$ and redshift are almost uncorrelated. Therefore, since
$\tau_{0}=\tau/(1+z)$ and redshift are uncorrelated, we used $\tau_{0}$
histogram of 385 LGRBs to generate more $\tau_{0}$ for mock LGRBs in the
Monte-Carlo simulation.

[width=0.7]GRB_Histogram/histogram_z_original.png

Figure :  The histogram of the redshift of 385 LGRBs detected by Swift (data
from the Swift archive.)

[width=0.7]GRB_Histogram/histogram_tau0_real.png

Figure :  The histogram of duration ($T_{90}$) of long GRBs in its comoving
frame that is derived from redshifts and duration of these 385 LGRBs detected
by Swift (data from the Swift archive).

[width=0.7]GRB_Histogram/BAT_Z_T_0.png

Figure :  $\tau_{0}$ is the duration of burst ($T_{90}/(1+z)$) in its rest
frame which is plotted versus burst redshift. (data from the Swift archive).)

Finally, we want to generate the histogram of observed LGRBs duration
($\tau=T_{90}$) for a cosmological model. From 1389 LGRBs in the Swift archive
(up to 23/04/2024), 1347 LGRBs have peak flux $\geq 10^{-7}erg/cm^{2}s$ and
their $T_{90}$ are determined but only redshifts of 385 LGRBs had measured.
Therefore, in a Monte Carlo simulation, we want to simulate
$\tau=\tau_{0}(1+z)$ for 1347 mock LGRBs and compare the histogram of $\tau$
for assumed cosmological model and observational histogram of $\tau$. Because
the duration of LGRBs in the Swift archive seems uncorrelated to their
redshift, We simulate $\tau=\tau_{0}(1+z)$ for 1347 mock LGRBs by using the
redshift distribution of Fig. and $\tau_{0}$ from Fig. distribution. The
histograms of $\tau$ for two cosmological models are depicted in Fig.. As we
expect, the $\tau$ histogram of LGRBs depends on the cosmological model. The
histogram of the $T_{90}$ of real data of Swift LGRBs archive is depicted in
Fig.. In the following section, we compare $\tau$ histogram of LGRBs to find
the best $\alpha$ for the cosmological model with a dark energy model of
$\omega(z)=-1+\alpha z$ based on $\chi^{2}$ test on LGRBs duration histogram.

[width=0.9]GRB_Histogram/histogram_tau_two_Model.png

Figure : Histogram of $\tau=\tau_{0}(1+z)$ regarding the efficiency in
detection of Swift/BAT for 1347 mock events and $\Lambda CDM$ model (blue) and
the cosmological model with dark energy model of $\omega(z)=-1+0.9z$ (red).

[width=0.7]GRB_Histogram/histogram_T_long_BAT.png

Figure :  The histogram of duration ($T_{90}$) of 1347 Swift LGRBs (data from
the Swift archive.).


#  MUTANT: A Multi-sentential Code-mixed Hinglish Dataset

Rahul Gupta
IIT Gandhinagar
Gandhinagar, Gujarat, India
<EMAIL_ADDRESS>
&Vivek Srivastava
TCS Research
Pune, Maharashtra, India
<EMAIL_ADDRESS>
&Mayank Singh
IIT Gandhinagar
Gandhinagar, Gujarat, India
<EMAIL_ADDRESS>

###### Abstract

The multi-sentential long sequence textual data unfolds several interesting
research directions pertaining to natural language processing and generation.
Though we observe several high-quality long-sequence datasets for English and
other monolingual languages, there is no significant effort in building such
resources for code-mixed languages such as Hinglish (code-mixing of Hindi-
English). In this paper, we propose a novel task of identifying multi-
sentential code-mixed text (MCT) from multilingual articles. As a use case, we
leverage multilingual articles from two different data sources and build a
first-of-its-kind multi-sentential code-mixed Hinglish dataset i.e., MUTANT.
We propose a token-level language-aware pipeline and extend the existing
metrics measuring the degree of code-mixing to a multi-sentential framework
and automatically identify MCT in the multilingual articles. The MUTANT
dataset comprises 67k articles with 85k identified Hinglish MCTs. To
facilitate future research, we make the dataset publicly available.

##  1 Introduction

Over the years, we have seen enormous downstream applications of multi-
sentential datasets in the areas such as question-answering Joshi et al.
(2017); Tapaswi et al. (2016), summarization Sharma et al. (2019); Cachola et
al. (2020), machine translation Bao et al. (2021), etc. The existing state-of-
the-art methods prove challenging to scale effectively and efficiently on
multi-sentential long sequence text Ainslie et al. (2020), which unplugs
several exciting research avenues. Unfortunately, to a large extent, the
majority of the research on multi-sentential data is dominated by a few
popular monolingual languages such as English, Chinese, and Spanish. Due to
this, code-mixed languages (among other low-resource and under-explored
languages) suffer from non-existent works in the aforementioned areas of
interest.

Figure 1: Example MCT and the corresponding article’s title form two
multilingual data sources: (A) Dainik Jagran news article and (B) Man-ki-baat
speech transcript. We color code the tokens as: English, Hindi, and language
independent.

Dataset | Task(s) | Data Source(s) | # Instances | Avg Tokens | Avg Sentences | Retrieval
---|---|---|---|---|---|---
Srivastava and Singh (2020) |  | Machine
---
Translation
| Social media posts
---
on Twitter & Facebook
13738 | 13 | 1.04 | Automatic
Khanuja et al. (2020) |  | Natural Language
---
Inference
| Hindi Bollywood
---
movie transcripts
2240 | 87 | 7.15 | Automatic
Mehnaz et al. (2021) |  | Dialogue
---
Summarization
| Manual translation of
---
dialogues and summaries
from Gliwa et al. (2019)
6830 | 31 | 7.85 | -
Srivastava and Singh (2021b) |  | Generation &
---
Evaluation
| IIT-B En-Hi parallel corpus
---
Kunchukuttan et al. (2018)
1974 | 20 | 1.05 | -
MUTANT | Summarization |  | Speech transcripts, press
---
releases, and news articles
84937 | 159 | 10.23 |  | Manual +
---
Automatic

Table 1: Comparison of the MUTANT dataset with the currently available
datasets in the Hinglish language.

We posit that due to several inherent challenges, the NLP community hold back
on building multi-sentential datasets for the low-resource and code-mixed
languages. One of the most significant bottlenecks in building such resources
is the unavailability of MCT on traditional and widely popular data sources
such as social media platforms where the short-length and noisy code-mixed
text is available in abundance. It presents several challenges such as the
difficulty in curating a large-scale multi-sentential dataset at ease. Another
major challenge is the lack of metrics to measure the degree of code-mixing in
the multi-sentential framework. The existing metrics such as code-mixing index
Das and Gambäck (2014) and multilingual-index Barnett et al. (2000) already
suffers from major limitations Srivastava and Singh (2021a) in the short-
length text format. In such a scenario, it gets mystifying to build a
retrieval pipeline to identify MCT and we need to depend heavily on the
expertise of human annotators which is a time and cost-demanding exercise. In
this work, we address both of these challenges. As a representative use case,
we base our work on Hinglish, a popular code-mixed language in the Indian
subcontinent. But the insights from our exploration could be extended to other
code-mixed language pairs.

To address the first challenge, we identify two non-traditional multilingual
data sources111these data sources have not been actively employed in building
datasets for the code-mixed languages i.e., political speeches and press
releases along with Hindi daily news articles (discussed in detail in Section
3). Figure 1 shows example Hinglish MCTs from two multilingual data sources.
To address the second challenge, we propose a token-level language-aware
pipeline and extend a widely popular metric (i.e., code-mixing index)
measuring the degree of code-mixing in a multi-sentential framework. We
demonstrate the effectiveness of the proposed pipeline with a minimal task-
specific annotation which significantly reduces the overall human effort
(discussed in detail in Section 4).

Eventually, we build a novel multi-sentential dataset for the Hinglish
language with 85k MCTs identified from 67k articles. In Table 1, we compare
MUTANT with four other Hinglish datasets Srivastava and Singh (2020); Khanuja
et al. (2020); Mehnaz et al. (2021); Srivastava and Singh (2021b) proposed for
a variety of tasks such as machine translation, natural language inference,
generation, and evaluation. The MUTANT dataset has a significantly higher
average number of sentences along with longer MCT (high average number of
tokens). Alongside, the dataset notably consists of a higher number of data
instances which is a rarity for the code-mixed datasets Srivastava and Singh
(2021a).

##  2 Multi-sentential Code-mixed Text Span (MCT)

Due to the absence of a formal definition of MCT in the literature, we propose
and use the following definition of MCT throughout this work:

MCT: Consider a multilingual article $A$ = {$s_{1}$, $s_{2}$, …, $s_{n}$}
consisting of $n$ sentences denoted by $s_{i}$ where $i\in$[1, $n$]. A unique
non-overlapping MCT $M_{p}$ in $A$ is a chunk of $m>1$ consecutive sentences
i.e. $M_{p}$ = {$s_{k}$, $s_{k+1}$, …, $s_{k+m-1}$}. $M_{p}$ should satisfy
the following two properties:

  1. 1.

$P1$: At least one $s_{k+j}$ in $M_{p}$ should be code-mixed. Trivially, at
most $m$-1 $s_{k+j}$ in $M_{p}$ could be monolingual. Here, $j\in$[0, $m$-1].

  2. 2.

$P2$: $s_{k}$ in $M_{p}$ is either the first sentence of the article or
preceded by a line break. Likewise, $s_{k+m-1}$ is either the last sentence of
the article or succeeded by a line break.

It should be noted that an article $A$ can have multiple non-overlapping
unique MCTs i.e. $A$ = {$M_{1}$, $M_{2}$, …, $M_{q}$} where $q\geq$0.

##  3 Multilingual and Multi-sentential Data Sources

Over the years, we observe several interesting and diverse code-mixed data
sources such as Twitter, Facebook, movie transcripts, etc. Social media sites
have acted as the cornerstone of the code-mixed data collection pipelines due
to the ease of availability of large-scale data. Nonetheless, they present
several challenges such as noisy data, short text, abusive, and multimodal
data. Given the requirements of MUTANT (i.e. multi-sentential and high-quality
data), we refrain from using social media sites in this work. Here, we focus
on two major data sources:

###  3.1 Political speeches and press releases

Here, we scrape data from five different web sources. Collectively, we denote
this data source as $D_{speech}$.

Aam Aadmi Party press releases (AAP): We scrape the press releases from the
official website of Aam Aadmi Party222https://aamaadmiparty.org/media/press-
releases. We have scraped 320 Hindi press releases from their website. The
website contains all the press releases in the last five years starting from
June 2017.

Indian National Congress speeches (INC): The official website of the INC
stores some of the speeches by major INC political leaders. We have extracted
112 of these speeches from their official
website333https://www.inc.in/media/speeches. The timeline for the scraped
speeches is between August 2018 to March 2022.

Man-ki-baat (MKB): Man-ki-baat is a radio program hosted by the Indian prime
minister Narendra Modi where he periodically addresses the people of the
nation. The MKB website444https://www.pmindia.gov.in/hi/mann-ki-baat/ stores
the official transcripts in Hindi and English languages. We have extracted the
transcripts of 67 of these programs between December 2015 to December 2021.

Press Information Bureau (PIB): The Press Information Bureau houses the
official press releases from all Indian government ministries including
President’s office, the Prime Minister’s office, Election Commission, etc. We
have extracted 30283 articles from the PIB website555https://www.pib.gov.in.
The timeline for these articles is from June 2017 to March 2022.

PM speech (PMS): Majority of the Indian Prime Minister speeches (different
from MKB speeches) are stored digitally on the PM India
website666https://www.pmindia.gov.in/hi/news-updates/. We have extracted 694
of these speeches that are recorded between November 2016 to October 2021.

###  3.2 Hindi news articles

Here, we scrape data from two major Hindi news daily websites. Collectively,
we denote this data source as $D_{news}$.

Dainik Bhaskar (DB): Dainik Bhaskar is one of the most popular Hindi
newspapers in India. It is ranked 4th in the world by circulation according to
World Press Trends
2016777https://web.archive.org/web/20170706110804/http://www.wptdatabase.org/world-
press-trends-2016-facts-and-figures. They have digitized the daily newspapers
on their website888https://www.bhaskar.com. Articles on DB website have been
divided into many categories such as ‘Entertainment’ and ‘Sports’. We have
extracted 115324 articles uploaded on the website between February 2019 to May
2022. In Table 2, we present the category-wise distribution of the articles
scraped from the DB website.

Category | DB | DJ
---|---|---
Business | 16012 | 4203
Entertainment | 18498 | 52173
Featured | 5536 | 19373
Lifestyle | 12189 | -
Miscellaneous | 20221 | -
National | 18615 | 160005
Politics | - | 33604
Sports | 9950 | -
World | 14303 | 42478
Total | 115324 | 311836
Table 2: Number of articles in various news categories in the DB and DJ
datasets.

Dainik Jagran (DJ): Dainik Jagran is another popular Indian Hindi newspaper.
According to World Press Trends 2016, DJ is ranked 5th in the world by
circulation. Similar to the DB website, they have also created a repository of
articles on their official website999https://www.jagran.com. Here, we extract
311836 of these articles from the website that were uploaded between April
2013 to May 2022. In Table 2, we present the category-wise distribution of the
articles scraped from the DJ website.

##  4 Experimental Setup

_Problem definition_ : Given a multilingual article $A$ comprising of $q$
multi-sentential text spans (MST) i.e. $A$ = {$M_{1}$, $M_{2}$, …, $M_{q}$},
we predict a binary outcome $L_{CM}$ for each MST $M_{i}$ i.e. $L(A)$ =
{$L_{CM}^{M_{1}}$, $L_{CM}^{M_{2}}$, …, $L_{CM}^{M_{q}}$,}. $L_{CM}^{M_{i}}$ =
1, if $M_{i}$ is code-mixed, otherwise 0. In a nutshell, a code-mixed MST
$M_{i}$ is a MCT and it satisfies the properties $P1$ and $P2$ (ref. section
2).

Figure 2 shows the architecture of the MCT identification pipeline. Next, we
discuss the various components of this pipeline in detail.

Figure 2: Architecture of MCT identification pipeline.

###  4.1 Token-level language annotation (TLA)

We exploit the token-level language information to identify MCT given a
multilingual article $A$. We annotate the words in $A$ using a code-mixed
language identification tool. Specifically, we use L3Cube-HingLID Nayak and
Joshi (2022) for this task. A word $w_{i}$ $\in$ $A$ can take either of the
three language tags from the set {$English$, $Hindi$, $Other$}. Given that
L3Cube-HingLID works only on the Roman script text, we use a Devanagari to
Roman script transliteration
tool101010https://github.com/ritwikmishra/devanagari-to-roman-script-
transliteration for the tokens written in Devanagari script. In Table 3, we
report the percentage of $Hindi$ and $English$ tokens. With an exception of
the AAP dataset, $Hindi$ is the predominant language in all the data sources.

| Articles | AW | AC | %H | %E
---|---|---|---|---|---
AAP | 320 | 1129 | 6033 | 53.97 | 45.09
INC | 112 | 2312 | 10691 | 63.83 | 33.12
MKB | 67 | 4151 | 20706 | 77.17 | 22.41
PIB | 30283 | 525 | 3015 | 80.96 | 17.59
PMS | 694 | 2591 | 13400 | 79.02 | 20.45
DB | 115324 | 382 | 1977 | 80.22 | 18.25
DJ | 311836 | 391 | 2037 | 79.28 | 19.60
$D_{speech}$ | 31476 | 590 | 3339 | 79.97 | 18.65
$D_{news}$ | 427160 | 388 | 2020 | 80.18 | 18.51
| $D_{speech}$
---
\+ $D_{news}$
458636 | 401 | 589 | 80.05 | 18.54

Table 3: Distribution of the scraped articles from various data sources. AW:
average number of words. AC: average number of characters. %E: percentage of
English tokens. %H: percentage of Hindi tokens.

###  4.2 Code-Mixing Index (CMI)

In the literature, we observe several metrics that has been proposed to
measure the degree of code-mixing in text such as code-mixing index (CMI, Das
and Gambäck (2014)), multilingual-index (M-index, Barnett et al. (2000)) and
integration-index (I-index, Guzmán et al. (2017)). Each of these metrics has
its own merits and limitations Srivastava and Singh (2021a). In this work, we
use the most widely used CMI metric due to the ease of interpretation and the
suitability for the task. CMI, by definition, measures the degree of code-
mixing in a text as:

$CMI=\begin{cases}100*[1-\frac{max(w_{i})}{n-u}]&n>u\\\ 0&n=u\end{cases}$ (1)

Here, $w_{i}$ is the number of words of the language $i$, max{$w_{i}$}
represents the number of words of the most prominent language, $n$ is the
total number of tokens, $u$ represents the number of language-independent
tokens (such as named entities, abbreviations, mentions, and hashtags). The
CMI score ranges from 0 to 100. A low CMI score suggests the prevalence of
only one language in the text whereas a high CMI score indicates a high degree
of code-mixing.

###  4.3 Small annotated dataset (SAnD)

We create a small manually annotated dataset comprising all seven data
sources. The objective of the annotation is to assign a binary label to each
MST such that we can identify if the MST is code-mixed or not from the
assigned label.

More formally, $SAnD$ = {$A_{1}$: $l_{1}$, $A_{2}$: $l_{2}$, …, $A_{u}$:
$l_{u}$}, represents $u$ manually annotated MST111111For distinctive
representation, we denote MST in $SAnD$ with $A$ instead of $M$. where
$l_{i}\in${0,1} $\forall$ $i\in$[1,$u$]. Here, $l_{i}$=1, if $A_{i}$ is code-
mixed, otherwise 0.

For this annotation task, we have selected a small number of articles (60 each
from $D_{speech}$ and $D_{news}$) randomly from the scraped articles. We leave
it to the judgment of the annotator to decide if a sentence (and subsequently
the MST) is code-mixed or not. The annotator has expert-level proficiency in
Hindi, English, and Hinglish languages. In Table 4, we show the distribution
of the annotated articles for each data source. In total, we annotate 120
articles and 568 MST where we identify 121 MST (21.3%) as code-mixed.

| Articles | MST
---|---|---
Total | Hing | E/H
AAP | 5 | 6 | 2 | 4
INC | 3 | 69 | 5 | 64
MKB | 3 | 66 | 25 | 41
PIB | 47 | 62 | 27 | 35
PMS | 2 | 36 | 13 | 23
DB | 30 | 207 | 48 | 159
DJ | 30 | 122 | 28 | 94
$D_{speech}$ | 60 | 239 | 72 | 167
$D_{news}$ | 60 | 329 | 76 | 253
| $D_{speech}$
---
\+ $D_{news}$
120 | 568 | 148 | 420
Table 4: $SAnD$ dataset statistics. Hing: Hinglish, E/H: English/Hindi.

###  4.4 Estimating multilinguality

Though CMI is widely used in numerous previous works, we couldn’t find any
discussion on the ideal CMI score thresholding criteria to identify a good
code-mixed text. The problem becomes even more challenging when we use the CMI
metric in a multi-sentential framework along with constraints $P1$ and $P2$
(ref section 2). Various works Khanuja et al. (2020) have used empirically
identified CMI thresholds to measure the degree of code-mixing in the text.
But, we couldn’t find any experimental justification for their findings.

Dual MEC score: Here, we propose a novel adoption of the CMI metric in a
constrained multi-sentential framework. For MST $M_{p}$ with $k$ sentences, we
compute the scores for dual multilinguality estimation criteria (MEC) as:

  1. 1.

Sentence-level CMI ($CMI$): We compute $CMI(s_{i})$ for the sentence
$s_{i}$$\in$ $M_{p}$ using the language-information of all the words in
$s_{i}$ and the formulation given in 1.

  2. 2.

Multilinguality ratio ($MR$): We compute $C_{MR}$ for the MST $M_{p}$ as:

$MR(M_{p})=\frac{N_{cm}}{k}$ (2)

Here, $N_{cm}$ and $k$ are the number of code-mixed and total sentences in
$M_{p}$ respectively.

Figure 3 shows the mean and standard deviation of dual MEC scores on seven
different data sources.

Figure 3: The mean and standard deviation of the dual MEC score for different
data sources. The CMI score is scaled between 0 to 1.

Formulation: We identify if the sentence $s_{i}$ is code-mixed or monolingual
using $CMI(s_{i})$ score as:

$f_{cm}(s_{i})=\begin{cases}1,&CMI(s_{i})>\alpha\\\ 0,&otherwise\end{cases}$
(3)

Here, $\alpha\in$ [0, 100] is the sentence-level CMI score threshold and
$f_{cm}(.)$ estimates the code-mixing status ($1$ being code-mixed and $0$
being monolingual) of the sentence under consideration. Using 3, we compute
$N_{cm}$ as:

$N_{cm}=\Sigma_{i=1}^{k}f_{cm}(s_{i})$ (4)

Using 2 and 4, we compute $MR(M_{p})$ as:

$MR(M_{p})=\frac{\Sigma_{i=1}^{k}f_{cm}(s_{i})}{k}$ (5)

We formulate the following function to identify if MST $M_{p}$ with $k$
sentences is code-mixed:

$g_{cm}(M_{p})=\begin{cases}1,&MR(M_{p})>\beta\\\ 0,&otherwise\end{cases}$ (6)

Here, $\beta\in$[0, 1] is the multilinguality ratio threshold and $g_{cm}(.)$
estimates the code-mixing status ($1$ being code-mixed and $0$ being
monolingual) of the MST under consideration.

###  4.5 Dual MEC threshold computation

The dual MEC formulation helps us to identify the MCT in a constrained setting
by jointly modeling the sentence-level and MST-level multilinguality
information. As discussed in Section 4.4, the ideal thresholds $\alpha$ and
$\beta$ are a conundrum that needs further exploration. Here, we propose to
use the $SAnD$ dataset to identify the dual MEC thresholds ($\alpha$ and
$\beta$). Algorithm 1 shows the procedure to compute the thresholds. The
algorithm takes $SAnD$ dataset $D$ with $u$ labeled MST. We represent the
parameter search space for $\alpha$ and $\beta$ with $\alpha_{cand}$ and
$\beta_{cand}$ respectively. $\alpha_{cand}$ ranges from $\alpha_{low}$ to
$\alpha_{high}$ with a step-size of $\alpha_{step}$ whereas $\beta_{cand}$
ranges from $\beta_{low}$ to $\beta_{high}$ with a step-size of
$\beta_{step}$. Based on our empirical observation, we set ($\alpha_{low}$,
$\alpha_{high}$, $\alpha_{step}$) with (0, 50, 1) and ($\beta_{low}$,
$\beta_{high}$, $\beta_{step}$) with (0, 0.5, 0.025).

We perform the grid search on each threshold combination of ($\alpha_{i}$,
$\beta_{j}$) to identify the best combination. For each threshold combination,
we identify the accuracy of identifying the MCT in $D$ leveraging $f_{cm}(.)$
and $g_{cm}(.)$ formulations. We select the threshold combination with the
highest accuracy as the final threshold ($\alpha$ and $\beta$). Table 5 shows
the best-identified thresholds on various data sources of the $SAnD$ dataset.
Figure 4 shows the mean and standard deviation of the accuracy on various dual
MEC threshold combinations for different data sources.

Algorithm 1 $compute_{\alpha,\beta}(D)$

1:$D$ = {$A_{1}$: $l_{1}$, $A_{2}$: $l_{2}$, …, $A_{u}$: $l_{u}$} where
$A_{i}$ = {$s_{1}$, $s_{2}$, …, $s_{k}$}

2:$\alpha_{cand}$ = [$\alpha_{low}$, $\alpha_{low}$+$\alpha_{step}$, …,
$\alpha_{high}$]

3:$\beta_{cand}$ = [$\beta_{low}$, $\beta_{low}$+$\beta_{step}$, …,
$\beta_{high}$]

4:$Accuracy$ = {}

5:for $\alpha_{i}$ in $\alpha_{cand}$ do

6: for $\beta_{j}$ in $\beta_{cand}$ do

7: $hits$ = 0

8: for $A_{p}$ $\in$ $D$ do

9: $F_{cm}$ = $f_{cm}(s_{q})$ $\forall$ $s_{q}$ $\in$ $A_{p}$

10: Compute $g_{cm}(A_{p})$ using $F_{cm}$

11: if $g_{cm}(A_{p})$ == $l_{p}$ then

12: $hits$ = $hits$ \+ 1

13: end if

14: end for

15: $Accuracy[(\alpha_{i},\beta_{j})]=100*(hits/u)$

16: end for

17:end for

18:$\alpha$ = $max_{value}(Accuracy).key()$[0]

19:$\beta$ = $max_{value}(Accuracy).key()$[1]

20:return $\alpha,\beta$

| $\alpha$ | $\beta$ | Accuracy(%)
---|---|---|---
AAP | 25 | 0.35 | 100
INC | 28 | 0.30 | 89
MKB | 22 | 0.35 | 64
PIB | 26 | 0.15 | 68
PMS | 21 | 0.45 | 89
DB | 18 | 0.40 | 72
DJ | 28 | 0.40 | 79
$D_{speech}$ | 24 | 0.35 | 72
$D_{news}$ | 29 | 0.475 | 78
| $D_{speech}$
---
\+ $D_{news}$
29 | 0.45 | 75
Table 5: Best identified thresholds ($\alpha$ and $\beta$) along with the
accuracy of identifying MCT on various data sources in the $SAnD$ dataset.
Figure 4: The mean and standard deviation of the accuracy on various dual MEC
threshold combinations. The red dot corresponding to each data source
indicates the accuracy against the best-identified thresholds.

###  4.6 Dual MEC threshold generalization

As evident from Table 5, the thresholds $\alpha$ and $\beta$ vary across the
data sources. So, it is important to identify which of these identified
thresholds will result in a robust and stable performance across datasets.
Here, we experiment with five dual MEC threshold generalisation techniques:

  1. 1.

Local Average (LA): For the data source $D_{i}$, we take the mean sentence-
level CMI score and mean MR score as the dual MEC thresholds.

  2. 2.

Global Average (GA): For the data source $D_{i}$, we take the mean sentence-
level CMI score and mean MR score of the corresponding category data-source
($D_{speech}$ or $D_{news}$) as the dual MEC thresholds.

  3. 3.

Average of LA and GA (ALG): For the data source $D_{i}$, we take the average
of LA and GA identified thresholds as the dual MEC thresholds.

  4. 4.

Single data source generalization (SDG): In this approach, we generalize the
dual MEC thresholds identified locally on a single data source $D_{i}$ (using
Algorithm 1) to identify MCT globally on other data sources.

  5. 5.

Multi data source generalization (MDG): In this approach, we use the dual MEC
threshold information from multiple sources and use the majority voting to
identify the best thresholds. For the data source $D_{i}$, we use the
thresholds identified on three data sources (using Algorithm 1), namely
$D_{i}$, $D_{speech}$ (if $D_{i}$ $\in$ $D_{speech}$, else $D_{news}$), and
$D_{speech}+D_{news}$. We then make an independent prediction on each of the
three thresholds and take majority voting for the final classification of
$M_{p}$.

##  5 MUTANT: A Multi-sentential Code-mixed Hinglish Dataset

We evaluate the performance of MCT identification pipeline and the five dual
MEC threshold generalization techniques using the three subsets of the $SAnD$
dataset: $D_{speech}$, $D_{news}$, and $D_{speech}+D_{news}$. We report the
following metric scores on each of the seven data sources:

  1. 1.

Accuracy: We compute accuracy as the ratio of the total correct prediction of
MCT and non-MCT to the total number of MST. We multiply this ratio by 100 and
report the accuracy percentage. A high accuracy % is preferred.

  2. 2.

False MCT Rate (FMR): We define FMR as the ratio of incorrectly identified MCT
to the total number of actual monolingual MST. We report the FMR% and a low
FMR% is preferred.

  3. 3.

Diversity@10 (D@10): We define D@10 as the percentage of articles in data
source $D_{i}$ having more than $10\%$ correctly identified MCT. A high D@10
score is preferred.

| Accuracy | FMR | D@10
---|---|---|---
| L | G | A | S | M | L | G | A | S | M | L | G | A | S | M
AAP | 62 | 66 | 64 | 72 | 74 | 15 | 21 | 20 | 17 | 17 | 49 | 46 | 51 | 60 | 62
INC | 63 | 66 | 64 | 73 | 74 | 17 | 21 | 20 | 16 | 12 | 49 | 46 | 51 | 59 | 59
MKB | 61 | 66 | 62 | 69 | 72 | 28 | 21 | 26 | 22 | 18 | 51 | 46 | 48 | 68 | 70
PIB | 62 | 66 | 64 | 67 | 72 | 24 | 21 | 24 | 30 | 17 | 53 | 46 | 55 | 73 | 74
PMS | 67 | 66 | 64 | 71 | 74 | 17 | 21 | 23 | 20 | 16 | 51 | 46 | 53 | 67 | 69
DB | 66 | 63 | 62 | 67 | 78 | 29 | 26 | 28 | 30 | 5 | 57 | 56 | 57 | 78 | 78
DJ | 62 | 63 | 64 | 75 | 78 | 26 | 26 | 26 | 6 | 5 | 48 | 56 | 49 | 73 | 74

Table 6: Results on $D_{speech}$ dataset. L: LA, G: GA, A: ALG, S: SDG, M:
MDG.

We report the results in Tables 6, 7, 8. The mean-based threshold
generalization techniques (LA, GA, and ALG) consistently show poor performance
on all the metrics. Given the nature of the problem, we prefer a low rate of
misidentification of monolingual MST as the MCT and at the same time a high
number of actual MCT should also be identified. MDG threshold generalization
technique satisfies both conditions with low FMR and high accuracy on all the
datasets. D@10 depicts if the threshold generalization technique is influenced
by the presence of a few outliers in the dataset. SDG and MDG both show
competitive results on the D@10 metric outperforming the mean-based threshold
generalization techniques by a large margin. The constant poor performance of
mean-based threshold generalization against SDG and MDG also shows the
efficacy of the proposed threshold computation strategy (Algorithm 1).

Finally, to build the MUTANT dataset, we use the MCT identification pipeline
with the MDG threshold generalization technique. Table 9 shows the statistics
of the MUTANT dataset. To facilitate future work on this novel task of MCT
identification, we will release the MUTANT dataset along with the initially
scraped data from all the data sources and the annotated $SAnD$ dataset. The
MUTANT dataset can be used for various tasks including but not limited to
question-answering, text summarization and machine translation for Hinglish
texts. This dataset could be used as a pre-training dataset to train efficient
NLU models for various tasks on Hinglish data.

| Accuracy | FMR | D@10
---|---|---|---
| L | G | A | S | M | L | G | A | S | M | L | G | A | S | M
AAP | 72 | 70 | 71 | 72 | 73 | 17 | 15 | 17 | 14 | 14 | 60 | 58 | 62 | 70 | 72
INC | 69 | 70 | 71 | 73 | 73 | 14 | 15 | 15 | 9 | 7 | 58 | 58 | 58 | 65 | 66
MKB | 66 | 70 | 68 | 70 | 72 | 25 | 15 | 21 | 21 | 15 | 73 | 58 | 71 | 79 | 80
PIB | 68 | 70 | 68 | 70 | 73 | 23 | 15 | 22 | 29 | 14 | 73 | 58 | 71 | 79 | 80
PMS | 61 | 70 | 69 | 74 | 73 | 14 | 15 | 18 | 14 | 12 | 63 | 58 | 63 | 71 | 69
DB | 66 | 69 | 67 | 68 | 71 | 28 | 22 | 26 | 29 | 3 | 76 | 72 | 74 | 84 | 85
DJ | 68 | 69 | 68 | 72 | 71 | 22 | 22 | 22 | 4 | 3 | 70 | 72 | 68 | 77 | 73

Table 7: Results on $D_{news}$ dataset. L: LA, G: GA, A: ALG, S: SDG, M: MDG.

| Accuracy | FMR | D@10
---|---|---|---
| L | G | A | S | M | L | G | A | S | M | L | G | A | S | M
AAP | 69 | 70 | 69 | 73 | 74 | 12 | 15 | 15 | 13 | 13 | 55 | 60 | 57 | 65 | 66
INC | 70 | 70 | 69 | 73 | 74 | 11 | 15 | 14 | 10 | 8 | 57 | 60 | 56 | 62 | 63
MKB | 67 | 70 | 69 | 70 | 72 | 21 | 15 | 19 | 17 | 14 | 62 | 60 | 65 | 68 | 65
PIB | 69 | 70 | 69 | 67 | 73 | 18 | 15 | 18 | 23 | 14 | 63 | 60 | 64 | 75 | 74
PMS | 62 | 70 | 70 | 72 | 74 | 13 | 15 | 17 | 16 | 12 | 57 | 60 | 59 | 65 | 69
DB | 67 | 68 | 67 | 67 | 75 | 23 | 19 | 22 | 24 | 4 | 64 | 62 | 62 | 76 | 75
DJ | 68 | 68 | 69 | 74 | 75 | 19 | 19 | 19 | 5 | 4 | 57 | 62 | 62 | 71 | 74

Table 8: Results on $D_{speech}$+$D_{news}$ dataset. L: LA, G: GA, A: ALG, S:
SDG, M: MDG.

##  6 Analysis and Discussion

| A | M |  M/A | Avg CMI | Avg Words | Avg Characters
---|---|---|---|---|---|---
A | M | H | A | M | H | A | M | H
AAP | 30 | 32 | 1.07 | 33.0 | 35.2 | 21.1 | 1347 | 1263 | 16 | 6993 | 6556 | 63
INC | 85 | 306 | 3.6 | 28.1 | 27.5 | - | 751 | 208 | - | 3368 | 935 | -
MKB | 58 | 243 | 4.19 | 20.1 | 22.4 | - | 1034 | 246 | - | 4843 | 1156 | -
PIB | 8473 | 8786 | 1.04 | 23.0 | 23.2 | 21.0 | 572 | 552 | 15 | 3139 | 3028 | 87
PMS | 597 | 3909 | 6.55 | 25.8 | 24.7 | 26.4 | 952 | 145 | 13 | 4585 | 700 | 79
DB | 12851 | 15433 | 1.20 | 21.0 | 21.2 | 20.2 | 107 | 89 | 24 | 528 | 440 | 123
DJ | 44913 | 56228 | 1.25 | 22.2 | 22.3 | 21.6 | 146 | 117 | 16 | 734 | 586 | 82
$D_{speech}$ | 9243 | 13276 | 1.44 | 23.2 | 23.8 | 21.3 | 604 | 420 | 15 | 3258 | 2268 | 87
$D_{news}$ | 57764 | 71661 | 1.24 | 21.9 | 22.0 | 21.2 | 137 | 111 | 18 | 688 | 555 | 91
$D_{speech}$ \+ $D_{news}$ | 67007 | 84937 | 1.27 | 22.0 | 22.3 | 21.2 | 201 | 159 | 17 | 1043 | 822 | 90
Table 9: MUTANT dataset statistics. A: Articles, M: MCT, and H: Headings. The
INC and MKB datasets contain generic and very-low informative headlines and we
do not include them in the final dataset.

| A | MST | CA | CKS | Acc | FMR | D@10
---|---|---|---|---|---|---|---
Hing | E/H
AAP | 5 | 5 | 2 | 3 | 1.0 | 100 | 0 | 100
INC | 5 | 82 | 10 | 67 | 0.76 | 88 | 10 | 80
MKB | 5 | 119 | 23 | 80 | 0.67 | 75 | 25 | 80
PIB | 5 | 5 | 2 | 3 | 1.0 | 80 | 0 | 50
PMS | 5 | 141 | 13 | 110 | 0.52 | 84 | 12 | 100
DB | 5 | 49 | 3 | 43 | 0.63 | 78 | 20 | 50
DJ | 5 | 18 | 2 | 15 | 0.77 | 88 | 13 | 100
$D_{speech}$ | 25 | 352 | 50 | 263 | 0.65 | 82 | 14 | 71
$D_{news}$ | 10 | 67 | 5 | 58 | 0.69 | 80 | 18 | 75
| $D_{speech}$
---
\+ $D_{news}$
35 | 419 | 55 | 321 | 0.65 | 82 | 15 | 74

Table 10: Qualitative evaluation of the MUTANT dataset. A: Articles, CA:
complete agreement between the annotators, Hing: Hinglish MST. E/H:
English/Hindi MST, CKS: Cohen’s kappa score.

In this section, we qualitatively evaluate the MUTANT dataset by employing two
human evaluators, different from the one used for the $SAnD$ to avoid any
biases in the evaluation. Both evaluators are proficient in English, Hindi,
and Hinglish languages. We randomly sample five articles from each of the
seven source datasets and share the originally scraped articles containing
both identified MCT and monolingual MST with both evaluators. During the
evaluation, we do not disclose which of the MSTs is identified as MCT and
share the following guidelines:

  1. 1.

Any MST containing only Hindi words or only English words is monolingual.

  2. 2.

Any named entity, date, number, or word common in both English and Hindi
languages should be considered a language-independent word.

In Table 10, we report our findings from the qualitative evaluation study. Out
of a total of 419 MST, we observe the complete agreement on 321 monolingual
MST and 55 code-mixed MST resulting in $\approx$ 90% complete agreement. A
complete agreement means that both annotators agree that any particular MST is
code-mixed or not. On MST with CA, we further compute the three metric scores
using MDG. The results strengthen our earlier findings from Section 5. In
Figure 5, we report two example MCT incorrectly identified by our MCT
identification pipeline. In the first example, both evaluators show complete
agreement whereas in the second example there is a disagreement between the
evaluators. We attribute this behavior to the poor state of the current code-
mixed LID systems Srivastava and Singh (2021a) and since the CMI metric and
our dual MEC formulation depend heavily on the code-mixed LID tools, the final
results get affected. This limitation further provides an opportunity for
future works to explore the problem from different perspectives such as a
token-level language-independent MCT identification pipeline. It will also be
interesting to see how this pipeline performs with other code-mixed languages,
especially in a low-resource setting.

Figure 5: False positive MCT. We color code the tokens as: Hindi, English, and
language independent.

##  7 Conclusion

In this paper, we present a novel task of identifying MCT from multilingual
documents. We propose an MCT identification pipeline by extending CMI to the
multi-sentential framework and leveraging the pipeline we build a dataset for
the Hinglish language. We highlight several challenges in building such
resources and our insights will be useful to future works in code-mixed and
low-resource languages.

##  8 Limitations

The limitations with the MUTANT dataset include but are not limited to:

  * •

Contrary to the previous works, all the data sources comprises the non social
media sites. This could potentially limit the diversity in the code-mixed text
as observed on social media platforms.

  * •

In the current form, the dataset is limited to only one code-mixed language.
We believe the proposed technique to extract MCT could be expanded to other
code-mixed languages in the future.

  * •

The data sources could potentially have their own biases (topical, style of
writing, etc). We expect future works to be cautious while generalizing the
results obtained on this dataset.

## References

  * Ainslie et al. (2020) Joshua Ainslie, Santiago Ontanon, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang, and Li Yang. 2020.  Etc: Encoding long and structured inputs in transformers.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 268–284.
  * Bao et al. (2021) Guangsheng Bao, Yue Zhang, Zhiyang Teng, Boxing Chen, and Weihua Luo. 2021.  G-transformer for document-level machine translation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 3442–3455.
  * Barnett et al. (2000) Ruthanna Barnett, Eva Codó, Eva Eppler, Montse Forcadell, Penelope Gardner-Chloros, Roeland Van Hout, Melissa Moyer, Maria Carme Torras, Maria Teresa Turell, Mark Sebba, et al. 2000.  The lides coding manual: A document for preparing and analyzing language interaction data version 1.1–july 1999.  _International Journal of Bilingualism_ , 4(2):131–271.
  * Cachola et al. (2020) Isabel Cachola, Kyle Lo, Arman Cohan, and Daniel S Weld. 2020.  Tldr: Extreme summarization of scientific documents.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 4766–4777.
  * Das and Gambäck (2014) Amitava Das and Björn Gambäck. 2014.  Identifying languages at the word level in code-mixed indian social media text.  In _Proceedings of the 11th International Conference on Natural Language Processing_ , pages 378–387.
  * Gliwa et al. (2019) Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019.  Samsum corpus: A human-annotated dialogue dataset for abstractive summarization.  In _Proceedings of the 2nd Workshop on New Frontiers in Summarization_ , pages 70–79.
  * Guzmán et al. (2017) Gualberto Guzmán, Joseph Ricard, Jacqueline Serigos, Barbara E Bullock, and Almeida Jacqueline Toribio. 2017.  Metrics for modeling code-switching across corpora.  _Proc. Interspeech 2017_ , pages 67–71.
  * Joshi et al. (2017) Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017.  Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1601–1611.
  * Khanuja et al. (2020) Simran Khanuja, Sandipan Dandapat, Sunayana Sitaram, and Monojit Choudhury. 2020\.  A new dataset for natural language inference from code-mixed conversations.  In _Proceedings of the The 4th Workshop on Computational Approaches to Code Switching_ , pages 9–16.
  * Kunchukuttan et al. (2018) Anoop Kunchukuttan, Pratik Mehta, and Pushpak Bhattacharyya. 2018.  The iit bombay english-hindi parallel corpus.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_.
  * Mehnaz et al. (2021) Laiba Mehnaz, Debanjan Mahata, Rakesh Gosangi, Uma Sushmitha Gunturi, Riya Jain, Gauri Gupta, Amardeep Kumar, Isabelle G Lee, Anish Acharya, and Rajiv Shah. 2021.  Gupshup: Summarizing open-domain code-switched conversations.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6177–6192.
  * Nayak and Joshi (2022) Ravindra Nayak and Raviraj Joshi. 2022.  L3cube-hingcorpus and hingbert: A code mixed hindi-english dataset and bert language models.  _arXiv preprint arXiv:2204.08398_.
  * Sharma et al. (2019) Eva Sharma, Chen Li, and Lu Wang. 2019.  Bigpatent: A large-scale dataset for abstractive and coherent summarization.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2204–2213.
  * Srivastava and Singh (2020) Vivek Srivastava and Mayank Singh. 2020.  Phinc: A parallel hinglish social media code-mixed corpus for machine translation.  In _Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020)_ , pages 41–49.
  * Srivastava and Singh (2021a) Vivek Srivastava and Mayank Singh. 2021a.  Challenges and limitations with the metrics measuring the complexity of code-mixed text.  In _Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching_ , pages 6–14.
  * Srivastava and Singh (2021b) Vivek Srivastava and Mayank Singh. 2021b.  Hinge: A dataset for generation and evaluation of code-mixed hinglish text.  In _Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems_ , pages 200–208.
  * Tapaswi et al. (2016) Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. 2016.  Movieqa: Understanding stories in movies through question-answering.  In _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 4631–4640. IEEE.


# Guided Decoding for Robot Motion Generation and Adaption

Nutan Chen1∗, Elie Aljalbout1, Botond Cseke1∗, Patrick van der Smagt1,2
1Machine Learning Research Lab, Volkswagen Group, Geramany <first name>.<last
<EMAIL_ADDRESS>of Informatics, Eötvös Loránd University,
Budapest, Hungary∗ These authors contributed equally to this work.

###### Abstract

We address motion generation for high-DoF robot arms in complex settings with
obstacles, via points, etc. A significant advancement in this domain is
achieved by integrating Learning from Demonstration (LfD) into the motion
generation process. This integration facilitates rapid adaptation to new tasks
and optimizes the utilization of accumulated expertise by allowing robots to
learn and generalize from demonstrated trajectories.

We train a transformer architecture on a large dataset of simulated
trajectories. This architecture, based on a conditional variational
autoencoder transformer, learns essential motion generation skills and adapts
these to meet auxiliary tasks and constraints. Our auto-regressive approach
enables real-time integration of feedback from the physical system, enhancing
the adaptability and efficiency of motion generation. We show that our model
can generate motion from initial and target points, but also that it can adapt
trajectories in navigating complex tasks, including obstacle avoidance, via
points, and meeting velocity and acceleration constraints, across platforms.

##  I Introduction

Motion generation is a challenging topic for robot arms with high degrees of
freedom (DoF). One main challenge is to generate smooth motion trajectories to
solve a given task while ensuring the robot and its environment’s safety, and
respecting the robot’s embodiment constraints. This problem is especially
complex when the task involves auxiliary objectives beyond reaching a target
point, such as avoiding obstacles, passing via specified points, or imposing
velocity or acceleration constraints. Another particularly intriguing aspect
of motion generation is the incorporation of learning from demonstration (LfD)
within the same process. Allowing users or agents to present a guiding
trajectory based on prior knowledge helps the rapid development of robotic
applications. This capability is crucial for adapting to novel tasks and
leveraging accumulated expertise effectively. Motion primitives offer a
promising avenue for achieving this goal [1, 2, 3, 4]. By encapsulating
fundamental motion behaviors into primitives, the robot can learn and
generalize from demonstrations, promoting versatility and adaptability in
complex tasks. Integrating learning from demonstration with the challenges of
motion generation presents an exciting direction for advancing the
capabilities of robot arms with high degrees of freedom.

In this work, we propose a framework for learning motion generation. We create
a large dataset of robot motion trajectories with multiple simulated robots
and train a transformer architecture to learn to generate motion based on
given initial and target points. We design a Transformer conditional
variational autoencoder. The trained generative model encapsulates basic
motion generation skills based on the training data. We further propose an
approach to efficiently adapt the generated motion to satisfy auxiliary tasks
and constraints such as obstacle avoidance, via points and velocity
constraints. The whole process is auto-regressive allowing for integrating
feedback from the real system during motion generation. Our approach allows
for generating motion based on an initial and target point only, or to imitate
a full trajectory given by a user or external agent. Our experiments show the
usability of our method for motion generation, under via point constraints,
obstacles, and velocity and acceleration constraints. We also show the
benefits of training with data collected using various robots. Additionally,
we show the ability to imitate and adapt user-given trajectories to a given
task.

##  II Methods

In this section, we present a Transformer-based autoencoder model in Sec. II-A
to II-C, followed by an introduction to adaptation methods based on an
autoregressive decoder in Section II-D and II-E (see Fig. 1).

###  II-A Inference model for the infilling task

We consider a dataset $\mathcal{D}=\\{x^{(i)}\\}^{N}_{i=1}$ of trajectories
$x^{(i)}=\\{x^{(i)}_{t}\\}_{t=1}^{T}$ and our goal is to learn representations
that can help us generate new trajectories given a starting point $x_{1}$ and
an endpoint $x_{T}$. We propose to do this with the Conditional VAE [5] model

$\displaystyle p_{\theta}(x_{2:T-1}|x_{1,T})=\int
p_{\theta}(x_{2:T-1}|x_{1,T},z)\>p_{\theta}(z|x_{1,T})\,dz$

where the latent variables $z$ are expected to encode latent features of paths
with starting point $x_{1}$ and endpoint $x_{T}$. We learn this by minimizing
the negative ELBO

$\displaystyle L(\theta,\phi)=$
$\displaystyle-\mathrm{E}_{\hat{p}(x_{1:T})}\mathrm{E}_{q_{\phi}(z;x_{1:T})}[\log
p_{\theta}(x_{2:T-1}|x_{1,T},z)]$
$\displaystyle+\mathrm{E}_{\hat{p}(x_{1:T})}\mathrm{KL}[q_{\phi}(z;x_{1:T})\,\|\,p_{\theta}(z;x_{1,T})].$

Here $p_{\theta}(z|x_{1,T})$ is the conditional prior model,
$p_{\theta}(x_{2:T-2}|x_{1,T},z)$ is the decoder/generation model, and
$q_{\phi}(z|\operatorname{x}_{1:T})$ is the encoder/recognition model. To
generate a trajectory, we sample the latent of the decoder input $z\sim
p_{\theta}(z|x_{1,T})$. We then sample with an auto-regressive decoder
$p_{\theta}(x^{\prime}_{t+1}|x^{\prime}_{2:t},x_{1,T},z)$. The training
process can be found in App. -A.

Either the prior or the autoregressive decoder is capable of generating
diverse samples. To enhance adaptability and generalization, our model
incorporates both for distinct purposes. With a fixed demonstration/posterior,
we can adjust the trajectories from the decoder. Conversely, by specifying a
starting and ending point, we can generate various trajectories using the
prior.

###  II-B PerceiverIO encoder

We implement the encoder $q_{\phi}(\operatorname{z}|\operatorname{x}_{1:T})$
via a PerceiverIO model [6], that is, as multi-layered, attention-based neural
architecture where each layer is composed of a cross-attention (CA) and as
self-attention (SA) block as

$\displaystyle h^{(l+1)}_{1:K}$
$\displaystyle=\mathrm{RMLP}_{\theta}\circ\mathrm{RSA}_{\theta}(h_{1:K}^{(l)}),\quad
l=1,\ldots,L$ $\displaystyle h^{(1)}_{1:K}$
$\displaystyle=\mathrm{RMLP}_{\theta}\circ\mathrm{RCA}_{\theta}(h^{(0)}_{1:K},\tilde{x}_{1:T}).$

The initial layer $h^{(0)}_{\theta}$ is a learned parameter, a part of
$\theta$. The values $\tilde{x}_{t}=W_{\theta}x_{t}+\tau_{t}$ are embedding
the data into the hidden space via a linear layer $W_{\theta}$ and a
positional embedding $\tau_{t}$. RCA and RSA denote a residual (R) cross-
attention and residual self-attention block defined as
$\mathrm{RCA}(h,z)=\mathrm{LayerNorm}(h+\mathrm{CA}(h,z))$ and
$\mathrm{RSA}(h)=\mathrm{LayerNorm}(h+\mathrm{SA}(h))$, respectively. The
encoder is then defined as a diagonal multivariate Gaussian distribution
$q_{\phi}(z|x_{1:T})=\mathcal{N}(z;\mu_{\theta}^{\mathrm{enc}}(h^{(L)}),\sigma_{\theta}^{\mathrm{enc}}(h^{(L)})^{2})$
with $\mu_{\theta}^{\mathrm{enc}}$ and $\sigma_{\theta}^{\mathrm{enc}}$ being
simple MLPs. The latent variable $z$ is thus designed to encode the trajectory
$x_{1:T},x_{t}\in\mathbb{R}^{d}$ to a shorter sequence of $K$ variables
$z_{k}\in\mathbb{R}^{m}$. Note that the choice $K=1$ results in a single
vector embedding of the trajectory $x_{1:T}$.

To implement the prior $p_{\theta}(z|x_{1,T})$ we use the same model having as
input only the pair $\\{x_{1},x_{T}\\}$.

This type of Perceiver architecture is designed to reduce the quadratic
($T\times T$) computational complexity of a Transformer encoder model (self-
attention layers) on $x_{1:T}$ by only cross-attending the sequence ($T\times
K$) and performing less computationally demanding self-attentions ($K\times
K$) on $z_{1:K}$. Generally, there is a large class of Perceiver models/blocks
with learnable queries or latents that are used for transforming data between
different modalities, for example, a Perceiver model with $K$ queries can be
used as a head-block in a classification model with $K$ classes.

###  II-C Autoregressive decoder

Figure 1: Model overview. Each PerceiverIO module gets a trajectory, but for
the prior it only contains the beginning and end points and a full trajectory
for the posterior. During training, minimizing the KL divergence between the
two makes the representation of the PerceiverIO module that only gets $x_{1}$
and $x_{T}$ generates the same latent distribution as the one getting the full
trajectory; therefore, it learns interpolation. The latent state then is used
by the autoregressive decoder to generate the next setpoint $x_{t+1}$. Dashed
lines indicate components used exclusively during the post-training adaptation
phase, not included in the initial training process.

A natural choice for the decoder in the context of attention-based models is a
conditional autoregressive model

$p_{\theta}(x_{2:T-1}|x_{1,T},z)=\prod_{t=1}^{T-1}p_{\theta}(x_{t+1}|x_{2:t},x_{1,T},z).$

Here, we again use an attention-based neural architecture, a standard
transformer decoder [7] defined as

$\displaystyle
h^{(l)}_{1:T}=\mathrm{RMLP}^{(l+1)}_{\theta}\circ\mathrm{RCA}_{\theta}^{(l+1)}(\cdot,z)\circ\mathrm{RCSA}_{\theta}^{(l+1)}(h^{(l)}_{1:T})$

with $h^{(0)}_{1:T}=\tilde{x}_{1:T}$ being the embeddings defined in the
previous section. The value $h^{(L)}_{t}$ is then used to define the
autoregressive model
$p(x_{t+1}|x_{2:t},x_{1,T},z)=\mathcal{N}(x_{t+1};\mu_{\theta}^{\mathrm{dec}}(h^{(L)}_{t}),\sigma_{\theta}^{\mathrm{dec}}(h^{(L)}_{t})^{2})$.
The additional “C” in the layer definition in $\mathrm{RCSA}$ denotes a casual
attention model implemented via triangular attention masks. Unlike
Transformers for text infilling [8, 9], we do not include segmentation
embedding since the length and position of the start and end points in the
model are already provided.

###  II-D Trajectory generator—Guided decoding

In addition to learning a conditional autoencoder model for trajectory
generation we would also like to extend the model to be able to perform online
trajectory adaptation, for example, add via-points and obstacle avoidance. To
achieve this we do online (approximate) probabilistic inference using
$p_{\theta}(x_{t+1}|x_{1:t},x_{T},z)$ without updating the model parameter
$\theta$. The obvious choice is a Bayesian approach where the adaptation can
be formulated as a Bayesian update of $p_{\theta}(x_{t+1}|x_{1:t},x_{T},z)$,
via a likelihood $p(y_{t+1}|x_{t+1})$. Alternatively, the likelihood can also
be used to implement constraints, for example, a Bayesian update with a
likelihood function $\mathrm{I}_{[a,b]}(x)$ can implement trajectory
boundaries.

For generic constraints we can formulate online trajectory adaptation as

$\displaystyle\min_{q(x_{t+1})}$
$\displaystyle\>\mathrm{KL}[q(x_{t+1})\,\|\,p_{\theta}(x_{t+1}|x_{1:t},x_{T},z)]$
(1) $\displaystyle\mathrm{s.t.}$
$\displaystyle\>\mathrm{E}_{q(x_{t+1})}[H(c(x_{t+1}))]\leq\alpha,$ (2)

where $H(\cdot)$ is the Heaviside function, $c(\cdot)$ is a constraint
function. The constraint in (2) expresses what proportion $0\leq\alpha\leq 1$
of the probability mass is allowed not to satisfy the constraint [10]. The KL
objective is convex in $q$, the constraints are linear, and the optimal
solution $(q^{\ast},\lambda^{\ast})$ satisfies

$\displaystyle q^{\ast}(x_{t+1})\propto
p_{\theta}(x_{t+1}|x_{1:t},x_{T},z)\times\exp\\{-\lambda^{\ast}H(c(x_{t+1}))\\}.$

Solving this optimization, however, even for as Gaussian $q$ and a $c$
implementing a boundary constraint can be numerically demanding. For more
general constraints such as obstacle avoidance, (2) can be analytically
intractable. For this reason, in the following we will only exploit the form
of $q^{\ast}(x_{t+1})$ to propose fast online trajectory adaptation methods.
Various trajectory adaptations can be conveniently formulated either as (1) by
choosing the appropriate constraint function $c(\cdot)$ or as a Bayesian
update with the appropriate likelihood. These will be detailed in the
following.

###  II-E Implementing specific constraints

####  II-E1 Obstacle avoidance with beam search

Beam search (BS) [11] is a classic heuristic in the field of natural language
processing (autoregressive models). It is used to generate trajectories with
high likelihood or other specific score values. The method recursively
generates trajectories as follows. A set of $S$ trajectories
$\\{x_{2:t}^{(s)}\\}$ are stored together with their scores, for example, the
path likelihood $\log p(x_{2:t}^{(s)}|x_{1,T},z)$. For each trajectory $S$ new
samples $x_{t+1}^{(s,r)}\sim p(x_{t+1}|x_{2:t}^{(s)},x_{1,T},z),r=1,\ldots,S$
are generated as candidates and the corresponding total scores are computed
for the extended trajectories $(x^{(s)}_{2:t},x_{t+1}^{(s,r)})$ for all
$(s,r)$ pairs. The overall top $S$ extended trajectories are selected for the
next step. Inspired by the form of $q^{\ast}(x_{t+1})$, we define the score by
adding $\tilde{s}(x_{t+1})=-\gamma\max(c(x_{t+1}),0)$ with
$c(x)=\delta^{2}-\|\,x-x_{\mathrm{obstacle}}\,\|^{2}$ to the likelihood.

####  II-E2 Position, velocity and acceleration bounds

Simple position and velocity bounds in an interval $[a,b]$ can be implemented
via a Bayesian update. In case of position bounds the posterior,
$q(x_{t+1})\propto\mathrm{I}_{[a,b]}(x_{t+1})\>p_{\theta}(x_{t+1}|x_{1:t},x_{T},z)$
is a truncated Gaussian from which we can easily sample. In case of velocity
bounds $[v_{\mathrm{min}},v_{\mathrm{max}}]$, we notice that
$v_{t}=(x_{t+1}-x_{t})/{\Delta t}$ is Gaussian to which we can apply a
Bayesian update via $\mathrm{I}_{[v_{\mathrm{min}},v_{\mathrm{max}}]}(v_{t})$.
Due to the fact that the predictive distribution is a diagonal (independent)
Gaussian, this results in a truncated Gaussian distribution on $v_{t+1}$ and
subsequently on $x_{t+1}$. Acceleration bounds follow the same principle
applied to $a_{t}=(\operatorname{x}_{t+1}-2x_{t}+x_{t-1})/{\Delta t^{2}}$.

####  II-E3 Via points through Brownian bridge

To add additional via-points to the autoregressive model
$p_{\theta}(x_{t+1}|x_{1:t},x_{T},z)$, say, $x_{s}$ at an intermediate time
$1<s<T$ we would have to compute the Bayesian posterior

$\displaystyle p(x_{t+1}$ $\displaystyle|x_{1:t},x_{s},x_{T},z)$
$\displaystyle\propto p(x_{s}|x_{t+1},x_{1:t},x_{T},z)\times
p(x_{t+1}|x_{1:t},x_{T},z).$

Since the likelihood term $p(x_{s}|x_{t+1},x_{1:t},x_{T},z)$ is intractable,
we propose to approximate it with a Gaussian likelihood

$\displaystyle
p(x_{s}|x_{t+1},x_{1:t},x_{T},z)\approx\mathcal{N}(x_{s}|x_{t+1},(s-t-1)\>\sigma_{\mathrm{vp}}^{2}).$

To have a stronger conditioning on the past $x_{1:t}$ we can alternatively use
a Brownian-bridge approximation

$\displaystyle\mathcal{N}\left(x_{t+1}|x_{t}+\frac{1}{s-t}(x_{s}-x_{t}),\sigma^{2}_{\mathrm{vp}}\frac{s-t-1}{s-t}\right).$

Both of the above approximations lead to a Gaussian approximation of
$p(x_{t+1}|x_{1:t},x_{s},x_{T},z)$.

####  II-E4 Constrained Beam Search for Brownian bridge

Drawing on the Constrained Beam Search (CBS) described by [12], we refine the
approach by integrating a controller. This addition drives the generated
trajectory towards the desired outcome, rather than substituting words.

We generate samples of $x_{t+1}$ for each sample $x_{t}$, using the
Transformer decoder incorporating via point method, i.e., Brownian Bridge. We
categorize the generated trajectories into three banks: Bank0 for trajectories
that do not meet the constraint; Bank1 for trajectories that have applied the
controller but still do not satisfy the constraint; and Bank2 for trajectories
that satisfy the constraint. For the trajectory $x_{1:t}$ that satisfies the
via-point constraint, we generate a single sample of $x_{t+1}$ using the
Transformer decoder exclusively for Bank2. If $x_{1:t}$ fails to satisfy the
constraint, we then generate one $x_{t+1}$ sample from the Transformer decoder
with the controller. In this case, if $x_{t+1}$ satisfies the constraint, the
sample is allocated to Bank2; otherwise, it goes to Bank1. Lastly, in cases
where $x_{1:t}$ does not satisfy the constraint, we generate $N_{s}$ $x_{t+1}$
samples from the Transformer decoder only. Here, if $x_{t+1}$ satisfies, the
sample is classified into Bank2; if not, into Bank0. Following this, we select
the samples from each bank with the order of Bank2, Bank1, Bank0, iteratively
based on the negative log-likelihood criterion until we have $N_{s}$ samples
in total for $x_{t+1}$.

Previous work [12] has demonstrated that CBS outperforms BS during guided
language generation, akin to via point task in robot trajectory generation.
However, for tasks such as obstacle avoidance, only BS is applicable, since
Bank0 and Bank1 cannot be defined.

##  III Related work

In robotics, movement primitives are used to specify behavior based on human-
demonstrated trajectories. Dynamic movement primitives (DMP) take inspiration
from dynamical systems to specify motion behavior [1, 2]. DMPs use human
demonstrations to fit a forcing term that augments the attractor dynamics to
match the demonstrated trajectories. DMPs allow motion adaptation, such as
avoidance of obstacles, by adding additional terms such as repulsive fields to
their driving equation [13, 14]. This process should be done carefully to
avoid competing objectives. Furthermore, DMPs applied in latent space [15, 16]
provide the capability to adjust motions within this latent space. Unlike
these works which adapt the latent space incrementally, our approach is a
sequence-to-sequence model. In our model, each trajectory is mapped to a
distinct point in the latent space, affording greater flexibility in guiding
the decoder’s output. Probabilistic movement primitives represent trajectory
distributions, which simplifies the blending of multiple trajectories [3].
Adapting movement to auxiliary tasks is also possible within this framework
[10, 17, 18]. Riemannian motion policies easily allow the combination of
multiple behaviors in both the task and the configuration space [4]. However,
this method lacks the advantages of probabilistic formulations.

##  IV Experiments

###  IV-A Data Collection

To train our models we collect a large dataset of robot motion trajectories in
simulation. We use the NVIDIA IsaacSim simulator to generate a large amount of
data in parallel. Our main experiments are based on data using the Franka
Emika Panda robot. However, for the multi-robot experiment we also introduce
motion data from different robots, namely Rethink Robotic’s Sawyer, Kuka’s LBR
IIWA 7, Jaco and Kinova Gen3 from Kinova Robotics, and Universal Robot’s UR5e.
We generate samples for each robot in 4096 environments in parallel and repeat
this process for multiple episodes. At the beginning of each episode, we
randomly sample starting and end-goal positions in the configuration space. We
then use a proportional derivative controller to drive the robot to the goal
position. To generate more diverse trajectories, we occasionally sample via
points at the beginning of an episode and use these as target points for a
short while at the beginning of the episode.

For the trajectory of the end-effector, we have $x\in\mathbb{R}^{3}$, while
$x\in\mathbb{R}^{7}$ for joint space. The robots which have 6-DOF are padded
zero to match this representation. Each sequence has a total length of $T=64$.

###  IV-B Distances to the original trajectories

Figure 2: Distances to the original trajectories for the Panda robot in
Cartesian space.  Figure 3: Distances to the original trajectories for IIWA
robot in joint space.

We split the dataset into 20,000 training, 2048 validation, and 2048 test
samples. A single robot was trained and then evaluated on its performance.
Fig. 2 and 3 illustrate the generalization from the prior and the
reconstruction. Evaluation metrics include the Root Mean Square Error (RMSE)
measured in degrees for joint movements, and an L2 Norm, measured in meters,
for assessing the trajectories for the end-effector. Assuming endpoint
distances are the same, a larger distance for trajectories from the prior
distribution to the original demos is advantageous, as it demonstrates the
diversity of the learned prior. Except that, lower distance values to the
original samples indicate better performance, where the distance reflects the
fidelity of the model’s output approximating the original trajectories. Rather
than determining the endpoint distance, we calculate the distance at
$x_{T-1}$, since the $x_{T}$ is given.

###  IV-C Generalization on an unknown robot

Figure 4: Distances to the original trajectories for the Panda robot in
Cartesian space with multiple-robot training and single-robot training.

Scaling up models across various robots presents significant challenges but
offers substantial benefits, as noted in [19]. To assess our model’s ability
to generalize, we conducted an experiment involving different robots. In this
experiment, the training dataset comprises IIWA, UR5e, and Sawyer robots
operating in Cartesian space. The validation dataset uses the Jaco robot,
while the Panda robot is the test dataset. For comparison of single-robot
training, the training dataset was divided into three subsets, each
corresponding to a specific robot, with the same validation and test datasets
used across all experiments. The results from the single-robot training are
aggregated for presentation. As illustrated in Fig. 4, training across
multiple robots demonstrates superior generalization to an unknown robot.

###  IV-D Adaptation and generalization

The experiments are trained on a Panda robot. Our adaption and generalization
are suitable for both samples from priors and posterior. To enhance
visualization and compare with demonstrations, we present all results based on
samples from posteriors, except for those in Sections IV-D2 and IV-D4 which
also use samples from priors.

####  IV-D1 Via points

Figure 5: Brownian bridge for via point with (upper) and without (lower) CBS.
Colors indicate the axes of the Cartesian coordinate.

We adapted the movements for via point (see Fig. 5) using the Brownian bridge.
We set the hyperparameter $\sigma_{\mathrm{vp}}$ in a similar range of the
output $\sigma$ from the autoregressive decoder. The experiments show the
comparisons between via points without and with CBS, where the beam search
size is set to five. The findings indicate that when CBS is used, the
trajectories appear more natural and closer to the original demos. On the
contrary, as CBS is a historical search method, a Brownian bridge without CBS
enables online trajectory generation.

####  IV-D2 Obstacle avoidance

Figure 6: Adaption for obstacle avoidance. (Upper) Obstacle avoidance with the
samples from the posterior. (Lower) Obstacle avoidance with samples from the
prior, where the original refers to the samples from the prior without
adaptation of the decoder. All of them successfully avoid the obstacles,
although the visualization is not perfect due to the viewpoint.

In the experiment, we used a beam search with a size of 25. The temperature of
the output, which is the scale of the output STD, was set to two. Fig. 6
demonstrates how the adapted trajectories effectively navigate to avoid
spherical obstacles for samples drawn from both the prior and posterior
distributions.

####  IV-D3 Position bound

Figure 7: Motion adaptation for position boundary. Colors indicate the axes of
the Cartesian coordinate.

As depicted in Figure 7, the generated trajectories are supposed to follow the
original demonstrations, with adaptations made to ensure compatibility within
the cuboid boundaries.

####  IV-D4 Velocity and acceleration bound

Table I: Velocity and acceleration limits. methods | samples | max values
---|---|---
velocity bound | prior | 0.050
| posterior | 0.050
| boundary | 0.050
| original data | 0.094
acceleration bound | prior | 0.020
| posterior | 0.020
| boundary | 0.020
| original data | 0.031

We have established boundaries for both velocity and acceleration, as detailed
in Table LABEL:tab:v_a_limit. Although the table presents only a single
boundary for each type of boundary, these boundaries can be set to arbitrary
values.

##  V Conclusion

In this paper we introduce a novel Transformer-based auto-encoder framework
for robot motion generation and adaptation, leveraging Learning from
Demonstration (LfD). The latent space of the autoencoder can be viewed a
motion representation similarly as in movement primitives. The conditioning of
the trajectory generation on goal an motion representation is implemented
through flexible cross-attention mechanisms. By using an autoregressive
decoder and online (approximate) inference methods our approach generates
trajectories that adapt to various task space constrains such as obstacle
avoidance, via-points, and various position, velocity, and acceleration
bounds. The experiments validated the framework’s robustness and adaptability
across different robots and various trajectory constraints.

## References

  * [1] S. Schaal, “Dynamic movement primitives-a framework for motor control in humans and humanoid robotics,” in _Adaptive motion of animals and machines_. Springer, 2006, pp. 261–280.
  * [2] A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal, “Dynamical movement primitives: learning attractor models for motor behaviors,” _Neural computation_ , vol. 25, no. 2, pp. 328–373, 2013.
  * [3] A. Paraschos, C. Daniel, J. R. Peters, and G. Neumann, “Probabilistic movement primitives,” _Advances in neural information processing systems_ , vol. 26, 2013.
  * [4] N. D. Ratliff, J. Issac, D. Kappler, S. Birchfield, and D. Fox, “Riemannian motion policies,” _arXiv preprint arXiv:1801.02854_ , 2018.
  * [5] K. Sohn, H. Lee, and X. Yan, “Learning structured output representation using deep conditional generative models,” _Advances in neural information processing systems_ , vol. 28, 2015.
  * [6] A. Jaegle, F. Gimeno, A. Brock, O. Vinyals, A. Zisserman, and J. Carreira, “Perceiver: General perception with iterative attention,” in _International conference on machine learning_. PMLR, 2021, pp. 4651–4664.
  * [7] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [8] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” _arXiv preprint arXiv:1810.04805_ , 2018.
  * [9] T. Wang and X. Wan, “T-cvae: Transformer-based conditioned variational autoencoder for story completion.” in _IJCAI_ , 2019, pp. 5233–5239.
  * [10] F. Frank, A. Paraschos, P. van der Smagt, and B. Cseke, “Constrained probabilistic movement primitives for robot trajectory adaptation,” _IEEE Transactions on Robotics_ , vol. 38, no. 4, pp. 2276–2294, 2021.
  * [11] P. Koehn, _Statistical machine translation_. Cambridge University Press, 2009.
  * [12] M. Post and D. Vilar, “Fast lexically constrained decoding with dynamic beam allocation for neural machine translation,” _arXiv preprint arXiv:1804.06609_ , 2018.
  * [13] D.-H. Park, H. Hoffmann, P. Pastor, and S. Schaal, “Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields,” in _Humanoids 2008-8th IEEE-RAS International Conference on Humanoid Robots_. IEEE, 2008, pp. 91–98.
  * [14] H. Hoffmann, P. Pastor, D.-H. Park, and S. Schaal, “Biologically-inspired dynamical systems for movement generation: Automatic real-time goal adaptation and obstacle avoidance,” in _2009 IEEE International Conference on Robotics and Automation_ , 2009, pp. 2587–2592.
  * [15] N. Chen, J. Bayer, S. Urban, and P. Van Der Smagt, “Efficient movement representation by embedding dynamic movement primitives in deep autoencoders,” in _2015 IEEE-RAS 15th international conference on humanoid robots (Humanoids)_. IEEE, 2015, pp. 434–440.
  * [16] N. Chen, M. Karl, and P. Van Der Smagt, “Dynamic movement primitives in latent space of time-dependent variational autoencoders,” in _2016 IEEE-RAS 16th international conference on humanoid robots (Humanoids)_. IEEE, 2016, pp. 629–636.
  * [17] D. Koert, G. Maeda, R. Lioutikov, G. Neumann, and J. Peters, “Demonstration based trajectory optimization for generalizable robot motions,” in _2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids)_ , 2016, pp. 515–522.
  * [18] D. Koert, J. Pajarinen, A. Schotschneider, S. Trick, C. Rothkopf, and J. Peters, “Learning intention aware online adaptation of movement primitives,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 4, pp. 3719–3726, 2019.
  * [19] A. Padalkar, A. Pooley, A. Jain, A. Bewley, A. Herzog, A. Irpan, A. Khazatsky, A. Rai, A. Singh, A. Brohan, _et al._ , “Open x-embodiment: Robotic learning datasets and rt-x models,” _arXiv preprint arXiv:2310.08864_ , 2023\.
  * [20] T. Sønderby, C. K.and Raiko, L. Maaløe, S. K. Sønderby, and O. Winther, “Ladder variational autoencoders,” _NeurIPS_ , 2016.
  * [21] S. R. Bowman, L. Vilnis, O. Vinyals, A. M. Dai, R. Jozefowicz, and S. Bengio, “Generating sentences from a continuous space,” _arXiv preprint arXiv:1511.06349_ , 2015.
  * [22] D. P. Kingma, T. Salimans, R. Jozefowicz, X. Chen, I. Sutskever, and M. Welling, “Improved variational inference with inverse autoregressive flow,” _Advances in neural information processing systems_ , vol. 29, 2016\.
  * [23] A. Roberts, J. Engel, C. Raffel, C. Hawthorne, and D. Eck, “A hierarchical latent vector model for learning long-term structure in music,” in _International conference on machine learning_. PMLR, 2018, pp. 4364–4373.
  * [24] D. J. Rezende and F. Viola, “Taming VAEs,” _CoRR_ , 2018.
  * [25] A. Klushyn, N. Chen, R. Kurle, B. Cseke, and P. van der Smagt, “Learning hierarchical priors in VAEs,” _Advances in Neural Information processing Systems_ , vol. 32, 2019.
  * [26] D. P. Bertsekas, _Nonlinear Programming: Second Edition_. Athena Scientific, 2003.
  * [27] L. Liu, H. Jiang, P. He, W. Chen, X. Liu, J. Gao, and J. Han, “On the variance of the adaptive learning rate and beyond,” _arXiv preprint arXiv:1908.03265_ , 2019.
  * [28] H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” _arXiv preprint arXiv:1710.09412_ , 2017.

###  -A Training

Training VAEs and CVAEs can be challenging due to local minima, posterior
collapse or over-regularization [20, 21, 22]. Sequence-to-sequence models with
strong decoders are often prone to posterior collapse. The authors in [23]
reduced the capability of the decoder using the hierarchical decoder.
Alternatively, reformulating the objective or using specialized training
strategies can also alleviate some optimization problems. The authors in [24]
reformulate training as a constrained optimization problem where the
reconstruction term
$-\mathrm{E}_{\hat{p}(x_{1:T})}\mathrm{E}_{q_{\phi}(z;x_{1:T})}[\log
p_{\theta}(x_{2:T-1}|x_{1,T},z)]$ term is constrained to achieve specific
reconstruction accuracy. The authors in [25] train a hierarchical VAE model
using a similar reformulation and use a scheduled pre-training of the
reconstruction term. To attain a required reconstruction loss (powerful
decoder) and to also avoid posterior collapse we formulate the optimization
problem as

$\displaystyle\min_{\theta,\phi}$ $\displaystyle\>1$
$\displaystyle\mathrm{s.t.}$
$\displaystyle\>\mathrm{E}_{\hat{p}(x_{1:T})}\mathrm{E}_{q_{\phi}(z;x_{1:T})}[-\log
p_{\theta}(x_{2:T-1}|x_{1,T},z)]\leq\xi_{\mathrm{rec}}$
$\displaystyle\>\mathrm{E}_{\hat{p}(x_{1:T})}\mathrm{KL}[q_{\phi}(z;x_{1:T})\,\|\,p_{\theta}(z;x_{1,T})]\leq\xi_{\mathrm{kl}}.$

The resulting Lagrangian will have two multipliers $\lambda_{\mathrm{rec}}$
and $\lambda_{\mathrm{kl}}$ that adaptively re-weight the reconstruction and
the KL (regularization) terms. We use the exponential method of multipliers
[26], that is, we apply the update
$\lambda_{\mathrm{kl}}^{(k+1)}=\lambda_{\mathrm{kl}}^{(k)}\cdot\exp\\{\eta(-\mathrm{E}_{\hat{p}(x_{1:T})}\mathrm{KL}[q_{\phi}(z;x_{1:T})\,\|\,p_{\theta}(z;x_{1,T})]+\xi_{\mathrm{kl}})\\}$
and a corresponding one for $\lambda_{\mathrm{rec}}$.

###  -B Architecture and computation

The computational analyses of this research used an NVIDIA GeForce GTX 1080 Ti
GPU, with PyTorch version 2.1.0 for implementation.

The model’s encoder depth was set to 3, with an embedding dimension of 16, and
a latent vector length of 32. The decoder was configured with 8 heads and 10
layers. For optimization, the RAdam optimizer [27], was chosen, configured
with beta values of 0.9 and 0.999, and a learning rate of 0.0005. Batch sizes
were 256 for the data in Cartesian space and 128 for the joint space. Mixup
[28] was used as a data augmentation for the joint space data.