# Sentence Simplification Using Paraphrase Corpus for Initialization

Kang Liu, Jipeng Qiang  Department of Computer Science, Yangzhou University
Jiangsu, China
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Neural sentence simplification method based on sequence-to-sequence framework
has become the mainstream method for sentence simplification (SS) task.
Unfortunately, these methods are currently limited by the scarcity of parallel
SS corpus. In this paper, we focus on how to reduce the dependence on parallel
corpus by leveraging a careful initialization for neural SS methods from
paraphrase corpus. Our work is motivated by the following two findings: (1)
Paraphrase corpus includes a large proportion of sentence pairs belonging to
SS corpus. (2) We can construct large-scale pseudo parallel SS data by keeping
these sentence pairs with a higher complexity difference. Therefore, we
propose two strategies to initialize neural SS methods using paraphrase
corpus. We train three different neural SS methods with our initialization,
which can obtain substantial improvements on the available WikiLarge data
compared with themselves without initialization.

###### Index Terms:

Sentence Simplificaiton, Paraphrase Corpus, Seq2Seq

##  I Introduction

The goal of sentence simplification (SS) task is to rephrase a sentence into a
form that is easier to read and understand, while still retaining the semantic
meaning, which can help people with reading difficulties such as non-native
speakers[1, 2], dyslexia[3] or autism[4]. Second language learners[5] and
people with low literacy[6] can also benefit from it.

Since the 2010 year, SS task have been addressed as a monolingual machine
translation problem translating from complex sentences to simplified
sentences. Existing SS methods have changed from statistical sentence
simplification methods [7, 8, 9] to neural sentence simplification methods
[10, 11, 12, 13]. Neural sentence simplification methods adopt sequence to
sequence (Seq2Seq) models. Seq2Seq models work very well only when provided
with a massive parallel corpus of complex and simplified sentences.
Unfortunately, these approaches are currently limited by the scarcity of
parallel corpus. For example, the biggest and widely-used SS training dataset
WikiLarge [10] is composed of 296,402 sentence pairs, which align sentences
from the ’ordinary’ English Wikipedia and the ’simple’ English Wikipedia.
WikiLarge has been criticized recently [14, 15] because they contain a large
proportion of noise data, which leads to systems that generalize poorly. Some
work [16, 12, lu2021unsupervised, 17] foucs on unsupervised SS method for
alleviating the need for SS supervised corpora. In this paper, we focus on how
to reduce the dependence on parallel corpus by leveraging a careful
initialization for neural SS methods.

There are large-scale paraphrase datasets [18, 19] for paraphrase generation
whose aim is to generate an output sentence that preserves the meaning of the
input sentence but contains variations in word choice and grammar. Comparing
with paraphrase dataset, SS dataset highlights that the two sentences of each
sentence pair should have difference in text complexity levels. We found that
there are a large proportion of sentence pairs in paraphrase dataset that
satisfy the expectations of SS task. For example, paraphrase dataset ParaBank
[19] was created automatically from bilingual text by pivoting over the non-
English language using neural machine translation (NMT) models. NMT models
usually tend to generate more high-frequency tokens and less low-frequency
tokens [20, 21]. Considering that the higher the word frequency, the simpler
the word is, this phenomenon could be beneficial to SS task. Table 1 shows two
sentence pairs from paraphrase corpus ParaBank. We can see that the translated
target sentence is easier than the source sentence.

Table 1 Two examples in ParaBank paraphrase corpus

Source |  This proposal will be communicated to the trader ’s
---|---
creditors.
Target |  This plan will be sent to the trader ’s creditors
Source |  It would be prudent for you not to be deceived by
your masquerade .
Target |  It would be wise for you not to be fooled by your
own masquerade .

In this paper, we will try to utilize paraphrase corpus to initialize neural
SS methods, and then fine-tune these methods on real SS dataset. Specifically,
we design two strategies for initialization. (1) We directly utilize the whole
paraphrase corpus to train an initial SS method. (2) Considering many
sentences pairs in paraphrase corpus cannot satisfy the expectations of SS
task, we only select these sentence pairs with a higher complexity difference
using text readability formula (Flesch reading ease score [22]), which is
designed to indicate how difficult a sentence is to understand. Experimental
results show that neural SS methods based on our initialization outperform
themselves without initialization.

The following sections are organized as follows: Section 2 describes the
related work; Section 3 presents how to initialize neural SS methods; Section
4 shows the experimental results; Section 5 summarizes the paper.

##  II Related Work

###  II-A Sentence Simplification

Automatic SS is a complicated natural language processing (NLP) task, which
consists of lexical and syntactic simplification levels. It has attracted much
attention recently as it could make texts more accessible to wider audiences,
and used as a pre-processing step, improve performances of various NLP tasks
and systems. Usually, hand-crafted, supervised, and unsupervised methods based
on resources like English Wikipedia and Simple English Wikipedia (EW-SEW) [23]
are utilized for extracting simplification rules. It is very easy to mix up
the automatic TS task and the automatic summarization task [7, 24]. TS is
different from text summarization as the focus of text summarization is to
reduce the length and redundant content.

At the lexical level, lexical simplification often substitutes difficult words
using more common words, which only require a large corpus of regular text to
obtain word embeddings to get words similar to the complex word [25, 26, 27,
28]. Woodsend and Lapata [29] presented a data-driven model based on a quasi-
synchronous grammar, a formalism that can naturally capture structural
mismatches and complex rewrite operations. Wubben et al. [30] proposed a
phrase-based machine translation (PBMT) model that is trained on ordinary-
simplified sentence pairs. Xu et al. [9] proposed a syntax-based machine
translation model using simplification-specific objective functions and
features to encourage simpler output.

Neural machine translation has shown to produce state-of-the-art results [31,
32, 21], which are based on sequence-to-sequence (Seq2Seq) architecture. In
recent years, many neural SS models based on Seq2Seq are proposed and achieve
good results [33, 10, 11, 12, 13, 34]. The main limitation of the
aforementioned neural SS models depended on the parallel ordinary-simplified
sentence pairs [15]. Because ordinary-simplified sentence pairs are expensive
and time-consuming to build, the available largest data is WikiLarge [10] that
only has 296,402 sentence pairs. The dataset is insufficiency for neural SS
model if we want to they can obtain the best parameters. Considering
paraphrase corpus includes a large number of sentence pairs that satisfy the
expectations of SS task. In this paper, we investigate the use of paraphrase
data for text simplification. We are the first to show that we can effectively
adapt paraphrase data for SS task.

###  II-B Unsupervised Sentence Simplification

To overcome the scarcity of parallel SS corpus, unsupervised SS methods
without using any parallel corpus have attracted much attention. Existing
unsupervised SS methods can be divided into two classifications. The first
scheme focuses on how to design an unsupervised SS method, and the second
scheme concentrates on how to build a parallel SS corpus.

[35] and [36] are the pipeline-based unsupervised framework, where the
pipeline of Narayan and Gardent is composed of lexical simplification,
sentence splitting, and phrase deletion, the pipeline of Kumar et al. includes
deletion, reordering, and lexical simplification. [37] proposed an
unsupervised neural text simplification based on a shared encoder and two
decoders, which only learn the neural network parameters from simple sentences
set and complex sentences set. In other languages, there are unsupervised
statistical machine translations for Japanese [38] and back-translation in
Spanish and Italian [39]. The performance of the above unsupervised SS methods
is however often below their supervised counterparts.

Some work [16, 12] constructed SS corpora by searching the most similar
sentences using sentence embedding modeling, and train SS methods using the
constructed SS corpora. [16] calculated the similarity between the sentences
from English Wikipedia by Word Mover’s distance [40]. [12] adopted
multilingual sentence embedding modeling LASER [32] to calculate the
similarity between the sentences from 1 billion sentences from CCNET [41].
Since the aim of the two works is to find the most similar sentences from a
large corpus, they cannot guarantee that the aligned sentences preserve the
same meanings. Lv et al. [42] construct large-scale pseudo parallel SS data by
taking the pair of the source sentences of translation corpus and the
translations of their references in a bridge language.

###  II-C Paraphrase Mining

Some work has focused on generating paraphrase corpus for neural machine
translation (NMT) systems using back-translation, where back-translation [43]
is a technique widely used in NMT to enhance the target monolingual data
during the training process. Specifically, the back-translation technique is
used by translating the non-English side of bitexts back to English[44] and
pairing translations with the references. Two large paraphrase corpora
(PARANMT[wieting2017paranmt] and PARABANK[19]) are built based on this idea,
and has been proven to have great potential in different translation-core
tasks. Round-trip translation is also used in mining paraphrases [45] by
translating sentences into another language then translating the result back
into the original language. Similar to machine translation, back-translation
is used to improve the performance of neural SS methods [38, 39, 46].
[mehta2020simplifythentranslate] trained a paraphrasing model by generating a
paraphrase corpus using back-translation, which is used to preprocess source
sentences of the low-resource language pairs before feeding into the NMT
system.

The above work for building a large paraphrase corpus is to serve for NMT and
other tasks, which is not fit for SS task. The difference of sentence
complexity between the original sentence and the translated sentence for each
sentence pair has not been taken into consideration, which is vitally
important for SS task. Therefore, we focus on how to build a sentence
simplification corpus, instead of a paraphrase corpus.

##  III Method

In this section, we will present how to utilize paraphrase corpus to
initialize neural SS models.

###  III-A Relation between Paraphrase Corpus and SS Corpus

Some work has focused on generating paraphrase corpus for neural machine
translation (NMT) systems using back-translation technique, where back-
translation [47] is a technique widely used in NMT to enhance the target
monolingual data during the training process. Specifically, the back-
translation technique is used by translating the non-English side of bitexts
back to English [48] and pairing translations with the references. We can see
that the two sentences of each sentence pair of paraphrase corpus should
preserve the same meaning.

Hypothesis 1: SS corpus can be regarded as a subset of paraphrase corpus.
Based on the definition of SS task, SS corpus should satisfy the following two
requirements: (1) The two sentences of each sentence pair should convey the
same meaning. (2) The two sentences of each sentence pair should have
difference in text complexity levels. Paraphrase only needs to satisfy the
first requirement, and SS corpus needs to satisfy both the requirements.

Hypothesis 2: Paraphrase corpus includes a large proportion of sentence pairs
belonging to SS corpus. Neural machine translation model usually tends to
generate more high-frequency tokens and less low-frequency tokens [20, 21].
The frequency of words is one of the most popular choices by sentence
simplification [26, 46]. In general, the higher the frequency, the easier the
word. Many empirical results supported the hypothesis, as shown in Table 1.

###  III-B Our Initialization Strategy

We provide two strategies to initialize neural SS models.

(1) First Initialization Strategy: Based on Hypothesis 2, we directly utilize
paraphrase corpus to train initial neural SS modeling. Here, we choose
ParaBank as our using paraphrase corpus. Due to the memory size of our
computer, we only randomly choose 2 million sentence pairs from ParaBank [19].
Finally, we train neural SS modeling on real SS corpus.

(2) Second Initialization Strategy: Our second initialization strategy is
shown in Figure 1. Different from the first one, we only select these sentence
pairs from paraphrase corpus that have difference in text complexity levels.
We measure the difference of text complexity using Flesch reading ease score
(FRES) [22], which is designed to indicate how difficult a sentence is to
understand, and is widely used to evaluate the performance of SS. FRES
proposed in 1975 is a classical formula in the field of text assessment, whose
coefficients are set by linguists. It is based on text features such as the
average sentence length and the average number of syllables per word. A higher
score indicates that the sentence is simpler to read. FRES grades the text
from 0 to 100. The higher scores indicate the sentences are easier to read. As
usual, the difference of one school grade level in FRES is 10, e.g., 5th grade
(100.00-90.00) and 6th grade (90.0-80.0). The formula of FRES is,

$206.835-1.015\left(\frac{\text{\\# words }}{\text{\\# sentences
}}\right)-84.6\left(\frac{\text{\\# syllables }}{\text{\\# words }}\right)$

(1)

Figure 1: The overview of our approach for training. A pseudo SS corpus is
synthesized by selecting these complex-simple sentence pairs with a higher
complexity difference. Then, we first train neural SS method using the pseudo
SS corpus, and train neural SS method using the real SS corpus.

To ensure simplicity, we only keep the sentence pairs with a FRES difference
higher than a threshold $h_{\textit{FRES}}$. In our experiments, we set
$h_{\textit{FRES}}$ = 10.0, where $h_{\textit{FRES}}=10.0$ means that for each
sentence pair, the simplified version should be at least one school level
simpler than its complex counterpart.

After obtaining pseudo SS corpus, we first initialize neural SS method using
pseudo SS corpus, and train neural SS method on real SS corpus.

(3) Statistics of our choosing paraphrase corpora:

We report the statistics of our choosing paraphrase corpora in Table 2. Here,
we report the statistics of real SS corpus WikiLarge for a comparison.
WikiLarge is the most popular and wildly used SS corpus. Because the SS task
is a paraphrase generation task using easier words, the length of the complex
sentence and the simple sentence are roughly the same, and the size of the
vocabulary in the simple sentence set should be smaller than the complex
sentence set. In contrast to the paraphrase corpora, the length of the complex
sentence in WikiLarge is longer than the simple sentence, because it focuses
on the deletion of content.

Table 2 Statistics of our choosing paraphrase corpora in two strategies
compared with WikiLarge. Avg(complex) and Avg(simple) are the average numbers
of words in the complex sentences and the simpler sentences, respectively.

| WikiLarge | First | Second
---|---|---|---
Vocab(complex) | 169,349 | 282,279 | 96,524
Vocab(simple) | 135,607 | 245,447 | 92,156
Avg(complex) | 21.93 | 12.04 | 10.49
Avg(simple) | 16.14 | 12.65 | 11.31
Total pairs | 296,402 | 2,000,000 | 321,900

##  IV Experinments

###  IV-A Experinmental setup

Neural SS methods: To validate that our two initialization strategies (First
and Second) are effective for different neural SS methods, we apply our two
initialization on the following three methods:

  * •

LSTM that is composed of RNN network and soft attention layer.

  * •

Transformer that is based solely on attention mechanisms.

  * •

Bart111https://dl.fbaipublicfiles.com/fairseq/models/bart.base.tar.gz that is
sequence-to-sequence model trained with denoising as pretraining objective

We implement the above three methods via the opensource toolkit fairseq[49].
We adopt the Adam optimizer with $\beta_{1}=0.9$, $\beta_{2}=0.98$,
$\epsilon=10^{-8}$ and Dropout is set 0.3 for the three methods. The initial
learning rate are set to $1\times 10^{-4}$, $1\times 10^{-4}$, $lr=1\times
10^{-5}$ for LSTM-based, Transformer-based and BART-based models,
respectively.

Evaluation Dataset: We select WikiLarge as the training SS corpus. For
evaluating neural SS methods, we select TurkCorpus [9] as our evaluation
benchmark dataset. The corpus consists of 2000 valid sentences and 359 test
sentences. In TurkCorpus, each complex sentence has 8 kinds of simplification
for reference.

Evaluation Metrics: SARI[9] is the main metric to evaluate text simplification
models, which calculates the arithmetic mean of the $n$-gram F1 scores of
three operations (keeping, adding, and deleting) through comparing the
generated sentences to multiple simplification references and the original
sentences. A Higher SARI score means better simplification performance. We use
SS evaluation tools Easse[50] to calculate the SARI metric.

###  IV-B Experinmental Results

Table 3 The evaluation result of the experiments.

Model |  Condition |  SARI
---|---|---
LSTM |  - |  35.77
First |  36.25
Second |  36.45
Transformer |  - |  37.29
First |  37.64
Second |  38.17
Bart |  - |  38.03
First |  38.29
Second |  38.77

The final evaluation results are shown in Table 3. We can see that the three
neural SS methods (LSTM, Transformer and Bart) with our First initialization
strategy outperform themselves without initialization. The results indicate
that our first initialization strategy is effective for neural SS methods. As
we expected, the Second initialization method with a selector indeed further
improves the performance of neural SS methods. With a selector, our paraphrase
corpus becomes more suitable for SS task. The selector makes SARI score get
0.68 improved for LSTM, 0.88 improved for Transformer, 0.74 improved for Bart-
based compared with themselves without initialization. From the simplified
sentences, we found that Second also improves the readability of
simplification results in varying degrees compared with the First
initialization method without a selector. This indicates that the noise
sentences in the paraphrase really harm the model training through the First
initialization method. We can conclude that Second is a more reasonable and
better method.

Table 4 The examples of simplified results generated by Transformer with the
second initialization.

Complex |  it was originally thought that the debris thrown up by the collision filled in the smaller craters .
---|---
Reference |  it was first thought that the debris thrown up by the collis-ion filled in the smaller craters .
Transformer |  it was originally thought that the debris thrown up by coll-ision filled in the smaller craters .
Second |  it was first thought that the debris thrown up by the collis-ion filled in the smaller craters .
Complex |  both names became defunct in 2007 when they were merged into the national museum of scotland .
Reference |  both names merged with each other in 2007 to becomethe national museum of scotland .
Transformer |  both names became defunct in 2007 when they were merged into the national museum of scotland .
Second |  both names became defunct in 2007 when they were joined into the national museum of scotland .

Table 4 shows the examples of the simplification result generated by
Transformer without initialization and Transformer with our second
initialization method. In the first example, we can find that our proposed
method replaces ’originally’ with ’first’ which is the same as the reference
while Transformer only repeats the original sentence. In the second example,
our method replaces ’merged’ with ’joined’ while Transformer still repeats.
This indicates that our initialization method can make more simplification
with word replacement compared with the baseline method.

##  V Conclusions

Considering the relationship between paraphrase corpus and SS corpus, we
propose two strategies to initialize neural sentence simplification (SS) model
using paraphrase corpus. Experimental results verify that neural SS methods
without our initialization outperform themselves without initialization. In
this paper, we use a small version of the paraphrase corpus. In future work,
we can use a bigger paraphrase corpus. In addition to that, we can also build
a new paraphrase corpus with different kinds of selectors not just FRES
selector.

## References

  * [1] G. H. Paetzold, Lexical Simplification for Non-Native English Speakers.  PhD thesis, University of Sheffield, 2016.
  * [2] J. Qiang, X. Lv, Y. Li, Y. Yuan, and X. Wu, “Chinese lexical simplification,” IEEE Transactions on Audio, Speech and Language Processing., vol. 29, pp. 1819–1828, 2021.
  * [3] L. Rello, R. Baeza-Yates, S. Bott, and H. Saggion, “Simplify or help? text simplification strategies for people with dyslexia,” in Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility, pp. 1–10, 2013.
  * [4] R. Evans, C. Orasan, and I. Dornescu, “An evaluation of syntactic simplification rules for people with autism,” Association for Computational Linguistics, 2014.
  * [5] M. Xia, E. Kochmar, and T. Briscoe, “Text readability assessment for second language learners,” in Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, (San Diego, CA), pp. 12–22, June 2016.
  * [6] W. M. Watanabe, A. C. Junior, V. R. Uzêda, R. P. d. M. Fortes, T. A. S. Pardo, and S. M. Aluísio, “Facilita: reading assistance for low-literacy readers,” in Proceedings of the 27th ACM international conference on Design of communication, pp. 29–36, 2009.
  * [7] Z. Zhu, D. Bernhard, and I. Gurevych, “A monolingual tree-based translation model for sentence simplification,” in Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pp. 1353–1361, 2010.
  * [8] S. Narayan and C. Gardent, “Hybrid simplification using deep semantics and machine translation,” in The 52nd annual meeting of the association for computational linguistics, pp. 435–445, 2014.
  * [9] W. Xu, C. Napoles, E. Pavlick, Q. Chen, and C. Callison-Burch, “Optimizing statistical machine translation for text simplification,” Transactions of the Association for Computational Linguistics, vol. 4, pp. 401–415, 2016\.
  * [10] X. Zhang and M. Lapata, “Sentence simplification with deep reinforcement learning,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, (Copenhagen, Denmark), pp. 584–594, Sept. 2017\.
  * [11] S. Surya, A. Mishra, A. Laha, P. Jain, and K. Sankaranarayanan, “Unsupervised neural text simplification,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, (Florence, Italy), pp. 2058–2068, July 2019.
  * [12] L. Martin, A. Fan, É. de la Clergerie, A. Bordes, and B. Sagot, “Multilingual unsupervised sentence simplification,” arXiv preprint arXiv:2005.00352, 2020.
  * [13] S. Agrawal, W. Xu, and M. Carpuat, “A non-autoregressive edit-based approach to controllable text simplification,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, (Online), pp. 3757–3769, Association for Computational Linguistics, Aug. 2021.
  * [14] W. Xu, C. Callison-Burch, and C. Napoles, “Problems in current text simplification research: New data can help,” Transactions of the Association for Computational Linguistics, vol. 3, pp. 283–297, 2015.
  * [15] S. Stajner, “Automatic text simplification for social good: Progress and challenges,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, (Online), pp. 2637–2652, Association for Computational Linguistics, Aug. 2021.
  * [16] T. Kajiwara and M. Komachi, “Text simplification without simplified corpora,” The Journal of Natural Language Processing, vol. 25, pp. 223–249, 2018\.
  * [17] J. Qiang, K. Liu, Y. Li, Y. Yuan, and Y. Zhu, “Parals: Lexical substitution via pretrained paraphraser,” arXiv preprint arXiv:2305.08146, 2023.
  * [18] J. Wieting and K. Gimpel, “Pushing the limits of paraphrastic sentence embeddings with millions of machine translations,” CoRR, vol. abs/1711.05732, 2017.
  * [19] J. E. Hu, R. Rudinger, M. Post, and B. Van Durme, “Parabank: Monolingual bitext generation and sentential paraphrasing via lexically-constrained neural machine translation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 6521–6528, 2019.
  * [20] S. Jiang, P. Ren, C. Monz, and M. de Rijke, “Improving neural response diversity with frequency-aware cross-entropy loss,” in The World Wide Web Conference, WWW ’19, (New York, NY, USA), p. 2879–2885, Association for Computing Machinery, 2019.
  * [21] S. Gu, J. Zhang, F. Meng, Y. Feng, W. Xie, J. Zhou, and D. Yu, “Token-level adaptive training for neural machine translation,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), (Online), pp. 1035–1046, Association for Computational Linguistics, Nov. 2020.
  * [22] J. P. Kincaid, R. P. Fishburne Jr, R. L. Rogers, and B. S. Chissom, “Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel,” tech. rep., Naval Technical Training Command Millington TN Research Branch, 1975.
  * [23] W. Coster and D. Kauchak, “Simple english wikipedia: a new text simplification task,” in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pp. 665–669, Association for Computational Linguistics, 2011.
  * [24] A. M. Rush, S. Chopra, and J. Weston, “A neural attention model for abstractive sentence summarization,” arXiv preprint arXiv:1509.00685, 2015\.
  * [25] G. Glavaš and S. Štajner, “Simplifying lexical simplification: do we need simplified corpora?,” in Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), vol. 2, pp. 63–68, 2015.
  * [26] J. Qiang, Y. Li, Y. Zhu, Y. Yuan, and X. Wu, “Lexical simplification with pretrained encoders,” pp. 8649–8656, 2020.
  * [27] J. Qiang, S. Zhu, Y. Li, Y. Zhu, Y. Yuan, and X. Wu, “Natural language watermarking via paraphraser-based lexical substitution,” Artificial Intelligence, p. 103859, 2023.
  * [28] J. Qiang, Y. Li, Y. Zhu, Y. Yuan, Y. Shi, and X. Wu, “Lsbert: Lexical simplification based on bert,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3064–3076, 2021.
  * [29] K. Woodsend and M. Lapata, “Learning to simplify sentences with quasi-synchronous grammar and integer programming,” in Proceedings of the conference on empirical methods in natural language processing, pp. 409–420, Association for Computational Linguistics, 2011.
  * [30] S. Wubben, A. V. D. Bosch, and E. Krahmer, “Sentence simplification by monolingual machine translation,” in Proceedings of the conference on empirical methods in natural language processing, pp. 1015–1024, In Proceedings of the 50th ACL, 2012.
  * [31] D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473, 2014\.
  * [32] M. Artetxe, G. Labaka, and E. Agirre, “Unsupervised statistical machine translation,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, (Brussels, Belgium), pp. 3632–3642, Association for Computational Linguistics, Oct.-Nov. 2018.
  * [33] S. Nisioi, S. Štajner, S. P. Ponzetto, and L. P. Dinu, “Exploring neural text simplification models,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), vol. 2, pp. 85–91, 2017.
  * [34] J. Qiang, P. Chen, W. Ding, T. Wang, F. Xie, and X. Wu, “Heterogeneous-length text topic modeling for reader-aware multi-document summarization,” ACM Transactions on Knowledge Discovery from Data (TKDD), vol. 13, no. 4, pp. 1–21, 2019.
  * [35] S. Narayan and C. Gardent, “Unsupervised sentence simplification using deep semantics,” pp. 111–120, 2015.
  * [36] D. Kumar, L. Mou, L. Golab, and O. Vechtomova, “Iterative edit-based unsupervised sentence simplification,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, (Online), pp. 7918–7928, Association for Computational Linguistics, July 2020.
  * [37] S. Surya, A. Mishra, A. Laha, P. Jain, and K. Sankaranarayanan, “Unsupervised neural text simplification,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, (Florence, Italy), pp. 2058–2068, Association for Computational Linguistics, July 2019.
  * [38] A. Katsuta and K. Yamamoto, “Improving text simplification by corpus expansion with unsupervised learning,” in 2019 International Conference on Asian Language Processing (IALP), pp. 216–221, IEEE, 2019.
  * [39] A. Palmero Aprosio, S. Tonelli, M. Turchi, M. Negri, and A. Di Gangi Mattia, “Neural text simplification in low-resource conditions using weak supervision,” in Workshop on Methods for Optimizing and Evaluating Neural Language Generation (NeuralGen), pp. 37–44, Association for Computational Linguistics (ACL), 2019.
  * [40] M. Kusner, Y. Sun, N. Kolkin, and K. Weinberger, “From word embeddings to document distances,” in International conference on machine learning, pp. 957–966, PMLR, 2015.
  * [41] G. Wenzek, M. Lachaux, A. Conneau, V. Chaudhary, F. Guzmán, A. Joulin, and E. Grave, “Ccnet: Extracting high quality monolingual datasets from web crawl data,” CoRR, vol. abs/1911.00359, 2019.
  * [42] X. Lu, J. Qiang, Y. Li, Y. Yuan, and Y. Zhu, “An unsupervised method for building sentence simplification corpora in multiple languages,” in Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 227–237, 2021.
  * [43] R. Sennrich, B. Haddow, and A. Birch, “Improving neural machine translation models with monolingual data,” arXiv preprint arXiv:1511.06709, 2015.
  * [44] J. Wieting, J. Mallinson, and K. Gimpel, “Learning paraphrastic sentence embeddings from back-translated bitext,” arXiv preprint arXiv:1706.01847, 2017.
  * [45] J. Mallinson, R. Sennrich, and M. Lapata, “Paraphrasing revisited with neural machine translation,” in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pp. 881–893, 2017.
  * [46] J. Qiang and X. Wu, “Unsupervised statistical text simplification,” IEEE Transactions on Knowledge and Data Engineering, vol. 33, no. 4, pp. 1802–1806, 2021.
  * [47] R. Sennrich, B. Haddow, and A. Birch, “Improving neural machine translation models with monolingual data,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), (Berlin, Germany), pp. 86–96, Aug. 2016.
  * [48] J. Wieting and K. Gimpel, “ParaNMT-50M: Pushing the limits of paraphrastic sentence embeddings with millions of machine translations,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), (Melbourne, Australia), pp. 451–462, July 2018.
  * [49] M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli, “fairseq: A fast, extensible toolkit for sequence modeling,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), (Minneapolis, Minnesota), pp. 48–53, June 2019.
  * [50] F. Alva-Manchego, L. Martin, C. Scarton, and L. Specia, “EASSE: Easier automatic sentence simplification evaluation,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, (Hong Kong, China), pp. 49–54, Nov. 2019\.


* [176] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: a method for automatic evaluation of machine translation,” in _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, 2002, pp. 311–318.
  * [177] A. Lavie and A. Agarwal, “METEOR: An automatic metric for mt evaluation with improved correlation with human judgments,” in _Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization_. Association for Computational Linguistics, 2007, pp. 65–72.
  * [178] C.-Y. Lin, “ROUGE: A package for automatic evaluation of summaries,” in _Text Summarization Branches Out: Proceedings of the ACL-04 Workshop_ , 2004, pp. 74–81.
  * [179] R. Vedantam, C. L. Zitnick, and D. Parikh, “CIDEr: Consensus-based image description evaluation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2015, pp. 4566–4575.
  * [180] P. Anderson, X. He, C. Buehler, D. Teney, M. Johnson, S. Gould, and L. Zhang, “SPICE: Semantic propositional image caption evaluation,” in _European Conference on Computer Vision (ECCV)_ , 2016, pp. 382–398.
  * [181] K. Pearson, “Note on regression and inheritance in the case of two parents,” _Proceedings of the Royal Society of London_ , vol. 58, no. 347-352, pp. 240–242, 1895.
  * [182] S. Kullback and R. A. Leibler, “On information and sufficiency,” _The Annals of Mathematical Statistics_ , vol. 22, no. 1, pp. 79–86, 1951.
  * [183] J. R. Quinlan, “Induction of decision trees,” _Machine Learning_ , vol. 1, no. 1, pp. 81–106, 1986.
  * [184] A. Stocco, B. Pulfer, and P. Tonella, “Mind the gap! a study on the transferability of virtual vs physical-world testing of autonomous driving systems,” _IEEE Transactions on Software Engineering_ , 2022.
  * [185] C. Zhang, R. Guo, W. Zeng, Y. Xiong, B. Dai, R. Hu, M. Ren, and R. Urtasun, “Rethinking closed-loop training for autonomous driving,” in _European Conference on Computer Vision_. Springer, 2022, pp. 264–282.
  * [186] S. Feng, H. Sun, X. Yan, H. Zhu, Z. Zou, S. Shen, and H. X. Liu, “Dense reinforcement learning for safety validation of autonomous vehicles,” _Nature_ , vol. 615, no. 7953, pp. 620–627, 2023.
  * [187] L. Li, X. Wang, K. Wang, Y. Lin, J. Xin, L. Chen, L. Xu, B. Tian, Y. Ai, J. Wang _et al._ , “Parallel testing of vehicle intelligence via virtual-real interaction,” _Science robotics_ , vol. 4, no. 28, p. eaaw4106, 2019.
  * [188] K. Othman, “Exploring the implications of autonomous vehicles: A comprehensive review,” _Innovative Infrastructure Solutions_ , vol. 7, no. 2, p. 165, 2022.
  * [189] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “Carla: An open urban driving simulator,” in _Conference on Robot Learning_. PMLR, 2017, pp. 1–16.
  * [190] P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y.-P. Flötteröd, R. Hilbrich, L. Lücken, J. Rummel, P. Wagner, and E. Wießner, “Microscopic traffic simulation using sumo,” in _International Conference on Intelligent Transportation Systems_. IEEE, 2018, pp. 2575–2582.
  * [191] L. Wen, D. Fu, S. Mao, P. Cai, M. Dou, and Y. Li, “LimSim: A long-term interactive multi-scenario traffic simulator,” _arXiv preprint arXiv:2307.06648_ , 2023.
  * [192] A. Zador, S. Escola, B. Richards, B. Ölveczky, Y. Bengio, K. Boahen, M. Botvinick, D. Chklovskii, A. Churchland, C. Clopath _et al._ , “Toward next-generation artificial intelligence: Catalyzing the neuroai revolution,” _arXiv preprint arXiv:2210.08340_ , 2022.
  * [193] X. Zhao, Y. Gao, S. Jin, Z. Xu, Z. Liu, W. Fan, and P. Liu, “Development of a cyber-physical-system perspective based simulation platform for optimizing connected automated vehicles dedicated lanes,” _Expert Systems with Applications_ , vol. 213, p. 118972, 2023.
  * [194] E. Leurent, “An environment for autonomous driving decision-making,” https://github.com/eleurent/highway-env, 2018.
  * [195] H. Caesar, J. Kabzan, K. S. Tan, W. K. Fong, E. Wolff, A. Lang, L. Fletcher, O. Beijbom, and S. Omari, “nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles,” _arXiv preprint arXiv:2106.11810_ , 2021.
  * [196] C. Gulino, J. Fu, W. Luo, G. Tucker, E. Bronstein, Y. Lu, J. Harb, X. Pan, Y. Wang, X. Chen, J. D. Co-Reyes, R. Agarwal, R. Roelofs, Y. Lu, N. Montali, P. Mougin, Z. Yang, B. White, A. Faust, R. McAllister, D. Anguelov, and B. Sapp, “Waymax: An accelerated, data-driven simulator for large-scale autonomous driving research,” in _Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks_ , 2023.
  * [197] M. W. Sayers, “Vehicle models for rts applications,” _Vehicle System Dynamics_ , vol. 32, no. 4-5, pp. 421–438, 1999.
  * [198] Hexagon, “Virtual test drive: Complete tool-chain for driving simulation applications,” https://hexagon.com/products/virtual-test-drive.
  * [199] Epic Games, “Unreal engine: The world’s most advanced real-time 3D creation tool for photoreal visuals and immersive experiences.” https://www.unrealengine.com/.
  * [200] Unity Technologies, “Unity engine: Unity’s real-time 3D development engine lets artists, designers, and developers collaborate to create amazing immersive and interactive experiences.” https://unity.com/products/unity-engine/.
  * [201] Q. Li, Z. Peng, L. Feng, Q. Zhang, Z. Xue, and B. Zhou, “MetaDrive: Composing diverse driving scenarios for generalizable reinforcement learning,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 45, no. 3, pp. 3461–3475, 2022.
  * [202] W. Li, C. Pan, R. Zhang, J. Ren, Y. Ma, J. Fang, F. Yan, Q. Geng, X. Huang, H. Gong _et al._ , “AADS: Augmented autonomous driving simulation using data-driven algorithms,” _Science Robotics_ , vol. 4, no. 28, p. eaaw0863, 2019.
  * [203] Z. Yang, Y. Chai, D. Anguelov, Y. Zhou, P. Sun, D. Erhan, S. Rafferty, and H. Kretzschmar, “SurfelGAN: Synthesizing realistic sensor data for autonomous driving,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 11 118–11 127.
  * [204] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “NeRF: Representing scenes as neural radiance fields for view synthesis,” _Communications of the ACM_ , vol. 65, no. 1, pp. 99–106, 2021.
  * [205] Z. Chen, C. Wang, Y.-C. Guo, and S.-H. Zhang, “StructNeRF: Neural radiance fields for indoor scenes with structural hints,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2023.
  * [206] X. Wu, J. Xu, Z. Zhu, H. Bao, Q. Huang, J. Tompkin, and W. Xu, “Scalable neural indoor scene rendering,” _ACM Transactions on Graphics_ , vol. 41, no. 4, 2022.
  * [207] W. Chang, Y. Zhang, and Z. Xiong, “Depth estimation from indoor panoramas with neural scene representation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 899–908.
  * [208] Y. Wei, S. Liu, J. Zhou, and J. Lu, “Depth-guided optimization of neural radiance fields for indoor multi-view stereo,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2023.
  * [209] M. Tancik, V. Casser, X. Yan, S. Pradhan, B. Mildenhall, P. P. Srinivasan, J. T. Barron, and H. Kretzschmar, “Block-NeRF: Scalable large scene neural view synthesis,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 8248–8258.
  * [210] A. Tonderski, C. Lindström, G. Hess, W. Ljungbergh, L. Svensson, and C. Petersson, “NeuRAD: Neural rendering for autonomous driving,” _arXiv preprint arXiv:2311.15260_ , 2023.
  * [211] L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, W. Zhang, B. Cui, and M.-H. Yang, “Diffusion models: A comprehensive survey of methods and applications,” _ACM Computing Surveys_ , 2022.
  * [212] G. Yan, Z. Liu, C. Wang, C. Shi, P. Wei, X. Cai, T. Ma, Z. Liu, Z. Zhong, Y. Liu _et al._ , “OpenCalib: A multi-sensor calibration toolbox for autonomous driving,” _Software Impacts_ , vol. 14, p. 100393, 2022.
  * [213] C. Jiang, A. Cornman, C. Park, B. Sapp, Y. Zhou, D. Anguelov _et al._ , “MotionDiffuser: Controllable multi-agent motion prediction using diffusion,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 9644–9653.
  * [214] Z. Zhong, D. Rempe, D. Xu, Y. Chen, S. Veer, T. Che, B. Ray, and M. Pavone, “Guided conditional diffusion for controllable traffic simulation,” in _IEEE International Conference on Robotics and Automation_. IEEE, 2023, pp. 3560–3566.
  * [215] X. Cai, W. Jiang, R. Xu, W. Zhao, J. Ma, S. Liu, and Y. Li, “Analyzing infrastructure lidar placement with realistic lidar simulation library,” in _2023 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2023, pp. 5581–5587.
  * [216] A. Swerdlow, R. Xu, and B. Zhou, “Street-view image generation from a bird’s-eye view layout,” _arXiv preprint arXiv:2301.04634_ , 2023.
  * [217] K. Yang, E. Ma, J. Peng, Q. Guo, D. Lin, and K. Yu, “BEVControl: Accurately controlling street-view elements with multi-perspective consistency via bev sketch layout,” _arXiv preprint arXiv:2308.01661_ , 2023.
  * [218] R. Gao, K. Chen, E. Xie, L. Hong, Z. Li, D.-Y. Yeung, and Q. Xu, “MagicDrive: Street view generation with diverse 3D geometry control,” _arXiv preprint arXiv:2310.02601_ , 2023.
  * [219] X. Li, Y. Zhang, and X. Ye, “DrivingDiffusion: Layout-guided multi-view driving scene video generation with latent diffusion model,” _arXiv preprint arXiv:2310.07771_ , 2023.
  * [220] J. Lu, Z. Huang, J. Zhang, Z. Yang, and L. Zhang, “WoVoGen: World volume-aware diffusion for controllable multi-camera driving scene generation,” _arXiv preprint arXiv:2312.02934_ , 2023.
  * [221] F. Jia, W. Mao, Y. Liu, Y. Zhao, Y. Wen, C. Zhang, X. Zhang, and T. Wang, “ADriver-I: A general world model for autonomous driving,” _arXiv preprint arXiv:2311.13549_ , 2023.
  * [222] D. Ha and J. Schmidhuber, “World models,” _arXiv preprint arXiv:1803.10122_ , 2018.
  * [223] A. Van Den Oord, O. Vinyals _et al._ , “Neural discrete representation learning,” _Advances in Neural Information Processing Systems_ , vol. 30, 2017.
  * [224] W. Zheng, W. Chen, Y. Huang, B. Zhang, Y. Duan, and J. Lu, “OccWorld: Learning a 3D occupancy world model for autonomous driving,” _arXiv preprint arXiv:2311.16038_ , 2023.
  * [225] Z. Zhang, A. Liniger, D. Dai, F. Yu, and L. Van Gool, “Trafficbots: Towards world models for autonomous driving simulation and motion prediction,” _arXiv preprint arXiv:2303.04116_ , 2023.
  * [226] A. Martino, M. Iannelli, and C. Truong, “Knowledge injection to counter large language model (llm) hallucination,” in _European Semantic Web Conference_. Springer, 2023, pp. 182–185.
  * [227] D. Lenat and G. Marcus, “Getting from generative ai to trustworthy AI: What llms might learn from Cyc,” _arXiv preprint arXiv:2308.04445_ , 2023.
  * [228] G. Agrawal, T. Kumarage, Z. Alghami, and H. Liu, “Can knowledge graphs reduce hallucinations in LLMs?: A survey,” _arXiv preprint arXiv:2311.07914_ , 2023.
  * [229] L. Chen, O. Sinavski, J. Hünermann, A. Karnsund, A. J. Willmott, D. Birch, D. Maund, and J. Shotton, “Driving with llms: Fusing object-level vector modality for explainable autonomous driving,” _arXiv preprint arXiv:2310.01957_ , 2023.
  * [230] R. Pfeifer and F. Iida, “Embodied artificial intelligence: Trends and challenges,” in _Embodied Artificial Intelligence: International Seminar, Dagstuhl Castle, Germany, July 7-11, 2003. Revised Papers_. Springer, 2004, pp. 1–26.
  * [231] L. Smith and M. Gasser, “The development of embodied cognition: Six lessons from babies,” _Artificial life_ , vol. 11, no. 1-2, pp. 13–29, 2005.
  * [232] J. Duan, S. Yu, H. L. Tan, H. Zhu, and C. Tan, “A survey of embodied AI: From simulators to research tasks,” _IEEE Transactions on Emerging Topics in Computational Intelligence_ , vol. 6, no. 2, pp. 230–244, 2022.
  * [233] X. Zhu, Y. Chen, H. Tian, C. Tao, W. Su, C. Yang, G. Huang, B. Li, L. Lu, X. Wang _et al._ , “Ghost in the Minecraft: Generally capable agents for open-world enviroments via large language models with text-based knowledge and memory,” _arXiv preprint arXiv:2305.17144_ , 2023.
  * [234] R. Law, K. J. Lin, H. Ye, and D. K. C. Fong, “Artificial intelligence research in hospitality: a state-of-the-art review and future directions,” _International Journal of Contemporary Hospitality Management_ , 2023.
  * [235] D. Driess, F. Xia, M. S. M. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, W. Huang, Y. Chebotar, P. Sermanet, D. Duckworth, S. Levine, V. Vanhoucke, K. Hausman, M. Toussaint, K. Greff, A. Zeng, I. Mordatch, and P. Florence, “PaLM-E: An embodied multimodal language model,” in _arXiv preprint arXiv:2303.03378_ , 2023.
  * [236] L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin _et al._ , “A survey on large language model based autonomous agents,” _arXiv preprint arXiv:2308.11432_ , 2023.
  * [237] J. A. Oravec, “The future of embodied AI: Containing and mitigating the dark and creepy sides of robotics, autonomous vehicles, and AI,” in _Good Robot, Bad Robot: Dark and Creepy Sides of Robotics, Autonomous Vehicles, and AI_. Springer, 2022, pp. 245–276.
  * [238] A. Keysan, A. Look, E. Kosman, G. Gürsun, J. Wagner, Y. Yu, and B. Rakitsch, “Can you text what is happening? integrating pre-trained language encoders into trajectory prediction models for autonomous driving,” _arXiv preprint arXiv:2309.05282_ , 2023.
  * [239] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of bert: smaller, faster, cheaper and lighter,” _arXiv preprint arXiv:1910.01108_ , 2019.
  * [240] C. Cui, Y. Ma, X. Cao, W. Ye, and Z. Wang, “Drive as you speak: Enabling human-like interaction with large language models in autonomous vehicles,” _arXiv preprint arXiv:2309.10228_ , 2023.
  * [241] H. Sha, Y. Mu, Y. Jiang, L. Chen, C. Xu, P. Luo, S. E. Li, M. Tomizuka, W. Zhan, and M. Ding, “LanguageMPC: Large language models as decision makers for autonomous driving,” _arXiv preprint arXiv:2310.03026_ , 2023.
  * [242] J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” _arXiv preprint arXiv:2301.12597_ , 2023.
  * [243] H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” _arXiv preprint arXiv:2304.08485_ , 2023.
  * [244] S. Zhang, D. Fu, Z. Zhang, B. Yu, and P. Cai, “TrafficGPT: Viewing, processing and interacting with traffic foundation models,” _arXiv preprint arXiv:2309.06719_ , 2023.
  * [245] C. Cui, Y. Ma, X. Cao, W. Ye, and Z. Wang, “Receive, reason, and react: Drive as you say with large language models in autonomous vehicles,” _arXiv preprint arXiv:2310.08034_ , 2023.
  * [246] J. Mao, Y. Qian, H. Zhao, and Y. Wang, “GPT-Driver: Learning to drive with GPT,” _arXiv preprint arXiv:2310.01415_ , 2023.
  * [247] T.-H. Wang, A. Maalouf, W. Xiao, Y. Ban, A. Amini, G. Rosman, S. Karaman, and D. Rus, “Drive anywhere: Generalizable end-to-end autonomous driving with multi-modal foundation models,” _arXiv preprint arXiv:2310.17642_ , 2023.
  * [248] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly _et al._ , “An image is worth 16x16 words: Transformers for image recognition at scale,” _arXiv preprint arXiv:2010.11929_ , 2020.
  * [249] J. Mao, J. Ye, Y. Qian, M. Pavone, and Y. Wang, “A language agent for autonomous driving,” _arXiv preprint arXiv:2311.10813_ , 2023.
  * [250] Anonymous, “3D dense captioning beyond nouns: A middleware for autonomous driving,” in _Submitted to The Twelfth International Conference on Learning Representations_ , 2023, under review. [Online]. Available: https://openreview.net/forum?id=8T7m27VC3S
  * [251] A. Awadalla, I. Gao, J. Gardner, J. Hessel, Y. Hanafy, W. Zhu, K. Marathe, Y. Bitton, S. Gadre, S. Sagawa, J. Jitsev, S. Kornblith, P. W. Koh, G. Ilharco, M. Wortsman, and L. Schmidt, “OpenFlamingo: An open-source framework for training large autoregressive vision-language models,” _arXiv preprint arXiv:2308.01390_ , 2023.
  * [252] X. Jia, Y. Gao, L. Chen, J. Yan, P. L. Liu, and H. Li, “Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2023, pp. 7953–7963.
  * [253] L. Wen, X. Yang, D. Fu, X. Wang, P. Cai, X. Li, T. Ma, Y. Li, L. Xu, D. Shang _et al._ , “On the road with GPT-4V (ision): Early explorations of visual-language model on autonomous driving,” _arXiv preprint arXiv:2311.05332_ , 2023.
  * [254] B. Peng, C. Li, P. He, M. Galley, and J. Gao, “Instruction tuning with GPT-4,” _arXiv preprint arXiv:2304.03277_ , 2023.
  * [255] J. Mai, J. Chen, B. Li, G. Qian, M. Elhoseiny, and B. Ghanem, “LLM as a robotic brain: Unifying egocentric memory and control,” _arXiv preprint arXiv:2304.09349_ , 2023.
  * [256] J. Li, X. Zhang, J. Li, Y. Liu, and J. Wang, “Building and optimization of 3D semantic map based on lidar and camera fusion,” _Neurocomputing_ , vol. 409, pp. 394–407, 2020.
  * [257] J. S. Berrio, M. Shan, S. Worrall, and E. Nebot, “Camera-lidar integration: Probabilistic sensor fusion for semantic mapping,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 23, no. 7, pp. 7637–7652, 2021.
  * [258] C. Premebida and U. Nunes, “Fusing lidar, camera and semantic information: A context-based approach for pedestrian detection,” _The International Journal of Robotics Research_ , vol. 32, no. 3, pp. 371–384, 2013.
  * [259] S. Wang, W. Li, W. Liu, X. Liu, and J. Zhu, “LiDAR2Map: In defense of lidar-based semantic map construction using online camera distillation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 5186–5195.
  * [260] J. de Curtò, I. de Zarzà, and C. T. Calafate, “Semantic scene understanding with large language models on unmanned aerial vehicles,” _Drones_ , vol. 7, no. 2, p. 114, 2023.
  * [261] S. Wu, H. Fei, L. Qu, W. Ji, and T.-S. Chua, “Next-GPT: Any-to-any multimodal llm,” _arXiv preprint arXiv:2309.05519_ , 2023.
  * [262] A. Elhafsi, R. Sinha, C. Agia, E. Schmerling, I. A. Nesnas, and M. Pavone, “Semantic anomaly detection with large language models,” _Autonomous Robots_ , pp. 1–21, 2023.
  * [263] X. Zhou, M. Liu, B. L. Zagar, E. Yurtsever, and A. C. Knoll, “Vision language models in autonomous driving and intelligent transportation systems,” _arXiv preprint arXiv:2310.14414_ , 2023.
  * [264] G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar, “Voyager: An open-ended embodied agent with large language models,” _arXiv preprint arXiv:2305.16291_ , 2023.
  * [265] A. Zhao, D. Huang, Q. Xu, M. Lin, Y.-J. Liu, and G. Huang, “Expel: LLM agents are experiential learners,” _arXiv preprint arXiv:2308.10144_ , 2023.
  * [266] K. Zhang, F. Zhao, Y. Kang, and X. Liu, “Memory-augmented LLM personalization with short-and long-term memory coordination,” _arXiv preprint arXiv:2309.11696_ , 2023.
  * [267] S. Wang, Y. Zhu, Z. Li, Y. Wang, L. Li, and Z. He, “ChatGPT as your vehicle co-pilot: An initial attempt,” _IEEE Transactions on Intelligent Vehicles_ , 2023.
  * [268] N. Shinn, F. Cassano, A. Gopinath, K. R. Narasimhan, and S. Yao, “Reflexion: Language agents with verbal reinforcement learning,” in _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.
  * [269] T. X. Olausson, J. P. Inala, C. Wang, J. Gao, and A. Solar-Lezama, “Demystifying GPT self-repair for code generation,” _arXiv preprint arXiv:2306.09896_ , 2023.
  * [270] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, “3D gaussian splatting for real-time radiance field rendering,” _ACM Transactions on Graphics (ToG)_ , vol. 42, no. 4, pp. 1–14, 2023.
  * [271] OpenAI, “GPT-4V(ision) system card,” https://openai.com/research/gpt-4v-system-card, 2023.
  * [272] F. Sammani, T. Mukherjee, and N. Deligiannis, “NLX-GPT: A model for natural language explanations in vision and vision-language tasks,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 8322–8332.
  * [273] P. Gao, J. Han, R. Zhang, Z. Lin, S. Geng, A. Zhou, W. Zhang, P. Lu, C. He, X. Yue, H. Li, and Y. Qiao, “LLaMA-Adapter V2: Parameter-efficient visual instruction model,” _arXiv preprint arXiv:2304.15010_ , 2023.
  * [274] W. Huang, C. Wang, R. Zhang, Y. Li, J. Wu, and L. Fei-Fei, “VoxPoser: Composable 3D value maps for robotic manipulation with language models,” _arXiv preprint arXiv:2307.05973_ , 2023.
  * [275] H. Ye, T. Liu, A. Zhang, W. Hua, and W. Jia, “Cognitive mirage: A review of hallucinations in large language models,” _arXiv preprint arXiv:2309.06794_ , 2023.
  * [276] V. Rawte, A. Sheth, and A. Das, “A survey of hallucination in large foundation models,” _arXiv preprint arXiv:2309.05922_ , 2023.