11institutetext: Department of Computer Science and Engineering, Shahjalal
University of Science and Technology
Kumargaon, Sylhet 3114, Bangladesh
11email<EMAIL_ADDRESS>22institutetext: 22email<EMAIL_ADDRESS>
# A Subword Level Language Model for Bangla Language

Aisha Khatun 11 0000-0001-6559-4493 Anisur Rahman 11 0000-0002-4616-4559
Hemayet Ahmed Chowdhury 11 0000-0002-5582-669X Md. Saiful Islam 11
0000-0001-9236-380X Ayesha Tasnim 1122 0000-0002-7143-3255

###### Abstract

Language models are at the core of natural language processing. The ability to
represent natural language gives rise to its applications in numerous NLP
tasks including text classification, summarization, and translation. Research
in this area is very limited in Bangla due to the scarcity of resources,
except for some count-based models and very recent neural language models
being proposed, which are all based on words and limited in practical tasks
due to their high perplexity. This paper attempts to approach this issue of
perplexity and proposes a subword level neural language model with the AWD-
LSTM architecture and various other techniques suitable for training in Bangla
language. The model is trained on a corpus of Bangla newspaper articles of an
appreciable size consisting of more than 28.5 million word tokens. The
performance comparison with various other models depicts the significant
reduction in perplexity the proposed model provides, reaching as low as 39.84,
in just 20 epochs.

###### Keywords:

Language Model Bangla AWD-LSTM Continuous-Space Language Model Neural Language
Model(NLM) Deep Learning Subword.

##  1 Introduction

There are generally two types of language models, continuous-space[18] and
count-based[19] language models. Count-based models calculate the n-gram
probabilities to make n-th order Markov assumptions by counting the frequency
of n-gram occurrences, followed by various smoothing techniques. Most of the
work in Bangla[8] is based on this approach. But n-gram models are extremely
sparse. Besides, this method of using n-grams to represent models is
linguistically uninformed, because it has to rely on exact matches of patterns
of tokens. Furthermore, the language model is limited to a specific window,
whereas humans can exploit a much larger context much easily.

Even though language models are at the heart of NLP, the newest techniques of
language modeling have not yet been adapted for Bangla. Specifically in the
reign of the continuous-space language model, very little has been done for
Bangla language. Feed-forward neural language models [18] and recurrent neural
language models (RNNs) have been implemented to solve the sparsity problems of
n-gram models. Here words are represented as vectors of continuous values
called word embeddings and these vector representations are fed into the
network for training. These vectors maintain the property that semantically
close words are are closer in the vector space they are in. Neural Language
models have also been used in various other levels of embedding including
sentence, corpus, and subword.

In this paper, we use a variant of the recurrent neural language model,
Average-Stochastic-Gradient-Descent Weight-Dropped LSTM[13], on subword
tokenized Bangla text, which was tokenized using Googles Sentencepiece
tokenizer. We used the unigram subword tokenization[11] to tokenize the
corpus. This process of tokenization helps provide structural information
about the language. Especially for a language with inflection[4], such as
Bangla. We also present a structure that includes various techniques to
optimize the training of the language model, producing significantly low
perplexities on data sets.

##  2 Related Works

###  2.1 On Bangla Language

Bangla is the seventh most spoken language by the total number of speakers in
the world, yet lacks standard resources. Works on modern language modeling are
next to existent. Most published works are those of count-based models[8],
which have been far outperformed by modern neural networks. Some considerable
work was done in [7]. Besides this, a comparative analysis was performed for
word embeddings [1]. But the need for more improved language models remains.
Notably, no language model was trained keeping in mind the level of inflection
that occurs in Bangla language, thus tokenizing in subword levels may offer
greatly towards downstream tasks such as text classification, summarization,
and translation.

###  2.2 On Language Models

Language modeling can be defined as the conditional probability of observing a
sequence of words and predicting the next word in the sequence. Traditionally
this was accomplished using word-count based methods, which are vastly
replaced by neural language models at present. A description of count-based
and neural language models is presented below.

####  2.2.1 Count-based language models

N-gram language models are the basic type of count-based methods based on
Markov assumption. This model states that the probability of occurrence of a
word depends on the last n-1 words that occur before it. A sentence or a
paragraph is predicted word by word in this manner. The Markov chain rule
states that :

$p(w_{n}\mid w_{1},w_{2},\ldots,w_{n-1})\approx p(w_{n}\mid
w_{n-m},\ldots,w_{n-2},w_{n-1})$ (1)

Here, ‘m‘ is the number of context words. The number of previous words/history
is the order of the model. The basic idea is to predict probability of $w_{n}$
from its preceding context. If the context contains only $w_{n-1}$, it is bi-
gram, and the result is obtained by dividing the frequency of $w_{n-1},w_{n}$
by frequency of $w_{n-1}$. Uni-gram on the other hand considers only the
frequency of $w_{n}$. Following a similar pattern, a tri-gram model would be
as follows:

$p(w_{3}\mid
w_{1},w_{2})=\frac{count(w_{1},w_{2},w_{2})}{\sum{}_{w}count(w_{1},w_{2},w)}$
(2)

Although a simple concept, n-gram models have several drawbacks. First, out-
of-sample combinations are given zero probability. This is called the sparsity
problem and tends to occur very frequently. An attempt to combat this is to
use back-off [10] and smoothing techniques [3]. Second is the curse of
dimensionality. Due to the enormous number of possible word combinations, it
becomes increasingly unfeasible to train on a larger number of context words.

Besides this, the semantic and syntactic similarity of sequences is not
recognized by such models, making it unable to ‘learn‘ language. True Markov
assumption is also not modeled due to the fixed length of context words taken
into consideration.

####  2.2.2 Continuous-space language models

Continuous-space language model, also called neural language model(NLM), is
the solution to sparsity and high dimensionality problems mentioned
previously. These models can represent distributed values and thus learn
syntactic and semantic features of the language, which may be otherwise
impossible to extract.

Recent works exploit NLM to achieve state-of-the-art performance using
recurrent neural networks or its variations such as LSTM, along with other
optimization and regularization techniques. Neural language models can be of
various architectural backgrounds as we see below:

  * •

Feed-Forward Language Models:

Feed-Forward models learn the probability of the next word for the previous
n-1 words with the help of a few feed-forward layers. An architecture of this
form contains a mapping from each word of the vocabulary V, to a continuous
vector space $\in R^{m}$, where m is the feature dimension. Therefore the
mapping C is a matrix of size $\mid V\mid\times m$, each row representing a
feature vector of a word. Then a composite function is formed with mappings
and a function for calculating conditional probability of $w_{t}$ with respect
to $(w_{t-n+1},\ldots,w_{t-1})$, which learns the features and parameters. It
can generalize better than the n-gram models and solves the sparsity problems,
however, long training and testing times pose a considerable threat to the
scalability of such methods.

Two models were proposed to combat this issue. One[17] works on clustering
similar words for calculation reduction by building a binary hierarchical tree
on the vocabulary words using expert knowledge, while another[16] uses data-
driven methods to do the same. 11.1% perplexity reduction was achieved from
the best HLBL model[16] comparing with the Kneser-Ney smoothed 5-gram LM.

  * •

Recurrent Neural Language Models:

In recurrent neural network based model(RNNLM)[14], there is no limitation to
the context size, so the information can keep circulating within the network
for as long as needed. These models provide better generalization by the
recurrently connected neurons, also called short term memory.

An improvement over RNNLM proposed in [15] implements factorization on the
output layer using classes, thus reducing the computational complexity of
RNNLM. This version is faster to train and test, and also performs better than
the original RNNLM.

###  2.3 On Neural Network Architectures

Recently numerous models have been proposed with more advanced structures and
strategies to improve on language modeling. Although most provide good
performance, AWD-LSTM[13], with its specialized optimization and
regularization techniques has been outperforming others in most cases. AWD-
LSTM stands for ASGD(Average Stochastic Gradient Descent) Weight-Dropped LSTM.
It uses DropConnect[21] as a means of regularization, a variant of Average-SGD
(NT-ASGD) and some other techniques in a very effective way.

LSTM follows a method to utilize short and long term memory of context texts,
which can be represented using these generic equations:

$i_{t}=\sigma(W^{i}x_{t}+U^{i}h_{t-1})$ (3)
$f_{t}=\sigma(W^{f}x_{t}+U^{f}h_{t-1})$ (4)
$o_{t}=\sigma(W^{o}x_{t}+U^{o}h_{t-1})$ (5)
$\widetilde{c}_{t}=tanh(W^{c}x_{t}+U^{c}h_{t-1})$ (6)
$c_{t}=i_{t}\odot\widetilde{c}_{t}+f_{t}\odot+\widetilde{c}_{t-1}$ (7)
$h_{t}=o_{t}\odot tanh(c_{t})$ (8)

Here, $W^{i},W^{f},W^{o},W^{c},U^{i},U^{f},U^{o},U^{c}$ represents weight
matrices, $x_{t}$ is the t-th time step vector input, $h_{t}$ is the present
hidden state, $c_{t}$ is the memory cell state, and $\odot$ is element-wise
multiplication.
The DropConnect comes in the scene to eliminate the conventional over-fitting
caused by RNN/LSTM layers. It is applied on the hidden-to-hidden activations,
$(U^{i},U^{f},U^{o},U^{c})$, randomly. This prevents over-fitting, still
maintaining RNNs long-term dependency retention. Figure 1 provides an
illustration of the DropConnect network.

Figure 1: DropConnect Network[21]

Unlike SGD, ASGD takes into account previous iterations in updates and returns
an average value. AWD-LSTM uses a variant of ASGD called NT-ASGD(non-
monotonically triggered ASGD), where ASGD is activated if the validation
metric does not improve for a fixed number of cycles.
Several other methods for effective learning includes variation in
backpropagation-through-time length, variational dropout[5], and embedding
Dropout[5]. These techniques improve the overall structure of the network,
making it easier to train a language model effectively, with significantly
small model size, and therefore is the first choice of architecture.

###  2.4 On Training Neural Networks

Various methods and techniques can be employed while training a model to make
it faster, more effective and perform better. When these techniques meet the
right architecture, state-of-the-art performances can be achieved. Some of
such methods used in this paper are described below.

Learning rates are an important and yet tedious hyper-parameter when training
a model. Finding the right learning rate is extremely crucial, but has been
solved over the years only through time expensive trial and error methods. An
impressive approach by Leslie Smith[20] addresses this problem by performing a
single trial run over the data. Training starts with a small learning rate and
exponential increase occurs per batch. The loss is recorded for every point.
The optimum learning rate is the highest one with a descending error.

Another training problem is local minima. Neural networks may often move
towards the local minima through gradient descent, instead of global minima,
and get stuck. According to [12], if the learning rate is abruptly increased,
the gradient descent will be able to jump out of local minima and start
looking for the global minima again. Experiments from [12] prove the
effectiveness of this approach.

##  3 Corpus

A large corpus on Bangla newspaper articles was created using a custom web
crawler containing 12 different topics. This corpus is relatively large
compared to the ones used for language modeling in the past for Bangla[8]. The
total number of word tokens in this dataset is 28533646 (28.5+ million), with
unique word count of 836509. The number of unique words is around 3% of the
entire vocabulary of the dataset. A summarized statistics of the dataset is
given in Table 1. The Dataset is imbalanced, which is apparent from Figure 2,
for the train set. 20% of the dataset was separated as a held-out dataset for
testing purposes.

Table 1: Dataset Statistics category | samples | total word | unique word
---|---|---|---
opinion | 8098 | 4185472 | 243968
international | 5155 | 1089780 | 86852
economics | 3449 | 909648 | 58932
art | 2665 | 1312571 | 154869
science | 2906 | 697899 | 76755
politics | 20050 | 6167418 | 196541
crime | 8655 | 2016342 | 128308
education | 12212 | 3963695 | 225348
sports | 11903 | 3087029 | 174677
accident | 6328 | 1086791 | 77171
environment | 4313 | 1347509 | 103783
entertainment | 10121 | 2669492 | 204902
average | 7988 | 2377803 | 144342
total | 95855 | 28533646 | 836509
Figure 2: Distribution of classes in the dataset

##  4 Methodology

###  4.1 Proposed Architecture

This paper employs an architecture called AWD-LSTM[13] for language modeling.
It is a variant of Recurrent Neural Network with LSTM gates[6] along with
special regularization techniques and optimization which allows for context
generalization and language understanding. The LSTM layers allow for the
memory of the context of a sentence but are prone to over-fitting. AWD-LSTM
employs dropconnect, among other techniques making it the choice of interest
for language modeling for a language such as Bangla.

This architecture consists of an embedding layer of size 400, followed by 3
regular LSTM layers with no attention. Additionally, it has some short-cut
connections and numerous drop-out hyper-parameters. Each layer has 1150
activation neurons. At last a softmax layer provides probabilities for the
next word. Adam optimizer is used along with flattened categorical cross-
entropy loss function. The architecture is illustrated in figure 3.

Figure 3: Language Model Architecture

###  4.2 Subword Tokenization

In traditional training, words are the tokens of the model with a fixed
vocabulary, where each token is represented by an integer. However, while
mapping words to integers we lose information about word structure, therefore
the language model requires more time and data to learn about languages with
inflections such as Bangla. To help the model learn at a deeper level, one
possibility is the use of characters as tokens. But this approach produces
much larger models and increases the computational costs many-fold, yet not
retaining as much information as required in the LSTM layers[2]. So we opt for
a method in the middle, and perform subword tokenization of the text using
SentencePiece tokenizer. This tokenizer performs unsupervised tokenization
directly from raw sentences irrespective of the language. The unigram
segmentation algorithm[11] was employed to create the subword vocabulary.

###  4.3 Training the language model

The SentencePiece tokenizer was set to a vocabulary size of 30,000[4], which
included <s>,</s>,<unk>tokens as the start token, end token, and the unknown
token. All tokens appearing less than 3 times were discarded and replaced with
the <unk>token. The language model was trained with a batch size of 32, back-
propagation-through-time window was set to 70. A dropout multiplier of 0.5 was
used for all the LSTM layers for regularization. Weight decay of 0.1 was used.

The layers were unfrozen, and weights were randomly initialized. Using the
cyclical learning rate finder technique[20], a learning rate of 1e-3 was
selected and the model was trained for 20 epochs. Stochastic gradient descent
with restarts (SGDR) mentioned in Section 2 was applied to each epoch. Given a
phrase, the language model at this point was capable of completing entire
paragraphs in Bangla language as illustrated in figure 4.

Figure 4: Example Predictions from the Language Model

##  5 Experiments

We attempt to run some experiments for the sake of comparison, recreating from
previously proposed models and evaluate their perplexities. Perplexity is a
common measure of language model performance. It calculates how well a model
predicts the next possible token, given a context of any length. In this case,
perplexity is a way to determine the uncertainty in text generation through
probability assignment. Low perplexity is a sign that the model is performing
well, whereas high perplexity means the language model was not able to capture
the language well. In simple words, perplexity can be calculated as the
exponentiation of the loss. The models used for analysis are:

###  5.1 Bi-gram language model

We compare our model with the bi-gram language model following [9], which has
been frequently used for language modeling in Bangla with variations. Let
$q_{t}=freq(w_{t-1},w_{t-2})$ be the representation of the frequency of
occurrence of a specific window of words $(w_{t-1},w_{t-2})$. Therefore the
conditional probability takes the following form:

$P(w_{t}|w_{t-1})=\alpha_{0}(q_{t})p_{0}+\alpha_{1}(q_{t})p_{1}(w_{t})+\alpha_{2}(q_{t})p_{2}(w_{t}|w_{t-1})$
(9)

With conditional weights $\alpha_{i}(q_{t})\geq
0,\sum{}_{i}\alpha_{i}(q_{t})=1$. Prediction is done using various window
size, including the following ways: $p_{0}=1/|V|,p_{1}$ is a uni-gram and
$p_{2}(i|j)$ is the bi-gram. Rest of the experimental measures are kept
similar to [8].

###  5.2 LSTM and CNN language models

In order to compare the proposed architecture with its modifications against
simpler plain models, we conduct experiments with a few other continuous space
neural models. We build a simple LSTM model with two layers that are trained
for 10 epochs. Each layer has 200 hidden units and all hidden states are
initialized by 0. We train with a batch size of 32, and an initial learning
rate of 1 is used. Besides this, we also try a character level CNN model,
similar to the classification model of [22]. The character set is of size 85
containing all Bangla letters, digits among others. The model consists of 2
convolutional layers followed by a fully connected layer. The length of the
input sequence is set as 1000 characters, other parameters are kept consistent
with [22].

###  5.3 AWD-LSTM word-level language model

For a much closer comparison, a strand of research by Chowdhury et al, word-
level language model using AWD-LSTM architecture was also compared. This model
was fed with text by only separating the words as tokens. Vocabulary size of
60,000 was used and all words with frequency less than 3 were eliminated and
replaced with <unk>. The model was trained for 4 epochs with a learning rate
of 1e-2, and then 3 more epochs with a learning rate of 1e-3 after which the
model started to overfit, so training was stopped. This model yielded a
perplexity of 51.2 at the end of training on the held-out test dataset.

##  6 Results and Discussion

The held-out test set is evaluated and the perplexities are recorded in the
Table 2.

Table 2: Perplexity Comparison of the Language Models Model | Perplexity on Test Set
---|---
AWD-LSTM subword(proposed) | 39.84
AWD-LSTM word | 51.2
Simple LSTM | 227
Character CNN | 125
Bi-gram | 860.1

From the table, it is clear that the subword level model significantly
outperformed all other models. The simple LSTM model gained a perplexity of
227 while using AWD-LSTM architecture lowers it up to 39.84. Character level
CNN improves the perplexity from simple LSTM model but it is still
outperformed by the proposed model significantly. The bi-gram model did not
work well at all, which shows the need for using a neural language model.
Furthermore, the inflection of Bangla language was leveraged by SentencePiece
tokenization providing a subword text structure, which when input to the model
improves the model performance from 51.2 to 39.84 perplexity.

##  7 Conclusion

In this paper, we discuss the ways to effectively train a weight-dropped LSTM
language model in Bangla using subword tokenization and other strategies. We
showed that the proposed subword level model outperforms the word-level models
with a perplexity of 39.84, showing the effectiveness of such tokenization.
This is a significantly low perplexity achieved in Bangla language so far. The
reasons for such results are we believe the architectural makeup of AWD-LSTM
using DropConnect mask on the hidden-to-hidden weight matrices as a technique
of regularization, which is a common problem in language modeling. Besides
this, the structure of Bangla language makes it difficult to perform modeling
tasks, which we attempted to solve using subword tokenization. By using
subwords instead of words, not only does it bring together the same words with
various extensions, but it also significantly reduces the vocabulary size
required to train the model to a similar level of accuracy. Language models
are being used widely for transfer learning in NLP and a well-trained model
that understands the language in question will be effectively helpful in
downstream tasks such as text summarization, classification, and translation.

##  8 Acknowledgement

This paper is a grateful recipient of the facilities and research environment
of the Department of Computer Science and Engineering, Shahjalal University of
Science and Technology (SUST) and SUST NLP Research Group. All authors of this
paper deserve equal credit providing major contributions to the research.

## References

  * [1] Ahmed Chowdhury, H., Imon, M., Saiful Islam, M.: A comparative analysis of word embedding representations in authorship attribution of bengali literature. 21st International Conference of Computer and Information Technology (ICCIT) (2018)
  * [2] Bojanowski, P., Joulin, A., Mikolov, T.: Alternative structures for character-level rnns. arXiv preprint arXiv:1511.06303 (2015)
  * [3] Chen, S.F., Goodman, J.: An empirical study of smoothing techniques for language modeling. Computer Speech & Language (1999)
  * [4] Czapla, P., Gugger, S., Howard, J., Kardas, M.: Universal language model fine-tuning for polish hate speech detection. Proceedings ofthePolEval2019Workshop p. 149 (2019)
  * [5] Gal, Y., Ghahramani, Z.: A theoretically grounded application of dropout in recurrent neural networks. Advances in neural information processing systems (2016)
  * [6] Gers, F.A., Schmidhuber, J., Cummins, F.A.: Learning to forget: Continual prediction with lstm. Neural Computation (2000)
  * [7] Haque, M., Habib, M., Rahman, M.: Automated word prediction in bangla language using stochastic language models. International Journal in Foundations of Computer Science & Technology (2015)
  * [8] Ismail, S., Rahman, M.S.: Bangla word clustering based on n-gram language model. 2014 International Conference on Electrical Engineering and Information Communication Technology (2014)
  * [9] Jelinek, F., Mercer, R.L.: Interpolated estimation of Markov source parameters from sparse data. Proceedings, Workshop on Pattern Recognition in Practice (1980)
  * [10] Kneser, R., Ney, H.: Improved backing-off for m-gram language modeling. 1995 International Conference on Acoustics, Speech, and Signal Processing (1995)
  * [11] Kudo, T.: Subword regularization: Improving neural network translation models with multiple subword candidates (2018)
  * [12] Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with restarts. CoRR (2016)
  * [13] Merity, S., Keskar, N.S., Socher, R.: Regularizing and optimizing LSTM language models. CoRR (2017)
  * [14] Mikolov, T., Karafiát, M., Burget, L., Černockỳ, J., Khudanpur, S.: Recurrent neural network based language model. Eleventh annual conference of the international speech communication association (2010)
  * [15] Mikolov, T., Kombrink, S., Burget, L., Černockỳ, J., Khudanpur, S.: Extensions of recurrent neural network language model. 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2011)
  * [16] Mnih, A., Hinton, G.E.: A scalable hierarchical distributed language model. Advances in neural information processing systems (2009)
  * [17] Morin, F., Bengio, Y.: Hierarchical probabilistic neural network language model. Aistats (2005)
  * [18] Schwenk, H., Dchelotte, D., Gauvain, J.L.: Continuous space language models for statistical machine translation. Proceedings of the COLING/ACL on Main Conference Poster Sessions (2006)
  * [19] Siivola, V., Pellom, B.L.: Growing an n-gram language model. INTERSPEECH (2005)
  * [20] Smith, L.N.: No more pesky learning rate guessing games. CoRR (2015)
  * [21] Wan, L., Zeiler, M., Zhang, S., Le Cun, Y., Fergus, R.: Regularization of neural networks using dropconnect. International conference on machine learning (2013)
  * [22] Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification (2015)


# Ultra-short pulse generation from mid-IR to THz range using plasma wakes and
relativistic ionization fronts

Zan Nie<EMAIL_ADDRESS>Yipeng Wu  Chaojie Zhang  Electrical and Computer
Engineering Department, University of California Los Angeles, Los Angeles,
California 90095, USA  Warren B. Mori  Electrical and Computer Engineering
Department, University of California Los Angeles, Los Angeles, California
90095, USA  Department of Physics, University of California Los Angeles, Los
Angeles, California 90095, USA  Chan Joshi  Electrical and Computer
Engineering Department, University of California Los Angeles, Los Angeles,
California 90095, USA  Wei Lu  Chih-Hao Pai  Jianfei Hua  Department of
Engineering Physics, Tsinghua University, Beijing, 100084, China  Jyhpyng Wang
Department of Physics, National Central University, Jhongli 32001, Taiwan
Institute of Atomic and Molecular Sciences, Academia Sinica, Taipei 10617,
Taiwan  Department of Physics, National Taiwan University, Taipei 10617,
Taiwan

###### Abstract

This paper discusses numerical and experimental results on frequency
downshifting and upshifting of a 10 $\mu$m infrared laser to cover the entire
wavelength (frequency) range from $\lambda$=1-150 $\mu$m ($\nu$=300-2 THz)
using two different plasma techniques. The first plasma technique utilizes
frequency downshifting of the drive laser pulse in a nonlinear plasma wake.
Based on this technique, we have proposed and demonstrated that in a tailored
plasma structure multi-millijoule energy, single-cycle, long-wavelength IR
(3-20 $\mu$m) pulses can be generated by using an 810 nm Ti:sapphire drive
laser. Here we extend this idea to the THz frequency regime. We show that sub-
joule, terawatts, single-cycle terahertz (2-12 THz, or 150-25 $\mu$m) pulses
can be generated by replacing the drive laser with a picosecond 10 $\mu$m CO2
laser and a different shaped plasma structure. The second plasma technique
employs frequency upshifting by colliding a CO2 laser with a rather sharp
relativistic ionization front created by ionization of a gas in less than half
cycle (17 fs) of the CO2 laser. Even though the electrons in the ionization
front carry no energy, the frequency of the CO2 laser can be upshifted due to
the relativistic Doppler effect as the CO2 laser pulse enters the front. The
wavelength can be tuned from 1-10 $\mu$m by simply changing the electron
density of the front. While the upshifted light with $5<\lambda(\mu$m$)<10$
propagates in the forward direction, that with $1<\lambda(\mu$m$)<5$ is back-
reflected. These two plasma techniques seem extremely promising for covering
the entire molecular fingerprint region.

##  I Introduction

Ultra-short radiation source in the mid-infrared (mid-IR) to terahertz (THz)
range (1–150 $\mu$m or 300–2 THz) is highly desirable in numerous physics,
material science, and biology applications [1, 2, 3, 4, 5, 6, 7, 8, 9]. These
ultra-short mid-IR/THz sources are ideal tools for pump-probe experiments in
the “molecular fingerprint” region [1, 2], ultrafast X-ray high harmonic
generation (HHG)[3], coherent IR spectroscopy [10, 11], time-resolved imaging
of molecular structures [12], and nanotip photoemission [13]. Furthermore,
intense single-cycle mid-IR/THz pulses are very useful in attosecond science
applications, for instance, sub-femtosecond control and metrology of bound-
electron dynamics in atoms [8], generation of attosecond [14] or even
zeptosecond pulses [15], and coherent control of lattice displacements through
nonlinear phononics [2]. Remarkable progress for generating such mid-IR/THz
pulses has been made through various optical methods in nonlinear crystals or
gases, such as optical parametric amplification (OPA) [16, 17], difference-
frequency generation (DFG) [18, 19, 20, 21], optical rectification [22, 23],
and two-color filamentation [24, 25]. Recent numerical and experimental
studies show that plasmas can also be utilized as nonlinear optical media to
generate ultra-short mid-IR/THz pulses with high efficiency [26, 27]. Unlike
in traditional nonlinear crystals, there is no damage concerns for the
power/intensity scaling in plasmas, which makes it possible to generate
relativistically intense pulses in the mid-IR to THz spectral range.

In this paper we discuss numerical and experimental results on two different
plasma techniques for frequency downshifting and upshifting of an IR laser to
cover the entire mid-IR to THz band that covers wavelengths from 1-150 $\mu$m
(or frequencies from 300-2 THz). In Sec. II we briefly describe the first
plasma technique that uses frequency downshifting (or photon deceleration) in
a nonlinear plasma wake. We first summarize our particle-in-cell (PIC)
simulations and experiments on generating multi-millijoule, single-cycle
pulses in the 3-20 $\mu$m wavelength range with a specially prepared plasma
density structure [26, 27]. In Sec. III we extend this frequency downshifting
scheme from LWIR to THz spectral range by replacing the drive laser with a
picosecond 10 $\mu$m CO2 laser. In this case we have to use a different
profile of the plasma density structure. By PIC simulations we show that sub-
joule, terawatts, single-cycle THz pulses can be produced using a picosecond
10 $\mu$m CO2 drive laser. In Sec. IV we introduce the second plasma technique
that frequency upshifts the incident IR laser pulse using a relativistic
ionization front and show corresponding numerical results. In this scheme, a
long-wavelength (e.g. 10 $\mu$m CO2 laser) pulse collides with a relativistic
ionization front produced by photoionization of a column of gas using a short-
wavelength (e.g. 800 nm Ti:sapphire laser) pulse. The frequency change of the
CO2 laser pulse and the direction in which the upshifted radiation is observed
is obtained by the double relativistic Doppler shift formulism using Lorentz
transformations. It is found that the frequency can be tuned by simply tuning
the gas and hence the plasma density.

##  II Extreme Frequency downshifting in a nonlinear plasma wake (mid-IR to
LWIR range)

It is known that an ultra-short intense laser pulse can excite a nonlinear
wake (density disturbance) while propagating through an underdense
($\omega_{p}<\omega_{0}$) plasma. Here $\omega_{p}=(4\pi
n_{p}e^{2}/m)^{\frac{1}{2}}$ is the plasma frequency, $n_{p}$ is the plasma
density, $m$ is the electron mass, $e$ is the electron charge, and
$\omega_{0}$ is the laser frequency. If the laser’s normalized vector
potential $a_{0}=eE/m\omega_{0}c>2$ ($E$ is the electric field of the laser
pulse and $c$ is the speed of light) and the pulse duration
$\tau<\sqrt{2\pi/\omega_{p}}$, the ponderomotive force of the laser pulse
eventually pushes out all the plasma electrons forward and outward, forming a
3D nonlinear wake [28]. According to 1D nonlinear theory, the refractive index
seen by the co-propagating laser photons varies as [29, 30]: $\eta\simeq
1-\frac{\omega_{p}^{2}}{2\omega_{0}^{2}}\frac{1}{1+\phi}$, where
$\phi=|e|\Phi/mc^{2}$ is the normalized scalar potential. The refractive index
gradient $\frac{\partial\eta}{\partial\zeta}$ with $\zeta=t-z/c$ being the
variable in the speed of light frame continuously alters the photon frequency
via self-phase modulation (SPM) such that the instantaneous frequency is given
by $\omega(t)=\omega_{0}-\omega_{0}\int{\frac{\partial\eta}{\partial\zeta}dt}$
(or $\frac{1}{\omega}\frac{\partial\omega}{\partial
t}=-\frac{\partial\eta}{\partial\zeta}$). Specifically, photons in the front
of the wake where $\frac{\partial\eta}{\partial\zeta}$ is positive are
frequency-downshifted, while photons in the tail of the wake where
$\frac{\partial\eta}{\partial\zeta}$ is negative are frequency-upshifted, and
the part in the central, near electron-free region experiences scarcely any
frequency change (Fig. 1). Due to the negative group velocity dispersion (GVD)
of the plasma, the longer wavelength photons generated by a positive
refractive index gradient travel with a smaller group velocity
($v_{g}(\omega)\simeq c[1-\frac{\omega_{p}^{2}}{2\omega^{2}}]$) than the
shorter wavelength photons. This is the reason why this phenomenon is also
called photon acceleration/deceleration [29, 30, 31, 32, 33]. This mechanism
has been used for pulse compression [34, 35, 36] or as a diagnostic for the
generation of wakes in plasmas [37, 38, 39].

When the laser pulse duration is roughly one plasma wavelength
($c\tau\simeq\lambda_{\text{p}}$, Fig. 1(a)), laser photons are frequency
downshifted at the front and upshifted at the back of the pulse just as in
usual SPM in fibers or neutral gases. However, when the laser pulse duration
is much less than one plasma wavelength ($c\tau\ll\lambda_{\text{p}}$, Fig.
1(b)), the whole pulse resides at the very front of the wake and experiences
only frequency downshifting (photon deceleration). This is so-called
asymmetric SPM. Using this asymmetric SPM concept, the possibility of
frequency downshifting near-IR drive pulses to mid-IR range in uniform-density
plasmas was explored numerically [40, 41] and experimentally [42, 43, 44] in
the last decade.

Figure 1:  Comparison of frequency downshifting for two different pulse
durations. The plasma density is $4\times 10^{18}\,\text{cm}^{-3}$. The
normalized vector potential is 3. The pulse duration is 40.0 fs in (a) and 9.6
fs in (b). The blue line, orange line, and dashed black line show the laser
electric field, the refractive index gradient, and the refractive index,
respectively.

Recently, we have extended this concept to the LWIR range by using a tailored
plasma structure [26, 27]. Using plasma theory, we found that there is an
optimal pulse duration for efficient frequency downshifting[26] expressed as
$c\tau\simeq\frac{0.52\lambda_{\text{p}}}{a_{0}}$, where $\lambda_{\text{p}}$
is the plasma wavelength. In such an optimal case, the longitudinal profile of
the laser pulse roughly overlaps with the longitudinal profile of the
refractive index gradient so that the whole pulse experiences frequency
downshifting with the optimum efficiency (Fig. 1(b)). For example, if
$a_{0}=3$, $n_{\text{p}}=4\times 10^{18}\,\text{cm}^{-3}$, then the optimal
pulse duration is 9.6 fs, which is the case shown in Fig. 1(b). Such short
($\sim$10 fs) pulses with high intensity ($a_{0}\sim 3$) are very difficult to
obtain even with the state-of-the-art techniques. To tackle this issue, we put
forward a general solution that uses a tailored plasma structure that first
compresses and then frequency downshifts commonly available longer ($\sim$30
fs) Ti:sapphire laser pulses to the LWIR range [26]. The plasma structure
consists of two sections: a relatively low-density but longer uniform density
region connected to a higher density but shorter density region by a sharp
density up-ramp. The 3D OSIRIS [45, 46] simulation showed that the originally
30 fs drive pulse is first compressed to $\sim$10 fs (the optimal pulse
duration for following frequency downshifting) in the first section, and then
experiences rapid and efficient frequency downshifting in the second section.
Using such a tailored plasma structure, multi-millijoule, near-single-cycle,
relativistic pulses in the LWIR region are generated [26]. The central
wavelength tunability in the range of 5-14 $\mu$m is achieved by varying the
peak density in the short high-density region.

Figure 2:  (a) Ideal plasma density profile for efficient frequency
downshifting. (b) Measured plasma density profile produced by a movable blade
covering a fraction of a gas jet. (c) Wavelength tunability in the range of
3-20 $\mu$m. Reproduced with permission from Nie et al., Nat. Photonics 12,
489-494 (2018) (Copyright 2018 Springer Nature) and Nie et al., Nat. Comms.
11, 2787 (2020) (Copyright 2020 The Authors).

We have experimentally demonstrated this frequency downshifting scheme by
employing a tailored plasma structure scheme[27]. A supersonic hydrogen gas
jet target with an insertable blade was used to shock-induce a density spike
[47, 48, 49] in the gas flow to produce a similar density profile as we have
described above as shown in Fig. 2(a) and (b). In the experiment, an ultra-
short (36$\pm$2 fs FWHM), energetic (580$\pm$9 mJ, 16 TW), 810 nm wavelength
drive laser pulse [50] passed through this tailored plasma structure, fully
ionized the neutral gas and produced a highly nonlinear wake [28]. The strong
time-dependent plasma density gradients formed during the expulsion of the
plasma electrons phase modulated the laser pulse downshifting the
instantaneous frequency of the photons as explained earlier. The generated
LWIR pulse was characterized by cross-correlation frequency-resolved optical
gating (XFROG) [51] method based on four-wave mixing (FWM) in argon gas. By
optimizing the peak density of the sharp high-density section of the plasma
structure, we have generated 3.4$\pm$1.1 mJ, 32.0 fs (FWHM), LWIR pulses with
a central wavelength of 9.4 $\mu$m, demonstrating the generation of
relativistic, near single-cycle LWIR pulses. We also showed that the central
wavelengths could be tuned in the range of 3–20 $\mu$m by varying several
experimental parameters, such as the gas density profile and laser energy.
These data are summarized in Fig. 2(c) where we show the measured energy and
the deduced peak $a_{0}$ of the frequency downshifted light from 3-20 $\mu$m
range.

##  III Frequency downshifting in a nonlinear plasma wake (THz range)

Now we extend this idea to obtain energetic THz radiation. Although research
on generating intense ultra-short THz pulses has made great strides in the
past decade, one of the current challenges facing THz technology is how to
generate sub-joule, terawatts level, single-cycle THz pulses. Most existing
methods, such as DFG [21], optical rectification [22, 23] and two-color
filamentation [25], rely on frequency conversion from commonly used near-
infrared lasers. The major issue of these methods is the low conversion
efficiency. One straightforward way to improve this problem is to use longer-
wavelength drive laser. CO2 laser is currently the most powerful LWIR
($\sim$10 $\mu$m) laser close to THz spectral range. In recent years, multi-
terawatt picosecond 10 $\mu$m CO2 lasers have been developed [52, 53], that
provides a better alternate path to nonlinear optics for generating intense
ultra-short THz pulses.

The concept of frequency downshifting in plasmas described in Sec. II can be
extended to THz range if switching the drive lasers from Ti:sapphire lasers
($\sim$800 nm) to CO2 lasers ($\sim$10 $\mu$m). Based on this idea, sub-joule,
terawatts, single-cycle THz pulses can be generated by using a picosecond 10
$\mu$m CO2 laser to excite a nonlinear wake in a tailored plasma structure
described in Sec. II; however, the details of the plasma density structure are
different.

Figure 3:  Quasi-3D OSIRIS simulation results on generation of sub-joule,
terawatts, single-cycle THz pulses. (a) The tailored plasma density profile.
(b,c) Evolution of pulse duration and spectrum with propagation distance in
the plasma. (d) Final spectrum and filtered single-cycle THz spectrum. (e) The
filtered single-cycle THz pulse.

Here, we present the PIC simulation results calculated by Quasi-3D OSIRIS code
[54, 55]. The drive CO2 laser pulse has an energy of 14.6 J with pulse
duration of 1 ps (FWHM). It is focused to a spot size of $w_{0}=130.5\,\mu$m
at the front of the tailored plasma structure. At the initial $a_{0}$ of 2,
the laser spot size is matched to the plasma at a density of $1.3\times
10^{16}\,\text{cm}^{-3}$. However, with the increase of the plasma density,
wake excitation leads to self-compression and self-focusing which increases
the $a_{0}$ and decreases the laser spot size. In our simulation, the laser
beam is seen to be self-guided in the wake throughout the whole propagation
distance [56]. As discussed in the previous case, the tailored plasma
structure in this case (Fig. 3(a)) also consists of two sections: the pulse
compressor and the THz converter. The pulse compressor section is a long
plasma up-ramp, which is different than the uniform density (flattop) profile
of the pulse compressor section in the tailored structure (Fig. 2(a) and (b))
proposed in Sec. II. The reason is that the normalized vector potential
$a_{0}$ is lower and the pulse length (scaled to drive laser wavelength) is
longer in this CO2 drive laser case compared with the previous Ti:sapphire
drive laser cases, which makes it more challenging to compress the CO2 pulse.
The long up-ramp plasma profile is designed to compress such a 1-ps drive
pulse in an as short a distance as possible. At the beginning of the plasma,
the plasma density should be low enough so that the plasma wavelength is
longer than the original pulse length to avoid pulse splitting by overlapping
with two wake buckets. As the laser pulse self-compresses, the plasma density
increases accordingly to speed up pulse shortening and minimize the
compression distance. In this way, such an up-ramp plasma profile compresses
the originally 1-ps pulse to less than 100 fs (Fig. 3(b), Fig. 4(b,e)).

Then in the THz converter section, the compressed drive laser is rapidly
frequency-downshifted in a higher-density plasma region to generate the THz
photons. The THz photons then slip backwards towards the center of the wake
due to their much smaller group velocity. The central region of the wake is
nearly devoid of plasma electrons ($\eta\sim 1$) and thus serves as a perfect
container for the THz pulse (Fig. 4(c,f)). Finally, the generated THz pulse
has a pulse duration of 271 fs (FWHM), corresponding to a single-cycle pulse
for the central frequency of 3.4 THz. The estimated THz energy (1-5 THz) is
293 mJ, with conversion efficiency of $\sim$2 % and peak power as high as 1.1
TW. Accordingly, the peak field of the THz pulse at the exit of plasma
structure reaches as high as 2.3 GV/cm.

Figure 4:  Three snapshots showing the evolution of the laser pulse and
wakefield. (a-c) The transverse electric field of the laser pulse and the
plasma density distribution (a) at the beginning of the plasma structure , (b)
at the end of pulse compressor, and (c) at the end of THz converter. (d-f) The
Wigner spectrograms of the on-axis transverse electric field and laser
envelope (orange dashed line) at the same position as in (a-c), respectively.

Furthermore, the output THz frequency can be tuned from 2-12 THz by varying
the plasma density and length in the THz converter section while keeping other
parameters unchanged. From Fig. 5, one can see that the conversion efficiency
roughly linearly scales with the output frequency, and the THz pulses in
different frequency cases remain pulse duration at around single cycle or even
sub cycle at lower frequency cases.

Figure 5:  Conversion efficiency of frequency downshifting and optical cycles
v.s. output central frequency.

Although the driving laser pulse duration used here (1 ps) is at present less
than the shortest available CO2 pulse duration (2 ps)[57], a great deal of
work is being carried out to shorten CO2 pulses to 1-ps or even sub-ps pulse
durations in the near future [58, 59]. Even if using the currently available
2-ps CO2 laser pulse, we can still use the same method with some adaptation. A
plasma channel may be used to confine the laser beam in long propagation
distance of the pulse compressor section, since the plasma density in the
pulse compressor has to be lower and self-guiding at lower densities may not
be possible.

##  IV Frequency upshifting by a relativistic ionization front

Now, we introduce another plasma technique: frequency upshifting by a
relativistic ionization front. This mechanism was first studied theoretically
[60, 61], and then observed using microwave radiation colliding with an
overdense ionization front [62, 63]. Here, we propose using this scheme to
frequency upshift a 10 $\mu$m CO2 laser by colliding it with a relativistic
ionization front produced by a short-wavelength ionizing laser pulse.

Figure 6:  Schematic of frequency upshifting of a CO2 laser pulse by a
relativistic ionization front produced by a short-wavelength ionizing laser.

The proposed schematic is shown in Fig. 6. The ionizing laser with a frequency
of $\omega_{ion}$ propagates into the neutral gas. It ionizes the neutral gas
and produces a relativistic ionization front. To the lowest order, the front’s
velocity $v_{0}$ is the group velocity of the ionizing laser in plasma. The
CO2 laser with a frequency of $\omega_{i}$ ($\omega_{i}\ll\omega_{ion}$)
counter-propagates with the ionizing laser and collides with the ionization
front. Then the frequency of the CO2 laser is upshifted due to relativistic
Doppler effect. Here, we use Lorentz transformations to investigate this
problem. Switching from the laboratory frame to the ionization front frame,
the incident CO2 laser frequency of $\omega_{i}$ will be upshifted to
$\omega_{f}\simeq 2\gamma_{0}\omega_{i}$, where
$\gamma_{0}=(1-\frac{v_{0}^{2}}{c^{2}})^{-\frac{1}{2}}$. Depending on the
plasma density of the ionization front, there are two possible regimes [61].
When the plasma density is high enough so that $\omega_{p}>\omega_{f}$
($\omega_{p}^{2}>2\omega_{ion}\omega_{i}$), it is in the overdense regime. The
CO2 laser will be reflected by the ionization front, and the frequency of the
reflected wave in the lab frame is $\omega_{r}\simeq
4\gamma_{0}^{2}\omega_{i}$.

When the plasma density $\omega_{p}<\omega_{f}$
($\omega_{p}^{2}<2\omega_{ion}\omega_{i}$), it is in the underdense regime.
The CO2 laser will enter the ionization front, and the frequency of the
transmitted wave is now $\omega_{t}=\gamma_{0}(\omega_{f}-v_{0}k_{tf})$, where
$k_{tf}=\frac{1}{c}\sqrt{\omega_{f}^{2}-\omega_{p}^{2}}$ is the wavenumber of
the transmitted wave in the ionization front frame. The transmitted frequency
increases with the increase of the plasma density and is bounded by
$\omega_{ion}$ (when $\omega_{p}^{2}=2\omega_{ion}\omega_{i}$). In the
underdense limit ($\omega_{p}^{2}\ll 2\omega_{ion}\omega_{i}$), the
transmitted frequency and wavenumber are approximately
$\omega_{t}\simeq\omega_{i}(1+\frac{\omega_{p}^{2}}{4\omega_{i}^{2}})$ and
$k_{t}\simeq\frac{\omega_{i}}{c}(1-\frac{\omega_{p}^{2}}{4\omega_{i}^{2}})$.
Interestingly, the transmitted wavenumber could be positive, zero, or
negative. When the wavenumber is zero, the group velocity of the wave goes to
zero and the wave will be absorbed by the plasma. When the wavenumber is
negative, the wave will travel backwards in the laboratory frame, moving with
a group velocity less than the velocity of the ionization front so that it is
still transmitted in the ionization front frame.

As observed in a previous study [61], the underdense regime instead of the
overdense regime is recommended in practical applications for two reasons. One
is that the frequency upshift in the underdense regime is independent of
$\gamma_{0}$, therefore not affected by the potential variation of
$\gamma_{0}$ due to the variation of the group velocity of ionizing laser when
ionizing the gas. The other advantage is that the plasma density needed is
lower than the overdense case for the same incident and upshifted frequencies.

We have run a series of 2D OSIRIS[45, 46] simulations to show frequency tuning
dependence on plasma density in the underdense regime. In our simulations, the
ionizing laser is a 40-fs 800-nm Ti:sapphire laser with a focused spot size of
200 $\mu$m and normalized vector potential $a_{0}=0.05$. The CO2 laser pulse
duration is 333 fs here (to save simulation time) with a focused spot size of
100 $\mu$m and normalized vector potential $a_{0}=0.05$. Hydrogen is used as
the neutral gas. The Ti:sapphire laser can ionize the hydrogen gas but the CO2
laser cannot since its peak intensity is below the tunneling ionization
threshold intensity for hydrogen. The output wavelength can be tuned in the
spectral range of 1-10 $\mu$m by simply tuning the gas density as shown in
Fig. 7(a). In the underdense limit the reflected wave is very weak, thus
transmission coefficient in the laboratory frame approaches unity and the
pulse duration will be compressed due to the conservation of oscillation
cycles [61] if not considering later-on pulse stretching by dispersion.
Therefore, the energy conversion efficiency approximately equals to the ratio
of the output wavelength and the initial wavelength
($\eta\simeq\lambda_{t}/\lambda_{i}$) as shown in Fig. 7(b).

Figure 7:  Frequency upshifting in the underdense regime. (a) Output
wavelength v.s. ionization front density. (b) Energy conversion efficiency
v.s. output wavelength. The dashed line marks the boundary of $k=0$.

For the sake of completeness, we should also mention that physically it is the
boundary conditions on $\mathbb{E}$ and $\mathbb{H}$ at the ionization front
that give rise to the reflected and transmitted waves. In addition, the plasma
ionization front can support currents. If the onset of ionization is sudden,
the ionization front width is much less than the wavelength of the CO2 . The
plasma electrons are produced at rest, therefore the current density
$\mathbb{J}=0$. Thereafter as the ionization front passes through the linearly
polarized CO2 laser pulse, its electric field gives rise to sinusoidal
oscillations of current that in turn give rise to regions of static magnetic
field at half the wavelength of the CO2 laser. This so-called magneto-static
mode [61, 64] is the third natural mode in an unmagnetized plasma in addition
to the Bohm-Gross waves and ion acoustic waves. This mode has not been
conclusively identified to date in plasmas because of the difficulty of
producing a periodic arrangement of quasi-dc currents in plasmas. The
interaction of an ionization front with a counter propagating electromagnetic
wave as described here will allow one to generate the conditions necessary to
excite this mode in addition to frequency upshifted light that is the topic of
this paper.

##  V Conclusions

In summary, we show in this paper that by using two different plasma
techniques we can generate mid-IR/THz pulses that cover the entire bandwidth
from 1-150 $\mu$m (300-2 THz).

By using frequency downshifting of a Ti:sapphire laser in a nonlinear wake,
the generation of relativistic, near single-cycle LWIR pulses tunable in 3-20
$\mu$m range has already been demonstrated using a tailored plasma structure.
Given that a few TW class, femtoseconds drive lasers in the near-IR are now
commonplace, one expects this technique to be adopted in many laboratories to
give intense, tunable LWIR pulses. Further extending this scheme into THz
range is achieved by using a picosecond CO2 driving laser instead of a
Ti:sapphire laser. By PIC simulations, we show that sub-joule, terawatts,
single-cycle THz pulses are generated and can be frequency tuned in the range
of 2-12 THz. At relativistic intensities afforded by such mid-IR/THz sources
it will now be possible to study laser wakefield and direct laser acceleration
by long-wavelength drivers not only in plasmas but also in dielectric and
semiconductor structures.

By using frequency upshifting of a CO2 laser by colliding it with a
relativistic ionization front, we show by PIC simulations that mid-IR pulses
tunable in the spectral range of 1-10 $\mu$m can be produced. The tunability
is straightforward by simply tuning the gas density. Such mid-IR pulses with
relatively lower intensities can be used for pump-probe experiments in the
molecular fingerprint region[1, 2], high harmonic generation[3, 4, 6], and
resonant or non-resonant nonlinear interactions in gases, solids, or
biological systems[7, 8, 9].

###### Acknowledgements.

This work was supported by the Office of Naval Research (ONR) MURI
(N00014-17-1-2075), AFOSR grant FA9550-16-1-0139, U.S. Department of Energy
grant DE-SC0010064 and NSF grant 1734315. The simulations were performed on
Sunway TaihuLight, Hoffman cluster at UCLA, and NERSC at LBNL.

## Data availability

The data that support the findings of this study are available from the
corresponding author upon reasonable request.

## References

## References

  * Meckel  _et al._ [2008] M. Meckel, D. Comtois, D. Zeidler, A. Staudte, D. Pavicic, H. C. Bandulet, H. Pepin, J. C. Kieffer, R. Dorner, D. M. Villeneuve, and P. B. Corkum, “Laser-Induced Electron Tunneling and Diffraction,” Science 320, 1478–1482 (2008).
  * Först  _et al._ [2011] M. Först, C. Manzoni, S. Kaiser, Y. Tomioka, Y. Tokura, R. Merlin, and A. Cavalleri, “Nonlinear phononics as an ultrafast route to lattice control,” Nature Physics 7, 854–856 (2011), 1101.1878 .
  * Popmintchev  _et al._ [2012] T. Popmintchev, M.-C. Chen, D. Popmintchev, P. Arpin, S. Brown, S. Alisauskas, G. Andriukaitis, T. Balciunas, O. D. Mucke, A. Pugzlys, A. Baltuska, B. Shim, S. E. Schrauth, A. Gaeta, C. Hernandez-Garcia, L. Plaja, A. Becker, A. Jaron-Becker, M. M. Murnane, and H. C. Kapteyn, “Bright Coherent Ultrahigh Harmonics in the keV X-ray Regime from Mid-Infrared Femtosecond Lasers,” Science 336, 1287–1291 (2012).
  * Ghimire  _et al._ [2011] S. Ghimire, A. D. DiChiara, E. Sistrunk, P. Agostini, L. F. DiMauro, and D. A. Reis, “Observation of high-order harmonic generation in a bulk crystal,” Nature Physics 7, 138–141 (2011).
  * Wolter  _et al._ [2015] B. Wolter, M. G. Pullen, M. Baudisch, M. Sclafani, M. Hemmer, A. Senftleben, C. D. Schröter, J. Ullrich, R. Moshammer, and J. Biegert, “Strong-Field Physics with Mid-IR Fields,” Physical Review X 5, 021034 (2015).
  * Vampa  _et al._ [2015] G. Vampa, T. J. Hammond, N. Thiré, B. E. Schmidt, F. Légaré, C. R. McDonald, T. Brabec, and P. B. Corkum, “Linking high harmonics from gases and solids,” Nature 522, 462–464 (2015).
  * Schubert  _et al._ [2014] O. Schubert, M. Hohenleutner, F. Langer, B. Urbanek, C. Lange, U. Huttner, D. Golde, T. Meier, M. Kira, S. W. Koch, and R. Huber, “Sub-cycle control of terahertz high-harmonic generation by dynamical Bloch oscillations,” Nature Photonics 8, 119–123 (2014).
  * Hassan  _et al._ [2016] M. T. Hassan, T. T. Luu, A. Moulet, O. Raskazovskaya, P. Zhokhov, M. Garg, N. Karpowicz, A. M. Zheltikov, V. Pervak, F. Krausz, and E. Goulielmakis, “Optical attosecond pulses and tracking the nonlinear response of bound electrons,” Nature 530, 66–70 (2016).
  * Pupeza  _et al._ [2020] I. Pupeza, M. Huber, M. Trubetskov, W. Schweinberger, S. A. Hussain, C. Hofer, K. Fritsch, M. Poetzlberger, L. Vamos, E. Fill, T. Amotchkina, K. V. Kepesidis, A. Apolonski, N. Karpowicz, V. Pervak, O. Pronin, F. Fleischmann, A. Azzeer, M. Žigman, and F. Krausz, “Field-resolved infrared spectroscopy of biological systems,” Nature 577, 52–59 (2020).
  * Auton and Cheung [1985] D. H. Auton and K. P. Cheung, “Coherent time-domain far-infrared spectroscopy,” J. Opt. Soc. Am. B 2, 606–612 (1985).
  * Matsubara, Nagai, and Ashida [2013] E. Matsubara, M. Nagai, and M. Ashida, “Coherent infrared spectroscopy system from terahertz to near infrared using air plasma produced by 10-fs pulses,” J. Opt. Soc. Am. B 30, 1627–1630 (2013).
  * Blaga  _et al._ [2012] C. I. Blaga, J. Xu, A. D. DiChiara, E. Sistrunk, K. Zhang, P. Agostini, T. A. Miller, L. F. DiMauro, and C. D. Lin, “Imaging ultrafast molecular dynamics with laser-induced electron diffraction,” Nature 483, 194–197 (2012).
  * Wimmer  _et al._ [2014] L. Wimmer, G. Herink, D. R. Solli, S. V. Yalunin, K. E. Echternkamp, and C. Ropers, “Terahertz control of nanotip photoemission,” Nature Physics 10, 432–436 (2014).
  * Silva  _et al._ [2015] F. Silva, S. M. Teichmann, S. L. Cousin, M. Hemmer, and J. Biegert, “Spatiotemporal isolation of attosecond soft X-ray pulses in the water window,” Nature Communications 6, 6611 (2015), 1411.1964 .
  * Hernández-García  _et al._ [2013] C. Hernández-García, J. A. Pérez-Hernández, T. Popmintchev, M. M. Murnane, H. C. Kapteyn, A. Jaron-Becker, A. Becker, and L. Plaja, “Zeptosecond High Harmonic keV X-Ray Waveforms Driven by Midinfrared Laser Pulses,” Physical Review Letters 111, 033002 (2013).
  * Fu  _et al._ [2018] Y. Fu, B. Xue, K. Midorikawa, and E. J. Takahashi, “TW-scale mid-infrared pulses near 3.3 $\mu$ m directly generated by dual-chirped optical parametric amplification,” Applied Physics Letters 112, 241105 (2018).
  * Sanchez  _et al._ [2016] D. Sanchez, M. Hemmer, M. Baudisch, S. L. Cousin, K. Zawilski, P. Schunemann, O. Chalus, C. Simon-Boisson, and J. Biegert, “7 $\mu$m, ultrafast, sub-millijoule-level mid-infrared optical parametric chirped pulse amplifier pumped at 2 $\mu$m,” Optica 3, 147 (2016).
  * Pupeza  _et al._ [2015] I. Pupeza, D. Sánchez, J. Zhang, N. Lilienfein, M. Seidel, N. Karpowicz, T. Paasch-Colberg, I. Znakovskaya, M. Pescher, W. Schweinberger, V. Pervak, E. Fill, O. Pronin, Z. Wei, F. Krausz, A. Apolonski, and J. Biegert, “High-power sub-two-cycle mid-infrared pulses at 100 MHz repetition rate,” Nature Photonics 9, 721–724 (2015).
  * Krogen  _et al._ [2017] P. Krogen, H. Suchowski, H. Liang, N. Flemens, K.-H. Hong, F. X. Kärtner, and J. Moses, “Generation and multi-octave shaping of mid-infrared intense single-cycle pulses,” Nature Photonics 11, 222–226 (2017).
  * Novák  _et al._ [2018] O. Novák, P. R. Krogen, T. Kroh, T. Mocek, F. X. Kärtner, and K.-H. Hong, “Femtosecond 8.5 $\mu$m source based on intrapulse difference-frequency generation of 2.1 $\mu$m pulses,” Optics Letters 43, 1335 (2018).
  * Vicario  _et al._ [2014] C. Vicario, A. V. Ovchinnikov, S. I. Ashitkov, M. B. Agranat, V. E. Fortov, and C. P. Hauri, “Generation of 09-mJ THz pulses in DSTMS pumped by a Cr:Mg_2SiO_4 laser,” Optics Letters 39, 6632 (2014).
  * Sell, Leitenstorfer, and Huber [2008] A. Sell, A. Leitenstorfer, and R. Huber, “Phase-locked generation and field-resolved detection of widely tunable terahertz pulses with amplitudes exceeding 100 MV/cm,” Optics Letters 33, 2767 (2008).
  * Junginger  _et al._ [2010] F. Junginger, A. Sell, O. Schubert, B. Mayer, D. Brida, M. Marangoni, G. Cerullo, A. Leitenstorfer, and R. Huber, “Single-cycle multiterahertz transients with peak fields above 10 MV/cm,” Optics Letters 35, 2645 (2010).
  * Fuji, Nomura, and Shirai [2015] T. Fuji, Y. Nomura, and H. Shirai, “Generation and Characterization of Phase-Stable Sub-Single-Cycle Pulses at 3000 cm-1,” IEEE Journal of Selected Topics in Quantum Electronics 21, 1–12 (2015).
  * Yoo  _et al._ [2017] Y.-J. Yoo, D. Kuk, Z. Zhong, and K.-Y. Kim, “Generation and Characterization of Strong Terahertz Fields From kHz Laser Filamentation,” IEEE Journal of Selected Topics in Quantum Electronics 23, 1–7 (2017).
  * Nie  _et al._ [2018] Z. Nie, C.-H. Pai, J. Hua, C. Zhang, Y. Wu, Y. Wan, F. Li, J. Zhang, Z. Cheng, Q. Su, S. Liu, Y. Ma, X. Ning, Y. He, W. Lu, H.-H. Chu, J. Wang, W. B. Mori, and C. Joshi, “Relativistic single-cycle tunable infrared pulses generated from a tailored plasma density structure,” Nature Photonics 12, 489–494 (2018).
  * Nie  _et al._ [2020] Z. Nie, C.-H. Pai, J. Zhang, X. Ning, J. Hua, Y. He, Y. Wu, Q. Su, S. Liu, Y. Ma, Z. Cheng, W. Lu, H.-H. Chu, J. Wang, C. Zhang, W. B. Mori, and C. Joshi, “Photon deceleration in plasma wakes generates single-cycle relativistic tunable infrared pulses,” Nature Communications 11, 2787 (2020).
  * Lu  _et al._ [2006] W. Lu, C. Huang, M. Zhou, W. B. Mori, and T. Katsouleas, “Nonlinear Theory for Relativistic Plasma Wakefields in the Blowout Regime,” Physical Review Letters 96, 165002 (2006).
  * Sprangle, Esarey, and Ting [1990a] P. Sprangle, E. Esarey, and A. Ting, “Nonlinear theory of intense laser-plasma interactions,” Physical Review Letters 64, 2011–2014 (1990a).
  * Sprangle, Esarey, and Ting [1990b] P. Sprangle, E. Esarey, and A. Ting, “Nonlinear interaction of intense laser pulses in plasmas,” Physical Review A 41, 4463–4469 (1990b).
  * Wilks  _et al._ [1989] S. C. Wilks, J. M. Dawson, W. B. Mori, T. Katsouleas, and M. E. Jones, “Photon accelerator,” Physical Review Letters 62, 2600–2603 (1989).
  * Esarey, Ting, and Sprangle [1990] E. Esarey, A. Ting, and P. Sprangle, “Frequency shifts induced in laser pulses by plasma waves,” Physical Review A 42, 3526–3531 (1990).
  * Mori [1997] W. Mori, “The physics of the nonlinear optics of plasmas at relativistic intensities for short-pulse lasers,” IEEE Journal of Quantum Electronics 33, 1942–1953 (1997).
  * Tsung  _et al._ [2002] F. S. Tsung, C. Ren, L. O. Silva, W. B. Mori, and T. Katsouleas, “Generation of ultra-intense single-cycle laser pulses by using photon deceleration,” Proceedings of the National Academy of Sciences of the United States of America 99, 29–32 (2002).
  * Gordon  _et al._ [2003] D. F. Gordon, B. Hafizi, R. F. Hubbard, J. R. Peñano, P. Sprangle, and A. Ting, “Asymmetric Self-Phase Modulation and Compression of Short Laser Pulses in Plasma Channels,” Physical Review Letters 90, 215001 (2003).
  * Faure  _et al._ [2005] J. Faure, Y. Glinec, J. J. Santos, F. Ewald, J.-P. P. Rousseau, S. Kiselev, A. Pukhov, T. Hosokai, and V. Malka, “Observation of laser-pulse shortening in nonlinear plasma waves,” Physical Review Letters 95, 205003 (2005).
  * Downer  _et al._ [2018] M. C. Downer, R. Zgadzaj, A. Debus, U. Schramm, and M. C. Kaluza, “Diagnostics for plasma-based electron accelerators,” Reviews of Modern Physics 90, 035002 (2018).
  * Murphy  _et al._ [2006] C. D. Murphy, R. Trines, J. Vieira, A. J. W. Reitsma, R. Bingham, J. L. Collier, E. J. Divall, P. S. Foster, C. J. Hooker, A. J. Langley, P. A. Norreys, R. A. Fonseca, F. Fiuza, L. O. Silva, J. T. Mendonça, W. B. Mori, J. G. Gallacher, R. Viskup, D. A. Jaroszynski, S. P. D. Mangles, A. G. R. Thomas, K. Krushelnick, and Z. Najmudin, “Evidence of photon acceleration by laser wake fields,” Physics of Plasmas 13, 033108 (2006).
  * Shiraishi  _et al._ [2013] S. Shiraishi, C. Benedetti, A. J. Gonsalves, K. Nakamura, B. H. Shaw, T. Sokollik, J. Van Tilborg, C. G. Geddes, C. B. Schroeder, C. Tóth, E. Esarey, and W. P. Leemans, “Laser red shifting based characterization of wakefield excitation in a laser-plasma accelerator,” Physics of Plasmas 20, 0–11 (2013).
  * Zhu, Palastro, and Antonsen [2012] W. Zhu, J. P. Palastro, and T. M. Antonsen, “Studies of spectral modification and limitations of the modified paraxial equation in laser wakefield simulations,” Physics of Plasmas 19, 033105 (2012).
  * Zhu, Palastro, and Antonsen [2013] W. Zhu, J. P. Palastro, and T. M. Antonsen, “Pulsed mid-infrared radiation from spectral broadening in laser wakefield simulations,” Physics of Plasmas 20, 073103 (2013).
  * Pai  _et al._ [2010] C.-H. Pai, Y.-Y. Chang, L.-C. Ha, Z.-H. Xie, M.-W. Lin, J.-M. Lin, Y.-M. Chen, G. Tsaur, H.-H. Chu, S.-H. Chen, J.-Y. Lin, J. Wang, and S.-Y. Chen, “Generation of intense ultrashort midinfrared pulses by laser-plasma interaction in the bubble regime,” Physical Review A 82, 063804 (2010).
  * Schreiber  _et al._ [2010] J. Schreiber, C. Bellei, S. P. D. Mangles, C. Kamperidis, S. Kneip, S. R. Nagel, C. A. J. Palmer, P. P. Rajeev, M. J. V. Streeter, and Z. Najmudin, “Complete Temporal Characterization of Asymmetric Pulse Compression in a Laser Wakefield,” Physical Review Letters 105, 235003 (2010), arXiv:arXiv:1010.3694v1 .
  * Streeter  _et al._ [2018] M. J. V. Streeter, S. Kneip, M. S. Bloom, R. A. Bendoyro, O. Chekhlov, A. E. Dangor, A. Döpp, C. J. Hooker, J. Holloway, J. Jiang, N. C. Lopes, H. Nakamura, P. A. Norreys, C. A. J. Palmer, P. P. Rajeev, J. Schreiber, D. R. Symes, M. Wing, S. P. D. Mangles, and Z. Najmudin, “Observation of Laser Power Amplification in a Self-Injecting Laser Wakefield Accelerator,” Physical Review Letters 120, 254801 (2018).
  * Fonseca  _et al._ [2002] R. A. Fonseca, L. O. Silva, F. S. Tsung, V. K. Decyk, W. Lu, C. Ren, W. B. Mori, S. Deng, S. Lee, T. Katsouleas, and J. C. Adam, “OSIRIS: A Three-Dimensional, Fully Relativistic Particle in Cell Code for Modeling Plasma Based Accelerators,” in  _Computational Science — ICCS 2002_, edited by C. T. P. Sloot, A. Hoekstra, Dongarra and J.Dongarra (Springer Berlin / Heidelberg, 2002) pp. 342–351.
  * Fonseca  _et al._ [2008] R. A. Fonseca, S. F. Martins, L. O. Silva, J. W. Tonge, F. S. Tsung, and W. B. Mori, “One-to-one direct modeling of experiments and astrophysical scenarios: pushing the envelope on kinetic plasma simulations,” Plasma Physics and Controlled Fusion 50, 124034 (2008), 0810.2460 .
  * Schmid  _et al._ [2010] K. Schmid, A. Buck, C. M. Sears, J. M. Mikhailova, R. Tautz, D. Herrmann, M. Geissler, F. Krausz, and L. Veisz, “Density-transition based electron injector for laser driven wakefield accelerators,” Physical Review Special Topics - Accelerators and Beams 13, 1–5 (2010).
  * Gonsalves  _et al._ [2011] A. J. Gonsalves, K. Nakamura, C. Lin, D. Panasenko, S. Shiraishi, T. Sokollik, C. Benedetti, C. B. Schroeder, C. G. Geddes, J. Van Tilborg, J. Osterhoff, E. Esarey, C. Toth, and W. P. Leemans, “Tunable laser plasma accelerator based on longitudinal density tailoring,” Nature Physics 7, 862–866 (2011).
  * Buck  _et al._ [2013] A. Buck, J. Wenz, J. Xu, K. Khrennikov, K. Schmid, M. Heigoldt, J. M. Mikhailova, M. Geissler, B. Shen, F. Krausz, S. Karsch, and L. Veisz, “Shock-front injector for high-quality laser-plasma acceleration,” Physical Review Letters 110, 1–5 (2013).
  * Hung  _et al._ [2014] T.-S. Hung, C.-H. Yang, J. Wang, S.-y. Chen, J.-Y. Lin, and H.-h. Chu, “A 110-TW multiple-beam laser system with a 5-TW wavelength-tunable auxiliary beam for versatile control of laser-plasma interaction,” Applied Physics B 117, 1189–1200 (2014).
  * Linden, Giessen, and Kuhl [1998] S. Linden, H. Giessen, and J. Kuhl, “XFROG — A New Method for Amplitude and Phase Characterization of Weak Ultrashort Pulses,” physica status solidi (b) 206, 119–124 (1998), (SICI)1521-3951(199803)206:1 [10.1002] .
  * Haberberger, Tochitsky, and Joshi [2010] D. Haberberger, S. Tochitsky, and C. Joshi, “Fifteen terawatt picosecond CO2 laser system,” Optics Express 18, 17865 (2010).
  * Polyanskiy, Pogorelsky, and Yakimenko [2011] M. N. Polyanskiy, I. V. Pogorelsky, and V. Yakimenko, “Picosecond pulse amplification in isotopic CO2 active medium,” Optics Express 19, 7717 (2011).
  * Lifschitz  _et al._ [2009] A. Lifschitz, X. Davoine, E. Lefebvre, J. Faure, C. Rechatin, and V. Malka, “Particle-in-Cell modelling of laser–plasma interaction using Fourier decomposition,” Journal of Computational Physics 228, 1803–1814 (2009).
  * Davidson  _et al._ [2015] A. Davidson, A. Tableman, W. An, F. Tsung, W. Lu, J. Vieira, R. Fonseca, L. Silva, and W. Mori, “Implementation of a hybrid particle code with a PIC description in r – z and a gridless description in $\phi$ into OSIRIS,” Journal of Computational Physics 281, 1063–1077 (2015), 1403.6890 .
  * Ralph  _et al._ [2009] J. E. Ralph, K. A. Marsh, A. E. Pak, W. Lu, C. E. Clayton, F. Fang, W. B. Mori, and C. Joshi, “Self-Guiding of Ultrashort, Relativistically Intense Laser Pulses through Underdense Plasmas in the Blowout Regime,” Physical Review Letters 102, 175003 (2009).
  * Polyanskiy  _et al._ [2020] M. N. Polyanskiy, I. V. Pogorelsky, M. Babzien, and M. A. Palmer, “ Demonstration of a 2 ps, 5 TW peak power, long-wave infrared laser based on chirped-pulse amplification with mixed-isotope CO2 amplifiers ,” OSA Continuum 3, 459 (2020).
  * Panagiotopoulos  _et al._ [2020] P. Panagiotopoulos, M. G. Hastings, M. Kolesik, S. Tochitsky, and J. V. Moloney, “Multi-terawatt femtosecond 10 $\mu$m laser pulses by self-compression in a CO2 cell,” OSA Continuum 3, 3040 (2020).
  * Tovey  _et al._ [2019] D. Tovey, S. Y. Tochitsky, J. J. Pigeon, G. J. Louwrens, M. N. Polyanskiy, I. Ben-Zvi, and C. Joshi, “ Multi-atmosphere picosecond CO2 amplifier optically pumped at 4.3 $\mu$m ,” Applied Optics 58, 5756 (2019).
  * Lampe, Ott, and Walker [1978] M. Lampe, E. Ott, and J. H. Walker, “Interaction of electromagnetic waves with a moving ionization front,” Physics of Fluids 21, 42 (1978).
  * Mori [1991] W. B. Mori, “Generation of tunable radiation using an underdense ionization front,” Physical Review A 44, 5118–5121 (1991).
  * Savage, Joshi, and Mori [1992] R. L. Savage, C. Joshi, and W. B. Mori, “Frequency upconversion of electromagnetic radiation upon transmission into an ionization front,” Physical Review Letters 68, 946–949 (1992).
  * Lai  _et al._ [1996] C. H. Lai, R. Liou, T. C. Katsouleas, P. Muggli, R. Brogle, C. Joshi, and W. B. Mori, “Demonstration of Microwave Generation from a Static Field by a Relativistic Ionization Front in a Capacitor Array,” Physical Review Letters 77, 4764–4767 (1996).
  * Fiuza, Silva, and Joshi [2010] F. Fiuza, L. O. Silva, and C. Joshi, “High-brilliance synchrotron radiation induced by the plasma magnetostatic mode,” Physical Review Special Topics - Accelerators and Beams 13, 1–5 (2010).