# Less is More!
A slim architecture for optimal language translation

Luca Herranz-Celotti1 Ermal Rrapaj2
1NECOTIS, Université de Sherbrooke, Canada
2 NERSC, Lawrence Berkeley National Laboratory, Berkeley, California 94720,
USA
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The softmax attention mechanism has emerged as a noteworthy development in the
field of Artificial Intelligence research, building on the successes of
Transformer-based architectures. However, their ever increasing sizes
necessitate ever increasing computational memory, that limits their usage. We
propose KgV, a sigmoid gating mechanism that, in conjunction with softmax
attention, significantly boosts performance without increasing architecture
size. To amend the size requirements, we leverage Tensor Chains to identify
and prune the excess parameters. We find that such excess resides primarily
within the embedding layer, and not in the output linear layer. To further
improve embedding and significantly reduce parameters, we introduce H-SoftPOS,
a hierarchical embedding layer which simultaneously enhances performance.
Remarkably, on the WMT14 English-German validation set, our approach yields a
threefold reduction in perplexity, surpassing the current state-of-the-art,
while reducing parameter counts also by a factor of 3. When we further reduce
the number of parameters up to sevenfold, we can still achieve a 21% decrease
in perplexity with respect to the baseline Transformer. To understand
generalization capabilities, we conduct experiments on the 7 language pairs of
the WMT17 dataset. Our method outperforms existing techniques in terms of test
loss while simultaneously halving the number of parameters. Moreover, we
observe a 70 times reduction in variance with respect to the prior state-of-
the-art. In conclusion, our proposed method yields significant improvements in
performance and much lower memory cost. We call the resulting architecture
Anthe.

##  1 Introduction

The Transformer architecture (Vaswani et al., 2017) has been a catalyst for
major breakthroughs in Artificial Intelligence, leading to outstanding
performance on a wide range of tasks, including language modeling (Brown et
al., 2020), translation (Vaswani et al., 2017), speech recognition (Radford et
al., 2022), and protein folding (Jumper et al., 2021), among others. Since
their inception, Transformer-based architectures have become increasingly
wider (Brown et al., 2020) and deeper (Wang et al., 2022), leading to a
massive increase in the number of parameters. For instance, ChatGPT-3 has 175
billion parameters (Radford et al., 2019; Brown et al., 2020), surpassing the
number of parameters of previous models by orders of magnitude. To address the
huge computational demands of such architectures, especially in handling long
sequences, researchers have proposed several approximate attention mechanisms,
such as sparse-approximation (Kitaev et al., 2020; Likhosherstov et al.,
2020), low-rank approximation (Choromanski et al., 2020; Katharopoulos et al.,
2020; Wang et al., 2020), their combination (Beltagy et al., 2020; Chen et
al., 2021; Zaheer et al., 2020), and I/O optimization techniques for
additional speed-up (Dao et al., 2022). However, not enough attention has been
given to the efficient use of limited parameters, and it often seems that a
reduction in parameters has to come with a degraded performance (Sanh et al.,
2019).

In this article we propose a novel gating mechanism placed before the softmax
attention that significantly improves performance, as evidenced by our
experimental findings. Additionally, we demonstrate that removing weight-
sharing between the output projection and both embeddings of the encoder and
decoder can further improve performance, at the cost of increasing the number
of parameters by 43%. To mitigate this increase in parameters without
compromising accuracy, we introduce two techniques: Hierarchical Soft Part of
Speech (H-SoftPOS) and Tensor Chain (TC). H-SoftPOS is based on the
observation that language elements, such as sub-words, words or sentences, can
have a very limited set of functional roles, and each of them can adapt its
role according to the context, which requires a soft aspect. Therefore, we
propose a method to assign a learnable Part of Speech (SoftPOS) to each
subword, which helps improve performance while decreasing the number of
parameters in the embedding. Additionally, TC allows us to represent a large
matrix as a tensor product contraction, drastically reducing the number of
learnable parameters in the architecture. It was originally proposed in
physics to characterize the short-range entanglement in one-dimensional
quantum systems (Verstraete et al., 2004; Pirvu et al., 2010), and has since
found many applications in physics and more recently in deep learning (Gao et
al., 2020). We name the resulting architecture the Anthe for Gates, and TC and
Hierarchical SoftPOS for Attention. In summary, we design the Anthe, a slim
architecture that improves performance over the Transformer and reduces the
number of parameters.

Our contributions are:

  * •

we introduce the KgV, a gating mechanism between values and keys in the
attention paradigm in Sec. 2.1;

  * •

we introduce Soft Part of Speech to reduce embedding parameters without loss
in performance by accounting for the limited functionality word sets play in
speech, in Sec. 2.2.1;

  * •

we introduce the TC to represent any matrix as a product of small tensors to
drastically reduce the amount of trainable parameters in Sec. 2.2.2;

  * •

we report improvements of Anthe over Transformer first on English-German
language translation, and then on other seven language pairs in Sec. 3.

##  2 Efficiency: higher performance with fewer parameters

In this section, we present our novel techniques for achieving higher
performance with fewer parameters by changing the Transformer setup with our
contributions. Our findings suggest that having independent weights for the
embedding matrices and output projection significantly improves performance,
contrary to prior work. Additionally, we replace the feed-forward layer with
the GEGLU layer (Shazeer, 2020; Chowdhery et al., 2022; Lin et al., 2022), as
we observe a small but statistically significant improvement in performance.
In the following, we proceed to propose KgV to improve performance, and
H-SoftPOS and TC to reduce the number of parameters.

(a)

(b)

(c)

Figure 1: KgV, H-SoftPOS and TC. (a) KgV uses the Key tensor to gate the Value
tensor, before feeding KQV to the attention. (b) Length two Tensor Chain (TC)
with bond dimension $b$ and external dimensions $N_{a}=a_{1}\cdot a_{2},\
N_{c}=c_{1}\cdot c_{2}$. (c) H-SoftPOS starts with a smaller matrix embedding
(green) that is concatenated with hierarchical temporal convolutions (blue)
and SoftPOS of each hierarchical level (red).

###  2.1 Improving performance through gating

Gating mechanisms have been widely used in recurrent neural networks to avoid
issues like gradient explosion (Hochreiter and Schmidhuber, 1997). The LSTM,
GRU, Neural Turing Machine, Differentiable Neural Computer, and Mogrifier LSTM
are well-known examples of models that employ gating mechanisms (Hochreiter
and Schmidhuber, 1997; Chung et al., 2015; Graves et al., 2014, 2016; Melis et
al., 2019). In fact, the latter closed the gap in language modeling tasks
between LSTM and Transformer. They observed a reduced variance in the LSTM
gates when an additional external gating system was introduced, pointing at a
stabilizing effect. In the context of the Transformer architecture, queries,
keys and values, are mapped linearly before applying the softmax attention,
such that $Q_{s}=W_{Q}Q,K_{s}=W_{K}K,V_{s}=W_{V}V$. In this work, for
convenience of notation, we refer to $\\{W_{q},W_{K},W_{V}\\}$ as pre-
attention, or patt. Inspired by the many successes of gates in the past, we
propose a novel gating mechanism before the attention

$\displaystyle V_{s}=$ $\displaystyle\ W_{V}V\sigma(W_{K}K)$ (1)
$\displaystyle Q_{s}=$ $\displaystyle\ W_{Q}Q$ $\displaystyle K_{s}=$
$\displaystyle\ K$ $\displaystyle Attention(Q_{s},K_{s},V_{s})=$
$\displaystyle\
softmax\Big{(}\frac{Q_{s}K_{s}^{T}}{\sqrt{d_{model}/d_{h}}}\Big{)}V_{s}$

where $\sigma$ is the sigmoid activation function, $d_{model}$ is the width of
the architecture and $d_{h}$ the number of heads in the multi-head attention.
Remarkably, it does not change the number of parameters with respect to the
original linear map. In addition, we also tested all other gate combinations
as shown in Tab. 4, which resulted in less effective improvements.

###  2.2 Reducing the number of parameters without compromising accuracy

Compressing deep neural networks to reduce their number of learnable
parameters while maintaining their prediction capabilities is a challenging
and important problem, especially given the rapid increase in size of recent
Transformer-based architectures. Such compression techniques can also help
lower the risk of overfitting. The linear transformations used in fully
connected, convolutional, and attention layers, contain the majority of the
learnable parameters to be trained and stored. In particular, the embedding
layers and output projection contribute significantly to the total number of
parameters in the Transformer. Each of them is represented by a
$d_{model}\times n_{vocab}$ matrix, resulting in three matrices of $16.3M$
parameters each, for standard choices of model width of $d_{model}=512$ and
vocabulary size of $n_{vocab}=32K$. To reduce the number of parameters,
Vaswani et al. (2017) proposed sharing the weights between these three layers.
However, such weight sharing can lead to suboptimal results, as we
demonstrate. Here, we propose two new approaches to significantly reduce the
number of parameters while maintaining performance, H-SoftPOS and TC. These
techniques allow us to keep the embeddings and output projection layers
independent, while avoiding an explosion in the number of parameters.

####  2.2.1 Hierarchical Soft Part of Speech (H-SoftPOS)

To reduce the number of parameters introduced by the embedding matrices, we
propose a new approach called Hierarchical Soft Part of Speech (H-SoftPOS).
The idea is that each sub-word can have a very limited set of functions in a
word (e.g. prefix, suffix, past tense of a verb, etc.). Similarly, each word
can have a very limited set of functions in the sentence (e.g. verb, noun,
adjective, etc.), and there’s a limited set of subordinate clauses in a
sentence (adverb, adjective, and noun clauses), and so on hierarchically.
Since each of the elements can play different roles depending on the context,
there is a soft aspect in the function assigned. We start with a much smaller
embedding dimension and propose a method to assign a learnable Part of Speech
(SoftPOS) to each subword. We use 1D convolutions to convert the sub-word
embedding into word and sentence embeddings, and at each level we assign a
hierarchical version of the SoftPOS idea. Finally, we concatenate the initial
small embedding with the convolution levels and the SoftPOS representations,
to have our full embedding representation. The matrix
$W_{sp}\in\mathbb{R}^{n_{sp}\times d_{sp}}$ represents a finite set $n_{sp}$
of possible POS functions, with dimension $d_{sp}$. We repeat the process at
$l_{sp}$ hierarchical levels. If $S$ is the batch of sequences of integers
that represent the input sentences, then

$\displaystyle Embedding_{sp}(d_{model})(S)=$ $\displaystyle\
Concat\bigcup_{l=1}^{l_{sp}}\Big{\\{}X_{l},SoftPOS(X_{l})\Big{\\}}$ (2)
$\displaystyle X_{1}=$ $\displaystyle\ Embedding(d_{emb})(S)$ $\displaystyle
X_{l}=$ $\displaystyle\ Conv1D(kernel=3,dilation=2^{l},pad=causal)(X_{l-1})$
$\displaystyle SoftPOS(X_{l})=$ $\displaystyle\
W_{sp}softmax(X_{l}[{:}{n_{sp}}])$

where in our implementation $l_{sp}=2$, $d_{sp}=\lfloor
d_{model}/2l_{sp}\rfloor$ and $d_{emb}=d_{model}-(2l_{sp}-1)d_{sp}$. We use
$X[{:}{n_{sp}}]$ to denote the first $n_{sp}$ elements of the vector. In our
setting, it results in an embedding of $d_{model}$ width, with four times
fewer parameters and same performance with respect to the original version.
The embedding $X_{1}$ has a matrix of size $d_{emb}\times n_{vocab}$ and sums
a non-learnable cosine positional encoding (Vaswani et al., 2017).

####  2.2.2 Tensor Chain (TC) representation of a linear layer

The Tensor Chain (TC) representation, also known as the Matrix Product
Operator (MPO), can be used to represent any linear transformation as a higher
order tensor which is factorised into a sequential product of smaller tensors
(Oseledets, 2011; Novikov et al., 2015). The TC representation is a useful
tool for quantum many-body systems, which are known to require extremely large
number of parameters that could grow exponentially with the system size.
Although one might assume that the quality of the representation is degraded
when the number of parameters is reduced, many applications in Condensed
Matter Physics and Quantum Information have shown that efficient and faithful
representations are possible (White, 1992; Vidal, 2003). In the context of
Deep Learning, Gao et al. (2020) replaced the linear layers with TCs in
LeNet-5 (Lecun et al., 1998), VGG (Simonyan and Zisserman, 2015), ResNet (He
et al., 2015), and DenseNet (Huang et al., 2016), without any loss of
prediction accuracy. A weight matrix $W_{N_{a},N_{c}}$ of size
$\mathbb{R}^{N_{a}\times N_{c}}$ can be decomposed as follows

$W_{N_{a},N_{c}}=\text{Reshape}\left\\{\text{Tr}_{b}\left[w^{(1)}_{a_{1},b,c_{1}}\left(\prod_{i=2}^{n-1}w^{(i)}_{a_{i},b,b,c_{i}}\right)w^{(n)}_{a_{n},b,c_{n}}\right],N_{a}\times
N_{c}\right\\},$ (3)

where, $N_{a}=\prod_{i=1}^{n}a_{i},\ N_{b}=\prod_{i=1}^{n}b_{i}$,
$w_{abc},w_{abbc}$ are tensors of order three and four in the chain and $n$ is
the length of the chain. The trace is taken over the common index $b$,
typically referred to as the bond. This internal index can vary between
consecutive tensors, but we chose it to be the same for simplicity. It can be
implemented through the well known einsum function, and here we provide a TC
of length 3 as illustration,

$\text{einsum}\Big{(}a_{1}b_{1}c_{1},\,a_{2}b_{1}b_{2}c_{2},\,a_{3}b_{2}c_{3}\rightarrow
a_{1}a_{2}a_{3}c_{1}c_{2}c_{3},\left[w^{(1)}_{a_{1},b_{1},c_{1}},\,w^{(2)}_{a_{2},b_{1},b_{2},c_{3}},\,w^{(3)}_{a_{3},b_{2},c_{3}}\right]\Big{)}.$
(4)

For optimal memory allocation, the full matrix representation is only implicit
since one can reshape the input in TC format, perform tensor contraction, and
reshape the output into the target size (Novikov et al., 2020).

As we apply TC to different parts of the network, we use the notation
TC${}_{\textit{where}:r}$ to indicate the use of TC in the location where, to
reduce the number of parameters by a factor $r$. We denote by emb, when the TC
is applied to the embedding, ff when applied to the feed-forward or the GEGLU
module, patt when applied to the pre-attention linear layers, layer when
applied both to patt and ff, and output when applied to the last linear layer
that outputs the logits. The parameter $r$ is a reduction factor, equal to the
ratio between the number of parameters in the full linear matrix, with respect
to the TC version. The bond parameter is the solution to the equation
$b(a_{1}c_{1}+a_{n}c_{n})+b^{2}\sum_{i=2}^{n-1}a_{i}b_{i}=rN_{a}N_{c}$, for a
given selection of parameters ($a_{i},c_{i}$). In our setup, the external
integer dimensions $a_{i}$ and $c_{i}$ of each tensor are chosen so as to be
close to the $n$-th root of $N_{a}$ and $N_{c}$. In principle, TC could be
used to reduce the number of parameters after training, by performing a
singular value decomposition (SVD) of the original matrix and pruning the
small singular values. Such post processing step would not reduce the memory
cost during training, but could be beneficial at inference time. In this work
we focus on using TC during training, and we use a length $n=2$ unless stated
otherwise. If the parameter reduction has minimal effect on loss, it can be
concluded that many of the parameters in the original linear layer were of no
functional importance.

##  3 Results

In this section we summarize the results of our experiments with an ablation
study of the improvements introduced above on the language translation task
from WMT14 English to German. Then, we proceed to confirm that the
improvements persist when applied to 7 new language pairs from the WMT17. All
our experiments were conducted in the same fashion: we train for a maximum of
two days with an NVIDIA V100 GPU and a batch size of 32. All our experiments
finished by early stopping on the validation loss with a patience of 10
epochs. Our baseline Transformer has a width of $d_{model}=512$, a number of
layers of $N=6$, a number of heads of $d_{h}=8$, a dropout probability of
$p_{dropout}=0.1$, and a width for the feed-forward layer of
$d_{\textit{ff}}=4d_{model}$. For simplicity we use the same tokenizer for all
the languages, which is a byte-pair encoding (Gage, 1994) with 32K sub-words,
trained on the WikiText-103 dataset (Merity et al., 2016). We use the Adam
optimizer (Kingma and Ba, 2015) with $\beta_{1}=0.9$, $\beta_{2}=0.98$,
$\epsilon=1e^{-9}$, and a learning rate of $lr=3.16e^{-5}$ without any
learning rate schedule. The learning rate was chosen after a grid search for
optimal performance of the Transformer architecture on the WMT14 development
set. To account for statistical fluctuations within the limits imposed by our
computational resources, every reported result is the mean and standard
deviation over 4 separate training experiments with different seeds. Notice
that our Transformer implementation results in 60M parameters because we used
32K sub-words, while the original 37K sub-words for the English-German pair,
results in 63M parameters, closer to the 65M reported in (Vaswani et al.,
2017).

###  3.1 Ablation study

| params | dev PPL
---|---|---
Anthe = B’ + KgV + H-SoftPOS + TC${}_{\text{{ff}:.005, {patt}:.07}}$ | 30M | $\textbf{3.5674}\pm\textbf{0.0130}$
B’ + KgV + H-SoftPOS + TC${}_{\text{{ff}:.1}}$ | 46M | $2.7690\pm 0.0048$
B’ + KgV + H-SoftPOS | 68M |<EMAIL_ADDRESS>
B’ + KgV + TC${}_{\text{{emb}:.2}}$ | 67M | $\textbf{1.4146}\pm\textbf{0.0799}$
B’ + KgV | 93M | $1.2642\pm 0.0035$
B’ = B + GEGLU | 93M | $2.5665\pm 0.0055$
B = Transformer + no-shared embeddings | 93M | $2.5987\pm 0.0155$
Transformer + shared embeddings | 60M | $3.8245\pm 0.0670$
Transformer 512 (Vaswani et al., 2017) | 65M | 4.66
Transformer 1024 (Vaswani et al., 2017) | 213M | 4.33
Table 1: Anthe on the English-to-German translation development set WMT14. We
use the same hyper-parameters as (Vaswani et al., 2017), for $d_{model}=512$,
and we report at the bottom their two best results for $d_{model}=512,1024$.
Our KgV results in better performance, while H-SoftPOS slightly improves
performance while reducing significantly the number of parameters. TC
drastically reduces the number of parameters while retaining a better
performance than the Transformer. The decrease in number of parameters with
respect of Transformer 1024 is sevenfold, while retaining an improvement in
performance.

TC length | params | dev PPL
---|---|---
2 | 33M | $3.5592\pm 0.0096$
4 | 29M | $3.7582\pm 0.1157$
3 | 33M | $3.9060\pm 0.0137$

Table 2: TCs. The Anthe variant that we use in this study is B’ + KgV +
H-SoftPOS + TC${}_{\text{{layer}:.1}}$. The length of the TC has an impact on
performance, the shortest being the best.

gate | dev PPL
---|---
B’ + KgV | $\textbf{1.2642}\pm 0.0035$
B’ + KgQ | $1.2771\pm 0.0012$
B’ + QgV | $2.5341\pm 0.0058$
B’ | $2.5665\pm 0.0055$
B’ + QgK | $2.6079\pm 0.0066$
B’ + VgK | $2.6113\pm 0.0091$
B’ + VgQ | $2.6315\pm 0.0093$

Table 3: Gatings. KgV outperforms all the other combinations of gating
mechanisms.

TC | params | dev PPL
---|---|---
B’ + KgV | 93M | $1.2642\pm 0.0035$
B’ + KgV + TC${}_{\textit{emb}:.8}$ | 86M | $1.3787\pm 0.0048$
B’ + KgV + TC${}_{\textit{emb}:.2}$ | 67M | $1.4146\pm 0.0799$
B’ + KgV + TC${}_{\textit{layer}:.2}$ | 61M | $4.4466\pm 0.0134$
B’ + KgV + TC${}_{\textit{layer}:.8}$ | 85M | $4.5861\pm 0.0157$
B’ + KgV + TC${}_{\textit{output}:.2}$ | 80M | $6.8445\pm 0.0985$
B’ + KgV + TC${}_{\textit{output}:.8}$ | 89M | $9.7301\pm 0.1794$
Table 4:  Finding the excess parameters. Parameters in the linear output layer
seem to be much more important, since when reduced by 20% and 80%, it results
in the strongest decrease in performance. However, decreasing the number of
learnable parameters in the embedding has less of an impact on the
performance. Notice the very non linear dependence with $r$.

gate | params | dev PPL
---|---|---
Anthe + no patt | 29M | $\textbf{3.2351}\pm\textbf{0.0186}$
Anthe | 30M | $3.5674\pm 0.0130$
Anthe + no patt \+ no ff | 29M | $4.3444\pm 0.0359$
Anthe + no ff | 30M | $4.9069\pm 0.0266$
Table 5: Removing or TC? Removing completely the pre-attention linear layers
improves performance with respect to using TC on them, while removing the
linear layers in the GEGLU worsens performance with respect to TC.

|  | CS/EN | DE/EN | FI/EN
---|---|---|---|---
dataset size |  | 1.6G | 8.1G | 3.9G
| params | test PPL | test PPL | test PPL
Anthe + no patt | 29M | $\textbf{4.6167}\pm\textbf{0.6501}$ | $\textbf{3.7891}\pm\textbf{0.0444}$ | $\textbf{4.2269}\pm\textbf{0.1164}$
Anthe | 30M | $\textbf{4.5259}\pm\textbf{0.5028}$ | $3.9822\pm 0.0131$ | $\textbf{4.1558}\pm\textbf{0.0202}$
AnthenoTC | 68M | $5.1874\pm 1.1889$ | $5.3560\pm 0.0480$ | $5.8806\pm 0.1741$
Transformer | 60M | $11.1995\pm 4.4999$ | $6.2168\pm 0.5581$ | $6.8927\pm 1.4090$

|  | LV/EN | RU/EN | TR/EN | ZH/EN
---|---|---|---|---|---
|  | 4.3G | 9G | 306M | 9G
| params | test PPL | test PPL | test PPL | test PPL
Anthe + no patt | 29M | $9.4998\pm 0.6706$ | $\textbf{4.2085}\pm\textbf{0.1171}$ | $4.1760\pm 0.0125$ | $\textbf{6.3080}\pm\textbf{0.2609}$
Anthe | 30M | $\textbf{8.8340}\pm\textbf{0.1074}$ | $\textbf{4.2563}\pm\textbf{0.0838}$ | $\textbf{4.1473}\pm\textbf{0.0046}$ | $\textbf{6.4599}\pm\textbf{0.0456}$
AnthenoTC | 68M | $\textbf{8.2546}\pm\textbf{0.1130}$ | $6.6637\pm 0.1169$ | $4.1697\pm 0.0190$ | $9.5552\pm 0.3533$
Transformer | 60M | $10.9181\pm 1.9593$ | $8.2324\pm 0.1390$ | $5.4590\pm 0.0262$ | $15.3343\pm 0.8857$

Table 6: Different Languages. Test perplexity on all the WMT17 language pairs.
The Anthe outperforms the Transformer in all the language pairs with just half
the parameters, with improvements in perplexity up to a factor of two, e.g.
CS/EN. We also report on experiments with complete removal of the pre-
attention linear layers which does not significantly improve Anthe, but also
does not cause degradation.

We introduce our innovations sequentially in Tab. 1, on the WMT14 English-
German dataset, validation split. For simplicity, we denote as a reference
baseline B the Transformer without weight sharing, and as B’ when we change in
B the feed-forward module by a GEGLU module. We first see that removing the
shared weights between encoder and decoder embedding and output projections,
increases the performance at the cost of an increment of $33M$ parameters, a
43%. Instead, the GEGLU module brings a minor but significant increase in
performance without an increase in parameters. The gating mechanism provided
by KgV drastically improves performance without any cost in terms of
parameters. In experiments not reported in the table, we observe an
improvement of the Transformer perplexity from $3.8245\pm 0.0670$ to
$3.4310\pm 0.0179$ with only the addition of the KgV to the original
Transformer with weight sharing between output linear layer and embeddings.
H-SoftPOS comes with a minor improvement in performance but a significant
improvement in parameters reduction. Finally, TC allows us to bring down the
number of parameters well below the original Transformer while maintaining a
better performance. We note that drastic reductions of the number of
parameters through TC eventually degrade the performance as shown in the
table. We explored a wide range of $r$ values in our experiments, and we
report the best combinations.

In contrast to the common practice of sharing the weights between embedding
and output layer, it is interesting to remark that the performance suffers the
most when the TC is applied to the linear layer at the output, while
performance suffers the least when TC is applied to the embedding, as it can
be seen in Tab. 4. This suggests that the embedding has an excess amount of
parameters that can be pruned, while the output linear layer might be more
important than often assumed. Additionally, we show in Tab. 4 that KgV
outperforms every other combination of gating mechanisms. Moreover, we find
combinations that degrade perplexity; an indication that care must be taken on
how gating is applied. Also we explore different lengths of TC in Tab. 4, and
a length of 2 gives the best results. Finally, we see in Tab. 5 that
completely removing pre-attention improves performance while the removal of ff
degrades it. However, as we have shown in Tab. 1, one can aggressively apply
TC to ff without negative effects.

###  3.2 Multiple language translation pairs

We consider all the 7 different language pairs from the WMT17 datasets in Tab.
6. We early stop on the validation set and we report on the test set. Both the
RU/EN and the ZH/EN pairs, exceeded the 9G in size, so we limited them to 9G
to make better use of our limited resources. We compare the Transformer with
our architecture with the least parameters and improved perplexity, and the
best one in terms of perplexity from Tab. 1. Remarkably the small Anthe
outperforms the Transformer in all language pairs, with only half its
parameters, and reduces the variance in the results up to 70 times, see FI/EN
pair. In addition, completely removing patt from Anthe generally causes a
small improvement, apart from the LV/EN and TR/EN language pairs.

##  4 Discussion and Conclusion

Introducing KgV, a sigmoid gating mechanism, as well as H-SoftPOS, a
hierarchical embedding layer, and TC, tensor chain representation, we were
able to significantly reduce the number of parameters required while enhancing
performance. We call Anthe the resulting architecture. Our experimental
results on both the WMT14 English-German validation set and the WMT17 test set
for seven language pairs, indicate that our proposed method outperforms
current state-of-the-art in terms of perplexity, while reducing parameter
counts by at least a factor of two.

Our analysis has confirmed that an excess of parameters exists within the
Transformer-based architectures, which we were able to identify using
H-SoftPOS and TC. In fact, and contrary to common practice, the embedding
layer can be significantly pruned without major losses in performance, while
the output linear layer needs all of its learnable parameters. We also observe
that the feed-forward layer can be pruned more than the pre-attention linear
projections. Surprisingly, our Anthe has more than half of all its parameters
in the linear readout layer.

In light of these findings, we believe that our approach holds great promise
for further advancing the field of Artificial Intelligence research in
language translation and perhaps in language modeling which could be studied
in future works.

## References

  * Beltagy et al. [2020] I. Beltagy, M. E. Peters, and A. Cohan.  Longformer: The long-document transformer.  _CoRR_ , abs/2004.05150, 2020.  URL https://arxiv.org/abs/2004.05150.
  * Brown et al. [2020] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei.  Language models are few-shot learners.  In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, _Advances in Neural Information Processing Systems_ , volume 33, pages 1877–1901. Curran Associates, Inc., 2020.  URL https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf.
  * Chen et al. [2021] B. Chen, T. Dao, E. Winsor, Z. Song, A. Rudra, and C. Ré.  Scatterbrain: Unifying sparse and low-rank attention approximation.  _CoRR_ , abs/2110.15343, 2021.  URL https://arxiv.org/abs/2110.15343.
  * Choromanski et al. [2020] K. Choromanski, V. Likhosherstov, D. Dohan, X. Song, A. Gane, T. Sarlós, P. Hawkins, J. Davis, A. Mohiuddin, L. Kaiser, D. Belanger, L. J. Colwell, and A. Weller.  Rethinking attention with performers.  _CoRR_ , abs/2009.14794, 2020.  URL https://arxiv.org/abs/2009.14794.
  * Chowdhery et al. [2022] A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al.  Palm: Scaling language modeling with pathways.  _arXiv preprint arXiv:2204.02311_ , 2022.
  * Chung et al. [2015] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio.  Gated feedback recurrent neural networks.  In _International conference on machine learning_ , pages 2067–2075. PMLR, 2015.
  * Dao et al. [2022] T. Dao, D. Y. Fu, S. Ermon, A. Rudra, and C. Ré.  FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness.  _arXiv e-prints_ , art. arXiv:2205.14135, May 2022.  doi: 10.48550/arXiv.2205.14135.
  * Gage [1994] P. Gage.  A new algorithm for data compression.  _C Users Journal_ , 12(2):23–38, 1994.
  * Gao et al. [2020] Z.-F. Gao, S. Cheng, R.-Q. He, Z. Y. Xie, H.-H. Zhao, Z.-Y. Lu, and T. Xiang.  Compressing deep neural networks by matrix product operators.  _Phys. Rev. Res._ , 2:023300, Jun 2020.  doi: 10.1103/PhysRevResearch.2.023300.  URL https://link.aps.org/doi/10.1103/PhysRevResearch.2.023300.
  * Graves et al. [2014] A. Graves, G. Wayne, and I. Danihelka.  Neural turing machines.  _arXiv preprint arXiv:1410.5401_ , 2014.
  * Graves et al. [2016] A. Graves, G. Wayne, M. Reynolds, T. Harley, I. Danihelka, A. Grabska-Barwińska, S. G. Colmenarejo, E. Grefenstette, T. Ramalho, J. Agapiou, et al.  Hybrid computing using a neural network with dynamic external memory.  _Nature_ , 538(7626):471–476, 2016.
  * He et al. [2015] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  _CoRR_ , abs/1512.03385, 2015.  URL http://arxiv.org/abs/1512.03385.
  * Hochreiter and Schmidhuber [1997] S. Hochreiter and J. Schmidhuber.  Long short-term memory.  _Neural computation_ , 9(8):1735–1780, 1997.
  * Huang et al. [2016] G. Huang, Z. Liu, and K. Q. Weinberger.  Densely connected convolutional networks.  _CoRR_ , abs/1608.06993, 2016.  URL http://arxiv.org/abs/1608.06993.
  * Jumper et al. [2021] J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko, et al.  Highly accurate protein structure prediction with alphafold.  _Nature_ , 596(7873):583–589, 2021.
  * Katharopoulos et al. [2020] A. Katharopoulos, A. Vyas, N. Pappas, and F. Fleuret.  Transformers are rnns: Fast autoregressive transformers with linear attention.  _CoRR_ , abs/2006.16236, 2020.  URL https://arxiv.org/abs/2006.16236.
  * Kingma and Ba [2015] D. P. Kingma and J. Ba.  Adam: A method for stochastic optimization.  In Y. Bengio and Y. LeCun, editors, _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , 2015.  URL http://arxiv.org/abs/1412.6980.
  * Kitaev et al. [2020] N. Kitaev, L. Kaiser, and A. Levskaya.  Reformer: The efficient transformer.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net, 2020.  URL https://openreview.net/forum?id=rkgNKkHtvB.
  * Lecun et al. [1998] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.  doi: 10.1109/5.726791.
  * Likhosherstov et al. [2020] V. Likhosherstov, K. Choromanski, J. Davis, X. Song, and A. Weller.  Sub-linear memory: How to make performers slim.  _CoRR_ , abs/2012.11346, 2020.  URL https://arxiv.org/abs/2012.11346.
  * Lin et al. [2022] T. Lin, Y. Wang, X. Liu, and X. Qiu.  A survey of transformers.  _AI Open_ , 2022.
  * Melis et al. [2019] G. Melis, T. Kočiskỳ, and P. Blunsom.  Mogrifier lstm.  In _International Conference on Learning Representations_ , 2019.
  * Merity et al. [2016] S. Merity, C. Xiong, J. Bradbury, and R. Socher.  Pointer sentinel mixture models, 2016.
  * Novikov et al. [2015] A. Novikov, D. Podoprikhin, A. Osokin, and D. P. Vetrov.  Tensorizing neural networks.  In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, _Advances in Neural Information Processing Systems_ , volume 28. Curran Associates, Inc., 2015.  URL https://proceedings.neurips.cc/paper/2015/file/6855456e2fe46a9d49d3d3af4f57443d-Paper.pdf.
  * Novikov et al. [2020] A. Novikov, P. Izmailov, V. Khrulkov, M. Figurnov, and I. Oseledets.  Tensor train decomposition on tensorflow (t3f).  _The Journal of Machine Learning Research_ , 21(1):1105–1111, 2020.
  * Oseledets [2011] I. V. Oseledets.  Tensor-train decomposition.  _SIAM Journal on Scientific Computing_ , 33(5):2295–2317, 2011.  doi: 10.1137/090752286.  URL https://doi.org/10.1137/090752286.
  * Pirvu et al. [2010] B. Pirvu, V. Murg, J. I. Cirac, and F. Verstraete.  Matrix product operator representations.  _New Journal of Physics_ , 12(2):025012, feb 2010\.  doi: 10.1088/1367-2630/12/2/025012.  URL https://dx.doi.org/10.1088/1367-2630/12/2/025012.
  * Radford et al. [2019] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9, 2019.
  * Radford et al. [2022] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever.  Robust speech recognition via large-scale weak supervision.  _arXiv preprint arXiv:2212.04356_ , 2022.
  * Sanh et al. [2019] V. Sanh, L. Debut, J. Chaumond, and T. Wolf.  Distilbert, a distilled version of bert: Smaller, faster, cheaper and lighter. arxiv 2019.  _arXiv preprint arXiv:1910.01108_ , 2019.
  * Shazeer [2020] N. Shazeer.  Glu variants improve transformer.  _arXiv preprint arXiv:2002.05202_ , 2020.
  * Simonyan and Zisserman [2015] K. Simonyan and A. Zisserman.  Very deep convolutional networks for large-scale image recognition.  In Y. Bengio and Y. LeCun, editors, _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , 2015.  URL http://arxiv.org/abs/1409.1556.
  * Vaswani et al. [2017] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin.  Attention is all you need.  In _NeurIPS_ , pages 5998–6008, 2017.
  * Verstraete et al. [2004] F. Verstraete, J. J. García-Ripoll, and J. I. Cirac.  Matrix product density operators: Simulation of finite-temperature and dissipative systems.  _Phys. Rev. Lett._ , 93:207204, Nov 2004.  doi: 10.1103/PhysRevLett.93.207204.  URL https://link.aps.org/doi/10.1103/PhysRevLett.93.207204.
  * Vidal [2003] G. Vidal.  Efficient classical simulation of slightly entangled quantum computations.  _Phys. Rev. Lett._ , 91:147902, Oct 2003.  doi: 10.1103/PhysRevLett.91.147902.  URL https://link.aps.org/doi/10.1103/PhysRevLett.91.147902.
  * Wang et al. [2022] H. Wang, S. Ma, L. Dong, S. Huang, D. Zhang, and F. Wei.  DeepNet: Scaling Transformers to 1,000 Layers.  _arXiv e-prints_ , art. arXiv:2203.00555, Mar. 2022.  doi: 10.48550/arXiv.2203.00555.
  * Wang et al. [2020] S. Wang, B. Z. Li, M. Khabsa, H. Fang, and H. Ma.  Linformer: Self-attention with linear complexity.  _CoRR_ , abs/2006.04768, 2020.  URL https://arxiv.org/abs/2006.04768.
  * White [1992] S. R. White.  Density matrix formulation for quantum renormalization groups.  _Phys. Rev. Lett._ , 69:2863–2866, Nov 1992.  doi: 10.1103/PhysRevLett.69.2863.  URL https://link.aps.org/doi/10.1103/PhysRevLett.69.2863.
  * Zaheer et al. [2020] M. Zaheer, G. Guruganesh, A. Dubey, J. Ainslie, C. Alberti, S. Ontañón, P. Pham, A. Ravula, Q. Wang, L. Yang, and A. Ahmed.  Big bird: Transformers for longer sequences.  _CoRR_ , abs/2007.14062, 2020.  URL https://arxiv.org/abs/2007.14062.


# Vertical Distribution of Optical Turbulence at the Peak Terskol Observatory
and Mount Kurapdag

Artem Yu. Shikhovtsev 1,∗ , Chun Qing 2,3,∗ , Evgeniy A. Kopylov 4, Sergey A.
Potanin5,6,
Pavel G. Kovadlo1

###### Abstract

Characterization of atmospheric turbulence is essential to understanding image
quality of astronomical telescopes and applying adaptive optics systems. In
this study, the vertical distributions of optical turbulence at the Peak
Terskol Observatory (43.27472 ° N 42.50083 ° E, 3127 m a.s.l.) using the Era-5
re-analysis, scintillation measurements and sonic anemometer data are
investigated. For the reanalysis grid node closest to the observatory,
vertical profiles of the structural constant of the air refractive index
turbulent fluctuations $C_{n}^{2}$ were obtained. The calculated
$C_{n}^{2}(z)$ vertical profiles are compared with the vertical distribution
of turbulence intensity obtained from tomographic measurements with Shack-
Hartmann sensor. The Fried parameter $r_{0}$ at the location of Terskol Peak
Observatory was estimated. Using combination of atmospheric models and scheme
paramaterization of turbulence, $C_{n}^{2}(z)$ profiles at Mt. Kurapdag were
obtained. The $r_{0}$ values at the Peak Terskol Observatory are compared with
estimated values of this length at the ten astronomical sites including Ali,
Lenghu and Daocheng.

##  1 Introduction

Adaptive optics systems correcting optical distortions in a wide field of view
require information about vertical profiles of optical turbulence and wind
speed. The key characteristics of the optical turbulence are the structural
constant of the air refractive index turbulent fluctuations $C_{n}^{2}$ and
the atmospheric coherence length, often associated with the Fried parameter
$r_{0}$ and seeing (Coulman, 1985; Tokovinin, 2023). These quantities
determine the possibilities of reaching a high angular resolution in
astronomical observations.

There are a number of techniques for measurements of optical turbulence
vertical profiles. In astronomical observations, the vertical profiles of
optical turbulence above telescope can be determined with a multidirectional
Shack–Hartmann wavefront sensor (Ran, 2024). In particular, the vertical
profiles of $C_{n}^{2}$ are calculated by applying the Solar Differential
Image Motion Monitor (S-DIMM+) or Slope Detection and Ranging (SLODAR)
techniques (Hickson, 2019, 2020; Wang Z., 2018; Subramanian, 2023;
Shikhovtsev, 2022; Joo, 2024; Bennoui, 2023; Deng, 2023; Wilson R., 2002). The
most complete review of methods for measuring optical turbulence vertical
profiles is given in a recent paper (Griffiths, 2024). Griffiths R. et al.
present the results of a intercomparison of turbulence profiling instruments
including Stereo Scintillation Detection and Ranging (S-SCIDAR), DIMM, 24-hour
Shack-Hartmann Image Motion Monitor (24hSHIMM), Ring-Image Next Generation
Scintillation Sensor (RINGSS), Multi-Aperture Scintillation Sensor-
Differential Image Motion Monitor (MASS-DIMM). It is important to emphasize
that, the comparison of these turbulence profiling instruments showed that the
largest deviations correspond to estimates of the isoplanatic angle
$\theta_{0}$, the values of which are highly sensitive to turbulence in the
upper atmospheric layers. These deviations depicted in the form of scattered
points are due to a limited accuracy in determination of the optical
turbulence strength at different atmospheric levels and, apparently,
differences in radiation propagation paths in measurements.

Measurement of optical turbulence vertical profiles is a hard procedure
(Zhang, 2022). Most often, the profile measurements are carried out only for a
separate set of nights or days. In order to estimate the optical turbulence
characteristics over long time intervals or, in other words, for a statistical
ensemble, special atmospheric models are widely used (Bolbasova, 2019; Qing,
2020; Bi, 2023; Yang, 2021; Quatresooz, 2023; Wu, 2024). The atmospheric
models are applied to estimate the characteristics of mesoscale,
micrometeorological and optical turbulence, on spatial scales comparable to
the aperture diameters of ground-based telescopes. Below, a few recent papers
demonstrating interesting results in this direction are discussed. A very
interesting approach to predict optical turbulence was demonstrated (Cuevas,
2024). Cuevas et al applied a combination of models to improve the forecasts
of $C_{n}^{2}$ profiles and seeing at Paranal using Weather Research and
Forecasting model. Also, Macatangay et al. (Macatangay, 2024) utilized
advanced numerical simulations with the Weather Research and Forecasting model
to predict astronomical seeing. Meteorological model used by the authors
reproduced variations very well, especially in comparison with smoothed DIMM
data. Using the 21-yr European Centre for Medium-Range Weather Forecasts’
fifth set of reanalysis (Era-5) data, the optical turbulence and the wind
speed characteristics were simulated at the Lenghu site (Bi, 2024). Analysis
of the optical turbulence statistics shows that calculated median value of
seeing is slightly lower than measured seeing. In particular, Era-5 derived
value is 0.72 ′′ and measured seeing is 0.75 ′′. Also, in our opinion,
significant fundamental results in the optical turbulence diagnostics and
forecasting with atmospheric models were obtained by English scientific teams:
new methods and tools were created for studying optical turbulence (Masciadri,
2001; Turchi, 2017).

For estimating optical turbulence characteristics with atmospheric models, a
key step is the selection of a small-scale turbulence parameterization scheme
(Shikhovtsev, 2024). Correct selection and adjustment of the parameterization
scheme to the atmospheric conditions of a given site are required to obtain
representative estimates of the key characteristics of optical turbulence.
Within mountainous regions, including Tibet and the Caucasus, turbulence
behavior is complex. The choice of parameterization scheme for mountain
regions should be based on analysis of preliminary measurement data. We should
emphasize also that modern science still needs to study dependencies in
turbulence evolution and determine to what extent these dependencies
corresponding to a given mountainous region can be applied for another region.

The present study focuses on the simulation of optical turbulence above the
Peak Terskol Observatory (43.27472 ° N 42.50083 ° E, 3127 m a.s.l.) and Mt.
Kurapdag (41.79609 ° N, 47.37428 ° E, 3553 m a.s.l.). These sites were chosen
as reference locations, for which data of special atmospheric measurements are
available (Bolbasova, 2023). The sites of the Terskol Peak Observatory with
high astroclimatic characteristics, Mount Kurapdag with low water vapor
content (Khaikin, 2022) as well as the Big Telescope Alt-Azimuthal Telescope
(BTA), the largest optical telescope with a non-segmented mirror in Eurasia
are shown in figure 1. (Shikhovtsev, 2024). Recent studies show that new
suitable locations for ground based telescopes include mountain regions of
Dagestan. In this regard, the figure shows the location of the mountain
village of Gunib as a place with potentially high characteristics.

In March 2023, the special experiments at the Peak Terskol Observatory have
been carried to characterize the optical turbulence. Together with the
measurements of optical turbulence profiles and meteorological characteristics
in the surface layer, variations of $C_{n}^{2}(z)$ have been simulated using a
gradient method. Below, we discuss the results of estimating optical
turbulence profiles using vertical shears of wind speed, mast meteorological
measurements and measured height distributions of optical turbulence.

##  2 Data

This study aims to estimate the image quality in nighttime at the Terskol Peak
Observatory, where optical turbulence monitoring is carried out, and at a new
location, on the slope of Mt.Kurapdag, considered as one of the reference
sites for methodological tests. In this study, we solved a number of tasks.
Firstly, using remote telescopic measurements we determined vertical
distributions of the optical turbulence intensity over the Peak Terskol
observatory. Vertical distributions of turbulence intensity were obtained
under conditions of good and mediocre image quality. Secondly, in order to
estimate the vertical profiles of the structural constant of air refractive
index turbulent fluctuations $C_{n}^{2}$, we used a gradient method, the
foundations of which have been created by Tatarsky. Using the gradient method,
the vertical profiles of optical turbulence over the Terskol Peak Observatory
and Mt. Kurapdag were determined.

Figure 1: Sites of the Terskol Peak Observatory, Big Telescope Alt-Azimuthal
Telescope (BTA), Gunib with high astroclimatic indicators and Mount Kurapdag
with low water vapor content. Red rectangle shows the BTA region

For simulation of $C_{n}(z)$ vertical profiles, we used the European Centre
for Medium-Range Weather Forecasts’ fifth set of reanalysis (Era-5) data
(Hersbach, 2020). The Era-5 reanalysis contains observational data from around
the world from 1940 to the present. Compared to previous databases, Era-5
demonstrates significant improvements in both spatial resolution and accuracy
of atmospheric characteristics (Huang, 2021; Rao, 2024). The re-analysis
contains assimilated meteorological characteristics including air temperature
and wind speed components at various pressure levels. The meteorological
characteristics are available with a flat horizontal resolution of about 30 km
and a continuous 1-hour temporal resolution. In particular, for the grid node
closest to the observatory, hourly values of air temperature and horizontal
components of wind speed at various pressure surfaces were used.

##  3 Results

###  3.1 Optical turbulence vertical distributions from Shack-Hartmann sensor
measurements at the Peak Terskol Observatory

In solar astronomy, Shack–Hartmann wavefront sensors with different fields of
views are typically employed for estimated wavefront distortions(Townson,
2015). The parameters of standard sensor depend on the optical turbulence
characteristics. In the paper, we present the results of an observing campaign
of the Shack-Hartmann sensor at the Peak Terskol Observatory in 2023 March. We
used the Shack-Hartmann sensor placed in the focal plane of MEADE LX200
Telescope installed inside the Schmidt-Cassegrain Telescope (figure 2). The
sensor is equipped with a high frame-rate FLIR Grasshopper GS3-U3-28S5M-C
camera. Subaperture size of the Shack-Hartmann sensor is 6x6 mm. Exposure time
is 5 ms. The sampling is 65 frame/sec. In experiments, we analyzed 20 full
time series and 10000 hartmannograms were obtained during the night of 19
March, 2023. Analysis of these measurement data made it possible to estimate
and analyze the character of optical turbulence above the telescope.

Figure 2: The dome of Schmidt-Cassegrain Telescope at the Peak Terskol
Observatory

Obtained data sets correspond to average seeing conditions suitable for
astronomical observations and thus were selected for processing. Assuming that
the turbulence structure obeys the Kolmogorov model, we can estimate the
height distribution of optical turbulence strength. For these calculations, we
used an expansion of the spatial spectrum of amplitude perturbations into
modes. Model spectrum of the scintillation is related to the height
distribution of optical turbulence strength:

$s_{k}=\sum_{i=1}^{Nu}W_{k}(z_{i})C_{n}^{2}(z_{i})\Delta z_{i},$ (1)

where $W_{k}(z_{i})$ is the weight function which depends on the turbulent
layer height and subaperture geometry, $\Delta z_{i}$ is the turbulent layer
thickness, $Nu$ is the number of turbulent layers. Details of the method are
described in the paper (Potanin, 2022).

y solving this equation in terms of $C_{n}^{2}(z_{i})\Delta z_{i}$ we obtain
the vertical distributions of optical turbulence strength above the Peak
Terskol Observatory. Night-time vertical distributions of optical turbulence
strength $C_{n}^{2}\Delta z$ at the Peak Terskol Observatory for different
image quality are shown in figures 3 \- 4. Image quality is estimated through
the $\beta$ parameter:

$\beta=\frac{0.98\lambda}{r_{0}},$ (2)

where variance of the differential displacements of the solar subimages
$\sigma_{\alpha}^{2}$ is related to the Fried parameter $r_{0}$ by the
following equation:

$\sigma_{\alpha}^{2}=K_{i}\lambda^{2}r_{0}^{-5/3}D^{-1/3},$ (3)

where $K_{i}$ is the numerical constant, $\lambda$ is the light wavelength and
$D$ is the telescope diameter.

Figure 3: Night-time vertical distributions of optical turbulence strength
$C_{n}^{2}\Delta z$ at the Peak Terskol Observatory for the average image
quality, 19 March, 2023. In the figure, we used a local time

(a)

(b)

Figure 4: Night-time vertical distributions of optical turbulence strength
$C_{n}^{2}\Delta z$ at the Peak Terskol Observatory for (a)) the best ($\beta$
= 0.63 ′′) and (b)) the worst atmospheric conditions ($\beta$ = 1.68 ′′), 19
March, 2023. In the figure, we used a local time

Analyzing these figures, we can see that pronounced turbulent layers are
formed at heights of 7 - 8 km and 2 - 2.5 km above the ground. The turbulent
layer at these heights leads to a significant decrease in image quality. For
example, the parameter $\beta$ determined by the integral $C_{n}^{2}$ over
height increases from 0.63 to 1.68 ′′. These changes are due to, in mainly,
generation of intensive turbulent fluctuations of air refractive index within
these layers.

It is necessary to emphasize that, conditionally, we consider three different
cases, including good seeing ($\beta$ ranges from 0.5 to 1.0 ′′), average
seeing ($\beta$ $\sim$ 1.2 ′′) and bad seeing ($\beta$ ranges from 1.5 to
2.0′′). Good seeing is associated with a thin lower turbulent layer ($\sim$ 50
- 100 m) and the absence of a layer at heights of 2 - 2.5 km. With a thickness
of the night boundary layer of 150-250 m, the seeing increases to $\sim$ 1.2
′′. Values higher than 1.5 ′′ are often associated with the development of
optical turbulence within the surface layer and an additional turbulent layer
at heights of 2 - 2.5 km.

###  3.2 Analysis of sonic measurements within the atmospheric surface layer.
Estimation of optical turbulence strength $C_{n}^{2}$

The lower part of the atmospheric boundary layer is known makes a major
contribution to the formation of integral optical turbulence (along line of
sight) and, as a consequence, to the resolving power of ground-based telescope
(Kornilov, 2014; Panchuk, 2011). Within this layer, variations in the
intensity of optical turbulence are most poorly described, both in time and in
height. At least, the accuracy of optical turbulence estimating and
forecasting based on standard atmospheric models or a small amount of optical
remote measurement data is not sufficient for planning an observing time. In
order to obtain correct shape of optical turbulence profiles within the lowest
atmospheric layers under the different atmospheric conditions we analyzed
measurement data carried out with a sonic anemometer. The sonic anemometer
”Meteo-2” has been placed on a 7-m meteorological mast at the Peak Terskol
Observatory site (figure 5). The observatory is located in the vicinity of the
Elbrus, on a flat platform at a height of 3100 m above sea level.

Below, for calculations of atmospheric turbulence characteristics, we used
long-period quasi-continuous measurement data of the sonic anemometer (from
January 2023 to December 2023). Thanks to the high frequency measurements (10
Hz) of the wind field and the sonic temperature, we calculated a number of the
turbulence characteristics including surface values of $C_{n}^{2}$.

(a)

(b)

Figure 5: a) The dome of Schmidt-Cassegrain Telescope and b) Meteo-2 sonic
anemometer Figure 6: The number of nights of sonic measurements by month, 2023

(a) Winter

(b) Spring

(c) Summer

(d) Autumn

Figure 7: Distributions of measured values of $C_{n}^{2}$ within the surface
layer of the atmosphere at the Peak Terskol Observatory site under average
atmospheric conditions, 2023

Based on the 1-year measurements with sonic anemometer, the 3 minute average
values of $C_{n}^{2}$ were obtained (Odintsov, 2019). In figure 6, we present
the number of nights sonic measurements for each month. The fewest number of
nights analyzed occurs in February-April. The largest number corresponds to
January as well as warm period of observations.

Distributions of measured values of $C_{n}^{2}$ within the surface layer of
the atmosphere for different seasons at the Peak Terskol Observatory site are
depicted in figures 7 and 8.

(a) Winter

(b) Spring

(c) Summer

(d) Autumn

Figure 8: Distributions of measured values of $C_{n}^{2}$ within the surface
layer of the atmosphere at the Peak Terskol Observatory site under clear sky
and low intensity of optical turbulence, 2023

Table 1 contains the calculated percentiles of $C_{n}^{2}$ in the surface
layer of the atmosphere. Calculated values of $C_{n}^{2}$ obtained from the
results of processing sonic measurement data are higher in comparison with
estimations derived from optical measurement data under clear sky conditions.
This fact is confirmed by previous studies (Lukin, 2015). Higher amplitudes of
$C_{n}^{2}$ are largely determined by the influence of dynamic processes on
the formation of small-scale turbulence in the surface layer of the
atmosphere. Due to the fact that astronomical observations are carried out
under good astro-optical conditions, when the intensity of optical turbulence
along the line of sight is minimal, calculated values in the upper part of
table1 differ from statistics corresponding only to clear sky conditions. In
this regard, we evaluated the statistics of surface turbulence for medium
atmospheric conditions and low intensity of optical turbulence.

Analysis of estimations shows that the intensity of turbulent fluctuations of
the air refractive index has a pronounced seasonal dependence. The highest
intensity of optical turbulence in the surface layer of the atmosphere is
observed in winter. During this period, the median values of $C_{n}^{2}$,
estimated for night time (from 0:00 to 6:00 local time), are equal to 8.6
$\cdot$ 10-15 m -2/3 and 2.6 $\cdot$ 10-15 m -2/3 for average atmospheric
condition and under clear sky, respectively. In spring, we can an increase in
the non-dimensional number of atmospheric situations $F_{c}$ with low
intensity optical turbulence. At this time, during periods of high image
quality, the median of $C_{n}^{2}$ is about 8.0 $\cdot$ 10-16 m -2/3.

In comparison with the winter and spring, in summer and autumn, turbulence in
the surface layer is significantly suppressed: the number of atmospheric
situations with strong turbulence $C_{n}^{2}\geq$ 10-14 m -2/3 does not exceed
7%. We should emphasize that the largest number of cases with the best
atmospheric conditions ($C_{n}^{2}\geq$ 10-16 m -2/3 ), occurs in summer and
autumn. In this period, about 7 - 10 % of the observing time corresponds to
weak optical turbulence.

Table 1: Percentiles of $C_{n}^{2}$ in the surface layer of the atmosphere at the Peak Terskol Observatory, 2023 Season | 10% , m-2/3 | 25% , m-2/3 | 50%, m-2/3 | 75%, m-2/3
---|---|---|---|---
Average atmospheric conditions
Winter | 2.7 $\cdot$ 10 -15 | 7.0 $\cdot$ 10 -15 | 8.6 $\cdot$ 10 -15 | 8.0 $\cdot$ 10 -14
Spring | 4.0 $\cdot$ 10 -16 | 6.8 $\cdot$ 10 -16 | 4.4 $\cdot$ 10 -15 | 1.5 $\cdot$ 10 -14
Summer | 3.0 $\cdot$ 10 -16 | 5.0 $\cdot$ 10 -16 | 1.1 $\cdot$ 10 -15 | 1.2 $\cdot$ 10 -14
Autumn | 3.4 $\cdot$ 10 -16 | 6.0 $\cdot$ 10 -16 | 3.8 $\cdot$ 10 -15 | 3.5 $\cdot$ 10 -14
Clear sky, light surface wind $V\leq$ 2.5 m/s
Winter | 5.0 $\cdot$ 10 -16 | 1.1 $\cdot$ 10 -15 | 2.6 $\cdot$ 10 -15 | 1.1 $\cdot$ 10 -14
Spring | 4.8 $\cdot$ 10 -16 | 5.7 $\cdot$ 10 -16 | 1.1 $\cdot$ 10 -15 | 3.1 $\cdot$ 10-15
Summer | 1.4 $\cdot$ 10 -16 | 4.2 $\cdot$ 10 -16 | 5.3 $\cdot$ 10 -16 | 2.2 $\cdot$ 10 -15
Autumn | 1.6 $\cdot$ 10 -16 | 3.5 $\cdot$ 10 -16 | 6.0 $\cdot$ 10 -16 | 3.0 $\cdot$ 10 -15

###  3.3 Vertical distribution of optical turbulence strength estimated from
Era-5 reanalysis

For simulation of the vertical profiles of optical turbulence, the gradient
method was used. Correction of the vertical profiles was performed using sonic
measurements and optical observations of image scintillation. The calculations
were based on the following equation:

$C_{n}^{2}(z)=\alpha L_{0}(z)^{4/3}M^{2}(z),$ (4)

where $\alpha$ is the numerical constant which equal to 2.8. The parameter
$M(z)$ is the vertical gradient of air refractive index, $L_{0}$ is the outer
scale of turbulence.

The calculation of the air refractive index gradients $M(z)$ at different
heights in the atmosphere is performed using the vertical gradients of the
natural logarithm of the air potential temperature $\frac{\partial
ln\theta}{\partial z}$:

$M=\left(\frac{-79\cdot 10^{-6}P}{T}\right)\frac{\partial ln\theta}{\partial
z},$ (5)

where $P$ is the atmospheric pressure, $T$ is the air temperature, $\theta$ is
the air potential air temperature. The potential air temperature is related to
the air temperature by the following formula:

$\theta=T\left(\frac{1000}{P}\right)^{0.286}.$ (6)

The key parameter is the outer scale of atmospheric turbulence $L_{0}$ Rao
(2023). In general, vertical changes of the outer scale $L_{0}$ can be
determined through vertical gradients of the horizontal component of the wind
velocity $S$:

$L_{0}^{4/3}=\begin{cases}0.1^{4/3}\cdot 10^{a+b\cdot
S},&\text{troposphere}\\\ 0.1^{4/3}\cdot 10^{c+d\cdot
S},&\text{stratosphere}\end{cases},$ (7)

where

$S=\left[\left(\frac{\partial u}{\partial z}\right)^{2}+\left(\frac{\partial
v}{\partial z}\right)^{2}\right]^{0.5},$ (8)

$u$ and $v$ are the horizontal components of wind velocity. Under conditions
of weak turbulence the coefficient $a$=1.64, $b$=42, $c$=0.506 and $d$=50.

Use of standard coefficients $a$=1.64, $b$=42, $c$=0.506 and $d$=50 gives
mediocre results. In particular, we calculated a vertical profile
$C_{n}^{2}(z)$ for 18 - 19 March, 2023 (figure 9 a)). By integrating this
profile using formula 9, we obtained the average estimation of $\beta$
parameter:

$\beta_{m}=\frac{0.98\lambda}{\left(0.423sec\alpha_{s}\left(\frac{2\pi}{\lambda}\right)^{2}\int_{0}^{H}C_{n}^{2}(z)dz\right)^{-3/5}},$
(9)

where $H$ is the height of the optically active atmosphere, $\alpha_{s}$ is
the zenith angle, $\lambda$ is the wavelength of light. In average, the model
value of $\beta_{m}$ is higher than the measured $\beta$ and and the estimated
$C_{n}^{2}$ in the upper atmospheric layers are higher than expected values.
In order to correct the vertical profile of $C_{n}^{2}(z)$ we calculated new
parameterization coefficients $a_{new}$ and $b_{new}$ by finding the best
fitting model linking measured and model near-surface values of $C_{n}^{2}$.

(a) $a$=1.64, $b$=42, $r_{0}$ = 5.66 cm, $\beta$ = 1.79 arc sec

(b) $a_{new}$=0.95, $b_{new}$=10, $r_{0}$ = 8.20 cm, $\beta$ = 1.23 arc sec

Figure 9: Calculated vertical profile of optical turbulence at the Peak
Terskol Observatory, 18 - 19 March 2023. Shading corresponds to the interval
between the first and the third quartiles estimated for all nights in March
2023

Specifically, new coefficients were calculated by minimizing the standard
deviation between the model values $C_{n}^{2}$ and the measured, sonic, values
of $C_{n}^{2}(m)$:

$\sum_{i}^{T_{L}}(C_{ni}^{2}(m)-C_{ni}^{2})^{2}\rightarrow min,$ (10)

where $C_{ni}^{2}(m)$ is the measured value of $C_{n}^{2}$ in the surface
layer, $C_{ni}^{2}$ is the model (calculated) value of $C_{n}^{2}(surf)$.
Summation is performed over the entire time implementation, the length of
which is $T_{L}$. Using this minimization condition, the weight coefficients
$a_{new}$ and $b_{new}$ are estimated. New parametrization coefficients at the
Terskol Peak Observatory differ significantly from standard values:
$a_{new}$=0.95, $b_{new}$=10. Figure 9 b) shows corrected vertical profile of
$C_{n}^{2}$ with the new parametrization coefficients. This profile
corresponds to 19 March, 2023.

One of the most important characteristics of optical turbulence is the height
of atmospheric boundary layer (ABL). For determination of the ABL height we
have used a well-known method based on the profile of the bulk Richardson
number. The ABL height is defined as the height where the bulk Richardson
number exceeds a critical threshold. From figure 9, we can see that the new
coefficients used to parameterize the night variations of $C_{n}^{2}$ made it
possible to reduce the height of ABL. Obtained night-time heights of the
atmospheric boundary layer ranges from 100 to 300 m. As a result, the
estimated strength of optical turbulent fluctuations decreases more rapidly
with height at night above Terskol Peak. We associate this decrease of
turbulent fluctuations strength within the lower atmospheric layer above the
Terskol Peak with the low heights of the atmospheric boundary layer at night.
Considering a large region, the minimum height of the atmospheric boundary
layer is observed above Terskol peak not only on individual nights, but also
on average, over long time period (figure 10). The high quality of
astronomical images at the Terskol Peak is related to the low height of the
atmospheric boundary layer. On some nights, its height is limited by 50 - 100
m. Mt Kurapdag and Big Telescope Alt-Azimuthal are located in worse
atmospheric conditions, the mean night-time heights of the atmospheric
boundary layer are $\sim$ 250 - 300 m and 250 - 350 m respectively (figure 10
a)). We believe that observed character of vertical changes is in good
agreement with theoretical and experimental data on the evolution of the
atmospheric boundary layer.

(a) Average atmospheric conditions, 2013 - 2023

(b) 19 March, 2023

Figure 10: Distributions of atmospheric boundary layer height (in meters)
within the Peak Terskol Observatory region.

The data of optical and meteorological measurements as well as the fields of
wind speed and air temperature extracted from the re-analysis Era-5 , served
as the basis for calculation of the average profiles of optical turbulence
above the observatory and Mt.Kurapdag. In figure 11 a) we demonstrate
calculated vertical profiles of optical turbulence above the Peak Terskol
Observatory averaged over January, 2023 and June, 2023.

Analyzing these profiles, it can be noted that the structure of turbulence in
the upper atmospheric layers changes significantly in summer. This fact is
indicated by deformations of vertical profiles of optical turbulence. On the
one hand, these deformations manifest in a greater vertical uniformity of
optical turbulence and suppression of turbulent fluctuations in the surface
layer of the atmosphere. On the other hand, the turbulence strength in the
upper atmospheric layers decreases in summer.

(a) Peak Terskol Observatory, 2023. Blue line corresponds to January. Orange
line corresponds to June. Shading corresponds to the interval between the
first and the third quartiles

(b) Mt. Kurapdag, 2013 - 2023. Blue line corresponds to the Peak Terskol
Observatory. Orange line corresponds to Mt. Kurapdag.

Figure 11: Night-time vertical profiles of optical turbulence above the Peak
Terskol Observatory and Mt.Kurapdag

Also, using Era-5 reanalysis data and identified patterns in optical
turbulence behaviour at the Peak Terskol Observatory site we calculated
averaged vertical profiles of $C_{n}^{2}(z)$ above the Mt. Kurapdag. For
comparison, the figure 11 b) shows vertical profiles above Terskol Peak and
Kurapdag Mountain over 2013 - 2023. Analysis of these vertical profiles
indicates the presence of an intense lower turbulence layer. It should be
noted that within the surface layer, the intensity of turbulence, in terms of
$C_{n}^{2}$, at the Mt. Kurapdag site exceeds by more than an order of
magnitude the values of $C_{n}^{2}$ observed at the Peak Terskol Observatory.
Moreover, at the Mt. Kurapdag site, turbulence in this atmospheric layer is
characterized by high intensity regardless of the parameterization
coefficients. The nature of the turbulence is largely due to strong vertical
wind speed shears near the ground.

##  4 Estimation of the Fried parameter at the sites with good astroclimatic
conditions

Figure 12 displays the monthly averaged Fried parameter estimated using Era-5
reanalysis data at the ten astronomical sites. European Southern Observatory
and Tibetan Plateau and the Ethiopian Plateau are some of the most good sites.
In contrast to the Fried parameter of the European Southern Observatory and
Tibetan Plateau (TP) sites, the Ethiopian Plateau has a different pattern,
possibly owing to different climate patterns. During the year, the Ethiopian
Plateau atmospheric coherence length was high in winter and low in summer. In
winter, atmospheric coherence length at the Ethiopian Plateau is better than
that at the ESO and TP sites, indicating that the Ethiopian Plateau has high
potential conditions that are comparable to some of the best astronomical
observatories in the world. Estimated Fried parameter at the Ali site located
in western China and the Tibetan plateau, one of the best astronomical sites,
is about 20 cm in the best months. These sites are presently under studies for
astronomy Hickson (2020). The Era-5 derived values of the Fried parameter
corresponding to spatial coherence length are consistent with measured values
at the Maidanak Observatory (Tillayev, 2023).

For comparison, we also compare the variation of the Fried parameter at the
site of Peak Terskol. Figure 13 shows variations of the Fried parameter
$r_{0}$ at the Terskol Peak Observatory. As you can see from the figure, the
site of Terskol Peak has a high image quality close to the best places in the
world. In particular, in summer $\beta$ is about 1.0 - 1.2 arc sec, in terms
of $r_{0}$, these values correspond to 8.3 - 10.0 cm. In addition to the
average monthly values of the Fried parameter obtained using the gradient
method, the figure shows the results of applying the Hufnagel–Vally (HV)
climate model to describe vertical changes in $C_{n}^{2}$ (Mahmood, 2022). The
HV model is based on the processing of experimental data and, on average,
describes the structure of optical turbulence well. Nevertheless, detailed
information about the vertical distribution of optical turbulence cannot be
reflected using this model. Analysis of the figure shows that the gradient
model and the model give similar results in terms of $r_{0}$.

Figure 12: Variations of the Fried parameter $r_{0}$=$0.98\lambda/\beta$
Figure 13: Variations of the Fried parameter $r_{0}$ at the Terskol Peak
Observatory

##  5 Discussion

In this paper, we present the results of studies of the optical turbulence in
the atmospheric boundary layer and free atmosphere, up to a height of 20 km.
In order to determine the structural constant of turbulent fluctuations of the
air refractive index, we refined the parameterization scheme for calculation
of this quantity. In the region of interest, this turbulent parameter was
calculated based on a comparison of reanalysis data, mast meteorological
measurements and observations of image scintillation. In particular, the
processing of mast measurement data made it possible not only to estimate
$C_{n}^{2}$ statistics in the atmospheric surface layer, but also to select
optimal parameterization coefficients that determine the contribution of wind
speed shear instability to the formation of turbulence. Using these
coefficients, we estimated $C_{n}^{2}$ values at different heights in the
atmospheric boundary layer more accurately. This made it possible to refine
the shapes of the vertical profiles of optical turbulence. The correctness of
our approach based on the using constant parametrization coefficients (within
only atmospheric boundary layer) is supported by the fact that the structure
of turbulence in the atmospheric boundary layer should be determined by
processes in the surface layer of the atmosphere, at least for long time
periods.

Correctness of the calculations is also confirmed by the results of estimating
the height of the atmospheric boundary layer. Figures 10 show distributions of
the atmospheric boundary layer height (in meters) within the Peak Terskol
Observatory region. Analysis of these distributions allows us to notice that a
fairly thin boundary layer is formed above the Observatory. This suggests that
turbulence develops predominantly in the layer near the ground. During the
measurements, the thickness of the atmospheric boundary layer was minimal
(compared to average atmospheric conditions (10 a)).

It can be noted that the gradient method outputs are in good agreement with
the data of optical measurements of turbulence at different height levels in
the atmosphere, at least for the Terskol Peak Observatory. Based on the
selected profile calculation scheme for the Terskol Peak Observatory, we
estimated the optical turbulence characteristics for other sites as well.
Analysis of spatial distribution of the atmospheric boundary layer height, as
well as calculated profiles of optical turbulence, can serve as the basis for
choosing optimal site for a ground-based astronomical telescope. Taking into
account the recent data on the spatial distribution of astroclimatic
characteristics within this macroregion, the following areas suitable for
optical telescopes are located: in mountain Dagestan, including Gunib, as well
as Shalbuzdag and Peak Terskol Observatory. The location of Mt. Kurapdag is
not good due to strong surface winds and strong surface optical turbulence,
which is the main reason of high values of $\beta$ (2- 3 ′′).

##  6 Conclusions

In this work we demonstrated that, using the gradient method and additional
measurement data, we can estimate the vertical profiles of optical turbulence
more accurately. The increase of accuracy of optical turbulence profiles
within the lower atmosphere is due to using both measured values of
$C_{n}^{2}$ and applying new parametrization coefficients between vertical
shears of wind speed and model values of $C_{n}^{2}$. Moreover, the gradient
method takes into account variations in the height of the atmospheric boundary
layer estimated from re-analysis data. Estimations of the atmospheric boundary
layer height and the vertical profiles of wind shears are important for
obtaining smooth vertical profiles of $C_{n}^{2}$ and $L_{0}$. Above the
atmospheric boundary layer, vertical profiles of optical turbulence are
reconstructed only on the basis of vertical profiles of wind speed shears with
parameterization coefficients $a$=1.64 and $b$=42, that describe $C_{n}^{2}$
variations well for weak turbulence. Thus, we can conclude that the hypotheses
of optical turbulence embedded in the gradient method are valid as, in
average, used atmospheric models converge to the measurements: the measured
and model values of $\beta$ are practically identical.

For the first time, by linking simulation outputs to measurement data, we have
obtained representative optical turbulence vertical profiles for the sites of
Peak Terskol Observatory and the Mt. Kurapdag. We have shown that the Kurapdag
mountain should not be considered as a potential site for a new optical
telescope.

## References

  * Coulman (1985) Coulman, C.E. Fundamental and applied aspects of astronomical “seeing”. Annu. Rev. Astron. Astrophys. 1985, 23, 19-57. Doi:10.1146/annurev.aa.23.090185.000315.
  * Tokovinin (2023) Tokovinin, A. The Elusive Nature of “Seeing”. Atmosphere 2023, 14, 11, 1694. Doi:10.3390/atmos14111694.
  * Ran (2024) Ran, X.; Zhang, L.; Rao, C. The AC-SLODAR: measuring daytime normalized optical turbulence intensity distribution based on slope autocorrelation. Mnras 2024, 528, 3, 3981-3991. Doi:10.1093/mnras/stae202.
  * Hickson (2019) Hickson, P.; Ma, B.; Shang, Z.; Xue, S. Multistar turbulence monitor: a new technique to measure optical turbulence profiles. Mnras 2019, 485, 2, 2532 – 2545, doi:10.1093/mnras/stz568.
  * Hickson (2020) Hickson, P.; Feng, L.; Hellemeier, J. A.; Shen, Z.; Xue, S.; Yao, Y.; Ma, B.; Chen, H.; Yang, R. Optical turbulence at Ali, China – results from the first year of lunar scintillometer observation. Mnras 2020, 494, 4, 5992 – 6000, doi:10.1093/mnras/staa1101.
  * Wilson R. (2002) Wilson, R. SLODAR: Measuring optical turbulence altitude with a Shack–Hartmann wavefront sensor. Mon. Not. R. Astron. Soc. 2002, 337, 103–108.
  * Deng (2023) Deng, J.; Song, T.-F.; Liu, Y. A Review of Daytime Atmospheric Optical Turbulence Profile Detection Technology. Chinese Astronomy and Astrophysics 2023, 47, 2, 257–284. Doi:10.1016/j.chinastron.2023.06.001.
  * Wang Z. (2018) Wang, Z.; Zhang, L.; Kong, L.; Bao, H.; Guo, Y.; Rao, X.; Zhong, L.; Zhu, L.; Rao, C. A modified S-DIMM+: Applying additional height grids for characterizing daytime seeing profiles. Mon. Not. R. Astron. Soc. 2018, 478, 1459–1467.
  * Subramanian (2023) Subramanian, S.K.; Rengaswamy, S. Forward modelling of turbulence strength profile estimation using S-DIMM+. Proceedings of SPIE 2023, 12638, 1263812. Doi:10.1117/12.2671735.
  * Shikhovtsev (2022) Shikhovtsev, A.Yu. A Method of Determining Optical Turbulence Characteristics by the Line of Sight of an Astronomical Telescope. Atmospheric and Oceanic Optics 2022, 35, 3, 303-309. Doi:10.1134/S1024856022030149.
  * Joo (2024) Joo, J.Y.; Ha, H.S.; Lee, J.H.; Kim, Y.S.; Butterley, T. SLODAR System Development for Vertical Atmospheric Disturbance Profiling at Geochang Observatory. Current Optics and Photonics 2024, 8, 1, 30-37. Doi:10.3807/COPP.2024.8.1.30.
  * Bennoui (2023) Bennoui, F.; Bahloul, D. Estimation of the Atmospheric Turbulence Parameters Using the Angle-of-Arrival Covariance Function. Atmospheric and Oceanic Optics 2023, 36, 5, 569-577. Doi:10.1134/S1024856023050202.
  * Griffiths (2024) Griffiths, R.; Bardou, L.; Butterley, T.; Osborn, J.; Wilson, R.; Bustos, E.; Tokovinin, A.; Le Louarn, M.; Otarola, A. A comparison of next-generation turbulence profiling instruments at Paranal. Monthly Notices of the Royal Astronomical Society 2024, 529, 1, 320–330. Doi:10.1093/mnras/stae434.
  * Zhang (2022) Zhang, K.; Luo, T.; Wang, F.-F.; Sun, G.; Liu, Q.; Qing, C.; Li, X.-B.; Weng, N.-Q.; Zhu, W.-Y. Influence of low clouds on atmospheric refractive index structure constant based on radiosonde data. Acta Physica Sinica 2022, 71, 089202. Doi:10.7498/aps.71.20211792.
  * Bolbasova (2019) Bolbasova, L.A.; Shikhovtsev, A.Yu.; Kopylov, E.A.; Selin, A.A.; Lukin, V.P.; Kovadlo, P.G. Daytime optical turbulence and wind speed distributions at the Baikal Astrophysical Observatory. Monthly Notices of the Royal Astronomical Society 2019, 482, 2619–2626. Doi:10.1093/mnras/sty2706.
  * Qing (2020) Qing, C.; Wu, X.; Li, X.; Luo, T.; Su, C.; Zhu, W. Mesoscale optical turbulence simulations above Tibetan Plateau: first attempt. Optics Express 2020, 28, 4, 4571–4586, doi:10.1364/OE.386078.
  * Bi (2023) Bi, C.; Qing, C.; Qian, X.; Luo, T.; Zhu, W.; Weng, N. Investigation of the Global Spatio-Temporal Characteristics of Astronomical Seeing. Remote Sens. 2023, 15, 2225, doi:10.3390/rs15092225.
  * Yang (2021) Yang, Q.; Wu, X.; Han, Y.; Chun, Q.; Wu, S.; Su, C.; Wu, P.; Luo, T.; Zhang, S. Estimating the astronomical seeing above Dome A using Polar WRF based on the Tatarskii equation. Opt. Express 2021, 29, 44000–44011. http://doi.org/10.1364/OE.439819.
  * Quatresooz (2023) Quatresooz, F.; Griffiths, R.; Bardou, L.; Wilson, R.; Osborn, J.; Vanhoenacker-Janvier, D.; Claude, O. Continuous daytime and nighttime forecast of atmospheric optical turbulence from numerical weather prediction models. Optics Express 2023, 31, 21, 33850–33872. Doi:10.1364/OE.500090.
  * Wu (2024) Wu, X.-Q.; Xiao, C.-Y.; Esamdin, A.; Xu, J.; Wang, Z.-W.; Xiao, L. Quantitative Analysis of Seeing with Height and Time at Muztagh-Ata Site Based on ERA5 Database. Research in Astronomy and Astrophysics 2024, 24, 1, 015006. Doi:10.1088/1674-4527/ad057d.
  * Cuevas (2024) Cuevas, O.; Marin, J.C.; Blazquez, J.; Meyer, C. Combining $C_{n}^{2}$ models to forecast the optical turbulence at Paranal. Monthly Notices of the Royal Astronomical Society 2024, 529, 3, 2208 - 2219. Doi:10.1093/mnras/stae630.
  * Bi (2024) Bi, C.; Qing, C.; Qian, X.; Zhu, W.; Luo, T.; Li, X.; Cui, S.; Weng, N. Astroclimatic parameters characterization at lenghu site with ERA5 products. Monthly Notices of the Royal Astronomical Society 2024, 527, 3, 4616 - 4631, doi:10.1093/mnras/stad3414.
  * Macatangay (2024) Macatangay, R.; Rattanasoon, S.; Butterley, T.; Bran, S. H.; Sonkaew, T.; Sukaum, B.; Sookjai, D.; Panya, M.; Supasri, T. Seeing and turbulence profile simulations over complex terrain at the Thai National Observatory using a chemistry-coupled regional forecasting model. Monthly Notices of the Royal Astronomical Society 2024, 530, 2, 1414 - 1423, doi:10.1093/mnras/stae727.
  * Masciadri (2001) Masciadri, E.; Jabouille, P. Improvements in the optical turbulence parameterization for 3D simulations in a region around a telescope. Astronomy and Astrophysics 2001, 376, 2, 727 – 734, doi:10.1051/0004-6361:20010999.
  * Turchi (2017) Turchi, A.; Masciadri, E.; Fini, L. Forecasting surface-layer atmospheric parameters at the Large Binocular Telescope site. Monthly Notices of the Royal Astronomical Society 2017, 466, 2, 1925 - 1943, doi:10.1093/mnras/stw2863.
  * Shikhovtsev (2024) Shikhovtsev, A. Reference optical turbulence characteristics at the Large Solar Vacuum Telescope site. Publications of the Astronomical Society of Japan 2024, psae031, doi:10.1093/pasj/psae031.
  * Bolbasova (2023) Bolbasova, L.A.; Kopylov E.A. Long-Term Trends of Astroclimatic Parameters above the Terskol Observatory. Atmosphere 2023, 14, 8, 1264. Doi:10.3390/atmos14081264.
  * Khaikin (2022) Khaikin V.B.; Shikhovtsev A.Yu.; Shmagin V.E.; Lebedev M.K.; Kopylov E.A.; Lukin V.P.; Kovadlo P.G. Eurasian Submillimeter Telescopes (ESMT) project. Possibility of submm image quality improvement using adaptive optics. Zhurnal radioelektroniki [Journal of Radio Electronics] 2022, 7, Doi:10.30898/1684-1719.2022.7.9.
  * Shikhovtsev (2024) Shikhovtsev, A.Yu.; Kovadlo P.G. Statistical estimations of the vapor content and optical thickness of the atmosphere using reanalysis and radiosonding data as applied to millimeter telescopes. Optika Atmosfery i Okeana 2024, 37, 2, 169-175. Doi:10.15372/AOO20240212.
  * Hersbach (2020) Hersbach, H.; Bell, B.; Berrisford, P.; Hirahara, S.; Horányi,A.; Muñoz-Sabater, J.; Nicolas,J.; Peubey, C.; Radu, R.; Schepers, D.; Simmons, A.; Soci, C.; Abdalla, S.; Abellan, X.; Balsamo, G.; Bechtold, P.; Biavati, G.; Bidlot, J.; Bonavita, M.; De Chiara, G.; Dahlgren, P.; Dee, D.; Diamantakis, M.; Dragani, R.; Flemming, J.; Forbes, R.; Fuentes, M.; Geer, A.; Haimberger, L.; Healy, S.; Hogan, R.J.; Hólm, E.; Janisková, M.; Keeley, S.; Laloyaux,P.; Lopez,P.; Lupu, C.; Radnoti, G.; de Rosnay, P.; Rozum, I.; Vamborg, F.; Villaume, S.; Thépaut, J.-N. The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society 2020, 146, 730, 1999-2049, doi: 10.1002/qj.3803.
  * Huang (2021) Huang, J.; Yin, J.; Wang, M.; He, Q.; Guo, J.; Zhang, J.; Liang, X.; Xie, Y. Evaluation of Five Reanalysis Products With Radiosonde Observations Over the Central Taklimakan Desert During Summer. Earth and Space Science 2021, 8, 5, 2021EA001707. Doi:10.1029/2021EA001707.
  * Rao (2024) Rao, P.; Wang, F.; Yuah, X.; Liu, Y.; Jiao, Y. Evaluation and comparison of 11 sets of gridded precipitation products over the Qinghai-Tibet Plateau. Atmospheric Research 2024, 302, 107315, doi:10.1016/j.atmosres.2024.107315.
  * Townson (2015) Townson, M.J; Kellerer A.; Saunter C.D. Improved shift estimates on extended Shack–Hartmann wavefront sensor images. Monthly Notices of the Royal Astronomical Society 2015, 452, 4022–4028, doi:10.1093/mnras/stv1503.
  * Potanin (2022) Potanin, S.A.; Kornilov, M.-V.; Savvin, A.D.; Safonov, B.S.; Ibragimov, M.A.; Kopylov, E.A.; Nalivkin, M.A.; Shmagin, V.E.; Huy, L.X.; Thao, N.T. A Facility for the Study of Atmospheric Parameters Based on the Shack–Hartmann Sensor. Astrophysical Bulletin 2022, 77, 214–221. Doi:10.1134/S1990341322020067.
  * Kornilov (2014) Kornilov V.; Safonov B.; Kornilov M.; Shatsky N.; Voziakova O.; Potanin S.; Gorbunov I.; Senik V.; Cheryasov D. Study on atmospheric optical turbulence above mount Shatdzhatmaz in 2007-2013. Publications of the Astronomical Society of the Pacific 2014, 126, 939, 482-495, doi:10.1086/676648.
  * Panchuk (2011) Panchuk V.E.; Afanas’ev V.L. Astroclimate of Northern Caucasus-myths and reality. Astrophysical Bulletin 2011, 66, 2, 233-254, doi:10.1134/S199034131102009X.
  * Odintsov (2019) Odintsov, S.L.; Gladkikh, V.A.; Kamardin, A.P.; Nevzorova, I.V. Determination of the Structural Characteristic of the Refractive Index of Optical Waves in the Atmospheric Boundary Layer with Remote Acoustic Sounding Facilities. Atmosphere 2019, 10, 711. doi:10.3390/atmos10110711.
  * Lukin (2015) Lukin, V.P.; Botygina, N.N.; Gladkikh, V.A.; Emaleev, O.N.; Konyaev, P.A.; Odintsov, S.L.; Torgaev, A.V. Joint measurements of atmospheric turbulence level with optical and acoustic meters. Atmospheric and Oceanic Optics 2015, 28, 3, 254 - 257. Doi: 10.1134/S1024856015030100.
  * Rao (2023) Rao, R. Effect of Outer Scale of Atmospheric Turbulence on Imaging Resolution of Large Telescopes. Guangxue Xuebao/Acta Optica Sinica 2023, 43, 24, 2400001, doi:10.3788/AOS231259.
  * Tillayev (2023) Tillayev, Y.; Azimov, A.; Ehgamberdiev, S.; Ilyasov, S. Astronomical Seeing and Meteorological Parameters at Maidanak Observatory. Atmosphere 2023, 14, 199, doi:10.3390/atmos14020199.
  * Mahmood (2022) Mahmood, D.A.; Naif, S.S.; Al-Jiboori, M.H.; Al-Rbayee, T. Improving Hufnagel-Andrews-Phillips model for prediction $C_{n}^{2}$ using empirical wind speed profiles. JASTP 2022, 240, 105952, doi:10.1016/j.jastp.2022.105952.