# CONFLATOR: Incorporating Switching Point based Rotatory Positional Encodings
for Code-Mixed Language Modeling

Mohsin Ali1 Kandukuri Sai Teja1 Neeharika Gupta2 Parth Patwa3
Anubhab Chatterjee2 Vinija Jain4 Aman Chadha4,5* Amitava Das6
1IIIT Sri City, India 2Wipro AI Labs, India 3University of California Los
Angeles, USA
4Stanford University, USA 5Amazon AI, USA 6University of South Carolina, USA
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

The mixing of two or more languages is called Code-Mixing (CM). CM is a social
norm in multilingual societies. Neural Language Models (NLMs) like
transformers have been very effective on many NLP tasks. However, NLM for CM
is an under-explored area. Though transformers are capable and powerful, they
cannot always encode positional/sequential information since they are non-
recurrent. Therefore, to enrich word information and incorporate positional
information, positional encoding is defined. We hypothesize that Switching
Points (SPs), i.e., junctions in the text where the language switches (L1
$\rightarrow$ L2 or L2 $\rightarrow$ L1), pose a challenge for CM Language
Models (LMs), and hence give special emphasis to switching points in the
modeling process. We experiment with several positional encoding mechanisms
and show that rotatory positional encodings along with switching point
information yield the best results.

We introduce CONFLATOR: a neural language modeling approach for code-mixed
languages. CONFLATOR tries to learn to emphasize switching points using
smarter positional encoding, both at unigram and bigram levels. CONFLATOR
outperforms the state-of-the-art on two tasks based on code-mixed Hindi and
English (Hinglish): (i) sentiment analysis and (ii) machine translation.

11footnotetext: Work does not relate to position at Amazon.

##  1 Code-Mixing: Juxtaposition of two Languages

Code-mixing is defined as the alternation of two or more languages during
articulation. Recently, code-mixing has gained a lot of attention in the area
of NLP due to the prevalence of language mixing in multilingual societies such
as India, Europe, US, South Africa, Mexico, etc. In such societies, code-
mixing is fairly commonplace, especially in informal conversations, where the
native language is often romanized and code-mixed with an auxiliary language.
This effect occasionally manifests in posts originating from the
aforementioned sources on social media platforms such as Twitter, Facebook,
etc. An example of Hindi and English code-mixing is shown in the following
phrase where an English word, dance, is mixed with Hindi romanized words:
Gaaye, aur, kare.

> GaayeHI aurHI danceEN kareHI
> English translation: sing and dance

With the proliferation of code mixing on the internet, it is important to
study language processing and language modeling for code-mixed languages.
While language modeling using neural networks has come a long way, replacing
n-gram language models with distributed neural representations Bengio et al.
(2003) to recent large transformer-based pre-trained language models (LMs)
such as GPT-x Radford et al. (2019), BERT Devlin et al. (2018a) etc., code-
mixed language modeling using state-of-the-art (SoTA) Transformer-based models
is still underexplored.

The biggest hindrance in the adoption of SoTA Transformer-based LMs for code-
mixing can be attributed to data scarcity. While Transformer-based Vaswani et
al. (2017b) architectures such as BERT and GPT have set new benchmarks in the
domain of language modeling, they are infamous for their low sample
efficiency. In other words, the voracious data appetite of Transformers and
the lack of substantial code-mixed datasets in the community is the primary
reason for the technological hindrances in the area of code-mixed language
modeling compared to vanilla language modeling.

To corroborate the aforementioned arguments, we experiment with Transformer-
based models such as GPT-2 and BERT for code-mixing. We empirically observe
that these models perform poorly on tasks involving code-mixed data. Our
hypothesis is as follows: Since information related to switching point is a
major component in the context of code mixed context, it should thus be
incorporated in downstream processing. Switching points are the bottleneck for
a model’s processing of code-mixed data and the reason for poor performance
using SoTA neural language models Chatterjere et al. (2020). Switching points
play a crucial factor when dealing with CM data. In the next few sections, we
discuss various positional encoding approaches, switching points, and our
approaches for language modeling on code-mixed data. Our key contributions
are:

  * •

We propose CONFLATOR, an LM system that incorporates switching point related
positional information.

  * •

Our system improves the performance of existing models and achieves a new
SoTA.

  * •

We investigate, experiment with, and introduce various switching point based
positional encoding techniques.

  * •

We introduce a novel Switching Point based Rotary matrix for Rotary Positional
Encoding (RoPE).

##  2 Related Work

It is important to study code-mixing as it is a part of most multilingual
societies and prevalent in social media. It is more complex to process code-
mixed text than monolingual text for NLP tasks Verma (1976). Similar line of
work was followed by Bokamba (1988) and Singh (1985) on the complexities of
multi-languages on the basis of syntactics and grammar. The difficulties of
processing code-mixed languages on social media is further exacerbated by
unusual spellings, many unique ways of writing the same word, unnecessary
capitalization etc Das and Gambäck (2014); Laddha et al. (2020).

With the growing popularity on social media, Various tasks like sentiment
analysis Patwa et al. (2020a); Chakravarthi et al. (2020), translation Dhar et
al. (2018); Srivastava and Singh (2020), hatespeech detection Bohra et al.
(2018); Banerjee et al. (2020), POS tagging Vyas et al. (2014), etc. have been
performed on code-mixed data. Methods to handle code-mixing for text
classification include the use of CNNs Aroyehun and Gelbukh (2018); Patwa et
al. (2020b), Transformer or BERT like models Samghabadi et al. (2020); Tang et
al. (2020), ensemble models Tula et al. (2021); Jhanwar and Das (2018), focal
loss Tula et al. (2022); Ma et al. (2020) etc.

Vaswani et al. (2017a) proposed transformers for neural language modeling
using masked language modeling (MLM) and next sentence prediction, which
achieved SoTA performance on many NLP tasks. Devlin et al. (2018b) released
mBERT, a model trained on multilingual corpus that includes 104 languages. A
cross lingual language model XLM was proposed in Lample and Conneau (2019)
which leveraged monolingual and crosslingual corpus for pretraining. Nayak and
Joshi (2022) present a bert pretrained on CM data. However, they do not make
changes to their language model or technique to handle code-mixed data in
particular. Sengupta et al. (2021) propose a Hierarchical transformer based
architecture that captures the semantic relationship among words and
hierarchically learns the sentence level semantics of code-mixed data. Ali et
al. (2022) Were one of the first to incorporate switching point information in
positional encoding. They utilize dynamic positional encodings whereas our
method, CONFLATOR infuses switching point information in rotatory positional
encodings and also uses both unigram and bigram tokens to get the final
embedding.

##  3 Data Extraction and Strategies

In this section, we discuss the details of code-mixed data extraction. Our
primary aim is to extract naturally distributed code-mixed data.

###  3.1 Qualitative and Quantitative Checkpoints for Hinglish Corpus

The performance of LMs is dependent on the training data size and quality,
along with the vocabulary size. Code-mixed language modeling suffers from the
following challenges: i) data scarcity, ii) Words from 2 (or more) languages
in the same sentence, iii) Hindi is written using English letters (i.e.
transliteration), hence, there is no standardization of spelling - which in
effect proliferates word forms Laddha et al. (2020, 2022), iii) Code-mixing is
usually found on social media and netizens often incorporate creativity in
their mixing along with wordplay. We consider two fundamental questions to
guide our data collection:

  1. 1.

The performance on any NLP task depends on the data complexity:

Empirical measurement: Consider two 4-word tweets - i) $T_{i}$ :
$w_{L1}w_{L1}w{L2}w{L2}$ and ii) $T_{j}$ : $w_{L1}w_{L2}w_{L1}w_{L2}$. Both
the tweets have 2 words each from the languages $L1$ and $L2$. Thus the mixing
ratio of both the tweets $T_{i}$ and $T_{j}$ is $(4-2)/4=0.50$. However,
$T_{i}$ only contains 1 code alternation point whereas $T_{j}$ contains 3
switches. It is likely that $T_{j}$ is harder to process. Hence, we need a
metric for the level of mixing between the languages. We use Code-Mixing-Index
Gambäck and Das (2016) (CMI) to measure such complexity. Please refer to
section 3.2 for more details on CMI.

  2. 2.

How much data is good enough?

Empirical measurement: When two languages blend, it is quite natural that the
number of unique word forms would be much higher in a Hinglish corpus in
comparison to monolingual English or Hindi corpus. Therefore, we ask an
essential question at the very beginning, how much data is good enough? We
decide to keep collecting data, until the Heaps’ curve starts converging so
that we cover most of the unique words.

Heaps’ law Gopalan and Hopkins (2020) states that the number of unique words
in a text of $n$ words is approximated by $V(n)=Kn^{\beta}$ where $K$ is a
positive constant and $\beta$ lies between $0$ and $1$, $K$ invariably lies
between $10$ and $100$ and $\beta$ between $0.4$ an $0.6$. Heaps’ law is often
considered to be a good estimator to calculate the vocabulary size. To
compare, from the figure 1, it can be seen that, for English Wiki, the
flattening of the Heaps’ law curve, starts at 40K-50K, whereas for monolingual
Hindi, it converges at 80K-90K, but for Hinglish the same behavior starts
around 800K vocabulary and 50M words.

Figure 1: Heaps’ plot on 50M word forms in English, Hindi and Hinglish
corpora. The $\beta$ values are $0.58$, $0.61$, $0.74$ respectively.

###  3.2 Code-Mixing Index (CMI)

As mentioned previously, we expect the difficulty of language processing tasks
to increase as the level of codemixing increases. To measure the level of
codemixing in our corpus, we use Code-mixing Index Gambäck and Das (2016) :

$\begin{aligned} C_{u}(x)&=w_{m}f_{m}(x)+w_{p}f_{p}(x)\\\
&=w_{m}\frac{N(x)-\max_{L_{i}\epsilon
L}(tLi)(x)}{N(x)}*100+w_{p}\frac{P(x)}{N(x)}*100\\\
&=100*\frac{w_{m}((N(x)-\max_{L_{i}\epsilon
L}(tLi)(x))+w_{p}P(x)}{N(x)}\end{aligned}$

(1)

Where x denotes utterance, N is the number of token in x belonging to any
language $L_{i}$, $w_{m}$ and $w_{n}$ are weights. Please refer to Gambäck and
Das (2016) for a detailed explanation of CMI.

###  3.3 Data Acquisition Pipeline

We follow a pipeline similar to Chatterjere et al. (2020). We collect CM data
from Twitter via the Twitter API. We need to use relevant keywords (words
unique to Hindi) in our search to get CM tweets. Words with lexical overlap
between Hindi and English should not be used for searching. for example, the
word do is confusing because it means two in Hindi. We start with the ICON
2017 Hinglish sentiment analysis dataset Patra et al. (2018), which is
annotated with word-level language. From this data, we create two vocabularies
$V_{HI}$ and $V_{EN}$, and generate a vocabulary of unqique Hindi words
$V_{HI-UNIQ}=V_{HI}-I$, where $I=V_{HI}\bigcap V_{EN}$. $V_{HI-UNIQ}$ set is
then sorted in descending order, based on the word frequency, and is used as
search words on the Twitter API. Once we get the tweets, we use a word-level
language identifier Barman et al. (2014) (having 90%+ accuracy) on the tweets
and calculate the CMI of the tweet. Once we get the word-level language
labels, we can also know where the switching points are. Tweets with CMI = 0
are discarded. Finally, we are left with 87k tweets. The CMI distribution of
our data is given in table 1.

CMI | # Tweets | Percentage
---|---|---
0-10 | 7,036 | 8.05%
11-20 | 16,481 | 18.9%
21-30 | 22,617 | 25.9%
31-40 | 22,722 | 26.0%
41-50 | 11,404 | 13.1%
50+ | 7,036 | 8.05%
Mean CMI: 28 | Total # of tweets: 87,296

Table 1: CMI distribution of the collected data. The total number of extracted
tweets is 87K.

Training and Testing data: We collect 87K sentences distributed over all CMI
ranges, instead of collecting equal data across the CMI ranges, so that the
resultant languages trained on this corpus would be able to handle real data.
We maintain the same distribution over both our training and testing corpora
(4:1 ratio), for our language models.

##  4 The Bottleneck of Code-mixed Language Modeling: Switching Points

Formally, Switching Points (SPs) are the tokens in text, where the language
switches. For code-mixed languages, consisting of a pair of languages, there
can be two types of switching points. Suppose the two languages as part of the
code-mixed language are L1 and L2, a switching point occurs when the language
in the text changes from L1 to L2 or L2 to L1. To explain it better, let us
consider the following sample in Hinglish:

> gaanaHI enjoyEN kareHI
> English Translation:  Enjoy the song.

In the above example, when the language switches from Hindi to English
(gaanaHI enjoyEN) a HI-EN (HIndi-ENglish) switching point occurs. Similarly, a
EN-HI(ENglish-HIndi) switching point occurs at - enjoyEN kareHI.

In the context of modeling code-mixed languages, switching points can be
considered as ordinary bigrams, that occur with other monolingual bigrams in a
corpus. It is easy to infer that particular SP bigrams will be relatively rare
in a given corpus. Hence, such sparse occurrences of switching point bigrams
make it difficult for any Language Model to learn their probabilities and
context. Since the language changes at the switching point, LMs are likely to
find it difficult to process these tokens. In order to counter this challenge,
we partition our code-mixed data into (i) switching points, and (ii) non-
switching points. We then build LMs specifically for switching points and non-
switching points, as discussed in the following sections.

CONFLATOR Hypothesis:  The CONFLATOR is built on 2 hypotheses. i) Positional
information is important for language models, especially when dealing with CM
text. ii) Switching points are the bottleneck for code mixed language models
(CMLM). We incorporate positional information of switching points into our
CMLM.

##  5 Positional Encoding Techniques

As discussed, SPs are the major bottleneck hence handling them separately is
needed. Positional encoding are neccesary for language models to learn
dependencies between tokens. Positional embedding was first introduced by
Vaswani et al. (2017b). The proposed sinusoidal positional encoding is
composed of sine and cosine values with position index as inputs. The encoding
techniques are further improved by Liu et al. (2020) where a dynamic function
is introduced to learn position with gradient flow and Shaw et al. (2018)
learned positional representation of relative positions using a learnable
parameter. We talk about different positional encoding techniques in detail in
the following subsections.

We experiment with several contemporary techniques and find that rotary
positional encoding Su et al. (2021) performs the best.

###  5.1 Sinusoidal Positional Encoding (SPE)

Vaswani et al. (2017b) introduced a predefined sinusoidal vector $p_{i}\in
R^{d}$ which is assigned to each position i. This $p_{i}$ is added to the word
embedding $x_{i}\in R^{d}$ at position i, and $x_{i}+p_{i}$ is used as input
to the model such that the Transformer can differentiate words coming from
different positions and this also assigns each token a position-dependent
attention. - equation 2.

$e_{ij}^{abs}=\frac{1}{\sqrt{d}}\left(\left(x_{i}+p_{i}\right)W^{Q,1}\right)\left(\left(x_{j}+p_{j}\right)W^{K,1}\right)^{T}$

(2)

Where W is the weight matrix, Q is query, K is key, l in the layer.

###  5.2 Dynamic Positional Encoding (DPE)

Instead of using predefined periodical functions like $sin$, Liu et al.
(2020), introduced a dynamic function $\Theta(i)$ at every encoder layer.
Improving upon sinusoidal PE, Dynamic PE learns $\Theta(i)$ instead of a
predefined $p_{i}$ to bring dynamic behavior to the model. At each utterance,
this learnable function $\Theta(i)$ tries to learn the best possible
representation for positional information with gradient flow. $\Theta(i)$ is
added to the word embedding $w_{i}$ as given in equation 3.

$e_{ij}=\frac{1}{\sqrt{d}}\left(\left(x_{i}+\Theta\left(i\right)\right)W^{Q,1}\right)\left(\left(x_{j}+\Theta\left(j\right)\right)W^{K,1}\right)^{T}$

(3)

###  5.3 Relative Positional Encoding (RPE)

In absolute PE, using different $p_{i}$ for different positions i helps the
transformer distinguish words at different positions. However, the absolute PE
is not effective in capturing the relative word order. Shaw et al. (2018)
introduced a learnable parameter $a_{i-j}^{l}$ which learns the positional
representation of the relative position i-j at encoder layer l. With the help
of this, we can explicitly capture word orders in our model as follows:

$e_{ij}^{rel}=\frac{1}{\sqrt{d}}\left(\left(x_{i}\right)^{l}W^{Q,l}\right)\left(\left(x_{i}\right)^{l}W^{K,l}+a_{i-j}^{l}\right)^{T}$

(4)

###  5.4 Switching Point-based Dynamic and Relative Positional Encoding
(SPDRPE)

Ali et al. (2022) introduce a novel, switching point based PE. For
illustration purposes, consider a code-mixed Hinglish text - yeHI gaanaHI
enjoyEN kareHI. SP-based indices (SPI) set the index to 0 whenever an SP
occurs. Indexing would normally be Index = (0, 1, 2, 3), but due to switching
point incorporation, this gets changed to SPI = (0, 1, 0, 0). These indices
are dynamically learned. In addition to this, they use a learning parameter
$a_{i-j}^{l}$, which encodes the relative position i-j at the encoder layer l.
This encoding approach learns representations dynamically based on SPs along
with the embedding $a_{i-j}^{l}$ so that it can also capture relative word
orders, as follows:

$e_{ij}=\frac{1}{\sqrt{d}}\left(\left(x_{i}+\Theta\left(S(l_{i})\right)\right)^{l}W^{Q,l}\right)\left(\left(x_{i}+\Theta\left(S(l_{j})\right)\right)^{l}W^{K,l}+a_{i-j}^{l}\right)^{T}$

(5)

###  5.5 Rotary Positional Encoding (RoPE)

Analogous to the idea of electromagnetic waves going through a polarizer to
preserve their relative amplitude, Su et al. (2021) came up with the idea of
Rotary Positional Encoding (RoPE). The idea is to use rotation matrices on the
embedding vectors to generate the positional values. The rotation negates any
absolute positional information and only retains information about the
relative angles between every pair of word embeddings in a sequence. We know
that the dot product between two vectors is a function of the magnitude of
individual vectors and the angle between them. Keeping this in mind, the
intuition for RoPE is to represent the embeddings as complex numbers and the
positions as pure rotations that we apply to them.
Mathematically, the formulations for a simple 2-dimensional case are defined
as follows:

$\begin{aligned} &f_{Q}(x_{i},i)=(W_{Q}x_{i})e^{\sqrt{-1}i\theta}\\\
&f_{Q}(x_{j},j)=(W_{K}x_{j})e^{\sqrt{-1}j\theta}\\\
&g(x_{i},x_{j},i-j)=Re[(W_{Q}x_{i})(W_{K}x_{i})\textsuperscript{*}e^{\sqrt{-1}(i-j)\theta}]\\\
\end{aligned}$

(6)

where $Re[·]$ is the real part of a complex number and
$(W\textsubscript{K}x\textsubscript{i})\textsuperscript{*}$ represents the
conjugate complex number of $(W\textsubscript{K}x\textsubscript{i})$.
$\theta\in R$ is a preset non-zero constant. Formulating
$f\textsubscript{(Q,K)}$ as a matrix multiplication, we get:

$f_{Q}(x_{i},i)=\left(\begin{smallmatrix}cosm\theta_{1}&-sinm\theta_{1}\\\
sinm\theta_{1}&cosm\theta_{1}\\\
\end{smallmatrix}\right)\left(\begin{smallmatrix}W_{Q,K}^{(11)}&W_{Q,K}^{(12)}\\\
W_{Q,K}^{(21)}&W_{Q,K}^{(22)}\\\
\end{smallmatrix}\right)\left(\begin{smallmatrix}x_{i}^{(1)}\\\ x_{i}^{(2)}\\\
\end{smallmatrix}\right)$

(7)

where
$(x\textsubscript{i}\textsuperscript{(1)},x\textsubscript{i}\textsuperscript{(2)})$
is $x\textsubscript{i}$ expressed in the form of 2D coordinates. In the same
way, we can turn function g into matrix form. By rotating the transformed
embedding vector by an angle in multiples of its position index, we are able
to incorporate relative position information. Due to this characteristic, it
is termed as Rotary Position Embedding.

In order to generalize the result in 2D to any $x\textsubscript{i}$ in
$R\textsubscript{d}$ where $d$ is even, they divide the d-dimension space into
$\frac{d}{2}$ sub-spaces and combine them in merit of the linearity of inner
product, turning the attention formulation:

$f_{Q,K}=e_{ij}^{rotary}=\frac{1}{\sqrt{d}}\left(RM_{\Theta,i}^{d}W^{Q,1}\left(x_{i}\right)\right)^{T}\left(RM_{\Theta,j}^{d}W^{K,1}\left(x_{j}\right)\right)$

(8)

$RM=\left(\begin{smallmatrix}cosm\theta_{1}&-sinm\theta_{1}&0&0&...&0&0\\\
sinm\theta_{1}&cosm\theta_{1}&0&0&...&0&0\\\
0&0&cosm\theta_{2}&-sinm\theta_{2}&...&0&0\\\
0&0&sinm\theta_{2}&cosm\theta_{2}&...&0&0\\\ .&.&.&.&...&.&.\\\
.&.&.&.&...&.&.\\\ 0&0&0&0&...&cosm\theta_{d/2}&-sinm\theta_{d/2}\\\
0&0&0&0&...&sinm\theta_{d/2}&cosm\theta_{d/2}\\\ \end{smallmatrix}\right)$

(9)

where RM is orthogonal and sparse matrix pre-defined parameters

$\Theta={\theta\textsubscript{i}=10000\textsuperscript{-2(i-1)/d},i\in[1,2,...,d/2]}.$
(10)

In contrast to the additive nature of the position embedding methods used by
other works, their approach is multiplicative. Moreover, RoPE naturally
incorporates relative position information through rotation matrix product
instead of altering terms in the expanded formulation of additive position
encoding when applied with self-attention.

##  6 Incorporation of Switching Point Information in CMLM

Positional encodings help the transformer learn dependencies between tokens at
different positions of the input sequence. To enhance the positional encodings
for code-mixed text, we modify the rotatory positional encoding to incorporate
switching point information.

###  6.1 Switching Point-based Rotary Matrix

Figure 2: Visual intuition for our rotary approach with switching point
incorporation. We consider a linearly polarized electromagnetic wave and show
the change in rotation whenever a switching point occurs.  Figure 3: This
diagram depicts the higher level understanding of the proposed positional
embeddings.  Figure 4: CONFALTOR architecture within the encoder layer. It
depicts how the unigrams and bigrams of the input statement are passed as
inputs to our encoder decoder architecture. In this framework, we generate a
rotary matrix and a switching point matrix. By performing element-wise
multiplication of the aforementioned matrices, we get our proposed novel
switching point based rotary matrix. We represent the embeddings as complex
numbers and their positions as pure rotations that we apply to them with the
help of our switching point based rotary matrix. Then, upon getting the output
layers for unigram and bigram statements separately. We introduce weighted
coefficients $a$ and $b$ for unigram outputs and bigram outputs, respectively.
We get our final output layer by adding these weighted unigram and bigram
outputs.

Switching points are the bottleneck for code mixing language modeling and to
address this problem, we incorporate switching point based rotary positional
encoding in our architecture. The intuition behind RoPE is electromagnetic
waves. The embeddings are represented as complex numbers and the positions are
represented as pure rotations that are applied to them. Keeping this in mind,
we address the problem of switching points (SP) with the help of angles that
participate in RoPE. Whenever we encounter a switching point, we change the
rotation, i.e., we change the direction of these angles. To implement the
rotation change, we define a switching point matrix. The switching point
matrix helps our model identify and learn the patterns of code mixing in the
corpus. Our matrix is defined with 1s and -1s. When there is a language shift
(L1 $\rightarrow$ L2) or (L2 $\rightarrow$ L1), i.e., when we encounter a
switching point, we annotate the column value as -1 and for the successive
words in L2, we annotate column values as 1 until another switching point
occurs.

$\begin{aligned} SPM&\in R_{n*n}^{d}&\\\ \text{if i}==\text{SP:}\\\
&SPM_{i}=-1\\\ \text{else:}\\\ &SPM_{i}=1\end{aligned}$

(11)

The visual intuition of our approach is shown in Figure 2. The switching point
matrix (SPM) with 1s and -1s is defined in such a way that it transposes the
rotary matrix, intuitively inverting the rotation at every switching point
encounter. Therefore, the final matrix, i.e., switching point rotary matrix
(SPRM) is a result of element-wise multiplication of the defined switching
point matrix (SPM) with rotary matrix (RM):

SPRM = SPM $\times$ RM (12)

$e_{ij}^{SPRotary}=\frac{1}{\sqrt{d}}\left(SPRM_{\Theta,i}^{d}W^{Q,1}\left(x_{i}\right)\right)^{T}\left(SPRM_{\Theta,j}^{d}W^{K,1}\left(x_{i}\right)\right)$

(13)

###  6.2 Bigram and Switching Point-based Rotary Positional Encoding (BSPRoPE)

(a) Sinusoidal PE

(b) Rotary PE

(c) Conflator

Figure 5: CONFLATOR is able to differentiate words coming from different
positions and give high attention when a switching point occurs (at bagEN and
kidarHI) while the other models cannot do so.

In this positional encoding method, we get positional information among the
bigrams in an utterance. We use the technique of switching point based rotary
positional encoding at a word-to-word level and at bigram level as depicted in
Figures 3 4 and mathematically expressed as Equation 16

$e_{ij}^{UniSPRotary}=\frac{1}{\sqrt{d}}\left(SPRM_{\Theta,i}^{d}W^{Q,1}\left(x_{i}\right)\right)^{T}\left(SPRM_{\Theta,j}^{d}W^{K,1}\left(x_{j}\right)\right)$

(14)

$e_{ij}^{BiSPRotary}=\frac{1}{\sqrt{d}}\left(SPRM_{\Theta,i}^{d}W^{Q,1}\left(x_{i}\right)\right)^{T}\left(SPRM_{\Theta,j}^{d}W^{K,1}\left(x_{j}\right)\right)$

(15)

prediction = $a*e_{ij}^{UnigramSPRotary}+b*e_{ij}^{BigramSPRotary}$

(16)

where $a$ and $b$ are learnable coefficients.

###  6.3 Conflator Architecture

The local dependencies for both Unigram and Bigram (Word2Vec trained from
scratch) along with unigram and bigram SPRM are passed to a 6-headed Multi-
Head attention in each encoder layer separately. This gives two attention
matrices. We introduced two learnable parameters $\alpha$ and $\beta$ that are
used as weighted coefficients for the unigram and bigram matrix respectively.
The final matrix is passed to the decoder layer. The embedding and
architecture in depicted in figures 3 and 4.

Models | Positional representation | Bigram | F1 (%)
---|---|---|---
Sin/Cos | Index | Dynamic | SPI | Relative | RM | SPRM |  |
Word2Vec + LSTM | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | 56
BERT | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | 60
3HA + Sinusoidal PE | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | 74.34
3HA + Dynamic PE | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | 75.02
3HA + Relative PE | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | 75.32
3HA + Rotary PE | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | 76.04
SOTA (PESTO) | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | 75.6
Unigram SP Relative (USPR) | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | 75
Bigram SP Relative BSPR) | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | 75
Unigram SPRoPE + Good Tuning | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✗ | 74.6
Unigram SPRoPE | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✗ | 75
Conflator (BSPRoPE) | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ | 76.23
Conflator with StableLM | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ | 76.11
Conflator with Alpaca | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ | 75.69
Conflator with LLaMA | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ | 76.45

Table 2: Results of various position sensitive experiments for Sentiment
Analysis on CM text.

Models | Positional representation | Bigram | BLEU
---|---|---|---
Sin/Cos | Index | Dynamic | SPI | Relative | RM | SPRM |  |
3HA + Sinusoidal PE | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | 17.2
3HA + Dynamic PE | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | 17.9
3HA + Relative PE | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | 18.4
3HA + Rotary PE | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | 24.9
SOTA (IIITH-mrinaldhar) | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | 28.4
Unigram SP Relative (USPR) | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | 9.8
Bigram SP Relative (BSPR) | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | 7.6
Unigram SPRoPE | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✗ | 29.1
Conflator (BSPRoPE) | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ | 25.16
Conflator with StableLM | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ | 29.06
Conflator with Alpaca | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ | 29.89
Conflator with LLaMA | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ | 30.15

Table 3: Results of position sensitive experiments for Machine Translation on
CM text. Higher BLEU is better.

##  7 Experiments and Results

For our base models, each training step takes about 0.5 seconds. We train the
base models for a total of 100,000 steps or 12 hours. For the big models like
bigram and SPM-based models, the step time is 1.0 seconds. The big models were
trained for 250,000 steps (2 days). We use ADAM optimizer with $\beta_{1}$ =
0.9, $\beta_{2}$ = 0.98 and $\epsilon$ = 1e-9. We use the method of varying
the learning rate over the course of training from Vaswani et al. (2017b).

We use two types of regularization during our training process: We apply
dropout to the output of each encoder and decoder layer followed by
Normalization. In addition, we apply dropout and normalization to the sums of
the word embeddings and the positional encodings in both the encoder and
decoder layers. We use a rate of Pdrop = 0.2.

###  7.1 Intrinsic Evaluation

The perplexity scores of different language models such as Transformer, GPT-2,
and BERT in comparison with CONFALTOR are shown in 4. We see that our model
performs much better than other models.

CMI Range | Transformer | GPT-2 | BERT | Conflator
---|---|---|---|---
0-10 | 1018.54 | 823.71 | 666.48 | 492.96
11-20 | 1210.11 | 967.01 | 782.19 | 501.44
21-30 | 1401.37 | 1334.72 | 1007.34 | 544.71
31-40 | 2688.00 | 2334.73 | 1007.34 | 800.62
41-50 | 4421.22 | 3905.87 | 4337.02 | 1095.12
Average | 2147.85 | 1873.20 | 1701.49 | 578

Table 4: Perplexity comparison between different models based on ranges of
CMI. Lower Perplexity is better.

###  7.2 Extrinsic Evaluation

We evaluate our model on two downstream tasks: (i) sentiment analysis, and
(ii) machine translation. For sentiment analysis, (Table. 2) we use the data
provided by Patwa et al. (2020a). CONFLATOR achieves 76.23% F1 score and
outperforms the SOTA Ali et al. (2022). The main reason for this is learning
SP by aggregating with the help of rotary positional encoding with a variable
length MHA framework. For the machine translation (Table. 3), we use the data
provided by Dhar et al. (2018). We achieve 29.1 bleu score and outperform the
SOTA Dhar et al. (2018) using the Unigram SPRoPE model which is able to learn
the patterns of language mixing with the help of switching point based rotary
positional encoding.

##  8 Conclusion & Takeaways

In this work, we report experiments on Hinglish sentiment analysis and Machine
translation problems through the lens of language modeling. Our contribution
could be seen as following:
(i) We introduce the idea of switching point based rotary positional encoding.
Whenever a switching point is encountered, we incorporate rotation change to
learn the patterns of language mixing.
(ii) We introduce CONFLATOR, a neural language modeling approach for code-
mixed languages. CONFLATOR tries to learn better representations by means of
switching point-based rotary positional encoding, initially at unigram level
and then at bigram level.
(iii) We empirically prove that CONFLATOR is learning the patterns of code-
mixing which other models with different positional encodings prove
unsuccessful, as shown in Figure 5.
(iv) It is also noteworthy that CONFLATOR achieves comparable to SOTA results
even without any pre-trained heavy language model.

##  9 Limitations

Although our bigram model achieves SOTA on sentiment analysis using unigram,
it is slightly behind the bigram model when it comes to machine translation,
where using bigram at the decoder level resulted in poor performance. Despite
conducting extensive experiments, there lacks a detailed explanation on why
the bigram-based approach for MT fails. Future experiments will focus on
exploring or understanding the issue of bigrams for MT and coming up with a
solution for the same.

## References

  * Ali et al. (2022) Mohsin Ali, Sai Teja Kandukuri, Sumanth Manduru, Parth Patwa, and Amitava Das. 2022.  Pesto: Switching point based dynamic and relative positional encoding for code-mixed languages (student abstract).  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 36.
  * Aroyehun and Gelbukh (2018) Segun Taofeek Aroyehun and Alexander Gelbukh. 2018.  Aggression detection in social media: Using deep neural networks, data augmentation, and pseudo labeling.  In _Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018)_ , pages 90–97, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Banerjee et al. (2020) Shubhanker Banerjee, Bharathi Raja Chakravarthi, and John P McCrae. 2020.  Comparison of pretrained embeddings to identify hate speech in indian code-mixed text.  In _2020 2nd International Conference on Advances in Computing, Communication Control and Networking (ICACCCN)_ , pages 21–25. IEEE.
  * Barman et al. (2014) Utsab Barman, Amitava Das, Joachim Wagner, and Jennifer Foster. 2014.  Code mixing: A challenge for language identification in the language of social media.  In _Proceedings of the First Workshop on Computational Approaches to Code Switching_ , pages 13–23, Doha, Qatar. Association for Computational Linguistics.
  * Bengio et al. (2003) Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003.  A neural probabilistic language model.  _J. Mach. Learn. Res._ , 3:1137–1155.
  * Bohra et al. (2018) Aditya Bohra, Deepanshu Vijay, Vinay Singh, Syed Sarfaraz Akhtar, and Manish Shrivastava. 2018.  A dataset of hindi-english code-mixed social media text for hate speech detection.  In _Proceedings of the second workshop on computational modeling of people’s opinions, personality, and emotions in social media_ , pages 36–41.
  * Bokamba (1988) Eyamba G Bokamba. 1988.  Code-mixing, language variation, and linguistic theory:: Evidence from bantu languages.  _Lingua_ , 76(1):21–62.
  * Chakravarthi et al. (2020) Bharathi Raja Chakravarthi, Vigneshwaran Muralidaran, Ruba Priyadharshini, and John P McCrae. 2020.  Corpus creation for sentiment analysis in code-mixed tamil-english text.  _arXiv preprint arXiv:2006.00206_.
  * Chatterjere et al. (2020) Arindam Chatterjere, Vineeth Guptha, Parul Chopra, and Amitava Das. 2020.  Minority positive sampling for switching points - an anecdote for the code-mixing language modeling.  In _Proceedings of The 12th Language Resources and Evaluation Conference_ , pages 6230–6238, Marseille, France. European Language Resources Association.
  * Das and Gambäck (2014) Amitava Das and Björn Gambäck. 2014.  Identifying languages at the word level in code-mixed indian social media text.
  * Devlin et al. (2018a) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018a.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Devlin et al. (2018b) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018b.  BERT: pre-training of deep bidirectional transformers for language understanding.  _CoRR_ , abs/1810.04805.
  * Dhar et al. (2018) Mrinal Dhar, Vaibhav Kumar, and Manish Shrivastava. 2018.  Enabling code-mixed translation: Parallel corpus creation and MT augmentation approach.  In _Proceedings of the First Workshop on Linguistic Resources for Natural Language Processing_ , pages 131–140, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Gambäck and Das (2016) Björn Gambäck and Amitava Das. 2016.  Comparing the level of code-switching in corpora.  In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)_ , pages 1850–1855, Portorož, Slovenia. European Language Resources Association (ELRA).
  * Gopalan and Hopkins (2020) Vinay Gopalan and Mark Hopkins. 2020.  Reed at semeval-2020 task 9: Sentiment analysis on code-mixed tweets.  _CoRR_ , abs/2007.13061.
  * Jhanwar and Das (2018) Madan Gopal Jhanwar and Arpita Das. 2018.  An ensemble model for sentiment analysis of hindi-english code-mixed data.  _arXiv preprint arXiv:1806.04450_.
  * Laddha et al. (2020) Abhishek Laddha, Mohamed Hanoosh, Debdoot Mukherjee, Parth Patwa, and Ankur Narang. 2020.  Understanding chat messages for sticker recommendation in messaging apps.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(08):13156–13163.
  * Laddha et al. (2022) Abhishek Laddha, Mohamed Hanoosh, Debdoot Mukherjee, Parth Patwa, and Ankur Narang. 2022.  Large scale multilingual sticker recommendation in messaging apps.  _AI Magazine_ , 42(4):16–28.
  * Lample and Conneau (2019) Guillaume Lample and Alexis Conneau. 2019.  Cross-lingual language model pretraining.
  * Liu et al. (2020) Xuanqing Liu et al. 2020.  Learning to encode position for transformer with continuous dynamical model.  In _ICML 2020_.
  * Ma et al. (2020) Yili Ma, Liang Zhao, and Jie Hao. 2020.  Xlp at semeval-2020 task 9: Cross-lingual models with focal loss for sentiment analysis of code-mixing language.  In _Proceedings of the fourteenth workshop on semantic evaluation_ , pages 975–980.
  * Nayak and Joshi (2022) Ravindra Nayak and Raviraj Joshi. 2022.  L3cube-hingcorpus and hingbert: A code mixed hindi-english dataset and bert language models.
  * Patra et al. (2018) Braja Gopal Patra, Dipankar Das, and Amitava Das. 2018.  Sentiment analysis of code-mixed indian languages: An overview of sail_code-mixed shared task @icon-2017.  _CoRR_ , abs/1803.06745.
  * Patwa et al. (2020a) Parth Patwa, Gustavo Aguilar, Sudipta Kar, Suraj Pandey, Srinivas PYKL, Björn Gambäck, Tanmoy Chakraborty, Thamar Solorio, and Amitava Das. 2020a.  SemEval-2020 task 9: Overview of sentiment analysis of code-mixed tweets.  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , pages 774–790, Barcelona (online). International Committee for Computational Linguistics.
  * Patwa et al. (2020b) Parth Patwa, Srinivas Pykl, Amitava Das, Prerana Mukherjee, and Viswanath Pulabaigari. 2020b.  Hater-O-genius aggression classification using capsule networks.  In _Proceedings of the 17th International Conference on Natural Language Processing (ICON)_ , pages 149–154, Indian Institute of Technology Patna, Patna, India. NLP Association of India (NLPAI).
  * Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.
  * Samghabadi et al. (2020) Niloofar Safi Samghabadi, Parth Patwa, Srinivas Pykl, Prerana Mukherjee, Amitava Das, and Thamar Solorio. 2020.  Aggression and misogyny detection using bert: A multi-task approach.  In _Proceedings of the second workshop on trolling, aggression and cyberbullying_ , pages 126–131.
  * Sengupta et al. (2021) Ayan Sengupta, Sourabh Kumar Bhattacharjee, Tanmoy Chakraborty, and Md Shad Akhtar. 2021.  Hit: A hierarchically fused deep attention network for robust code-mixed language representation.
  * Shaw et al. (2018) Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. 2018.  Self-attention with relative position representations.  In _NAACL 2018_.
  * Singh (1985) Rajendra Singh. 1985.  Grammatical constraints on code-mixing: Evidence from hindi-english.  _Canadian Journal of Linguistics/Revue canadienne de linguistique_ , 30(1):33–45.
  * Srivastava and Singh (2020) Vivek Srivastava and Mayank Singh. 2020.  Phinc: A parallel hinglish social media code-mixed corpus for machine translation.  _arXiv preprint arXiv:2004.09447_.
  * Su et al. (2021) Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. 2021.  Roformer: Enhanced transformer with rotary position embedding.  _CoRR_ , abs/2104.09864.
  * Tang et al. (2020) Tiancheng Tang, Xinhuai Tang, and Tianyi Yuan. 2020.  Fine-tuning bert for multi-label sentiment analysis in unbalanced code-switching text.  _IEEE Access_ , 8:193248–193256.
  * Tula et al. (2021) Debapriya Tula, Prathyush Potluri, Shreyas Ms, Sumanth Doddapaneni, Pranjal Sahu, Rohan Sukumaran, and Parth Patwa. 2021.  Bitions@DravidianLangTech-EACL2021: Ensemble of multilingual language models with pseudo labeling for offence detection in Dravidian languages.  In _Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages_ , pages 291–299, Kyiv. Association for Computational Linguistics.
  * Tula et al. (2022) Debapriya Tula, MS Shreyas, Viswanatha Reddy, Pranjal Sahu, Sumanth Doddapaneni, Prathyush Potluri, Rohan Sukumaran, and Parth Patwa. 2022.  Offence detection in dravidian languages using code-mixing index-based focal loss.  _SN Computer Science_ , 3(5):330.
  * Vaswani et al. (2017a) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017a.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc.
  * Vaswani et al. (2017b) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017b.  Attention is all you need.  _CoRR_ , abs/1706.03762.
  * Verma (1976) S.K. Verma. 1976.  Code-switching: Hindi-english.  _Lingua_ , 38(2):153–165.
  * Vyas et al. (2014) Yogarshi Vyas, Spandana Gella, Jatin Sharma, Kalika Bali, and Monojit Choudhury. 2014.  Pos tagging of english-hindi code-mixed social media content.  In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_ , pages 974–979.


# Dressed Energy Levels in Strongly Interacting Atoms

Seyed Mostafa Moniri1<EMAIL_ADDRESS>Marjan Fani2<EMAIL_ADDRESS>Elnaz Darsheshdar3<EMAIL_ADDRESS>1Faculty of Science, Golpayegan
University of Technology, P.O. Box 87717-67498, Golpayegan, Iran  2Laser and
Plasma Research Institute, Shahid Beheshti University, Tehran 19839-69411,
Iran  3Departamento de Física, Universidade Federal de São Carlos, P.O. Box
676, 13565-905, São Carlos, São Paulo, Brazil

###### Abstract

We investigate the effect of strong interaction in the dressed energy levels
of the two level emitters. Strong dipole-dipole interactions give rise to new
sidebands in the fluorescence spectrum due to specific couplings among the
collective dressed levels which in turn depends on the spatial configuration
of atoms. These couplings are the main responsible for the frequencies and
variety of sidebands. We explain the general method for finding the dressed
energy levels for a system of any number of strongly coupled atoms and we
solve this problem for two different spatial configurations of three coupled
two-level emitters. We show that the coupling among dressed levels and
consequently energies and number of sidebands in the fluorescence spectrum are
different for each configuration. Thus the fluorescence spectrum of strongly
interacting atoms contains information about the number and configuration of
atoms.

##  I Introduction

Dressing of atoms with the incident laser field -in both its classical and
quantum descriptions-, or with individual photons is a well-known phenomenon
in quantum optics theories. Resulted coupled atom-field energy levels describe
well the spectral properties of the system. Fluorescence spectrum is one of
the simplest and yet most useful case of the light-matter coupling. It
describes the emission of a two-level system (TLS) when is driven strongly by
the external laser field. One smart physical interpretation of this spectrum
describes a TLS dressed by the laser field [1, 2]. This leads to new
eigenstates $\left|\pm\right\rangle$ which are a combination of states
$\left|\uparrow,n\right\rangle$ and $\left|\downarrow,n+1\right\rangle$ where
$\,\left|\downarrow\right\rangle,\,\left|\uparrow\right\rangle$ are the
“ground” and “excited” states of the atom and $n$ is the number of photons. A
group of these states builds “manifolds of excitation”. In each manifold,
eigenstates are split by the Rabi frequency of the driving field, and the
successive manifolds separated by ${{\omega}_{L}}$ which is the one photon
energy. The transitions between neighbor manifolds explain the main property:
a carrier centered at the laser frequency, and two sidebands symmetrically
shifted away by the Rabi frequency of the driving field [3, 4, 5].

All mentioned studies so far, are for the single atom effects. In this
context, the coupling of the emitters, gives access to new control parameter,
as new resonances rise from the interaction. Including the interaction between
atoms [6, 7, 8] new sidebands appear at twice the Rabi frequency from the
carrier. Recently it has been shown that these extra sidebands, called the
“baby Mollow”, should be observable in a large cloud of cold atoms interacting
by the light-mediated dipole-dipole interaction, provided the optical
thickness is large enough [9]. Spectral properties have been studied in a two
atoms system when atoms are weakly interacting by light mediated dipole-dipole
interaction, but only one atom is driven by external field. In this case the
fluorescence spectrum will not be affected by the interaction and additional
sidebands would not appear. Dressed energy levels are still single atom ones
[10]. In the limit of strong dipole-dipole interaction the appearance of the
collective dressed energy levels has been shown and photon correlations have
been studied in a two atoms system [11]. Although the appearance of collective
effects have been reported in the last mentioned work, a description for the
collective dressed energy levels for a system of many atoms with the strong
interaction have never been done.

In this paper we will explain the general method for obtaining dressed energy
levels for systems containing $N$ number of strongly interacting atoms, and
the reason of higher number of sidebands in their fluorescence spectrum by
solving an example of three atoms system. We also investigate the effect of
atomic configuration in the dressed energy levels of strongly interacting
atoms and consequently the fluorescence spectrum. For this goal we consider an
example of two different spatial configurations of three two-levels atoms and
will study the properties of the fluorescence spectrum in this system. We will
show that coupling between dressed levels and the additional sidebands will
change by changing the atomic spatial configuration. The example of three
atoms can be extended to a desired $N$ number of atoms with the cost of
computation for diagonalizing the Hamiltonian which is an $N$ by $N$ matrix.
For high number of $N$ it will be an intractable task and approximated methods
will be useful.

##  II Fluorescence Spectrum

Dynamical studies of a system of two-level systems(TLS) driven by an external
laser beam and undergoing cooperative emission due to long range interaction
mediated by light, can be done by a master equation approach. More
specifically we consider identical two-level system (TLS), each of them being
described by the spin half angular momentum algebra, at positions
${{\textbf{r}}_{i}}$ and with $\Gamma$ the decay rate of the excited state.
Experimentally, this can be either two atoms, two molecules or two quantum
dots. The corresponding master equation in the rotating frame of the laser, in
the Born, Markov, and rotating-wave approximations is then given by [12, 13]
(we set $\hbar=1$ along the paper)

$\displaystyle\frac{\partial\rho}{\partial
t}=i\left[\rho,{{H}_{\sigma}}\right]+{\mathcal{L}_{\sigma}}\rho,$ (1)
$\displaystyle{{H}_{\sigma}}=\sum\limits_{i}{{H}_{\sigma
i}}+\sum\limits_{i,j\neq i}{{H}_{\sigma
ij}}=\frac{{{\Omega}}}{2}\sum\limits_{i}({{\operatorname{e}}^{-i\textbf{k}.{{\textbf{r}}_{i}}}}\sigma_{i}^{-}+{{\operatorname{e}}^{i\textbf{k}.{{\textbf{r}}_{i}}}}\sigma_{i}^{+})+\Delta\sum\limits_{i}{\sigma_{i}^{+}\sigma_{i}^{-}}+\Gamma\sum\limits_{i,j\neq
i}{{\Omega}_{ij}}\sigma_{i}^{+}\sigma_{j}^{-},$ (2)

where $\rho$ is the density matrix and $\sigma_{i}^{-}$ ($\sigma_{i}^{+}$) is
the annihilation (creation) operator of the $i$-th TLS with free energy
${{\omega}_{a}}$. The detuning is the difference between ${{\omega}_{a}}$ and
the laser frequency $\Delta={{\omega}_{a}}-{{\omega}_{L}}$, $\Omega$ is the
Rabi frequency of the driving field and k is the wave vector. The Hamiltonian
of the system is written in the rotating frame of the laser in Eq. (2), and
the first term is related to atom-field interaction, while the second term
corresponds to the atom-atom interaction. The Lindblad super-operator is

$\displaystyle{\mathcal{L}_{\sigma}}\rho=\frac{\Gamma}{2}\sum\limits_{i}{\mathcal{L}_{\sigma
i}}\rho+\frac{\Gamma}{2}\sum\limits_{i,j\neq
i}{{\Gamma}_{ij}}{\mathcal{L}_{\sigma
ij}}\rho=\frac{\Gamma}{2}\sum\limits_{i}(2\sigma_{i}^{-}\rho\sigma_{i}^{+}-\sigma_{i}^{+}\sigma_{i}^{-}\rho-\rho\sigma_{i}^{+}\sigma_{i}^{-})$
(3) $\displaystyle+\frac{\Gamma}{2}\sum\limits_{i,j\neq
i}{{\Gamma}_{ij}}(2\sigma_{j}^{-}\rho\sigma_{i}^{+}-\sigma_{i}^{+}\sigma_{j}^{-}\rho-\rho\sigma_{i}^{+}\sigma_{j}^{-}),$

where ${{\Gamma}_{ij}}$ and ${{\Omega}_{ij}}$ are

$\displaystyle{{\Gamma}_{ij}}=\frac{3}{2}\left(1-\cos^{2}{{\theta_{ij}}}\right)\frac{\sin{(k{r}_{ij})}}{{k{r}_{ij}}}+\frac{3}{2}\left(1-3\cos^{2}{{\theta_{ij}}}\right)\left(\frac{\cos{(k{r}_{ij})}}{(kr_{ij})^{2}}-\frac{\sin{(k{r}_{ij})}}{(kr_{ij})^{3}}\right),$
(4)
$\displaystyle{{\Omega}_{ij}}=-\frac{3}{4}\left(1-\cos^{2}{{\theta_{ij}}}\right)\frac{\cos{(k{r}_{ij})}}{{k{r}_{ij}}}+\frac{3}{4}\left(1-3\cos^{2}{{\theta_{ij}}}\right)\left(\frac{\sin{(k{r}_{ij})}}{(kr_{ij})^{2}}+\frac{\cos{(k{r}_{ij})}}{(kr_{ij})^{3}}\right),$
(5)

while $\theta_{ij}$ is the angle between the direction of atom dipole moments
and the vector joining the $i$th and the $j$th atom, whose distance is
${{r}_{ij}}=\left|{\mathbf{\hat{{r}_{i}}}}-{\mathbf{\hat{{r}_{j}}}}\right|$.
The dipole-dipole interaction can be described based on virtual photons
exchange between atoms and become the strongest at small interatomic
distances. The last term of Eq. (3), describes the cooperative emission.

We are interested to the one photon spectrum (1PS) of the system which is
defined by the Fourier transform of the first-order correlation function of
the electric field as bellow [9]

$\displaystyle
S\left(\omega\right)=\underset{T\to\infty}{\mathop{\lim}}\,\underset{t\to\infty}{\mathop{\lim}}\,\int_{-T}^{T}{d\tau{{g}^{(1)}}\left(t,\tau\right){{e}^{-i\omega\tau}}}.$
(6)

This spectrum is valid in the steady state of the system $t\to\infty$ and the
first order correlation function emitted in the direction of the normalized
vector $\mathbf{\hat{n}}$ is given by

$\displaystyle{{g}^{(1)}}\left(t,\tau\right)=\frac{\left\langle
E\left(\mathbf{\hat{n}},t\right){{E}^{\dagger}}\left(\mathbf{\hat{n}},t+\tau\right)\right\rangle}{\left\langle
E\left(\mathbf{\hat{n}},t\right){{E}^{\dagger}}\left(\mathbf{\hat{n}},t\right)\right\rangle},$
(7)

with the far field electric field operator of
${{E}^{\dagger}}\left(\mathbf{\hat{n}},t\right)=\sum\limits_{j=1}^{N}{\sigma_{j}^{-}\left(t\right){{e}^{-ik\mathbf{\hat{n}}.{{\mathbf{r}}_{j}}}}}$.
The quantum regression theorem, commonly used for two-time observables is used
in calculation of ${{g}^{(1)}}\left(t,\tau\right)$ [14, 15, 16, 17]. We
calculated it numerically using qutip toolbox. We integrated Eq. (1)
numerically, for enough long times to obtain the density matrix of the system
at the steady state, then we calculated two-time correlation function of
${{g}^{(1)}}\left(t,\tau\right)$.

##  III Dressed Energy Levels

In this section we study the collective $N$ atom dressed energy levels. Since
this study needs an exact diagonalization of the Hamiltonian which is a large-
scale NP problem and intractable due to the exponential growth of the
computational costs with the input size, we solve this problem setting $N=3$,
the extension to the large scale system is straightforward but very
computational resource demanding. We will select two spatial configuration of
three atoms located in vertices of an equilateral triangle and in vertices of
an isosceles triangle. For each spatial configuration we will obtain the
dressed levels and their transitions related to various sidebands in the 1PS.

###  III.1 Equilateral Triangle Spatial Configuration

Dressed eigenstates of the single atoms that are at resonance with the driving
field can be written as
$\left|\pm\right\rangle=\frac{1}{\sqrt{2}}\left(\left|\uparrow,n-1\right\rangle\pm\left|\downarrow,n\right\rangle\right)$.
It was shown that turning on the strong interaction in two emitters strongly
driven by a laser field leads the collective effects in their dressed energy
levels [11]. Increasing the number of atoms up to three when they are located
in vertices of an equilateral triangle cause that the dipole-dipole
interaction in Eq. (2) generates collective single- or double-excitation
eigenstates of

$\displaystyle\left|\phi_{1}\right\rangle=a_{1}\left(\left|\uparrow\downarrow\downarrow\right\rangle+\left|\downarrow\uparrow\downarrow\right\rangle+\left|\downarrow\downarrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{2}\right\rangle=a_{2}\left|\uparrow\downarrow\downarrow\right\rangle+a_{3}\left(\left|\downarrow\uparrow\downarrow\right\rangle+\left|\downarrow\downarrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{3}\right\rangle=a_{4}\left(\left|\downarrow\uparrow\downarrow\right\rangle-\left|\downarrow\downarrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{4}\right\rangle=a_{5}\left(\left|\uparrow\uparrow\downarrow\right\rangle+\left|\uparrow\downarrow\uparrow\right\rangle+\left|\downarrow\uparrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{5}\right\rangle=a_{6}\left|\uparrow\uparrow\downarrow\right\rangle+a_{7}\left(\left|\uparrow\downarrow\uparrow\right\rangle+\left|\downarrow\uparrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{6}\right\rangle=a_{8}\left(\left|\uparrow\downarrow\uparrow\right\rangle-\left|\downarrow\uparrow\uparrow\right\rangle\right),$
(8)

in addition to the ground and three excitation states of

$\displaystyle\left|\phi_{0}\right\rangle=\left|\downarrow\downarrow\downarrow\right\rangle$
$\displaystyle\left|\phi_{7}\right\rangle=\left|\uparrow\uparrow\uparrow\right\rangle.$
(9)

Eigenstates introduced in Eqs. (III.1, III.1) can be obtained numerically by
diagonalization of the Hamiltonian

$\displaystyle{{H_{lab}}_{,\sigma}}=\sum\limits_{i}{{H^{\prime}}_{\sigma
i}}+\sum\limits_{i,j\neq i}{{H}_{\sigma
ij}}=\sum\limits_{i}{{{\omega}_{a}}\sigma_{i}^{z}}+\Gamma\sum\limits_{i,j\neq
i}{{\Omega}_{ij}}\sigma_{i}^{+}\sigma_{j}^{-},$ (10)

in the lab frame where $\sigma_{i}^{z}$ is the energy operator of the ith
atom. Then $a_{i}$ in Eqs. (III.1, III.1) are obtained by normalization
condition. In this paper we consider the strong atom-atom interaction of
$\left|{{\Omega}_{ij}}\right|\gg 1$ and ${{\Gamma}_{ij}}\approx 1$, and strong
driving which is defined by the relation
$\Omega^{2}\gg\Gamma^{2}+4|\Gamma\Omega_{ij}|^{2}$. In addition to the Rabi
frequency of the driving field, this condition can be obtained by tuning the
atomic distances $k{{r}_{ij}}$, and the orientation of the atomic dipoles
${{\theta}_{ij}}$, in Eqs. (5) and (4).

(a)

(b)

Figure 1: (a) Collective dressed energy levels whose transitions between
manifolds account for the main phenomenology of 1PS, shown in the rotating
frame of the laser. (b) Fluorescence spectrum. Both (a) and (b) are related to
three strongly interacting atoms in the equilateral triangle spatial
configuration. Different peaks are indicated with different $T_{i}$ signs
corresponding to the related transitions in dressed levels. $T^{\prime}_{i}$
are not shown in the spectrum as they are symmetrical with the $T_{i}$
transitions. The inset shows frequency range of $\omega=[-230,-180]$ in order
to see it in higher resolution. The results were obtained by driving three
strongly interacting TLS resonantly with $\Omega=200\Gamma$, $\Gamma$ which is
the decay rate of the individual atoms sets the frequency unit throughout the
text. The interaction between atoms is determined by setting $kr=0.01$ and
$\theta={{\cos}^{-1}}(\frac{1}{\sqrt{3}})$ in Eqs. (3) and (4). Also
${{\omega}_{L}}$ (laser frequency) , is taken as a reference for the
frequency.

We follow the approach of [18] to study collective three atoms dressed levels.
We consider the following basis

$\displaystyle\left|\phi_{0}^{n}\right\rangle=\left|\phi_{0}\right\rangle\left|n\right\rangle$
$\displaystyle\left|\phi_{i}^{n}\right\rangle=\left|\phi_{i}\right\rangle\left|n-1\right\rangle\quad
i=1,2,3$
$\displaystyle\left|\phi_{j}^{n}\right\rangle=\left|\phi_{j}\right\rangle\left|n-2\right\rangle\quad
j=4,5,6$
$\displaystyle\left|\phi_{7}^{n}\right\rangle=\left|\phi_{7}\right\rangle\left|n-3\right\rangle.$
(11)

The above eight-dimensional subspace is characterized by an eigenvalue of the
operator
${{N}_{T}}={N_{\nu}}+\sum\limits_{i}{\frac{\sigma_{i}^{z}}{2}}+\frac{1}{2}$,
where ${N_{\nu}}$ is the photon number. For $N$ particles system the above
subspace is ${{2}^{N}}$-dimensional and the photon number is restored to
obtain hybrid atom-field states having the same eigenvalue of ${{N}_{T}}$. In
this new basis, the eigenstates of the Hamiltonian in Eq. (2), which are
“atoms-light” collective dressed state, reads

$\displaystyle\left|u_{0}^{n}\right\rangle=b_{1}\left|\phi_{0}^{n}\right\rangle+b_{2}\left|\phi_{1}^{n}\right\rangle+b_{3}\left|\phi_{4}^{n}\right\rangle+b_{4}\left|\phi_{7}^{n}\right\rangle$
$\displaystyle\left|u_{1}^{n}\right\rangle=b_{5}\left|\phi_{0}^{n}\right\rangle+b_{6}\left|\phi_{1}^{n}\right\rangle+b_{7}\left|\phi_{4}^{n}\right\rangle+b_{8}\left|\phi_{7}^{n}\right\rangle$
$\displaystyle\left|u_{2}^{n}\right\rangle=b_{9}\left|\phi_{2}^{n}\right\rangle+b_{10}\left|\phi_{3}^{n}\right\rangle+b_{11}\left|\phi_{5}^{n}\right\rangle+b_{12}\left|\phi_{6}^{n}\right\rangle$
$\displaystyle\left|u_{3}^{n}\right\rangle=b_{13}\left|\phi_{2}^{n}\right\rangle+b_{14}\left|\phi_{3}^{n}\right\rangle+b_{15}\left|\phi_{5}^{n}\right\rangle+b_{16}\left|\phi_{6}^{n}\right\rangle$
$\displaystyle\left|u_{4}^{n}\right\rangle=b_{17}\left|\phi_{0}^{n}\right\rangle+b_{18}\left|\phi_{1}^{n}\right\rangle+b_{19}\left|\phi_{4}^{n}\right\rangle+b_{20}\left|\phi_{7}^{n}\right\rangle$
$\displaystyle\left|u_{5}^{n}\right\rangle=b_{21}\left|\phi_{2}^{n}\right\rangle+b_{22}\left|\phi_{3}^{n}\right\rangle+b_{23}\left|\phi_{5}^{n}\right\rangle+b_{24}\left|\phi_{6}^{n}\right\rangle$
$\displaystyle\left|u_{6}^{n}\right\rangle=b_{25}\left|\phi_{2}^{n}\right\rangle+b_{26}\left|\phi_{3}^{n}\right\rangle+b_{27}\left|\phi_{5}^{n}\right\rangle+b_{28}\left|\phi_{6}^{n}\right\rangle$
$\displaystyle\left|u_{7}^{n}\right\rangle=b_{29}\left|\phi_{0}^{n}\right\rangle+b_{30}\left|\phi_{1}^{n}\right\rangle+b_{31}\left|\phi_{4}^{n}\right\rangle+b_{32}\left|\phi_{7}^{n}\right\rangle,$
(12)

where $\left|u_{i}^{n}\right\rangle$ and $b_{i}$ are obtained by numerical
diagonalization of the Hamiltonian (2) and then writing its eigenstates in the
basis of III.1, in addition to the normalization condition.

The “atoms-light” collective dressed states of this configuration is thus an
octet of $\left|u_{n}^{i}\right\rangle$, with $i=0,1,..,7$ and numerically
obtained eigenenergies $E_{n}^{i}$. Dressed states of each manifold are non-
equally spaced by

$\Delta_{ij}=-\Delta_{ji}\equiv E_{n}^{i}-E_{n}^{j},$ (13)

and the energy difference between two neighboring manifolds is equal to
${{\omega}_{L}}$. Manifolds of collective dressed states of three strongly
interacting atoms in equilateral triangle configuration are shown in Fig.
(1a).

The spectral shape of 1PS can be obtained by solving the master equation of
Eq. (1), and this spectrum is understood as transitions between the dressed
levels. Similar to the one atom dressed levels, transitions from one level of
higher energy manifold (with $E_{n}^{i}$) to the same energy level in the
lower energy one (with $E_{n-1}^{i}$), leads to the central peak. It leaves
the state of the three TLS unchanged
($\left|u_{n}^{i}\right\rangle\to\left|u_{n-1}^{i}\right\rangle$ for
$i=0,1,..,7$), and the transition frequency is ${{\omega}_{L}}$. Other various
transitions and the related peaks are shown in Figs. (1a) and (1b). All
transitions are hereafter given in the laser frame.

Dressed energy levels are characterized by entanglement between atomic and
field states, and various transitions between these levels that are matched
with the peaks of the 1PS will be explained based on these couplings. For
instance as it can be seen from the Eqs. (III.1)
$\left|u_{n}^{2}\right\rangle$ and $\left|u_{n}^{3}\right\rangle$ are coupled
only with $\left|u_{n}^{5}\right\rangle$ and $\left|u_{n}^{6}\right\rangle$
thus they have transitions only between each other in the successive
manifolds. Note that $\left|u_{n}^{2}\right\rangle$ and
$\left|u_{n}^{3}\right\rangle$ are degenerate, the same holds true for the
$\left|u_{n}^{5}\right\rangle$ and $\left|u_{n}^{6}\right\rangle$. Other
states that are not coupled to these four energy levels do not have
transitions to (or from) them. Consequently it can be seen from Fig. (1) that
$\left|u_{n}^{i}\right\rangle$ with $i=0,1,4,7$ transit only between each
other. These couplings are the main responsible for the fourteen number of
peaks in this configuration. In the following of the text we will study the
couplings and sidebands of the fluorescence spectrum when atoms are located in
the vertices of an isosceles triangle.

The fluorescence spectrum of Fig. (1b) indicates pure collective
(configuration induced) effects by the presence of fourteen sidebands, with
frequency differences compared to the central peak of $\pm\Delta_{ij}$, in
this configuration of atoms.

###  III.2 Isosceles Triangle Spatial Configuration

When three atoms located at the vertices of an isosceles triangle, the
collective single- and double -excitation eigenstates, together with the
ground and three exited states are:

$\displaystyle\left|\phi_{0}\right\rangle=\left|\downarrow\downarrow\downarrow\right\rangle$
$\displaystyle\left|\phi_{1}\right\rangle=a_{1}\left|\uparrow\downarrow\downarrow\right\rangle+a_{2}\left(\left|\downarrow\uparrow\downarrow\right\rangle+\left|\downarrow\downarrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{2}\right\rangle=a_{3}\left(\left|\downarrow\uparrow\downarrow\right\rangle-\left|\downarrow\downarrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{3}\right\rangle=a_{4}\left|\uparrow\downarrow\downarrow\right\rangle+a_{5}\left(\left|\downarrow\uparrow\downarrow\right\rangle+\left|\downarrow\downarrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{4}\right\rangle=a_{6}\left|\downarrow\uparrow\uparrow\right\rangle+a_{7}\left(\left|\uparrow\uparrow\downarrow\right\rangle+\left|\uparrow\downarrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{5}\right\rangle=a_{8}\left(\left|\uparrow\uparrow\downarrow\right\rangle-\left|\uparrow\downarrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{6}\right\rangle=a_{9}\left|\downarrow\uparrow\uparrow\right\rangle+a_{10}\left(\left|\uparrow\uparrow\downarrow\right\rangle+\left|\uparrow\downarrow\uparrow\right\rangle\right)$
$\displaystyle\left|\phi_{7}\right\rangle=\left|\uparrow\uparrow\uparrow\right\rangle.$
(14)

Since the configuration of atoms is different from the equilateral triangle,
the basis of (III.2) which is equivalent to the basis of (III.1,III.1), i.e.
the eigenstates of the (10) in the lab frame has different couplings. In this
structure of the atoms, since the $\left|\phi_{i}\right\rangle$ have the same
atomic excitation compared to the equilateral triangle configuration, we can
again consider the basis of Eq.(III.1) and obtain eigenstates of the
Hamiltonian of (2) in this basis.

The eigenstates of the atom-light system are composed by the collective
dressed states incorporating the eigenstates of Hamiltonian (2) for atoms with
light-mediated dipole-dipole interactions, and the photon number states of the
light field i.e., the $n$-excitation manifold for our system is given by

$\displaystyle\left|u_{0}^{n}\right\rangle=$ $\displaystyle
d_{1}\left|\phi_{0}^{n}\right\rangle+d_{2}\left|\phi_{1}^{n}\right\rangle+d_{3}\left|\phi_{3}^{n}\right\rangle+d_{4}\left|\phi_{4}^{n}\right\rangle$
$\displaystyle+d_{5}\left|\phi_{6}^{n}\right\rangle+d_{6}\left|\phi_{7}^{n}\right\rangle$
$\displaystyle\left|u_{1}^{n}\right\rangle=$ $\displaystyle
d_{7}\left|\phi_{0}^{n}\right\rangle+d_{8}\left|\phi_{1}^{n}\right\rangle+d_{9}\left|\phi_{3}^{n}\right\rangle+d_{10}\left|\phi_{4}^{n}\right\rangle$
$\displaystyle+d_{11}\left|\phi_{6}^{n}\right\rangle+d_{12}\left|\phi_{7}^{n}\right\rangle$
$\displaystyle\left|u_{2}^{n}\right\rangle=$ $\displaystyle
d_{13}\left|\phi_{2}^{n}\right\rangle+d_{14}\left|\phi_{5}^{n}\right\rangle$
$\displaystyle\left|u_{3}^{n}\right\rangle=$ $\displaystyle
d_{15}\left|\phi_{0}^{n}\right\rangle+d_{16}\left|\phi_{1}^{n}\right\rangle+d_{17}\left|\phi_{3}^{n}\right\rangle+d_{18}\left|\phi_{4}^{n}\right\rangle$
$\displaystyle+d_{19}\left|\phi_{6}^{n}\right\rangle+d_{20}\left|\phi_{7}^{n}\right\rangle$
$\displaystyle\left|u_{4}^{n}\right\rangle=$ $\displaystyle
d_{21}\left|\phi_{0}^{n}\right\rangle+d_{22}\left|\phi_{1}^{n}\right\rangle+d_{23}\left|\phi_{3}^{n}\right\rangle+d_{24}\left|\phi_{4}^{n}\right\rangle$
$\displaystyle+d_{25}\left|\phi_{6}^{n}\right\rangle+d_{26}\left|\phi_{7}^{n}\right\rangle$
$\displaystyle\left|u_{5}^{n}\right\rangle=$ $\displaystyle
d_{27}\left|\phi_{2}^{n}\right\rangle+d_{28}\left|\phi_{5}^{n}\right\rangle$
$\displaystyle\left|u_{6}^{n}\right\rangle=$ $\displaystyle
d_{29}\left|\phi_{0}^{n}\right\rangle+d_{30}\left|\phi_{1}^{n}\right\rangle+d_{31}\left|\phi_{3}^{n}\right\rangle+d_{32}\left|\phi_{4}^{n}\right\rangle$
$\displaystyle+d_{33}\left|\phi_{6}^{n}\right\rangle+d_{34}\left|\phi_{7}^{n}\right\rangle$
$\displaystyle\left|u_{7}^{n}\right\rangle=$ $\displaystyle
d_{35}\left|\phi_{0}^{n}\right\rangle+d_{36}\left|\phi_{1}^{n}\right\rangle+d_{37}\left|\phi_{3}^{n}\right\rangle+d_{38}\left|\phi_{4}^{n}\right\rangle$
$\displaystyle+d_{39}\left|\phi_{6}^{n}\right\rangle+d_{40}\left|\phi_{7}^{n}\right\rangle.$
(15)

As it can be seen from Eq. (III.2), $\left|u_{2}^{n}\right\rangle$ and
$\left|u_{5}^{n}\right\rangle$ are coupled only with each other thus their
transitions in successive manifolds are limited only between themselves. As a
result other dressed levels of $\left|u_{i}^{n}\right\rangle$, $i=0,1,3,4,6,7$
have transitions only between each other since they do not coupled to
$\left|u_{2}^{n}\right\rangle$ and $\left|u_{5}^{n}\right\rangle$. This can be
understood well by checking the energy of the peaks in 1PS and compare them
with the (numerically obtained) transition energies between various dressed
levels. Transitions between dressed energy levels and the related peaks in the
1PS are shown in Figs. (2).

Having the same atomic excitation for two states leads to the same value for
the $N_{\nu}$ -the photon number operator- as well, thus transitions between
them in successive manifolds may lead to a peak at $\omega_{L}$ which is set
at the origin of the 1PS plots. One example of these transitions is
$\left|u_{4}^{n}\right\rangle\to\left|u_{6}^{n-1}\right\rangle$. Checking
carefully the sidebands energies of the 1PS, one can see that there is no peak
matched with the transition energy of these two levels in the successive
manifolds and since in this transition only the photon excitation number $n$
will be decreased, thus it is equivalent to the
$\left|u_{i}^{n}\right\rangle\to\left|u_{i}^{n-1}\right\rangle$. Note that in
the equilateral triangle configuration $\left|u_{4}^{n}\right\rangle$ and
$\left|u_{6}^{n}\right\rangle$ are not coupled to each other and we didn’t see
this effect.

(a)

(b)

Figure 2: (a) Collective dressed energy levels whose transitions between
manifolds account for the main phenomenology of 1PS, shown in the rotating
frame of the laser. (b) Fluorescence spectrum. Both (a) and (b) are related to
three strongly interacting atoms in the isosceles triangle spatial
configuration. Different peaks are indicated with different $T_{i}$ signs
corresponding to the related transitions in dressed levels. $T^{\prime}_{i}$
are not shown in the spectrum as they are symmetrical with the $T_{i}$
transitions. Note that the transitions of $T_{7}$ and $T_{12}$ are degenerate,
the same holds true for the $T_{9}$ and $T_{11}$. The inset shows frequency
range of $\omega=[-300,-100]$ in order to see it in higher resolution. The
results were obtained by driving three strongly interacting TLS resonantly
with $\Omega=200\Gamma$, $\Gamma$ which is the decay rate of the individual
atoms sets the frequency unit throughout the text. The interaction between
atoms is determined by setting $kr=0.01$ for the two equal sides of the
triangle when the angle between them is $\pi/2$ and
$\theta={{\cos}^{-1}}(\frac{1}{\sqrt{3}})$ in Eqs. (3) and (4). Also
${{\omega}_{L}}$ (laser frequency) , is taken as a reference for the
frequency.

The transformation into a new couplings between dressed states, due to the new
configuration of dipole-dipole interactions, leads to twenty-four sidebands in
the 1PS as shown in Fig. (2b). These sidebands are collective and
configuration induced, corresponding to twenty-four resonant frequencies
$\pm{{\Delta}_{ij}}$ not presented for equilateral triangle.

##  IV Conclusion and perspectives

Strong interactions between three two-level emitters give rise to a series of
new sidebands in the fluorescence spectrum, whose shifts from the atomic
transition and their number depend on the couplings among dressed levels which
in turn depend on spatial configuration of atoms, in addition to the
interaction strength and the driving field. Similarly to the three emitters,
for a system of many particles the appearance of more additional sidebands is
expected. This can be shown by exact diagonalization of the Hamiltonian which
is a large-scale NP problem and intractable due to the exponential growth of
the computational costs with the input size. To circumvent this problem, one
needs to deal with approximated methods, for instance the Bogoliubov-Born-
Green-Kirkwood-Yvon (BBGKY) hierarchy [9]. Benchmarking of this approximation
against exact results was discussed in [9] only for the limit of weak
interaction explained in [11] which occurs when the distance between the
emitters is comparable or larger than the optical wavelength. In this regime
the BBGKY hierarchy is in very good agreement with the exact results obtained
by the “Quantum Toolbox in Python” [9].

Besides the presented results in this paper, we also have monitored the BBGKY
hierarchy results (without a discrete phase space representation of atoms [19,
20] which is used in [9, 21]), and compared them with the exact solution in
the strong interaction regime. We used BBGKY hierarchy considering up to third
order of correlation which is the best approximation for four particles and
suitable for the quantity of 7 as it is a two time two particle correlation
function. We neglect the 3-operators terms of
$\left\langle\sigma_{i}^{\alpha}\sigma_{j}^{\beta}\sigma_{k}^{\gamma}\right\rangle$
with $i\neq j,k$. Furthermore, we neglect the 4-operator terms, since they
involve 3-operators with $i\neq j\neq k$.

For a chain of four atoms with $kr=0.01$, $\theta=\cos^{-1}(1/\sqrt{3})$,
$\Omega=200$, the above explained BBGKY hierarchy didn’t give the correct
number and energies of the sidebands, as compared to the exact solution (not
shown here). The same holds true for five atoms at the same spatial
configuration and interaction parameters. This confirms that despite the
method is highly accurate for large scale systems with weak long-range
interactions, the accuracy is not happening in the strong interaction regime.
Note that based on the transparency-induced dark states [22, 23, 24, 25],
which allows for the generation of subwavelength cold atom structures [26, 27,
28, 29], some designs have been suggested to surpass limitations of
manipulating atoms in scales below the diffraction limit. It is needed to
mention that the strong interaction regime can be obtained by tuning both the
atomic distances and also the angle between atoms dipole moments and vector
joining them.

## Acknowledgment

E. D. benefited from Grant from São Paulo Research Foundation (FAPESP, Grant
Number 2018/10813-2.

## References

  * Cohen-Tannoudji and Reynaud [1977] C. Cohen-Tannoudji and S. Reynaud, J. Phys. B 10, 365 (1977).
  * Reynaud [1983] S. Reynaud, Ann. Phys. 8, 315 (1983).
  * Carmichael and Walls [1976] H. J. Carmichael and D. F. Walls, J. Phys. B 9, 1199 (1976).
  * Scully and Zubairy [1997] M. O. Scully and M. S. Zubairy,  _Quantum Optics_ (Cambridge University Press, 1997).
  * Cohen-Tannoudji  _et al._ [1998] C. Cohen-Tannoudji, J. Dupont-Roc, and G. Grynberg,  _Atom—Photon Interactions_ (Wiley, 1998).
  * Senitzky [1978] I. R. Senitzky, Phys. Rev. Lett. 40, 1334 (1978).
  * Agarwal  _et al._ [1980] G. S. Agarwal, R. Saxena, L. M. Narducci, D. H. Feng, and R. Gilmore, Phys. Rev. A 21, 257 (1980).
  * Ben-Aryeh and Bowden [1988] Y. Ben-Aryeh and C. Bowden, IEEE J. Quantum Electron. 24, 1376 (1988).
  * Pucci  _et al._ [2017] L. Pucci, A. Roy, T. S. do Espirito Santo, R. Kaiser, M. Kastner, and R. Bachelard, Phys. Rev. A 95, 053625 (2017).
  * an Peng  _et al._ [2019] Z. an Peng, G. qing Yang, Q. lin Wu, and G. xiang Li, Phys. Rev. A 99, 033819 (2019).
  * Darsheshdar  _et al._ [2020] E. Darsheshdar, M. Hugbart, R. Bachelard, and C. J. Villas-Boas, arXiv e-prints , arXiv:2012.03735 (2020), arXiv:2012.03735 [quant-ph] .
  * Agarwal [1974] G. S. Agarwal, in  _Springer Tracts in Modern Physics_ (Springer Berlin Heidelberg, 1974).
  * Das  _et al._ [2008] S. Das, G. S. Agarwal, and M. O. Scully, Phys. Rev. Lett. 101, 153601 (2008).
  * Gardiner and Zoller [2014] C. Gardiner and P. Zoller,  _The Quantum World of Ultra-Cold Atoms and Light Book I: Foundations of Quantum Optics_ (Imperial College Press, 2014).
  * Gisin [1993] N. Gisin, J. Mod. Opt. 40, 2313 (1993).
  * Brun and Gisin [1996] T. A. Brun and N. Gisin, J. Mod. Opt. 43, 2289 (1996).
  * Breuer  _et al._ [1997] H.-P. Breuer, B. Kappler, and F. Petruccione, Phys. Rev. A 56, 2334 (1997).
  * Compagno  _et al._ [1995] G. Compagno, R. Passante, and F. Persico, in  _Atom-Field Interactions and Dressed Atoms_ (Cambridge University Press, 1995).
  * Wootters [1987] W. K. Wootters, Ann. Phys. (N. Y.) 176, 1 (1987).
  * Schachenmayer  _et al._ [2015] J. Schachenmayer, A. Pikovski, and A. Rey, Phys. Rev. X 5, 011022 (2015).
  * Pucci  _et al._ [2016] L. Pucci, A. Roy, and M. Kastner, Phys. Rev. B 93, 174302 (2016).
  * Agarwal and Kapale [2006] G. S. Agarwal and K. T. Kapale, J. Phys. B 39, 3437 (2006).
  * Cho [2007] J. Cho, Phys. Rev. Lett. 99, 020502 (2007).
  * Yavuz and Proite [2007] D. D. Yavuz and N. A. Proite, Phys. Rev. A 76, 041802 (2007).
  * Gorshkov  _et al._ [2008] A. V. Gorshkov, L. Jiang, M. Greiner, P. Zoller, and M. D. Lukin, Phys. Rev. Lett. 100, 093005 (2008).
  * Miles  _et al._ [2013] J. A. Miles, Z. J. Simmons, and D. D. Yavuz, Phys. Rev. X 3, 031014 (2013).
  * Wang  _et al._ [2018] Y. Wang, S. Subhankar, P. Bienias, M. Łacki, T.-C. Tsui, M. A. Baranov, A. V. Gorshkov, P. Zoller, J. V. Porto, and S. L. Rolston, Phys. Rev. Lett. 120, 083601 (2018).
  * Subhankar  _et al._ [2019] S. Subhankar, P. Bienias, P. Titum, T.-C. Tsui, Y. Wang, A. V. Gorshkov, S. L. Rolston, and J. V. Porto, New J. Phys. 21, 113058 (2019).
  * Tsui  _et al._ [2020] T.-C. Tsui, Y. Wang, S. Subhankar, J. V. Porto, and S. L. Rolston, Phys. Rev. A 101, 113058 (2020).