# CALCS 2021 Shared Task: Machine Translation for Code-Switched Data

Shuguang Chen 1, Gustavo Aguilar 1, Anirudh Srinivasan 2, Mona Diab 3 Thamar
Solorio 1
University of Houston 1
The University of Texas at Austin 2
Meta AI 3
{schen52, gaguilaralas<EMAIL_ADDRESS>1
<EMAIL_ADDRESS>2
<EMAIL_ADDRESS>3

###### Abstract

To date, efforts in the code-switching literature have focused for the most
part on language identification, POS, NER, and syntactic parsing. In this
paper, we address machine translation for code-switched social media data. We
create a community shared task. We provide two modalities for participation:
supervised and unsupervised. For the supervised setting, participants are
challenged to translate English into Hindi-English (Eng-Hinglish) in a single
direction. For the unsupervised setting, we provide the following language
pairs: English and Spanish-English (Eng-Spanglish), and English and Modern
Standard Arabic-Egyptian Arabic (Eng-MSAEA) in both directions. We share
insights and challenges in curating the "into" code-switching language
evaluation data. Further, we provide baselines for all language pairs in the
shared task. The leaderboard for the shared task comprises 12 individual
system submissions corresponding to 5 different teams. The best performance
achieved is 12.67% BLEU score for English to Hinglish and 25.72% BLEU score
for MSAEA to English.

##  1 Introduction

Linguistic code-switching refers to the linguistic phenomenon of alternating
between two or more languages or varieties of language both across sentences
and within sentences (aka intra sentential code-switching, which is the type
we focus on in this work)(Joshi, 1982). The wide use of social media platforms
(e.g, Twitter, Facebook, Reddit, etc.) where users communicate with each other
more spontaneously, rendering significantly more written code-switched data.
Accordingly, this raises an increasing demand for more tools and resources to
process code-switched data. However, current NLP technology is lagging in the
development of resources and methodologies that can effectively process such
phenomena. This is true for even the large multilingual pre-trained models
such as mBART (Liu et al., 2020) and mT5 (Xue et al., 2021). At the same time,
the growing adoption of smart devices and automated assistants that rely on
speech interfaces makes it even more pressing for the NLP field to deal with
code-switched language data.

Figure 1: Examples from the dataset. The irregular grammar and non-standard
syntactic rules make it difficult to translate from one language to the other.

In this work, we present the new task of machine translation (MT) from and to
Code Switched language. We create a community task, defining and curating
datasets and evaluation metrics. We leverage already existing standard code-
switching datasets that were created in the community as used in the previous
workshops on Computational Approaches to Linguistic Code-Switching (CALCS). We
created a series of shared tasks focusing primarily on enabling technology for
code-switching, including language identification (LID) (Solorio et al.,
2014), part of speech (POS) tagging (Molina et al., 2016), and named entity
recognition (NER) (Aguilar et al., 2018). In our task, we leverage previously
created data sets and extend them for the task of machine translation (MT)
under code-switching settings in multiple language combinations and
directions. General challenges to the processing of code-switched language
persist in this setting but notably, the following challenges are especially
significant to the MT problem space: 1) irregular grammar and spelling of
utterances; 2) non-standard syntactic rules; and 3) ambiguous words (e.g,
‘car‘ in French and English), etc. For example, in Figure 1, the irregular
grammar and non-standard syntactic rules make it difficult to translate from
one language to the other. With this in mind, we host a new community task for
machine translation on code-switched data. The goal is to provide code-
switched MT datasets and to continue to motivate new research and energize the
NLP community to take on the challenges posed by code-switched data.

For the community task, we provide two modes for participation: supervised and
unsupervised. For the supervised setting, we ask participants to translate
English into Hindi-English (Eng-Hinglish) in a single direction. For the
unsupervised setting, we provide the following language pairs: English and
Spanish-English (Eng-Spanglish), and English and Modern Standard Arabic-
Egyptian Arabic (Eng-MSAEA) in both directions. We received 12 individual
public system submissions from 5 different teams. The best performance
achieved is 12.67% BLEU score for English to Hinglish and 25.72% BLEU score
for MSAEA to English. Our contributions are multi-fold:

  1. 1.

We introduce the novel task of translating into code-switched language;

  2. 2.

We create new standard datasets for evaluating MT for code-switched data:
English$\rightarrow$Hinglish, English-Modern Standard Arabic-Egyptian Arabic,
and English-Spanglish;

  3. 3.

We provide baseline systems for MT for several pairs of languages: English-
Hinglish, English-Spanglish;

  4. 4.

We present and discuss some of the challenges of generating evaluation data
for code-switched language.

##  2 Related Work

Code-switching is a phenomenon where multilingual speakers communicate by
switching back and forth between the languages they speak or write when
communicating with other multilingual speakers (Joshi, 1982). It tends to be
more common in informal settings such as social media platforms where
interactions are more casual. Previous work has studied code-switching for
many languages from different perspectives. Toribio (2001) presents new
methodologies for gathering code-switched data, examining permissible and
unacceptable language alternations. Goyal et al. (2003) introduces a bilingual
syntactic parser that operates on input strings from Hindi and English and
develops grammars for Hindi and English. Lately, researchers mainly focus on
exploring this phenomenon at the core level of NLP pipelines, including
language identification (Solorio and Liu, 2008; Aguilar and Solorio, 2020),
language modeling (Garg et al., 2018; Gonen and Goldberg, 2019), and Named
Entity Recognition (Winata et al., 2019; Aguilar et al., 2018), etc.

Recent advances in large pre-trained models (Devlin et al., 2019; Lewis et
al., 2020) have led to significant performance gains in many multilingual
tasks. Without further pre-training and extra neural network components, these
large pre-trained models (e.g, mBART (Liu et al., 2020) and mT5 (Xue et al.,
2021)) can be easily adapted to code-switching tasks. However, the pre-
training tasks of these models are performed on monolingual text without
language alternation. Applying these techniques to the task of code-switching
machine translation may not result in a good performance due to the challenges
mentioned in Section 1.

Machine translation on code-switched data has been recently receiving
increasing attention in the NLP community. For this task, there are two main
focused efforts: 1) data resources, and 2) efficient methods. For data
resources, Srivastava and Singh (2020) presents a parallel corpus for Hindi-
English (Hinglish) and English and proposes a translation pipeline built on
top of Google Translate. Tarunesh et al. (2021) adapts a neural machine
translation model to automatically generate code-switching Hindi-English data.
However, due to the lack of resources, this phenomenon remains challenging,
especially for social media genres where annotated data is limited. In this
work, in addition to the Hinglish-English dataset, we further propose
standardized datasets for translating from monolingual data to code-switched
data. Critically, we are proposing standard datasets for evaluating to and
from code-switched data. For efficient methods, Singh and Solorio (2017)
employs the n-best-list generation approach to address the spelling variation
in the text and also enrich the resource for translation. Yang et al. (2020)
proposes a simple yet effective pre-training approach to align the cross-
lingual information between the source and target language. Xu and Yvon (2021)
designs a new method to simultaneously disentangle and translate the two mixed
languages. In this paper, we compare different systems from the participating
teams in terms of methodology (e.g, model architecture and training strategy)
and performance. We further provide in-depth analysis. In the spirit of
reproducibility, we make our data and experiments publicly available to
support future research in this area.

##  3 Task Description

The goal of the community task is to motivate new research and energize the
community to take on challenges posed by code-switched data. Most research to
date has focused on translation from code-switched language to monolingual
language, albeit without establishing standard data sets for the task. In our
work, we establish standard datasets in three different language pairs.
Moreover, we introduce the novel task of translating into code-switched
language. Generating plausible code-switched language is a challenge for
systems in general and it is still an untapped novel territory in NLP. As
mentioned earlier, code-switching is pervasive among multilingual speakers as
well as among even monolingual speakers given the global nature of
communication. Catering to such users requires systems that are able to
produce more natural language consistent and coherent with their manner of
speaking, e.g. in conversational AI. We contend that such natural language
interactions with code-switching understanding and generation will be the next
frontier in human language technology interaction. Hence an initial challenge
is to create standardized evaluation datasets that reflect the code-switched
phenomenon. Machine translation serves both ends exemplifying understanding
and generation. With this in mind, we hosted a new shared community task on
machine translation for code-switched language. The shared task mainly focuses
on translating a source sentence into a target language while one of the
directions contains an alternation between two languages.111We constrain the
problem space to code-switching being on one side of the directions for this
initial iteration of the task, however, we plan on extending the datasets to
include code-switched data source to code-switched data target in the near
future. Given a monolingual/code-switched sentence, the goal is to translate
it into a code-switched/monolingual sentence, respectively. In this shared
task, we provide the following two settings for participation:

#### Supervised settings

We primarily focus on translating from monolingual text to code-switched text.
This setting only has one language pair with a single direction, i.e,
English$\rightarrow$Hinglish. We provide parallel training data and
participants are required to build systems for translating an English sentence
to a Hinglish sentence.

#### Unsupervised settings

This setting comprises two language pairs: English to/from Spanish-English
(Eng-Spanglish), and English to/from Modern Standard Arabic-Egyptian Arabic
(Eng-MSAEA). We focus on machine translation in two directions, either from
monolingual text to code-switched text or from code-switched text to
monolingual text. To this end, we provide raw data with no reference
translations (no parallel data). Participants are required to provide
translations for both directions for either or both language pairs.

####

Additionally, in both settings, participants are allowed to use any external
resources. We employ the BLEU score metric for evaluation as proposed by Post
(2018). Participants can submit their predictions for test data to the
Linguistic Code-switching Evaluation (LinCE) platform (Aguilar et al., 2020).
The leaderboard in the LinCE platform averages the BLEU scores across multiple
language pairs and directions to determine the position. Participants can
submit their results while comparing with others in real-time.

##  4 Datasets

| Supervised | Unsupervised
---|---|---
| Eng$\rightarrow$Hinglish | Eng$\rightarrow$Spanglish | Spanglish$\rightarrow$Eng | Eng$\rightarrow$MSAEA | MSAEA$\rightarrow$Eng
| Train | Dev | Test | Raw | Test | Raw | Test | Raw | Test | Raw | Test
Sentences | 8,060 | 960 | 942 | 15,000 | 5,000 | 15,000 | 6,500 | 15,000 | 5,000 | 12,000 | 6,500
Tokens | 93,025 | 11,465 | 11,849 | 166,649 | 212,793 | 228,484 | 189,465 | 166,649 | 104,429 | 219,796 | 303,998
Eng tokens | 33,886 | 4,273 | 4,333 | 116,617 | 97,646 | 43,869 | 100,000 | 116,617 | 38,967 | - | 181,890
Non-Eng tokens | 58,381 | 7,130 | 7,420 | - | 62,587 | 98,412 | 41,839 | - | 36,676 | 149,087 | 80,780
Other tokens | 758 | 62 | 96 | 50,032 | 52,560 | 86,203 | 47,626 | 50,032 | 28,831 | 70,709 | 41,328

Table 1: Distribution Statistics for the data sets for each language pair.
Only English-Hinglish had training and development data. English-Spanglish and
English-MSAEA both had test data with associated gold translations, as well as
raw data. For the raw sets, we just provide the raw data in one of directions
without gold translations. Other tokens refer to social media special tokens,
e.g, username mentions, emoji expressions and urls.

###  4.1 Eng-Hinglish

#### Data Curation

The CMU Document Grounded Conversations Dataset Zhou et al. (2018) contains a
set of conversations between users, with each conversation being grounded in a
Wikipedia article about a particular movie. A subset of this dataset is
translated into Hinglish (code-switched Hindi-English) and is available as the
CMU Hinglish Document Grounded Conversations Dataset Gupta et al. (2019). The
English sentences from the former dataset and their corresponding Hinglish
translations make up the parallel corpus used for our current task. The
dataset has multiple translations for some of the English sentences. These
sentences are pre-processed to remove newline and tab characters. These
parallel sentences could be used for machine translation in either of the 2
directions (English $\rightarrow$ Hinglish or Hinglish $\rightarrow$ English).
We chose the former as it involves the generation of code-switched text and
represents a more challenging task overall. It is worth noting that this setup
renders a single Hinglish translation per English sentence.

#### Data distribution:

The data statistics are shown in Table 1. We use the same train/dev/test
splits as originally created by Zhou et al. (2018).

###  4.2 Eng-Spanglish

#### Data Curation

For the Eng-Spanglish language pair, we leverage the English-Spanish language
identification corpus introduced in the first CALCS shared task Solorio et al.
(2014) and the second CALCS shared task Molina et al. (2016). We randomly
sample a subset from this dataset where we add translations for both
directions, namely the task for this language pair comprises two directions:
Eng $\rightarrow$ Spanglish and Spanglish $\rightarrow$ Eng. As the original
data was collected from Twitter, we normalized some social media special
tokens, e.g, replace username mentions with ‘<username>’. Since this data is
used in the unsupervised setting, we do not provide training data. We only
provide test data. For the latter data, we curate reference translations for
both directions from three translators.

#### Data distribution

The data statistics for both directions Eng $\rightarrow$ Spanglish and
Spanglish $\rightarrow$ Eng are shown in Table 1. We also provide raw data
from the code-switched Spanglish data from previous collections. Those
statistics are also shown in Table 1. The idea for providing the raw data is
to provide the participants with a sense of the data genres and domains and
the code-switching style of the data used in the task. As shown in Table 1, it
is worth noting that there are no Spanish tokens in the raw data for Eng
$\rightarrow$ Spanish. Likewise, the ratios of English tokens for Spanglish
$\rightarrow$ Eng in raw and test data are 19.2% and 19.5%. The skewed
distribution poses a great challenge considering that the model may not see
enough words in the target language. However, we think that the skewness can
be reasonably handled with the provided data. Moreover, the raw and test sets
draw a very similar data distribution, which can also help adapt the learning
from training to testing.

###  4.3 Eng-MSAEA

#### Data Curation

For the Eng-MSAEA language pair, we combine the datasets introduced in the
CALCS-2016 (Molina et al., 2016) and CALCS-2018 (Aguilar et al., 2018) as the
new corpora for the current machine translation shared task. The data was
collected from the Twitter platform and Blog commentaries. The tweets that
have been deleted or belong to the user whose accounts have been suspended
were removed and eliminated. We also perform normalization on each tweet to
reduce the impact of social media special tokens. Similar to the Eng-Spanglish
setting, we provide only test data and raw data.

#### Data distribution

We have two directions for Eng-MSAEA language pair: Eng $\rightarrow$ MSAEA
and MSAEA $\rightarrow$ Eng. The data statistics for each direction are listed
in Table 1. From Table 1, we can see that there are no tokens from the target
language in the raw data.

###  4.4 General Translation Guidelines

The following are some of the overall guidelines used for curating
translations: The translators are required to follow the standard grammar
rules and typos should be corrected in the translation. The translated text
should also be normalized. Slang, special forms, and expressions are to be
translated to literal equivalents or to a more formal/standard form. The typos
should be eliminated in translation, e.g, “Happo Birthday" should be
translated to “Feliz Cumpleaños". Additionally, speech effects should not be
reflected in translation, e.g, “Happy Birthdayyy" should be translated to
“Feliz Cumpleaños". Social media special tokens, including username mentions,
URLs, hashtags, and emojis, should not be translated even if they are
transferable/translatable. Moreover, measurement units should not be converted
or localized, e.g, “2LB" should not be translated to “1kg". When translating
from Spanglish into English, The translators were instructed to reword the
English fragments when the translation warrants it. For translation from code-
switched text to monolingual text, we requested one gold reference. For the
case where the target translation is in the code-switched language, we
requested three gold references. It is worth noting that we did not prescribe
how and when the translations should include a code-switch. We only emphasized
that we expect the resulting language to be a plausible code-switching of the
two languages. We also provided ample examples per language pair. We expect
there to be significant variation among the translations, as different
translators would choose to code-switch at different points. We hope that this
elicited data set could be eventually compared to naturally occurring code-
switched data to investigate where and how they vary.

###  4.5 Dataset Evaluation

Datasets | Code-switching Evaluation Metrics
---|---
CMI | M-Index | I-Index | LE | SE | Burstiness
Eng$\rightarrow$Hinglish | 36.81 | 0.90 | 0.53 | 0.95 | 2.13 | -0.07
Eng$\rightarrow$Spanglish | 49.99 | 2.13 | 0.40 | 1.06 | 1.60 | -0.19
Spanglish$\rightarrow$Eng | 48.98 | 2.19 | 0.39 | 1.06 | 1.57 | -0.15
Eng$\rightarrow$MSAEA | 9.54 | 0.71 | 0.62 | 0.59 | 2.31 | -0.10
MSAEA$\rightarrow$Eng | 10.07 | 0.52 | 0.76 | 0.58 | 3.72 | -0.03

Table 2: Code-switching statistics on each dataset.

To measure the quality of the datasets we created, and understand the inherent
characteristics of code-switched corpora, we evaluate our datasets following
the statistics proposed by Guzmán et al. (2017) and Gambäck and Das (2014),
including:

  * •

Code-Mixing Index (CMI): The fraction of total words that belong to languages
other than the most dominant language in the text;

  * •

Multilingual Index (M-Index): A word-count-based measure that quantifies the
inequality of the distribution of language tags in a corpus of at least two
languages;

  * •

Integration Index (I-Index): A proportion of how many switch points exist
relative to the number of language-dependent tokens in the corpus;

  * •

Language Entropy (LE): the number of bits of information are needed to
describe the distribution of language tags;

  * •

Span Entropy (SE): the number of bits of information are needed to describe
the distribution of language spans;

  * •

Burstiness: A measure of quantifying whether switching occurs in bursts or has
a more periodic characteristic.

The code-switching evaluation statistics of our datasets are shown in Table 2.

##  5 Methods

We received submissions from five different teams. Four teams submitted system
responses for Eng-Hinglish, making this language pair the most popular in this
shared task. On the other hand, we had no external submissions for Spanglish-
Eng.

Below we provide a brief description of the two baselines as well as
participant systems:

  * •

baseline1 We use mBART (Liu et al., 2020) as the baseline model trained for 5
epochs with a batch size of 8 and a learning rate of 5e-5. For the supervised
setting, we simply fine-tune it on the parallel data. For the unsupervised
setting, we directly generate translations without fine-tuning.

  * •

Echo (baseline) This baseline simply passes inputs as outputs and the goal is
to measure how much the overlap in input/output can contribute to the final
performance. It is inspired by preliminary observations that there is a high
token overlap between the source and target sentences. Also, many tokens
common in social media (e.g, username mentions, URLs, and emoticons) have no
clear translations and annotators left them unchanged.

  * •

UBC_HImt (Jawahar et al., 2021). They propose a dependency-free method for
generating code-switched data from bilingual distributed representations and
adopt a curriculum learning approach where they first fine-tune a language
model on synthetic data then on gold code-switched data.

  * •

IITP-MT (Appicharla et al., 2021). They propose an approach to create a code-
switching parallel corpus from a clean parallel corpus in an unsupervised
manner. Then they train a neural machine translation model on the gold corpus
along with the generated synthetic code-switching parallel corpus.

  * •

UBC_ARmt (Nagoudi et al., 2021). They collect external parallel data from
online resources and fine-tune a sequence-to-sequence transformer-based model
on external data then on gold code-switched data.

  * •

CMMTOne (Dowlagar and Mamidi, 2021). They present a gated convolutional
sequence to sequence encoder and decoder models for machine translation on
code-switched data. The sliding window inside the convolutional model renders
it able to handle contextual words and extract rich representations.

  * •

LTRC-PreCog (Gautam et al., 2021). They propose to use mBART, a pre-trained
multilingual sequence-to-sequence model and fully utilize the pre-training of
the model by transliterating the roman Hindi words in the code-mixed sentences
to Devanagri script.

In Table 3, we listed the main components and strategies used by the
participating systems. Most systems are based on transformer architectures,
including multilingual transformers such as mT5 and mBART. It is worth noting
that all teams applied deep neural networks techniques. Among the five
participating teams, three of them (IITP-MT, UBC_ARmt, CMMTOne) trained the
model from scratch while two of them (UBC_HImt, LTRC-PreCog) fine-tuned the
models with pre-trained knowledge on monolingual data. To tackle the challenge
of low-resource data, two teams (IITP-MT, LTRC-PreCog) transliterated Hindi to
Devanagari during training to fully utilize monolingual resources available in
the native Devanagari script, and then back-transliterated Devanagari to Hindi
to improve performance. Additionally, four teams (UBC_HImt, IITP-MT, UBC_ARmt,
LTRC-PreCog) leveraged external resources (e.g, parallel datasets and text
processing libraries). Moreover, two teams (UBC_HImt, IITP-MT) applied data
augmentation techniques to generate synthetic data to increase the size of the
training data.

System | Base Model | Training | Transliteration | Ext Resources | Data Augmentation
---|---|---|---|---|---
UBC_HImt | mT5 | two-stage fine-tuning |  | ✓ | ✓
IITP-MT | transformer | from scratch | ✓ | ✓ | ✓
UBC_ARmt | transformer | from scratch |  | ✓ |
CMMTOne | seq2seq | from scratch |  |  |
LTRC-PreCog | mBART | fine-tuning | ✓ | ✓ |

Table 3: The comparison of the main components and strategies used by the
participating systems. Ext Resources stands for external resources such as
parallel datasets and text processing libraries.

##  6 Evaluation and Results

###  6.1 Evaluation Metric

The evaluation of the shared task was conducted through a dedicated platform,
where participants can obtain immediate feedback of their submissions after
uploading translations for the test data. The platform then scores the
submissions and publishes the results in a public leaderboard for each
language pair direction.

We use BLEU Papineni et al. (2002) to rank participating systems. BLEU is a
score for comparing a candidate translation of the text to one or more
reference translations.

###  6.2 Results

System | Eng $\rightarrow$ Hinglish | Spanglish $\rightarrow$ Eng | Eng $\rightarrow$ Spanglish | MSAEA $\rightarrow$ Eng | Eng $\rightarrow$ MSAEA
---|---|---|---|---|---
Baselines
baseline1 | 11.00 | 33.97 | 55.05 | 1.55 | 36.11
Echo | 6.84 | 43.86 | 65.46 | 1.80 | 40.83
Participating Systems
UBC_HImt | 12.67 |  |  |  |
IITP-MT | 10.09 |  |  |  |
UBC_ARmt |  |  |  | 25.72 |
CMMTOne | 2.58 |  |  |  |
LTRC-PreCog | 12.22 |  |  |  |

Table 4: The results of the participating systems in all language pairs. The
scores are based on the average BLEU metric. The highlighted systems are the
best scores for each language pair.

As listed in Table 4 5 teams participated in this shared task. 4 teams
submitted systems for Eng$\rightarrow$Hinglish language directions, and one
team submitted systems for the MSAEA$\rightarrow$Eng language direction.

#### Baselines

For the supervised task (i.e, Eng$\rightarrow$Hinglish), baseline1 achieves a
BLEU score of 11.00%. It outperformed the Echo baseline by 4.16%, indicating
that the multilingual model has acquired some useful knowledge about code-
switching during pretraining. This is aligned with findings in literature
about the usefulness of pretraining transformer models with in domain data
(Doddapaneni et al., 2021). For the unsupervised tasks, we simply use the
model to make predictions based on the pre-trained knowledge it has learned on
monolingual data. Surprisingly, on all four unsupervised tasks, the results
from baseline1 are lower than the ones from Echo, suggesting that the pre-
trained knowledge from the monolingual data cannot be effectively adapted to
the code-switched data.

#### Participating systems

Table 4 shows the BLEU score results for all the teams/systems. For
Eng$\rightarrow$Hinglish, two teams (UBC_HImt, LTRC-PreCog) outperform the
baseline. The best performance achieved was from UBC_HImt with a BLEU score of
12.67%, outperforming the baseline by 1.67%. For the task of
MSAEA$\rightarrow$Eng, the only one submission received was from UBC_ARmt with
a BLEU Score of 25.72%, outperforming the baseline by 23.92%.

##  7 Analysis

# | Sentence | Gold Standard/Reference Translation | System Prediction
---|---|---|---
1 |  |  Yes you definitely should
---
|  haan tumhe definitely dekhna chahiye
---
|  haan, [jarur aapako]
---
2 |  |  It’s about a mute cleaner and she works at a secret government lab
---
|  ye ek mute cleaner ke baare mein hai jo secret government lab mein work
karti hai
---
|  yah ek mute cleaner ke baare mein hai aur [wo] secret government lab par
kaam karta hai
---
3 |  |  I wasn’t sure about those two to start out with, they just didn’t seem like they’d be good together
---
|  mujhe un donon ke baare mein nishchit nahin tha ki ve shuroo kar sakate
hain, unhen aisa nahin lagata tha ki ve ek saath achchhe honge
---
|  mujhe un dono ke baare mein [start out mein sure nahi tha], unhe aisa
[lagta hai wo acche sath mein rehne nahin lagte hain]
---
4 |  |  I mean not rated well
---
|  mujhe lagta hai ki ise thik se rate nahi kiya gaya
---
|  mera mathalab [yah nahem tha ki]
---
5 |  |  The shape of water is a great movie
---
|  The shape of water ek bahut achchi movie hai
---
|  [shailee] kee ek mahaan philm hai
---

Table 5:  Poor translation Examples for the task of Eng$\rightarrow$Hinglish.
Errors in translations from participating systems are highlighted in bold in
brackets (e.g, [shailee]). Reference translations are provided for comparison.

Although most of the scores reported by the participants outperformed the
baselines for both Eng$\rightarrow$Hinglish and MSAEA$\rightarrow$Eng, we
notice that the BLEU metric takes overlapped tokens into consideration, which
may potentially result in a higher score. Additionally, the overall results
are arguably lower compared to results achieved by the state-of-the-art
systems on monolingual data for these language pairs, i.e. in the absence of
code-switching. We conduct error analysis to understand and expose the
bottleneck of machine translation on code-switched data.

###  7.1 Evaluation Metrics

In Table 4, it is worth noting that the Echo system, with inputs as outputs,
could outperform the transformer-based baseline1 system in four out of five
language translation directions. We suspect the high scores from the Echo
system are because of the high overlap between the training and testing data.
Therefore, to minimize such impact, we normalize the text by removing the
overlapping tokens from the data, including username mentions, emoji
expressions, and URLs. The results of BLEU and normalized BLEU scores on each
system for the task of Eng$\rightarrow$Hinglish are shown in Table 6 where we
note significant drops in performance ranging from 0.43% to 6.13 BLEU score.
After normalizing the text, the Echo system can only reach 0.71 BLEU points,
which is far lower than the performance of baseline1. Although the
performances of participating systems also decrease drastically, some can
still outperform the baseline1.

System | BLEU | BLEU_norm | Drop
---|---|---|---
Baselines
Baseline1 | 11.00 | 6.05 | 4.95
Echo | 6.84 | 0.71 | 6.13
Participating Systems
UBC_HImt | 12.67 | 7.58 | 5.09
IITP-MT | 10.09 | 6.33 | 3.76
CMMTOne | 2.58 | 2.15 | 0.43
LTRC-PreCog | 12.22 | 6.70 | 5.52

Table 6: The BLEU and the normalized BLEU scores on the task of
Eng$\rightarrow$Hinglish. The largest number for each column is in bold.

###  7.2 Error Analysis

Although most of the scores reported by the participants outperform the
baseline, there are many mistakes made by the models. Thus, we manually
inspect the predictions from participating systems. Table 5 shows some cases
where the participating systems produce low quality translations for the task
of Eng$\rightarrow$Hinglish.

Based on our findings, there are two main errors among the predictions:
grammar errors and semantic errors. For the grammar errors, we find that many
predicted sentences have incorrect word/phrase orders. Although linguistic
theories of code-switching argue that code-switching does not violate
syntactic constraints of the languages involved (Poplack, 1980, 2013), it may
still impact models ability to correctly capture grammatical structures from
the involved languages. Moreover, we observe that some translations may have
extra punctuation or incorrect pronouns. Gender translation errors are also
common in MT systems for monolingual data (Stanovsky et al., 2019; Saunders
and Byrne, 2020). For the semantic errors, it is worth noting that many
sentences can only be translated partially. Understanding the semantics of
code-switching is difficult as the model has to learn the word representations
and dependencies for all involved languages. This may result in incomplete or
irrelevant translations.

##  8 Conclusion

This paper summarizes the insights gained from creating a task on MT for code-
switched language. Given a monolingual/code-switched sentence, the goal is to
translate it into a code-switched/monolingual sentence. We introduced a
machine translation dataset focused on code-switched social media text for
three language pairs: English-Hinglish, English-Spanglish, and English-Modern
Standard Arabic-Egyptian Arabic with one or two directions for each pair. We
received 12 submissions from 5 teams, 4 of them submitted to
English$\rightarrow$Hinglish and 1 of them submitted to Modern Standard
Arabic-Egyptian Arabic$\rightarrow$English. While the participating systems
are different, we observed that there is a strong trend favoring the use of
transformer-based models (e.g, mBART) as a key ingredient in the proposed
architectures. The best performances achieved were 12.67% BLEU score for
English$\rightarrow$Hinglish and 25.72% BLEU score for Modern Standard Arabic-
Egyptian Arabic$\rightarrow$English. Compared to monolingual formal text, the
reported BLEU scores are significantly lower, which highlights the difficulty
of both, the linguistic properties in code-switched data, and the noise
attributed to the social media genre. More investigations need to be carried
out to disentangle the effects of each factor. However, this shared task
demonstrates that performing machine translation on code-switched data is a
new horizon of interest in the NLP community and more work needs to be done to
bring performance on par to monolingual systems. It is also likely that
advances in MT for code-switched data will lead to groundbreaking
contributions to the general field of MT.

## Acknowledgements

As organizers of the CALCS 2021 shared task, we gratefully acknowledge all the
teams for their interest and participation. We also would like to thank the
Alan W. Black, Sunayana Sitaram, Victor Soto and Emre Yilmaz for their
invaluable collaboration in organizing this shared task.

## References

  * Aguilar et al. (2018) Gustavo Aguilar, Fahad AlGhamdi, Victor Soto, Mona Diab, Julia Hirschberg, and Thamar Solorio. 2018.  Named entity recognition on code-switched data: Overview of the CALCS 2018 shared task.  In _Proceedings of the Third Workshop on Computational Approaches to Linguistic Code-Switching_ , pages 138–147, Melbourne, Australia. Association for Computational Linguistics.
  * Aguilar et al. (2020) Gustavo Aguilar, Sudipta Kar, and Thamar Solorio. 2020.  LinCE: A centralized benchmark for linguistic code-switching evaluation.  In _Proceedings of the 12th Language Resources and Evaluation Conference_ , pages 1803–1813, Marseille, France. European Language Resources Association.
  * Aguilar and Solorio (2020) Gustavo Aguilar and Thamar Solorio. 2020.  From English to code-switching: Transfer learning with strong morphological clues.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8033–8044, Online. Association for Computational Linguistics.
  * Appicharla et al. (2021) Ramakrishna Appicharla, Kamal Kumar Gupta, Asif Ekbal, and Pushpak Bhattacharyya. 2021.  IITP-MT at CALCS2021: English to Hinglish neural machine translation using unsupervised synthetic code-mixed parallel corpus.  In _Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching_ , pages 31–35, Online. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Doddapaneni et al. (2021) Sumanth Doddapaneni, Gowtham Ramesh, Anoop Kunchukuttan, Pratyush Kumar, and Mitesh M Khapra. 2021.  A primer on pretrained multilingual language models.  _arXiv preprint arXiv:2107.00676_.
  * Dowlagar and Mamidi (2021) Suman Dowlagar and Radhika Mamidi. 2021.  Gated convolutional sequence to sequence based learning for English-hingilsh code-switched machine translation. In _Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching_ , pages 26–30, Online. Association for Computational Linguistics.
  * Gambäck and Das (2014) Björn Gambäck and Amitava Das. 2014.  On measuring the complexity of code-mixing.  In _Proceedings of the 11th International Conference on Natural Language Processing, Goa, India_ , pages 1–7.
  * Garg et al. (2018) Saurabh Garg, Tanmay Parekh, and Preethi Jyothi. 2018.  Code-switched language models using dual RNNs and same-source pretraining.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3078–3083, Brussels, Belgium. Association for Computational Linguistics.
  * Gautam et al. (2021) Devansh Gautam, Prashant Kodali, Kshitij Gupta, Anmol Goel, Manish Shrivastava, and Ponnurangam Kumaraguru. 2021.  CoMeT: Towards code-mixed translation using parallel monolingual sentences.  In _Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching_ , pages 47–55, Online. Association for Computational Linguistics.
  * Gonen and Goldberg (2019) Hila Gonen and Yoav Goldberg. 2019.  Language modeling for code-switching: Evaluation, integration of monolingual data, and discriminative training.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4175–4185, Hong Kong, China. Association for Computational Linguistics.
  * Goyal et al. (2003) P Goyal, Manav R Mital, A Mukerjee, Achla M Raina, D Sharma, P Shukla, and K Vikram. 2003.  A bilingual parser for hindi, english and code-switching structures.  In _10th Conference of The European Chapter_ , page 15.
  * Gupta et al. (2019) Shaleen Kumar Gupta, Sahitya Potluri, Aishwarya Reganti, Harsh Lara, and Alan W. Black. 2019.  Cmu hinglish grounded conversations dataset.
  * Guzmán et al. (2017) Gualberto A Guzmán, Joseph Ricard, Jacqueline Serigos, Barbara E Bullock, and Almeida Jacqueline Toribio. 2017.  Metrics for modeling code-switching across corpora.  In _INTERSPEECH_ , pages 67–71.
  * Jawahar et al. (2021) Ganesh Jawahar, El Moatez Billah Nagoudi, Muhammad Abdul-Mageed, and Laks Lakshmanan, V.S. 2021.  Exploring text-to-text transformers for English to Hinglish machine translation with synthetic code-mixing.  In _Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching_ , pages 36–46, Online. Association for Computational Linguistics.
  * Joshi (1982) Aravind K. Joshi. 1982.  Processing of sentences with intra-sentential code-switching.  In _Coling 1982: Proceedings of the Ninth International Conference on Computational Linguistics_.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Liu et al. (2020) Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020.  Multilingual denoising pre-training for neural machine translation.  _Transactions of the Association for Computational Linguistics_ , 8:726–742.
  * Molina et al. (2016) Giovanni Molina, Fahad AlGhamdi, Mahmoud Ghoneim, Abdelati Hawwari, Nicolas Rey-Villamizar, Mona Diab, and Thamar Solorio. 2016.  Overview for the second shared task on language identification in code-switched data.  In _Proceedings of the Second Workshop on Computational Approaches to Code Switching_ , pages 40–49, Austin, Texas. Association for Computational Linguistics.
  * Nagoudi et al. (2021) El Moatez Billah Nagoudi, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2021.  Investigating code-mixed Modern Standard Arabic-Egyptian to English machine translation.  In _Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching_ , pages 56–64, Online. Association for Computational Linguistics.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_ , pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  * Poplack (1980) Shana Poplack. 1980.  Sometimes i’ll start a sentence in spanish y termino en espanol: toward a typology of code-switching1.
  * Poplack (2013) Shana Poplack. 2013.  “sometimes i’ll start a sentence in spanish y termino en español”: Toward a typology of code-switching.  _Linguistics_ , 51(s1):11–14.
  * Post (2018) Matt Post. 2018.  A call for clarity in reporting BLEU scores.  In _Proceedings of the Third Conference on Machine Translation: Research Papers_ , pages 186–191, Brussels, Belgium. Association for Computational Linguistics.
  * Saunders and Byrne (2020) Danielle Saunders and Bill Byrne. 2020.  Reducing gender bias in neural machine translation as a domain adaptation problem.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7724–7736, Online. Association for Computational Linguistics.
  * Singh and Solorio (2017) Thoudam Doren Singh and Thamar Solorio. 2017.  Towards translating mixed-code comments from social media.  In _International Conference on Computational Linguistics and Intelligent Text Processing_ , pages 457–468. Springer.
  * Solorio et al. (2014) Thamar Solorio, Elizabeth Blair, Suraj Maharjan, Steven Bethard, Mona Diab, Mahmoud Ghoneim, Abdelati Hawwari, Fahad AlGhamdi, Julia Hirschberg, Alison Chang, and Pascale Fung. 2014.  Overview for the first shared task on language identification in code-switched data.  In _Proceedings of the First Workshop on Computational Approaches to Code Switching_ , pages 62–72, Doha, Qatar. Association for Computational Linguistics.
  * Solorio and Liu (2008) Thamar Solorio and Yang Liu. 2008.  Learning to predict code-switching points.  In _Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing_ , pages 973–981, Honolulu, Hawaii. Association for Computational Linguistics.
  * Srivastava and Singh (2020) Vivek Srivastava and Mayank Singh. 2020.  PHINC: A parallel Hinglish social media code-mixed corpus for machine translation.  In _Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020)_ , pages 41–49, Online. Association for Computational Linguistics.
  * Stanovsky et al. (2019) Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019.  Evaluating gender bias in machine translation.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1679–1684, Florence, Italy. Association for Computational Linguistics.
  * Tarunesh et al. (2021) Ishan Tarunesh, Syamantak Kumar, and Preethi Jyothi. 2021.  From machine translation to code-switching: Generating high-quality code-switched text.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 3154–3169, Online. Association for Computational Linguistics.
  * Toribio (2001) Almeida Jacqueline Toribio. 2001.  Accessing bilingual code-switching competence.  _International Journal of Bilingualism_ , 5:403–436.
  * Winata et al. (2019) Genta Indra Winata, Zhaojiang Lin, and Pascale Fung. 2019.  Learning multilingual meta-embeddings for code-switching named entity recognition.  In _Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)_ , pages 181–186, Florence, Italy. Association for Computational Linguistics.
  * Xu and Yvon (2021) Jitao Xu and François Yvon. 2021.  Can you traducir this? machine translation for code-switched input.  In _Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching_ , pages 84–94, Online. Association for Computational Linguistics.
  * Xue et al. (2021) Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021.  mT5: A massively multilingual pre-trained text-to-text transformer.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 483–498, Online. Association for Computational Linguistics.
  * Yang et al. (2020) Zhen Yang, Bojie Hu, Ambyera Han, Shen Huang, and Qi Ju. 2020.  CSP:code-switching pre-training for neural machine translation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2624–2636, Online. Association for Computational Linguistics.
  * Zhou et al. (2018) Kangyan Zhou, Shrimai Prabhumoye, and Alan W Black. 2018.  A dataset for document grounded conversations.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 708–713, Brussels, Belgium. Association for Computational Linguistics.


# Are Leptons as elementary as Quarks?

Aharon Davidson<EMAIL_ADDRESS>https://physics.bgu.ac.il/~davidson
Physics Department, Ben-Gurion University of the Negev, Beer-Sheva 84105,
Israel

(20 Dec, 2022)

###### Abstract

Nucleons and electrons were once considered to be elementary particles, a role
now taken by quarks and leptons. Here, by trading $SU(3)_{C}$ for
$SU(3)_{C}\times SU(3)_{\ell C}$ confining gauge symmetry, we suggest that
nucleons and electrons share in fact the same level of compositeness. Standard
model leptons, equipped with inherited Yukawa couplings, make their appearance
at the intermediate trinification stage as (gauge protected) three pre-lepton
composites. Adding an extra $SU(3)$ flavor link to the chain, as exclusively
prescribed on multifold family grounds, preserves the anti-symmetric structure
of the single family wave function, and furthermore predicts a novel anomaly
free lepto/dark portal.

Introduction

A long time ago, in what seems to be another era, nucleons (protons and
neutrons) and electrons were considered indivisible elementary particles.
Primarily owing to their discretely quantized electric charges (Millikan
integer sequence), they govern atomic physics, and consequently the structure
of matter at the Bohr scale and beyond. Quarks have entered the game through
the group theoretical ”eightfold way” path, and their strong local $SU(3)_{C}$
color interactions have been proposed and experimentally confirmed a bit
later. The integer charged baryons and mesons have eventually become
composites of fractionally charged confined quarks.

The leptons, on the other hand, have fully preserved their fundamental point-
like perception during the quark revolution. Moreover, together with quarks
they share the standard $SU(3)_{C}\times SU(2)_{L}\times U(1)_{Y}$
electro/nuclear model, and in particular, contribute their part to cancel the
otherwise harmful ABJ anomalies. The latter feature takes us beyond the
standard model, crossing the gauge desert all the way to grand unification.
While staying essentially the same for $SU(5)$, anomaly cancelation becomes
automatic for $SO(10)$ and $E(6)$.

By consistently embedding $SU(3)_{C}$ within a parent $SU(4)_{PS}$, the unity
of quarks and leptons has been formally demonstrated in Pati-Salam’s ”Lepton
number as the fourth color” PS . Numerically speaking, $SU(4)_{PS}$ unitarity
has been simply translated into the fundamental sum rule
$\frac{1}{3}+\frac{1}{3}+\frac{1}{3}-1=0$. While such an imaginative idea has
forcefully opened the door for the left-right symmetric grand unification
variants $SO(10)\subset E(6)$, once the spontaneous symmetry breaking
$SU(4)_{PS}\rightarrow SU(3)_{C}\times U(1)_{B-L}$ takes place, quarks and
leptons are not treated on equal symmetric footing any more. After all, one
can always build an integer out of thirds, but cannot vice versa build a third
out of integers.

The idea of accompanying $SU(3)_{C}$ by extra $SU(3)$ factors (with or without
an underlying discrete symmetry) is not new, with $E(6)$-embeddable E6
trinification 333

$G_{tri}=SU(3)_{C}\times SU(3)_{L}\times SU(3)_{R}$ (1)

being the prototype example. Other examples include LR-symmetric chiral color
$SU(3)_{CL}\times SU(3)_{CR}$ chiralC , confining hypercolor (technicolor)
$SU(3)_{HC}$ of various sorts (notably the Rishon model Rishon  and some of
its derivatives HCcomp ), spontaneously broken flavor $SU(3)_{F}$ to account
for the fermion threefold family replication 3F , and even leptocolor
$SU(3)_{\ell C}$ aimed to enhance quark/lepton symmetry qlsym . On symmetry
grounds, with ”lepton number as the fourth color” in mind, there is no group
theoretical reason why not considering ”baryon number as the fourth
leptocolor” as well. This would bring nucleons and electrons, which anyhow
share integer electric charges, back to the same level of elementariness
(actually compositeness now), as used to be the case prior to the introduction
of quarks.

Such an idea can be minimally realized within the framework of local
$SU(3)_{C}\times SU(3)_{\ell C}$ symmetry. In fact, the latter symmetry has
already been introduced qlsym  in the literature. However, the orthodox role
of $SU(3)_{\ell C}$ as a spontaneously broken symmetry at heavy mass scale is
herby replaced by a totally different scenario. Namely, an unbroken symmetry,
characterized by a leptocolor scale substantially larger than the ordinary
color scale $\Lambda_{\ell C}\gg\Lambda_{C}$ (with a gauge coupling constant
$g_{\ell C}\gtrsim g_{C}$ at $\Lambda_{\ell C}$). Leptocolor is expected to
enter its short range confining mode much earlier than $SU(3)_{C}$, at the
stage where trinification 333  prevails. As long as it stays unbroken, the
latter local symmetry is capable of gauge protecting a flavor-chiral
combination of spectator quarks and composite leptons.

Composite model constructions are heavily restricted by ’t Hooft’s anomaly
matching equations tHooft . General solutions are hard to find, and the more
so solutions with integer coefficients (even in cases where the restrictive
Apellquist-Carazzone decoupling condition AC  is not applicable). In fact, ’t
Hooft himself has pointed out that $QCD$ with more than two flavors is not
natural, and made the interesting remark that to construct tenable models with
complete naturalness for elementary particles one may need more types of
confining gauge theories besides $QCD$ (an example is provided in Ter ). At
any rate, contrary to ’t Hooft’s $3$-quark model, where the associated
$SU(3)_{L}\times SU(3)_{R}\times U(1)_{V}$ flavor symmetry is notably global,
the latter symmetry is almost fully gauged in our case, except for the vector
$U(1)_{V}$ remnant. Unfortunately, our model is not natural in ’t Hooft sense,
with the problematic matching equations being of course those involving
$U(1)_{V}$. However, within the context of a flavor-chiral model (unlike QCD),
it makes it a bit easier for us to assume that fermion masslessness is
protected by anomaly free gauge invariance. While chiral symmetry suffices for
this purpose, anomaly cancellation is stronger, and furthermore serves as a
field theoretical consistency tool.

The threefold family structure cannot and does not stay out of the game. It
integrates in naturally and inductively by means of adding another local
$SU(3)_{F}$ link to the $SU(3)$ chain 3chain . And quite remarkably, it does
it without upsetting the overall Pauli anti-symmetric structure of the lepton
wave function, a phenomenon which we have already encountered at the single
family level.

Composite leptons, spectator quarks

Let our $q\leftrightarrow\bar{\lambda}$ symmetric, anomaly free, flavor-chiral
representation of left handed fermions transform as

$\psi_{L}=\begin{array}[]{|c ||c|c||c|c|}\hline\cr{}{}\hfil&C&\ell C&L&R\\\
\hline\cr\hline\cr~{}q_{L}{}&3&1&~{}3^{\star}&1\\\
\hline\cr~{}\bar{q}_{L}{}&~{}3^{\star}&1&1&3\\\
\hline\cr~{}\lambda_{L}{}&1&~{}3^{\star}&3&1\\\
\hline\cr~{}\bar{\lambda}_{L}{}&1&3&1&~{}3^{\star}\\\ \hline\cr\end{array}$
(2)

under the local quartification gauge group

$G_{quad}=[SU(3)_{C}\times SU(3)_{\ell C}]\times[SU(3)_{L}\times
SU(3)_{R}]~{}.$ (3)

Note that $q\leftrightarrow\bar{\lambda}$ under $\\{C\leftrightarrow\ell
C,L\leftrightarrow R\\}$, so that one cannot really tell quarks from pre-
leptons at this stage. The representation is $C\ell C$ and $LR$ real,
separately, yet overall complex, forming a moose chain. Conspicuous by its
absence is the $(1,1;3,3^{\star}$) leptonic piece which has been bifurcated
into the $\lambda+\bar{\lambda}$ pair. Starting with $G_{quad}$, we end up
with unbroken $SU(3)_{C}\times SU(3)_{\ell C}\times U(1)_{Q}$ once the dust
settles down. The electric charge reads

$Q=T_{3L}+T_{3R}+\frac{1}{2}\left(Y_{L}+Y_{R}\right)~{},$ (4)

so that $Q[q]+Q[\lambda]$=0. At any rate, we keep the option of inductively
enlarging $G_{quad}$, if needed, when proceeding beyond the scope of the
single family level.

In analogy with the color singlet baryons which are made of three quarks, we
now attempt to construct leptocolor singlet composites using

$\lambda_{L}(1,3^{\star};3,1)~{},\quad\lambda_{R}(1,3^{\star};1,3)$ (5)

as our building blocks. Here, invoking the traditional basis, $f_{R}$ denotes
the right handed Weyl partner of the left handed $\bar{f}_{L}$ (they transform
into each other under charge conjugation). In turn, $\lambda_{L}$ and
$\lambda_{R}$ transform alike, that is $\sim 3^{\star}$ under $S(3)_{\ell C}$.
We are after a set of left handed composites which would compensate for the
$SU(3)_{L,R}$ triangle anomalies contributed by the quark spectators. The fact
that $\lambda_{R}\lambda_{R}\lambda_{R}$ and
$\lambda_{L}\lambda_{L}\lambda_{R}$ are right-handed leaves us with the two
left-handed candidates:

(i) $\lambda_{L}\lambda_{L}\lambda_{L}$

From $SU(2)_{L}\times SU(2)_{R}^{\dagger}$ Lorentzian point of view, we can
have $(2+2+4,1)$, spinors nicely included (in fact, there are two of them).
But this turns out irrelevant given that under local $SU(3)_{L}\times
SU(3)_{R}$, the resulting configuration $(1+8+8+10,1)$ does not contain
triplets nor anti-triplets. Put it in other words, no standard model leptonic
candidates are available. Note in passing that, unlike complex $(10,1)$, the
real representations $(1+8+8,1)$ cannot anyhow be protected by gauge
invariance. The $(1,1)$ piece will play a role in a later stage.

(ii) $\lambda_{L}\lambda_{R}\lambda_{R}$

The Lorentz representation $(2,1+3)$ also contains the mandatory $(2,1)$
piece, but unlike in the previous case, one faces now $(3,3^{\star}+6)$ under
$SU(3)_{L}\times SU(3)_{R}$, including the mandatory $(3,3^{\star})$
representation. With this in hands, we can now complete our composite level
lepton identification, namely

$\ell_{L}+\bar{\ell}_{L}=(1,1;3,3^{\star})~{}.$ (6)

Together with the quarks and anti quarks, hereby treated as spectators, the
leptocolor singlets close upon

$\psi_{L}^{tri}=\begin{array}[]{|c ||c|c|c|}\hline\cr{}\hfil&C&L&R\\\
\hline\cr\hline\cr~{}q_{L}{}&3&~{}3^{\star}&1\\\
\hline\cr~{}\bar{q}_{L}{}&~{}3^{\star}&1&3\\\
\hline\cr~{}\ell_{L}+\bar{\ell}_{L}{}&1&3&~{}3^{\star}\\\
\hline\cr\end{array}$ (7)

forming a complex single family anomaly free, that is $3\times(-1)+9\times
0+3\times 1=0$, trinification representation. It is important to note that the
$SU(3)_{R}$ triangle anomaly $A(6)=-7A(3^{\star})$ makes the $(1,1;3,6)$
representation irrelevant for our anomaly cancelation purposes. Had we
incorporated $(1,1;3,6)$, then owing to $A_{3L}+A_{3R}=3(2+7)\neq 0$, there
would be no way to simultaneously cancel both $SU(3)_{L,R}^{3}$ anomalies
$A_{3L,3R}$, respectively. As long as $G_{tri}$ stays unbroken, this complex
set is protected and remains massless. Other composite states, such as (say)
$\lambda_{L}\lambda_{L}\lambda_{L}$ which belongs to $SU(3)_{L}/Z_{3}$, are
not protected from picking up the large $\Lambda_{\ell C}$ mass scale.

The Yukawa interactions, introduced at the fundamental Lagrangian level,
forcefully reappear at the intermediate trinification stage. As depicted in
Fig.(1), spectator quarks and composite leptons share the one and the same
Yukawa coupling diagrams. As far as the leptons are concerned, only one pre-
lepton is involved in the coupling, while its two companions stay inert. A
similar situation occurs in the leading order of beta decay.

Figure 1: Yukawa at trinification: Spectator quarks and composite leptons
share similar type Yukawa diagrams, involving the one and the same Higgs
scalar.Only one pre-lepton is actively involved, while two others stay inertly
out of the game.

Threefold family structure

Once the single quark/lepton family model has been established, the obvious
challenge is to incorporate the underlying threefold family structure, and do
it naturally without any superfluous replication. With the underlying motto of
”One three to rule of flavors”, we are driven to the quite traditional step of
introducing an extra local $SU(3)_{F}$ group factor ($F$ stands for Family).
One has to be careful though, especially when dealing with composite models,
to ensure the overall anti-symmetry of the lepton wave function under the
interchange of the two identical $\lambda_{R}$s. This is however not automatic
and should be regarded as a consistency test for the model under construction.

Without loosing generality, let $\lambda_{L}\sim 3^{\star}$ under $SU(3)_{F}$,
so that

$\lambda_{L}\lambda_{L}\lambda_{L}\sim 3^{\star}\times 3^{\star}\times
3^{\star}=1+8+8+10^{\star}~{}.$ (8)

There are still two possibilities regarding the assignments of
$\bar{\lambda}_{L}$ (or equivalently of $\lambda_{R}$). To choose between
these two, one needs to first check the associated transformation laws:

(i) $\bar{\lambda}_{L}\sim 3$ ($\lambda_{R}\sim 3^{\star}$) implies

$\lambda_{L}\lambda_{R}\lambda_{R}\sim 3^{\star}\times 3^{\star}\times
3^{\star}=1+8+8+10^{\star}~{}.$ (9)

(ii) $\bar{\lambda}_{L}\sim 3^{\star}$ ($\lambda_{R}\sim 3$) implies

$\lambda_{L}\lambda_{R}\lambda_{R}\sim 3^{\star}\times 3\times
3=3+3+6^{\star}+15~{}.$ (10)

And since the left handed composites must include a $3$ (or a $3^{\star}$) to
consistently enter the family picture, it is only the second option which can
deliver. The main bonus is fixing the inner structure of the tenable composite
fermion, namely choosing $\lambda_{L}\lambda_{R}\lambda_{R}$ rather than
$\lambda_{L}\lambda_{L}\lambda_{L}$. It is by no means trivial that the very
same conclusion was drawn previously, where only
$\lambda_{L}\lambda_{R}\lambda_{R}$ was shown to transform as required, that
is via $(3,3^{\star})$, under the electro/weak $SU(3)_{L}\times SU(3)_{R}$.
Still, see eq.(10), there are two $\lambda_{L}\lambda_{R}\lambda_{R}$ triplet
candidates. Using $SU(3)_{F}$ spinor indices, they are

$\displaystyle\text{anti-
symmetric:}\quad\epsilon^{iqp}\epsilon_{pjk}\lambda_{Lq}\lambda_{R}^{j}\lambda_{R}^{k}~{},~{}$
(11a)
$\displaystyle\text{symmetric:}\quad\lambda_{Lp}(\lambda_{R}^{i}\lambda_{R}^{p}+\lambda_{R}^{p}\lambda_{R}^{i})~{}.$
(11b)

The choice between these two candidates will be eventually made on symmetry
(or anti-symmetry) grounds when the full picture will be in front of us.

A word of caution is now in order. Note that
$\lambda_{L}\lambda_{R}\lambda_{R}$ is expected to transform according to the
$(\frac{1}{2},0)$ representation under Lorentz group $SU(2)_{L}\times
SU(2)_{R}^{\dagger}$. This means that $\lambda_{R}\lambda_{R}$ must be treated
as a Lorentz scalar, that is $\epsilon_{ij}\lambda_{R}^{i}\lambda_{R}^{j}$
(here $i,j$ are $SU(2)_{R}$ spinor indices). To sharpen this point, with the
focus on its underlying antisymmetric Lorentz structure, a somewhat better
notation for the composite is perhaps $\lambda_{L}[\lambda_{R}\lambda_{R}]$.
The same should hold of course for the left handed composite fermions
$\lambda_{L}\lambda_{L}\lambda_{L}$ as well, whose inner Lorentz structure is
better manifested via $\lambda_{L}[\lambda_{L}\lambda_{L}]$.

At any rate, recalling that $\lambda_{L}$ and $\bar{\lambda}_{L}$ are both
$SU(3)_{F}$ anti-triplets, there seems to be just one way to cancel the
potentially newly induced ABJ anomalies. Quarks and anti-quarks are called to
the rescue, and they do it quite naturally by means of

$\lambda_{L},\bar{\lambda}_{L}\sim 3^{\star}\quad\Longrightarrow\quad
q_{L},\bar{q}_{L}\sim 3~{},$ (12)

to be regarded a fundamental feature of the quark/lepton symmetry.

While $\lambda_{L},\bar{\lambda}_{L}\sim 3^{\star}$, notice that the emerging
composite lepton state $\lambda_{L}\lambda_{R}\lambda_{R}$, previously
identified as $\ell_{L}+\bar{\ell}_{L}$, transforms under $SU(3)_{F}$
precisely like $q_{L},\bar{q}_{L}$, that is
$\lambda_{L}\lambda_{R}\lambda_{R}\sim 3$. In turn, with the addition of
$SU(3)_{F}$, the attractive trinification scheme, dynamically encountered at
the single family level following leptoquark confinement, is apparently in
jeopardy. The accumulated anomaly of twenty seven $SU(3)_{F}$ triplets must be
taken care of in order to gauge protect the masslessness of the three
$\\{q_{L},\bar{q}_{L},\ell_{L}+\bar{\ell}_{L}\\}$ flavor-chiral standard
trinification families. One is thus after a leptocolor singlet, $SU(3)_{F}$
non-trivial composite fermion which, on self consistency grounds, must further
transform as a total multi-singlet $(1,1,1)$ under $SU(3)_{C}\times
SU(3)_{L}\times SU(3)_{R}$.

Figure 2: Lepto/Dark portal: $\phi\ell\Lambda$ Yukawa coupling involves the
same Higgs scalar which couples (and generates masses upon developing VEV) to
standard model fermions. $\phi_{S}\sim 6$ under $SU(3)_{F}$ couples
symmetrically in family space.

It comes with no surprise that precisely such a composite fermion does exist
in the model and is at our service (see anomaly details below). We refer of
course to $\Lambda_{L}\equiv\lambda_{L}\lambda_{L}\lambda_{L}\sim 10^{\star}$
which has already made its appearance in Eq.(8). The fact that $\Lambda_{L}$
is totally symmetric in its $SU(3)_{F}$ spinor indices is crucial. By the same
symmetry token, it chooses for us the symmetric
$\lambda_{L}\lambda_{R}\lambda_{R}$ triplet candidate Eq.(11b). The anti-
symmetric structure of the lepton wave function under identical pre-lepton
interchange, encountered already at the single family model, is preserved (it
could not have been done differently) in the modified threefold family
version.

In 4-dim, the $SU(n)$ triangle anomaly table anomaly  offers the relation

$A(\square\square\square)=\frac{1}{2}(n+6)(n+3)A(\square)~{},$ (13)

where the box stands for the fundamental representation, and in general we
have $A(r^{\star})=-A(r)$. In particular for $n=3$, the above formula reads

$A(10^{\star})+27A(3)=0~{},$ (14)

thereby elevating $\Lambda_{L}=\lambda_{L}\lambda_{L}\lambda_{L}$ to the level
of a mandatory supplement to the threefold quark/lepton family (spectators $+$
composites) flavor chiral representation. Altogether, the latter is given
explicitly by

$\psi_{L}^{tri}=\begin{array}[]{|c ||c|c|c||c|}\hline\cr{}\hfil&C&L&R&F\\\
\hline\cr\hline\cr~{}q_{L}{}&3&~{}3^{\star}&1&3\\\
\hline\cr~{}\bar{q}_{L}{}&~{}3^{\star}&1&3&3\\\
\hline\cr~{}\ell_{L}+\bar{\ell}_{L}\equiv\lambda_{L}\lambda_{R}\lambda_{R}{}&1&3&~{}3^{\star}&3\\\
\hline\cr~{}\Lambda_{L}\equiv\lambda_{L}\lambda_{L}\lambda_{L}{}&1&1&1&~{}10^{\star}\\\
\hline\cr\end{array}$ (15)

The overall anomaly is now $9\times 1+9\times 1+9\times 1-1\times 27=0$. Had
we started from $SU(n)_{F}$, thereby facing the anomaly equation $3\times
9\times 1-1\times\frac{1}{2}(n+6)(n+3)=0$, the special case of $n=3$ would
have been singled out on anomaly free and Pauli exclusion grounds. This
establishes a novel group theoretical interplay between the single fermionic
family and its threefold replication.

It is only when $SU(3)_{F}$ gets broken down, and leaves no residual gauge
symmetry to protect $\Lambda_{L}$ from acquiring the heavy mass scale
involved, that one finally faces the familiar $E(6)$-unifiable trinification
scheme (threefold ’superfluous’ replication included). Be reminded that
Lorentz invariance allows for two types of mass terms. Dirac mass term
${f_{1L}}^{\dagger}f_{2R}$ involves two opposite helicity Weyl fermions
$f_{1L}$ and $f_{2R}$ which must then carry the same conserved charges
$Q(f_{1L})-Q(f_{2R})=0$. Majorana mass term, say $f_{1L}f_{2L}$, involves two
(may be the same) same helicity Weyl fermions, with $Q(f_{1L})+Q(f_{2L})=0$.
By acquiring a heavy Majorana mass $m_{\Lambda}\Lambda_{L}\Lambda_{L}$ at this
level (no $\Lambda_{R}$ at our disposal to produce a Dirac mass as well),
$\Lambda_{L}$ plays here a role similar to that of the $\nu_{R}$ in the
standard model. In fact, appreciating its electro/nuclear neutrality, it may
trigger the very same neutrino seesaw mechanism provided it can couple to
standard model particles. A possible (there may be others) tree level
mechanism by which $\Lambda_{L}$ can acquire its Majorana mass is suggested in
Fig.(3). Owing to the $SU(3)_{F}$ multiplication feature $10\times 10\subset
6\times 6\times 6$, it involves exactly three Yukawa couplings. The relevant
inner fermion lines are associated with heavy Majorana neutrinos (there are
two per trinification family) which live outside minimal
$G_{LR}=SU(2)_{L}\times SU(2)_{R}\times U(1)_{L}\times U(1)_{R}$. Note that
$\Lambda_{L}$ Yukawa mix with both heavy LR components $(2_{1},2_{-1})$ and
$(1_{-2},1_{2})$ of $\ell_{L}+\bar{\ell}_{L}$. Two of the three scalar lines
end with a $\langle 2_{-1},2_{1}\rangle$ VEV, while the third one ends with a
$\langle(1_{2},1_{-2})\rangle$ VEV, respectively.

Figure 3: Dark mass origin: Depicted is a tree level diagram, involving
exactly three Yukawa couplings, by which the trinification singlet fermion
$\Lambda_{L}$ acquires its Majorana mass.

To refresh the Higgs mechanism scheme in the presence of the newly introduced
$SU(3)_{F}$ group, we first observe that the Yukawa interactions are now
governed by $\phi(1,1;3^{\star},3;3^{\star}+6)$. They couple symmetrically
($\phi_{S}\sim 6$) and/or anti-symmetrically ($\phi_{A}\sim 3^{\star}$) in the
family space. Given the odd number of families, $\phi_{A}$ is not allowed to
Yukawa couple by itself, as otherwise one family stays massless and the other
two degenerate. This makes the symmetric Yukawa $\phi_{S}$ couplings
essential. The novel ingredient is that the ordinary Yukawa couplings,
depicted in Fig.(1), are now supplemented, as is demonstrated in Fig.(2), by
$\phi_{S}(\ell_{R}+\bar{\ell}_{R})\Lambda_{L}$. This can be interpreted as a
lepto/dark portal. After all, $\Lambda_{L}$ is a Majorana heavy trinification
singlet, striped from all electro/nuclear interactions, capable of seesaw
mixing $(\theta\sim m_{\ell}/m_{\Lambda})$ with standard model fermions.

Epilogue

While quarks and leptons are equal rights members of the standard
electro/nuclear theory, they are not really treated on symmetrical footing.
Lepton number can serve as the fourth ”color”, but it is harder to imagine
baryon number serving as the fourth ”leptocolor”. The fact that an integer can
be built out of thirds, but not vice versa, naively suggests that leptons
(integer charges) are made out of three pre-leptons (fractional charges), such
that one would not be able to tell quarks from pre-leptons at the fundamental
level. A group theoretical realization of this idea is provided in this
Letter, with $E(6)$-unifiable trinification emerging in the intermediate
stage, and with the threefold family structure inductively incorporated. To
answer the question in the title, leptons need not be, at least theoretically,
as elementary as quarks.

A few comments are in order:
(i) In analogy with LR symmetry, one may anticipate $C\ell C$ symmetry to be
spontaneously violated, such that leptocolor becomes the first interaction to
confine. Until such a symmetry breaking mechanism is introduced, one has to
explicitly assume $g_{\ell C}\gtrsim g_{C}$.
(ii) We can say nothing compelling at this stage about the quark/lepton mass
spectrum and mixings. While we rely on the dynamical emergence of the
intermediate trinification phase, we have not incorporated any particular
trinification model (from existing ones 333  or beyond) for probing fine
details.
(iii) It is not clear what ingredients can be added to our model in order to
achieve (or at least tenably avoid) the ’t Hooft naturalness requirement. In
addition, we have not investigated at this stage what could be the benefits,
if at all, of incorporating supersymmetry.

###### Acknowledgements.

I would like to dedicate this Letter to the memory of Prof. Kameshwar C. Wali
(Syracuse University). A scholar, a mentor, a colleague and a very dear
friend.

## References

  * (1) J.C. Pati and A. Salam, Phys. Rev. D10, 275 (1974).
  * (2) F. Gursey, P. Ramond and P. Sikivie, Phys. Lett. 60B, 177 (1976); Y. Achiman and B. Stech, Phys. Lett. 77B, 389 (1978); Q. Shafi, Phys. Lett. B79, 301 (1978); R. Barbieri and D.V. Nanopoulos, Phys. Lett. 91B, 369 (1980).
  * (3) A. de Rujula, H. Georgi and S.L. Glashow, in Fifth Workshop on Grand Unification, edited by K. Kang, H. Fried and P. Frampton, World Scientific (1984); K.S. Babu, X.G. He and S. Pakvasa, Phys. Rev. D33, 763 (1986); J.E. Kim, Phys. Lett. B591, 119 (2004); J. Sayre, S. Wiesenfeldt and S. Willenbrock, Phys. Rev. D73, 035013 (2006); J. Hetzel and B. Stech, Phys. Rev. D91, 055026 (2015); G.M. Pelaggi, A. Strumia and S. Vignali, JHEP 08, 130 (2015); K.S. Babu, B. Bajc, M. Nemevšek and Z. Tavartkiladze, AIP Conf. Proc. 1900, 020002 (2017); Z.W. Wang, A. Al Balushi, R. Mann and H.M. Jiang, Phys. Rev. D99, 115017 (2019); K.S. Babu, S. Jana and A. Thapa, JHEP 02, 051 (2022).
  * (4) P. H. Frampton and S. L. Glashow, Phys. Lett. B190,157 (1987); A. Davidson and K.C. Wali Phys. Lett. B194, 247 (1987).
  * (5) H. Harari and N. Seiberg, Phys.Lett. B98, 269 (1981); ibid Nucl. Phys. B204, 141 (1982); A. Davidson and J. Sonnenschein, Phys. Lett. B123, 299 (1983).
  * (6) T. Kobayashi, Phys. Rev. D31, 2340 (1985); Y. Koide, Il Nuovo Cimento 95, 160 (1986); J.J. Dugne, S. Fredriksson and J. Hansson, Euro. Lett. 60, 188 (2002).
  * (7) T. Yanagida, Phys. Rev. D20, 2986 (1979); Z.G. Berezhiani, Phys. Lett. B150, 177 (1985).
  * (8) R. Foot and H. Lew, Phys. Rev. D41, 3502 (1990); R. Foot, H. Lew and R.R. Volkas, Phys. Rev. D44, 1531 (1991); K.S. Babu, E. Ma and S. Willenbrock, Phys. Rev. D69, 051301(R) (2004); A. Demaria, C.I. Low and R.R. Volkas, Phys. Rev. D72, 075007 (2005); K.S. Babu, T.W. Kephart and H. Pas, Phys. Rev. D77, 116006 (2008); C. Kownacki, E. Ma, N. Pollard, O. Popov and M. Zakeri, Phys. Lett, 769B, 267 (2017).
  * (9) G. ‘t Hooft, in Recent Developments in Gauge Theories, (Plenum Press, New York, 1980) , p.135; reprinted in Unity of Forces in the Universe, A. Zee, editor (World Scientific, Singapore, 1982), Vol.II, p.1004; S. Dimopoulos, S. Raby and L. Susskind, Nucl. Phys. B173, 208 (1980); Y. Frishman, A.Schwimmer, T.Banks and S.Yankielowicz, Nucl. Phys. B177, 157 (1981); I. Bars, Phys. Lett. 106B, 105 (1981).
  * (10) J. Terning, Phys. Rev. Lett. 80, 2517 (1998).
  * (11) E. Ma, Phys. Lett. B593, 198 (2004); E. Ma, Mod. Phys. Lett. A20, 1953 (2005); E. Ma, M. Mondragon and G. Zoupanos, JHEP 12, 026 (2005); Q.H. Cao, S.L. Chen, E. Ma and G. Rajasekaran, Phys. Rev. D73, 015009 (2006). K.S. Babu, T.W. Kephart and H. Pas, Phys. Rev. D77,116006 (2008); T. Banks and J.F. Fortin, JHEP 07, 046 (2009); C. Kownacki, E. Ma, N. Pollard, O. Popov and M. Zakeri, Phys. Lett. B769, 267 (2017); ibid Nucl. Phys. B928, 520 (2018); ibid Phys. Lett. B777, 121 (2018); E. Ma, Phys. Lett. B820, 136561 (2021).
  * (12) T. Appelquist and J. Carrazone, Phys. Rev. D11, 2856 (1975).
  * (13) J. Banks and H. Georgi, Phys. Rev. D14, 1159 (1976); J. Patera and R.T. Sharp, J. Math. Phys. 22, 2352 (1981); U. Sarkar, IJMPA 22, 931 (2007).