# Supporting Undotted Arabic with Pre-trained Language Models

Aviad Rom  Kfir Bar
The Data Science Institute, Reichman University, Herzliya, Israel
<EMAIL_ADDRESS>
###### Abstract

utf8

utf8

##  1 Introduction

Arabic is a highly inflected Semitic language, spoken by almost 400 million
native speakers around the world. Arabic words are highly ambiguous, mostly
due to the lack of short vowels, represented by diacritic vocalization marks,
which are typically omitted in standard writing. Modern Standard Arabic (MSA),
is the language that is used in official settings, while the dialectal
variants of Arabic are used in day-to-day conversations. In addition to
vocalization marks, some Arabic letters carry dots, called i’jaam (إِعْجَام>),
which are used to distinguish between consonants represented by the same
orthographic form, or rasm (رَسْم>) in Arabic. For example, the letters Tā
(تـ>), Yā (يـ>), Thā (ثـ>), Bā (بـ>), and Nun (نـ>) have exactly the same
orthographic shape, excluding the number and location of the dots they carry.
Without the dots, the letter remains ambiguous. Nevertheless, some dots are
sometimes forgotten in handwritten scripts, forcing the reader to use the
surrounding context in order to resolve such ambiguities. It becomes slightly
more complicated when some of the letters turn into other Arabic letters after
their dots are being removed. For example, by removing the three dots from the
letter Shīn (ش>) we get the letter Sīn (س>), and that makes different words
look the same. For example, the reader may have difficulty understanding the
meaning of the word سعب> (sa’b), which can be interpreted without the dots as
"sigh", and with the dots شعب> (sha’b) as "people". Additional examples are
provided in Table 2. Fortunately, dots are strictly used in digitized texts.
However, we have noticed a recent trend of removing dots from Arabic posts on
social media Drißner (2021)111https://arabic-for-nerds.com/dotless-arabic/,
where people use special keyboards and applications to naturally write without
dots, mainly for bypassing automatic content-filtering algorithms to avoid
having their message classified as offensive. It seems like most native Arabic
speakers can still understand the meaning of the text, even if provided
dotless. Table 1 shows an example for a text and its undotted version. The use
of dots for distinguishing between consonants was introduced to the Arabic
language after the rise of Islam, when non native speakers started showing
interest in the new religion. Until that time, the knowledge of how to
pronounce undotted text was based on the reader’s memory and the surrounding
context Daniels (2014). The use of Transformer Vaswani et al. (2017) in
natural language processing (NLP) has become fundamental to achieve state-of-
the-art results in different downstream tasks, including content filtering.
Since Transformer-based language models are trained with digitized texts, the
vocabulary acquired from the data is represented with dots. Therefore, the
undotted letters that are not part of the official Arabic language, are not
recognized by the model, even if they exist in the Unicode character set
(e.g., "Dotless Archaic Beh" [ٮ>]). In this work, we study the effect of
removing dots from text written in Arabic, on the performance of a
Transformer-based language model, employed as a typical content-filtering
classifier. Our results show that replacing the dotted MSA letters with their
corresponding dotless versions, causes a strong adversarial effect on the
performance of the language model that was fine-tuned on various downstream
tasks. We describe our attempts to handle undotted Arabic, none of them
require re-training the language model, and discuss their results and
potential contributions.

Original |  وتعد "قاروه" إحدى الجزر الكويتية التسع التي تنتشر في المياه الإقليمية الكويتية>
---|---
Undotted |  وٮعد "ڡاروه" إحدى الحرر الكوٮٮىه الٮسع الٮى ٮٮٮسر ڡى المٮاه الإڡلٮمىه الكوٮٮٮه>
Translation | Qaruh is one of the nine Kuwaiti islands in the Kuwaiti territorial waters
Table 1: Arabic text, given with and without the dots. The text was taken from
https://arabic.cnn.com/travel/article/2021/07/13/qaruh-island-kuwait.

##  2 Related Work

###  2.1 Arabic Transformer Models

Multilingual BERT, or mBERT Devlin et al. (2019), was the first pre-trained
language model to include Arabic. It covers only MSA, and usually do not
perform well enough on downstream Arabic NLP tasks, due to the relatively
small Arabic training data it was trained on. AraBERT Antoun et al. (2020) and
GigaBERT Lan et al. (2020) are two language models that were trained on a much
larger portion of Arabic texts, still only MSA. Both offer better performance
on downstream tasks. Two recent models, MARBERT Abdul-Mageed et al. (2021a),
and CAMeLBERT Inoue et al. (2021), include Dialectical Arabic in their
training data, reaching better performance on relevant tasks. None of these
models have been used with undotted Arabic, which is the main focus of our
work.

###  2.2 Adversarial Inputs in NLP

Adversarial inputs are crafted examples to deceive neural networks at
inference time. Such attacks have already been introduced and discussed by
Szegedy et al. 2013 and Goodfellow et al. 2015, focusing mostly on adversarial
perturbations in vision tasks. Generating adversarial inputs in NLP is
considered to be more challenging than in computer vision, mostly due to the
relatively large importance every word has in a given input text, comparing to
the small importance a single pixel has in an input image. Nonetheless, it has
been recently addressed by Jin et al. (2020), who presented an efficient way
of generating adversarial textual inputs for a BERT Devlin et al. (2019), by
modifying the texts semantically based on some word statistics taken from the
language model itself. They showed that while their modified texts are
understandable by human readers, their BERT-based models have struggled to
produce the correct output. In this work, we evaluate a more natural approach
for fooling an Arabic language model, simply by converting some letters to
their undotted versions, keeping the modified text understandable for human
readers.

##  3 Handling Undotted Arabic

We begin by fine-tuning a Transformer-based Arabic language model on two
downstream tasks, and evaluate their performance on undotted inputs. Following
that, we develop different computational approaches for recovering the missing
information that was lost with undotting, without pre-training the language
model itself. We evaluate the different approaches on the same downstream
tasks, and report on the results in the following section. For all our
experiments we use the recent CAMeLBERT-Mix base model Inoue et al. (2021),
which was pre-trained on a mix of MSA, Classical Arabic, and Dialectical
Arabic texts.

###  3.1 Undotting

In order to remove dots from the text, we created a mapping for all the Arabic
characters available in the Unicode character set, for which we match the most
resemblant undotted character. The mapping table is provided in Appendix A.
Some Arabic letters have different forms, depending on whether they appear at
the beginning, middle or end of a word. Therefore, we map all the forms of a
relevant letter. Undotting an input text is a simple replacement of all
relevant letters with their orthographic equivalents.

###  3.2 Supporting Undotted Arabic

As reported in the following section, fine-tuned Arabic language models do not
perform well on undotted texts. Therefore, we suggest two ways to handle
undotted texts. In one way, we make changes to the tokenizer of the model, and
in another way we develop an algorithm for restoring the dots of the input
text, which runs as a pre-processing step before submitting the text to the
language model.

####  3.2.1 Changing the Tokenizer

Before processing the text with a pre-trained language model, it is necessary
to break it into tokens using the same tokenizer that was used during the pre-
training phase of the model. CAMeLBERT-Mix uses a standard BERT tokenizer,
provided by Hugging Face222https://github.com/huggingface/tokenizers, with a
vocabulary of 30,000 tokens. Each token has a numeric identifier. We take two
different approaches for changing the configuration of the tokenizer in order
to handle undotted texts without having to pre-train the language model, nor
fine-tuning it on a downstream task.

##### Undotting the Tokenizer Vocabulary.

According to this approach, we undot the entire vocabulary of the tokenizer,
thereby enabling it to seamlessly recognize undotted letters and words.
Obviously, after undotting the vocabulary some of the tokens (5,852 out of the
original 30,000 tokens, or $19.52\%$) become identical, leaving some of the
token identifiers unused; therefore, the model’s vocabulary get smaller. Since
we suspect that working with a smaller vocabulary may be detrimental to the
performance of the model on downstream tasks, we suggest another approach for
modifying the tokenizer.

##### Extending the Tokenizer Vocabulary.

Under this approach, we extend the tokenizer’s vocabulary by adding the
undotted version of the relevant tokens and mapping them to the same
identifier of their original token. This way the tokenizer keeps the original
dotted version of every token, and thus can accept both, dotted and undotted
inputs. We add the undotted version of a token only if it is not already part
of the vocabulary; overall, we added 17,280 undotted versions. The resulting
vocabulary has about $57\%$ token identifiers that are mapped to two token
versions.

####  3.2.2 AReDotter: Restoring Arabic Dots

As opposed to the previous approach, here we develop an algorithm for pre-
processing the input undotted text to restore its dots. The language model
itself remains unmodified. We train a sequence-to-sequence machine-translation
(MT) model on the unlabeled 10M Arabic tweets dataset published with the
second NADI shared task Abdul-Mageed et al. (2021b). The tweets were posted
from multiple geographies. For creating parallel texts for training, every
tweet from the original corpus was paired with its automatically generated
undotted version, using the mappings provided in Appendix A. We remove from
the tweets URLs, user mentions, and hashtags. Our MT model is based on the
pre-trained Arabic-to-English Marian MT Tiedemann and Thottingal (2020)
architecture333Specifically, we used the Helsinki-NLP/opus-mt-ar-en model from
Hugging Face., which was fine-tuned for "undotted Arabic"-to-Arabic
translation. We fine-tune our model on the entire parallel dataset for two
epochs.

Undotted | Option 1 (pronunciation, meaning) | Option 2 (pronunciation, meaning)
---|---|---
ڡٮحٮ> |  فيجب> (fyajib, "must") |  فتحت> (fatahat, "opened")
ٮڡارٯ> |  تفارق> (tafaruq, "leave") |  بفارق> (bifariq, "difference")
ٮحار> |  نجار> (najaar, "carpenter") |  بحار> (bahaar, "seas")
حٮوٮ> |  حبوب> (hubub, "cereal") |  جنوب> (janub, "south")
Table 2: Examples of undotted ambiguous words. We do not provide all the
possible pronunciations in each row.

##  4 Experimental Results

| ArSarcasm V2 - Sentiment | ANERCorp
---|---|---
Original Text | $70.55$ | $81.39$
Undotted Text | $44.86$ | $9.16$
Undotted Text + Undotted Tokenizer | $64.50$ | $72.85$
Undotted Text + Extended Tokenizer | $65.03$ | $71.68$
AReDotter | $68.27$ | $67.97$
Table 3: Model performance on downstream tasks, using different undotted text
handling approaches.

To evaluate our proposed methods, we fine-tune CAMeLBERT on two tasks,
sentence level and token level. For the sentence-level task we use the
sentiment analysis subtask of ArSarcasm-v2 Abu Farha et al. (2021), designed
as a three-labels (positive, negative, neutral) classification task. As we did
with NADI, we preprocess the text to remove URLs, user mentions, and hashtags.
For evaluation, we use the official evaluation objective metric, defined as
macro average F1 score of both non-neutral labels. For a token-level
downstream task, we evaluate our language model on the named-entity
recognition (NER) task using the ANERcorp dataset Benajiba et al. (2007). We
use the modified version of the dataset, which was recently released by Obeid
et al. (2020). Following previous works on NER, we use the micro average F1
metric for evaluation. For each task, we fine-tune CAMeLBERT on the original
preprocessed training data for 10 epochs, using the official train/test split,
and evaluate it on the undotted version of the test set. We use the standard
Hugging Face’s pipelines, AutoModelForSequenceClassification and
AutoModelForTokenClassification444https://huggingface.co/transformers/model_doc/auto.html
for the sentiment analysis and NER tasks, respectively. We evaluate the models
under different conditions of supporting undotted texts, as described in the
previous section.

###  4.1 Results

The results, reported in Table 3, demonstrate the adversarial effect of
processing undotted Arabic with a vanilla, unmodified CAMeLBERT model. The
first row lists the results we get by working with the original texts. In the
second row we provide the results of using the same model, but this time
applied on the undotted version of the texts. As observed, the metrics
measured for the two tasks dropped significantly on undotted texts.
Unsurprisingly, the tokenizer of the vanilla language model does not recognize
tokens with undotted letters, which are excluded from the modern Arabic
script, and thus treating them as “unknown” tokens. The two tokenizer-updating
approaches, whose results are reported in the 3rd and 4th rows, prove to be
effective for undotted texts, in both tasks. This improvement is achieved
mainly due to the reduction in the number of unknown tokens the model is
assigned with. Among the two, we observe that the extended tokenizer is
slightly better on the sentiment analysis task, while the undotted tokenizer
is better on the NER task. However, the difference in those results is
insignificant. Interestingly, AReDotter, our MT dots restoration model, which
we run as a preprocessor before submitting the text to the language model,
provides competitive results in both task. It is slightly better than the
tokenizer-updating techniques on sentiment analysis, but slightly worse on the
NER task. Naturally, a sequence-to-sequence translation model may sometimes
generate some out-of-context tokens in the target sequence. We believe that
NER is more sensitive to this type of mistakes than sentiment analysis task.
For future work, we plan to work with a simple sequential-tagging model
instead of the sequence-to-sequence MT model, to avoid generating such tokens.
The results we get from AReDotter are encouraging; it provides an elegant way
to support undotted text without modifying the model or the tokenizer.

##  5 Conclusion

Undotting has been recently adopted by social-media users in order to bypass
content-filtering gateways. We studied the effect of undotting on the
performance of a standard pre-trained language model. Our results show that
processing undotted text with a vanilla, unmodified language model, has a
detrimental effect in two downstream NLP tasks. By simply editing the
tokenizer, which is used by the language model, we are able to show
significant improvements over the vanilla model. Our third approach, which
does not require changing the tokenizer, is using a machine-translation model
for restoring the missing dots. With this technique we show competitive
results to the tokenizer-updating techniques, without having to modify the
model or its tokenizer. We believe that our study provides some conclusions as
for how undotted texts should be treated with modern Transformer-based
language models. We recommend that at least one of our techniques will be
adopted as a standard step in a common Arabic NLP pipeline.

## References

  * Abdul-Mageed et al. (2021a) Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021a.  ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic.  In _Proceedings of the ACL-IJCNLP 2021 Main Conference_. Association for Computational Linguistics.
  * Abdul-Mageed et al. (2021b) Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2021b.  NADI 2021: The second nuanced Arabic dialect identification shared task.  In _Proceedings of the Sixth Arabic Natural Language Processing Workshop_ , pages 244–259, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  * Abu Farha et al. (2021) Ibrahim Abu Farha, Wajdi Zaghouani, and Walid Magdy. 2021.  Overview of the WANLP 2021 shared task on sarcasm and sentiment detection in Arabic.  In _Proceedings of the Sixth Arabic Natural Language Processing Workshop_ , pages 296–305, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  * Antoun et al. (2020) Wissam Antoun, Fady Baly, and Hazem Hajj. 2020.  AraBERT: Transformer-based model for Arabic language understanding.  In _Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection_ , pages 9–15, Marseille, France. European Language Resource Association.
  * Benajiba et al. (2007) Yassine Benajiba, Paolo Rosso, and José Miguel BenedíRuiz. 2007.  ANERsys: An Arabic named entity recognition system based on maximum entropy.  In _Computational Linguistics and Intelligent Text Processing_ , pages 143–153, Berlin, Heidelberg. Springer Berlin Heidelberg.
  * Daniels (2014) Peter T. Daniels. 2014.  _The Type and Spread of Arabic Script_, pages 25 – 39. Brill, Leiden, The Netherlands.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Drißner (2021) Gerald Drißner. 2021.  Social media & palestine: Dotless Arabic outsmarts algorithms.
  * Goodfellow et al. (2015) Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015.  Explaining and harnessing adversarial examples.  In _International Conference on Learning Representations_.
  * Inoue et al. (2021) Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021\.  The interplay of variant, size, and task type in Arabic pre-trained language models.  In _Proceedings of the Sixth Arabic Natural Language Processing Workshop_ , Kyiv, Ukraine (Online). Association for Computational Linguistics.
  * Jin et al. (2020) Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020.  Is BERT really robust? A strong baseline for natural language attack on text classification and entailment.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020_ , pages 8018–8025. AAAI Press.
  * Lan et al. (2020) Wuwei Lan, Yang Chen, Wei Xu, and Alan Ritter. 2020.  An empirical study of pre-trained transformers for Arabic information extraction.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4727–4734, Online. Association for Computational Linguistics.
  * Obeid et al. (2020) Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020.  CAMeL tools: An open source python toolkit for Arabic natural language processing.  In _Proceedings of the 12th Language Resources and Evaluation Conference_ , pages 7022–7032, Marseille, France. European Language Resources Association.
  * Szegedy et al. (2013) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013.  Intriguing properties of neural networks.  _arXiv preprint arXiv:1312.6199_.
  * Tiedemann and Thottingal (2020) Jörg Tiedemann and Santhosh Thottingal. 2020.  OPUS-MT — Building open translation services for the World.  In _Proceedings of the 22nd Annual Conferenec of the European Association for Machine Translation (EAMT)_ , Lisbon, Portugal.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc.

##  Appendix A Appendix A: Undotting Table

|  | Initial/Medial | Terminal
---|---|---|---
Letter | MSA | Undotted | Undotted
Ba |  ب> |  ٮـ> |  ٮ>
Ta |  ت> |  ٮـ> |  ٮ>
Tha |  ث> |  ٮـ> |  ٮ>
Jim |  ج> |  حـ> |  ح>
Kha |  خ> |  حـ> |  ح>
Dhal |  ذ> |  د> |  د>
Zayn |  ز> |  ر> |  ر>
Shin |  ش> |  سـ> |  س>
Dad |  ض> |  صـ> |  ص>
Za’ |  ظ> |  طـ> |  ط>
Ghayn |  غ> |  عـ> |  ع>
Fa |  ف> |  ڡـ> |  ڡ>
Qaf |  ق> |  ٯـ> |  ٯ>
Nun |  ن> |  ٮـ> |  ں>
Ya |  ي> |  ٮـ> |  ى>
Table 4: Character mapping used for our undotting function, specifying only
the letters which are not identical to their undotted version.


[1]Angela Giraldi 11footnotetext: For the CMS Collaboration.

#  $\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$/$\mathrm{t}\mathrm{H}$
production at CMS

###### Abstract

After the Higgs boson discovery, a main focus at the CERN LHC has been the
measurement of its properties. Observing the Higgs boson associated production
with a top quark-antiquark pair ($\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$)
is particularly interesting because it provides tree-level access to measuring
the Higgs boson–top quark Yukawa coupling. However, the production of a single
top quark in association with the Higgs boson ($\mathrm{t}\mathrm{H}$) is also
sensitive to the sign of this coupling. This coupling is essential for the
stability of the Higgs potential at high energy scales and can also be a probe
for physics beyond the standard model (SM). The latest measurements of the
$\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ and $\mathrm{t}\mathrm{H}$
associated production rates performed by the CMS Collaboration with
proton–proton collision events at $\sqrt{s}=13$ TeV in the diphoton and
multilepton channels are presented here. Events are categorized according to
the lepton and jet multiplicities, and multivariate classifiers are used to
distinguish signal from background processes. The results are consistent with
the SM predictions and feature the first observation of
$\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ production in a single Higgs boson
decay channel along with the first tests of its CP structure.

##  1 Introduction

With the discovery of the Higgs boson (H) by the ATLAS [1, 2] and CMS [3, 4,
5] Collaborations in 2012, the last missing piece of the standard model (SM)
of particle physics was observed. The thorough investigation of its properties
is still of crucial relevance. These proceedings focus on the Yukawa
interaction ($y_{t}$) between the Higgs boson and the top quark. The
production of the Higgs boson in association with a top quark–antiquark pair
($\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$) or with a single top quark
($\mathrm{t}\mathrm{H}$) involve Higgs-boson – top-quark couplings at tree
level. In the SM, the $\mathrm{t}\mathrm{H}$ production cross section is
expected to be particularly small, due to the negative interference of the two
leading order diagrams, one in which the Higgs boson couples to a top quark,
and one in which it couples to a W boson. However, if the relative sign of
these couplings is inverted, the interference becomes constructive, hence
boosting the cross section by an order of magnitude. The measurement of
$\mathrm{t}\mathrm{H}$ production thus offers sensitivity to the sign of the
top Yukawa coupling $y_{t}$.

The $\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ process has a complex and rich
variety of final states resulting from the combinations of
$\mathrm{t}\overline{\mathrm{t}}$ and Higgs boson decay channels. The first
observation of $\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ was based on a
combined analysis of proton-proton collision data at $\sqrt{s}=7,8$ and
$13\,$TeV recorded with the CMS detector and corresponding to an integrated
luminosity of up to 5.1, 19.7, and 35.9 $\mathrm{fb^{-1}}$, by taking into
account the Higgs boson decays into pairs of W and Z bosons, photons, $\tau$
leptons, or bottom quark jets [6]. Since then, the measurements in the
$\gamma\gamma$ and multilepton final states of the Higgs boson have been
updated using the full LHC Run-2 data set, which corresponds to an integrated
luminosity of 138 $\mathrm{fb^{-1}}$. These proceedings provide an overview of
the latest results.

##  2 Measurement in the $\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$
($\mathrm{H}\to\gamma\gamma$) final state

Despite the low branching fraction of the Higgs boson decaying into photons,
diphoton signatures can be measured very precisely with the CMS detector, thus
allowing to resolve the Higgs boson system in
$\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$($\mathrm{H}\to\gamma\gamma$)
candidate events with little ambiguity. A measurement of
$\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ production [7] is performed using
events in which the diphoton invariant mass is required to be in the range of
100 < $m_{\gamma\gamma}$ < 180 GeV, and the top quark pair final state
includes the fully hadronic and lepton+jets channels. A boosted decision tree
(BDT) is trained in each channel to discriminate between signal and background
events. Only events with high BDT scores are further classified into four
regions in the lepton+jets and hadronic final states, respectively. A
simultaneous maximum likelihood fit based on the diphoton invariant mass
distributions is performed using a double-sided Crystal Ball parametrization,
to extract the signal strength
$\mu_{\mathrm{t}\overline{\mathrm{t}}\mathrm{H}}$, defined as the ratio of the
measured and the expected cross section. This yields a signal strength
$\mu_{\mathrm{t}\overline{\mathrm{t}}\mathrm{H}}=1.38^{+0.36}_{-0.29}$,
corresponding to an observed (expected) significance of 6.6 (4.7) standard
deviations, marking the first observation of the
$\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ process in a single Higgs boson
decay channel. The CP structure of the Higgs boson couplings to fermions is
also tested by using a dedicated BDT trained to distinguish between CP-even
and CP-odd contributions to the coupling. The sample is categorized in bins of
the BDT output representing the $\mathcal{D}_{0-}$ observable [8], and a
simultaneous fit to the diphoton mass distributions is performed with
$\mu_{\mathrm{t}\overline{\mathrm{t}}\mathrm{H}}$ left unconstrained. The
results are shown in Figure 1 (left), with the fractional contribution of the
CP-odd component measured to be $f_{\mathrm{CP}}^{\mathrm{Htt}}=0.00\pm 0.33$,
with $|f_{\mathrm{CP}}^{\mathrm{Htt}}|<0.67$ at 95% confidence level (C.L.).
The hypothesis of a pure pseudoscalar CP-odd structure of the Htt coupling
($f_{\mathrm{CP}}^{\mathrm{Htt}}=1$) is excluded at 3.2 standard deviations.

Figure 1: Left: event distributions in bins of the $\mathcal{D}_{0-}$
discriminant; the inner panel shows the likelihood scan for
$f_{\mathrm{CP}}^{\mathrm{Htt}}$ [7]. Right: observed results of the STXS fit;
the lower panel shows the ratio of the fitted values to the SM predictions
[9].

In the context of measurements of Higgs boson production cross sections and
couplings in events where the Higgs boson decays into a pair of photons, the
first measurement of the $\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$
production in five regions of the Higgs boson transverse momentum is performed
[9]. The same categorization as in Ref. [7] is used to construct
$\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ enriched categories, which are
further divided to provide sensitivity to individual simplified-template-
cross-section (STXS) bins. The $\mathrm{t}\mathrm{H}$ process is also measured
in a dedicated analysis category enriched in tHq events where the top decays
leptonically. The results are shown in Figure 1 (right): the observed
(expected) upper limit for the $\mathrm{t}\mathrm{H}$ production cross section
at 95% C.L. is found to be 14 (8) times the SM prediction.

##  3 Measurements in the multilepton final state

The search for $\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ production in
multilepton final states makes use of Higgs boson decays into WW∗, ZZ∗, and
$\tau^{+}\tau^{-}$, and semileptonic or hadronic decays of the top quark pair
[10]. Although the signal has a significantly larger branching ratio than
$\mathrm{H}\to\gamma\gamma$ decay, this analysis suffers from considerable
background contributions. Events are selected and classified into ten disjoint
signal regions, according to the lepton (e and $\mu$) and hadronically
decaying $\tau$ ($\tau_{h}$) multiplicities; among these, three experimental
categories are constructed to increase the sensitivity to
$\mathrm{t}\mathrm{H}$ events. In the $\mathrm{t}\mathrm{H}$ enriched
categories, artificial neural networks are used to separate signal from
background, and then events are further categorized based on the lepton flavor
and the b-tagged jet multiplicity. The rest of the categories uses a BDT that
discriminates the $\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ signal process
from background processes. The reducible backgrounds of the analysis largely
consists of contributions from non-prompt leptons and misidentified $\tau_{h}$
leptons, as well as electrons of misidentified charge. The contribution from
these processes in the signal regions is estimated using a data-driven method.
Irreducible backgrounds are dominated by $\mathrm{t}\overline{\mathrm{t}}$W
and $\mathrm{t}\overline{\mathrm{t}}$Z production in the main signal regions,
and dedicated control regions are used to constrain their contributions.

The signal strengths are extracted by performing a maximum likelihood fit to
the data in all signal and control region subcategories. The fitted signal
strength is 0.92 $\pm$ 0.19 (stat) ${}^{+0.17}_{-0.13}$ (syst) for the
$\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ signal and 5.7 $\pm$ 2.7 (stat)
$\pm$ 3.0 (syst) for the $\mathrm{t}\mathrm{H}$ signal, and is thus compatible
compatible with the SM expectations (see Figure 2 (left)). The values for the
parameters modifying the $\mathrm{t}\overline{\mathrm{t}}$W and
$\mathrm{t}\overline{\mathrm{t}}$Z production rates after the fit to the data
are $\mu_{\mathrm{t}\overline{\mathrm{t}}W}$ = 1.43 $\pm$ 0.21 (stat+syst) and
$\mu_{\mathrm{t}\overline{\mathrm{t}}Z}$ = 1.03 $\pm$ 0.14 (stat+syst). The
observed (expected) significance for
$\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ production is 4.7 (5.2) standard
deviations, while it is 1.4 (0.3) for the $\mathrm{t}\mathrm{H}$ signal. The
observed event yields are also interpreted in terms of coupling modifiers of
the Higgs boson to the top quark ($\kappa_{t}$) and the W and Z bosons
($\kappa_{V}$). The compatibility of the observed data is compared to a grid
of $(\kappa_{t},\kappa_{V})$ hypotheses, computing a likelihood score to
construct C.L. regions in the $(\kappa_{t},\kappa_{V})$ plane and confidence
intervals in $\kappa_{t}$. The measurements constrains $-$0.9 < $\kappa_{t}$ <
$-$0.7 or  0.7 < $\kappa_{t}$ < 1.1 at 95% C.L.

Additionally, a measurement of the CP structure of the Yukawa interaction
between the Higgs boson and one or two top quarks is performed [11]. Machine
learning techniques are applied to enhance the separation of CP-even from CP-
odd scenarios. Two-dimensional confidence regions are set on the ratios
$\kappa_{t}$ and $\tilde{\kappa}_{t}$ of the couplings of CP-even and CP-odd
Lagrangian terms, respectively, to the SM expectation for the top-Higgs Yukawa
coupling. These regions are found to be $-$1.09 < $\kappa_{t}$ < $-$0.74 or
0.77 < $\kappa_{t}$ < 1.30, $-$1.4 < $\tilde{\kappa}_{t}$ < 1.4 at 95% C.L.
Similar to the $\mathrm{H}\to\gamma\gamma$ analysis, the fractional
contribution of pseudo-scalar couplings is measured to be
$|f_{\mathrm{CP}}^{\mathrm{Htt}}|=0.59$, with an interval of (0.24, 0.81) at
68% C.L. The results are combined with previously published analyses of the
$H\rightarrow\gamma\gamma$ [7] and $H\rightarrow ZZ$ [12] decay modes,
yielding two- and one-dimensional confidence regions on $\kappa_{t}$ and
$\tilde{\kappa}_{t}$, as shown in Figure 2 (right). The modifier
$|f_{\mathrm{CP}}^{\mathrm{Htt}}|$ is measured to be 0.28, with an interval of
$|f_{\mathrm{CP}}^{\mathrm{Htt}}|<0.55$ at 68% C.L.

##  4 Conclusion

The Yukawa interactions between the top quark and Higgs boson are crucial to
establish the validity of the standard model (SM) of particle physics. This
contribution report the measurements of the production of the Higgs boson in
association with a top quark–antiquark pair
($\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$) or with a single top quark
($\mathrm{t}\mathrm{H}$), in the diphoton and multilepton final states, using
the data set collected by the CMS experiment during the second data-taking
period and with improved analysis techniques. The results obtained are
consistent with the SM expectations.

Figure 2: Left: Two-dimensional contours of the likelihood function as a
function of the production rates of the
$\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ and $\mathrm{t}\mathrm{H}$ signal
processes [10]. Right: Likelihood scan as function of $\kappa_{t}$ and
$\tilde{\kappa}_{t}$ and two-dimensional confidence intervals for single final
states and the combination of the three channels [11].

## References

  * [1] ATLAS Collaboration, _The ATLAS experiment at the CERN Large Hadron Collider_ , _JINST_ 3 (2008) S08003.
  * [2] ATLAS Collaboration, _Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC_ , _Phys. Lett. B_ 716 (2012) 1 [1207.7214].
  * [3] CMS Collaboration, _The CMS Experiment at the CERN LHC_ , _JINST_ 3 (2008) S08004.
  * [4] CMS Collaboration, _Observation of a New Boson at a Mass of 125 GeV with the CMS Experiment at the LHC_ , _Phys. Lett. B_ 716 (2012) 30 [1207.7235].
  * [5] CMS collaboration, _Observation of a New Boson with Mass Near 125 GeV in $pp$ Collisions at $\sqrt{s}$ = 7 and 8 TeV_, _JHEP_ 06 (2013) 081 [1303.4571].
  * [6] CMS Collaboration, _Observation of $\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ production_, _Phys. Rev. Lett._ 120 (2018) 231801 [1804.02610].
  * [7] CMS Collaboration, _Measurements of $\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ Production and the CP Structure of the Yukawa Interaction between the Higgs Boson and Top Quark in the Diphoton Decay Channel_, _Phys. Rev. Lett._ 125 (2020) 061801 [2003.10866].
  * [8] A.V. Gritsan, R. Röntsch, M. Schulze and M. Xiao, _Constraining anomalous higgs boson couplings to the heavy-flavor fermions using matrix element techniques_ , _Phys. Rev. D_ 94 (2016) 055023.
  * [9] CMS Collaboration, _Measurements of Higgs boson production cross sections and couplings in the diphoton decay channel at $\sqrt{\mathrm{s}}$ = 13 TeV_, _JHEP_ 07 (2021) 027 [2103.06956].
  * [10] CMS Collaboration, _Measurement of the Higgs boson production rate in association with top quarks in final states with electrons, muons, and hadronically decaying tau leptons at $\sqrt{s}=$ 13 TeV_, _Eur. Phys. J. C_ 81 (2021) 378 [2011.03652].
  * [11] CMS Collaboration, _Search for CP violation in $\mathrm{t}\overline{\mathrm{t}}\mathrm{H}$ and $\mathrm{t}\mathrm{H}$ production in multilepton channels at $\sqrt{s}=13~{}\mathrm{TeV}$_, CMS Physics Analysis Summary CMS-PAS-HIG-21-006, CERN, Geneva (2022).
  * [12] CMS Collaboration, _Constraints on anomalous Higgs boson couplings to vector bosons and fermions in production and decay in the $\mathrm{H}\to 4\ell$ channel_, CMS Physics Analysis Summary CMS-PAS-HIG-19-009, CERN, Geneva (2020).