# Non-contrastive sentence representations via self-supervision

Marco Farina
Bloomberg
<EMAIL_ADDRESS>
&Duccio Pappadopulo11footnotemark: 1
Bloomberg
<EMAIL_ADDRESS>
Alphabetical order

###### Abstract

Sample contrastive methods, typically referred to simply as _contrastive_ are
the foundation of most unsupervised methods to learn text and sentence
embeddings. On the other hand, a different class of self-supervised loss
functions and methods have been considered in the computer vision community
and referred to as _dimension contrastive_. In this paper, we thoroughly
compare this class of methods with the standard baseline for contrastive
sentence embeddings, SimCSE (Gao et al., 2021). We find that self-supervised
embeddings trained using dimension contrastive objectives can outperform
SimCSE on downstream tasks without needing auxiliary loss functions.

##  1 Introduction

Text embeddings are an important tool for a variety of NLP tasks. They provide
a general and compute efficient solution to problems like topic
classification, document clustering, text mining and information retrieval,
among others.

Most modern techniques to learn text embeddings rely on minimizing a
contrastive loss Chopra et al. (2005); van den Oord et al. (2019). This
requires identifying, for each example $x$ in the training set, a _positive_
example $x^{+}$ and a set of _negative_ examples $x^{-}_{i}$ associated to
$x$. The choice of $x^{+}$ and $x_{i}^{-}$ is one of the main factors
differentiating these techniques. Unsupervised methods Zhang et al. (2020);
Giorgi et al. (2021); Chuang et al. (2022) rely on in-batch negatives for the
$x_{i}^{-}$ and data augmentation for $x^{+}$. Supervised or weakly supervised
methods Reimers and Gurevych (2019); Ni et al. (2022b); Wang et al. (2022); Su
et al. (2022); Muennighoff (2022); Ni et al. (2022a) rely either on mining
heuristics or annotated datasets to build the positive and negative pairs, for
instance a common choice is to use entailment and contradiction pairs
respectively, as in SNLI Bowman et al. (2015a) and MNLI Williams et al.
(2018a).

In this work we approach the problem of learning text embedding from the point
of view of which objective function to use. We consider two self-supervised
representation learning algorithms introduced in the computer vision
literature: Barlow Twins (BT) Zbontar et al. (2021) and VICReg Bardes et al.
(2022).

What teases apart these two methods is their nature of being _dimension
contrastive_ according to the classification of Garrido et al. (2022): while
the usual contrastive method, defined by Garrido et al. (2022) as _sample
contrastive_ , avoids the collapse of the learned representations by
penalizing similarity of the embeddings corresponding to different data
points, dimension contrastive methods regularize the objective function by de-
correlating the embeddings across their dimensions. Both sample and dimension
contrastive methods rely on data augmentation in the unsupervised setting.
While good augmentation functions are known and routinely used for image data,
augmentation of textual data is usually considered trickier (Feng et al.,
2021). One of the breakthrough of SimCSE is the realization that using the
model stochastic dropout mask to define the augmented views of the same data
point is an effective choice.

The main goal of this paper is to compare the embeddings learned through
sample-contrastive and dimension-contrastive techniques and explore different
augmentation strategies. We use SimCSE Gao et al. (2021) as our sample-
contrastive baseline and compare it against BT and VICReg111To the best of our
knowledge we are first to use VICReg as an objective to train sentence
embeddings.. Our main findings are: i) Barlow Twins is competitive with
unsupervised SimCSE as a standalone objective function and outperforms it on a
majority of MTEB tasks with a RoBERTa based architectures. This is partly at
odds with the finding of Klein and Nabi (2022) and Xu et al. (2023) which
include new terms in the loss with the motivation that BT alone does not get
better performances than SimCSE. A thorough comparison of dimension and sample
contrastive methods does not exist in the literature. ii) VICReg underperforms
Barlow Twins and SimCSE: we find it harder to optimize it and we cannot
exclude that more hyperparameter exploration and better data augmentation
would lead to better results. iii) We obtain mixed results by using
supervision (for instance from NLI datasets) in place of data augmentation: in
no case supervision leads to better performances across all MTEB downstream
task categories.

##  2 Contrastive techniques

|  | dropout ($p_{\textrm{do}}$) | EDA ($\alpha$) | shuffle ($p_{\textrm{shuffle}}$)
---|---|---|---|---
|  | 0.05 | 0.1 | 0.2 | 0.1 | 0.2 | 0.05 | 0.1 | 0.2 | 0.3 | 0.5
Barlow Twins
BERT | max | 77.9 | 74.0 | 73.5 | 74.3 | 73.9 | 76.6 | 77.8 | 78.9 | 79.5 | 79.6
| q75 | 75.1 | 73.2 | 72.4 | 72.9 | 72.4 | 75.0 | 76.7 | 78.0 | 78.8 | 78.6
| q50 | 74.0 | 72.6 | 72.2 | 72.5 | 71.6 | 73.7 | 75.8 | 76.0 | 77.6 | 77.7
RoBERTa | max | 80.0 | 80.5 | 78.1 | 76.0 | 77.2 | 79.5 | 80.4 | 80.2 | 80.4 | 80.8
| q75 | 78.6 | 77.4 | 77.0 | 74.2 | 75.8 | 78.2 | 80.0 | 79.9 | 80.1 | 80.0
| q50 | 78.0 | 75.2 | 74.4 | 73.1 | 74.4 | 77.6 | 78.7 | 79.4 | 79.8 | 79.5
VICReg
BERT | max | 76.2 | 75.3 | 75.5 | 76.0 | 76.3 | 77.6 | 76.8 | 77.4 | 78.1 | 78.5
| q75 | 74.8 | 74.2 | 74.0 | 75.0 | 75.1 | 76.4 | 75.4 | 77.2 | 77.8 | 77.7
| q50 | 74.5 | 73.5 | 73.0 | 74.2 | 74.2 | 75.3 | 73.8 | 77.0 | 75.9 | 77.2
RoBERTa | max | 81.2 | 81.0 | 81.6 | 80.2 | 80.4 | 82.0 | 81.9 | 81.6 | 82.2 | 82.0
| q75 | 80.7 | 80.4 | 80.3 | 79.0 | 79.3 | 79.7 | 80.9 | 81.3 | 81.3 | 81.8
| q50 | 80.4 | 80.0 | 79.7 | 78.0 | 77.3 | 79.0 | 80.0 | 81.2 | 81.0 | 81.3
Table 1:  max, upper quartile, and median for the distribution of STS-B
Spearman’s correlations on the dev set as a function of the data augmentation.
Bold: overall best score per model, underlined: best score per augmentation.
For VICReg we only ran EDA with with $\alpha=0.1$.

All the techniques that we experiment with in the following can be described
in a unified way. Consider a batch of data points $s_{n}$, $n=1,\ldots,N$
(sentences in this work).222We use $n,m$ to denote different members of the
same batch and $i,j,k$ to denote different dimensions in the same embedding.
The representation $\bf{e}_{n}$ for each point is obtained through a
parametrized sentence encoder (BERT and RoBERTa are what we will use in this
paper): ${\bf{e}}_{n}=E_{\theta}(s_{n})$. In order to consider data
augmentation of any type, we assume that $E_{\theta}$ allows for a second
(possibly random) parameter $\bf{\epsilon}$ specifying the augmentation
${\bf{e}}^{\prime}_{n}=E_{\theta}(s_{n},\bf{\epsilon})$. When training
$E_{\theta}$ in the self-supervised setting we create two embeddings (_views_)
of each point in the batch, ${\bf{e}}^{(A,B)}_{n}$. Each of them is projected
to a high-dimensional space by means of a parametrized _projector_
${\bf{z}}_{n}\equiv P_{\theta}({\bf{e}}_{n})$. The resulting D-dimensional
vectors ${\bf{z}}_{n}$ are then used in the method specific loss function.

SimCSE $-$ Our baseline for sample contrastive methods is SimCSE Gao et al.
(2021). According to the previous definitions the unsupervised version of
SimCSE minimizes the contrastive loss

$\Delta
L_{\textrm{SimCSE}}=-\log\frac{e^{{\textrm{sim}}({\bf{z}}^{(A)}_{n},{\bf{z}}^{(B)}_{n})/\tau}}{\sum_{m}e^{{\textrm{sim}}({\bf{z}}^{(A)}_{n},{\bf{z}}^{(B)}_{m})/\tau}}$
(1)

summed over the batch $n=1,\ldots,N$. sim is a similarity function, in this
case the standard cosine similarity. Unsupervised SimCSE uses different
dropout masks applied to the same input data point to obtain the two views of
the same sample.

Barlow Twins $-$ BT Zbontar et al. (2021) is one of the two dimension
contrastive methods we consider. Each batch contributes to the loss by an
amount

$\Delta L_{BT}=\sum_{i}(1-\rho_{ii})^{2}+\lambda_{\textrm{BT}}\sum_{j\neq
i}\rho_{ij}^{2}$ (2)

where $\rho_{ij}$ is the Pearson correlation between the $i$-th and $j$-th
entry of the embeddings of ${\bf{z}}^{(A)}$ and ${\bf{z}}^{(B)}$. The first
term in Eq. 2 enforces that the embedding of the two views A and B are
perfectly correlated; the second term on the other hand regularizes the first
and requires different embedding components to be uncorrelated and ideally to
encode different information about the data.

VICReg $-$ The second example of dimension contrastive technique that we
examine is VICReg Bardes et al. (2022). In this case the loss function
combines three terms

$\displaystyle
L_{\textrm{VICReg}}=\frac{\lambda_{I}}{N}\sum_{n}||{\bf{z}}_{n}^{(A)}-{\bf{z}}_{n}^{(B)}||^{2}+$
(3)
$\displaystyle\frac{\lambda_{V}}{D}\sum_{i,I}H\left(\sqrt{C^{(I)}_{ii}+\epsilon}\right)+\frac{\lambda_{C}}{D}\sum_{i\neq
j,I}C^{(I)}_{ij}{}^{2}$

where $I=A,B$, and $H=\max(0,1-x)$. The $D\times D$ matrix $C$ in Eq. 3 is the
covariance matrix for the component of the ${\bf{z}}^{(A,B)}$ vectors
estimated within a batch. Similarly to BT the first term in the loss drives
two views of the same data point to be represented by the same vector while
the other two terms are introduced to prevent embeddings’ collapse. The last
term in Eq. 3 has similarities with the regularization criteria used by BT,
and it tries to de-correlate different components of the vectors
${\bf{z}}^{(A,B)}$; the second term is a hinge loss that encourages the
variance of each of the components of the same vectors to be of order 1.

There is extensive work trying to understand the representation learned by
contrastive (Wang and Isola (2020) _inter alia_) and non-contrastive methods
(Balestriero and LeCun (2022); Garrido et al. (2022); Shwartz-Ziv et al.
(2022) _inter alia_) and the reason of their success. Among these works we
wish to point out Garrido et al. (2022) in which the similarities between
sample-contrastive and dimension-contrastive objectives are extensively
discussed and the different performances of the two classes of methods, albeit
in the vision domain, are attributed to architectural and hyperparameter
choices. Ultimately which of these methods work better in the text modality is
an empirical question and attempting to answer this question is the main goal
of this paper.

##  3 Methods

In order to compare with Gao et al. (2021), we use the same Wikipedia
dataset333The dataset can be downloaded at this link. they used to train the
unsupervised models.

For our supervised experiments we try two datasets. The first, used also by
Gao et al. (2021), is the set of entailment pairs from SNLI Bowman et al.
(2015b) and MNLI Williams et al. (2018b). Only the positive pairs are used, as
hard negatives cannot be easily incorporated in our objectives. The other is
WikiAuto Jiang et al. (2020), a set of sentences from English Wikipedia
aligned to their simplified English Wikipedia version.

We consider two base models for our experiments, BERT-base and RoBERTa-base.
In each case the embedding $E_{\theta}$ that we use for downstream tasks is
the embedding of the [CLS] token. The projector $P_{\theta}$ for SimCSE is a
linear layer with the same dimension as the transformer dimension, followed by
$\tanh$ activation. For BT and VICReg we follow Bardes et al. (2022) and use
two linear layers with batch normalization and ReLU activation, followed by an
additional linear layer all of dimension 8192. Larger dimensions give similar
results and smaller ones progressively degrade performances.

The SimCSE models are trained with a temperature $\tau=0.05$, and a learning
rate of $3\times 10^{-5}$ for BERT and $10^{-5}$ for RoBERTa, which were
identified with an hyperparameter sweep.

We experiment with three basic types of augmentations for BT and VICReg.
Dropout: as in Gao et al. (2021) we apply different dropout masks to each view
of the same data point; this augmentation is parametrized by the dropout
probability $p_{\textrm{do}}=\\{0.05,0.1,0.2\\}$. Shuffling: for both branches
we select a fraction $p_{\textrm{s}}=\\{0.05,0.1,0.2,0.3,0.5\\}$ of the input
tokens and apply a random permutation. EDA Wei and Zou (2019): we apply EDA to
each branch with the same parameter $\alpha=\\{0.1,0.2\\}$ for synonym
replacement, random insertions, random swaps, and random deletions. For each
augmentation we perform a hyperparameter scan to select the best value of the
remaining parameters (learning rate and the loss coefficients in Eqs. 2 and
3). We measure the Spearman’s rank correlation on the STS-B Cer et al. (2017)
validation set to select the best checkpoints as in Gao et al. (2021).

Results are shown in Tab. 1. Across models and loss functions, smaller
$p_{\textrm{do}}$ and larger $p_{\textrm{shuffle}}$ values are preferred, and
the effect is more pronounced with BT. EDA underperforms in all cases. For
more details about the scans we refer to Appendix A.

##  4 Results

We evaluate the embedding on a variety of downstream tasks using the Massive
Text Embedding Benchmark (MTEB) Muennighoff et al. (2023) and report both
average performances on the test set and a breakdown by task category in Table
6444We refer to Appendix C for a summary of the task contained in the
benchmark and a complete breakdown of the scores by task..

Method | Class. | Clust. | PairClass. | Rerank. | Retr. | STS | Summ. | Avg. | $\ell_{\textrm{align}}$ | $\ell_{\textrm{unif}}$
---|---|---|---|---|---|---|---|---|---|---
BERT
avg. | 61.7 | 30.1 | 56.3 | 43.4 | 10.6 | 54.4 | 29.8 | 38.3 | 0.20 | -1.62
SimCSE | 63.7 | 30.5 | 73.1 | 47.0 | 21.5 | 74.8 | 31.2 | 46.6 | 0.21 | -2.62
VICReg ($p_{\textrm{do}}=0.05$) | 62.9 | 33.0 | 61.8 | 46.0 | 17.4 | 67.8 | 29.3 | 43.9 | 0.16 | -2.22
VICReg ($p_{\textrm{shuffle}}=0.5$) | 59.0 | 33.3 | 63.8 | 46.1 | 19.3 | 67.7 | 29.8 | 43.7 | 0.20 | -2.67
Barlow Twins ($p_{\textrm{do}}=0.05$) | 63.7 | 29.9 | 69.4 | 46.3 | 18.7 | 70.0 | 30.1 | 44.6 | 0.24 | -2.88
Barlow Twins ($p_{\textrm{shuffle}}=0.5$) | 59.1 | 27.9 | 73.4 | 45.7 | 16.6 | 70.6 | 29.0 | 42.9 | 0.34 | -3.08
RoBERTa
avg. | 60.0 | 21.6 | 54.1 | 40.2 | 5.8 | 53.8 | 29.6 | 34.6 | 0.01 | -0.16
SimCSE | 64.6 | 30.8 | 74.5 | 47.3 | 23.6 | 74.4 | 27.7 | 47.4 | 0.20 | -2.59
VICReg ($p_{\textrm{do}}=0.2$) | 61.3 | 33.4 | 68.2 | 46.1 | 19.9 | 70.5 | 28.7 | 45.1 | 0.06 | -0.86
VICReg ($p_{\textrm{shuffle}}=0.5$) | 63.0 | 32.4 | 70.7 | 47.3 | 20.7 | 70.6 | 29.2 | 45.7 | 0.03 | -0.44
Barlow Twins ($p_{\textrm{do}}=0.1$) | 65.2 | 33.9 | 70.6 | 47.3 | 24.1 | 71.1 | 28.9 | 47.5 | 0.05 | -0.70
Barlow Twins ($p_{\textrm{shuffle}}=0.5$) | 59.4 | 28.1 | 73.1 | 45.3 | 21.5 | 72.2 | 27.6 | 44.5 | 0.07 | -0.72
Barlow Twins (NLI) | 60.3 | 36.8 | 71.2 | 47.6 | 25.1 | 70.0 | 27.5 | 47.1 | 0.01 | -0.12
Barlow Twins (WikiAuto) | 58.1 | 33.5 | 67.7 | 45.6 | 25.9 | 70.6 | 31.1 | 46.0 | 0.01 | -0.11
Table 2: MTEB test performances aggregated by task category for (Ro)BERT(a):
average of last layers, SimCSE666SimCSE scores differ from those reported in
Muennighoff et al. (2023) because we evaluate unsupervised models without
projector consistently with what done in Gao et al. (2021). and our best
hypertuned models from Tab. 1. We display the performances of the best models
for both dropout and shuffle augmentations with overall best scores in bold.
We also include results from best RoBERTa Barlow Twins models trained on
alternative datasets underlying best scores. Alignment and uniformity are also
shown.

While BERT scores trail behind SimCSE by a few percent points for both BT and
VICReg for the majority of tasks, RoBERTa with BT and dropout outperforms
SimCSE with two notable exceptions: pair classification and STS. For pair
classification we notice that embeddings trained using shuffle augmentation
outperform those trained with dropout irrespectively of model architecture or
objective. The STS results seem to indicate some degree of overfitting to the
STS-B dev set. This seems more severe for VICReg for which the dev set
performances in Tab. 1 are above BT.

Evaluating on STS tasks is a common practice which we also follow to select
model checkpoints. However, this has been criticized due to the lack of
correlation between STS performances and downstream task performances Reimers
et al. (2016); Wang et al. (2021); Abe et al. (2022). Finally we notice that
models trained on supervised datasets can outperform unsupervised methods on
certain downstream tasks, but there is no clear winner. This aligns with the
finding of Muennighoff et al. (2023) in which single model performance on
different tasks varies a lot with no single model winning across all tasks.

We also report _alignment_ and _uniformity_ , two metrics which are commonly
considered when analyzing sample contrastive embedding techniques: the
standard sample contrastive objective optimizes them in the limit of
infinitely many negative samples (Wang and Isola, 2020). They are shown to
empirically correlate to the embedding performances on downstream tasks, but
an understanding of why uniformity is needed is lacking. Huang et al. (2023)
derives an upper bound on the error rate for classification tasks based on
three metrics, alignment, _divergence_ , and _concentration_. Intuitively, the
latter two represent how separated the centroids of the various classes are in
the embedding space and how concentrated around such centroid are the
representation of the augmented members of each class. Huang et al. (2023)
show that both the InfoNCE van den Oord et al. (2019) and BT satisfy these
criteria. We refer to Appendix B for further discussions of alignment an
uniformity.

##  5 Conclusions

In this work, we compare sample contrastive (SimCSE) and dimension contrastive
(Barlow Twins, VICReg) training objectives to learn sentence embeddings. Our
results shows how these alternative self-supervision objectives can learn good
representations, performing as well as or better than those obtained from
SimCSE. Dimension contrastive techniques are largely unexplored outside the
computer vision literature and we hope this work could be a step in the
direction of popularizing them in the NLP community.

## Limitations

The goal of this short paper is to make the point that dimension contrastive
objectives are a viable alternative to standard sample contrastive
techiniques.

While we used SimCSE as our baseline, it would be interesting to use sample
contrastive loss functions on methods like DiffCSE Chuang et al. (2022),
InfoCSE Wu et al. (2022) and PromptBERT Jiang et al. (2022) and see whether
the same improvement in performance obtained using the standard contrastive
loss function would apply to BT or VICReg.

It would be interesting to study different model architectures like decoder-
only models Muennighoff (2022) or encoder-decoder ones Ni et al. (2022a).

Additionally, while our study is limited to sentence embeddings for English
documents, the methods are applicable to multilingual corpora and it would be
worth exploring them in this context.

## References

  * Abe et al. (2022) Kaori Abe, Sho Yokoi, Tomoyuki Kajiwara, and Kentaro Inui. 2022.  Why is sentence similarity benchmark not predictive of application-oriented task performance? In _Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems_ , pages 70–87, Online. Association for Computational Linguistics.
  * Agirre et al. (2014) Eneko Agirre, Carmen Banea, Claire Cardie, Daniel M Cer, Mona T Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014\.  Semeval-2014 task 10: Multilingual semantic textual similarity.  In _SemEval@ COLING_ , pages 81–91.
  * Agirre et al. (2012) Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012.  Semeval-2012 task 6: A pilot on semantic textual similarity.  In _* SEM 2012: The First Joint Conference on Lexical and Computational Semantics–Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)_ , pages 385–393.
  * Agirre et al. (2013) Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013\.  * sem 2013 shared task: Semantic textual similarity.  In _Second joint conference on lexical and computational semantics (* SEM), volume 1: proceedings of the Main conference and the shared task: semantic textual similarity_ , pages 32–43.
  * Balestriero and LeCun (2022) Randall Balestriero and Yann LeCun. 2022.  Contrastive and non-contrastive self-supervised learning recover global and local spectral embedding methods.  In _Advances in Neural Information Processing Systems_ , volume 35, pages 26671–26685. Curran Associates, Inc.
  * Bardes et al. (2022) Adrien Bardes, Jean Ponce, and Yann LeCun. 2022.  Vicreg: Variance-invariance-covariance regularization for self-supervised learning.
  * Bowman et al. (2015a) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015a.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Bowman et al. (2015b) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015b.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Casanueva et al. (2020) Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić. 2020.  Efficient intent detection with dual sentence encoders.
  * Cer et al. (2017) Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017\.  SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation.  In _Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)_. Association for Computational Linguistics.
  * Chopra et al. (2005) S. Chopra, R. Hadsell, and Y. LeCun. 2005.  Learning a similarity metric discriminatively, with application to face verification.  In _2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)_ , volume 1, pages 539–546 vol. 1.
  * Chuang et al. (2022) Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljacic, Shang-Wen Li, Scott Yih, Yoon Kim, and James Glass. 2022.  DiffCSE: Difference-based contrastive learning for sentence embeddings.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4207–4218, Seattle, United States. Association for Computational Linguistics.
  * Cohan et al. (2020) Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel S. Weld. 2020.  Specter: Document-level representation learning using citation-informed transformers.
  * Fabbri et al. (2020) Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2020.  Summeval: Re-evaluating summarization evaluation.
  * Feng et al. (2021) Steven Y. Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. 2021.  A survey of data augmentation approaches for NLP.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 968–988, Online. Association for Computational Linguistics.
  * FitzGerald et al. (2022) Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh, Swetha Ranganath, Laurie Crist, Misha Britan, Wouter Leeuwis, Gokhan Tur, and Prem Natarajan. 2022.  Massive: A 1m-example multilingual natural language understanding dataset with 51 typologically-diverse languages.
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  SimCSE: Simple contrastive learning of sentence embeddings.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Garrido et al. (2022) Quentin Garrido, Yubei Chen, Adrien Bardes, Laurent Najman, and Yann Lecun. 2022\.  On the duality between contrastive and non-contrastive self-supervised learning.
  * Geigle et al. (2021) Gregor Geigle, Nils Reimers, Andreas Rücklé, and Iryna Gurevych. 2021.  Tweac: Transformer with extendable qa agent classifiers.
  * Giorgi et al. (2021) John Giorgi, Osvald Nitski, Bo Wang, and Gary Bader. 2021.  DeCLUTR: Deep contrastive learning for unsupervised textual representations.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 879–895, Online. Association for Computational Linguistics.
  * Huang et al. (2023) Weiran Huang, Mingyang Yi, Xuyang Zhao, and Zihao Jiang. 2023.  Towards the generalization of contrastive self-supervised learning.
  * Jiang et al. (2020) Chao Jiang, Mounica Maddela, Wuwei Lan, Yang Zhong, and Wei Xu. 2020.  Neural CRF model for sentence alignment in text simplification.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7943–7960, Online. Association for Computational Linguistics.
  * Jiang et al. (2022) Ting Jiang, Jian Jiao, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haizhen Huang, Denvy Deng, and Qi Zhang. 2022.  PromptBERT: Improving BERT sentence embeddings with prompts.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 8826–8837, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Klein and Nabi (2022) Tassilo Klein and Moin Nabi. 2022.  SCD: Self-contrastive decorrelation of sentence embeddings.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 394–400, Dublin, Ireland. Association for Computational Linguistics.
  * Lan et al. (2017) Wuwei Lan, Siyu Qiu, Hua He, and Wei Xu. 2017.  A continuously growing dataset of sentential paraphrases.  In _Proceedings of The 2017 Conference on Empirical Methods on Natural Language Processing (EMNLP)_ , pages 1235–1245. Association for Computational Linguistics.
  * Li et al. (2021) Haoran Li, Abhinav Arora, Shuohui Chen, Anchit Gupta, Sonal Gupta, and Yashar Mehdad. 2021.  MTOP: A comprehensive multilingual task-oriented semantic parsing benchmark.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 2950–2962, Online. Association for Computational Linguistics.
  * Liu et al. (2018) Xueqing Liu, Chi Wang, Yue Leng, and ChengXiang Zhai. 2018.  Linkso: a dataset for learning to retrieve similar question answer pairs on software development forums.  In _Proceedings of the 4th ACM SIGSOFT International Workshop on NLP for Software Engineering_ , pages 2–5.
  * Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011.  Learning word vectors for sentiment analysis.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 142–150, Portland, Oregon, USA. Association for Computational Linguistics.
  * McAuley and Leskovec (2013) Julian McAuley and Jure Leskovec. 2013.  Hidden factors and hidden topics: Understanding rating dimensions with review text.  In _Proceedings of the 7th ACM Conference on Recommender Systems_ , RecSys ’13, page 165–172, New York, NY, USA. Association for Computing Machinery.
  * Muennighoff (2022) Niklas Muennighoff. 2022.  Sgpt: Gpt sentence embeddings for semantic search.
  * Muennighoff et al. (2023) Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023.  MTEB: Massive text embedding benchmark.  In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_ , pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistics.
  * Ni et al. (2022a) Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022a.  Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 1864–1874, Dublin, Ireland. Association for Computational Linguistics.
  * Ni et al. (2022b) Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, and Yinfei Yang. 2022b.  Large dual encoders are generalizable retrievers.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 9844–9855, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * O’Neill et al. (2021) James O’Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota, and Danushka Bollegala. 2021.  I wish i would have loved this one, but i didn’t – a multilingual dataset for counterfactual detection in product reviews.
  * Reimers et al. (2016) Nils Reimers, Philip Beyer, and Iryna Gurevych. 2016.  Task-oriented intrinsic evaluation of semantic textual similarity.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , pages 87–96, Osaka, Japan. The COLING 2016 Organizing Committee.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  * Saravia et al. (2018) Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. 2018\.  CARER: Contextualized affect representations for emotion recognition.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3687–3697, Brussels, Belgium. Association for Computational Linguistics.
  * Shah et al. (2018) Darsh Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, and Preslav Nakov. 2018\.  Adversarial domain adaptation for duplicate question detection.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1056–1063, Brussels, Belgium. Association for Computational Linguistics.
  * Shwartz-Ziv et al. (2022) Ravid Shwartz-Ziv, Randall Balestriero, and Yann LeCun. 2022.  What do we maximize in self-supervised learning?
  * Su et al. (2022) Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2022.  One embedder, any task: Instruction-finetuned text embeddings.
  * Thakur et al. (2021) Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021.  Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models.
  * van den Oord et al. (2019) Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2019.  Representation learning with contrastive predictive coding.
  * Wang et al. (2021) Kexin Wang, Nils Reimers, and Iryna Gurevych. 2021.  TSDAE: Using transformer-based sequential denoising auto-encoderfor unsupervised sentence embedding learning.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 671–688, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Wang et al. (2022) Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022.  Text embeddings by weakly-supervised contrastive pre-training.
  * Wang and Isola (2020) Tongzhou Wang and Phillip Isola. 2020.  Understanding contrastive representation learning through alignment and uniformity on the hypersphere.  In _Proceedings of the 37th International Conference on Machine Learning_ , ICML’20. JMLR.org.
  * Wei and Zou (2019) Jason Wei and Kai Zou. 2019.  EDA: Easy data augmentation techniques for boosting performance on text classification tasks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 6382–6388, Hong Kong, China. Association for Computational Linguistics.
  * Williams et al. (2018a) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018a.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Williams et al. (2018b) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018b.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Wu et al. (2020) Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, et al. 2020.  Mind: A large-scale dataset for news recommendation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 3597–3606.
  * Wu et al. (2022) Xing Wu, Chaochen Gao, Zijia Lin, Jizhong Han, Zhongyuan Wang, and Songlin Hu. 2022\.  InfoCSE: Information-aggregated contrastive learning of sentence embeddings.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 3060–3070, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Xu et al. (2023) Jiahao Xu, Wei Shao, Lihui Chen, and Lemao Liu. 2023.  Imsimcse: Improving contrastive learning for sentence embeddings from two perspectives.
  * Xu et al. (2015) Wei Xu, Chris Callison-Burch, and William B Dolan. 2015.  Semeval-2015 task 1: Paraphrase and semantic similarity in twitter (pit).  In _Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015)_ , pages 1–11.
  * Zbontar et al. (2021) Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. 2021.  Barlow twins: Self-supervised learning via redundancy reduction.
  * Zhang et al. (2020) Yan Zhang, Ruidan He, Zuozhu Liu, Kwan Hui Lim, and Lidong Bing. 2020.  An unsupervised sentence embedding method by mutual information maximization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1601–1610, Online. Association for Computational Linguistics.

##  Appendix A Hyperparameters

In the hyperparameter search the model architectures are fixed both in terms
of the base models (BERT and RoBERTa) and in terms of the projectors that are
used (see Sec. 3). We furthermore fix the batch size to 256 as we did not
observe significant gains with larger batches.

All models are trained for 2 epochs. We evaluate every 60 steps and the final
metric we use for checkpoint selection is the Spearman’s correlation on the
STS-B dev set.

###  A.1 Barlow Twins

For BT we use a grid scan to explore hyperparameters and data augmentations.
We use the values reported in Table 3 for both BERT and RoBERTa models.
Augmentations are not combined, but for each augmentation we scan learning
rate and the loss coefficient ($\lambda_{\textrm{BT}}$).

Parameter | Domain
---|---
learning rate | $\\{1,2,5\\}\times 10^{-5}$
dropout | $\\{0.05,0.1,0.2\\}$
shuffle | $\\{0.5,1,2,3,5\\}\times 10^{-1}$
EDA | $\\{0.1,0.2\\}$
Barlow Twins
$\lambda_{\textrm{BT}}$ | $\\{0.5,1,2.5,5,7.5,10,25\\}\times 10^{-3}$
Barlow Twins
$\log_{10}\lambda_{\textrm{C}}$ | $[-3,-1]$
$\log_{10}\lambda_{\textrm{V}}$ | $[1,4]$
shuffle | $\\{0.5,1,2,3,5\\}\times 10^{-1}$
Table 3:  BERT and RoBERTa values for the BT and VICReg hyperparameter scan.
The scan over $\lambda_{\textrm{C, V}}$ is uniform in log space. For VICReg we
only use $\alpha=0.1$ for EDA augmentation.

We find the performances to be quite insensitive to the choice of the learning
rate, but quite sensitive to $\lambda_{\textrm{BT}}$ for both model
architectures. This is shown in Fig. 1. We thus constrain
$\lambda_{\textrm{BT}}\leq 0.05$ for BERT and $\leq 0.025$ for RoBERTa. We
show the development set performances as a function of the augmentation in
Tab. 1.

Figure 1:  STS-B performances as a function of the $\lambda_{\textrm{BT}}$
coefficient. We show both the max and the upper quartile of the metric
distribution after binning by the value of the parameter.  Figure 2:  STS-B
performances as a function of $\lambda_{\textrm{C}}$ and
$\lambda_{\textrm{V}}$. We show both the max and the upper quartile of the
metric distribution after binning by the value of the parameter.

###  A.2 VICReg

The parameter space of VICReg is larger than the one of BT: the loss function
depends on 3 parameters $\lambda_{\textrm{V},\textrm{I},\textrm{C}}$. We fix
$\lambda_{\textrm{I}}=1$ and scan the remaining two parameters. Since the
parameter is larger we use SMAC instead of grid search. Table 3 report the
parameters of the scan. Similarly to BT augmentations are not combined, but
for each augmentation we scan learning rate, $\lambda_{\textrm{V}}$, and
$\lambda_{\textrm{C}}$. For each augmentation strategy we run a total of 50
jobs.

Similarly to BT, there is little sensitivity to the learning rate. We find
that the scan favors small values of $\lambda_{\textrm{C}}$ and large values
of $\lambda_{\textrm{V}}$. The dev set performances as a function of the
augmentation are shown in Tab. 1.

##  Appendix B Alignment and uniformity

Figure 3:  Alignment and uniformity numbers for the models reported in Tab. 6.
[CLS]-(Ro)BERT(a) represent text embedding models obtained by using the last
layer [CLS] token as the embedding. Lower values are better for both metrics.

We calculate the alignment and uniformity metrics Wang and Isola (2020) for
the unsupervised models shown in Tab. 6. Optimizing the unsupervised
objective, either sample or dimension contrastive, improve uniformity in all
cases while it typically degrades alignment. We notice that these effects are
particularly pronounced for the sample contrastive objective optimized by
SimCSE, in particular in terms of the improvement in uniformity.

(a) RoBERTa [CLS] (b) SimCSE RoBERTa (c) Barlow Twins dropout RoBERTa (d)
VICReg shuffle RoBERTa

Figure 4: Histograms of cosine similarity between pairs of sentences from the
STS-B test set computed with different RoBERTa models, vertically divided in
groups according to human ratings of similarity. Notice the different scale of
the horizontal axis.

For both BT and VICReg, and in particular for RoBERTa, uniformity improves
only marginally through training. However this does not seem to hurt
performances on downstream tasks as shown in Tab. 6. This is consistent with
the discussion of Huang et al. (2023).

Another representation of this fact is Fig. 4 which shows the distribution of
cosine similarities of sentence pairs on the STS-B test set stratified by the
similarity rating assigned by human annotators. We see that both SimCSE, BT,
and VICReg training increase the divergence of the distributions across
buckets, but SimCSE tends, on average, to achieve that by spreading the
embeddings apart on the hypersphere (notice the different horizontal scale of
the 3 bottom panels in Fig. 4)

##  Appendix C MTEB

The MTEB (Massive Text Embedding Benchmark) Muennighoff et al. (2023) is a
comprehensive evaluation tool designed to assess the performance of text
embedding models. It includes well established benchmarks, and spans a wide
range of tasks and domains.

We report results on the 56 English language datasets. They are divided in the
following tasks (associated evaluation metrics in parenthesis): Classification
(accuracy), Clustering (v-measure), Pair Classification (average precision),
Rerank (MAP), Retrieval (nDCG@10), STS (Spearman correlation), and
Summarization (Spearman correlation). A breadkdown of all datasets, compiled
with results from our RoBERTa models, is shown in Tab. 4.

Dataset | SimCSE | VICReg | VICReg | Barlow Twins | Barlow Twins | Barlow Twins | Barlow Twins
---|---|---|---|---|---|---|---
|  | (dropout) | (shuffle) | (dropout) | (shuffle) | (NLI) | (WikiAuto)
Class.
AmazonCounterfactualClassification O’Neill et al. (2021) | 65.5 | 64.2 | 65.2 | 65.0 | 64.1 | 60.9 | 60.5
AmazonPolarityClassification McAuley and Leskovec (2013) | 76.6 | 63.3 | 64.6 | 72.9 | 62.9 | 62.7 | 62.1
AmazonReviewsClassification McAuley and Leskovec (2013) | 35.0 | 29.0 | 29.8 | 33.1 | 28.7 | 28.8 | 30.4
Banking77Classification Casanueva et al. (2020) | 78.1 | 77.3 | 76.9 | 77.9 | 76.1 | 75.6 | 67.6
EmotionClassification Saravia et al. (2018) | 46.8 | 42.9 | 44.3 | 44.5 | 46.0 | 42.7 | 40.5
ImdbClassification Maas et al. (2011) | 73.5 | 64.9 | 65.0 | 72.0 | 62.4 | 63.0 | 57.4
MassiveIntentClassification FitzGerald et al. (2022) | 61.5 | 61.1 | 64.7 | 64.8 | 57.6 | 60.5 | 58.8
MassiveScenarioClassification FitzGerald et al. (2022) | 69.4 | 70.0 | 73.6 | 73.7 | 62.0 | 70.9 | 69.5
MTOPDomainClassification Li et al. (2021) | 85.1 | 85.9 | 88.1 | 88.0 | 80.9 | 84.4 | 81.4
MTOPIntentClassification Li et al. (2021) | 61.3 | 59.8 | 64.8 | 68.3 | 59.0 | 56.0 | 51.0
ToxicConversationsClassification (url) | 68.6 | 66.4 | 66.8 | 69.9 | 64.2 | 66.3 | 66.5
TweetSentimentExtractionClassification (url) | 54.0 | 50.4 | 51.8 | 52.4 | 48.9 | 51.3 | 51.6
Clust.
ArxivClusteringP2P♢ | 32.9 | 34.9 | 33.7 | 35.2 | 33.1 | 38.6 | 33.5
ArxivClusteringS2S♢ | 21.4 | 21.8 | 23.5 | 23.0 | 17.9 | 25.8 | 23.6
BiorxivClusteringP2P♢ | 30.1 | 31.5 | 30.4 | 31.7 | 30.8 | 36.0 | 30.0
BiorxivClusteringS2S♢ | 22.1 | 22.9 | 24.6 | 23.9 | 16.1 | 26.1 | 22.0
MedrxivClusteringP2P♢ | 26.9 | 29.0 | 27.4 | 28.5 | 28.8 | 31.2 | 28.0
MedrxivClusteringS2S♢ | 24.9 | 25.4 | 26.0 | 26.0 | 21.3 | 28.3 | 25.6
RedditClustering Geigle et al. (2021) | 33.9 | 40.1 | 35.0 | 41.2 | 28.7 | 47.0 | 41.7
RedditClusteringP2P♢ | 47.2 | 48.8 | 43.1 | 50.4 | 46.3 | 52.5 | 46.9
StackExchangeClustering Geigle et al. (2021) | 46.3 | 48.2 | 49.3 | 50.9 | 38.0 | 51.9 | 49.1
StackExchangeClusteringP2P♢ | 29.5 | 30.7 | 30.0 | 30.0 | 28.5 | 30.5 | 33.1
TwentyNewsgroupsClustering (url) | 23.8 | 33.5 | 33.1 | 31.9 | 19.4 | 37.2 | 34.8
PairClass.
SprintDuplicateQuestions Shah et al. (2018) | 86.4 | 70.7 | 77.1 | 74.1 | 88.5 | 84.2 | 84.2
TwitterSemEval2015 Xu et al. (2015) | 56.8 | 56.3 | 56.3 | 59.1 | 51.8 | 51.3 | 43.6
TwitterURLCorpus Lan et al. (2017) | 80.4 | 77.6 | 78.8 | 78.8 | 78.9 | 78.3 | 75.4
Rerank.
AskUbuntuDupQuestions (url) | 53.3 | 51.7 | 51.9 | 52.5 | 51.9 | 52.2 | 50.4
MindSmallReranking Wu et al. (2020) | 29.4 | 29.2 | 30.3 | 29.6 | 27.9 | 30.0 | 31.1
SciDocsRR Cohan et al. (2020) | 66.9 | 65.5 | 68.7 | 67.5 | 62.0 | 69.7 | 66.0
StackOverflowDupQuestions Liu et al. (2018) | 39.8 | 38.1 | 38.2 | 39.6 | 39.5 | 38.4 | 34.8
Retr.♠
ArguAna | 34.7 | 43.8 | 42.6 | 43.9 | 35.6 | 44.1 | 40.6
ClimateFEVER | 14.5 | 12.8 | 13.0 | 19.2 | 14.2 | 18.2 | 22.0
CQADupstackRetrieval | 20.4 | 13.9 | 17 | 20.0 | 18.7 | 19.4 | 18.3
DBPedia | 15.7 | 12.0 | 13.2 | 15.2 | 12.8 | 17.6 | 17.2
FEVER | 28.4 | 12.6 | 15.9 | 28.4 | 17.1 | 25.2 | 33.7
FiQA2018 | 12.6 | 11.6 | 11.3 | 14.4 | 10.3 | 16.1 | 11.3
HotpotQA | 31.4 | 16.5 | 16.8 | 25.0 | 29.7 | 26.7 | 36.2
MSMARCO | 8.8 | 5.4 | 6.1 | 7.8 | 7.8 | 8.6 | 12.6
NFCorpus | 14.3 | 9.1 | 10.6 | 11.7 | 10.1 | 15.6 | 18.7
NQ | 12.3 | 7.3 | 8.9 | 13.6 | 9.0 | 12.3 | 15.4
QuoraRetrieval | 80.4 | 78.5 | 79.5 | 79.6 | 78.3 | 78.2 | 75.0
SCIDOCS | 6.9 | 5.7 | 6.6 | 7.4 | 7.2 | 10.5 | 9.5
SciFact | 34.1 | 27.3 | 24.3 | 25.6 | 34.7 | 35.2 | 34.5
Touche2020 | 10.9 | 10.4 | 9.7 | 11.9 | 10.6 | 13.1 | 10.5
TRECCOVID | 28 | 30.9 | 35.1 | 38.0 | 26.1 | 36.7 | 33.6
STS
BIOSSES (url) | 67.7 | 51.1 | 56.9 | 56.9 | 69.5 | 58.8 | 68.6
SICK-R Agirre et al. (2014) | 68.9 | 67.9 | 70.1 | 70.6 | 64.8 | 64.3 | 67.4
STS12♡ | 70.2 | 64.2 | 63.2 | 62.5 | 65.4 | 66.5 | 66.3
STS13♡ | 81.8 | 78.7 | 77.3 | 77.6 | 77.7 | 77.3 | 77.2
STS14♡ | 73.2 | 68.1 | 66.6 | 68.1 | 70.5 | 67.7 | 67.4
STS15♡ | 81.4 | 78.5 | 76.3 | 76.2 | 80.4 | 75.3 | 74.1
STS16♡ | 80.7 | 77.5 | 77.4 | 79.3 | 76.0 | 75.0 | 74.7
STS17♡ | 81.8 | 81.2 | 81.6 | 82.0 | 80.8 | 78.2 | 79.8
STS22♡ | 57.7 | 60.2 | 59.8 | 61.0 | 60.8 | 61.9 | 55.5
STSBenchmark♡ | 80.1 | 78.0 | 76.9 | 76.6 | 75.6 | 75.1 | 75.2
Summ.
SummEval Fabbri et al. (2020) | 27.6 | 28.7 | 29.2 | 28.9 | 27.6 | 27.5 | 31.1
Table 4: MTEB performances of RoBERTa models on all English datasets grouped
by task. We display the scores for both dropout and shuffle augmentations with
overall best scores in bold. We also include scores from best Barlow Twins
models trained on alternative datasets underlying best scores. $\diamondsuit$:
custom clustering datasets created for MTEB, for details we refer to
Muennighoff et al. (2023). $\spadesuit$: retrieval datasets are a subset of
the BEIR benchmark Thakur et al. (2021). $\heartsuit$: tasks from the original
STS benchmark Agirre et al. (2012, 2013).


# On monogenity of certain pure number fields defined by $x^{2^{u}\cdot
3^{v}\cdot 5^{t}}-m$

Lhoussain El Fadil  Faculty of Sciences Dhar El Mahraz, P.O. Box 1796 Atlas-
Fez , Sidi Mohamed Ben Abdellah University, Fez– Morocco
<EMAIL_ADDRESS>
###### Abstract.

Let $K=\mathbb{Q}(\alpha)$ be a pure number field generated by a root $\alpha$
of a monic irreducible polynomial $F(x)=x^{2^{u}\cdot 3^{v}\cdot 5^{t}}-m$,
with $m\neq\pm 1$ a square free rational integer, $u$, $v$ and $t$ three
positive integers. In this paper, we study the monogenity of $K$. We prove
that if $m\not\equiv 1\ \mbox{\rm(mod }{4})$, $m\not\equiv\pm 1\ \mbox{\rm(mod
}{9})$, and $m\not\in\\{\pm 1,\pm 7\\}\ \mbox{\rm(mod }{25})$, then $K$ is
monogenic. But if $m\equiv 1\ \mbox{\rm(mod }{4})$ or $m\equiv 1\
\mbox{\rm(mod }{9})$ or $m\equiv-1\ \mbox{\rm(mod }{9})$ and $u=2k$ for some
odd integer $k$ or $u\geq 2$ and $m\equiv 1\ \mbox{\rm(mod }{25})$ or
$m\equiv-1\ \mbox{\rm(mod }{25})$ and $u=2k$ for some odd integer $k$ or
$u=v=1$ and $m\equiv\pm 82\ \mbox{\rm(mod }{5^{4}})$, then $K$ is not
monogenic.

###### Key words and phrases:

Power integral basis, Theorem of Ore, prime ideal factorization, Newton
polygons

###### 2010 Mathematics Subject Classification:

11R04, 11R16, 11R21

##  1\. Introduction

Let $K=\mathbb{Q}(\alpha)$ be a number field generated by a root $\alpha$ of a
monic irreducible polynomial $F(x)\in\mathbb{Z}[x]$ and $\mathbb{Z}_{K}$ its
ring of integers. It is well know that the ring $\mathbb{Z}_{K}$ is a free
$\mathbb{Z}$-module of rank $n=[K:\mathbb{Q}]$. Thus by the fundamental
theorem of finitely generated Abelian groups, the Abelian group
$\mathbb{Z}_{K}/\mathbb{Z}[\alpha]$ is finite. Its cardinal order is called
the index of $\mathbb{Z}[\alpha]$, and denoted by
$(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])$. The ring $\mathbb{Z}_{K}$ is said to be
monogenic if it has a power integral basis as a $\mathbb{Z}$-module. That is
$(1,\theta,\cdots,\theta^{n-1})$ is a $\mathbb{Z}$-basis of $\mathbb{Z}_{K}$
for some $\theta\in\mathbb{Z}_{K}$. $K$ is said to be not monogenic otherwise.
Monogenity of number fields is a classical problem of algebraic number theory,
going back to Dedekind, Hasse, and Hensel (see for instance [22]). The problem
of testing the monogenity of number fields and the construction of power
integral bases has been intensively studied these last four decades, mainly by
Gaál, Nakahara, Pohst, and their collaborators (see for instance [2, 21, 22,
23, 34]). In [19], Funakura, calculated integral bases and studied monogenity
of pure quartic fields. In [24], Gaál and Remete, calculated the elements of
index $1$ in pure quartic fields generated by $m^{\frac{1}{4}}$ for
$1<m<10^{7}$ and $m\equiv 2,3\ \mbox{\rm(mod }{4})$. In [1], Ahmad, Nakahara,
and Husnine proved that if $m\equiv 2,3\ \mbox{\rm(mod }{4})$ and
$m\not\equiv\pm 1\ \mbox{\rm(mod }{9})$, then the sextic number field
generated by $m^{\frac{1}{6}}$ is monogenic. They also showed in [2], that if
$m\equiv 1\ \mbox{\rm(mod }{4})$ and $m\not\equiv\pm 1\ \mbox{\rm(mod }{9})$,
then the sextic number field generated by $m^{\frac{1}{6}}$ is not monogenic.
In [8], based on prime ideal factorization, El Fadil showed that if $m\equiv
1\ \mbox{\rm(mod }{4})$ or $m\equiv 1\ \mbox{\rm(mod }{9})$, then the sextic
number field generated by $m^{\frac{1}{6}}$ is not monogenic. Hameed and
Nakahara proved that if $m\equiv 1\ \mbox{\rm(mod }{4})$, then the octic
number field generated by $m^{1/8}$ is not monogenic, but if $m\equiv 2,3\
\mbox{\rm(mod }{4})$, then it is monogenic ([27]). In [25], by applying the
explicit form of the index equation, Gaál and Remete obtained deep new results
on monogenity of number fields generated by $m^{\frac{1}{n}}$, with $3\leq
n\leq 9$ and $m\neq\pm 1$ a square free integer. In [3, 4, 18, 8, 9, 10, 11,
12, 13], based on Newton polygon’s techniques, El Fadil et al. studied the
monogenity of some pure number fields. Also El Fadil, Chouli, and Kchit
studied the monogenity of pure number fields defined by
$x^{60}-m\in\mathbb{Z}[x]$ with $m$ a square free integer. The goal of this
paper is to study the monogenity of pure number fields defined by
$x^{2^{u}\cdot 3^{v}\cdot 5^{t}}-m$, where $m\neq\pm 1$ is a square free
integer, $u$, $v$, and $t$ are three natural integers. The cases $uvt=0$ have
been studied in [3, 28, 4, 18]. Also the case $u=2$ and $t=v=1$ has been
studied by in [15]. Our proofs are based on Newton’s polygon techniques and on
index divisors as introduced by Hensel as follows: The index of a field $K$ is
defined by $i(K)=gcd\\{(\mathbb{Z}_{K}:\mathbb{Z}[\theta])\mid
K=\mathbb{Q}(\theta)\mbox{ and }\theta\in\mathbb{Z}_{K}\\}$. A rational prime
$p$ dividing $i(K)$ is called a prime common index divisor of $K$. If
$\mathbb{Z}_{K}$ has a power integral basis, then $i(K)=1$. Therefore a field
having a prime common index divisor is not monogenic.

##  2\. Main results

Let $K$ be a number field generated by a complex root $\alpha$ of a monic
irreducible polynomial $F(x)=x^{2^{u}\cdot 3^{v}\cdot 5^{t}}-m$, with
$m\neq\pm 1$ a square free rational integer, $u$, $v$, and $t$ three positive
integers.

######  Theorem 2.1.

The ring $\mathbb{Z}[\alpha]$ is the ring of integers of $K$ if and only if
$m\not\equiv 1\ \mbox{\rm(mod }{4})$, $m\not\equiv\pm 1\ \mbox{\rm(mod }{9})$,
and $m\not\in\\{\pm 1,\pm 7\\}\ \mbox{\rm(mod }{25})$.

######  Remark.

If $m\equiv 1\ \mbox{\rm(mod }{4})$, $m\not\equiv\pm 1\ \mbox{\rm(mod }{9})$,
and $m\not\in\\{\pm 1,\pm 7\\}\ \mbox{\rm(mod }{25})$, then
$\mathbb{Z}[\alpha]$ is not integrally closed. But in this case, Theorem 2.1
cannot decide on monogenity of $K$. The following theorems give an answer.

######  Theorem 2.2.

  1. (1)

$m\equiv 1\ \mbox{\rm(mod }{4})$, then $2$ divides $i(K)$.

  2. (2)

$m\equiv 1\ \mbox{\rm(mod }{9})$ or $m\equiv-1\ \mbox{\rm(mod }{9})$ and
$u=2k$ for some odd integer $k$ or $u=1$, $m\equiv-1\ \mbox{\rm(mod }{27})$,
and $v\geq 3$, then $3$ divides $i(K)$.

  3. (3)

$m\equiv\pm 1\ \mbox{\rm(mod }{25})$ and $u=2k$ for some odd integer $k$ or
$u=1$, $m\equiv\pm 1\ \mbox{\rm(mod }{125})$, and $t\geq 2$ or $u=v=1$,
$m\equiv\pm 82\ \mbox{\rm(mod }{5^{4}})$, and $t\geq 3$, then $5$ divides
$i(K)$.

In particular, if one of these following conditions holds, then $K$ is not
monogenic.

######  Corollary 2.3.

Let $a\neq\pm 1$ be a square free rational integer, $u$, $v$ and $t$ three
positive integers, and $s<{2^{u}\cdot 3^{v}\cdot 5^{t}}$ a positive integer,
which is coprime to $30$. Then $F(x)=x^{2^{u}\cdot 3^{v}\cdot 5^{t}}-a^{s}$ is
irreducible over $\mathbb{Q}$. Let $K$ be the number field generated by a
complex root $\alpha$ of the monic irreducible polynomial $F(x)$.

  1. (1)

If $a\not\equiv 1\ \mbox{\rm(mod }{4})$, $m\not\equiv\pm 1\ \mbox{\rm(mod
}{9})$, and $m\not\in\\{\pm 1,\pm 7\\}\ \mbox{\rm(mod }{25})$ then $K$ is
monogenic.

  2. (2)

If one of the following conditions holds

    1. (a)

$a\equiv 1\ \mbox{\rm(mod }{4})$,

    2. (b)

$a\equiv 1\ \mbox{\rm(mod }{9})$,

    3. (c)

$a\equiv-1\ \mbox{\rm(mod }{9})$ and $u=2k$ for some positive odd integer $k$,

    4. (d)

$a\equiv-1\ \mbox{\rm(mod }{27})$ and $v\geq 3$,

    5. (e)

$a\equiv 1\ \mbox{\rm(mod }{25})$ and $u\geq 2$,

    6. (f)

$a\equiv-1\ \mbox{\rm(mod }{25})$ and $u=2k$ for some positive odd integer
$k$,

    7. (g)

$a\equiv\pm 82\ \mbox{\rm(mod }{5^{4}})$, $u=v=1$, and $t\geq 3$,

then $K$ is not monogenic.

##  3\. Preliminaries

Let $K=\mathbb{Q}(\alpha)$ be a number field generated by a root $\alpha$ of a
monic irreducible polynomial $F(x)\in\mathbb{Z}[x]$, $\mathbb{Z}_{K}$ its ring
of integers, and $ind(\alpha)=(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])$ the index
of $\mathbb{Z}[\alpha]$ in $\mathbb{Z}_{K}$. For a rational prime integer $p$,
if $p$ does not divide $(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])$, then a well
known theorem of Dedekind says that the factorization of $p\mathbb{Z}_{K}$ can
be derived directly from the factorization of $\overline{F(x)}$ in
$\mathbb{F}_{p}[x]$. Namely,
$p\mathbb{Z}_{K}=\prod_{i=0}^{r}\mathfrak{p}_{i}^{l_{i}}$, where every
$\mathfrak{p}_{i}=p\mathbb{Z}_{K}+\phi_{i}(\alpha)\mathbb{Z}_{K}$ and
$\overline{F(x)}=\prod_{i=1}^{r}\overline{\phi_{i}(x)}^{l_{i}}$ modulo $p$ is
the factorization of $\overline{F(x)}$ into powers of monic irreducible
coprime polynomials of $\mathbb{F}_{p}[x]$. So,
$f(\mathfrak{p}_{i})={\mbox{deg}}(\phi_{i})$ is the residue degree of
$\mathfrak{p}_{i}$ (see [32, Chapter I, Proposition 8.3]). In order to apply
this theorem in an effective way, one needs a criterion to test whether $p$
divides the index $(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])$. In $1878$, Dedekind
proved the well known Dedekind’s criterion which allows to test if a prime
integer $p$ divides $(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])$ [5, Theorem 6.1.4]
and [6]. When Dedekind’s criterion fails, that is, $p$ divides the index
$(\mathbb{Z}_{K}:\mathbb{Z}[\theta])$ for every primitive element
$\theta\in\mathbb{Z}_{K}$ of $K$, then it is not possible to obtain the prime
ideal factorization of $p\mathbb{Z}_{K}$ by applying Dedekind’s theorem. In
1928, Ore developed an alternative approach for obtaining the index
$(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])$, the absolute discriminant, and the
prime ideal factorization of the rational primes in a number field $K$ by
using Newton polygons (see [31, 33]). For the convenience of the reader, as it
is necessary for the proof of our main results, we refer to our paper [20].
We start by recalling some fundamental facts about Newton polygons applied in
algebraic number theory. For more details, we refer to [16, 17, 26]. For a
prime integer $p$ and for a monic polynomial $\phi\in\mathbb{Z}[x]$ whose
reduction is irreducible in $\mathbb{F}_{p}[x]$, let $\mathbb{F}_{\phi}$ be
the field $\frac{\mathbb{F}_{p}[x]}{(\overline{\phi})}$. For any monic
polynomial $F(x)\in\mathbb{Z}[x]$, upon the Euclidean division by successive
powers of $\phi$, we expand $F(x)$ as
$F(x)=\sum_{i=0}^{l}a_{i}(x)\phi(x)^{i}$, called the $\phi$-expansion of
$F(x)$ (for every $i$, $deg(a_{i}(x))<deg(\phi)$). The $\phi$-Newton polygon
of $F(x)$ with respect to $p$, is the lower boundary of the convex envelope of
the set of points $\\{(i,\nu_{p}(a_{i}(x))),\,a_{i}(x)\neq 0\\}$ in the
Euclidean plane, which is denoted by $N_{\phi}(F)$. Let
$S_{1},\,S_{2},\dots,S_{t}$ be the sides of $N_{\phi}(F)$. For every side $S$
of $N_{\phi}(F)$, the length of $S$, denoted by $l(S)$, is the length of its
projection to the $x$-axis, its height, denoted by $H(S)$, is the length of
its projection to the $y$-axis. Let $\lambda=H(S)/l(S)$, then $-\lambda$ is
the slope of $S$. If $\lambda\neq 0$, then $\lambda=h/e$ with $e$ and $h$ two
positive coprime integer. Notice that $e=l(S)/d$, called the ramification
index of $S$ and $h=H(S)/d$, where $d=$gcd$(l(S),H(S))$ is called the degree
of $S$. Thus $N_{\phi}(F)$ is the join of its different sides ordered by
increasing slopes, which we can express by
$N_{\phi}(F)=S_{1}+S_{2}+\dots+S_{t}$. The principal $\phi$-Newton polygon of
$F(x)$ , denoted by $N_{\phi}^{+}(F)$, is the part of the polygon
$N_{\phi}(F)$, which is determined by all sides of negative slopes of
$N_{\phi}(F)$. For every side $S$ of $N_{\phi}^{+}(F)$, with initial point
$(s,u_{s})$ and length $l$, and for every $i=0,\dots,l$, we attach the
following residue coefficient $c_{i}\in\mathbb{F}_{\phi}$ as follows:

$c_{i}=\left\\{\begin{array}[]{ll}0,&\mbox{ if }(s+i,{\it u_{s+i}})\mbox{ lies
strictly above }S,\\\ \left(\dfrac{a_{s+i}(x)}{p^{{\it u_{s+i}}}}\right)\,\,\
\mbox{\rm(mod }{(p,\phi(x))}),&\mbox{ if }(s+i,{\it u_{s+i}})\mbox{ lies on
}S.\end{array}\right.$

where $(p,\phi(x))$ is the maximal ideal of $\mathbb{Z}[x]$ generated by $p$
and $\phi$.
Let $\lambda=-h/e$ be the slope of $S$, where $h$ and $e$ are two positive
coprime integers. Then $d=l/e$ is the degree of $S$. Notice that, the points
with integer coordinates lying on $S$ are exactly
$(s,u_{s}),(s+e,u_{s}-h),\cdots,(s+de,u_{s}-dh)$. Thus, if $i$ is not a
multiple of $e$, then $(s+i,u_{s+i})$ does not lie on $S$, and so $c_{i}=0$.
Let
$F_{S}(y)=t_{d}y^{d}+t_{d-1}y^{d-1}+\cdots+t_{1}y+t_{0}\in\mathbb{F}_{\phi}[y]$,
called the residual polynomial of $F(x)$ associated to the side $S$, where for
every $i=0,\dots,d$, $t_{i}=c_{ie}$. The theorem of Ore plays a key role for
proving our main theorems:
Let $\phi\in\mathbb{Z}[x]$ be a monic polynomial, with $\overline{\phi(x)}$
irreducible in $\mathbb{F}_{p}[x]$. As defined in [17, Def. 1.3], the
$\phi$-index of $F(x)$, denoted $ind_{\phi}(F)$, is deg$(\phi)$ multiplied by
the number of points with natural integer coordinates that lie below or on the
polygon $N_{\phi}^{+}(F)$, strictly above the horizontal axis, and strictly
beyond the vertical axis (see Figure 1).

$0$$S_{1}$$S_{2}$$S_{3}$ Figure 1.  $N_{\phi}^{+}(F)$.

Assume that $\overline{F(x)}=\prod_{i=1}^{r}\overline{\phi_{i}}^{l_{i}}$ is
the factorization of $\overline{F(x)}$ in $\mathbb{F}_{p}[x]$, where
$\phi_{1},\dots,\phi_{r}$ are monic polynomials lying in $\mathbb{Z}[x]$ and
$\overline{\phi_{1}},\dots,\overline{\phi_{r}}$ are pairwise coprime
irreducible polynomials over $\mathbb{F}_{p}$. For every $i=1,\dots,r$, let
$N_{\phi_{i}}^{+}(F)=S_{i1}+\dots+S_{ir_{i}}$ be the principal part of the
$\phi_{i}$-Newton polygon of $F$ with respect to $p$. For every
$j=1,\dots,r_{i}$, let
$F_{S_{ij}}(y)=\prod_{s=1}^{s_{ij}}\psi_{ijs}^{a_{ijs}}(y)$ be the
factorization of $F_{S_{ij}}(y)$ into powers of monic irreducible polynomials
of $\mathbb{F}_{\phi_{i}}[y]$. Then we have the following theorem of Ore (see
[17, Theorem 1.7 and Theorem 1.9], [16, Theorem 3.9], and [31]):

######  Theorem 3.1.

(Theorem of Ore)

  1. (1)

$\nu_{p}((\mathbb{Z}_{K}:\mathbb{Z}[\alpha]))\geq\sum_{i=1}^{r}ind_{\phi_{i}}(F).$

The equality holds if $a_{ijs}=1$ for every $i,j,s$.

  2. (2)

If $a_{ijs}=1$ for every $i,j,s$, then

$p\mathbb{Z}_{K}=\prod_{i=1}^{r}\prod_{j=1}^{r_{i}}\prod_{s=1}^{s_{ij}}\mathfrak{p}^{e_{ij}}_{ijs}$

is the factorization of $p\mathbb{Z}_{K}$ into powers of prime ideals of
$\mathbb{Z}_{K}$ lying above $p$, where $e_{ij}$ is the ramification index of
the side $S_{ij}$ and
$f_{ijs}=\mbox{deg}(\phi_{i})\times\mbox{deg}(\psi_{ijs})$ is the residue
degree of $\mathfrak{p}_{ijs}$ over $p$ for every $i=1,\dots,r$,
$j=1,\dots,r_{i}$, and $s=1,\dots,s_{ij}$.

######  Corollary 3.2.

Under the assumptions above Theorem 3.1, if for every $i=1,\dots,r$, $l_{i}=1$
or $N_{\phi_{i}}(F)=S_{i}$ has a single side of height $1$, then
$\nu_{p}((\mathbb{Z}_{K}:\mathbb{Z}[\alpha]))=0$.

The following lemma allows to evaluate the $p$-adic valuation of the binomial
coefficient $\binom{p^{r}}{j}$. For its proof, we refer to [4].

######  Lemma 3.3.

Let $p$ be a rational prime integer and $r$ be a positive integer. Then
$\nu_{p}\left(\binom{p^{r}}{j}\right)=r-\nu_{p}(j)$ for any integer
$j=1,\dots,p^{r}-1$.

The following lemma allows to determine the $\phi$-Newton polygon of $F(x)$.

######  Lemma 3.4.

Let $F(x)=x^{n}-m\in\mathbb{Z}[x]$ be an irreducible polynomial and $p$ a
prime integer which divides $n$ and does not divide $m$. Let $n=p^{r}t$ in
$\mathbb{Z}$ with $p$ does not divide $t$. Then
$\overline{F(x)}=\overline{(x^{t}-m)}^{p^{r}}$. Let $v=\nu_{p}(m^{p}-m)$ and
$\phi\in\mathbb{Z}[x]$ be a monic polynomial, whose reduction modulo $p$
divides $\overline{F(x)}$. Let us denote $(x^{t}-m)=\phi(x)Q(x)+R(x)$. Then
$\nu_{p}(R)\geq 1$.

  1. (1)

If $\nu_{p}(m^{p-1}-1)\leq r$, then $N_{\phi}^{+}(F)$ is the lower boundary of
the convex envelope of the set of the points
$\\{(0,v)\\}\cup\\{(p^{j},r-j),\,j=0,\dots,r\\}$.

  2. (2)

If $\nu_{p}(m^{p-1}-1)\geq r+1$, then $N_{\phi}^{+}(F)$ is the lower boundary
of the convex envelope of the set of the points
$\\{(0,V)\\}\cup\\{(p^{j},r-j),\,j=0,\dots,r\\}$ for some integer $V\geq r+1$.

##  4\. Proofs of main results

###### Proof.

of Theorem 2.1.
The proof of Theorem 2.1 can be done by using Dedekind’s criterion as it was
shown in the proof of [29, Theorem $6.1$]. But as the other results are based
on Newton polygon’s techniques, let us use theorem of index with ”if and only
if” as it is given in [26, Theorem 4.18], which says that:
$\nu_{p}(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])=0$ if and only if $ind_{1}(F)=0$,
where $ind_{1}(F)$ is the index given in Theorem 3.1. Since
$\triangle(F)=\mp(2^{u}\cdot 3^{v}\cdot 5^{t})^{2^{u}\cdot 3^{v}\cdot
5^{t}}m^{2^{u}\cdot 3^{v}\cdot 5^{t}-1}$, by the formula
$\nu_{p}(\triangle(F))=2\nu_{p}(ind(F))+\nu_{p}(d_{K})$, where $d_{K}$ is the
absolute discriminant of $K$ and $ind(F)=(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])$,
we conclude that $\mathbb{Z}[\alpha]$ is integrally closed if and only if $p$
does not divide $(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])$ for every rational prime
integer $p$ dividing $30m$. Let $p$ be a rational prime dividing $m$, then
$F(x)\equiv\phi^{2^{u}\cdot 3^{v}\cdot 5^{t}}(\ mod\ p)$, where $\phi=x$. As
$m$ is a square free integer, the $\phi$-principal Newton polygon with respect
to $\nu_{p}$, $N_{\phi}^{+}(F)=S$ has a single side of height $\nu_{p}(m)$. As
$l(S)=2^{u}\cdot 3^{v}\cdot 5^{t}\geq 2$, $ind_{\phi}(F)=0$ if and only if $S$
has height $1$, which means $\nu_{p}(m)=1$. It follows that the unique prime
candidates to divide the index $(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])$ are $2$,
$3$, and $5$.
For $p=2$ and $2$ does not divide $m$, let $\phi\in\mathbb{Z}[x]$ be a monic
polynomial, whose reduction is an irreducible factor of $(x^{3^{v}\cdot
5^{t}}-1)$ in $\mathbb{F}_{2}[x]$. Again as $l(N_{\phi}^{+}(F))=2^{u}\geq 2$,
$ind_{\phi}(F)=0$ if and only if $N_{\phi}^{+}(F)$ has a single side of height
$1$, which means by Lemma 3.4 that $\nu_{2}(1-m)=1$; $m\not\equiv 1\
\mbox{\rm(mod }{4})$.
Similarly, for $p=3$ and $3$ does not divide $m$, let $\phi\in\mathbb{Z}[x]$
be a monic polynomial, whose reduction is an irreducible factor of
$(x^{2^{u}\cdot 5^{t}}-m)$ in $\mathbb{F}_{3}[x]$. Again as
$l(N_{\phi}^{+}(F))=3^{v}\geq 2$, $ind_{\phi}{F}=0$ if and only if
$N_{\phi}^{+}(F)$ has a single side of height $1$, which means by Lemma 3.4
that $\nu_{3}(m^{2}-1)=1=1$; $m\not\equiv\mp 1\ \mbox{\rm(mod }{9})$.
Again, for $p=5$ and $5$ does not divide $m$, let $\phi\in\mathbb{Z}[x]$ be a
monic polynomial, whose reduction is an irreducible factor of $(x^{2^{u}\cdot
3^{v}}-m)$ in $\mathbb{F}_{5}[x]$. As $l(N_{\phi}^{+}(F))=5^{t}\geq 2$,
$ind_{\phi}{F}=0$ if and only if $N_{\phi}^{+}(F)$ has a single side of height
$1$, which means by Lemma 3.4 that $\nu_{5}(m^{4}-1)=1=1$; $m\not\in\\{\mp
1,\pm 7\\}\ \mbox{\rm(mod }{25})$. ∎

The existence of prime common index divisors was first established in $1871$
by Dedekind who exhibited examples in fields of third and fourth degrees, for
example, he considered the cubic field $K$ defined by $F(x)=x^{3}-x^{2}-2x-8$
and he showed that the prime $2$ splits completely. So, if we suppose that $K$
is monogenic, then we would be able to find a cubic polynomial generating $K$,
that splits completely into distinct polynomials of degree $1$ in
$\mathbb{F}_{2}[x]$. Since there are only $2$ distinct polynomials of degree
$1$ in $\mathbb{F}_{2}[x]$, this is impossible. Based on these ideas and using
Kronecker’s theory of algebraic numbers, Hensel gave a necessary and
sufficient condition on the so-called ”index divisors” for any prime integer
$p$ to be a prime common index divisor [30]. $($For more details see [29]$)$.
For the proof of Theorem 2.2, we need the following lemma, which characterizes
the prime common index divisors of $K$. We need to use only one way, which is
an immediate consequence of Dedekind’s theorem.

######  Lemma 4.1.

Let $p$ be a rational prime integer and $K$ be a number field. For every
positive integer $f$, let $\mathcal{P}_{f}$ be the number of distinct prime
ideals of $\mathbb{Z}_{K}$ lying above $p$ with residue degree $f$ and
$\mathcal{N}_{f}$ the number of monic irreducible polynomials of
$\mathbb{F}_{p}[x]$ of degree $f$. Then $p$ is a prime common index divisor of
$K$ if and only if $\mathcal{P}_{f}>\mathcal{N}_{f}$ for some positive integer
$f$.

######  Remark.

As it was shown in the proof of Theorem 2.1, the unique prime candidates to be
a prime common index divisors of $K$ are $2$, $3$, and $3$, because if
$p\not\in\\{2,3,5\\}$, then $p$ does not divide the index
$(\mathbb{Z}_{K}:\mathbb{Z}[\alpha])$, and so the factorization of
$p\mathbb{Z}_{K}$ is analogous to the factorization of $x^{2^{u}\cdot
3^{v}\cdot 5^{t}}-m$ in $\mathbb{F}_{p}[x]$.

######  Remark.

In order to prove Theorem 2.2, we don’t need to determine the factorization of
$p\mathbb{Z}_{K}$ explicitly. But according to Lemma 4.1, we need only to show
that $\mathcal{P}_{f}>\mathcal{N}_{f}$ for an adequate positive integer $f$.
So in practice the second point of Theorem 3.1, could be replaced by the
following: If $l_{i}=1$ or $d_{ij}=1$ or $a_{ijk}=1$ for some $(i,j,k)$
according to notation of Theroem 3.1, then $\psi_{ijk}$ provides a prime ideal
$\mathfrak{p}_{ijk}$ of $\mathbb{Z}_{K}$ lying above $p$ with residue degree
$f_{ijk}=m_{i}\times t_{ijk}$, where $t_{ijk}=$deg$(\psi_{ijk})$ and
$p\mathbb{Z}_{K}=\mathfrak{p}_{ijk}^{e_{i}j}I$, where the factorization of the
ideal $I$ can be derived from the other factors of each residual polynomials
of $F(x)$.

###### Proof.

of Theorem 2.2.

  1. (1)

Assume that $m\equiv 1\ \mbox{\rm(mod }{4})$. Let $3^{v}\cdot 5^{t}=3s$ for
some odd integer $s\in\mathbb{Z}$. Then
$\overline{F(x)}=\overline{((x^{3}-1)(U(x))^{2^{u}}}=\overline{(x^{2}+x+1)T(x)}^{2^{u}}$
in $\mathbb{F}_{2}[x]$ for some monic polynomials $U$ and $T$ in
$\mathbb{Z}[x]$ such that $\overline{(x^{2}+x+1)}$ and $\overline{T(x)}$ are
coprime over $\mathbb{F}_{2}[x]$ because $\overline{(x^{3s}-1)}$ is separable
over $\mathbb{F}_{2}[x]$. Let $\phi={x^{2}+x+1}$ and $x^{3s}-1=\phi(x)T(x)+2a$
for some integer $a\in\mathbb{Z}$.

    1. (a)

$\nu_{2}(1-m)=2$, then by Lemmas 3.4 and 3.3, $N_{\phi}^{+}(F)=S$ has a single
side of degree $d=2$. By using
$F(x)=((x^{3s}-1)+1)^{2^{u}}-m=(\phi(x)T(x)+2a)^{2^{u}}+\sum_{j=1}^{2^{u}-1}\binom{2^{u}}{j}(T\phi+2a)^{3^{v}-j}\cdots+1-m$,
we have $F_{S}(y)=t^{2}y^{2}+ty+1$, where $t\equiv T(x)\ \mbox{\rm(mod
}{2,\phi})$ is a nonzero element of $\mathbb{F}_{\phi}$ (because
$\overline{\phi}$ does not divide $\overline{T(x)}$ in $\mathbb{F}_{2}[x]$).
Hence $F_{S}(y)=(ty-x)(ty-x^{2})$ in $\mathbb{F}_{\phi}[y]$. Thus by Remark
Remark, $\phi$ provides $2$ distinct prime ideals of $\mathbb{Z}_{K}$ lying
above $2$ with residue degree $2$ each. If $\nu_{2}(1-m)=3$ , then by Lemmas
3.4 and 3.3, $N_{\phi}^{+}(F)$ has two sides $S_{1}$ and $S_{2}$ joining the
point $(0,3)$, $(2^{u-1},1)$, and $(2^{u},0)$ (see $FIGURE\,2$). Thus $S_{1}$
is a side of degree $2$ and $S_{2}$ is a side of degree $1$. By using
$F(x)=((x^{3s}-1)+1)^{2^{u}}-m=(\phi(x)T(x)+2a)^{2^{u}}+\sum_{j=1}^{2^{u}-1}\binom{2^{u}}{j}(T\phi+2a)^{3^{v}-j}\cdots+1-m$,
we conclude that $F_{S_{1}}(y)=t^{2}y^{2}+ty+1=(ty-x)(ty-x^{2})$, where
$t\equiv T(x)\ \mbox{\rm(mod }{2,\phi})$ is a nonzero element of
$\mathbb{F}_{\phi}$ and $F_{S_{2}}(y)$ is of degree $1$. By Remark Remark,
$\phi$ provides three prime ideals of $\mathbb{Z}_{K}$ lying above $2$ with
residue degree deg$(\phi)=2$ each. As $x^{2}+x+1$ is the unique monic
irreducible polynomial of degree $2$ in $\mathbb{F}_{2}[x]$, by Lemma 4.1, $2$
divides $i(K)$ and $K$ is not monogenic.

$0$$2^{u-2}$$2^{u-1}$$2^{u}$$1$$2$$3$$S_{1}$$S_{2}$ Figure 2.
$N_{\phi}^{+}(F)$ for $v_{2}=3$.

    2. (b)

If $v_{2}\geq 4$, then by Lemma 3.4, $N_{\phi}^{+}(F)$ has $g\geq 2$ sides for
which the last two sides $S_{g}$ and $S_{g-1}$ are of height $1$ each (see
$FIGURE3$). Thus, $F_{S_{g}}(y)$ and $F_{S_{g-1}}(y)$ are of degree $1$. By
Remark Remark, $\phi$ provides at least two prime ideals of $\mathbb{Z}_{K}$
lying above $2$ with residue degree deg$(\phi)=2$ each. As $x^{2}+x+1$ is the
unique monic irreducible polynomial of degree $2$ in $\mathbb{F}_{2}[x]$, by
Lemma 4.1, $2$ divides $i(K)$ and $K$ is not monogenic.

$0$$2^{u-2}$$2^{u-1}$$2^{u}$$1$$2$$3$$\nu_{2}$$S_{g-1}$$S_{g}$ Figure 3.
$N_{\phi}^{+}(F)$ for $v_{2}\geq 4$.

  2. (2)

Assume $m\equiv 1\ \mbox{\rm(mod }{9})$. Let $2^{u}\cdot 5^{t}=2s$ for some
integer $s\in\mathbb{Z}$. Then
$\overline{F(x)}=(x^{2s}-1)^{3^{v}}=((x-1)(x+1)U(x))^{3^{v}}$ in
$\mathbb{F}_{3}[x]$ for some monic polynomial $U(x)\in\mathbb{F}_{3}[x]$. Let
$\phi_{1}=x-1$, $\phi_{2}=x+1$, and $v_{3}=\nu_{3}(1-m)$. If $v_{3}\geq 2$,
then by Lemmas 3.4 and 3.3, $N_{\phi_{i}}^{+}(F)$ has $g\geq 2$ sides of which
the last two sides $S_{ig}$ and $S_{ig-1}$ are of height $1$ each for every
$i=1,2$ (see $FIGURE4$ and $FIGURE5$). Thus $F_{S_{ig}}(y)$ and
$F_{S_{ig-1}}(y)$ are of degree $1$ for every $i=1,2$. By Remark Remark, every
$\phi_{i}$ provides at least $2$ prime ideals $\mathfrak{p}_{ij}$ of
$\mathbb{Z}_{K}$ lying above $3$ with residue degree $f_{ij}=1$ for every
$i,j=1,2$. Therefore, there are at least $4$ prime ideals of $\mathbb{Z}_{K}$
lying above $3$ with residue degree $1$ each. As there is only $3$ monic
irreducible polynomial of degree $1$ in $\mathbb{F}_{3}[x]$, by Lemma 4.1, $3$
divides $i(K)$ and $K$ is not monogenic.

$0$$3^{v-2}$$3^{v-1}$$3^{v}$$1$$2$$v_{3}$$S_{i2}$$S_{i1}$ Figure 4.
$N_{\phi_{i}}^{+}({F})$ for $v\geq 2$ and $v_{3}=3$.
$0$$1$$3$$1$$2$$v_{3}$$S_{ig}$$S_{ig-1}$ Figure 5.  $N_{\phi_{i}}^{+}({F})$
for $v=1$ and $v_{3}\geq 3$.

  3. (3)

Assume that $m\equiv-1\ \mbox{\rm(mod }{9})$ and $u=2k$ for some odd integer
$k$. Then $2^{u}\cdot 5^{t}=4s$ for some odd nonnegative integer $s$. Thus
$\overline{F(x)}=\overline{(x^{4}+1)U(x)^{3^{v}}}=((x^{2}+x-1)(x^{2}-x-1)U(x))^{3^{v}}$
in $\mathbb{F}_{3}[x]$ for some monic polynomial $U\in\mathbb{Z}[x]$. Let
$\phi_{1}=x^{2}+x-1$, $\phi_{2}=x^{2}-x-1$, and $v_{3}=\nu_{3}(1-m)$. Since
$3$ does not divide $4s=2^{u}\cdot 5^{t}$, $\overline{(x^{4s}+1)}$ is
separable over $\mathbb{F}_{3}$. So, $\overline{\phi_{1}\phi_{2}(x)}$ and
$\overline{U(x)}$ are coprime in $\mathbb{F}_{3}[x]$. By Lemmas 3.4 and 3.3,
for every $i=1,2$, $N_{\phi_{i}}^{+}(F)$ has at least two sides $S_{i1}$ and
$S_{i2}$ with height $1$ each. Thus $F_{S_{ij}}(y)$ is irreducible over
$\mathbb{F}_{\phi_{i}}$ as it is of degree $1$ for every $i,j=1,2$. By Remark
Remark, every factor $\phi_{i}$ provides at least two distinct prime ideals of
$\mathbb{Z}_{K}$ lying above $3$ with residue degree $f=2$ each. Thus there
are at least four distinct prime ideals of $\mathbb{Z}_{K}$ lying above $3$
with residue degree $f=2$ each. As $x^{2}+1$, $x^{2}+x-1$, and $x^{2}-x-1$ are
the unique monic irreducible polynomials of degree $2$ in $\mathbb{F}_{3}[x]$,
by Lemma 4.1, $3$ divides $i(K)$, and so $K$ is not monogenic.

  4. (4)

Similarly, if we assume that $u=1$, $m\equiv\pm 1\ \mbox{\rm(mod }{81})$, and
$v\geq 3$, then let $\phi=x^{2}+1$. By Lemmas 3.4 and 3.3, $N_{\phi}^{+}(F)$
has at least four sides of degree $1$ each. Thus $\phi$ provides at least four
prime ideals of $\mathbb{Z}_{K}$ lying above $3$ with residue degree $1$ each.

  5. (5)

Assume that $m\equiv\pm 1\ \mbox{\rm(mod }{125})$ and $t\geq 2$. Then
$\overline{F(x)}$ has two monic factors $\phi_{1}$ and $\phi_{2}$ of degree
$1$ each in $\mathbb{F}_{5}[x]$. Let $\overline{\phi}$ be one fixed factor of
$\overline{F(x)}$ of degree $1$. Since $m\equiv\pm 1\ \mbox{\rm(mod }{125})$,
by Lemmas 3.4 and 3.3, we conclude that $N_{\phi}^{+}(F)$ has at least $3$
sides of degree $1$ each. Thus $\phi$ provides at least $3$ prime ideals of
$\mathbb{Z}_{K}$ lying above $5$ with residue degree $1$ each. Hence the two
factors $\phi_{1}$ and $\phi_{2}$ provide at least $6$ prime ideals of
$\mathbb{Z}_{K}$ lying above $5$ with residue degree $1$ each. Therefore $5$
divides $i(K)$, and so $K$ is not monogenic.

  6. (6)

Assume that $m\equiv 1\ \mbox{\rm(mod }{25})$ and $u\geq 2$. By the litle
Fermat’s theorem, $\overline{x^{4}-1}$ has four distinct monic factors of
degree $1$ each in $\mathbb{F}_{5}[x]$. Let $\overline{\phi}$ be one fixed
factor of degree $1$ of $\overline{F(x)}$ in $\mathbb{F}_{5}[x]$. Since
$m\equiv 1\ \mbox{\rm(mod }{25})$, by Lemmas 3.4 and 3.3, we conclude that
$N_{\phi}^{+}(F)$ has at least $2$ sides of degree $1$ each. Thus $\phi$
provides at least $2$ prime ideals of $\mathbb{Z}_{K}$ lying above $5$ with
residue degree $2$ each. It follows that the four factors provide at least $8$
prime ideals of $\mathbb{Z}_{K}$ lying above $5$ with residue degree $1$ each.
Thus, $5$ divides $i(K)$, and so $K$ is not monogenic.

  7. (7)

If $m\equiv-1\ \mbox{\rm(mod }{25})$ and $u=2k$, then $\overline{x^{12}+1}$
divides $\overline{F(x)}$ in $\mathbb{F}_{5}[x]$ and
$\overline{x^{12}+1}=\overline{(x^{2}+4x+2)(x^{2}+3x+3)(x^{2}+2)(x^{2}+x+2)(x^{2}+2x+3)(x^{2}+3)}$
in $\mathbb{F}_{5}[x]$. Let $\overline{\phi}$ be one fixed of these factors.
Since $m\equiv 1\ \mbox{\rm(mod }{25})$, by Lemmas 3.4 and 3.3,
$N_{\phi}^{+}(F)$ has at least $2$ sides of degree $1$ each. Thus $\phi$
provides at least $2$ prime ideals of $\mathbb{Z}_{K}$ lying above $5$ with
residue degree $2$ each. Hence there are at least $12$ prime ideals of
$\mathbb{Z}_{K}$ lying above $5$ with residue degree $2$ each. As there are
only $10$ monic irreducible polynomial of degree $2$ in $\mathbb{F}_{5}[x]$,
by Lemma 4.1, $5$ divides $i(K)$, and so $K$ is not monogenic.

  8. (8)

Assume that $u=v=1$, $m\equiv\pm 82\ \mbox{\rm(mod }{5^{4}})$, and $t\geq 3$.
Since $\overline{x^{6}-2}=\overline{(x^{2}+4x+2)(x^{2}+x+2)(x^{2}+2)}$ and
$\overline{x^{6}+2}=\overline{(x^{2}+3)(x^{2}+3x+3)(x^{2}+2x+3)}$ in
$\mathbb{F}_{5}[x]$. Let $\overline{\phi}$ be one fixed irreducible factor of
$\overline{F(x)}$ of degree $2$. Since $m\equiv\pm 82\ \mbox{\rm(mod
}{5^{4}})$, we have $\nu_{5}(m^{4}+1)\geq 4$. So, by Lemmas 3.4 and 3.3, we
conclude that $N_{\phi}^{+}(F)$ has at least $4$ sides of degree $1$ each.
Thus $\phi$ provides at least $4$ prime ideals of $\mathbb{Z}_{K}$ lying above
$5$ with residue degree $2$ each. Since $\overline{F(x)}$ has $3$ distinct
monic irreducible factors in $\mathbb{F}_{5}[x]$ of degree $2$ each, we
conclude that there are at least $12$ prime ideals of $\mathbb{Z}_{K}$ lying
above $5$ with residue degree $2$ each. As there is only $10$ monic
irreducible polynomial of degree $2$ in $\mathbb{F}_{5}[x]$, by Lemma 4.1, $5$
divides $i(K)$, and so $K$ is not monogenic.

∎

###### Proof.

of Theorem 2.3.
Since gcd$(k,30)=1$, let $(x,y)\in\mathbb{Z}^{2}$ be the unique solution of
the equation $k\cdot x-2^{u}\cdot 3^{v}\cdot 5^{t}\cdot y=1$ and
$\theta=\frac{\alpha^{x}}{a^{y}}$. Then $\theta^{2^{u}\cdot 3^{v}\cdot
5^{t}}=a$, and so $g(x)=x^{2^{u}\cdot 3^{v}\cdot 5^{t}}-a$ is the minimal
polynomial of $\theta$ over $\mathbb{Q}$; $\theta\in\mathbb{Z}_{K}$ is a
primitive element of $K$. Since $a\neq\pm 1$ is a square free integer, we can
apply Theorems 2.1 and 2.2, and get the desired result. ∎

## References

  * [1] S. Ahmad, T. Nakahara, and S. M. Husnine, Power integral bases for certain pure sextic fields, Int. J. of Number Theory v:10, No 8 (2014) 2257–2265.
  * [2] S. Ahmad, T. Nakahara, and A. Hameed, On certain pure sextic fields related to a problem of Hasse, Int. J. Alg. and Comput. 26(3) (2016) 577–583
  * [3] H. Ben Yakou and L. El Fadil, on power integral bases for certain pure number fields defined by $x^{p^{r}}-m$, Int. J. Number theory, 2021, doi:10.1142/S1793042121500858
  * [4] H. Ben yakkou, L. El Fadil, and A. Chillali, On Power integral bases for certain pure number fields defined by $x^{2^{r}\cdot 5^{s}}-m$, Comm. in Algebra, 2021, doi: 10.1080/0092782.2021.1883642
  * [5] H. Cohen, A Course in Computational Algebraic Number Theory, GTM 138, Springer-Verlag Berlin Heidelberg (1993)
  * [6] R. Dedekind, Über den Zusammenhang zwischen der Theorie der Ideale und der Theorie der höheren Kongruenzen, Göttingen Abhandlungen 23 (1878) 1–23
  * [7] L. El Fadil, Computation of a power integral basis of a pure cubic number field, Int. J. Contemp. Math. Sci. 2(13-16)(2007) 601–606
  * [8] L. El Fadil, On Power integral bases for certain pure sextic fields (To appear in a forthcoming issue of Bol. Soc. Paran. Math.)
  * [9] L. El Fadil, On Power integral bases for certain pure sextic fieldsJ, J. Number Theory, 228 (2021) 375–389
  * [10] L. El Fadil, On Power integral bases for certain pure number fields  (To appear in a forthcoming issue of Publicationes Mathematicae)
  * [11] L. El Fadil, On Power integral bases for certain pure number fields defined by $x^{24}-m$ , Stud. Sci. Math. Hung. 57(3) (2020) 397–407
  * [12] L. El Fadil, On Power integral bases for certain pure number fields defined by $x^{18}-m$ , (To appear in Comm. Math. Uni. of Carolina)
  * [13] L. El Fadil, On Power integral bases for certain pure number fields defined by $x^{2\cdot 3^{k}}-m$ ( To appear in a forthcoming issue of Act. Arith.)
  * [14] L. El Fadil, On Power integral bases for certain pure number fields defined by $x^{2\cdot 3^{k}}-m$ , Colloquium Mathematicum, 2022, doi:10.4064/cm8574-6-2021
  * [15] L. El Fadil, On Power integral bases for certain pure number fields defined by $x^{60}-m$ , (To appear in Acta Mathematica Vietnamica)
  * [16] L. El Fadil,On Newton polygon’s techniques and factorization of polynomial over Henselian valued fields, J. of Algebra and its Appl. (2020), doi: S0219498820501881
  * [17] L. El Fadil, J. Montes and E. Nart, Newton polygons and $p$-integral bases of quartic number fields, J. Algebra and Appl. 11(4) (2012) 1250073
  * [18] L. El Fadil and A. Najim, On Power integral bases for certain pure number fields defined by $x^{2^{u}\cdot 3^{v}}-m$ (To appear in ASM)
  * [19] T. Funakura, On integral bases of pure quartic fields, Math. J. Okayama Univ. 26 (1984) 27–41
  * [20] L. El Fadil and I. Gaál, On integral bases and monogeneity of pure octic number fields with non-square free parameters, (To appear in Glasnik Mat.)
  * [21] I. Gaál, Power integral bases in algebraic number fields, Ann. Univ. Sci. Budapest. Sect. Comp. 18 (1999) 61–87
  * [22] I. Gaál, Diophantine equations and power integral bases, Theory and algorithm, Second edition, Boston, Birkhäuser, 2019
  * [23] I. Gaál, P. Olajos, and M. Pohst, Power integral bases in orders of composite fields, Exp. Math. 11(1) (2002) 87–90.
  * [24] I. Gaál and L. Remete, Binomial Thue equations and power integral bases in pure quartic fields, JP Journal of Algebra Number Theory Appl. 32(1) (2014) 49–61
  * [25] I. Gaál and L. Remete, Power integral bases and monogenity of pure fields, J. of Number Theory 173 (2017) 129–146
  * [26] J. Guardia, J. Montes and E. Nart,Newton polygons of higher order in algebraic number theory, J. trans. of ams 364(1) (2012) 361–416
  * [27] A.Hameed and T.Nakahara, Integral bases and relative monogenity of pure octic fields, Bull. Math. Soc. Sci. Math. R épub. Soc. Roum. 58(106) No. 4(2015) 419–433
  * [28] A. Hameed, T. Nakahara, S. M. Husnine, On existence of canonical number system in certain classes of pure algebraic number fields, J. Prime Res. Math. 7(2011) 19–24.
  * [29] H. Smith, The monogenity of radical extension, Acta Arithmitica, 198(2021), 313–327.
  * [30] K. Hensel, Arithmetische Untersuchungen über die gemeinsamen ausserwesentlichen Discriminantentheiler einer Gattung, J. Reine Angew. Math., 113:128–160, 1894. ISSN 0075-4102. doi: 10.1515/crll.1894.113.128.
  * [31] J. Montes and E. Nart, On a theorem of Ore, J. Algebra 146(2) (1992) 318–334
  * [32] J. Neukirch, Algebraic Number Theory, Springer-Verlag, Berlin, 1999.
  * [33] O. Ore, Newtonsche Polygone in der Theorie der algebraischen Korper, Math. Ann., 99 (1928), 84–117
  * [34] A. Pethö and M. Pohst, On the indices of multiquadratic number fields, Acta Arith. 153(4) (2012) 393–414