# EEG2IMAGE: Image Reconstruction from EEG Brain Signals

###### Abstract

Reconstructing images using brain signals of imagined visuals may provide an
augmented vision to the disabled, leading to the advancement of Brain-Computer
Interface (BCI) technology. The recent progress in deep learning has boosted
the study area of synthesizing images from brain signals using Generative
Adversarial Networks (GAN). In this work, we have proposed a framework for
synthesizing the images from the brain activity recorded by an
electroencephalogram (EEG) using small-size EEG datasets. This brain activity
is recorded from the subject’s head scalp using EEG when they ask to visualize
certain classes of Objects and English characters. We use a contrastive
learning method in the proposed framework to extract features from EEG signals
and synthesize the images from extracted features using conditional GAN. We
modify the loss function to train the GAN, which enables it to synthesize
$128\times 128$ images using a small number of images. Further, we conduct
ablation studies and experiments to show the effectiveness of our proposed
framework over other state-of-the-art methods using the small EEG dataset.

Index Terms—  Deep Learning, EEG, GAN

##  1 Introduction

Human visual system is considered a highly advanced intelligent information
processor that generates rich 3D visuals with semantic construction. The most
challenging problem is to train artificial machines to construct images from
brain activity directly to semantic categories [1]. The possibility of brain-
to-image construction significantly contributes to advancing the Brain-
Computer Interface (BCI) technology. The core purpose of BCI using invasive or
non-invasive techniques is to provide communication and control of external
devices by thought alone or using minimal muscular activity. This effort would
be highly relevant in neuro-rehabilitation, i.e. to support patients with
disabilities to have better everyday communication in their lives. Decoding
brain responses to imagination/visual stimuli would greatly benefit the
communication exchange for the disabled people. The most widely employed brain
imaging modality with high temporal precision is Electroencephalography (EEG)
due to its relatively lower cost and portability.

EEG is a non-invasive technique which makes it the most practical methodology
to record the electrophysiological dynamics of the brain. EEG signals have
been used to analyze a wide spectrum of research from aspects of cognition to
clinical aspects [2]. For decades, EEG signals have been widely employed for
classifying several disorders or understanding brain dynamics. The successful
implication of the past results can be seen in the BCI. A previous effort made
by the research community has shown promising results to augment healthy
individuals with additional sensory or motor capabilities [3]. The most
intriguing task is to decode the content of the mind using brain signals and
draw a link between them. Two most challenging endeavors in this space are to
reconstruct the visualized images [4] and decode imagined speech-to-text [5]
based solely on recorded brain signals. Vision neuroscientists made the
initial attempts [6, 7, 8] to provide an evidence of visual stimuli features
represented in recorded brain activity. These attempts initiated the
classification of image categories using brain signals using deep learning and
further led to the reconstruction and generation of the images [9].

Our contributions are as follows: 1) A framework that can synthesize images
using a small EEG dataset, 2) Use of semi-hard triplet loss [10] to learn
features from EEG signals that show better k-means accuracy than the softmax
counterpart, as shown in Figs. [2,3] and 3) Use of mode seeking regularization
[11] and data augmentation [12] based modification to GAN for synthesizing
high-quality images using conditional GAN as shown in Fig.
4(b).111https://github.com/prajwalsingh/EEG2Image

(a) EEG feature extractor

(b) EEG2Image

Fig. 1: This figure illustrates the proposed framework for EEG feature
extraction and image generation. a) Shows the LSTM network with $128$ hidden
units that transforms EEG signal into $128D$ feature vector. b) Shows the GAN
network with a data augmentation block that prevents the discriminator from
memorizing the small dataset and helps the generator synthesize high-quality
images.

##  2 Related Works

The development of advanced deep generative architectures in recent times has
made it possible to see images from brain signals. The initial study by
Kavasidis _et al._ implemented long short-term memory (LSTM) stacked with
generative techniques to generate seen images from 40 Image Net classes [4].
Thoughtviz [13] encouraged the design of conditional GAN (cGAN) to decode EEG
signals using a small dataset consisting of imagination tasks comprising
digits, characters, and objects. Several architectures have been developed
using CNN and LSTM on the time-series data of most biological areas. The
capability of LSTM for identifying the sequential pattern and CNN to locate
the neighborhood features was recently combined with spectral normalization
generative adversarial network (SNGAN) to yield seen images from EEG encodings
[14]. Researchers are putting effort into reconstructing geometrical shapes
from brain activities, primarily in generating precise edges and other low-
level details. Further advancement in GAN leads to synthesizing natural
geometrical shapes, which enforces semantic alignment constraints to construct
natural shapes at pixel-level [15, 16]. Recently, a siamese network was
utilized to maximize the relationship between extracted manifold brain feature
representation and visual features [17]. The obtained representation
demonstrated better image classification and saliency detection performance on
the learned manifold. Khare _et al._ proposed conditional progressive growing
of GANs (CProGAN) to develop perceived images [18] and showed higher inception
than previous related work. Recent work on contrastive self-supervised
approach has been shown to maximize the mutual information between visual
stimulus and corresponding EEG latent representations [19]. They proposed an
approach that employed cross-modal alignment enforcing image retrieval at the
instance level rather than pixel-level generation.

Fig. 2: t-SNE [20] visualization of Object test dataset [21] EEG feature space
which is learned using label supervision with test classification accuracy
0.75 and k-means accuracy 0.18.

##  3 Proposed Method

In this work, we proposed a framework shown in Fig.1, for visualizing the
brain activity EEG signals. The framework consists of a two-phase approach: 1)
extracting good features from the EEG signals with a contrastive learning
approach and 2) a conditional data-efficient GAN to transform the extracted
EEG features to image. In our case, a good feature implies useful information
about an image that can help GAN to reconstruct that image.

Fig. 3: t-SNE [20] visualization of Object test dataset [21] EEG feature space
which is learned using triplet loss with test k-means accuracy 0.53. Each
cluster’s equivalent EEG-based generated images are also visualized in this
plot.

(a) ThoughtViz [13]

(b) EEG2Image (Ours)

Fig. 4: Qualitative comparison between the images generated by EEG signals
using the ThoughtViz method (left) and our proposed framework (right). Images
in the red bounding box are the sample images from the Object test dataset
[21]. These images are visualized by the participants, and the respective
brain activity EEG signals is used here for reconstruction.

Feature Extraction. Recent works [17] have shown that the contrastive
learning-based approach outperforms the supervised setting in the case of
generalized feature learning for downstream tasks such as object detection,
classification, saliency map from EEG signals, etc. Building on this, we have
used a triplet loss-based contrastive learning [10] approach in the proposed
framework for EEG feature learning. Triplet loss aims to minimize the distance
between the two data with the same labels and maximize the distance between
the two data with different labels. To prevent the feature extraction network
from squashing the representation of each data into a small cluster, a margin
term is used in triplet loss. It ensures that the distance between the feature
of the same label data is close to zero and greater than the margin for
different label data. The formulation of triplet loss is as follows:

$\displaystyle\min_{\theta}\mathbb{E}\big{[}||f_{\theta}(x^{a})-f_{\theta}(x^{p})||_{2}^{2}-||f_{\theta}(x^{a})-f_{\theta}(x^{n})||_{2}^{2}+\beta\big{]}$
(1)

where, $f$ is parameterized function on $\theta$ that maps EEG signals to a
feature space i.e. $f_{\theta}:\mathbb{R}^{C\times
T}\xrightarrow{}\mathbb{R}^{128}$. The goal of Eqn. 1 is to minimize the
distance between anchor (a) EEG signal and positive (p) EEG signal of the same
class as the anchor and maximize the distance between anchor EEG signal and
negative (n) EEG signal of different class with margin distance. This
formulation is also known as metric learning or contrastive learning. The idea
behind using the formulation is to ensure that the EEG signals generated by
the brain activity for similar images should be close to each other in the
learned feature space [22]. For learning better features, we have used semi-
hard hard triplets, where the distance of the negative sample is more than
positive but less than the margin, and also used an online hard-triplet mining
strategy similar to [10].

Image Generation. In the proposed framework, we have used a Generative
Adversarial Network (GAN) [23] to synthesize the image from the extracted EEG
feature. A GAN architecture consists of two sub-networks: Generator (G) and
Discriminator (D). The purpose of a Generator is to learn the transformation
between a latent distribution ($p_{\mathbb{Z}}$) and real-world data
distribution ($p_{data}$). In our case, we assume latent distribution as an
isotropic Gaussian $\mathcal{N}(0,I)$ from which we sample a noise vector
$z\in\mathbb{R}^{128}$. The discriminator learns to distinguish real images
from synthesized images. The complete GAN architecture is trained in a min-max
optimization setting. Where the discriminator tries to maximize the score for
real images $D(x)$ and minimize the score for generated images $D(G(z))$, in
contrast to the discriminator, the generator tries to minimize $(1-D(G(z))$
and the minimizing of the term is only possible if generator synthesizes
photorealistic images. The complete GAN optimization process can also be
represented below:

$\displaystyle\min_{G}\max_{D}V(D,G)=\mathbb{E}_{x\sim
p_{data}(x)}[log(D(x))]+$ $\displaystyle\mathbb{E}_{x\sim
p_{\mathcal{Z}(z)}}[log(1-D(G(z))))]$ (2)

Method | Inception Score
---|---
AC-GAN [24] | 4.93
ThoughtViz [13] | 5.43
EEG2Image (Ours) | 6.78
Table 1: Comparison of Inception Score values (on all classes of Object
dataset [21]).

(a) ThoughtViz [13]

(b) EEG2Image (Ours)

Fig. 5: Qualitative comparison between the images generated by the ThoughtViz
method (left) and our proposed framework (right). Images in the red bounding
box are the sample images from the Character test dataset [21], these images
are visualized by the participants, and the respective brain activity EEG
signals is used here for reconstruction.

(a) no modeloss and dataaug,

inception score 3.61.

(b) with modeloss and no dataaug,

inception score 4.27.

(c) no modeloss and with dataaug,

inception score 6.5.

Fig. 6: Ablation study showing the qualitative result on Object dataset [21]
using different loss combinations for training the GAN network.

Object Class |  | Apple
---
(n07739125)
| Car
---
(n02958343)
| Dog
---
(n02084071)
| Gold
---
(n03445326)
| Mobile
---
(n02992529)
| Rose
---
(n12620196)
| Scooter
---
(n03791053)
| Tiger
---
(n02129604)
| Wallet
---
(n04548362)
| Watch
---
(n04555897)
All
Mean | 6.09 | 6.15 | 6.99 | 6.98 | 7.33 | 5.44 | 5.81 | 5.67 | 6.48 | 6.67 | 6.78
| SD
---
0.05 | 0.084 | 0.031 | 0.082 | 0.030 | 0.089 | 0.077 | 0.057 | 0.086 | 0.037 | 0.086

Table 2: Mean and standard deviation (SD) of Inception scores for each class
of Objects dataset [21].

Similar to [13], we aim to develop a framework that can utilize a small-size
EEG dataset for generating images from EEG signals. To overcome the problem of
small dataset [13] has used the trainable weighted Gaussian layer [25], which
learns the mean ($\mu$) and variance ($\sigma$) for the encoded EEG signal. In
this work, we follow a different strategy than [13]. Instead, we have used a
Conditional DCGAN [26] architecture with the following modification 1)
following the work of [27], we have used hinge loss for stable training of
GAN, 2) we have added a differentiable data augmentation block between
generator and discriminator which helps the network in learning from small
datasize [12], and 3) to ensure the mode diversity of GAN we have also used a
mode seeking regularization as proposed in work [11]. In work [27], Lim _et
al._ have shown that the vanilla formulation of GAN suffers from mode collapse
and unstable training problems. To solve these issues, they formulated an SVM
separating hyperplane approach, which is known as GAN Hinge Loss as below:

$\displaystyle\mathcal{L}_{D}$ $\displaystyle=\mathbb{E}_{x\sim
p_{data}(x)}[max(0,1-D(x))]\text{ }+$
$\displaystyle\qquad\qquad\mathbb{E}_{x\sim
p_{\mathcal{Z}(z)}}[max(0,1+D(G(z)))]$ (3) $\displaystyle\mathcal{L}_{G}$
$\displaystyle=-\mathbb{E}_{x\sim p_{\mathcal{Z}(z)}}[D(G(z))]$ (4)

Training a GAN for synthesizing photorealistic images requires a large number
of data [12], and other deep learning approaches also face the same data
scarcity issues. Zhao et al. [12] in their work have shown that the problem of
sparse data for training a GAN can be resolved by adding a Differentiable Data
Augmentation (DiffAug) block between the generator and discriminator, which is
illustrated in Fig.1(b). The issue with sparse data is discriminator can
easily memorize the data, which causes the vanishing gradient problem for the
generator. The data augmentations we have used for our GAN network are
translation and color jittering. The final loss term we aim to optimize for
the proposed EEG2Image is given below:

$\displaystyle\mathcal{L}_{D}$ $\displaystyle=\mathbb{E}_{(x,\psi)\sim
p_{data}(x)}[max(0,1-D({\color[rgb]{1,0,0}T}(x),\psi))]\text{ }+$
$\displaystyle\qquad\mathbb{E}_{x\sim p_{\mathcal{Z}(z)},\psi\sim
p_{data}(x)}[max(0,1+D({\color[rgb]{1,0,0}T}(G(z,\psi)),\psi))]$ (5)
$\displaystyle\mathcal{L}_{ms}$
$\displaystyle=\min_{G}\bigg{(}\frac{d_{I}(G(\psi,z_{1}),G(\psi,z_{2}))}{d_{z}(z_{1},z_{2})}\bigg{)}^{-1}$
(6) $\displaystyle\mathcal{L}_{G}$ $\displaystyle=-\mathbb{E}_{x\sim
p_{\mathcal{Z}(z)},\psi\sim
p_{data}(x)}[D({\color[rgb]{1,0,0}T}(G(z,\psi)),\psi)]+\alpha*\mathcal{L}_{ms}$
(7)

where $\mathcal{L}_{D}$ is discriminator loss, $\mathcal{L}_{G}$ is generator
loss, $\mathcal{L}_{ms}$ is mode seeking regularizer term [11],
${\color[rgb]{1,0,0}T}$ is DiffAugment [12] function, $\psi$ is EEG feature
vector and $\alpha$ is regularizer weight term which kept as $1.0$ for all the
experiments.

##  4 Experiments and Results

In the first part of this section, we will discuss the experimental setup we
used to train the feature extraction and generative network, including the
dataset. Later in this section, we will discuss all the ablation studies done
to justify choices for the proposed framework.

Datasets. We have used the EEG data from [21]. This dataset consists of EEG
signals for $3$ different subjects: Digits, Characters, and Objects. In our
study, we have only used Characters and Object data because these are more
diverse and complex data to show the effectiveness of the proposed framework.
The Characters dataset consists of ten English alphabet classes and the subset
of Chars74K [28]. Similarly, the Objects dataset consists of ten different
object classes and the subset of ImageNet [29]. While collecting brain
activity EEG signals of the participants, they were asked to think about one
of these characters/objects at a time. To record the EEG signals, Emotiv EPOC+
[30] device is used, which has $14$ channels with a sampling rate of $128$ Hz
per channel. For each dataset, $23$ participants were asked to visualize every
ten classes. Thus we have $230$ EEG samples per dataset. For our work, we have
used the EEG data provided by the authors with train-test splits [13]. We
would like to thank the authors for making it publicly available.

EEG2Feature. The first stage of our proposed framework is to convert EEG
signals into useful features for image generation. For this, we design two
regimes. In the first regime, we train a classification network for extracting
EEG features as done in [13]. The classifier is a LSTM [31] network with $128$
hidden units using softmax cross-entropy loss. We use k-means clustering [32]
as a metric for the learned EEG feature, i.e., higher k-means accuracy implies
better learned representation [22]. The first regime gives us $74.3\%$ &
$75.4\%$ classification accuracy on test data of Object dataset & Character
dataset [21] and k-means accuracy of $17.8\%$ and $16.3$%, further we plot
t-SNE map [20] to visualize the clustering of test data features from Object
dataset in Fig. 2. For the second regime, we used a contrastive learning
approach to learning the feature of an EEG signal. As discussed in the Sec. 3
we used semi-hard triplet loss for training the LSTM [31] network with $128$
hidden units. The goal of triplet loss is to structured the feature space in
such a way that positive pairs are in close proximity to each other while
negative pairs are positioned far apart. The k-means accuracy we got on the
test data of the Object dataset is $53\%$, and the Character dataset is
$49\%$. Further, we plot t-SNE map [20] to visualize the clustering of test
data features from Object dataset Fig. 3. We can see that the k-means accuracy
and t-SNE plot are better for the second regime. Therefore we decided to use
the contrastive learning method as an EEG feature extractor for our proposed
framework.

Feature2Image. The second stage of our proposed framework is to synthesize
photorealistic images from extracted EEG features using the first stage. For
synthesizing the image, we have used Conditional DCGAN [26] with modification
as discussed in Sec. 3. We have used Inception Score (IS) [33] as a metric for
image quality comparison with other methods. Table 1 shows our proposed GAN
method performed better in synthesizing the images from less number of EEG
data. In Table 2 we have shown per class inception score for test data of
Object dataset [21]. We also performed the qualitative analysis of synthesized
images for both the dataset Object and Character, which are shown in Figs. [4,
5]. We have performed several ablation studies to verify the importance of
each loss in training the GAN network for the proposed framework. For that, we
trained the proposed conditional GAN (cGAN) for three different regimes on the
Object dataset [21]. In the first regime, we train the cGAN without mode
seeking regularization and DiffAugment, shown in Fig. 6(a). which has an
inception score of $3.61$. In the second regime, we have added a mode seeking
regularization term only and trained the cGAN from scratch, Fig. 6(b) shows
improvement with an inception score of $4.27$. In the third and last regime,
we train the cGAN with the DiffAugment block, showing a large improvement in
the synthesized image as shown in Fig. 6(c) with an inception score of $6.5$.
Based on these experiments, we used both the mode-seeking regularization term
and the DiffAugment block in the proposed framework.

##  5 Conclusion

This work proposes a framework that uses a small-sized dataset for generating
images from brain activity EEG signals. Our proposed framework has a better
inception score than the previously proposed method for the small-sized EEG
dataset and synthesized images of size $128\times 128$. The framework consists
of a contrastive learning approach to learn the good features of EEG data,
which is empirically shown to perform better than the softmax-based supervised
learning method. We have performed several ablation studies to demonstrate the
effectiveness of modified GAN loss function in synthesizing high-quality
images. As future work, we plan to tackle large-size EEG datasets and approach
for complete self/un-supervised learning for extracting features from EEG data
and image synthesis.

## References

  * [1] Yoichi Miyawaki, Hajime Uchida, Okito Yamashita, Masa-aki Sato, Yusuke Morito, Hiroki C Tanabe, Norihiro Sadato, and Yukiyasu Kamitani,  “Visual image reconstruction from human brain activity using a combination of multiscale local image decoders,”  Neuron, vol. 60, no. 5, pp. 915–929, 2008.
  * [2] Vangelis Sakkalis,  “Applied strategies towards eeg/meg biomarker identification in clinical and cognitive research,”  Biomarkers in medicine, vol. 5, no. 1, pp. 93–105, 2011.
  * [3] Mahdi Bamdad, Homayoon Zarshenas, and Mohammad A Auais,  “Application of bci systems in neurorehabilitation: a scoping review,”  Disability and Rehabilitation: Assistive Technology, vol. 10, no. 5, pp. 355–364, 2015.
  * [4] Isaak Kavasidis, Simone Palazzo, Concetto Spampinato, Daniela Giordano, and Mubarak Shah,  “Brain2image: Converting brain signals into images,”  in 25th ACM MM, 2017.
  * [5] Zhenhailong Wang and Heng Ji,  “Open vocabulary electroencephalography-to-text decoding and zero-shot sentiment classification,”  in AAAI, 2022, vol. 36, pp. 5350–5358.
  * [6] Thomas Carlson, David A Tovar, Arjen Alink, and Nikolaus Kriegeskorte,  “Representational dynamics of object vision: the first 1000 ms,”  Journal of vision, vol. 13, no. 10, pp. 1–1, 2013.
  * [7] Thomas A Carlson, Hinze Hogendoorn, Ryota Kanai, Juraj Mesik, and Jeremy Turret,  “High temporal resolution decoding of object position and category,”  Journal of vision, vol. 11, no. 10, pp. 9–9, 2011.
  * [8] Koel Das, Barry Giesbrecht, and Miguel P Eckstein,  “Predicting variations of perceptual performance across individuals from neural activity using pattern classifiers,”  Neuroimage, vol. 51, no. 4, pp. 1425–1437, 2010.
  * [9] Concetto Spampinato, Simone Palazzo, Isaak Kavasidis, Daniela Giordano, Nasim Souly, and Mubarak Shah,  “Deep learning human mind for automated visual classification,”  in IEEE CVPR, 2017, pp. 6809–6817.
  * [10] Florian Schroff, Dmitry Kalenichenko, and James Philbin,  “Facenet: A unified embedding for face recognition and clustering,”  in IEEE CVPR, 2015, pp. 815–823.
  * [11] Qi Mao, Hsin-Ying Lee, Hung-Yu Tseng, Siwei Ma, and Ming-Hsuan Yang,  “Mode seeking generative adversarial networks for diverse image synthesis,”  in IEEE/CVF CVPR, 2019.
  * [12] Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, and Song Han,  “Differentiable augmentation for data-efficient gan training,”  NeurIPS, vol. 33, pp. 7559–7570, 2020.
  * [13] Praveen Tirupattur, Yogesh Singh Rawat, Concetto Spampinato, and Mubarak Shah,  “Thoughtviz: Visualizing human thoughts using generative adversarial network,”  in Proceedings of the 26th ACM International Conference on Multimedia, New York, NY, USA, 2018, MM ’18, p. 950–958, ACM.
  * [14] Xiao Zheng, Wanzhong Chen, Mingyang Li, Tao Zhang, Yang You, and Yun Jiang,  “Decoding human brain activity with deep learning,”  Biomedical Signal Processing and Control, vol. 56, pp. 101730, 2020\.
  * [15] Xiang Zhang, Xiaocong Chen, Manqing Dong, Huan Liu, Chang Ge, and Lina Yao,  “Multi-task generative adversarial learning on geometrical shape reconstruction from eeg brain signals,”  arXiv preprint arXiv:1907.13351, 2019.
  * [16] Ahmed Fares, Sheng-hua Zhong, and Jianmin Jiang,  “Brain-media: A dual conditioned and lateralization supported gan (dcls-gan) towards visualization of image-evoked brain activities,”  in 28th ACM MM, 2020, pp. 1764–1772.
  * [17] Simone Palazzo, Concetto Spampinato, Isaak Kavasidis, Daniela Giordano, Joseph Schmidt, and Mubarak Shah,  “Decoding brain representations by multimodal learning of neural activity and visual features,”  IEEE PAMI, vol. 43, no. 11, pp. 3833–3849, 2020.
  * [18] Sanchita Khare, Rajiv Nayan Choubey, Loveleen Amar, and Venkanna Udutalapalli,  “Neurovision: perceived image regeneration using cprogan,”  Neural Computing and Applications, vol. 34, no. 8, pp. 5979–5991, 2022.
  * [19] Zesheng Ye, Lina Yao, Yu Zhang, and Silvia Gustin,  “See what you see: Self-supervised cross-modal retrieval of visual stimuli from brain activity,”  arXiv preprint arXiv:2208.03666, 2022.
  * [20] Laurens Van der Maaten and Geoffrey Hinton,  “Visualizing data using t-sne.,”  JMLR, vol. 9, no. 11, 2008.
  * [21] Pradeep Kumar, Rajkumar Saini, Partha Pratim Roy, Pawan Kumar Sahu, and Debi Prosad Dogra,  “Envisioned speech recognition using eeg sensors,”  Personal and Ubiquitous Computing, vol. 22, no. 1, pp. 185–199, Feb 2018.
  * [22] Yaling Tao, Kentaro Takagi, and Kouta Nakata,  “Clustering-friendly representation learning via instance discrimination and feature decorrelation,”  preprint arXiv:2106.00131, 2021.
  * [23] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio,  “Generative adversarial networks,”  Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020.
  * [24] Augustus Odena, Christopher Olah, and Jonathon Shlens,  “Conditional image synthesis with auxiliary classifier gans,”  in ICML. PMLR, 2017, pp. 2642–2651.
  * [25] Swaminathan Gurumurthy, Ravi Kiran Sarvadevabhatla, and R Venkatesh Babu,  “Deligan: Generative adversarial networks for diverse and limited data,”  in IEEE CVPR, 2017.
  * [26] Alec Radford, Luke Metz, and Soumith Chintala,  “Unsupervised representation learning with deep convolutional generative adversarial networks,”  preprint arXiv:1511.06434, 2015.
  * [27] Jae Hyun Lim and Jong Chul Ye,  “Geometric gan,”  arXiv preprint arXiv:1705.02894, 2017.
  * [28] Teófilo Emídio De Campos, Bodla Rakesh Babu, Manik Varma, et al.,  “Character recognition in natural images.,”  VISAPP (2), vol. 7, no. 2, 2009.
  * [29] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei,  “Imagenet: A large-scale hierarchical image database,”  in IEEE CVPR. Ieee, 2009, pp. 248–255.
  * [30] “EPOC+ - 14 Channel EEG — emotiv.com,” https://www.emotiv.com/epoc/,  [Accessed 14-Oct-2022].
  * [31] Sepp Hochreiter and Jürgen Schmidhuber,  “Long short-term memory,”  Neural computation, vol. 9, no. 8, 1997.
  * [32] Xin Jin and Jiawei Han,  K-Means Clustering, pp. 563–564,  Springer US, Boston, MA, 2010.
  * [33] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen,  “Improved techniques for training gans,”  in 30th NeurIPS, Red Hook, NY, USA, 2016, NIPS’16, p. 2234–2242, Curran Associates Inc.


# On some realizable metabelian $5$-groups

###### Abstract

Let $G$ be a $5$-group of maximal class and $\gamma_{2}(G)=[G,G]$ its derived
group. Assume that the abelianization $G/\gamma_{2}(G)$ is of type $(5,5)$ and
the transfers $V_{H_{1}\to\gamma_{2}(G)}$ and $V_{H_{2}\to\gamma_{2}(G)}$ are
trivial, where $H_{1}$ and $H_{2}$ are two maximal normal subgroups of $G$.
Then $G$ is completely determined with the isomorphism class groups of maximal
class. Moreover the group $G$ is realizable with some fields $k$, which is the
normal closure of a pure quintic field.

Fouad ELMOUHIB (the corresponding author)
Department of Mathematics and Computer Sciences,
Mohammed First University, Oujda, Morocco,
Correspondence<EMAIL_ADDRESS>
Mohamed TALBI
Regional Center of Professions of Education and Training,
Oujda, Morocco
<EMAIL_ADDRESS>

Abdelmalek AZIZI
Department of Mathematics and Computer Sciences,
Mohammed First University, Oujda, Morocco,
<EMAIL_ADDRESS>
Key words: Groups of maximal class, Metabelian $5$-groups, Transfer, $5$-class
groups.
AMS Mathematics Subject Classification: 11R37, 11R29, 11R20, 20D15.

##  1 Introduction

The coclass of a $p$-group $G$ of order $p^{n}$ and nilpotency class $c$ is
defined as $cc(G)=n-c$, and a $p$-group $G$ is called of maximal class, if it
has $cc(G)=1$. These groups have been studied by various authors, by
determining there classification, the position in coclass graph and the
realization of these groups. Blackburn’s paper [1], is considered as reference
of the basic materials about these groups of maximal class. Eick and Leendhan-
Green in [5] gave a classification of $2$-groups. Blackburn’s classification
in [1], of the $3$-groups of coclass $1$ implies that these groups exhibit
behaviour similar to that proved for $2$-groups. The $5$-groups of maximal
class have been investigated in detail in [2] [3] [4] [8] [13].
With an arbitrary prime $p\geq 2$, let $G$ be a metabelian $p$-group of order
$|G|=p^{n}$ and $cc(G)=1$, where $n\geq 3$. Then $G$ is of maximal class and
the commutator factor group $G/\gamma_{2}(G)$ of $G$ is of type $(p,p)$ [1],
[11]. By $G_{a}^{(n)}(z,w)$ we denote the representative of an isomorphism
class of the metabelian $p$-groups $G$, which satisfies the relations of
theorem 2.1, with a fixed system of exponents $a,w$ and $z$.
In this paper we shall prove that some metabelian $5$-groups are completely
determined with the isomorphism class groups of maximal class, furthermore
they can be realized.
For that we consider $k=\mathbb{Q}(\sqrt[5]{p},\zeta_{5})$, the normal closure
of the pure quintic field $\Gamma=\mathbb{Q}(\sqrt[5]{p})$, and also a cyclic
Kummer extension of degree $5$ of the $5^{th}$ cyclotomic field
$k_{0}=\mathbb{Q}(\zeta_{5})$, where $p$ is a prime number, such that
$p\equiv-1(\mathrm{mod}\,25)$. According to [6], if the $5$-class group of
$k$, denoted $C_{k,5}$, is of type $(5,5)$, we have that the rank of the
subgroup of ambiguous ideal classes under the action of
$Gal(k/k_{0})=\langle\sigma\rangle$, denoted $C_{k,5}^{(\sigma)}$, is rank
$C_{k,5}^{(\sigma)}=1$. Whence by class field theory the relative genus field
of the extension $k/k_{0}$, denoted $k^{*}=(k/k_{0})^{*}$, is one of the six
cyclic quintic extension of $k$.
By $F_{5}^{(1)}$ we denote the Hilbert $5$-class field of a number field $F$.
Let $G=\mathrm{Gal}\left((k^{*})_{5}^{(1)}/k_{0}\right)$, we show that $G$ is
a metabelian $5$-group of maximal class, and has two maximal normal subgroups
$H_{1}$ and $H_{2}$, such that the transfers $V_{H_{1}\to\gamma_{2}(G)}$ and
$V_{H_{2}\to\gamma_{2}(G)}$ are trivial. Moreover $G$ is completely determined
with the isomorphism class groups of maximal class. The theoretical results
are underpinned by numerical examples obtained with the computational number
theory system PARI/GP [15].

##  2 PRELIMINARY

Let $G$ be a metabelian $p$-group of order $p^{n}$, $n\geq 3$, with
abelianization $G/\gamma_{2}(G)$ is of type $(p,p)$, where
$\gamma_{2}(G)=[G,G]$ is the commutator group of $G$. The subgroup $G^{p}$ of
$G$, generated by the $p^{th}$ powers is contained in $\gamma_{2}(G)$, which
therefore coincides with the Frattini subgroups
$\phi(G)=G^{p}\gamma_{2}(G)=\gamma_{2}(G)$. According to the basis theorem of
Burnside, the group $G$ can thus be generated by two elements $x$ and $y$,
$G=<x,y>$. If we declare the lower central series of $G$ recursively by

$\begin{cases}\gamma_{1}(G)=G\\\
\gamma_{j}(G)=[\gamma_{j-1}(G),G]\,\,\text{for}\,\,j\geq 2,\\\ \end{cases}$

Then we have Kaloujnine’s commutator relation
$[\gamma_{j}(G),\gamma_{l}(G)]\subseteq\gamma_{j+l}(G)$, for $j,l\geq 1$, and
for an index of nilpotence $c\geq 2$ the series

$G=\gamma_{1}(G)\supset\gamma_{2}(G)\supset.....\supset\gamma_{c-1}(G)\supset\gamma_{c}(G)=1$

becomes stationary.
The two-step centralizer

$\chi_{2}(G)=\\{g\in G\,|\,[g,u]\in\gamma_{4}(G)\text{for all
u}\in\gamma_{2}(G)\\}$

of the two-step factor group $\gamma_{2}(G)/\gamma_{4}(G)$, that is the
largest subgroup of $G$ such that
$[\chi_{2}(G),\gamma_{2}(G)]\subset\gamma_{4}(G)$. It is characteristic,
contains the commutator subgroup $\gamma_{2}(G)$. Moreover $\chi_{2}(G)$
coincides with $G$ if and only if $n=3$. For $n\geq 4$, $\chi_{2}(G)$ is one
of the $p+1$ normal subgroups of $G$.
Let the isomorphism invariant $k=k(G)$ of $G$, be defined by
$[\chi_{2}(G),\gamma_{2}(G)]=\gamma_{n-k}(G)$, where $k=0$ for $n=3$ and
$0\leq k\leq n-4$ if $n\geq 4$, also for $n\geq p+1$ we have
$k=min\\{n-4,p-2\\}$.
$k(G)$ provides a measure for the deviation from the maximal degree of
commutativity $[\chi_{2}(G),\gamma_{2}(G)]=1$ and is called _defect of
commutativity_ of $G$.
With a further invariant $e$, it will be expressed, which factor
$\gamma_{j}(G)/\gamma_{j+1}(G)$ of the lower central series is cyclic for the
first time [12], and we have $e+1=min\\{3\leq j\leq
m\,|\,1\leq|\gamma_{j}(G)/\gamma_{j+1}|\leq p\\}$.
In this definition of $e$, we exclude the factor
$\gamma_{2}(G)/\gamma_{3}(G)$, which is always cyclic. The value $e=2$ is
characteristic for a group $G$ of maximal class.

###  2.1 On the $5$-class group of maximal class

Let $G$ be a metabelian $5$-group of order $5^{n},\,n\geq 4$, such that
$G/\gamma_{2}(G)$ is of type $(5,5)$, then $G$ admits six maximal normal
subgroups $H_{1},...,H_{6}$, which contain the commutator group
$\gamma_{2}(G)$ as a normal subgroup of index $5$. We have that $\chi_{2}(G)$
is one of the groups $H_{i}$. We fix $\chi_{2}(G)=H_{1}$. We have the
following theorem

######  Theorem 2.1.

Let $G$ be a metabelian $5$-group of order $5^{n},\,n\geq 4$, with the
abelianization $G/\gamma_{2}(G)$ is of type $(5,5)$ and $k=k(G)$ its invariant
defined before. Assume that $G$ is of maximal class, then $G$ can be generated
by two elements, $G=<x,y>$, be selected such that $x\in G\setminus\chi_{2}(G)$
and $y\in\chi_{2}(G)\setminus\gamma_{2}(G)$. Let $s_{2}=[y,x]\in\gamma_{2}(G)$
and $s_{j}=[s_{j-1},x]\in\gamma_{j}(G)$ for $j\geq 3$. Then we have:

$s_{j}^{5}s_{j+1}^{10}s_{j+2}^{10}s_{j+3}^{5}s_{j+4}=1$ for $j\geq 2$.

$x^{5}=s_{n-1}^{w}$ with $w\in\\{0,1,2,3,4\\}$.

$y^{5}s_{2}^{10}s_{3}^{10}s_{4}^{5}s_{5}=s_{n-1}^{z}$ with
$z\in\\{0,1,2,3,4\\}$.

$[y,s_{2}]=\prod\limits_{i=1}^{k}s_{n-i}^{a_{n-i}}$ with
$a=(a_{n-1},...a_{n-k})$ exponents such that $0\leq a_{n-i}\leq 4$.

###### Proof.

See [[11], Theorem 2] for $p=5$. ∎

The six maximal normal subgroups $H_{1}....H_{6}$ are arranged as follow:
$H_{1}=\langle y,\gamma_{2}(G)\rangle=\chi_{2}(G)$, $H_{i}=\langle
xy^{i-2},\gamma_{2}(G)\rangle$ for $2\leq i\leq 6$. The order of the
abelianization of each $H_{i}$, for $1\leq i\leq 6$, is given by the following
theorem.

######  Theorem 2.2.

Let $G$, $H_{i}$ and the invariant $k$ as before. Then for $1\leq i\leq 6$,
the order of the commutator factor groups of $H_{i}$ is given by:

If $n=2$ we have : $|H_{i}/\gamma_{2}(H_{i})|=5$ for $1\leq i\leq 6$.

If $n\geq 3$ we have : $|H_{i}/\gamma_{2}(H_{i})|=5^{2}$ for $2\leq i\leq 6$,
and $|H_{1}/\gamma_{2}(H_{1})|=5^{n-k-1}$

###### Proof.

See [[10], Theorem 3.1] for $p=5$. ∎

######  Lemma 2.1.

Let $G$ be a $5$-group of order $|G|=5^{n},\,n\geq 4$. Assume that the
commutator group $G/\gamma_{2}(G)$ is of type $(5,5)$. Then $G$ is of maximal
class if and only if $G$ admits a maximal normal subgroup with factor
commutator of order $5^{2}$. Furthermore $G$ admits at least five maximal
normal subgroups with factor commutator of order $5^{2}$.

###### Proof.

Assume that $G$ is of maximal class, then by theorem 2.2, we conclude that $G$
has five maximal normal subgroups with the order of commutator factor is
$5^{2}$ if $n\geq 4$, and has six when $n=3$. Conversely, Assume that
$cc(G)\geq 2$, the invariant $e$ defined before is greater than 3, and since
each maximal normal subgroup $H$ of $G$ verify $|H/\gamma_{2}(H)|\geq 5^{e}$
we get that $|H/\gamma_{2}(H)|>5^{2}$ ∎

###  2.2 On the transfer concept

Let $G$ be a group and let $H$ be a subgroup of $G$. The transfer from $G$ to
$H$ can be decomposed as follows: Also we note $\bar{V}$ instead of $V_{G\to
H}$.

$G$$H/\gamma_{2}(H)$ $G/\gamma_{2}(G)$ Figure 1: Transfer diagram$\bar{V}$

######  Definition 2.1.

Let $G$ be a group, $H$ be a normal subgroup of $G$, and let $g\in G$ such
that, $f$ is the order of $gH$ in $G/H$, $r=\frac{[G:H]}{f}$ and
$g_{1},....g_{r}$ be a representative system of $G/H$, then the transfer from
$G$ to $H$, noted $V_{G\to H}$, is defined by:

$V_{G\to H}$ : $G/\gamma_{2}(G)\longrightarrow H/\gamma_{2}(H)$

$g\gamma_{2}(G)\longrightarrow\prod_{i=1}^{r}g_{i}^{-1}g^{f}g_{i}\gamma_{2}(H)$

In the special case that $G/H$ is cyclic group of order $5$ and $G=\langle
h,H\rangle$, then the transfer $V_{G\to H}$ is given as:

$(1)$ If $g\in H$; then $V_{G\to
H}(g\gamma_{2}(G))=g^{1+h+h^{2}+h^{3}+h^{4}}\gamma_{2}(H)$

$(2)$ $V_{G\to H}(h\gamma_{2}(G))=h^{5}\gamma_{2}(H)$

##  3 MAIN RESULTS

In this section we investigate the purely group theoretic results to determine
the invariants of metabelian $5$-group of maximal class developed in theorem
2.1. Furthermore we show that a such metabelian $5$-group is realized by the
Galois group of some fields tower.

###  3.1 Invariants of metabelian $5$-group of maximal class

In this paragraph, we keep the same hypothesis on the group $G$ and the
generators $G=\langle x,y\rangle$, such that $x\in G\setminus\chi_{2}(G)$ and
$y\in\chi_{2}(G)\setminus\gamma_{2}(G)$. The six maximal normal subgroups of
$G$ are as follows: $H_{1}=\chi_{2}(G)=\langle y,\gamma_{2}(G)\rangle$ and
$H_{i}=\langle xy^{i-2},\gamma_{2}(G)\rangle$ for $2\leq i\leq 6$.
In the case that the transfers from two subgroups $H_{i}$ and $H_{j}$ to
$\gamma_{2}(G)$ are trivial, we can determine completely the $5$-group $G$.

######  Proposition 3.1.

Let $G$ be a metabelian $5$-group of maximal class of order $5^{n},\,n\geq 4$.
If the transfers $V_{\chi_{2}(G)\to\gamma_{2}(G)}$ and
$V_{H_{2}\to\gamma_{2}(G)}$ are trivial, then $n\leq 6$ and $\gamma_{2}(G)$ is
of exponent $5$. Furthermore:
\- If $n=6$ then $G\sim G_{a}^{(6)}(1,0)$ where $a=0$ or $1$.
\- If $n=5$ then $G\sim G_{a}^{(5)}(0,0)$ where $a=0$ or $1$.
\- If $n=4$ then $G\sim G_{0}^{(4)}(0,0)$.

###### Proof.

Assume that $n\geq 7$, then $\gamma_{5}(G)=\langle
s_{5},\gamma_{6}(G)\rangle$, because $G$ is of maximal class and
$|\gamma_{5}(G)/\gamma_{6}(G)|=5$. By [[1], lemma 3.3] we have
$y^{5}s_{5}\in\gamma_{6}(G)$, thus $\gamma_{5}(G)=\langle
s_{5}^{4},\gamma_{6}(G)\rangle=\langle
y^{5}s_{5}s_{5}^{4},\gamma_{6}(G)\rangle=\langle y^{5},\gamma_{6}(G)\rangle$,
and since $V_{\chi_{2}(G)\to\gamma_{2}(G)}(y)=y^{5}=1$, because the transfers
are trivial by hypothesis, we get that
$\gamma_{5}(G)=\gamma_{6}(G)$, which is impossible, whence $n\leq 6$ and
According to [[1], lemma 3.2], $\gamma_{2}(G)$ is of exponent $5$.
If $n=6$, we have $V_{\chi_{2}(G)\to\gamma_{2}(G)}$ and
$V_{H_{2}\to\gamma_{2}(G)}$ are trivial, so by theorem 2.1 we obtain
$x^{5}=s_{5}^{w}=1$ which imply $w=0$, because $0\leq w\leq 4$. Since
$\gamma_{2}(G)$ is of exponent $5$, we have $s_{2}^{5}=1$ and by theorem 2.1
the relation $s_{4}^{5}s_{5}^{10}s_{6}^{10}s_{7}^{5}s_{8}=1$ gives
$s_{4}^{5}=1$, also $s_{3}^{5}s_{4}^{10}s_{5}^{10}s_{6}^{5}s_{7}=1$ gives
$s_{3}^{5}=1$. We replace in
$y^{5}s_{2}^{10}s_{3}^{10}s_{4}^{5}s_{5}=s_{5}^{z}$ and we get
$s_{5}=s_{5}^{z}$, whence $z=1$. We have
$[\chi_{2}(G),\gamma_{2}(G)]\subset\gamma_{6-k}(G)\subset\gamma_{4}(G)$ then
$6-k\geq 4$, and $0\leq k\leq 2$, thus $[y,s_{2}]=s_{4}^{\alpha\beta}$,
$a=(\alpha,\beta)$. If $k=0$, then $a=0$ and $G\sim G_{0}^{(6)}(1,0)$, if
$k=1$ then $a=1$ and $G\sim G_{1}^{(6)}(1,0)$ and if $k=2$ then $G\sim
G_{a}^{(6)}(1,0)$.
If $n=5$, we have
$[\chi_{2}(G),\gamma_{2}(G)]\subset\gamma_{5-k}(G)\subset\gamma_{4}(G)$ then
$5-k\geq 4$, and $0\leq k\leq 1$. We have $s_{4}^{5}=1$,
$s_{2}^{5}=s_{3}^{5}=1$ and $[y,s_{2}]=s_{4}^{a}$. the relation
$y^{5}s_{2}^{10}s_{3}^{10}s_{4}^{5}s_{5}=s_{4}^{z}$ imply $s_{4}^{z}=1$ so
$z=0$. As $n=6$ we obtain $w=0$. If $k=0$ then $G\sim G_{0}^{(5)}(0,0)$ and if
$k=1$ $G\sim G_{a}^{(5)}(0,0)$.
If $n=4$, Since
$[\chi_{2}(G),\gamma_{2}(G)]\subset\gamma_{5-k}(G)\subset\gamma_{4}(G)$ we
have $4-k\geq 4$, and $k=0$, thus $[y,s_{2}]=1$, i.e $a=0$. By the same way in
this case we have $w=z=0$, therefor $G\sim G_{0}^{(4)}(0,0)$. ∎

######  Proposition 3.2.

Let $G$ be a metabelian $5$-group of maximal class of order $5^{n}$. If the
transfers $V_{H_{2}\to\gamma_{2}(G)}$ and $V_{H_{i}\to\gamma_{2}(G)}$, $3\leq
i\leq 6$, are trivial, then we have:
\- If $n=5$ or $6$ then $G\sim G_{a}^{(n)}(0,0)$.
\- If $n\geq 7$ then $G\sim G_{0}^{(n)}(0,0)$ .

###### Proof.

If $n=5$ or $6$, by [[1], theorem 1.6] we have $[\chi_{2}(G),\gamma_{2}(G)]=1$
and $[\chi_{2}(G),\gamma_{2}(G)]\subset\gamma_{4}(G)$ elementary, and
$(\gamma_{2}(\chi_{2}(G)))^{5}=1$ and
$\prod_{i=2}^{3}[\gamma_{i}(G),\gamma_{4}(G)]=1$, we conclude that
$(xy)^{5}=x^{5}y^{5}s_{2}^{10}s_{3}^{10}s_{4}^{5}s_{5}$ and we have
$y^{5}s_{2}^{10}s_{3}^{10}s_{4}^{5}s_{5}=s_{n-1}^{z}$ then
$(xy)^{5}=x^{5}s_{n-1}^{z}$ and since $V_{H_{2}\to\gamma_{2}(G)}$ and
$V_{H_{3}\to\gamma_{2}(G)}$ are trivial then
$(xy)^{5}=x^{5}=s_{n-1}^{z}=s_{n-1}^{w}=1$, thus $z=w=0$. Since
$[\chi_{2}(G),\gamma_{2}(G)]=\gamma_{n-k}\subset\gamma_{4}(G)$ we have
$n-k\geq 4$, whence $0\leq k\leq 2$ because $n=5$ or $6$ then $G\sim
G_{a}^{(n)}(0,0)$.
If $n\geq 7$, according to corollary page 69 of [1] we have,
$(\gamma_{j}(\chi_{2}(G)))^{5}=\gamma_{j+4}(G)$ for $j\geq 2$, and since
$y^{5}s_{2}^{10}s_{3}^{10}s_{4}^{5}s_{5}=s_{n-1}^{z}$ we obtain:

$y^{5}=s_{n-1}^{z}s_{5}^{-1}s_{4}^{-1}s_{3}^{-10}s_{2}^{-10}\equiv
s_{n-1}^{z}s_{5}^{-1}\,\mathrm{mod\gamma_{6}(G)}$

because $s_{2}^{5}\in\gamma_{6}(G)$, $s_{3}^{5}\in\gamma_{6}(G)$ and
$s_{4}^{5}\in\gamma_{6}(G)$, and since $n\geq 7$ we have
$s_{n-1}\in\gamma_{6}(G)$, therefor $V=V_{H_{3}\to\gamma_{2}(G)}(y)\equiv
s_{5}^{-1}\,\mathrm{mod\gamma_{6}(G)}$. Thus Im$(V)\subset\gamma_{5}(G)$, In
fact Im$(V)=\gamma_{5}(G)$, and also we have $y\notin\mathrm{ker}(V)$ and
$\forall f\geq 2$ $y^{k}s_{f}^{l}\notin\mathrm{ker}(V)$. The kernel of $V$ is
formed by elements of $\gamma_{2}(G)$ of exponent $5$, its exactly
$\gamma_{n-4}(G)$, and since $G$ is of maximal class then the rank of
$\gamma_{2}(G)$ is $2$ and $\gamma_{2}(G)$ admits exactly $25$ elements of
exponent $5$, these elements form $\gamma_{n-4}(G)$. We conclude that
$|\chi_{2}(G)/\gamma_{2}(\chi_{2}(G))|=|\gamma_{n-4}(G)|\times|\gamma_{5}(G)|=5^{4}\times
5^{n-5}=5^{n-1}=|\chi_{2}(G)|$, whence $\chi_{2}(G)$ is abelian because
$\gamma_{2}(\chi_{2}(G))=1$, consequently $[y,s_{2}]=1$, thus $a=0$. As the
cases $n=5$ or $6$ we obtain $(xy)^{5}=x^{5}s_{n-1}^{z}$, therefor $z=w=0$,
hence $G\sim G_{0}^{(n)}(0,0)$.
In the case when $V_{H_{2}\to\gamma_{2}(G)}$ and $V_{H_{i}\to\gamma_{2}(G)}$,
$4\leq i\leq 6$ are trivial, according to [[1], theorem 1.6] we have
$(xy^{\mu})^{5}=x^{5}(y^{5}s_{2}^{10}s_{3}^{10}s_{4}^{5}s_{5})^{\mu}=s_{n-1}^{w}s_{n-1}^{\mu
z}$ with $\mu=2,3,4$, then we can admit the same reasoning to prove the
result. ∎

######  Proposition 3.3.

Let $G$ be a metabelian $5$-group of maximal class of order $5^{n}$. If the
transfers $V_{H_{i}\to\gamma_{2}(G)}$ and $V_{H_{j}\to\gamma_{2}(G)}$, where
$i,j\in\\{3,4,5,6\\}$ and $i\neq j$, are trivial, then we have: $G\sim
G_{0}^{(n)}(0,0)$.

###### Proof.

Assume that $H_{i}=\langle xy^{\mu_{1}},\gamma_{2}(G)\rangle$ and
$H_{j}=\langle xy^{\mu_{2}},\gamma_{2}(G)\rangle$ where
$\mu_{1},\mu_{2}\in\\{1,2,3,4\\}$ and $\mu_{1}\neq\mu_{2}$. According to [[1],
theorem 1.6] we have already prove that
$(xy^{\mu_{1}})^{5}=s_{n-1}^{w+\mu_{1}z}$ and
$(xy^{\mu_{2}})^{5}=s_{n-1}^{w+\mu_{2}z}$. Since $V_{H_{i}\to\gamma_{2}(G)}$
and $V_{H_{j}\to\gamma_{2}(G)}$ are trivial, we obtain
$s_{n-1}^{w+\mu_{1}z}=s_{n-1}^{w+\mu_{2}z}=1$ then $w+\mu_{1}z\equiv
w+\mu_{2}z\equiv 0\,(\mathrm{mod}\,5)$ and since $5$ does not divide
$\mu_{1}-\mu_{2}$ we get $z=0$ and at the same time $w=0$. To prove $a=0$ we
admit the same reasoning as proposition 3.2. ∎

###  3.2 APPLICATION

Through this section we denote by:

  * -

$p$ a prime number such that $p\equiv-1(\mathrm{mod}\,25)$.

  * -

$k_{0}=\mathbb{Q}(\zeta_{5})$ the $5^{th}$ cyclotomic field,
$(\zeta_{5}=e^{\frac{2\pi i}{5}}).$

  * -

$k=k_{0}(\sqrt[5]{p})$ a cyclic Kummer extension of $k_{0}$ of degree $5$.

  * -

$C_{k,5}$ the $5$-ideal class group of $k$.

  * -

$k^{*}=(k/k_{0})^{*}$ the relative genus field of $k/k_{0}$.

  * -

$F_{5}^{(1)}$ the absolute Hilbert $5$-class field of a number field $F$.

  * -

$G=\mathrm{Gal}\left((k^{*})_{5}^{(1)}/k_{0}\right)$.

We begin by the following theorem.

######  Theorem 3.1.

Let $k=\mathbb{Q}(\sqrt[5]{p},\zeta_{5})$ be the normal closure of a pure
quintic field $\mathbb{Q}(\sqrt[5]{p})$, where $p$ a prime congruent to $-1$
modulo 25. Let $k_{0}$ be the the $5^{th}$ cyclotomic field. Assume that the
$5$-class group $C_{k,5}$ of $k$, is of type $(5,5)$, then $Gal(k^{*}/k_{0})$
is of type $(5,5)$, and two sub-extensions of $k^{*}/k_{0}$ admit a trivial
$5$-class number.

###### Proof.

By $C_{k,5}^{(\sigma)}$ we denote the subgroup of ambiguous ideal classes
under the action of $Gal(k/k_{0})=\langle\sigma\rangle$. According to [[6],
theorem 1.1], in this case of the prime $p$ we have rank
$C_{k,5}^{(\sigma)}=1$, and by class field theory, since
$[k^{*}:k]=|C_{k,5}^{(\sigma)}|$, we have that $k^{*}/k$ is a cyclic quintic
extension, whence $Gal(k^{*}/k_{0})$ is of type $(5,5)$.
Since $p\equiv-1(\mathrm{mod}\,25)$, then $p$ splits in $k_{0}$ as
$p=\pi_{1}\pi_{2}$, where $\pi_{1},\,\pi_{2}$ are primes of $k_{0}$. By [[7],
theorem 5.15] we have explicitly the relative genus field $k^{*}$ as
$k^{*}=k(\sqrt[5]{\pi_{1}^{a_{1}}\pi_{2}^{a_{2}}})=k_{0}(\sqrt[5]{\pi_{1}\pi_{2}},\sqrt[5]{\pi_{1}^{a_{1}}\pi_{2}^{a_{2}}})$
with $a_{1},a_{2}\in\\{1,2,3,4\\}$ such that $a_{1}\neq a_{2}$. Its clear that
the extension $k^{*}/k_{0}$ admits six sub-extensions, where $k$ is one of
them, and the others are $k_{0}(\sqrt[5]{\pi_{1}^{a_{1}}\pi_{2}^{a_{2}}})$,
$k_{0}(\sqrt[5]{\pi_{1}^{a_{1}+1}\pi_{2}^{a_{2}+1}})$,
$k_{0}(\sqrt[5]{\pi_{1}^{a_{1}+2}\pi_{2}^{a_{2}+2}})$,
$k_{0}(\sqrt[5]{\pi_{1}^{a_{1}+3}\pi_{2}^{a_{2}+3}})$ and
$k_{0}(\sqrt[5]{\pi_{1}^{a_{1}+4}\pi_{2}^{a_{2}+4}})$. Since
$a_{1},a_{2}\in\\{1,2,3,4\\}$, we can see that the extensions
$L_{1}=k_{0}(\sqrt[5]{\pi_{1}})$ and $L_{2}=k_{0}(\sqrt[5]{\pi_{2}})$ are sub-
extensions of $k^{*}/k_{0}$.
In [[7], section 5.1], we have an investigation of the rank of ambiguous
classes of $k_{0}(\sqrt[5]{x})/k_{0}$, denoted $t$. We have $t=d+q^{*}-3$,
where $d$ is the number of prime divisors of $x$ in $k_{0}$, and $q^{*}$ an
index defined as [[7], section 5.1]. For the extensions $L_{i}/k_{0}$,
$(i=1,2)$, we have $d=1$ and by [[7], theorem 5.15] we have $q^{*}=2$, hence
$t=0$.
By $h_{5}(L_{i})$, $(i=1,2)$, we denote the class number of $L_{i}$, then we
have $h_{5}(L_{1})=h_{5}(L_{2})=1$. Otherwise $h_{5}(L_{i})\neq 1$, then there
exists an unramified cyclic extension of $L_{i}$, denoted $F$. This extension
is abelian over $k_{0}$, because $[F:k_{0}]=5^{2}$, then $F$ is contained in
$(L_{i}/k_{0})^{*}$ the relative genus field of $L_{i}/k_{0}$. Since
$[(L_{i}/k_{0})^{*}:L_{i}]=5^{t}=1$, we get that $(L_{i}/k_{0})^{*}=L_{i}$,
which contradicts the existence of $F$. Hence the $5$-class number of $L_{i}$,
$(i=1,2)$, is trivial. ∎

In what follows, we denote by $L_{1}$ and $L_{2}$ the two sub-extensions of
$k^{*}/k_{0}$, which verify theorem 3.1, and by $\tilde{L}$ the three
remaining sub-extensions different to $k$. Let
$G=Gal((k^{*})_{5}^{(1)}/k_{0})$, we have
$\gamma_{2}(G)=Gal((k^{*})_{5}^{(1)}/k^{*})$, then
$G/\gamma_{2}(G)=Gal(k^{*}/k_{0})$ is of type $(5,5)$, therefore $G$ is
metabelian $5$-group with factor commutator of type $(5,5)$, thus $G$ admits
exactly six maximal normal subgroups as follows:

$H=Gal((k^{*})_{5}^{(1)}/k)$, $H_{L_{i}}=Gal((k^{*})_{5}^{(1)}/L_{i})$,
$(i=1,2)$, $\tilde{H}=Gal((k^{*})_{5}^{(1)}/\tilde{L})$

. With $\chi_{2}(G)$ is one of them.
Now we can state our principal result.

######  Theorem 3.2.

Let $G=Gal((k^{*})_{5}^{(1)}/k_{0})$ be a $5$-group of order $5^{n}$, $n\geq
4$, then $G$ is a metabelian of maximal class. Furthermore we have:
\- If $\chi_{2}(G)=H_{L_{i}}(i=1,2)$ then: $G\sim G_{a}^{(n)}(z,0)$ with
$n\in\\{4,5,6\\}$ and $a,z\in\\{0,1\\}$.
\- If $\chi_{2}(G)=\tilde{H}$ then : $G\sim G_{1}^{(n)}(0,0)$ with $n=5$ or
$6$.

………… $G\sim G_{0}^{(n)}(0,0)$ with $n\geq 7$ such that $n=s+1$ where
$h_{5}(\tilde{L})=5^{s}$.

###### Proof.

Let $G=Gal((k^{*})_{5}^{(1)}/k_{0})$ and $H=Gal((k^{*})_{5}^{(1)}/k)$ its
maximal normal subgroup, then
$\gamma_{2}(H)=Gal((k^{*})_{5}^{(1)}/k_{5}^{(1)})$, therefor
$H/\gamma_{2}(H)=Gal(k_{5}^{(1)}/k)\simeq C_{k,5}$, and as $C_{k,5}$ is of
type $(5,5)$ by hypothesis we get that $|H/\gamma_{2}(H)|=5^{2}$. Lemma 2.1
imply that $G$ is a metabelian $5$-group of maximal class, generated by two
elements $G=\langle x,y\rangle$, such that, $x\in G\setminus\chi_{2}(G)$ and
$y\in\chi_{2}(G)\setminus\gamma_{2}(G)$. Since $\chi_{2}(G)=\langle
y,\gamma_{2}(G)\rangle$, we have $\chi_{2}(G)\neq H$. Otherwise we get that
$|H/\gamma_{2}(H)|=5^{2}$ which contradict theorem 2.1.
According to theorem 3.1, we have $h_{5}(L_{1})=h_{5}(L_{2})=1$ then the
transfers $V_{H_{L_{i}}\to\gamma_{2}(G)}$ are trivial.
If $\chi_{2}(G)=H_{L_{i}}$ the results are nothing else than proposition 3.1.
If $\chi_{2}(G)=\tilde{H}$ and $n=4$ then $\gamma_{4}(G)=1$ and
$[\chi_{2}(G),\gamma_{2}(G)]=\gamma_{2}(\tilde{H})$, also
$[\chi_{2}(G),\gamma_{2}(G)]=\gamma_{4}(G)=1$ then $\chi_{2}(\tilde{H})=1$,
whence $\tilde{H}$ is abelian. Consequently
$\tilde{H}/\gamma_{2}(\tilde{H})=C_{\tilde{L},5}$, so
$h_{5}(\tilde{L})=|\tilde{H}|=5^{3}$ because its a maximal subgroup of $G$.
Since $\tilde{L}$ and $k$ have always the same conductor, we deduce that
$h_{5}(k)$ and $h_{5}(\tilde{L})$ verify the relations
$5^{5}h_{\tilde{L}}=uh_{\Gamma}^{4}$ and $5^{5}h_{k}=uh_{\Gamma}^{4}$, given
by C. Parry in [14], where $u$ is a unit index and a divisor of $5^{6}$. Using
the $5$-valuation on these relations we get that $h_{5}(\tilde{L})=5^{s}$
where $s$ is even, which contradict the fact that $h_{5}(\tilde{L})=5^{3}$,
hence $n\geq 5$.
The results of the theorem are exactly application of propositions 3.2, 3.3.
According to proposition 3.2, if $n\geq 7$ we have $|\chi_{2}(G)|=5^{n-1}$ and
since
$h_{5}(\tilde{L})=|\tilde{H}/\gamma_{2}(\tilde{H})|=|\tilde{H}|=5^{n-1}=5^{s}$
we deduce that $n=s+1$. ∎

##  4 Numerical examples

For these numerical examples of the prime $p$, we have that $C_{k,5}$ is of
type $(5,5)$ and rank $C_{k,5}^{(\sigma)}\,=\,1$, which mean that $k^{*}$ is
cyclic quintic extension of $k$, then by theorem 3.2 we have a completely
determination of $G$. We note that the absolute degree of $(k^{*})_{5}^{(1)}$
surpass $100$, then the task to determine the order of $G$ is definitely far
beyond the reach of computational algebra systems like MAGMA and PARI/GP.

Table 1: $k\,=\,\mathbb{Q}(\sqrt[5]{p},\zeta_{5})$ with $C_{k,5}$ is of type
$(5,5)$ and rank $C_{k,5}^{(\sigma)}\,=\,1$.

$p$ | $p\,(\mathrm{mod}\,25)$ | $h_{k,5}$ | $C_{k,5}$ | rank $(C_{k,5}^{(\sigma)})$
---|---|---|---|---
149 | -1 | 25 | $(5,5)$ | 1
199 | -1 | 25 | $(5,5)$ | 1
349 | -1 | 25 | $(5,5)$ | 1
449 | -1 | 25 | $(5,5)$ | 1
559 | -1 | 25 | $(5,5)$ | 1
1249 | -1 | 25 | $(5,5)$ | 1
1499 | -1 | 25 | $(5,5)$ | 1
1949 | -1 | 25 | $(5,5)$ | 1
1999 | -1 | 25 | $(5,5)$ | 1
2099 | -1 | 25 | $(5,5)$ | 1

## References

  * [1] N.Blackburn, _On a special class of p-groups_ , Acta Math. 100 (1958), 45-92.
  * [2] H.Dietrich, B.Eick, and D.Feichtenschlager. _Investigating p-groups by coclass with GAP_. Contemp. Math. AMS 470, 45-61, 2008.
  * [3] H.Dietrich. _Periodic patterns in the graph of p-groups of maximal class._ J. Group Theory 13, 851-871, 2010.
  * [4] H.Dietrich. _A new periodic pattern in the graph of p-groups of maximal class._ Bull. London Math. Soc. 42, 1073-1088, 2010.
  * [5] B.Eick and C.Leedham-Green, _Classification of prime-power groups by coclass_ , Bull. London Math. Soc. 40, 274-288, 2008.
  * [6] F.Elmouhib, M.Talbi, and A.Azizi, _5-rank of ambiguous class groups of quintic Kummer extensions_ , Accepted for publication in Proceedings-Mathematical Sciences.
  * [7] M.Kulkarni, D.Majumdar, B.Sury, _$l$ -class groups of cyclic extension of prime degree $l$_, J. Ramanujan Math. Soc. 30, No.4 (2015), 413-454.
  * [8] C.R.Leedham-Green and S.McKay. _On the classification of p-groups of maximal class_. Quart. J. Math. Oxford Ser. (2) 35, 293-304, 1984.
  * [9] D.C.Mayer, _Transfers of metabelian p-groups_ , Monatsh. Math. 166 (2012), no. 3-4, 467-495.
  * [10] D.C.Mayer, _The second $p$-class group of a number field_, Int. J. Number Theory 8 (2012), no. 2, 471-505.
  * [11] R.J.Miech, _Metabelian $p$-groups of maximal class_, Trans. Amer. Math. Soc. 152 (1970), 331-373.
  * [12] B.Nebelung, _Klassiffication metabesher 3-gruppen mit Faktorkommutatogruppe von typ $(3,3$) und anwendung auf das Kapitulationsproblem_, ThÃ¨se de doctorat (1989), Kolon.
  * [13] M.F.Newman. _Groups of prime-power order_. Groups Canberra 1989, 49-62, Lecture Notes in Math. 1456, Springer, Berlin, 1990.
  * [14] C. Parry, _Class number relations in pure quintic fields_ , Symposia Mathematica. 15 (1975), 475-485.
  * [15] The PARI Group, PARI/GP, Version 2.4.9, Bordeaux, 2017, http://pari.math.u-bordeaux.fr.


# Galactic ChitChat: Using Large Language Models to Converse with Astronomy
Literature

Ioana Ciucă
Research School of Astronomy & Astrophysics, Australian National University,
Cotter Rd., Weston, ACT 2611, Australia.

School of Computing, Australian National University,
Acton, ACT 2601, Australia.
Yuan-Sen Ting
Research School of Astronomy & Astrophysics, Australian National University,
Cotter Rd., Weston, ACT 2611, Australia.

School of Computing, Australian National University,
Acton, ACT 2601, Australia.

###### Abstract

We demonstrate the potential of the state-of-the-art OpenAI GPT-4 large
language model to engage in meaningful interactions with Astronomy papers
using in-context prompting. To optimize for efficiency, we employ a
distillation technique that effectively reduces the size of the original input
paper by 50%, while maintaining the paragraph structure and overall semantic
integrity. We then explore the model’s responses using a multi-document
context (ten distilled documents). Our findings indicate that GPT-4 excels in
the multi-document domain, providing detailed answers contextualized within
the framework of related research findings. Our results showcase the potential
of large language models for the astronomical community, offering a promising
avenue for further exploration, particularly the possibility of utilizing the
models for hypothesis generation.

Computationalmethods(1965)---Galacticarchaeology(2178)

††software: OpenAI GPT-4 API (OpenAI, 2023), langchain (Chase, 2022), FAISS
(Johnson et al., 2019).

##  1 Introduction

Large language models (LLMs) have significantly advanced natural language
processing, allowing machines to process and generate intricate text with
remarkable quality (e.g., Devlin et al., 2018; Brown et al., 2020; Chowdhery
et al., 2022; Bubeck et al., 2023). In this study, we employ ‘in-context
prompting’ 111While developing this project, we came across paperqa (White,
2023), which performs QA on academic papers, similar to our approach. However,
we employ a distillation technique and consider the chat history. for the SOTA
GPT-4 model (OpenAI, 2023) as a practical solution to tap into the model’s
capabilities without the need for fine-tuning (e.g., Howard & Ruder, 2018; Wei
et al., 2022) or incorporating specialized adapters (e.g, Pfeiffer et al.,
2020; Hu et al., 2021). This technique enables us to harness the potential of
GPT-4, which has not been specifically trained for Astronomy, to engage with
Astronomy literature, a field characterized by its complexity and richness of
information. Our goal for the future is to use LLMs in conjunction with expert
knowledge to identify possible gaps in our knowledge base and generate new
scientific hypotheses in Astronomy.

##  2 Methods

Figure 1: The GPT-4 model responses to expert-level questioning in no-context
vs. multi-document context settings. No-context responses may lack recent
findings and sometimes hallucinate details. For example, Belokurov et al.
(2018) did not use GALAH data. Context enhances performance in summarisation,
study comparisons, and idea generation. For example, the model accurately
identifies that it only has access to Kawata et al. (2018), as shown in
Comparative Analysis III.

We select ten papers in Galactic Archaeology, including Helmi et al. (2018);
Belokurov et al. (2018); Kawata et al. (2018); Massari et al. (2019); Myeong
et al. (2018); Horta et al. (2020); Ciucă et al. (2023). We extract the text
from the PDF and divide it, then pass to the GPT-4 powered ChatGPT (OpenAI,
2023) using a distillation prompt that contains: ‘Distill each paragraph of
the given text, maintaining the same number of paragraphs and structure. Limit
the word count to 50% of the original, and ensure references are included.’ We
essentially ‘compress’ the paper to half its size. Unlike summarisation, this
process ensures that the distilled paper inherits the original paper’s
paragraph structure and associated information. The distillation process
allows us to use more processed papers efficiently for a context window with a
given $\sim$8K token limit.

Our approach employs the langchain (Chase, 2022) framework. The distilled
input is embedded using OpenAI’s text-embedding-ada-002 model and stored in a
large vector store. The expert query and chat history are processed by GPT-4
to generate a standalone question, which is also embedded. We then use FAISS
(Facebook AI Similarity Search) (Johnson et al., 2019) to perform a similarity
search between the embedded question and the input and retrieve a relevant
context for the model. Finally, the GPT-4 model, accessed through the OpenAI
API, uses this context and the standalone question to generate a response. The
model uses the following system prompt: ‘Engage in insightful conversations
with humans, providing meaningful, concise answers based on the provided
documents. Include pertinent study citations, such as Example et al. (2020).’

##  3 Results and conclusion

We explore the performance of GPT-4 when responding to expert-level inquiries
concerning summarization, comparative analysis, and idea generation, as
illustrated in Figure 1. The native model may generate imprecise information
through a phenomenon called ‘hallucination’ (e.g., Shuster et al., 2021; Ji et
al., 2022; Peng et al., 2023), or fail to provide answers when presented with
recent papers beyond its training data. However, when given access to context,
the model’s performance significantly improves, as shown by the different
responses in the first two left panels of Figure 1 between the native and in-
context GPT-4, which we denote by Origins222We provide an example of a
conversation with Origins, the in-galactic-archaeology-context GPT-4 model, at
https://www.youtube.com/watch?v=cufBNDDBgJ4.. The in-context prompting allows
the model to explore connections and differences across multiple papers, as
exemplified by the Comparative Analysis II in Figure 1, where the model
correctly identifies the difference in the focus of the two papers provided.

The Comparative Analysis III in Figure 1 demonstrates that the in-context
GPT-4 model can identify if it has access to a particular paper, indexed by
the first author(s) and year of publication, and then uses that to answer the
question. We recognize the potential for idea generation as shown in the lower
right panel, with the model identifying a possible new link between machine
learning, which was employed to study the Milky Way disc by Ciucă et al.
(2023), with the vast available data for globular clusters.

To conclude, this research note emphasizes the importance of utilizing in-
context prompting with large language models to engage with Astronomy papers
effectively. In the future, we plan to investigate the model’s response
quality as a function of the number of input papers and other variables, such
as the focus of the papers and the broadness of the research. On the technical
aspect, we plan to explore how the responses vary with the distillation level
and how they compare to those from a fine-tuned model.

The authors thank the OpenAI Team for access to the GPT-4 model. IC is
grateful for the Joint Jubilee Fellowship at the ANU. YST acknowledges
financial support from the Australian Research Council through DECRA
Fellowship DE220101520. IC is grateful to Lachezar Todorov, who provided
access to the GPT-4 API and to the langchain community, in particular Hamel
Husain and Mayo Oshin, for their open-source code. IC is grateful to Dr Josh
Peek and Dr Kartheik Iyer for their insightful comments. IC and YST thank the
Kavli Institute for Theoretical Physics, where this research was undertaken as
part of the ‘Building a Physical Understanding of Galaxy Evolution with Data-
driven Astronomy’. This research was in part supported by the National Science
Foundation under Grant No. NSF PHY-1748958.

## References

  * Belokurov et al. (2018) Belokurov, V., Erkal, D., Evans, N. W., Koposov, S. E., & Deason, A. J. 2018, MNRAS, 478, 611, doi: 10.1093/mnras/sty982
  * Brown et al. (2020) Brown, T. B., Mann, B., Ryder, N., et al. 2020, Language Models are Few-Shot Learners.  https://arxiv.org/abs/2005.14165
  * Bubeck et al. (2023) Bubeck, S., Chandrasekaran, V., Eldan, R., et al. 2023, Sparks of Artificial General Intelligence: Early experiments with GPT-4.  https://arxiv.org/abs/2303.12712
  * Chase (2022) Chase, H. 2022, LangChain.  https://github.com/hwchase17/langchain
  * Chowdhery et al. (2022) Chowdhery, A., Narang, S., Devlin, J., et al. 2022, arXiv e-prints, arXiv:2204.02311, doi: 10.48550/arXiv.2204.02311
  * Ciucă et al. (2023) Ciucă, I., Kawata, D., Ting, Y.-S., et al. 2023, MNRAS, doi: 10.1093/mnrasl/slad033
  * Devlin et al. (2018) Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. 2018, arXiv e-prints, arXiv:1810.04805, doi: 10.48550/arXiv.1810.04805
  * Helmi et al. (2018) Helmi, A., Babusiaux, C., Koppelman, H. H., et al. 2018, Nature, 563, 85–88, doi: 10.1038/s41586-018-0625-x
  * Horta et al. (2020) Horta, D., Schiavon, R. P., Mackereth, J. T., et al. 2020, MNRAS, 493, 3363, doi: 10.1093/mnras/staa478
  * Howard & Ruder (2018) Howard, J., & Ruder, S. 2018, arXiv e-prints, arXiv:1801.06146, doi: 10.48550/arXiv.1801.06146
  * Hu et al. (2021) Hu, E. J., Shen, Y., Wallis, P., et al. 2021, arXiv e-prints, arXiv:2106.09685, doi: 10.48550/arXiv.2106.09685
  * Ji et al. (2022) Ji, Z., Lee, N., Frieske, R., et al. 2022, Survey of Hallucination in Natural Language Generation.  https://arxiv.org/abs/2202.03629
  * Johnson et al. (2019) Johnson, J., Douze, M., & Jégou, H. 2019, IEEE Transactions on Big Data, 7, 535
  * Kawata et al. (2018) Kawata, D., Baba, J., Ciucǎ, I., et al. 2018, MNRAS, 479, L108, doi: 10.1093/mnrasl/sly107
  * Massari et al. (2019) Massari, D., Koppelman, H. H., & Helmi, A. 2019, A&A, 630, L4, doi: 10.1051/0004-6361/201936135
  * Myeong et al. (2018) Myeong, G. C., Evans, N. W., Belokurov, V., Sanders, J. L., & Koposov, S. E. 2018, ApJ, 863, L28, doi: 10.3847/2041-8213/aad7f7
  * OpenAI (2023) OpenAI. 2023, ArXiv, abs/2303.08774
  * Peng et al. (2023) Peng, B., Galley, M., He, P., et al. 2023, arXiv e-prints, arXiv:2302.12813, doi: 10.48550/arXiv.2302.12813
  * Pfeiffer et al. (2020) Pfeiffer, J., Vulić, I., Gurevych, I., & Ruder, S. 2020, in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (Online: Association for Computational Linguistics), 7654–7673, doi: 10.18653/v1/2020.emnlp-main.617
  * Shuster et al. (2021) Shuster, K., Poff, S., Chen, M., Kiela, D., & Weston, J. 2021, Retrieval Augmentation Reduces Hallucination in Conversation.  https://arxiv.org/abs/2104.07567
  * Wei et al. (2022) Wei, J., Bosma, M., Zhao, V. Y., et al. 2022, Finetuned Language Models Are Zero-Shot Learners.  https://arxiv.org/abs/2109.01652
  * White (2023) White, A. 2023, Paper QA.  https://github.com/whitead/paper-qa