Somnath Roy

# Understanding the Impact of Post-Training Quantization on Large Language
Models

###### Abstract

Large language models (LLMs) are rapidly increasing in size, with the number
of parameters becoming a key factor in the success of many commercial models,
such as ChatGPT, Claude, and Bard. Even the recently released publicly
accessible models for commercial usage, such as Falcon and Llama2, come
equipped with billions of parameters. This significant increase in the number
of parameters makes deployment and operation very costly. The remarkable
progress in the field of quantization for large neural networks in general and
LLMs in particular, has made these models more accessible by enabling them to
be deployed on consumer-grade GPUs. Quantized models generally demonstrate
comparable performance levels to their unquantized base counterparts.
Nonetheless, there exists a notable gap in our comprehensive understanding of
how these quantized models respond to hyperparameters, such as temperature,
max new tokens, and top_k, particularly for the next word prediction.

The present analysis reveals that nf4 and fp4 are equally proficient 4-bit
quantization techniques, characterized by similar attributes such as inference
speed, memory consumption, and the quality of generated content. Nevertheless,
these quantization methods exhibit distinct behaviors at varying temperature
settings, both in the context of smaller and larger models. Furthermore, the
study identifies nf4 as displaying greater resilience to temperature
variations in the case of the llama_2 series of models at lower temperature,
while fp4 and fp4-dq proves to be a more suitable choice for falcon series of
models. It is noteworthy that, in general, 4-bit quantized models of varying
sizes exhibit higher sensitivity to temperature in the range of 0.5 to 0.8,
unlike their unquantized counterparts. Additionally, int8 quantization is
associated with significantly slower inference speeds, whereas unquantized
bfloat16 models consistently yield the fastest inference speeds across models
of all sizes.

Index Terms: post-training quantization, LLM, nf4, fp4, nf4-dq, fp4-dq

##  1 Introduction

With the emergence of the Transformer architecture [1], a significant
breakthrough was achieved, enabling the effective retention of extensive long-
range dependencies in tasks related to natural language processing, speech,
and vision. The transformer architecture enables highly parallel training due
to sequence parallelism, which makes it possible to pretrain LLMs with
hundreds of billions of parameters [2, 3, 4]. The Big-bench [5] introduced
over 200 benchmarks designed to assess the capabilities of Large Language
Models(LLMs) through quantification and extrapolation. This diverse and
intricately elaborated set of benchmarks significantly contributed to the
intensification of the race surrounding LLM development and advancement.

The widespread adoption of LLMs on a substantial scale gained traction
following the successful establishment of ChatGPT (including GPT-3 and
subsequent iterations) [2]. The pre-training of large transformer language
models with 7 billion parameters and beyond demands a considerable amount of
GPU computation, which can translate to costs amounting to millions of
dollars. Such level of expenditure is beyond what academic research and small
organizations can typically afford. Despite the high cost of deploying and
operating large language models (LLMs), the recent release of the Falcon [6]
and Llama2 [7] models has sparked optimism among small organizations and has
increased their desire to deploy their own custom LLMs.

The efficient deployment of decoder only LLMs are challenging in practice
because the generative inference proceeds sequentially, where the computation
for each token depends on the previously generated tokens [8]. It is
noteworthy that caching the attention key and value tensors of each layer can
significantly improve the inference speed of smaller decoder-only models that
fit on a single GPU memory. However, this is not possible for models that do
not fit into the memory of a single GPU. To address the need for expensive
high-end GPUs to support the deployment of these models, diverse forms of
quantization have been put forward as potential solutions. The application of
quantization methods to transformers emerges as a efficacious approach for
mitigating sampling latency, while incurring minimal to negligible impact on
overall performance [9]. Quantization techniques can be mainly characterized
into three forms namely - i) quantization aware training [10, 11], ii)
quantization aware fine-tuning [12, 13, 14], and iii) post training
quantization (PTQ) [15, 16, 17]. In [18], the investigation primarily centers
on evaluating the impact of diverse post-training quantization methods,
employing perplexity scores as a benchmark. The perplexity scores are computed
on datasets such as Wiki [19], PTB [20], and C4 [21], which mostl likely have
served as foundational datasets during the training of most of the LLMs. It
should be noted that these datasets are predisposed to exhibit favorable
perplexity scores across all models, owing to their utilization in model
training. Furthermore, it is acknowledged that perplexity, as a metric, may
not effectively capture instances of repetitive generation within LLMs.
Following outlines the primary contributions of the present study.

  1. 1.

This study offers a systematic examination of the influence exerted by three
pivotal hyper-parameters, namely, max new tokens, temperature, and top_k, on
LLMs that have undergone quantization through widely adopted post-training
quantization techniques such as [15]111https://github.com/IST-DASLab/gptq
(hereafter, gptq) and [14,
12]222https://github.com/TimDettmers/bitsandbytes333https://github.com/artidoro/qlora
(hereafter, bitsandbytes).

  2. 2.

It explores how these hyper-parameters exert their influence across a range of
model sizes, spanning from 3 billion to 70 billion parameters.

  3. 3.

The process involves generating a total of 6,300 samples for each quantization
method, achieved by constructing ten smaller prompts that encompass a diverse
spectrum of domains for every model.

  4. 4.

LLMs typically exhibit a tendency towards repetitive generation, and it is
often challenging to discern such repetition through perplexity scores.
Therefore, to identify and quantify repetitive generation, the primary metric
employed is the number of duplicate content words.

  5. 5.

It scrutinizes quantization methods that share similar inference speeds but
manifest differing effects on accuracy.

  6. 6.

Finally, it aims to discern the optimal quantization method for deployment,
considering specific constraints and requirements.

##  2 Quantization

Quantization is a well defined mechanism for reducing the number of bits used
to represent a value. In the context of large neural network models,
quantization reduces the precision of the model's parameters and/or
activations. Moreover, it has been found that the quantized large models are
often competitive to its base ones in terms of accuracy while reducing the
computational requirements.
In the context of LLMs, the quantization process can be divided into two types
namely i) simulated and, ii) pure quantization. In simulated quantization,
some operations are performed in floating-point arithmetic, which requires the
dequantization of quantized parameters back to full precision during
inference. [22, 23, 24, 25]. Pure quantization uses integer-only quantization,
which eliminates the need for dequantization during inference [26, 27, 28, 12,
29]. The main difference between these two process of quantization is shown
below in Table 1.

Features |  Simulated Quantization |  Pure Quantization
---|---|---
Operations |  Floating and Fixed point |  Fixed-point
Need for dequantization |  Yes |  No
Inference speed |  Slower |  Comparatively Faster
Table 1: General understanding of simulated vs. pure quantization in
transformer based LLMs

However, it is crucial to note that pure quantization is a more aggressive
approach and can also lead to a greater loss of accuracy. On the other hand,
simulated quantization is a conservative approach and can achieve significant
speedups without sacrificing too much accuracy. Pure quantization can be
further categorized into W8A8 and W4A4, where the weights and activations are
quantized to 8-bit integers and 4-bit integers, respectively [29] [27].

###  2.1 GPTQ

It is a layer-wise quantization method based on the Optimal Brain Quantization
(OBQ) [30]. The goal is to find a quantized weight matrix $\widetilde{W}$ that
minimizes the squared error between the quantized layer output
$\widetilde{W}X$ and the full-precision layer output $WX$ as shown below.

$\underset{\widetilde{W}}{\operatorname{argmin}}\lVert{WX}-\widetilde{W}X\rVert^{2}$

The OBQ algorithm iteratively quantizes one weight at a time, while the GPTQ
algorithm utilizes a vectorized implementation that allows it to efficiently
handle multiple rows of the weight matrix in parallel. This makes GPTQ
significantly faster than OBQ, especially for large models.

####  2.1.1 GPU Memory Consumption in 4-bit GPTQ Quantization

It is well-established that the goal of quantization is to deploy LLMs on
consumer-grade GPUs having at most 24 GB.The distribution of GPU memory
utilised by the models during GPTQ 4-bit quantization is shown below in Table
2. GPTQ quantization has following limitations.

  * •

It is very GPU memory intensive process.

  * •

Even 4-bit quantization of 40B model throws out of memory (OOM) on 80GB A100
GPU machine. Moreover, it is not possible to quantize 7B models on 24GB A10
GPU machines.

Model |  GPU Memory(GB)
---|---
stablelm_3b |  19.54
redpajama_3b |  9.58
falcon_7b |  23.64
llama2_7b |  24.83
llama2_13b |  40.46
falcon_40b and llama2_70b |  OOM on single A100 80GB GPU
Table 2: Distribution of GPU memory consumed by GPTQ 4-bits quantization for
different models evaluated on Nvidia A100 80GB GPU machine

####  2.1.2 Layerwise Error induced by GPTQ

GPTQ 4-bit quantization reduces the size of a model by more than 80%, i.e., a
model of 14 GB is reduced to around 2 GB post quantization. It is important to
note that the quantization error introduced by GPTQ is different for different
models, as shown in Table 3. This is because the models shown in Table 3 have
different architectures, including the number of heads, number of layers,
embedding dimension, number of query groups in multi-query attention, block
size, and hidden dimension.

Model | mlp.proj | att.proj | mlp.fc | attn.attn
---|---|---|---|---
stablelm_3b | 52850.7 | 12638.9 | 383200.9 | 844806.3
redpajama_3b | 23448.1 | 1048.9 | 137061.9 | 138947.9
falcon_7b | 19194.83 | 2362.39 | 149962.4 | 32886.3
llama2_7b | 22773.0 | 3198.7 | 170837.6 | 248520.0
llama2_13b | 27829.5 | 5470.5 | 247389.2 | 301002.0
Table 3: Quantization error introduced by GPTQ in mlp projection, attention
projection, fully connected and attention layers.

###  2.2 bitsandbytes Quantizations

bitsandbytes (bnb) provides implementation of five powerful and state-of-the-
art quantization techniques namely i) int8, ii) fp4, iii) nf4, iv) fp4-dq444dq
stands for double quantization, and v) nf4-dq. The int8 quantization
procedure[14] uses vector-wise quantization with separate normalization
constants for each inner product in the matrix multiplication. However, they
have found around 0.1% dominant activation outliers that has the potential to
degrade the quality especially in bigger LLMs. Therefore, the precision for
these dominant outliers are kept in float16. This scheme isolates the outlier
feature dimensions into a 16-bit matrix multiplication, while still allowing
more than 99.9% of the values to be multiplied in 8 bits.

QLoRA[12] introduced a new data type called 4-bit normal-float (nf4), which is
optimal for normally distributed weights, double quantization to reduce the
memory footprint, and paged optimization to manage memory spikes. These
techniques together yield excellent inference speed without sacrificing the
quality of generation. In nf4 quantization, the base model weights are stored
in nf4 data type and computation is performed in bfloat16. However, the model
weights are dequantized to bfloat16 in the foward pass for inference [31]. The
bnb quantizations compress the model footprint in the range of 40% (int8) to
70% (nf4-dq). It is important to emphasize here that int8 quantization for
llama2_70B throws OOM error on A100 80GB GPU machine. Rest of the details of
compressed model size corresponding to bnb quantizations are described in the
following sections.

##  3 Experiment

This section provides a detailed description of the models, prompts, decoding
approach, and related hyper-parameters used to generate the data for the
analysis.

###  3.1 Model Description

A total of six pre-trained models with 3 billion to 70 billion parameters were
selected for next-word prediction. These models are decoder-only, and their
architecture-specific details are shown in Table 4. As can be seen, these
models differ from each other in terms of the number of heads, number of
layers, embedding dimension, number of query groups used in multi-query
attention, sequence length, and intermediate size.

###  3.2 Prompts Selection and Proposed Hypothesis

Ten prompts are designed to access the quality and inference speed of pre-
trained models for next word generation. These prompts are selected on the
simple proposed hypothesis and shown below in Table 5.

Hypothesis 1: All pre-trained LLMs trained on billions or trillions of tokens
can be ideally conceptualized as a large tree, where each node represents a
topic and the text continuations associated with that topic. As we traverse
down the tree, the text continuations become more specific and focused.
Conversely, as we traverse up the tree, the text continuations become more
general and abstract.

Hypothesis 2: The quality of a pre-trained model can be assessed based on its
ability to accurately identify the correct topic node and then traverse to the
sub-topic node for focused next word prediction.

Model | n_head | n_layer | embed_dim | n_query_groups_mqa | block_size | intermediate_size
---|---|---|---|---|---|---
stablelm_3b | 32 | 16 | 4096 | 32 | 4096 | 16384
redpajama_3b | 32 | 32 | 2560 | 32 | 2048 | 10240
falcon_7b | 71 | 32 | 4544 | 1 | 2048 | 18176
llama2_7b | 32 | 32 | 4096 | 32 | 4096 | 11008
llama2_13b | 40 | 40 | 4096 | 32 | 5120 | 13824
falcon_40b | 128 | 60 | 8192 | 8 | 2048 | 32768
llama2_70b | 64 | 80 | 8192 | 8 | 5120 | 28672
Table 4: Descrption of most relevant architectural specs of the pre-trained models used during the experiment Prompt | General Expected Continuation
---|---
Life in London | Travel/Cultural/Work-Related/London specific stuff
It is easy to be a techie | Comparison of techie with other probable roles in tech sector
Stock brokers are earning | Stock brokers and their earning style, sources, etc
It looks like written by Shakespeare | Shakespeare style text comparison
Hello, my name is | Chat or Introduction
Global warming and AI | Global Warming and AI in general as well as their +ive and -ive association
Current world order | Essay/Discussion/Power and Politics related tow world order
Percentage of people adore actors and singers | Stats on people following their favourite actors/singers and discussion on the related topic
Exercise and eating habits for | Eating habits and exercise routine in general (pros and cons)
Millennial and genz | Comparison and contrast between millennial and genz
Table 5: Prompts Description

###  3.3 Decoder Description

The current experiment uses a bare top_k sampling decoder without any
additional features, such as repetition penalty. To assess the model's
potential, we use a list of max new tokens, temperature as well as top_k. The
max new tokens, temperature and top_k are [50, 100, 150, 200, 250, 300, 350,
400, 450, 500], [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 1.0] and [1, 5, 10,
20, 50, 100, 200] respectively. The completion text is generated for every
quantized models using all the combinations of max new tokens, temperature and
top_k. The reason for high top_k such as 200 is that it might allow models to
choose more diverse, less repetitive, and semantically coherent text.

##  4 Analysis

A total of 6300 (10 prompts $\times$ 10 max new tokens $\times$ 9 temperature
$\times$ 7 top_k) completion text is generated for each quantized model except
falcon_40b and llama2_70b for 16bit 555Both falcon_40b and llama2_70b
encounter OOM errors on an 80GB GPU machine. falcon_40b throws an OOM error
when the max new tokens exceeds 400, while llama2_70b faces an OOM error
during the loading process.. The evaluation of these completion texts is
conducted through the computation of counting the duplicate content words,
serving as a metric for assessing the quality of the generated text. The
content words are the remaining words after removing the stop words.
Additionally, the model's size in gigabytes (GB) serves as a key measure for
quantifying GPU memory consumption, while tokens/sec is employed as a metric
to gauge the model's inference speed.

###  4.1 Memory Consumption and Inference Speed

Model | bnb.nf4 | bnb.nf4-dq | bnb.fp4 | bnb.f4-dq | bnb.int8 | bfloat16
---|---|---|---|---|---|---
stablelm_3b | 3.22 | 3.20 | 3.22 | 3.06 | 4.68 | 7.42
redpajama_3b | 2.31 | 2.17 | 2.31 | 2.17 | 3.52 | 5.60
falcon_7b | 5.72 | 5.37 | 5.72 | 5.37 | 8.71 | 14.50
llama2_7b | 4.58 | 4.27 | 4.58 | 4.27 | 7.82 | 13.53
llama2_13b | 8.83 | 7.8 | 8.83 | 7.8 | 14.2 | 26.23
falcon_40b | 26.40 | 24.64 | 26.55 | 24.64 | 44.52 | 80.85
llama2_70b | 40.23 | 38.2 | 40.4 | 38.2 | 70.44 | -
Table 6: The distribution of memory consumed (lower is better) for all the
models for different quantization evaluated on Nvidia A100 80GB GPU machine.

The utilization of int8 quantization demonstrates a significant reduction in
memory consumption, approximately in the range of 40% to 50%, when compared to
bfloat16, as illustrated in Table 6. Nonetheless, it is important to note that
this enhancement is accompanied by a corresponding trade-off in inference
speed, with int8 exhibiting a slowdown of roughly 75% to 80% in comparison to
bfloat16, as indicated in Table 7.

When evaluating memory consumption between the fp4 and nf4 quantization
approaches for model sizes up to 13 billion, their distinctions are
negligible. However, nf4 quantization exhibits a slight advantage over fp4 in
terms of memory consumption for larger models such as falcon_40b and
llama2_70b. Nevertheless, fp4-dq is found to be better in memory consumption
(i.e., takes less memory) across the models compared to its counterpart, as
shown in Table 6. It is worth noting that while double quantization offers a
clear advantage in memory consumption, it results in an inference speed
reduction of approximately 10% to 25% compared to the absence of double
quantization, as outlined in Table 7.

In conclusion, among the various quantization methods, bfloat16 stands out as
the least efficient in terms of memory consumption. However, it excels in
terms of inference speed, except in the case of stablelm_3b.

Model | bnb.nf4 | bnb.nf4-dq | bnb.fp4 | bnb.f4-dq | bnb.int8 | bfloat16
---|---|---|---|---|---|---
stablelm_3b | (37.76, 62.79) | (38.7, 53.37) | (42.99, 63.11) | (38.7, 53.03) | (7.91, 16.81) | (37.76, 49.88)
redpajama_3b | (24.2, 32.29) | (22.59, 27.04) | (25.64, 31.37) | (15.49, 27.08) | (2.52, 3.24) | (29.35, 37.85)
falcon_7b | (29.09, 37.54) | (22.71, 30.04) | (24.77, 37.41) | (22.23, 30.7) | (3.13, 12.63) | (35.79, 48.05)
llama2_7b | (23.09, 29.88) | (19.32, 23.44) | (23.01, 28.65) | (17.85, 23.41) | (1.32, 8.87) | (28.39, 36.35)
llama2_13b | (15.9, 23.14) | (13.22, 18.84) | (12.0, 22.98) | (10.83, 18.22) | (6.49, 7.14) | (24.12, 29.34)
falcon_40b | (11.93, 16.59) | (11.57, 14.51) | (12.12, 16.61) | (10.42, 12.76) | (3.56, 4.63) | (12.37, 13.99)
llama2_70b | (8.67, 10.39) | (6.47, 9.07) | (8.52, 10.23) | (6.39, 8.82) | (2.79, 3.76) | -
Table 7: The distribution of minimum and maximum inference speed (higher is
better) in tokens/sec for different quantization evaluated on Nvidia A100 80GB
GPU machine.

###  4.2 Temperature vs. Quality of Generation

A common pattern emerges within all quantization approaches, wherein an
increase in temperature correlates with an elevation in number of duplicate
content words except for bfloat16. However, it is worth noting that some
models are more sensitive to even temperature lower than 0.5 compared to
others.

When comparing the performance of stablelm_3b and redpajama_3b models, it
becomes evident that the fp4 and nf4-dq quantization methods exhibit
suboptimal results, characterized by an increased occurrence of duplicate
words at lower temperature settings. However, the situation varies when
considering falcon models, where nf4 quantization consistently demonstrates
inferior performance across the entire temperature spectrum in comparison to
other quantization methods.

In contrast, when assessing llama2 models, the situation becomes more nuanced,
with most quantization approaches contributing significantly to repetitive
generation. In this context, determining a clear front-runner among these
methods proves to be a challenging task. Nevertheless, it is noteworthy that
for the llama2_70b model, both fp4 and fp4-dq quantization methods outshine
the others in terms of performance.

The analysis reveals that the int8-quantized model demonstrates effective
control over the occurrence of duplicate content words for both llama2_13b and
llama2_70b, effectively limiting them in the range of 40. In contrast, the
bfloat16 models exhibit a characteristic of independence from temperature
scaling, as they consistently generate a comparable number of repetitive words
across all temperature settings except redpajama_3b.

###  4.3 Max Returned Tokens vs. Quality of Generation

The term max returned tokens encompasses the combined value of max new tokens
and the length of the input prompt in terms of tokens. the analyis reveals
that an the count of duplicate words generated linearly increases with the
increase of max returned tokens across all models and quantization methods.

###  4.4 Top_k vs. Quality of Generation

The analysis offers a somewhat surprising insight, indicating that setting
top_k equal to 1 tends to result in the lowest occurrence of duplicate words
across models and quantization methods. Nonetheless, it's noteworthy that this
effect reaches a point of saturation and loses distinctiveness when top_k is
equal to or greater than 5.

###  4.5 Overall Comparison

In terms of the average number of duplicate content words666The total number
of content words generated for the unquantized model lies in the range of
1.34M to 1.45M and the maximum duplicate number of words is around 80K.
generated in absolute terms, our analysis reveals the following insights:

  * •

For fp4 and fp4-dq compared to nf4 and nf4-dq across various models (except
llama2 series), there is a consistent reduction in repetitive generation,
typically ranging from 12% to 20% relative.

  * •

In the case of nf4 and nf4-dq for llama2 models of different sizes, there is a
more noticeable advantage, with relative reduction of 9% to 11% in repetitive
generation.

  * •

Int8 quantization has a more pronounced limitation on the number of generated
words, producing approximately 30-50% fewer content words than 4-bit
quantization. Additionally, it produces 25-40% more duplicate content words
relative to 4-bit quantization at normalized scale.

  * •

When comparing bfloat16 with 4-bit quantization, it's noteworthy that bfloat16
generally produces more number of content words, often ranging from
approximately 3% to 10%. Nonetheless, bfloat16 tends to generate a marginally
higher duplicate words, indicating relative inferiority of 1% to 3.5% with
4bit quantization.

The computation of average perplexity scores, with a token stride of 512, is
conducted for all quantization levels across each model. An examination of
these scores reveals that the perplexity values for all models reside within a
relatively constrained range, typically ranging from 12 to 15. Consequently,
it is discerned that perplexity, within this context, may not serve as a
suitable metric for assessing the quality of the generated text.

##  5 Conclusions

In scenarios where GPU memory is not a limiting factor and the utmost priority
is placed on achieving both high inference speed and accuracy, it is advisable
to prioritize the utilization of bfloat16 for models up to 7 billions. This
preference arises due to its reduced susceptibility to variations in
temperature and max new tokens. Moreover, model upto 7 billion size
effectively fits into a consumer grade GPU machine. Alternatively, nf4 and fp4
serves as the default choice for individuals seeking a balance between GPU
utilization, accuracy and inference speed, thus offering a middle-ground
solution that combines all aspects effectively.

It's worth noting that the adoption of double quantization, such as fp4-dq and
nf4-dq, can lead to a marginal reduction in memory footprint. However, it is
accompanied by a relatively decreased inference speed. Hence, the
recommendation leans toward using quantization without the doubling approach.
Additionally, when considering the nf4 and fp4 precision combination, it is
recommended to use a temperature of less than 0.5, exactly 1.0, or a
combination of these values to achieve optimal performance.

The current evaluation does not consider int8 to be a feasible alternative to
other quantization methods. While int8 reduces memory usage, it significantly
slows down inference and produces around 30-50% fewer words than other
quantization methods.

It is important to note that the current experiment did not achieve
satisfactory results in terms of accuracy and inference speed when using gptq
4-bit quantization. Further investigation is needed to replicate the
comparable performance that has been reported in other
studies777https://github.com/PanQiWei/AutoGPTQ. Therefore, this result is not
included in the analysis presented.

##  6 Limitations and Future Work

The current study is conducted on 7 models ranging in size from 3 billion to
70 billion parameters, and 10 prompts are used for next-word prediction using
various combinations of hyperparameters. Further study with more models
(containing $\leq$ 1 billion parameters) and prompts might provide more
insights into the effects of these hyperparameters on relatively smaller
quantized LLMs.

Future work will focus on the primary causes of repetitive generation and
their relationship to Hypothesis 1 and Hypothesis 2. Moreover, the results
show that falcon has a faster inference speed than llama2 in the 7B category.
However, falcon has a higher number of overall parameters than llama2.
Therefore, future research will focus on model-specific factors that affect
inference speed.

## References

  * [1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, ``Attention is all you need,'' _Advances in neural information processing systems_ , vol. 30, 2017.
  * [2] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell _et al._ , ``Language models are few-shot learners,'' _Advances in neural information processing systems_ , vol. 33, pp. 1877–1901, 2020.
  * [3] A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann _et al._ , ``Palm: Scaling language modeling with pathways,'' _arXiv preprint arXiv:2204.02311_ , 2022.
  * [4] S. Smith, M. Patwary, B. Norick, P. LeGresley, S. Rajbhandari, J. Casper, Z. Liu, S. Prabhumoye, G. Zerveas, V. Korthikanti _et al._ , ``Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model,'' _arXiv preprint arXiv:2201.11990_ , 2022.
  * [5] A. Srivastava, A. Rastogi, A. Rao, A. A. M. Shoeb, A. Abid, A. Fisch, A. R. Brown, A. Santoro, A. Gupta, A. Garriga-Alonso _et al._ , ``Beyond the imitation game: Quantifying and extrapolating the capabilities of language models,'' _arXiv preprint arXiv:2206.04615_ , 2022.
  * [6] G. Penedo, Q. Malartic, D. Hesslow, R. Cojocaru, A. Cappelli, H. Alobeidli, B. Pannier, E. Almazrouei, and J. Launay, ``The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only,'' _arXiv preprint arXiv:2306.01116_ , 2023.
  * [7] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale _et al._ , ``Llama 2: Open foundation and fine-tuned chat models,'' _arXiv preprint arXiv:2307.09288_ , 2023.
  * [8] R. Pope, S. Douglas, A. Chowdhery, J. Devlin, J. Bradbury, J. Heek, K. Xiao, S. Agrawal, and J. Dean, ``Efficiently scaling transformer inference,'' _Proceedings of Machine Learning and Systems_ , vol. 5, 2023.
  * [9] C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, ``Accelerating large language model decoding with speculative sampling,'' _arXiv preprint arXiv:2302.01318_ , 2023.
  * [10] G. Yang, D. Lo, R. Mullins, and Y. Zhao, ``Dynamic stashing quantization for efficient transformer training,'' _arXiv preprint arXiv:2303.05295_ , 2023\.
  * [11] Z. Liu, B. Oguz, C. Zhao, E. Chang, P. Stock, Y. Mehdad, Y. Shi, R. Krishnamoorthi, and V. Chandra, ``Llm-qat: Data-free quantization aware training for large language models,'' _arXiv preprint arXiv:2305.17888_ , 2023\.
  * [12] T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, ``Qlora: Efficient finetuning of quantized llms,'' _arXiv preprint arXiv:2305.14314_ , 2023.
  * [13] S. J. Kwon, J. Kim, J. Bae, K. M. Yoo, J.-H. Kim, B. Park, B. Kim, J.-W. Ha, N. Sung, and D. Lee, ``Alphatuning: Quantization-aware parameter-efficient adaptation of large-scale pre-trained language models,'' _arXiv preprint arXiv:2210.03858_ , 2022.
  * [14] T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer, ``Llm. int8 (): 8-bit matrix multiplication for transformers at scale,'' _arXiv preprint arXiv:2208.07339_ , 2022.
  * [15] E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh, ``Gptq: Accurate post-training quantization for generative pre-trained transformers,'' _arXiv preprint arXiv:2210.17323_ , 2022.
  * [16] Z. Yuan, L. Niu, J. Liu, W. Liu, X. Wang, Y. Shang, G. Sun, Q. Wu, J. Wu, and B. Wu, ``Rptq: Reorder-based post-training quantization for large language models,'' _arXiv preprint arXiv:2304.01089_ , 2023.
  * [17] J. Lin, J. Tang, H. Tang, S. Yang, X. Dang, and S. Han, ``Awq: Activation-aware weight quantization for llm compression and acceleration,'' _arXiv preprint arXiv:2306.00978_ , 2023.
  * [18] Z. Yao, C. Li, X. Wu, S. Youn, and Y. He, ``A comprehensive study on post-training quantization for large language models,'' _arXiv preprint arXiv:2303.08302_ , 2023.
  * [19] S. Merity, C. Xiong, J. Bradbury, and R. Socher, ``Pointer sentinel mixture models,'' _arXiv preprint arXiv:1609.07843_ , 2016.
  * [20] M. Marcus, B. Santorini, and M. A. Marcinkiewicz, ``Building a large annotated corpus of english: The penn treebank,'' 1993.
  * [21] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, ``Exploring the limits of transfer learning with a unified text-to-text transformer,'' _The Journal of Machine Learning Research_ , vol. 21, no. 1, pp. 5485–5551, 2020.
  * [22] S. Shen, Z. Dong, J. Ye, L. Ma, Z. Yao, A. Gholami, M. W. Mahoney, and K. Keutzer, ``Q-bert: Hessian based ultra low precision quantization of bert,'' in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 05, 2020, pp. 8815–8821.
  * [23] A. H. Zadeh, I. Edo, O. M. Awad, and A. Moshovos, ``Gobo: Quantizing attention-based nlp models for low latency and energy efficient inference,'' in _2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)_. IEEE, 2020, pp. 811–824.
  * [24] H. Bai, W. Zhang, L. Hou, L. Shang, J. Jin, X. Jiang, Q. Liu, M. Lyu, and I. King, ``Binarybert: Pushing the limit of bert quantization,'' _arXiv preprint arXiv:2012.15701_ , 2020.
  * [25] W. Zhang, L. Hou, Y. Yin, L. Shang, X. Chen, X. Jiang, and Q. Liu, ``Ternarybert: Distillation-aware ultra-low bit bert,'' _arXiv preprint arXiv:2009.12812_ , 2020.
  * [26] S. Kim, A. Gholami, Z. Yao, M. W. Mahoney, and K. Keutzer, ``I-bert: Integer-only bert quantization,'' in _International conference on machine learning_. PMLR, 2021, pp. 5506–5518.
  * [27] Z. Yao, C. Li, X. Wu, S. Youn, and Y. He, ``A comprehensive study on post-training quantization for large language models,'' _arXiv preprint arXiv:2303.08302_ , 2023.
  * [28] G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, and S. Han, ``Smoothquant: Accurate and efficient post-training quantization for large language models,'' in _International Conference on Machine Learning_. PMLR, 2023, pp. 38 087–38 099.
  * [29] X. Wu, C. Li, R. Y. Aminabadi, Z. Yao, and Y. He, ``Understanding int4 quantization for transformer models: Latency speedup, composability, and failure cases,'' _arXiv preprint arXiv:2301.12017_ , 2023.
  * [30] E. Frantar and D. Alistarh, ``Optimal brain compression: A framework for accurate post-training quantization and pruning,'' _Advances in Neural Information Processing Systems_ , vol. 35, pp. 4475–4488, 2022.
  * [31] Y. Belkada, T. Dettmers, A. Pagnoni, S. Gugger, and S. Mangrulkar, ``Making llms even more accessible with bitsandbytes, 4-bit quantization and qlora,'' 2023\. [Online]. Available: https://huggingface.co/blog/4bit-transformers-bitsandbytes


# On Small-World Networks: Survey and Properties Analysis

Alaa Eddin Alchalabi  _Graduate School of Electronics and Computer
Engineering_
Istanbul Sehir University, Istanbul, Turkey
<EMAIL_ADDRESS>
###### Abstract

Complex networks has been a hot topic of research over the past several years
over-crossing many disciplines, starting from mathematics and computer science
and ending by the social and biological sciences. Random graphs were studied
to observe the qualitative features they have in common in planetary-scale
data sets which helps us to project the insights proven to real-world
networks.

In this paper, We survey the particular case of small-world phenomena and
decentralized search algorithms. We start by explaining the first empirical
study for the “six degrees of separation” phenomenon in social networks; then
we review some of the probabilistic network models based on this work,
elaborating how these models tried to explain the phenomenon’s properties, and
lastly, we review few of the recent empirical studies empowered by these
models. Finally, some future works are proposed in this area of research.

###### Index Terms:

Small-World, Complex Networks, Lattices and Random Graphs, Search Algorithms.

##  I Introduction

Recently, the study of complex networks has emerged in various range of
disciplines and research areas. The World Wide Web has revolutionized the way
we deal with everything one deals in daily life. Computer scientists were
curious to find a way to handle the wheel of controlling the complexity and
the enormous growth of the Internet. Social networks’ data scale is
unpredictably uncontrollable by social scientists. The biological interactions
in cell’s metabolism are expected to define its pathways and could provide
insights to biologists [13]. The urge a new born science is needed in order to
be able to manipulate networks before networks manipulate our needs [8].

The study of complex networks evolved since the study of randomly generated
graphs by Erdos and Renyi [4], and the appearance of a large-scale network
data had leashed tremendous work in multi-disciplinary areas including the
real and the virtual world [13]. The efforts were put to describe the
properties of random graphs in large networks which raised more and more
technical questions to be answered. To mimic real-networks, a randomly
produced stylized network model is adopted in order to generalize the
resulting conclusions and properties onto real-networks. Simple models fails
to capture the complexity of a realistic network’s structure and features
offering a strong mathematical basis which futures investigations can be build
upon.

In the next sections of this paper, we survey the “small-world phenomenon” and
few related problems. We start with the famous psychologist Stanley Milgram’s
social experiment, that captures the main aspects of the phenomenon [11],
[14]; we review few of the models based on random graphs that tries to explain
the problem [7], [9], [12], [15], [16]; and then we mention recent work that
has applied the traditional insights of these models on large data sets
extracted from famous web applications [2], [10]. Lastly, some suggested
further extensions to small-world networks are discussed, along with some
future works followed by their relevance to this field.

##  II small-world phenomenon

The small-world phenomenon has recently been the hot topic of both theoretical
and practical research, and it has been given huge attention from most, if not
all, multi-disciplinary researchers. The term “small-world”, linked by all
means to the “short chains of acquaintances”, or the “six degrees or
separation” [5][6][16], refers to the human social network’s graph; where
nodes replaces people, and edges between two nodes mimic if the two
corresponding persons know each other on a first-name basis [8]. The graph is
described to be a “small-world” because of the fact that any two random pairs
of nodes are separated by relatively a small number of nodes, generally less
than 6. Although the first-name basis rule is a bit naive for an edge
definition, the resulted graph behaves as a real-world network.

Small-world networks are of great importance because of adoption to the
limitations of either of the end extreme networks types; random networks and
regular lattices. Small-world networks proved their ability to be used as
frameworks for the study of interaction networks of complex systems [1].

The most important key of the small-world study is to prove the hypothesis
that assumes the qualitatively shared structure among a variety of networks
across variant fields. A common property arises in large networks which is the
existence of short paths between most of the nodes pairs although nodes in
network have a high degree of clustering. Nodes can also be reached and
navigated with no need of universal understanding of the whole network. Such
properties contributed in describing large-scale social networks behavior, and
additionally, they gave important insights to create applications to the
internal design of decentralized peer-to-peer systems.

###  II-A Milgram’s Experiment

Stanley Milgram, the famous social psychologist, made an experiment to measure
people connectivity in the USA in the 1960s and to test the small-world
property [11][14]. The experiment questions the probability of an arbitrarily
two selected individuals from a large data set to know each other in person. A
target person was selected in the state of Massachusetts who was a Boston
stockbroker, and 296 arbitrarily selected individuals as “starting persons”
from Nebraska and Boston were asked to generate acquaintance chains to the
stockbroker in Boston. The selected group were given a document of the
described study with the target’s name. The senders were asked to choose a
recipient which they think that he/she will contribute to carry the message to
the target person in the shortest way possible. The concept of “roster” was
introduced to prevent the message goes back to a previous sender “loop” and to
track the number of node the message reached.

The results of the experiment were quite astonishing. Sixty-four chains made
their way to the target person. The mean number of the intermediaries was 5.2
with median of 6. Boston starting chains exhibits shorter range chains than
Nebraska’s chains. Additional experiments by Korte and Milgram proved that
these numbers are quite stable [14].

Some comments on Milgram’s experiments exhibit the inability of this model to
be generalized to larger networks. Varying the information about the target
person might affect the decisions taken by senders, and here we meant
psychological and sociological factors take place.

##  III Small-world based empirical models

###  III-A Watts and Strogatz’s Model

Watts and Strogatz came up with a model that aims to explain the small-world
property. After Bollobas and de la Vega [3] introduced the theorem which
proves the logarithmic property in the path length with respect to the number
of nodes _O(log n)_ in small-world networks, Watts and Strogatz felt that
there was something missing in the theorem. The model proposed considered
small-world networks to be highly-structured with relatively a few number of
random links added within. Long-range connections in this model plays a
crucial rule in creating short paths through all nodes [15].

The model adopts the idea of rewiring edges between nodes with a certain
probability to generate a graph. The probability allows the change between
regular networks (p=0) and random networks (p=1). The model starts by
generating a ring of n connected nodes (average degree of k). Then, the
rewiring of each edge with the probability p and the landing node is also
chosen randomly. The clustering coefficient _$(C_{p})$_ is a measure which
reflects the fraction of the connected neighbours to a node in a graph
compared to all possible connections of the neighboured averaged over all
nodes [15].

The results derived for a regular graph (p = 0) a highly clustered (_$C\sim$_
3/4) and path length _$L\sim n/2k$_ where _$n >k>ln(n)>1$_ should be chosen.
For random graphs (p = 1) the resulted network is poorly with a low clustering
coefficient (_$C\sim k/n$_) and a small path length _$L\sim ln(n)/ln(k)$_.
Their research also included three empirical real-world examples of small-
world models, and their main finding was that the average path length of the
chosen example was slightly higher than the random model, while the clustering
coefficient had clearly a much higher value than the random model. Using their
results, they reasoned how infectious disease spread rapidly in small-world
societies.

Some drawbacks of Watts and Strgatz’s model is that it cannot be generalized
to all small-world models. Some extended works by other scientists tried to
fill in gaps.

###  III-B Classes of Small-World Networks

Due to the limited vision of the Watts and Strogatz model, new explanation was
needed. Trying to look at the dilemma from another prospective, Amaral et al.
tried to classify small-world networks to three classes reporting an empirical
study of real-world networks [1]. The study covers mainly the statistical
properties of real-world networks, and it was enough to prove the existence of
three classes of small-world networks: Scale-free, broad-scale, and single
scale[1].

####  III-B1 Scale-free

The networks which is characterized by a vertex connectivity distribution
which decays as a power law.

####  III-B2 Broad-scale

The networks characterized by a connectivity distribution that has a power-low
region and followed by a sharp cutt-off.

####  III-B3 Single-scale

The networks characterized by a connectivity distribution with a fast decaying
tail.

The research also gave an answer to why such taxonomy exist, and they reasoned
that by mentioning two types of factors. The first factor was the aging of the
vertices, which in time old nodes will stop being as effective in the network,
and an example of that can be the actors network. The second type of factor
was the cost of adding new links to the vertices which limited by the vertex
capacity. An example of this can be the airports map where placing too many
vertices are pricey and not practical.

###  III-C Kleinbergs’s Algorithmic Prospective

Kleinberg’s way of explaining small-world properties was a bit close to Watts
and Strogatz but with slight differences [7]. Kleinberg used a _n x n_ grid of
nodes to represent the network, and to add the small-world flavour, a number
of long-range connection edges were added and not rewired. After adding the
edges, the probability of connecting two random vertices (v,w) is proportional
to _$1/d(v,w)^{q}$_ where q is the clustering coefficient [9].

Kleinberg came up with theorems to quantify the decentralized algorithms’
delivery time which generalized the results in [3] by Bollobás and de la Vega
of the logarithmic behavior of short pathways in networks. He proved that the
time needed is not always logarithmic but it depends on other parameters. A
new parameter was introduced ($\alpha$ >=0) that controls the long-range
connections. Interestingly, the delivery time varies depends on $\alpha$ as
follows:

####  III-C1 For 0 <$\alpha$ <2

the delivery time of any decentralized algorithm in the grid-based model is
$\Omega$ ($n^{(2-a)/3}$).

####  III-C2 For $\alpha$ = 2

the delivery time of any decentralized algorithm in the grid-based model is
O($log^{2}n$).

####  III-C3 For $\alpha$ >2

the delivery time of any decentralized algorithm in the grid-based model is
$\Omega$ ($n^{(a-2)/(a-1)}$) [7].

##  IV Recent real-world Empirical experiments

###  IV-A Dodds, Muhammad, and Watts Experiment

Dodds et al. tried to mimic Milgram’s experiment with the electronic messaging
systems. Around 60,000 randomly selected email users attempted to reach 18
target persons in 13 different countries.

The findings were quite unexpected. Successful social chains passed through
intermediate to weak strength ties [12]. This finding proves that the highly
connected hubs’ effect is negligible. The attrition of message chains showed
that messages could make it to the target in a median of five to seven. The
cool fact about attrition rate was the constancy of its value for a certain
period of time. The 384 completed chains (out of 24,163) had an average chain
length of 4.05. This number was considered misleading by the authors, which
made them evaluate the experiment using new metrics.

The general results showed that the the network structure alone is not enough
to interpret the network. Actions and perceptions of the individuals are big
contributors.

###  IV-B Leskovec et al. on a Large Instant-Messaging Network

Leskovec et al. presented a study in 2003 which captured a month of
communication activities within the Microsoft Messenger instant-messaging
system [10]. The data set contained about 30 million conversations between 240
million people, and a graph was constructed containing 180 million nodes and
1.3 billion undirected edges. The network represents accounts that were active
during June 2006.

The resulted average path length among users was 6.6 with 6 being the median.
The results showed that users with similar age, language, and location tend to
communicate with each other. Users with different genders tend to communicate
more and also for longer conversations [10]. Conversations tends to decrease
with the increase in the distance. However, communication chains through
relatively long distances tend to carry longer conversations [10].

###  IV-C Bakhshandeh’s Degrees of Separation in Twitter

Bakhshandeh et al. did an interesting analysis to identify the degree
separation between two Twitter users. They used a new search technique to
provide near-optimal solutions more optimized than greedy approaches [2]. The
average separation path length was 3.43 between any two random Twitter users
which required 67 requests from Twitter, while the near-optimal was 3.88 using
only 13.3 requests on average. Surprisingly, Twitter’s 3.43 degree of
separation is small and the reason they have claimed was the indicative of
changing social norms in the modern connected world.

##  V Further Extensions and future works

There is no doubt that small-world networks are still and will still be a hot
research topic due to its nature. In this section, we would like to propose
some ideas for future extensions which might propose solutions for unanswered
or vaguely answered questions.

Introducing machine learning techniques to small-world networks, in my
opinion, is a good idea. Constructing networks should be smart enough in order
to be controllable not only interpretable. Small-world networks could be build
to mimic the brain neural-map which might give us more insight on how the
human brain works. ML techniques can be also used to conserve the “six degrees
of separation rule” or even to break it which completely depends on the
application.

Introducing local reference nodes in such networks could be a new idea to be
implemented. Reference nodes could have some regional knowledge about the
surrounding nodes. They can control the “hubs” and determine how new links can
be distributed among reference nodes. We can think of routers as examples. The
uniqueness of the node is somehow unrealistic for some applications, and that
shows the urge of introducing a new concept.

##  VI Conclusion

At this paper, we discussed the famous phenomenon of small-world networks and
its importance in various areas. Few of the small-world driven models were
surveyed. Then recent real-world experiments in the context of complex
networks were mentioned. Later, further extensions and future works were
proposed. In the future, we will try to implement that the suggested ideas
practically on a given data set. By taking into account their bros and cons,
the ideas will be later evaluated against the other state-of-art
implementations.

## References

  * [1] Amaral, Luıs A. Nunes, et al. ”Classes of small-world networks.” Proceedings of the national academy of sciences 97.21 (2000): 11149-11152.
  * [2] Bakhshandeh, Reza, et al. ”Degrees of separation in social networks.” Fourth Annual Symposium on Combinatorial Search. 2011.
  * [3] Bollobas, B., de la Vega,W. F., The diameter of random regular graphs. (1982), 125–134.
  * [4] Erdos, P., and Renyi, A., On the Evolution of Random Graphs. Magyar Tud. Akad. Mat. Kutató Int. Közl. 5 (1960), 17–61.
  * [5] Fass, Craig, Mike Ginelli, and Brian Turtle. Six Degrees of Kevin Bacon. Plume Books, 1996.
  * [6] Guare, John. Six degrees of separation: A play. Vintage, 1990.
  * [7] J. Kleinberg. The small-world phenomenon: An algorithmic perspective. Proc. 32nd ACM Symposium on Theory of Computing, 2000.
  * [8] J. Kleinberg. Complex Networks and Decentralized Search Algorithms. Proceedings of the International Congress of Mathematicians (ICM), 2006.
  * [9] Kleinberg, J. Navigation in a Small World. Nature 406(2000), 845.
  * [10] Leskovec, Jure, and Eric Horvitz. ”Planetary-scale views on a large instant-messaging network.” Proceedings of the 17th international conference on World Wide Web. ACM, 2008.
  * [11] Milgram, Stanley. ”The small world problem.” Psychology today 2.1 (1967): 60-67.
  * [12] Peter Sheridan Dodds, Roby Muhamad, Duncan J. Watts. An Experimental Study of Search in Global Social Networks. Science 301(2003), 827.
  * [13] Strogatz, Steven H. ”Exploring complex networks.” Nature 410.6825 (2001): 268-276.
  * [14] Travers, Jeffrey, and Stanley Milgram. ”An experimental study of the small world problem.” Sociometry (1969): 425-443.
  * [15] Watts, D. J. and S. H. Strogatz. Collective dynamics of ’small-world’ networks. Nature 393:440-42(1998).
  * [16] Watts, Duncan J. Six degrees: The science of a connected age. WW Norton and Company, 2004.

|  John Doe Hello, here is some text without a meaning. This text should show
what a printed text will look like at this place. If you read this text, you
will get no information. Really? Is there no information? Is there a
difference between this text and some nonsense like “Huardest gefburn”? Kjift
– not at all! A blind text like this gives you information about the selected
font, how the letters are written and an impression of the look. This text
should contain all letters of the alphabet and it should be written in of the
original language. There is no need for special content, but the length of
words should match the language.
---|---


# Monitoring Observations of SMC X-1’s Excursions (MOOSE) I: Program
Description and Initial High-State Spectral Results

Kristen C. Dage1,2 McKinley Brumback,3 Joey Neilsen, 4 Chin-Ping Hu,5 Diego
Altamirano,6 Arash Bahramian,7 Philip A. Charles 6 William I. Clarkson,8 Daryl
Haggard,1,2 Ryan C. Hickox,9 Jamie Kennea10
1Department of Physics, McGill University, 3600 University Street, Montréal,
QC H3A 2T8, Canada
2McGill Space Institute, McGill University, 3550 University Street, Montréal,
QC H3A 2A7, Canada
3 Cahill Center for Astronomy and Astrophysics, California Institute of
Technology, 1216 E California Blvd, Pasadena, CA 91125, USA
4 Villanova University, Department of Physics, Villanova, PA 19085, USA;
5 Department of Physics, National Changhua University of Education, Changhua,
50007, Taiwan
6Physics & Astronomy, University of Southampton, Southampton, Hampshire SO17
1BJ, UK
7International Centre for Radio Astronomy Research $-$ Curtin University, GPO
Box U1987, Perth, WA 6845, Australia
8Department of Natural Sciences, University of Michigan-Dearborn, 4901
Evergreen Rd. Dearborn, MI, 48128
9 Department of Physics & Astronomy, Dartmouth College, 6127 Wilder
Laboratory, Hanover, NH 03755, USA
10 Department of Astronomy and Astrophysics, The Pennsylvania State
University, University Park, PA 16802, USA E-mail<EMAIL_ADDRESS>
(Accepted XXX. Received YYY; in original form ZZZ)

###### Abstract

SMC X-1 has exhibited three super-orbital period excursions since the onset of
X-ray monitoring beginning with RXTE’s launch in 1995. NICER has recently
probed a fourth observed excursion beginning in 2021 with our program
Monitoring Observations of SMC X-1’s Excursions (MOOSE). These sensitive new
MOOSE data probe different super-orbital periods and phases within them.
Spectral fits to the high-state continuum during April 2021 to January 2022
show that the intrinsic spectral shapes are characterised by a soft
(kT$\sim$0.19 keV) disc component and a hard ($\Gamma$ $\sim$0.7) power-law
tail. When the 2021-2022 NICER observations, taken during an excursion, are
compared to 2016 XMM-Newton observations (outside of an excursion), we find
little evidence for intrinsic spectral variability across the high-states, but
find evidence for a $>$3 $\sigma$ change in the absorption, although we
caution that there may be calibration differences between the two instruments.
Thus, over different lengths of super-orbital periods, we see little evidence
for intrinsic spectral changes in the high-state. Upcoming studies of the
pulse profiles may shed light on the mechanism behind the excursions.

###### keywords:

accretion, accretion discs – stars: pulsars: individual: SMC X-1 – X-rays:
binaries

††pubyear: 2022††pagerange: Monitoring Observations of SMC X-1’s Excursions
(MOOSE) I: Program Description and Initial High-State Spectral
Results–Monitoring Observations of SMC X-1’s Excursions (MOOSE) I: Program
Description and Initial High-State Spectral Results

##  1 Introduction

Modulation in X-ray binary lightcurves has long been used as a diagnostic to
understand key physical processes in the system. One such modulation that has
garnered interest over the last $\sim$50 years are observations of super-
orbital periodicities.

Super-orbital periods$--$ the detection of a periodicity in an X-ray binary
which is longer than the orbital period of the binary$-$ have been
significantly detected in many different types of systems. These include
supergiant fast X-ray transients, wind-accretion systems, as well as
persistent (Roche lobe overflow), high-mass X-ray binaries and low-mass X-ray
binaries with both black hole and neutron star primaries (Kotze & Charles,
2012; Corbet & Krimm, 2013). Recent optical studies also showed evidence for
super-orbital periodicites in the black hole transient MAXI J1820+070 (Thomas
et al., 2022).

One explanation is that super-orbital period modulation is due to a warped,
precessing accretion disc that is observed edge-on, such that the disc
obscures the accretor during precession Clarkson et al. (2003b). As such, the
super-orbital period is a direct observable of the accretion disc
configuration. For systems like the neutron star X-ray binary SMC X-1, one
generally accepted cause of warps in X-ray binary accretion discs is radiation
driven warping (Pringle, 1996; Ogilvie & Dubus, 2001). The stability of the
warped disc structure is determined by binary parameters such as radius and
mass ratio (e.g. Clarkson et al., 2003a, b). In systems that do not fall in
the stability range, like SMC X-1, fluctuations in the geometry of the
accretion disc can be observed as fluctuations in super-orbital behaviour
(Ogilvie & Dubus, 2001).

SMC X-1 is a well-studied high mass X-ray binary with a known neutron star
accretor and a B0 I supergiant companion (Reynolds et al., 1993), The system
has a 3.89 day orbital period (Schreier et al., 1972) and the neutron star
produces a $\sim$ 0.7s pulse period (Lucke et al., 1976), which, since the
1970s, its $\sim$ 0.7 s pulse period has steadily been spun up (Inam et al.,
2010). Hu et al. (2019) report the spin-up frequency derivative as $\sim$ 2.5
$\times 10^{-11}$ s-2. Early studies of SMC X-1’s X-ray spectral properties
identified both hard and soft spectral components (Marshall et al., 1983).
These components likely originate from different physical processes (Naik &
Paul, 2004). Hickox & Vrtilek (2005) modeled the hard and soft emission with a
twisted inner disc, which indicated that the hard emission may be due to the
illuminating pulsar beam, and the soft component caused by the reprocessed
emission of the inner accretion disc.

SMC X-1 is also well known for its super-orbital period, which is typically
measured close to 60 days, but can take excursions to periods as low as 45
days (Clarkson et al., 2003a; Dage et al., 2019; Hu et al., 2019). The
characterisation and physical origins of the super-orbital period ($P_{sup}$)
modulation have been the subject of many studies over the last few decades
(see, e.g. Gruber & Rothschild, 1984; Wojdowski et al., 1998; Clarkson et al.,
2003a; Trowbridge et al., 2007; Hu et al., 2011, 2013; Dage et al., 2019; Hu
et al., 2019, and references therein), with many techniques implemented to
accurately measure these long-term modulations, including dynamical power
spectrum and Hilbert-Huang (Huang et al., 1998) transform analysis.

Spectral studies by Wojdowski et al. (2000) determined that the spectral
properties are at odds with predictions from line-driven wind models (Blondin
& Woo, 1995). RXTE spectral monitoring by Inam et al. (2010) found little
evidence of spectral variation, except that $N_{\textrm{H}}$ increased as the
X-ray flux dropped, and monitoring of the spectrum by Vrtilek et al. (2005)
showed that the super-orbital period cycle influences the spectrum, as the low
state spectra were linked to the orbital phase, but not the high state
spectra. The role that radiation-driven warping plays in SMC X-1’s super-
orbital period is very much a mystery, as recent analysis by Pradhan et al.
(2020) suggests that a warped accretion disk is not the only cause of
$P_{sup}$, but variability in the power-law normalization could imply that
intrinsic changes from the source itself are a factor in producing the
observed super-orbital periodicity.

Other tests to determine the geometry of the disc can also be probed via
studies of the pulse profiles: variation in the soft pulse profiles may be
related to precession in the disc (Neilsen et al., 2004). Brumback et al.
(2020) implemented X-ray tomography techniques to directly model the warped
accretion disc, in particular, the inner part near the magnetosphere, and
showed that the direct (hard) pulse profile and reprocessed (soft) pulse
profile can both be separately traced by X-ray spectra. The hard and soft
pulse profile offsets changed within a given 60 day super-orbital period
cycle, but the pulse profiles at the beginning and end of the 60 day cycle
were almost identical$-$ implying that the inner accretion disc had completed
an entire precessed cycle.

Observational and theoretical tests can be made to determine if SMC X-1’s disc
warp and roving super-orbital period is attributed to radiation driven
warping, where irradiation from a central source warps the accretion disc.
Smoothed particle hydrodynamical simulations by Foulkes et al. (2006, 2010)
suggest this is quite possible. Such a scenario would imply a link between the
change in super-orbital period and the mass transfer rate. Dage et al. (2019)
searched for such a link, but they were not able to find conclusive observable
evidence. Hu et al. (2019)’s complete analysis of the super-orbital period
from 1999-2019, along with analysis of spin periods thanks to MAXI GSC found
no evidence that the mass transfer rate was strongly associated with the
super-orbital period, but suggested that the super-orbital period modulation
could be due to a torque change (which in turn is dependent on the mass
transfer rate and the warp angle).

Understanding the nature of SMC X-1’s warped, precessing accretion disc has
risen to greater importance with the discovery of the pulsating ultraluminous
X-ray sources (ULXPs; e.g., Bachetti et al. (2014)). SMC X-1 may be a nearby
analog to the ULXPs, as it also exhibits pulse dropouts (Pike et al., 2019;
Bachetti et al., 2020). Many ULXs are now thought to exhibit super-orbital
periods and possibly also accretion disc warping (see, e.g., Motch et al.,
2014; Brightman et al., 2019; Walton et al., 2016, among others), see also
Townsend & Charles (2020) and references therein. Understanding SMC X-1’s
super-orbital behaviour can shed light on the mysterious, and more distant
ULXPs, as well as provide a unique understanding on the physical processes
responsible for X-ray binary variability on many timescales.

Our NICER observing program, Monitoring Observations Of SMC X-1’s Excursions
(MOOSE) is designed to fully utilise NICER’s monitoring capabilities to
provide information on SMC X-1’s spectral shape and soft timing properties as
the sources enters a new state of super-orbital period modulation. These
properties have not yet been probed during an epoch of super-orbital period
excursion, and MOOSE will provide a rich and diverse data-set to shed light on
the physical properties of the system during this time. This paper is the
first in the series focusing on the high-state spectra. Detailed timing
analysis of the system will be presented in Brumback et al. (in prep). In
Section 2.1 we describe the MOOSE campaign; Section 2 discusses initial
spectral analysis. These results are discussed in Section 3, while Section 4
considers the next steps to be undertaken in the analysis of MOOSE’s dataset.

##  2 Data and Analysis

NICER’s superb timing resolution ($<$300 nsec) and monitoring capabilities,
along with sensitivity in the soft (0.2-12 keV) energy range (Gendreau et al.,
2016) present an excellent opportunity to study SMC X-1 throughout the
duration of the newest super-orbital period excursions. The observations are
spaced roughly 10 days apart, and sample different stages of the super-orbital
cycle. The data quality is a function of the observation length and the phase
of the super-orbital cycle that is observed (as observations taken in the low
state have a much lower count rate). Figure 1 shows the observations plotted
against the Swift/BAT monitoring lightcurve, with each observation flagged by
data quality.

As discussed in Section 2.2, the super-orbital cycles are not consistent with
each other for the duration of the monitoring interval (see also Figure 2).

###  2.1 Data Reduction and Reprocessing

The NICER observations were processed with HEAsoft 6.29 nicerl2 task, with the
overonly range extended to 1.5, as the overonly ranges of many of the data
often exceeded this value (see Table 1). Overshoots (the overonly range)
measure the high energy particle backgrounds, while undershoots (the underonly
range) are detector resets which measure red noise and any optical light.
111https://heasarc.gsfc.nasa.gov/docs/nicer/data_analysis/workshops/NICER-
Workshop-Filtering-Markwardt-2021.pdf The spectra were extracted with xselect,
and we implemented optimal binning
222https://heasarc.gsfc.nasa.gov/docs/nicer/analysis_threads/spectrum-
grouping/ with no minimum number of counts (Kaastra & Bleeker, 2016). The
background spectra were generated using the nibackgen3C50 tool (Remillard et
al., 2021), and we used the nicerarf and nicerrmf tasks to generate individual
ARF and RMF files for each spectrum.

Figure 1: Rebinned Swift/BAT monitoring of SMC X-1 overlaid with observation times from the NICER monitoring campaign. Each observation is differentiated by data quality (green diamond are the highest count rates, yellow star is intermediate, red dot is the lowest), which is a function of observation length and where within the super-orbital period cycle the snapshot was taken. Table 1: NICER observations log. ObsID | Date | Duration | Overonly Range | Underonly Range | Count Rate
---|---|---|---|---|---
|  | (sec) |  | (ct/sec) |
4509012401 | 2022-01-05 | 1474 | <15 | <300 | 151
4509012301 | 2021-12-23 | 2411 | <1 | <306 | 5
4509012201 | 2021-12-10 | 1476 |  $<$11 | <200 | 249
4509012101 | 2021-11-27 | 1513 |  $<$100 | <200 | 6
4509012001 | 2021-11-17 | 955 |  $<$75 | <200 | 10
4509011901 | 2021-11-07 | 1259 |  $<$30 | <200 | 176
4509011701 | 2021-10-17 | 2533 | <1 | <200 | 195
4509011601 | 2021-10-05 | 1107 |  $<$10 | <200 | 10
4509011501 | 2021-10-03 | 350 |  $<$16 | <200 | 9
4509011401 | 2021-09-21 | 1476 |  $<$35 | <200 | 245
4509011301 | 2021-09-01 | 1898 |  $<$0.5 | <200 | 189
4509011201 | 2021-08-24 | 1327 |  $<$20 | <200 | 52
4509011101 | 2021-08-11 | 1123 |  $<$1 | <200 | 9
4509011001 | 2021-08-01 | 1235 |  $<$50 | <200 | 72
4509010901 | 2021-07-22 | 821 |  $<$20 | <200 | 179
4509010803 | 2021-07-11 | 944 |  $<$30 | <200 | 232
4509010802 | 2021-06-28 | 796 |  $<$20 | <200 | 10
4509010801 | 2021-06-26 | 1684 |  $<$1.5 | <200 | 139
4509010701 | 2021-06-16 | 1279 |  $<$2 | <200 | 206
4509010601 | 2021-05-21 | 1024 |  $<$4 | <200 | 146
4509010501 | 2021-05-16 | 1446 |  $<$140 | <200 | 7
4509010401 | 2021-05-07 | 967 |  $<$40 | <200 | 2
4509010301 | 2021-04-24 | 436 |  $<$20 | <200 | 195
4509010103 | 2021-04-13 | 483 |  $<$7 | <200 | 220
4509010102 | 2021-04-12 | 1362 |  $<$3 | <200 | 214
4509010101 | 2020-04-01 | 1006 |  $<$11 | <200 | 33

###  2.2 Determining changes in $P_{sup}$ from BAT monitoring

SMC X-1 has been observed to undergo three separate epochs of super-orbital
period excursion, which last for 2-3 years, occurring in 1997, 2006, and 2014
(Hu et al., 2019). While it is difficult to accurately predict the onset of
the next super-orbital period excursion, the $\sim$ 6 year difference gap
between previous super-orbital period excursions suggested that SMC X-1 would
undergo some type of super-orbital period excursion beginning in 2021.

Most techniques to perform accurate measurements of super-orbital
periodicities, such as the Lomb-Scargle periodogram (Scargle, 1982) often
require hundreds of days worth of data to perform a single period measurement.
While there is always a concern that red noise can cause problematic false
detections (Vaughan et al., 2016), SMC X-1’s super-orbital period is well-
known and has been observed with many repetitions.

Given that our monitoring campaign has not yet exceeded one year and is
comprised of short observations with days-long gaps, techniques such as the
Lomb-Scargle periodogram are not the most effective methods to constrain the
super-orbital period. However, phase dispersion minimization is one of the
techniques enabling us to measure periods from these data sets.

To quantify any super-orbital period changes between the 2016 and 2021 epochs,
we implement the python package P4J, where the criterion for period detection
is based on maximizing the Cauchy-Schwarz Quadratic Mutual Information (Huijse
et al., 2018) 333https://github.com/phuijse/P4J to measure the best super-
orbital period of the BAT data taken from MJD 59290-59606, which covers 5
super-orbital cycles (we removed the cycle before MJD 59290 because the data
gap was problematic), and interpolated a sine model from the best fit period
($P_{sup}$=51d). For comparison, we implemented the same methodology on 5
cycles of the BAT data from MJD 57535-57788 (during the time the comparison
XMM-Newton observations were taken). P4J returned the best fit period of 53
days, and the 2016 cycles appear to be self-consistent. We superimpose these
models over-top the Swift/BAT monitoring in Figure 2. While the XMM-Newton
epoch can be represented by a sinusoidal model with the period derived from
P4J, the NICER epoch cannot all be fit by the modeled P4J period for that
epoch, which implies that the super-orbital period is not a constant value
during this time. This suggests that the MOOSE data set is unique because it
provides both spectral and timing windows into unprobed regimes of $P_{sup}$
for SMC X-1.

Figure 2: Comparison of the super-orbital period cycle during the 2016 XMM-
Newton observations and the current NICER monitoring campaign. We fit the two
different epochs of Swift/BAT observations with P4J and overlay the model with
best-fit parameters on the image. The dashed blue lines on the left panel are
the cycle that was observed by XMM-Newton, while the dotted lines show the
best-fit period over the whole data-set used to measure the period. The left
panel shows the P4J model during the NICER monitoring.

###  2.3 Monitoring of the High-State Continuum during the excursion

Figure 3: Best-fit spectrum and residuals of ObsID 4509011701. The spectrum
was fit with tbabs*(bbody+cutoffpl+gaussian+gaussian+gaussian). The Gaussian
components for line energy and $\sigma$ were fixed to the values from Brumback
et al. (2020). The xspec model components are as follows: bbody (red dotted
line), cutoffpl (black dash-dotted line), and the three Gaussian components
(blue dashed line). The uncertainties in the XMM-Newton observations are
likely underestimated given the high reduced $\chi^{2}$ values. Figure 4:
Best-fit parameters of $N_{\textrm{H}}$, kT and $\Gamma$ for high-state
observations, with Swift/BAT light curve for reference. These parameters can
be found in Table 2.

As an initial step to monitor the continuum, we fit spectra of the high-state
observations using xspec version 12.12.0, with solar abundances from Wilms et
al. (2000) and $\chi^{2}$ statistics, with photo-electric absorption cross-
sections from Verner et al. (1996). All reported uncertainties are at 90%
confidence, and we implemented optimal binning
444https://heasarc.gsfc.nasa.gov/docs/nicer/analysis_threads/spectrum-
grouping/. The high-state observations can be described by the model
tbabs*(bbody+cutoffpl) with three gaussian components added to model emission
lines (see Figure 3). We restricted the search range of the high-energy cut-
off value between 12 keV and 20keV to allow the fits to more closely match
Brumback et al. (2020). While NICER does not extend to this range, the
inclusion of a hard component is necessary to fit the spectrum. SMC X-1 is
known to produce emission from Fe K$\alpha$ (6.4 keV, broad), Ne X Ly$\alpha$
(1.2 keV), and O VIII Ly$\alpha$ (0.65 keV), and we fix our emission lines to
the same line energy and width as Brumback et al. (2020). The low/intermediate
state spectra are not well-fit by this model. We will undertake a
comprehensive examination of all of the low/intermediate state data once the
full MOOSE dataset has been acquired.

For comparison, we fit the same models to the high state XMM-Newton
observations from Brumback et al. (2020) (ObsIDs 0784570201, 0784570301,
0784570501; see Brumback et al. 2020 for analysis details), which were
obtained in 2016 when the source was not in a state of super-orbital period
modulation.

The best-fit parameters for $N_{\textrm{H}}$, kT and $\Gamma$ are presented in
Table 2 and can be seen in Figure 4. We note that given the spectral
resolution of XMM-Newton, the models we use are likely over-simplistic,
however, we use the same models for XMM-Newton and NICER for sake of
consistency and ease of comparison across the epochs.

Table 2: Best-fit spectral parameters for high-state observations from each super-orbital period epoch. The NICER and XMM-Newton observations are fit in the 0.3-12 keV energy range. ObsID | Date |  $N_{\textrm{H}}$ (1021cm-2) | kT (keV) | $\Gamma$ |  $\chi^{2}$ /d.o.f.
---|---|---|---|---|---
07845770201 | 2016-09-08 | 2.02 $\pm$ 0.05 | 0.197 $\pm$ 0.002 | 0.569 $\pm$ 0.021 | 1043.28/208
07845770301 | 2016-09-19 | 2.02 $\pm$ 0.06 | 0.198 $\pm$ 0.003 | 0.603 $\pm$ 0.026 | 888.42/206
07845770501 | 2016-10-24 | 2.1 $\pm$ 0.04 | 0.195 $\pm$ 0.002 | 0.63 $\pm$ 0.018 | 2026.07/216
4509010103 | 2021-04-13 | 2.4 $\pm$ 0.1 | 0.20 $\pm$ 0.01 | 0.75 $\pm$0.11 | 172.35/138
4509010301 | 2021-04-24 | 2.6 $\pm$ 0.1 | 0.19 $\pm$ 0.01 | 0.58 0.11 | 152.06/134
4509010701 | 2021-06-16 | 2.4 $\pm$ 0.1 | 0.20 $\pm$ 0.01 | 0.63 $\pm$ 0.07 | 286.20 /153
4509010803 | 2021-07-11 | 2.5 $\pm$ 0.1 | 0.20 $\pm$ 0.01 | 0.73 $\pm$ 0.08 | 191.66/150
4509010901 | 2021-08-01 | 2.4 $\pm$ 0.1 | 0.19 $\pm$ 0.01 | 0.66 $\pm$ 0.08 | 198.70/147
4509011301 | 2021-09-01 | 2.4 $\pm$ 0.1 | 0.20 $\pm 0.01$ | 0.61 $\pm$0.01 | 211.42 /153
4509011401 | 2021-09-21 | 2.5 $\pm$ 0.1 | 0.19 $\pm$ 0.01 | 0.78 $\pm$ 0.07 | 272.90/152
4509011701 | 2021-10-17 | 2.5 $\pm$ 0.1 | 0.20 $\pm$ 0.01 | 0.66 $\pm$0.05 | 299.49/151
4509011901 | 2021-11-07 | 2.5 $\pm$ 0.1 | 0.20 $\pm$ 0.01 | 0.67 $\pm$0.05 | 196.29/145
4509012201 | 2021-12-10 | 2.4 $\pm$ 0.1 | 0.20 $\pm$ 0.01 | 0.73 $\pm$0.07 | 239.19/150
4509012401 | 2022-01-05 | 2.7 $\pm$ 0.1 | 0.19 $\pm$ 0.01 | 0.75 $\pm$ 0.03 | 208.44/138

##  3 Results and Discussion

We fit XMM-Newton and NICER high state spectra of SMC X-1 through different
$P_{sup}$ values and phases and find no significant spectral changes, except
for an apparent moderate variability in $\Gamma$. As the best-fit value for
$\Gamma$ depends on the high-energy cut-off, a parameter NICER is not
sensitive to (as the best-fit values are sometimes outside the NICER energy
range), we cannot yet conclude whether it is variable or not. As seen in
Figure 4, the best-fit $N_{\textrm{H}}$ increases between the XMM-Newton and
NICER observations (from an average XMM-Newton value of 2.1 (+/- 0.08)
$\times$ $10^{21}$ cm-2 to an average NICER value 2.5 ($\pm$ 0.01)$\times$
$10^{21}$ cm-2 ). However, the temperature remains relatively stable with both
XMM-Newton and NICER giving an average kT of 0.20 keV to a precision of
$<$0.04 keV.

There appears to be slight variability in $\Gamma$, although that may be due
to a degeneracy with the high-energy cut-off parameter, which NICER is not
sensitive to (see next section and Fig 5). Thus, this study cannot comment on
any potential variability of the power-law index. Our observations show that
the overall spectral shape remains remarkably steady across the different
superorbital modulations and phases covered here.

###  3.1 Comparison to Previous Spectral Studies of SMC X-1

Some comparisons can be made between spectral fits to the MOOSE high-state
observations and SMC X-1 spectral fits from the literature. In particular,
Naik & Paul (2004) observed SMC X-1 with BeppoSAX while SMC X-1 was in the
onset of the first observed super-orbital period excursion (early 1997). Their
spectra contained similar features to ASCA observations from Paul et al.
(2002): a 6.4 keV iron emission line, a power-law component and a soft excess.
However, the BeppoSAX observations were not sufficiently long enough to place
constraints on variation in the soft excess between Paul et al. (2002) and
Naik & Paul (2004). We find that the MOOSE observations have the same
features, and remarkably similar X-ray fits (over the same energy ranges)
across several years and different instruments, e.g. Naik & Paul (2004),
yielding soft component kT values between 0.16-0.19 keV. Fits by Brumback et
al. (2020) find that within different super-orbital phases, the overall
spectral shape of the high-state remains similar, with any changes occurring
in the normalization of the Fe line, or small differences in the primary
power-law index.

Figure 5: Best fit power-law index compared to best fit high energy cut value.
NICER is not sensitive to the same energy ranges as XMM-Newton, and thus had a
wider range of best-fit values for the high energy cut, which in turn impacted
the corresponding value for the power-law index.

The hard bands of SMC X-1 have also been observed many times, with RXTE,
Suzaku and NuSTAR: Inam et al. (2010) fit RXTE/PCA (3-25 keV) observations
with a cutoff power-law only, but find that the only parameter that changes
within the error-bars is $N_{\textrm{H}}$, which is anti-correlated to the
X-ray flux. Fits by Pike et al. (2019) found that the spectrum in the low-
state was poorly constrained due to the low number of source counts, and that
a low-temperature (kT$<$ 0.5 keV) component was necessary to model excess soft
flux. This component was not recovered by Pradhan et al. (2020) fits to Suzaku
and NuStar spectra, but they did find evidence for the hard X-ray spectrum
becoming steeper as the flux increased.

While we do detect some changes in the best fit values for $N_{\textrm{H}}$
between the 2016 XMM-Newton observations and the 2021-2022 NICER observations,
it is not clear at this point whether the differences are due to calibration
(see Brumback et al. 2020 for more detail on the XMM-Newton observations), or
potential disc changes during excursion. $\Gamma$ also appears to show some
variance within the NICER observations.However, we note that there is a
degeneracy between the best-fit value for the high-energy cut-off, which
extends beyond NICER’s energy range, and thus it is not clear whether the
power-law index shows intrinsic variability, or rather if the fit was not well
constrained (see Figure 5). Thus, this study is not well-equipped to comment
on variability in the power-law index.

###  3.2 Comparison of SMC X-1 to ULX Pulsars with Super-Orbital Periods

SMC X-1 is known to reach high X-ray luminosities that are consistent with
being near or above its Eddington limit (for a 1.1 solar mass NS; Bonnet-
Bidaud & van der Klis (1981); Van der Meer et al. (2007); Pike et al. (2019).
The richness of observational data on SMC X-1 makes it an ideal target for
comparison with extragalactic ultraluminous X-ray pulsars, as several have
been found to exhibit evidence for super-orbital periodicities, including NGC
7793 P13; (Motch et al., 2014), NGC 5907 ULX1; (Walton et al., 2016), M82 X-2;
(Brightman et al., 2019), M51 ULX7; (Brightman et al., 2020). Due to the
distance of these galaxies, it is challenging to perform detailed measurements
of the pulse profiles, and thus the main observables utilised to understand
these systems are the spectral shape and spectral evolution. Thus, it is
potentially useful to be able to compare the long-term monitoring of SMC X-1’s
spectral behaviour to studies of these systems.

While it is challenging to perform spectral studies of M82 X-2 due to its
nearby neighbour (Bachetti et al., 2020), the other three systems have been
well-studied. Broad-band spectroscopy by Walton et al. (2018) found that the
spectrum of NGC 7793 P13 was best fit by two soft components as well as a
third, hard continuum component. X-ray spectroscopy of high cadence
observations by Lin et al. (2022) found that NGC 7793 P13 has both a hard and
a soft component, although it is unclear if the source’s spectrum shows
statistically significant changes over the observations.

Rodríguez Castillo et al. (2020) determined that M51 ULX7 was best fit by a
black-body disc with a second (hotter) black-body model. A study of the
source’s spectral evolution by Brightman et al. (2022) found that there is not
sufficient evidence for variation of the spectral shape either as a function
of super-orbital phase or X-ray luminosity.

Studies of NGC 5907 ULX1 found that the source’s spectrum was well
characterised by a soft disc component with a power-law tail. Unlike the
previous studies of the aforementioned ULXPs, spectral fits to observations of
NGC 5907 ULX1 in 2003, 2012, and 2013 suggest that there is some evidence for
spectral change as a function of phase (Fürst et al., 2017).

In the new MOOSE spectral monitoring of SMC X-1’s high-state continuum, where
we have probed a number of different super-orbital phases, we currently do not
find strong evidence of intrinsic changes to SMC X-1’s high-state continuum
spectrum, although there may be some evidence for changes in $N_{\textrm{H}}$
or $\Gamma$. Given that past studies have found that many systems find state
changes linked to the mass transfer rate (Kotze & Charles, 2012, and
references therein), the lack of clear state changes during SMC X-1’s
excursion suggest that the relationship between the super-orbital period
excursion and the high-state spectral parameters is somewhat complicated.

##  4 Summary and Future Studies

We present the first full monitoring coverage which affords both high quality
spectra and pulse profiles of SMC X-1 as it enters its fourth observed epoch
of super-orbital period excursion.

Our initial findings are summarized as follows:

  * •

We find that the new observations probe many different super-orbital phases
and different super-orbital period values, and as such will provide a rich and
diverse data-set that will be key to understanding the driving causes behind
super-orbital period modulations.

  * •

Fits to the spectral continuum are best-characterised by an absorbed black
body disk with kT $\sim 0.19$ keV, and a hard power-law component
($\Gamma\sim$ 0.7). Despite having probed different super-orbital periods and
super-orbital period phases, we do not detect clear and significant evidence
of changes to the continuum.

  * •

We note a $>$3 $\sigma$ increase in $N_{\textrm{H}}$ (with kT relatively
fixed) from the 2016 XMM-Newton observations to the 2021-2022 NICER
monitoring, although this could be due to differences in the instruments (e.g.
Schellenberger et al., 2015), rather than any physical changes.

  * •

While the time averaged X-ray spectroscopy is a useful tool to make
comparisons with other systems which may have similar accretion mechanisms, we
see little evidence for spectral changes to the high-state as a function of
super-orbital excursion.

  * •

NICER monitoring of SMC X-1 via MOOSE will continue through early 2023. We
will release all data products at the end of the monitoring period to cement
the legacy value of this program.

The next step will be to perform a full study of pulse profiles (Brumback et
al in prep), as well as phase-resolved spectroscopy and detailed study of any
variability in the emission lines as a function of orbital and super orbital
period, to delve into the physical origins of the super-orbital period
modulation.

## Acknowledgements

The authors thank the anonymous referee for helpful feedback. KCD and DH
acknowledge funding from the Natural Sciences and Engineering Research Council
of Canada (NSERC), the Canada Research Chairs (CRC) program, and the McGill
Bob Wares Science Innovation Prospectors Fund. KCD acknowledges fellowship
funding from the McGill Space Institute. DA acknowledges support from the
Royal Society. The authors thank Elizabeth Ferrara and the NICER team for
their efforts with the observation scheduling. This research has made use of
data and/or software provided by the High Energy Astrophysics Science Archive
Research Center (HEASARC), which is a service of the Astrophysics Science
Division at NASA/GSFC and NASA’s Astrophysics Data System Bibliographic
Services, as well as the software packes numpy (Van der Walt et al., 2011) and
matplotlib (Hunter, 2007).

## Data Availability

The NICER and XMM-Newton observations are publicly available on HEASARC, and
the Swift/BAT (Krimm et al., 2013) lightcurves can be downloaded from
https://swift.gsfc.nasa.gov/results/transients/SMCX-1/

## References

  * Bachetti et al. (2014) Bachetti M., et al., 2014, Nature, 514, 202
  * Bachetti et al. (2020) Bachetti M., et al., 2020, ApJ, 891, 44
  * Blondin & Woo (1995) Blondin J. M., Woo J. W., 1995, ApJ, 445, 889
  * Bonnet-Bidaud & van der Klis (1981) Bonnet-Bidaud J. M., van der Klis M., 1981, A&A, 97, 134
  * Brightman et al. (2019) Brightman M., et al., 2019, ApJ, 873, 115
  * Brightman et al. (2020) Brightman M., et al., 2020, ApJ, 895, 127
  * Brightman et al. (2022) Brightman M., et al., 2022, ApJ, 925, 18
  * Brumback et al. (2020) Brumback M. C., Hickox R. C., Fürst F. S., Pottschmidt K., Tomsick J. A., Wilms J., 2020, ApJ, 888, 125
  * Clarkson et al. (2003a) Clarkson W. I., Charles P. A., Coe M. J., Laycock S., Tout M. D., Wilson C. A., 2003a, MNRAS, 339, 447
  * Clarkson et al. (2003b) Clarkson W. I., Charles P. A., Coe M. J., Laycock S., 2003b, MNRAS, 343, 1213
  * Corbet & Krimm (2013) Corbet R. H. D., Krimm H. A., 2013, ApJ, 778, 45
  * Dage et al. (2019) Dage K. C., Clarkson W. I., Charles P. A., Laycock S. G. T., Shih I. C., 2019, MNRAS, 482, 337
  * Foulkes et al. (2006) Foulkes S. B., Haswell C. A., Murray J. R., 2006, MNRAS, 366, 1399
  * Foulkes et al. (2010) Foulkes S. B., Haswell C. A., Murray J. R., 2010, MNRAS, 401, 1275
  * Fürst et al. (2017) Fürst F., Walton D. J., Stern D., Bachetti M., Barret D., Brightman M., Harrison F. A., Rana V., 2017, ApJ, 834, 77
  * Gendreau et al. (2016) Gendreau K. C., et al., 2016, in den Herder J.-W. A., Takahashi T., Bautz M., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 9905, Space Telescopes and Instrumentation 2016: Ultraviolet to Gamma Ray. p. 99051H, doi:10.1117/12.2231304
  * Gruber & Rothschild (1984) Gruber D. E., Rothschild R. E., 1984, ApJ, 283, 546
  * Hickox & Vrtilek (2005) Hickox R. C., Vrtilek S. D., 2005, ApJ, 633, 1064
  * Hu et al. (2011) Hu C.-P., Chou Y., Wu M.-C., Yang T.-C., Su Y.-H., 2011, ApJ, 740, 67
  * Hu et al. (2013) Hu C.-P., Chou Y., Yang T.-C., Su Y.-H., 2013, ApJ, 773, 58
  * Hu et al. (2019) Hu C.-P., Mihara T., Sugizaki M., Ueda Y., Enoto T., 2019, ApJ, 885, 123
  * Huang et al. (1998) Huang N. E., et al., 1998, Proceedings of the Royal Society of London Series A, 454, 903
  * Huijse et al. (2018) Huijse P., Estévez P. A., Förster F., Daniel S. F., Connolly A. J., Protopapas P., Carrasco R., Príncipe J. C., 2018, ApJS, 236, 12
  * Hunter (2007) Hunter J. D., 2007, Computing in Science and Engineering, 9, 90
  * Inam et al. (2010) Inam S. Ç., Baykal A., Beklen E., 2010, MNRAS, 403, 378
  * Kaastra & Bleeker (2016) Kaastra J. S., Bleeker J. A. M., 2016, A&A, 587, A151
  * Kotze & Charles (2012) Kotze M. M., Charles P. A., 2012, MNRAS, 420, 1575
  * Krimm et al. (2013) Krimm H. A., et al., 2013, ApJS, 209, 14
  * Lin et al. (2022) Lin L. C.-C., Hu C.-P., Takata J., Li K.-L., Hui C. Y., Kong A. K. H., 2022, ApJ, 924, 65
  * Lucke et al. (1976) Lucke R., Yentis D., Friedman H., Fritz G., Shulman S., 1976, ApJ, 206, L25
  * Marshall et al. (1983) Marshall F. E., White N. E., Becker R. H., 1983, ApJ, 266, 814
  * Motch et al. (2014) Motch C., Pakull M. W., Soria R., Grisé F., Pietrzyński G., 2014, Nature, 514, 198
  * Naik & Paul (2004) Naik S., Paul B., 2004, A&A, 418, 655
  * Neilsen et al. (2004) Neilsen J., Hickox R. C., Vrtilek S. D., 2004, ApJ, 616, L135
  * Ogilvie & Dubus (2001) Ogilvie G. I., Dubus G., 2001, MNRAS, 320, 485
  * Paul et al. (2002) Paul B., Nagase F., Endo T., Dotani T., Yokogawa J., Nishiuchi M., 2002, ApJ, 579, 411
  * Pike et al. (2019) Pike S. N., et al., 2019, ApJ, 875, 144
  * Pradhan et al. (2020) Pradhan P., Maitra C., Paul B., 2020, ApJ, 895, 10
  * Pringle (1996) Pringle J. E., 1996, MNRAS, 281, 357
  * Remillard et al. (2021) Remillard R. A., et al., 2021, arXiv e-prints, p. arXiv:2105.09901
  * Reynolds et al. (1993) Reynolds A. P., Hilditch R. W., Bell S. A., Hill G., 1993, MNRAS, 261, 337
  * Rodríguez Castillo et al. (2020) Rodríguez Castillo G. A., et al., 2020, ApJ, 895, 60
  * Scargle (1982) Scargle J. D., 1982, ApJ, 263, 835
  * Schellenberger et al. (2015) Schellenberger G., Reiprich T. H., Lovisari L., Nevalainen J., David L., 2015, A&A, 575, A30
  * Schreier et al. (1972) Schreier E., Giacconi R., Gursky H., Kellogg E., Tananbaum H., 1972, ApJ, 178, L71
  * Thomas et al. (2022) Thomas J. K., Charles P. A., Buckley D. A. H., Kotze M. M., Lasota J.-P., Potter S. B., Steiner J. F., Paice J. A., 2022, MNRAS, 509, 1062
  * Townsend & Charles (2020) Townsend L. J., Charles P. A., 2020, MNRAS, 495, 139
  * Trowbridge et al. (2007) Trowbridge S., Nowak M. A., Wilms J., 2007, ApJ, 670, 624
  * Van der Meer et al. (2007) Van der Meer A., Kaper L., van Kerkwijk M. H., Heemskerk M. H. M., van den Heuvel E. P. J., 2007, A&A, 473, 523
  * Van der Walt et al. (2011) Van der Walt S., Colbert S. C., Varoquaux G., 2011, Computing in Science and Engineering, 13, 22
  * Vaughan et al. (2016) Vaughan S., Uttley P., Markowitz A. G., Huppenkothen D., Middleton M. J., Alston W. N., Scargle J. D., Farr W. M., 2016, MNRAS, 461, 3145
  * Verner et al. (1996) Verner D. A., Ferland G. J., Korista K. T., Yakovlev D. G., 1996, ApJ, 465, 487
  * Vrtilek et al. (2005) Vrtilek S. D., Raymond J. C., Boroson B., McCray R., 2005, ApJ, 626, 307
  * Walton et al. (2016) Walton D. J., et al., 2016, ApJ, 827, L13
  * Walton et al. (2018) Walton D. J., et al., 2018, MNRAS, 473, 4360
  * Wilms et al. (2000) Wilms J., Allen A., McCray R., 2000, ApJ, 542, 914
  * Wojdowski et al. (1998) Wojdowski P., Clark G. W., Levine A. M., Woo J. W., Zhang S. N., 1998, ApJ, 502, 253
  * Wojdowski et al. (2000) Wojdowski P. S., Clark G. W., Kallman T. R., 2000, ApJ, 541, 963