# Mitigating Word Bias in Zero-shot Prompt-based Classifiers

Adian Liusie, Potsawee Manakul, Mark J. F. Gales
ALTA Institute, Department of Engineering, University of Cambridge
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Prompt-based classifiers are an attractive approach for zero-shot
classification. However, the precise choice of the prompt template and label
words can largely influence performance, with semantically equivalent settings
often showing notable performance difference. This discrepancy can be partly
attributed to word biases, where the classifier may be biased towards classes.
To address this problem, it is possible to optimise classification thresholds
on a labelled data set, however, this mitigates some of the advantages of
prompt-based classifiers. This paper instead approaches this problem by
examining the expected marginal probabilities of the classes. Here,
probabilities are reweighted to have a uniform prior over classes, in an
unsupervised fashion. Further, we draw a theoretical connection between the
class priors and the language models’ word prior, and offer the ability to set
a threshold in a zero-resource fashion. We show that matching class priors
correlates strongly with the oracle upper bound performance and demonstrate
large consistent performance gains for prompt settings over a range of NLP
tasks.111code available on github at https://github.com/adianliusie/robust-
prompt-classifier

##  1 Introduction

Large language models (LLM) have shown impressive general ability for natural
language processing (NLP) tasks. LLMs can effectively handle a range of NLP
tasks through ‘prompting’, where a natural language instruction is added to
the input, conditioning the model to the task at hand. Prompting can either be
an emergent ability learned through scaling up model size Brown et al. (2020);
Wei et al. (2022) or an ability learned through instruction tuning Wei et al.
(2021); Chung et al. (2022); Ouyang et al. (2022). Despite the recent
popularity of prompting, there is a known sensitivity of prompt-based LLMs to
elements such as prompt template and label words Gao et al. (2021); Schick and
Schütze (2021). Previous works have demonstrated that prompt templates can
significantly impact task performance (Shin et al., 2020; Zhou et al., ) and
that factors such as chosen label words can influence system performance for
classification tasks (Zhao et al., 2021; Holtzman et al., 2021).

Figure 1: Instead of using the raw LM output probabilities of the label words,
we consider mitigating bias by finding weights that make the classifier
unbiased over classes. This is connected to normalising by word priors, which
we use as a zero-resource de-biasing approach.

This work focuses on the influence of ‘word biases’ for prompt-based
classifiers. i.e. the bias that prompts may have towards certain classes,
independent of the input text. To account for this bias, one could use a
labelled dataset to find optimal class decision thresholds. This, however,
requires labelled task data, which may limit the zero-shot benefits of prompt-
based classifiers. We propose a simple unsupervised solution of re-weighting
probabilities, where we use unlabelled data to search for weight parameters
that ensure a uniform prior over classes. We show that this prior matching
leads to greater robustness for diverse prompt settings and that the
unsupervised weights which debias the classifier is highly correlated with the
oracle weights that maximise accuracy. Further, we provide theoretical
analysis that draws a connection between word priors and inherent class bias,
which we use to motivate a zero-resource normalisation approach that is
competitive with prior matching. Overall, we demonstrate that our unsupervised
approach highly reduces sensitivity to the chosen prompt and label words, and
that settings which initially fail can often be made effective through a
simple probability re-weighting.

Our contributions are 1) We propose a simple unsupervised probability re-
weighting method, and empirically demonstrate greater robustness to prompt and
label word choice, with large accuracy gains across prompt settings for a
range of standard NLP tasks. 2) We theoretically connect the weight parameters
to word priors and use this to motivate a zero-resource re-weighting approach.
3) We show that the weights of prior matching are highly correlated with the
optimal oracle weights that maximize accuracy, illustrating that our approach
is a near-optimal use of a system’s output probabilities.

##  2 Mitigiating Bias by Re-weighting

Prompt-based classifiers Given an input sequence
$\bm{\mathrm{x}}\\!\in\\!\mathcal{X}$, large language models (LLMs) model
$P_{\theta}(\bm{\mathrm{w}}|\bm{\mathrm{x}})$, the output probability
distribution over all possible sequences
$\bm{\mathrm{w}}\\!\in\\!\mathcal{X}$. For a classification task
$\mathcal{T}$, a prompt-based classifier 1) reformats the input text $x$ to
prompt $\bm{\mathrm{p}}\in\mathcal{X}$ by including the task instruction, and
2) selects class words $\\{w_{i}\\}_{1:K}$ which are associated to each output
class $\\{y_{i}\\}_{1:K}$. For example in sentiment classification, one can
use prompt ‘what is the sentiment of the following review? <x>’, (where <x> is
the current input $x$, e.g. ‘Inception was absolutely brilliant’), and class
words $w_{0}$=bad and $w_{1}$=good for the negative and positive classes
respectively. For a prompt classifier,
$Q=\\{\bm{\mathrm{p}},\\{w_{i}\\}_{1:K}\\}$, class probabilities can be set to
be proportional to the probability of the associated class word, where the
final decision $\hat{y}$ is the class with the highest probability Zhao et al.
(2021); Jiang et al. (2020).

$\displaystyle\tilde{P}_{\theta}(y_{k}|\bm{\mathrm{x}},Q)$
$\displaystyle=\frac{P_{\theta}(w_{k}|\bm{\mathrm{p}}(\bm{\mathrm{x}}))}{\sum_{w_{i}}P_{\theta}(w_{i}|\bm{\mathrm{p}}(\bm{\mathrm{x}}))}$
(1) $\displaystyle\hat{y}$
$\displaystyle=\underset{k}{\text{argmax}}\;\tilde{P}_{\theta}(y_{k}|\bm{\mathrm{x}},Q)$
(2)

However, as a large language model, the prompt-based classifier may return
probabilities that are influenced by distributional statistics of words
Gardner et al. (2021); Liusie et al. (2022). This may lead to inherent class
bias, where label words may have high probability not because they better
answer the prompt, but because they have a high LM prior.

Optimal Weights To account for this, one can define weight parameters
$\bm{\alpha}=\\{\alpha_{i}\\}_{1:K}$, where each $\alpha_{i}\in\mathbb{R}^{+}$
scales the probabilities of the classifier,

$\displaystyle\hat{P}_{\theta}(y_{k}|\bm{\mathrm{x}},Q,\bm{\alpha})=\frac{\alpha_{k}\tilde{P}_{\theta}(y_{k}|\bm{\mathrm{x}},Q)}{\sum_{i}\alpha_{i}\tilde{P}_{\theta}(y_{i}|\bm{\mathrm{x}},Q)}$
(3)

Given labelled task dataset
$\mathcal{D}=\\{(\bm{\mathrm{x}}^{(j)},y^{(j)})\\}_{j=1}^{N}$, one can then
find the optimal weights $\bm{\alpha}^{*}$ that maximises the accuracy of the
prompt classifier $\hat{P}_{\theta}(y_{k}|\bm{\mathrm{x}},Q,\bm{\alpha})$ over
the dataset,

$\displaystyle\bm{\alpha}^{*}$
$\displaystyle=\underset{\bm{\alpha}}{\text{argmax}}\;\text{Accuracy}(Q,\bm{\alpha},\mathcal{D})$
(4)

method | inputs | labels | imdb | rt | amazon | snli | mnli | qqp
---|---|---|---|---|---|---|---|---
baseline | ✗ | ✗ | 85.4$\pm$12.7 | 78.8$\pm$14.0 | 86.0$\pm$13.8 | 45.2$\pm$13.7 | 43.5$\pm$11.3 | 65.4$\pm$14.0
null-input | ✗ | ✗ | 92.1$\pm$3.2 | 89.1$\pm$3.8 | 95.0$\pm$1.8 | 75.2$\pm$10.4 | 66.1$\pm$9.7 | 77.4$\pm$6.6
prior-match | ✓ | ✗ | 93.1$\pm$3.3 | 90.9$\pm$1.6 | 96.0$\pm$0.8 | 78.5$\pm$9.3 | 69.8$\pm$9.7 | 79.1$\pm$2.4
optimal | ✓ | ✓ | 93.5$\pm$2.7 | 91.2$\pm$1.5 | 96.1$\pm$0.7 | 79.4$\pm$8.2 | 70.8$\pm$8.6 | 82.3$\pm$2.8
Table 1: Average dataset accuracy and standard deviations, over all prompts
and label words. baseline and null-input are zero-resource classification
methods, prior matching uses the text inputs but not labels, while optimal is
an oracle approach that uses the labels to search for the best thresholds.
Results for FlanT5 large

Prior-Matching The previous approach requires labelled data, which may limit
the benefit of using prompt-based classifiers. As an alternative, one can find
the values $\bar{\bm{\alpha}}$ that ensure that the classifier is unbiased,
such that the class prior $\hat{P}(y_{k}|Q,\bm{\alpha})$ matches the true
prior $P(y_{k})$

$\displaystyle\hat{P}_{\theta}(y_{k}|Q,\bm{\alpha})$
$\displaystyle=\mathbb{E}_{\bm{\mathrm{x}}}\\{\hat{P}_{\theta}(y_{k}|\bm{\mathrm{x}},Q,\bm{\alpha})\\}$
(5)
$\displaystyle\approx\frac{1}{N}\sum_{j=1}^{N}\hat{P}_{\theta}(y_{k}^{(j)}|\bm{\mathrm{x}}^{(j)},Q,\bm{\alpha})$
(6) $\bar{\bm{\alpha}}=\underset{\bm{\alpha}}{\text{argmin}}\;\sum_{\forall
y_{k}}|\hat{P}_{\theta}(y_{k}|Q,\bm{\alpha})-P(y_{k})|$ (7)

A deterministic solution that exactly matches the distributions exists, which
can be found with a search with 1 degree of freedom (that can be accounted for
by setting $\alpha_{1}=1$). If there is no expected class bias, one can assume
equal probabilities over all classes,
$P(y_{k})=\mathcal{U}(y_{k})=\frac{1}{N}$. This approach is therefore
unsupervised and only requires text inputs
$\mathcal{D}_{x}=\\{\bm{\mathrm{x}}^{(j)}\\}_{j=1}^{M}$, which therefore can
be applied at inference to any test set.

Null-Input Approximation The dependence of prior-matching on unlabelled
dataset $\mathcal{D}_{x}$ is a drawback. In Appendix A, we show that one can
make the analytical approximation

$\bar{\alpha}_{k}\approx\frac{1}{\mathbb{E}_{x}\\{P_{\theta}(w_{k}|\bm{\mathrm{x}},Q)\\}}=\frac{1}{P_{\theta}(w_{k}|Q)}$
(8)

Inspired by Zhao et al. (2021), we consider a resource-free approximation of
the word prior (equation 8) by considering the output word probabilities of
the null input $\emptyset$ (i.e. an empty string).

$\displaystyle P_{\theta}(w_{k}|Q)$ $\displaystyle\approx
P_{\theta}(w_{k}|\bm{\mathrm{p}}(\emptyset))$ (9)

This enables a zero-resource approximation of weight parameters
$\bar{\bm{\alpha}}$.

##  3 Experiments

###  3.1 Experimental Setup

Data Experimental results are run on standard NLP benchmarks, including
sentiment classification (IMDB Maas et al. (2011), Rotten Tomatoes Pang and
Lee (2005) and amazon), natural language inference (SNLI Bowman et al. (2015)
and MNLI Williams et al. (2018)) and paraphrase detection (QQP Wang et al.
(2018)). Evaluation is reported on standard test sets, except for amazon
polarity where 5000 test examples were randomly sampled.

Models We use FlanT5 large222https://huggingface.co/google/flan-t5-large Chung
et al. (2022), a T5 with further instruction-tuning stage where the system was
trained in a multi-task fashion over 1,836 tasks, each prepended with a
natural instruction prompt. This work evaluates FlanT5 for different NLP tasks
with arbitrary prompting set ups. For each task, we select 6 prompt templates
and for binary classification tasks consider 25 possible class words pairs,
while for NLI we have 64 class word triplets (where all permutations of valid
class words are considered). All prompts and label words used are given in
Appendix B. Further experiments for FlanT5 base and Llama-2-chat can be found
in Appendix D.

Methods We consider 4 different methods to leverage LLM probabilities for
classification. Class word probability via equation 1 (baseline). Normalised
probabilities calculated using null-inputs priors via equation 9 (null-input).
Optimising $\alpha_{k}$ with a search to have unbiased class prior via
equation 7 (prior-match). The oracle upper-bound performance, found by
optimising the optimal accuracy threshold via equation 4 (optimal).

###  3.2 Experimental Results

Figure 2: boxplots of the accuracy of all label-word pairs for rotten
tomatoes, over all the considered prompts

Classification Robustness Table 1 shows the mean and standard deviation of
accuracies among all prompt and class word settings for a given task. We
observe large consistent gains from both re-weighting approaches, with prior-
matching increasing baseline accuracy by between 6.7% to 12.1% for sentiment
classification, 13.7% for qqp, and over 25% for natural language inference.
Prior-matching also demonstrates performance very similar to the oracle upper-
bound, often within 1%, showing that the unsupervised prior-match approach is
competitive with the supervised threshold search. Prior-matching also performs
better than null-input by a small margin in all tasks, where this small gap
confirms that the word-prior normalisation is a very reasonable zero-shot
approximation.

Prompt Robustness Figure 2 illustrates a boxplot of rotten tomatoes
performance over all class-words for each considered method, over all 6
prompts. As observed in Table 1, naively using raw label word probabilities
(dark blue) leads to considerable fluctuations in accuracy; some prompt and
label word settings lead to reasonable accuracy (92%+ accuracy), however there
is observed brittleness to label word choice, with many settings demonstrating
poor performance. Prior matching (green) leads to significant robustness, with
nearly all sensible settings above 85% accuracy. We further find that, as
shown in Table 1, the unsupervised approach has accuracies very comparable to
those when using optimal thresholds.

In Figure 3, we consider similar boxplots for SNLI and observe larger gains
through reweighting. This was as higher probabilities are often assigned to
the entailment and contradiction labels words, leading to under-classification
of the neutral class. We observe greater sensitivity to prompt choice and
label words for snli than as observed in rotten tomatoes, even with
reweighting.

Figure 3: boxplots of the accuracy of all label-word sets for snli, for the
first 3 prompts

Weight Alignment Figure 4 shows a scatter plot of the weights found by the
optimal threshold search $\bm{\alpha}^{*}$ (equation 4), with those found from
the unsupervised prior matching method $\bar{\bm{\alpha}}$ (equation 7) and
the zero-resource word prior approximation (equation 9). We see a clear linear
relationship between optimal and prior-match, illustrating that accounting for
the marginal bias is almost equivalent with maximising accuracy, however,
achieved in an unsupervised fashion. Null-input is also well correlated with
the optimal thresholds, but there is a less direct relationship. Similar
linear relationships are observed also for other binary-classification tasks
and prompts, as shown in Appendix C.

Figure 4: Scatter plot of the optimal weights $\bm{\alpha}^{*}$ (equation 4)
with the prior match weights $\bm{\bar{\alpha}}$ (equation 7) and the
approximation via null-input (equation 9), for all settings of prompt 1 on
amazon

##  4 Conclusions

This paper analyzes prompt-based classifiers and demonstrates that inherent
class bias is a significant factor that influences the sensitivity of the
system to prompt and label words. We propose an unsupervised approach of prior
matching, which we demonstrate performs competitively to the supervised
alternative of searching for optimal thresholds, while avoiding the need for
labelled data. We relate prior matching with word biases, and motivate a zero-
resource approach of debiasing model probabilities. We show that our methods
lead to practical approaches that reduce the sensitivity to design choices
such as prompts and label words.

## Limitations

This work considered sentiment classification, natural language inference, and
paraphrase detection, and could have been extended over a greater suite of
tasks to guarantee its effectiveness. Further, this paper ran experiments on
FlanT5 and Llama2, and this work has not yet explored a larger range of
prompted language models. FlanT5 has also been instruction-tuned on similar
tasks, so the findings may be limited in scenarios where known capabilities
have to be elicited from models robustly.

## Ethical Considerations

Though this work suggests methods to improve the robustness of prompt-based
classifiers to prompts and label words, this does not imply that all design
choices will work. In some set ups, the system may be ineffective and have
poor generalisation over the task. Deploying machine learning classifiers in
real-world classification settings has many associated risks, and careful
analysis should be made before deploying such systems.

## Acknowledgements

This work is supported by Cambridge University Press & Assessment (CUP&A), a
department of The Chancellor, Masters, and Scholars of the University of
Cambridge, and the Cambridge Commonwealth, European & International Trust.

## References

  * Bowman et al. (2015) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015\.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022.  Scaling instruction-finetuned language models.  _arXiv preprint arXiv:2210.11416_.
  * Gao et al. (2021) Tianyu Gao, Adam Fisch, and Danqi Chen. 2021.  Making pre-trained language models better few-shot learners.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 3816–3830.
  * Gardner et al. (2021) Matt Gardner, William Merrill, Jesse Dodge, Matthew E Peters, Alexis Ross, Sameer Singh, and Noah A Smith. 2021.  Competency problems: On finding and removing artifacts in language data.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 1801–1813.
  * Holtzman et al. (2021) Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021\.  Surface form competition: Why the highest probability answer isn’t always right.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7038–7051.
  * Jiang et al. (2020) Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020.  How can we know what language models know?  _Transactions of the Association for Computational Linguistics_ , 8:423–438.
  * Liusie et al. (2022) Adian Liusie, Vatsal Raina, Vyas Raina, and Mark Gales. 2022.  Analyzing biases to spurious correlations in text classification tasks.  In _Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 78–84, Online only. Association for Computational Linguistics.
  * Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011.  Learning word vectors for sentiment analysis.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 142–150, Portland, Oregon, USA. Association for Computational Linguistics.
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022\.  Training language models to follow instructions with human feedback.  _Advances in Neural Information Processing Systems_ , 35:27730–27744.
  * Pang and Lee (2005) Bo Pang and Lillian Lee. 2005.  Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales.  In _Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05)_ , pages 115–124, Ann Arbor, Michigan. Association for Computational Linguistics.
  * Schick and Schütze (2021) Timo Schick and Hinrich Schütze. 2021.  Exploiting cloze-questions for few-shot text classification and natural language inference.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 255–269.
  * Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020.  Autoprompt: Eliciting knowledge from language models with automatically generated prompts.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4222–4235.
  * Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018.  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ , pages 353–355, Brussels, Belgium. Association for Computational Linguistics.
  * Wei et al. (2021) Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021.  Finetuned language models are zero-shot learners.  In _International Conference on Learning Representations_.
  * Wei et al. (2022) Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022\.  Emergent abilities of large language models.  _arXiv preprint arXiv:2206.07682_.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Zhao et al. (2021) Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021.  Calibrate before use: Improving few-shot performance of language models.  In _International Conference on Machine Learning_ , pages 12697–12706. PMLR.
  * (19) Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba.  Large language models are human-level prompt engineers.  In _NeurIPS 2022 Foundation Models for Decision Making Workshop_.

##  Appendix A Derivation of Zero-Resource Equation

For a prompt classifier $Q=\\{\bm{\mathrm{p}},\\{w_{i}\\}_{1:K}\\}$, class
probabilities are assumed to be proportional to the probability of the
associated class word,

$\displaystyle\tilde{P}_{\theta}(y_{k}|\bm{\mathrm{x}},Q)$
$\displaystyle=\frac{P_{\theta}(w_{k}|\bm{\mathrm{p}}(\bm{\mathrm{x}}))}{\sum_{w_{i}}P_{\theta}(w_{i}|\bm{\mathrm{p}}(\bm{\mathrm{x}}))}$
(10)

Given the task dataset
$\mathcal{D}=\\{\\{\bm{\mathrm{x}}^{(j)},y^{(j)}\\}\\}_{j=1}^{N}$, one can
calculate the assumed prior of the prompt classifier over the output classes,

$\displaystyle\tilde{P}_{\theta}(y_{k}|Q)$
$\displaystyle=\mathbb{E}_{\bm{\mathrm{x}}}\big{\\{}\tilde{P}_{\theta}(y_{k}|Q,\bm{\mathrm{x}})\big{\\}}$
(11)
$\displaystyle\approx\frac{1}{N}\sum_{j=1}^{N}\tilde{P}_{\theta}(y_{k}|Q,\bm{\mathrm{x}}^{(j)})$
(12)
$\displaystyle=\frac{1}{N}\sum_{j=1}^{N}\frac{P_{\theta}(w_{k}|\bm{\mathrm{p}}(\bm{\mathrm{x}}^{(j)}))}{\sum_{w_{i}}P_{\theta}(w_{i}|\bm{\mathrm{p}}(\bm{\mathrm{x}}^{(j)}))}$
(13)

This can be compared to the actual prior of the task/domain,

$\displaystyle P(y_{k})\approx P(y_{k}|\mathcal{D})$
$\displaystyle=\frac{1}{N}\sum_{j=1}^{N}\mathbbm{1}(y,y^{(j)})$ (14)

If $\mathcal{D}$ is sufficiently large, then an unbiased classifier should
have a class prior similar to that approximated via the labels. However, if
they diverge, one may wish to debias the classifier by scaling class
probabilities by factors $\alpha_{k}$,

$\displaystyle\hat{P}_{\theta}(y_{k}|\bm{\mathrm{x}},Q,\bm{\alpha})$
$\displaystyle=\frac{\alpha_{k}\tilde{P}_{\theta}(y_{k}|\bm{\mathrm{x}},Q)}{\sum_{i}\alpha_{i}\tilde{P}_{\theta}(y_{i}|\bm{\mathrm{x}},Q)}$
(15)
$\displaystyle=\frac{\alpha_{k}P_{\theta}(w_{k}|\bm{\mathrm{x}},Q)}{Z(\bm{\mathrm{x}},Q,\bm{\alpha})}$
(16)

Where
$Z(\bm{\mathrm{x}},Q,\bm{\alpha})=\sum_{i}\alpha_{i}P_{\theta}(w_{i}|\bm{\mathrm{x}},Q,\bm{\alpha})$
and $P_{\theta}(w_{k}|\bm{\mathrm{x}},Q)\equiv
P_{\theta}(w_{k}|\bm{\mathrm{p}}(\bm{\mathrm{x}}))$. The parameters
$\bar{\bm{\alpha}}$ that lead to an unbiased classifier can then be determined
in a deterministic fashion.

$\bar{\bm{\alpha}}=\underset{\bm{\alpha}}{\text{argmin}}\;\sum_{\forall
y_{k}}|\hat{P}_{\theta}(y_{k}|Q,\bm{\alpha})-P(y_{k})|$ (17)

Note that by constraining $\alpha_{1}=1$, there will exist a deterministic
solution that ensures that $\hat{P}_{\theta}(y_{k}|Q,\bm{\alpha})=P(y_{k})$.
For given weight parameters $\bm{\alpha}$, Consider the prompt-classifier
priors, $\hat{P}_{\theta}(y_{k}|Q,\bm{\alpha})$. One can approximate this
using a Taylor series of the expectation of a ratio, yielding

$\displaystyle\hat{P}_{\theta}(y_{k}|Q,\bm{\alpha})$
$\displaystyle=\mathbb{E}_{\bm{\mathrm{x}}}[\hat{P}_{\theta}(y_{k}|\bm{\mathrm{x}},Q,\bm{\alpha})]$
(18)
$\displaystyle=\mathbb{E}_{\bm{\mathrm{x}}}[\frac{\alpha_{k}P_{\theta}(w_{k}|\bm{\mathrm{x}},Q)}{Z(\bm{\mathrm{x}},Q,\bm{\alpha})}]$
(19)
$\displaystyle\approx\frac{\mathbb{E}_{\bm{\mathrm{x}}}[\alpha_{k}P_{\theta}(w_{k}|\bm{\mathrm{x}},Q)]}{\mathbb{E}_{\bm{\mathrm{x}}}[Z(\bm{\mathrm{x}},Q,\bm{\alpha})]}$
(20) $\displaystyle=\frac{\alpha_{k}P_{\theta}(w_{k}|Q)}{Z(Q,\bm{\alpha})}$
(21)

By equating the predicted prior with the true prior, we find an approximation
for $\bar{\bm{\alpha_{k}}}$

$\displaystyle\hat{P}_{\theta}(y_{k}|Q)$ $\displaystyle=P(y_{k}|\mathcal{D})$
(22) $\displaystyle\frac{\alpha_{k}P_{\theta}(w_{k}|Q)}{Z(Q)}$
$\displaystyle=P(y_{k}|\mathcal{D})$ (23) $\displaystyle\Rightarrow\alpha_{k}$
$\displaystyle=\frac{Z(Q)\cdot P(y_{k}|\mathcal{D})}{P_{\theta}(w_{k}|Q)}$
(24)

A final insight is that in many cases it is assumed that there should be no
inherent class bias, and so $P(y_{k}|\mathcal{D})$ can be assumed to be
uniform and be included in the normalisation term.

##  Appendix B Prompts and Label Words

###  B.1 Sentiment Classification

prompt
---
classify the following review:
how was the movie?
which word best describes the text?
what is the sentiment?
what is the reviewer’s verdict?
is the following movie good or bad?
Table 2: sentiment classification prompts positive | negative
---|---
good | bad
great | terrible
amazing | poor
fantastic | horrible
positive | negative
Table 3: label words for sentiment classification

###  B.2 Natural Language Inference

prompt
---
is the second text an entailment of the first text?
does the second text directly follow from the first text?
are the texts related?
are the texts consistent?
does text 1 imply text 2?
can text 2 be logically derived from text 1?
does the hypothesis logically follow the premise?
Table 4: NLI prompts entailment | neutral | contradiction
---|---|---
yes | maybe | no
correct | unclear | incorrect
yeah | potentially | nope
follows | neutral | contradiction
Table 5: label words for NLI

###  B.3 Paraphrase Identification

prompt
---
is the second text a paraphrase of the first text?
are the two texts semantically equivalent?
are the texts paraphrases of each other?
do the texts have the same meaning?
is the meaning of text 1 the same as in text 2?
would the two texts be classified as paraphrases?
Table 6: NLI prompts paraphrase | not paraphrase
---|---
yes | no
correct | incorrect
yeah | not
positive | negative
true | false
Table 7: label words for sentiment classification

##  Appendix C Threshold Alignment Plots

Figure 5: weights alignment plot for rotten tomatoes Figure 6: weights
alignment plot for imdb

##  Appendix D Impact of LLM Choice

method | inputs | labels | imdb | rt | amazon | snli | mnli | qqp
---|---|---|---|---|---|---|---|---
baseline | ✗ | ✗ | 82.1$\pm$11.1 | 70.1$\pm$11.7 | 83.4$\pm$12.8 | 37.4$\pm$6.1 | 37.0$\pm$4.3 | 52.5$\pm$11.4
null-input | ✗ | ✗ | 87.5$\pm$3.2 | 78.5$\pm$4.8 | 91.1$\pm$2.3 | 41.8$\pm$3.8 | 40.2$\pm$4.1 | 53.9$\pm$10.2
prior-match | ✓ | ✗ | 89.1$\pm$2.4 | 80.8$\pm$2.9 | 92.0$\pm$1.3 | 44.7$\pm$6.3 | 41.8$\pm$3.8 | 58.5$\pm$5.5
optimal | ✓ | ✓ | 89.3$\pm$2.0 | 81.2$\pm$2.9 | 92.1$\pm$1.4 | 47.6$\pm$5.9 | 43.5$\pm$3.7 | 65.3$\pm$2.9
Table 8: Robustness performance when using FlanT5 base as the base LLM (set-up equivalent to Table 1). method | inputs | labels | imdb | rt | amazon | snli | mnli | qqp
---|---|---|---|---|---|---|---|---
baseline | ✗ | ✗ | 85.8$\pm$8.7 | 78.4$\pm$10.3 | 86.4$\pm$10.3 | 35.1$\pm$2.7 | 36.9$\pm$3.2 | 51.0$\pm$11.6
null-input | ✗ | ✗ | 87.4$\pm$6.9 | 83.2$\pm$6.6 | 90.7$\pm$6.4 | 37.9$\pm$5.4 | 39.4$\pm$3.7 | 51.8$\pm$8.4
prior-match | ✓ | ✗ | 90.5$\pm$3.1 | 86.3$\pm$3.7 | 93.1$\pm$2.4 | 39.5$\pm$5.4 | 41.2$\pm$3.1 | 52.6$\pm$1.9
optimal | ✓ | ✓ | 90.8$\pm$2.8 | 86.7$\pm$3.6 | 93.2$\pm$2.4 | 42.8$\pm$4.6 | 42.8$\pm$2.3 | 66.8$\pm$0.4
Table 9: Robustness performance when using Llama-2-chat 7B as the base LLM.

Tables 8 and 9 show the prompt-based classifier performance for the different
methods when using different FlanT5-base and LLama-2-chat 7B respectively. For
sentiment classification and natural language tasks, we similarly observe that
the various re-weighting methods lead to considerable boosts in accuracy. Both
null-norm and prior-match again lead to performance near that to the optimal
weights, with considerable performance boost over the baseline. However, for
paraphrase detection, we only observe moderate performance boosts over the
baseline setting with a larger performance discrepancy with the optimal
weights.

##  Appendix E Boxplots

Figure 7: boxplots of the accuracy of all label-word pairs on IMDB, over all
the considered prompts Figure 8: boxplots of the accuracy of all label-word
pairs on amazon, over all the considered prompts Figure 9: boxplots of the
accuracy of all label-word pairs on qqp, over all the considered prompts


# Correlations of the Riemann zeta function

Michael J. Curran<EMAIL_ADDRESS>Mathematical Institute,
University of Oxford, Oxford, OX2 6GG, United Kingdom.

###### Abstract.

Assuming the Riemann hypothesis, we investigate the shifted moments of the
zeta function

$M_{\bm{\alpha},\bm{\beta}}(T)=\int_{T}^{2T}\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt$

introduced by Chandee [4], where
$\bm{\alpha}=\bm{\alpha}(T)=(\alpha_{1},\ldots,\alpha_{m})$ and
$\bm{\beta}=(\beta_{1}\ldots,\beta_{m})$ satisfy $|\alpha_{k}|\leq T/2$ and
$\beta_{k}\geq 0$. We shall prove that

$M_{\bm{\alpha},\bm{\beta}}(T)\ll_{\bm{\beta}}T(\log
T)^{\beta_{1}^{2}+\cdots+\beta_{m}^{2}}\prod_{1\leq j<k\leq
m}|\zeta(1+i(\alpha_{j}-\alpha_{k})+1/\log T)|^{2\beta_{j}\beta_{k}}.$

This improves upon the previous best known bounds due to Chandee [4] and Ng,
Shen, and Wong [21], particularly when the differences
$|\alpha_{j}-\alpha_{k}|$ are unbounded as $T\rightarrow\infty$. The key
insight is to combine work of Heap, Radziwiłł, and Soundararajan [13] and work
of the author [7] with the work of Harper [12] on the moments of the zeta
function.

##  1\. Introduction

This paper is concerned with the shifted moments

$M_{\bm{\alpha},\bm{\beta}}(T)=\int_{T}^{2T}\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt,$
(1)

where $\bm{\alpha}=\bm{\alpha}(T)=(\alpha_{1},\ldots,\alpha_{m})$ and
$\bm{\beta}=(\beta_{1}\ldots,\beta_{m})$ satisfy $|\alpha_{k}|\leq T/2$ and
$\beta_{k}\geq 0$. These were first studied in general by Chandee [4], who
gave lower bounds assuming the $\beta_{k}$ are integers,
$\alpha_{k}=O(\log\log T)$, and $|\alpha_{j}-\alpha_{k}|=O(1)$. Chandee also
gave upper bounds assuming RH when $|\alpha_{j}-\alpha_{k}|=O(1)$ and
$\alpha_{k}=O(\log T)$ which are sharp up to a $(\log T)^{\varepsilon}$ loss.
Subsequently Ng, Shen, and Wong [21] removed the $(\log T)^{\varepsilon}$ loss
in the special case where $\bm{\beta}=(\beta,\beta)$ by using the work of
Harper [12] on the moments of the zeta, and they also gave bounds in the
larger regime $|\alpha_{1}+\alpha_{2}|\leq T^{0.6}$. More precisely, in this
range they proved

$M_{(\alpha_{1},\alpha_{2}),(\beta,\beta)}(T)\ll T(\log
T)^{2\beta^{2}}F(\alpha_{1},\alpha_{2},T)^{2\beta^{2}}$

where

$F(\alpha_{1},\alpha_{2},T)=\begin{cases}\min\left(|\alpha_{1}-\alpha_{2}|^{-1},\log
T\right)&|\alpha_{1}-\alpha_{2}|\leq 1/100\\\
\log(2+|\alpha_{1}-\alpha_{2}|)&|\alpha_{1}-\alpha_{2}|>1/100\end{cases}.$

Some special cases of the shifted moments $M_{\bm{\alpha},\bm{\beta}}(T)$ and
related objects have been studied unconditionally. For example the integral

$\int_{T}^{2T}\zeta(\tfrac{1}{2}+i(t+\alpha_{1}))\zeta(\tfrac{1}{2}-i(t+\alpha_{2}))dt$

akin to $M_{\bm{\alpha},\bm{\beta}}(T)$ with
$\bm{\beta}=(\tfrac{1}{2},\tfrac{1}{2})$ is fairly well understood. Here the
current state of the art comes from Atkinson’s formula for the mean square of
zeta [1] and Bettin’s work on the second moment of zeta with shifts of size
$T^{2-\varepsilon}$ [2]. The current state of the art for
$M_{\bm{\alpha},\bm{\beta}}(T)$ with $\bm{\beta}=(1,1)$ is due to Motohashi’s
explicit formula for the fourth moment of zeta [19, 20] and Kovaleva’s work
showing Motohashi’s formula holds for shifts of size $T^{6/5-\varepsilon}$
[17]. Finally in the case where $\bm{\beta}=(\beta,\beta)$, sharp upper bounds
for $\beta\leq 1$ and lower bounds for all $\beta\geq 0$ were obtained by the
author [7]. The goal of this paper is, assuming RH, to extend the work of Ng,
Shen, and Wong [21] to arbitrary $\bm{\alpha}$ and $\bm{\beta}$ and to give
stronger bounds in the regime where the $|\alpha_{j}-\alpha_{k}|$ are
unbounded.

######  Theorem 1.1.

If $\beta_{k}\geq 0$ and $|\alpha_{k}|\leq T/2$ for $k=1,\ldots,m,$ then

$M_{\bm{\alpha},\bm{\beta}}(T)\ll T(\log
T)^{\beta_{1}^{2}+\cdots+\beta_{m}^{2}}\prod_{1\leq j<k\leq
m}|\zeta(1+i(\alpha_{j}-\alpha_{k})+1/\log T)|^{2\beta_{j}\beta_{k}}.$

######  Remark.

Throughout this paper we will assume $T$ is large in terms of $\bm{\beta}$,
and whenever we use the Vinogradov notation $\ll$ we allow the implicit
constant to depend on $\bm{\beta}$. When we use big-Oh notation, however, then
the implicit constants will be absolute unless there is a subscript
$O_{\bm{\beta}}$ to indicate $\bm{\beta}$ dependence.

Note that our result agrees with Chandee’s conjecture for the size of
$M_{(\alpha_{1},\alpha_{2}),(\beta,\beta)}(T)$ in the regime
$|\alpha_{1}-\alpha_{2}|=O(1)$, and without this restriction our bound is the
same order of magnitude predicted by the famous recipe of Conrey, Farmer,
Keating, Rubinstein, and Snaith [6]. At heart, Theorem 1.1 is a statement
about how $\zeta(\tfrac{1}{2}+it)$ and $\zeta(\tfrac{1}{2}+i(t+\alpha))$ are
correlated for $t\in[T,2T]$ and $|\alpha|\leq T/2$. More precisely, it
predicts that $\zeta(\tfrac{1}{2}+it)$ and $\zeta(\tfrac{1}{2}+i(t+\alpha))$
are perfectly correlated on average when $|\alpha|\leq 1/\log T$, and
decorrelate like $|\zeta(1+i\alpha)|$ for $|\alpha|>1/\log T$. When
$\alpha\leq 1$, the Laurent expansion for zeta shows that we obtain the same
correlations predicted from random matrix theory. For larger $\alpha$, the
correlation is of order 1 on average, which can be seen by estimating moments
of zeta to the right of the 1-line. There are, however, long range
correlations coming from the primes; more precisely, from the extreme values
of zeta on the one line. This is to be expected, for the Keating Snaith
philosophy only predicts that random matrix theory is a good model for
$\zeta(\tfrac{1}{2}+it)$ in short intervals.

The starting point of the proof is to use the method of Soundararajan [24] and
Harper [12] to bound $\log|\zeta(\tfrac{1}{2}+it)|$ by a short Dirichlet
polynomial. Instead of following the argument of Harper, however, we treat the
exponential of this short Dirichlet polynomial in a way similar to the work of
Heap, Radziwiłł, and Soundararajan [13]. Using this method, the integrals that
arise can be evaluated by simply using the mean value theorem for Dirichlet
polynomials. These mean values are much easier to evaluate uniformly in the
shifts $\alpha_{k}$ than the integrals of products of shifted cosines that
appear when using Harper’s method [12] as Ng, Shen, and Wong do in [21]. This
difference allows us to obtain upper bounds for general shifts $\bm{\alpha}$
and exponents $\bm{\beta}$. The final ingredient is a more precise estimate of
the following sum

$\sum_{p\leq X}\frac{\cos(\delta\log p)}{p}$

coming from the theory of pretentious multiplicative functions, see Lemma 2.6.
This idea appeared in the author’s previous work on studying the second moment
of moments of zeta in short intervals [7]. This more precise estimate is what
allows us to improve the bound of Ng, Shen, and Wong [21] in the regime where
$|\alpha_{j}-\alpha_{k}|$ is unbounded.

## Acknowledgements

The author would like to thank Paul Bourgade, Hung Bui, and Maksym Radziwiłł
for bringing the author’s attention to the works of Chandee [4] and Ng, Shen,
and Wong [21]. The author would also like to thank his advisor Jon Keating,
Valeriya Kovaleva, and Maksym Radziwiłł for useful comments.

##  2\. Preliminary tools and notation

We will start by using the following lemma, which is due to Soundararajan [24]
and Harper [12].

######  Lemma 2.1.

Assume RH, let $t\in[T,2T]$, and $|\alpha|\leq T/2$. Then for $2\leq X\leq
T^{2}$

$\displaystyle\log|\zeta(\tfrac{1}{2}$
$\displaystyle+i(t+\alpha))|\leq\textup{Re }\sum_{p\leq
X}\frac{1}{p^{1/2+1/\log X+i(t+\alpha)}}\frac{\log X/p}{\log X}$
$\displaystyle+$ $\displaystyle\sum_{p\leq\min(\sqrt{X},\log
T)}\frac{1}{2p^{1+2i(t+\alpha)}}+\frac{\log T}{\log X}+O(1).$

Throughout it will be useful to break the set of primes into certain
intervals. Set

$\beta_{\ast}:=\sum_{k\leq m}\max(1,\beta_{k}).$

We choose a sequence with $T_{0}=2$ and

$T_{j}=T^{e^{j-1}/(\log_{2}T)^{2}}$

for $j>0$ where we use $\log_{j}$ to denote the $j$-fold iterated logarithm.
We will choose $L$ to be the largest integer such that $T_{L}\leq
T^{1/200\beta_{\ast}^{2}}$. Given any $1\leq j\leq L$ define

$\mathcal{P}_{j,X}(s)=\sum_{p\in(T_{j-1},T_{j}]}\frac{1}{p^{s}}\frac{\log
X/p}{\log X}.$

More generally, given an interval $I_{j}\subseteq(T_{j-1},T_{j}]$ we will set

$\mathcal{P}_{I_{j},X}(s)=\sum_{p\in I_{j}}\frac{1}{p^{s}}\frac{\log X/p}{\log
X}.$

If $\mathcal{P}_{I_{j},X}(s)$ is not too large, then we will be able to
efficiently approximate $\exp(\beta\mathcal{P}_{I_{j},X}(s))$ with its Taylor
series. Indeed, if we choose cutoff parameters
$K_{j}=(\log_{2}T)^{3/2}e^{-j/2}$ for $j\geq 1$ and set

$\mathcal{N}_{I_{j},X}(s;\beta):=\sum_{n\leq
20\beta_{\ast}K_{j}}\frac{\beta^{n}\mathcal{P}_{I_{j},X}(s)^{n}}{n!}$

where $\beta\leq\beta_{\ast}$, then we have the following analog of lemma 1 of
[13]:

######  Lemma 2.2.

If $\beta\leq\beta_{\ast}$, $I_{j}\subset(T_{j-1},T_{j}]$, and
$|\mathcal{P}_{I_{j},X}(s)|\leq 2K_{j}$ for some $1\leq j\leq L$ then

$\exp(2\beta\textup{Re
}\mathcal{P}_{I_{j},X}(s))\leq(1+e^{-10K_{j}\beta_{\ast}})^{-1}|\mathcal{N}_{I_{j},X}(s;\beta)|^{2}.$

###### Proof.

Since $|\mathcal{P}_{I_{j},X}(s)|\leq 2K_{j}$, series expansion gives

$|\exp(\beta\mathcal{P}_{I_{j},X}(s))|-e^{-20\beta_{\ast}K_{j}}\leq|\mathcal{N}_{I_{j},X}(s;\beta)|.$

By assumption
$\exp(-2K_{j}\beta_{\ast})\leq|\exp(\beta\mathcal{P}_{I_{j},X}(s))|\leq\exp(2K_{j}\beta_{\ast})$,
so the claim readily follows. ∎

We will first bound the shifted moment of zeta when all of the shifts
$t+\alpha_{k}$ lie in the “good” set

$\mathcal{G}:=\left\\{t\in[T/2,5T/2]:|P_{j,T_{L}}(1+1/\log T_{L}+it)|\leq
K_{j}\text{ for all }1\leq j\leq L\right\\}.$ (2)

In this case we may use Lemma 2.1 with $X=T_{L}$ in tandem with Lemma 2.2 to
reduce the problem to computing the mean value of certain Dirichlet
polynomial. We will accomplish this with the following mean value theorem of
Montgomery and Vaughan (see for example theorem 9.1 of [10]).

######  Lemma 2.3.

Given any complex numbers $a_{n}$

$\int_{T}^{2T}\left|\sum_{n\leq
N}\frac{a_{n}}{n^{it}}\right|^{2}dt=(T+O(N))\sum_{n\leq N}|a_{n}|^{2}.$

We will also make use of the property that Dirichlet polynomials supported on
distinct sets of primes are approximately independent in the mean square
sense. The precise formulation we will use is the following splitting lemma
which appears in equation (16) of [14]

######  Lemma 2.4.

Suppose for $1\leq j\leq L$ we have $j$ disjoint intervals $I_{j}$ and
Dirichlet polynomials $A_{j}(s)=\sum_{n}a_{j}(n)n^{-s}$ such that $a_{j}(n)$
vanishes unless $n$ is composed of primes in $I_{j}$. Then if $\prod_{j\leq
L}A_{j}(s)$ is a Dirichlet polynomial of length $N$

$\displaystyle\int_{T}^{2T}\prod_{j\leq
L}|A_{j}(\tfrac{1}{2}+it)|^{2}dt=(T+O(N))\prod_{j\leq
L}\left(\frac{1}{T}\int_{T}^{2T}|A_{j}(\tfrac{1}{2}+it)|^{2}dt\right)$

The following variant due to Soundararajan [24, lemma 3] will also be useful
for handling moments of Dirichlet polynomials supported on primes.

######  Lemma 2.5.

Let $k$ be a natural number and suppose $N^{k}\leq T/\log T$. Then given any
complex numbers $a_{p}$

$\int_{T}^{2T}\left|\sum_{p\leq N}\frac{a_{p}}{p^{it}}\right|^{2k}dt\ll
Tk!\left(\sum_{p\leq N}|a_{p}|^{2}\right)^{k}.$

During the main mean value calculation, we will need to bound a certain
product over primes. This product will be controlled with the following lemma,
which is a special case of lemma 3.2 of [16]:

######  Lemma 2.6.

Given $\delta\in\mathbb{R}$ and $X\geq 2$

$\displaystyle\sum_{p\leq X}$ $\displaystyle\frac{\cos(\delta\log
p)}{p}=\log|\zeta(1+1/\log X+i\delta)|+O(1).$

To handle the shifted moment of zeta when some of the shifts $t+\alpha_{k}$
lie in the “bad” set $[T/2,5T/2]\setminus\mathcal{G}$, we take advantage of
the incremental structure present. For each $1\leq j\leq L$, define

$\displaystyle\mathcal{B}_{j}:=\\{t\in[T/2$
$\displaystyle,5T/2]:|\mathcal{P}_{r,T_{s}}(\tfrac{1}{2}+1/\log T_{s}+it)|\leq
K_{j}\text{ for all }1\leq r<j\text{ and }r\leq s\leq L$ $\displaystyle\text{
but }|\mathcal{P}_{j,T_{s}}(\tfrac{1}{2}+1/\log T_{s}+it)|>K_{j}\text{ for
some }j\leq s\leq L\\}.$

Notice that

$[T/2,5T/2]\setminus\mathcal{G}=\bigsqcup_{j\leq L}\mathcal{B}_{j}.$

On the bad sets $\mathcal{B}_{j}$ the series expansion $\mathcal{N}_{j,T_{s}}$
of $\exp(P_{j,T_{s}})$ is a poor approximation, so we are forced to estimate
$\log\zeta$ using only the primes up to $T_{j-1}$. While the resulting
Dirichlet polynomial is too short to obtain sharp bounds, we can hedge this
loss by multiplying by a suitably large even power of $|P_{j,T_{s}}|/K_{j}$,
which is larger than 1 on $\mathcal{B}_{j}$. If we then extend the range of
integration to all of $[T,2T]$, we can still win as the event
$|\mathcal{P}_{j,T_{s}}(1+1/\log T_{s}+it)|>K_{j}$ is quite rare. For example,
we will make use of the following bound.

######  Lemma 2.7.

If $\text{meas}(S)$ denotes the measure of a set $S\subset[T/2,5T/2],$ then

$\text{meas}(\mathcal{B}_{1})\ll Te^{-(\log_{2}T)^{2}/5}.$

###### Proof.

Taking the union bound over all $1\leq s\leq L$, we may bound the measure of
$\mathcal{B}_{1}$ by

$\sum_{s\leq
L}\frac{1}{K_{1}^{2k}}\int_{T/2}^{5T/2}|\mathcal{P}_{1,T_{s}}(\tfrac{1}{2}+1/\log
T_{s}+it)|^{2k}dt$

for some positive integer $k$. Trivially bounding $\log(T_{s}/p)/\log T_{s}$
by 1, Lemma 2.3 shows this quantity has order at most

$(T+T_{1}^{k})Lk!\left(\frac{e}{(\log_{2}T)^{2}}\right)^{k}.$

We may now conclude by taking $k=\lceil(\log_{2}T)^{2}/e\rceil$ and using
Stirling’s approximation, noting that $L\leq\log_{3}T$. ∎

Finally we will need to control the contribution of the prime squares. To this
end denote for each $1\leq l\leq\log_{2}T$ the Dirichlet polynomials

$Q_{l}(s):=\sum_{e^{l}<p\leq e^{l+1}}\frac{1}{2p^{2s}}$

and the sets

$\displaystyle\mathcal{C}_{l}=\\{t\in[T/2,5T/2]:|Q_{l}(\tfrac{1}{2}+it)|>J_{l}\text{
and }|Q_{r}(\tfrac{1}{2}+it)|\leq J_{r}\text{ for all }l<r\leq\log_{2}T\\},$

where we take $J_{l}:=e^{-l/10}$ and follow the convention that
$t\in\mathcal{C}_{0}$ if $t$ does not lie in $\mathcal{C}_{l}$ for any $1\leq
l\leq\log_{2}T$. If $t\in\mathcal{C}_{0}$ then the contribution of the prime
squares is negligible. On the other hand the integrals over the
$\mathcal{C}_{l}$ are handled in a similar fashion to the integrals over the
$\mathcal{B}_{j}$: first by multiplying by a large even power of
$|Q_{l}(\tfrac{1}{2}+it)|/J_{l}$ then by extending the range of integration
and using the mean value theorem for Dirichlet polynomials. As with the
$\mathcal{B}_{j}$, the events $\mathcal{C}_{l}$ are quite rare:

######  Lemma 2.8.

For $1\leq l\leq\log_{2}T$

$\text{meas}(\mathcal{C}_{l})\ll Te^{-le^{3l/4}}.$

###### Proof.

The same reasoning used in Lemma 2.7 shows that the measure of
$\mathcal{C}_{l}$ is bounded by

$\frac{1}{J_{l}^{2k}}\int_{T/2}^{5T/2}|Q_{l}(\tfrac{1}{2}+it)|^{2k}dt.$

Applying Lemma 2.5 with $k=\lceil le^{4l/5}\rceil$ to this integral readily
gives the claim. ∎

The proof of Theorem 1.1 is based on the following partition of $[T,2T]$:
Given a subset $A$ of $[m]:=\\{1,\ldots,m\\}$ define

$\mathcal{G}_{A}:=\left\\{t\in[T,2T]:t+\alpha_{k}\in\mathcal{G}\text{ if and
only if }k\in A\right\\}.$

Then we can decompose $[T,2T]$ into the disjoint union

$[T,2T]=\bigsqcup_{A\subseteq[m]}\mathcal{G}_{A}.$ (3)

In section 3, we will handle the integral over the set $\mathcal{G}_{[m]}$
where all of the shifts $t+\alpha_{k}$ are good. In section 4, we will handle
the cases where some of the shifts $t+\alpha_{k}$ are bad. We will have to
further partition the sets $\mathcal{G}_{A}$ with $A\subsetneq[m]$ according
to which of the sets $\mathcal{B}_{j}$ the bad shifts $t+\alpha_{k}$ lie in.
There are really no additional new ideas in this section, however, and the
difficulty here mostly comes from the need for rather messy notation.

##  3\. Moments over good shifts

We will further partition $\mathcal{G}_{[m]}$ using the subsets

$\mathcal{C}_{\ell}=\\{t\in[T,2T]:t+\alpha_{k}\in\mathcal{C}_{\ell(k)}\\}$

where $\ell:[m]\rightarrow\\{0,1,\ldots,\lfloor\log_{2}T\rfloor\\}$ is an
arbitrary function. First we will show that we may restrict our attention to
functions $\ell$ such that $|\ell|_{\infty}\leq 2\log_{3}T$, where
$|\ell|_{\infty}$ denotes the maximum of $\ell$.

######  Proposition 3.1.

For all $\ell$

$\int_{\mathcal{C}_{\ell}}\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt\ll
T\exp\left(-\frac{|\ell|_{\infty}}{2}\exp\left(3|\ell|_{\infty}/4\right)\right)(\log
T)^{2\beta_{\ast}^{2}}.$

Therefore

$\sum_{\begin{subarray}{c}\ell\\\
|\ell|_{\infty}>2\log_{3}T\end{subarray}}\int_{\mathcal{C}_{\ell}}\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt\ll
T\exp(-(\log_{2}T)^{3/2}(\log_{3}T)/10).$

######  Remark.

Since the shifts satisfy $|\alpha_{j}-\alpha_{k}|\leq T$, the error term in
this proposition (and in later calculations) is of smaller order then the
upper bound in Theorem 1.1 by the standard estimate $|\zeta(1+1/\log
T+it)|\gg\zeta(2+2/\log T)/\zeta(1+1/\log T)\gg 1/\log T$.

###### Proof.

First note that $\mathcal{C}_{\ell}$ is contained in a translate of
$\mathcal{C}_{|\ell|_{\infty}}$. Therefore by Lemma 2.8, Harper’s bound (say)
for the moments of zeta [12], and Hölder’s inequality we find that

$\displaystyle\int_{\mathcal{C}_{\ell}}\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt$
$\displaystyle\ll\left(\text{meas}(\mathcal{C}_{|\ell|_{\infty}})\right)^{1/2}\left(\int_{T}^{2T}\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{4\beta_{k}}dt\right)^{1/2}$
$\displaystyle\ll T\exp$
$\displaystyle\left(-\frac{|\ell|_{\infty}}{2}\exp\left(3|\ell|_{\infty}/4\right)\right)(\log
T)^{2\beta_{\ast}^{2}}.$

To bound the sum over $\ell$, first note that the number of $\ell$ with
$|\ell|_{\infty}=l$ at most $(l+1)^{m}$. Then the second claim follows from
the geometric series formula, say, after using the crude bound
$\exp(3|\ell|_{\infty}/4)\geq(\log_{2}T)^{3/2}$. ∎

We shall therefore restrict our attention to $\ell$ with
$|\ell|_{\infty}\leq\log_{3}T$ going forward. Now for each $\ell$ denote by
$I_{j,\ell}$ the interval $(T_{j-1},T_{j}]\cap(2^{|\ell|_{\infty}+1},\infty)$.
Our assumption on $\ell$ implies the interval $I_{j,\ell}$ only differs from
$(T_{j-1},T_{j}]$ when $j=1$; however, it will be notationally simpler to use
$I_{j,\ell}$ for all $j$ rather than to treat the first interval separately.
Notice also that for $t\in\mathcal{C}_{l}$

$\left|\textup{Re }\sum_{p\leq e^{l+1}}\frac{1}{p^{1/2+1/\log
T_{L}+it}}\frac{\log(X/p)}{\log X}+\textup{Re
}\sum_{p\leq\log_{2}T}\frac{1}{2p^{1+2it}}\right|=O(e^{l/2}).$ (4)

Therefore for fixed $\ell$ we may use Lemma 2.1 with $X=T_{L}$ to find

$\displaystyle\qquad\qquad\qquad\quad\int_{\mathcal{G}_{[m]}\cap\mathcal{C}_{\ell}}\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt$
$\displaystyle\ll\int_{\mathcal{G}_{[m]}\cap\mathcal{C}_{\ell}}\prod_{k=1}^{m}$
$\displaystyle\exp\left(2\beta_{k}\textup{Re
}\left(\sum_{j=1}^{L}\mathcal{P}_{j,T_{L}}(\tfrac{1}{2}+1/\log
T_{L}+i(t+\alpha_{k}))+\sum_{l\leq\log_{2}T}Q_{l}(\tfrac{1}{2}+i(t+\alpha_{k}))\right)\right)dt$
$\displaystyle\ll~{}$ $\displaystyle
e^{O(\beta_{\ast}e^{|\ell|_{\infty}/2})}\int_{\mathcal{G}_{[m]}\cap\mathcal{C}_{\ell}}\prod_{k=1}^{m}\prod_{j=1}^{L}\exp\left(2\beta_{k}\textup{Re
}\mathcal{P}_{I_{j,\ell},T_{L}}(\tfrac{1}{2}+1/\log
T_{L}+i(t+\alpha_{k}))\right)dt$

Next recall $K_{1}=(\log_{2}T)^{3/2}e^{-1/2}$, so by (4) and the assumption
that $|\ell|_{\infty}\leq 2\log_{3}T$ it follows that

$|\mathcal{P}_{I_{1,\ell},T_{L}}(\tfrac{1}{2}+1/\log
T_{L}+i(t+\alpha_{k}))|\leq 2K_{1}$

for all $k\leq m$ and $t\in\mathcal{G}_{[m]}\cap\mathcal{C}_{\ell}$.
Furthermore when $1<j\leq L$

$|\mathcal{P}_{I_{j,\ell},T_{L}}(\tfrac{1}{2}+1/\log
T_{L}+i(t+\alpha_{k}))|\leq K_{j}$

for all $k\leq m$ and $t\in\mathcal{G}_{[m]}\cap\mathcal{C}_{\ell}$ because
$I_{j,\ell}=(T_{j-1},T_{j}]$ for $j>1$. Therefore Lemma 2.2 allows us to bound
the integral over $\mathcal{G}_{[m]}\cap\mathcal{C}_{\ell}$ by

$\displaystyle\ll
e^{O(\beta_{\ast}e^{|\ell|_{\infty}/2})}\int_{\mathcal{G}_{[m]}\cap\mathcal{C}_{\ell}}\prod_{k=1}^{m}\prod_{j=1}^{L}$
$\displaystyle(1+e^{-10K_{j}\beta_{\ast}})^{-1}|\mathcal{N}_{I_{j,\ell},T_{L}}(\tfrac{1}{2}+1/\log
T_{L}+i(t+\alpha_{k});\beta_{k})|^{2}dt$ $\displaystyle\ll
e^{O(\beta_{\ast}e^{|\ell|_{\infty}/2})}\int_{T}^{2T}$
$\displaystyle\left|e^{|\ell|_{\infty}/10}Q_{|\ell|_{\infty}}(\tfrac{1}{2}+it+i\alpha_{k_{\ast}})\right|^{2\lceil|\ell|_{\infty}e^{4|\ell|_{\infty}/5}\rceil}$
$\displaystyle\times\prod_{j=1}^{L}\prod_{k=1}^{m}|\mathcal{N}_{I_{j,\ell},T_{L},\alpha_{k}}(1+1/\log
T_{L}+it;\beta_{k})|^{2}dt,$ (5)

where $k_{\ast}$ is arbitrarily chosen so that
$\ell(k_{\ast})=|\ell|_{\infty}$ and

$\mathcal{N}_{I_{j,\ell},X,\alpha}(s;\beta):=\mathcal{N}_{I_{j,\ell},X}(s+i\alpha;\beta).$

Here we have used that
$|e^{|\ell|_{\infty}/10}Q_{|\ell|_{\infty}}(\tfrac{1}{2}+it+i\alpha_{k_{\ast}})|\geq
1$ on $\mathcal{C}_{\ell}$, that
$\prod_{j=1}^{L}(1+e^{-10K_{j}\beta_{\ast}})^{-1}<\infty$, and nonnegativity
to extend the integral to the entire interval $[T,2T]$.

We are now in a setting where me may use the mean value theorem for Dirichlet
polynomials. First note that $Q_{l}(s)^{\lceil le^{4l/5}\rceil}$ has length at
most $\exp(5(\log_{3}T)^{2}(\log_{2}T)^{8/5})=T^{o(1)}$ for $l\leq 2\log_{3}T$
and that
$\prod_{k=1}^{m}\mathcal{N}_{I_{j,\ell},T_{L},\alpha_{k}}(s;\beta_{k})$ has
length at most $T_{j}^{20\beta_{\ast}^{2}K_{j}}$. Therefore $\prod_{j\leq
L}\prod_{k=1}^{m}\mathcal{N}_{I_{j,\ell},T_{L},\alpha_{k}}(s;\beta_{k})$ has
length at most $T^{1/2}$ as $T_{L}\leq T^{1/200\beta_{\ast}^{2}}$, so the
integrand in (3) is a short Dirichlet polynomial. Thus by Lemma 2.4 it
suffices to compute the mean values

$\int_{T}^{2T}\left|e^{|\ell|_{\infty}/10}Q_{|\ell|_{\infty}}(\tfrac{1}{2}+it+i\alpha_{k_{\ast}})\right|^{2\lceil|\ell|_{\infty}e^{4|\ell|_{\infty}/5}\rceil}dt$

and

$\int_{T}^{2T}\prod_{k=1}^{m}|\mathcal{N}_{I_{j,\ell},T_{L},\alpha_{k}}(1+1/\log
T_{L}+it;\beta_{k})|^{2}dt$

for each $1\leq j\leq L$. The estimate for the moment of $Q_{|\ell|_{\infty}}$
is straightforward.

######  Proposition 3.2.

For fixed $\ell$

$\int_{T}^{2T}\left|e^{|\ell|_{\infty}/10}Q_{|\ell|_{\infty}}(\tfrac{1}{2}+it+i\alpha_{k_{\ast}})\right|^{2\lceil|\ell|_{\infty}e^{4|\ell|_{\infty}/5}\rceil}dt\ll
Te^{-|\ell|_{\infty}e^{|\ell|_{\infty}/2}}.$

###### Proof.

By Lemma 2.5

$\displaystyle\int_{T}^{2T}\big{|}Q_{|\ell|_{\infty}}$
$\displaystyle(\tfrac{1}{2}+it+i\alpha_{k_{\ast}})\big{|}^{2\lceil|\ell|_{\infty}e^{4|\ell|_{\infty}/5}\rceil}dt$
$\displaystyle\ll
T\lceil|\ell|_{\infty}e^{4|\ell|_{\infty}/5}\rceil!\left(\sum_{e^{|\ell|_{\infty}}<p\leq
e^{|\ell|_{\infty}+1}}\frac{1}{2p^{2}}\right)^{\lceil|\ell|_{\infty}e^{4|\ell|_{\infty}/5}\rceil}.$

Because

$\sum_{e^{|\ell|_{\infty}}<p\leq
e^{|\ell|_{\infty}+1}}\frac{1}{2p^{2}}\ll\frac{1}{|\ell|_{\infty}e^{|\ell|_{\infty}}}$

the claim follows by a routine application of Stirling’s approximation. ∎

To estimate the mean values of the
$\mathcal{N}_{I_{j,\ell},X,\alpha}(s;\beta)$ we must analyze their
coefficients. Denote $a_{X}(p):=\log(X/p)/\log X$. We may then write

$\mathcal{N}_{I_{j,\ell},X,\alpha}(s;\beta)=\sum_{\begin{subarray}{c}p\mid
n\Rightarrow p\in I_{j,\ell}\\\ \Omega(n)\leq
20\beta_{\ast}K_{j}\end{subarray}}\frac{\beta^{\Omega(n)}g_{X}(n)n^{-i\alpha}}{n^{s}}$

where $\Omega(n)$ is the number of prime factors of $n$ counting multiplicity
and

$g_{X}(n):=\prod_{p^{r}\|n}\frac{a_{X}(p)^{r}}{r!}.$

Note that $g_{X}(n)$ is a multiplicative function and $a_{X}(p)\in[0,1]$. We
do not need to completely understand all of the coefficients of the
$\mathcal{N}_{I_{j,\ell},X,\alpha}(s;\beta)$; the following lemma will
suffice.

######  Lemma 3.3.

Write

$\prod_{k=1}^{m}\mathcal{N}_{I_{j,\ell},X,\alpha_{k}}(s;\beta_{k})=\sum_{n\geq
1}\frac{b_{I_{j,\ell},X,\bm{\alpha},\bm{\beta}}(n)}{n^{s}}.$

Then $b_{I_{j,\ell},X,\bm{\alpha},\bm{\beta}}$ is supported on integers
composed of primes in $I_{j,\ell}$, and for $p\in I_{j,\ell}$

$b_{I_{j,\ell},X,\bm{\alpha},\bm{\beta}}(p)=a_{X}(p)\sum_{k=1}^{m}\beta_{k}p^{-i\alpha_{k}}.$

In general we have the bound

$|b_{I_{j,\ell},X,\bm{\alpha},\bm{\beta}}(p^{r})|\leq\frac{\beta_{\ast}^{r}k^{r}}{r!}.$

###### Proof.

The first two properties are immediate by expanding the coefficients
$b_{I_{j,\ell},X,\bm{\alpha},\bm{\beta}}(n)$ using the Dirichlet convolution.
To deduce the upper bound, first note that the $p^{r}$ coefficient of each
$\mathcal{N}_{I_{j,\ell},X,\alpha}(s;\beta)$ is bounded by $\beta^{r}/r!$.
Whence

$\displaystyle|b_{I_{j,\ell},X,\bm{\alpha},\bm{\beta}}(p^{r})|\leq\sum_{r_{1}+\cdots+r_{k}=r}\prod_{l=1}^{k}\frac{\beta_{l}^{r_{l}}}{r_{l}!}\leq\frac{\beta_{\ast}^{r}}{r!}\sum_{r_{1}+\cdots+r_{k}=r}\binom{r}{r_{1},\ldots,r_{k}}=\frac{\beta_{\ast}^{r}k^{r}}{r!}.$

∎

We can now compute

######  Proposition 3.4.

For $1\leq j\leq s\leq L$ and

$\int_{T}^{2T}\prod_{k=1}^{m}|\mathcal{N}_{I_{j,\ell},T_{s},\alpha_{k}}(1+1/\log
T_{s}+it;\beta_{k})|^{2}dt\leq(T+O(T^{1/2}))\prod_{p\in
I_{j,\ell}}\left(1+\frac{|b_{I_{j,\ell},X,\bm{\alpha},\bm{\beta}}(p)|^{2}}{p}+O_{\bm{\beta}}\left(\frac{1}{p^{2}}\right)\right).$

###### Proof.

By Lemma 2.3 and multiplicativity the mean value of interest equals

$(T+O(T^{1/2}))\sum_{\begin{subarray}{c}p\mid n\Rightarrow p\in I_{j,\ell}\\\
\Omega(n)\leq
20\beta_{\ast}K_{j}\end{subarray}}\frac{|b_{I_{j,\ell},X,\bm{\alpha},\bm{\beta}}(n)|^{2}}{n^{1+1/\log
T_{s}}}\leq(T+O(T^{1/2}))\prod_{p\in I_{j,\ell}}\left(1+\sum_{r\geq
1}\frac{|b_{I_{j,\ell},X,\bm{\alpha},\bm{\beta}}(p^{r})|^{2}}{p^{r}}\right),$

where in the final step we disposed of the condition on $\Omega(n)$ using
nonnegativity. Now just notice the contribution of terms with $r\geq 2$ is
$\ll 1/p^{2}$ by the upper bound in Lemma 3.3. ∎

Now we can use Lemma 2.4 to estimate (3). If we combine Propositions 3.1, 3.2,
and 3.4 and sum over $\ell$ with $|\ell|_{\infty}\leq 2\log_{3}T$ we conclude

######  Proposition 3.5.

On the good set we have the bound

$\displaystyle\int_{\mathcal{G}_{[m]}}\prod_{k\leq
m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt\ll T(\log
T)^{\beta_{1}^{2}+\cdots+\beta_{m}^{2}}\prod_{1\leq j<k\leq
m}|\zeta(1+i(\alpha_{j}-\alpha_{k})+1/\log T)|^{2\beta_{j}\beta_{k}}.$

###### Proof.

Recalling that at most $(l+1)^{k}$ of the functions $\ell$ have
$|\ell|_{\infty}=l$, we see the integral of interest is of order at most

$\displaystyle T\sum_{0\leq l\leq
2\log_{3}T}(l+1)^{k}e^{-le^{l/2}}\prod_{2^{l+1}<p\leq T_{L}}$
$\displaystyle\left(1+\frac{1}{p}\sum_{j,k\leq
m}\frac{\beta_{j}\beta_{k}}{p^{i(\alpha_{j}-\alpha_{k})}}+O_{\bm{\beta}}\left(\frac{1}{p^{2}}\right)\right)$
$\displaystyle\ll T\sum_{0\leq l\leq
2\log_{3}T}(l+1)^{k}e^{-le^{l/2}}\times(l+1)^{\beta_{\ast}^{2}}$
$\displaystyle\prod_{p\leq T_{L}}\left(1+\frac{1}{p}\sum_{j,k\leq
m}\frac{\beta_{j}\beta_{k}}{p^{i(\alpha_{j}-\alpha_{k})}}+O_{\bm{\beta}}\left(\frac{1}{p^{2}}\right)\right).$

This is

$\ll T\prod_{p\leq
T_{L}}\left(1+\frac{\beta_{1}^{2}+\cdots+\beta_{m}^{2}}{p}\right)\prod_{1\leq
j<k\leq L}\prod_{p\leq
T_{L}}\left(1+\frac{2\beta_{j}\beta_{k}\cos((\alpha_{j}-\alpha_{k})\log
p)}{p}\right).$

The first product is visibly $\ll(\log
T_{L})^{\beta_{1}^{2}+\cdots+\beta_{m}^{2}}$, and we can bound the remaining
products with Lemma 2.6. The claim now follows since $\log T_{L}\asymp\log T$.
∎

##  4\. Moments over bad shifts

We now consider the integral over $\mathcal{G}_{A}$ where $A$ is a proper
subset of $[m]$. We will partition $\mathcal{G}_{A}$ using maps
$f:[m]\setminus A\rightarrow[L]$ such that
$f(\alpha_{k})\in\mathcal{B}_{f(k)}$ for $k\not\in A$. If we write

$\mathcal{G}_{A,f}=\\{t\in\mathcal{G}_{A}:f(\alpha_{k})\in\mathcal{B}_{f(k)}\text{
for }k\not\in A\\},$

then $\mathcal{G}_{A}$ is the disjoint union of the $\mathcal{G}_{A,f}$ over
all $f:[m]\setminus A\rightarrow[L]$. Therefore the task at hand is bounding
shifted moments over the $\mathcal{G}_{A,f}$. Without loss of generality we
will write $A=[m]\setminus[a]$. That is we assume that the first $a$ shifts
are bad for some $a\in[m]$, so $f$ is a map from $[a]$ to $[L]$. For such a
set $A$ and a map $f$, we will write $\mathcal{G}_{a,f}$ for
$\mathcal{G}_{A,f}$. We may also assume that $f$ is an increasing function,
and by Proposition 3.1 we restrict the range of integration to
$\mathcal{G}[A]\cap\mathcal{C}_{\ell}$ for some
$\ell:[m]\rightarrow\\{0,\ldots,\lfloor 2\log_{3}T\rfloor\\}$. Since $f$ is
increasing, we can only work with the primes up to $T_{f(1)-1}$. We first
handle the case with $f(1)=1$.

######  Proposition 4.1.

If $f(1)=1$, then

$\int_{\mathcal{G}_{a,f}}\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt\ll
Te^{-(\log_{2}T)^{2}/20}.$

Therefore

$\sum_{\begin{subarray}{c}f:[a]\rightarrow[L]\\\
f(1)=1\end{subarray}}\int_{\mathcal{G}_{a,f}}\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt\ll
Te^{-(\log_{2}T)^{2}/30}.$

###### Proof.

Because $\mathcal{G}_{a,f}$ is contained in a translate of $\mathcal{B}_{1}$
the first bound is a consequence of Lemma 2.7 and the Cauchy-Schwarz
inequality. To bound the sum over $f$, just notice that there are at most
$L^{a}$ such maps, and $L\leq\log_{3}T$. ∎

Now for fixed $a,f,$ and $\ell$ with $f(1)>1$ we may use Lemma 2.1 with
$X=T_{f(1)-1}$ in the same manner as the previous section to find

$\displaystyle\int_{\mathcal{G}_{a,f}\cap\mathcal{C}_{\ell}}$
$\displaystyle\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt$
$\displaystyle\ll$
$\displaystyle\int_{\mathcal{G}_{[m]}\cap\mathcal{C}_{\ell}}\prod_{k=1}^{m}\exp\Bigg{(}2\beta_{k}\textup{Re
}\Bigg{(}\sum_{j<f(1)}\mathcal{P}_{j,T_{f(1)-1}}(\tfrac{1}{2}+1/\log
T_{f(1)-1}+i(t+\alpha_{k}))$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad+\sum_{l\leq\log_{2}T}Q_{l}(\tfrac{1}{2}+i(t+\alpha_{k}))+(\log_{2}T)^{2}e^{2-f(1)}\Bigg{)}\Bigg{)}dt$
$\displaystyle\ll$
$\displaystyle~{}e^{O(\beta_{\ast}e^{|\ell|_{\infty}/2})+e^{2-f(1)}\beta_{\ast}(\log_{2}T)^{2}}$
$\displaystyle\qquad\qquad\times\int_{\mathcal{G}_{[m]}\cap\mathcal{C}_{\ell}}\prod_{k=1}^{m}\prod_{j<f(1)}\exp\left(2\beta_{k}\textup{Re
}\mathcal{P}_{I_{j,\ell},T_{f(1)-1}}(\tfrac{1}{2}+1/\log
T_{f(1)-1}+i(t+\alpha_{k}))\right)dt$ $\displaystyle\ll$
$\displaystyle~{}e^{O(\beta_{\ast}e^{|\ell|_{\infty}/2})+e^{2-f(1)}\beta_{\ast}(\log_{2}T)^{2}}$
$\displaystyle\qquad\qquad\times\sum_{s\leq
L}\int_{T}^{2T}\prod_{k=1}^{m}\prod_{j<f(1)}\exp\left(2\beta_{k}\textup{Re
}\mathcal{P}_{I_{j,\ell},T_{f(1)-1}}(\tfrac{1}{2}+1/\log
T_{f(1)-1}+i(t+\alpha_{k}))\right)$
$\displaystyle\qquad\qquad\qquad\qquad\times|\mathcal{P}_{f(1)}(\tfrac{1}{2}+1/\log
T_{s}+i(t+\alpha_{1}))/K_{f(1)}|^{2\lceil K_{f(1)}^{2}/2\rceil}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\times\left|e^{|\ell|_{\infty}/10}Q_{|\ell|_{\infty}}(\tfrac{1}{2}+it+i\alpha_{k_{\ast}})\right|^{2\lceil|\ell|_{\infty}e^{4|\ell|_{\infty}/5}\rceil}dt$

where $\ell(k_{\ast})=|\ell|_{\infty}$ as in (3), only now we have also used
the definition of the bad set $\mathcal{B}_{f(1)}$ and used a union bound over
all $s\leq L$. By Lemma 2.4, all that remains is to control the moments of
$\mathcal{P}_{f(1)}$ slightly off of the half line.

######  Proposition 4.2.

For each $s\leq L$

$\int_{T}^{2T}|\mathcal{P}_{f(1)}(\tfrac{1}{2}+1/\log
T_{s}+i(t+\alpha_{1}))/K_{f(1)}|^{2\lceil K_{f(1)}^{2}/2\rceil}dt\ll
Te^{-(\log_{2}T)^{3}e^{-f(1)}/3}.$

###### Proof.

Trivially bounding $p^{-i\alpha_{1}}$ and $\log(X_{s}/p)/\log(X_{s})$ by 1,
Lemma 2.5 gives a bound of

$TK_{f(1)}^{-2k}k!\left(\sum_{p\in(T_{j-1},T_{j}]}\frac{1}{p}\right)^{k}$

where $k=\lceil K_{f(1)}^{2}/2\rceil$. The quantity in parentheses is
asymptotic to 1, so is at most $2$ for large $T$, say. The conclusion is now
routine. ∎

We now have all the necessary tools to bound the shifted moment (1) over bad
sets.

######  Proposition 4.3.

On the bad sets

$\sum_{a,\ell,f}\int_{\mathcal{G}_{a,f}\cap\mathcal{C}_{\ell}}\prod_{k=1}^{m}|\zeta(\tfrac{1}{2}+i(t+\alpha_{k}))|^{2\beta_{k}}dt\ll
T(\log T)^{\beta_{1}^{2}+\cdots+\beta_{m}^{2}}\prod_{1\leq j<k\leq
m}|\zeta(1+i(\alpha_{j}-\alpha_{k})+1/\log T)|^{2\beta_{j}\beta_{k}},$

where the summation is taken over $1\leq a\leq m$, functions
$\ell:[m]\rightarrow\\{0,1,\ldots,\lfloor 2\log_{3}T\rfloor\\}$ and increasing
functions $f:[a]\rightarrow[L]$.

###### Proof.

First recall for each $l\leq 2\log_{3}T$ there are at most $(l+1)^{m}$
functions $\ell$ with $|\ell|_{\infty}=l$, and given $b\in[L]$, there are at
most $L^{a}$ functions $f$ with $f(1)=b$. By Proposition 4.1 we may also
assume $b>1$. Using Lemma 2.4 along with Propositions 3.2, 3.4, and 4.2, we
may bound the sum with $b\geq 2$ by

$\displaystyle T\sum_{\begin{subarray}{c}a\leq m\\\ 2\leq b\leq L\\\ 0\leq
l\leq 2\log_{3}T\end{subarray}}L^{a}$
$\displaystyle(l+1)^{m}\exp\left(O\left(\beta_{\ast}e^{l/2}\right)+e^{2-b}\beta_{\ast}(\log_{2}T)^{2}-le^{l/2}-e^{-b}(\log_{2}T)^{3}/3\right)$
$\displaystyle\qquad\times\prod_{2^{l+1}<p\leq T_{b-1}}\left(1+\sum_{1\leq
j,k\leq
m}\frac{\beta_{j}\beta_{k}}{p^{i(\alpha_{j}-\alpha_{k})}}+O_{\bm{\beta}}\left(\frac{1}{p^{2}}\right)\right)$
$\displaystyle\ll T\sum_{b\leq L}L^{m}\exp$
$\displaystyle\left(e^{2-b}\beta_{\ast}(\log_{2}T)^{2}-e^{-b}(\log_{2}T)^{3}/3\right)\prod_{p\leq
T_{b-1}}\left(1+\sum_{1\leq j,k\leq
m}\frac{\beta_{j}\beta_{k}}{p^{i(\alpha_{j}-\alpha_{k})}}\right).$

Next notice that

$\prod_{p\in(T_{b-1},T_{L}]}\left(1+\sum_{1\leq j,k\leq
m}\frac{\beta_{j}\beta_{k}}{p^{i(\alpha_{j}-\alpha_{k})}}\right)\ll(\log
T)^{\beta_{\ast}^{2}}.$

Therefore the desired sum is

$\ll T(\log T)^{\beta_{\ast}^{2}}\sum_{b\leq
L}L^{m}\exp\left(-c_{\bm{\beta}}e^{-b}(\log_{2}T)^{3}\right)\prod_{p\leq
T_{L}}\left(1+\sum_{1\leq j,k\leq
m}\frac{\beta_{j}\beta_{k}}{p^{i(\alpha_{j}-\alpha_{k})}}\right).$

for some $c_{\bm{\beta}}>0$. Since $L\leq\log_{3}T$, this quantity is

$\ll T(\log
T)^{\beta_{\ast}^{2}}L^{m+1}\exp\left(-c_{\bm{\beta}}(\log_{2}T)^{2}\right)\prod_{p\leq
T_{L}}\left(1+\sum_{1\leq j,k\leq
m}\frac{\beta_{j}\beta_{k}}{p^{i(\alpha_{j}-\alpha_{k})}}\right).$

This is admissible as $L^{m+1}(\log
T)^{\beta_{\ast}^{2}}e^{-c_{\bm{\beta}}(\log_{2}T)^{2}}$ is $\ll_{A}(\log
T)^{-A}$ for all $A>0$. ∎

In view of (3), this completes the proof of Theorem 1.1.

## References

  * [1] F. V. Atkinson. The mean value of the Riemann zeta-function, Acta Math. 81, 353–376 (1949).
  * [2] S. Bettin, The second moment of the Riemann zeta function with unbounded shifts. Int. J. Number Theory 6, no. 8, 1933–1944 (2010).
  * [3] P. Bourgade, Mesoscopic fluctuations of the zeta zeros, Probab. Theory Related Fields 148, no. 3-4, 479–500 (2010).
  * [4] V. Chandee. On the correlation of shifted values of the Riemann zeta function. Q. J. Math. 62, no. 3, 545-572 (2011).
  * [5] J. B. Conrey, D. W. Farmer, J. P. Keating, M. O. Rubinstein, and N. C. Snaith, Autocorrelation of random matrix polynomials, Commun. Math. Phys 237 3, 365-395 (2003).
  * [6] J. B. Conrey, D. W. Farmer, J. P. Keating, M. O. Rubinstein, and N. C. Snaith, Integral moments of $L$-functions, Proc. London Math. Soc. 91 no. 3, 33–104 (2005).
  * [7] M. J. Curran, Freezing transition and moments of moments of the Riemann zeta function, arXiv preprint arXiv:2301.10634 (2023).
  * [8] A. Granville and K. Soundararajan, Multiplicative number theory: The pretentious approach. To appear.
  * [9] A. E. Ingham, Mean-value theorems in the theory of the Riemann zeta function, Proc. London Math. Soc. 27, 273–300 (1926).
  * [10] H. Iwaniec and E. Kowalski. Analytic number theory, American Mathematical Society, vol. 53, Providence, RI (2004).
  * [11] G. H. Hardy and J. E. Littlewood, Contributions to the theory of the Riemann zeta-function and the theory of the distribution of primes. Acta Mathematica 41, 119 - 196 (1918).
  * [12] A. J. Harper, Sharp conditional bounds for moments of the Riemann zeta function, arXiv.1305.4618 (2013).
  * [13] W. Heap., M. Radziwiłł, K. Soundararajan, Sharp upper bounds for fractional moments of the Riemann zeta function, Q. J. Math. 70, no. 4, 1387–1396 (2019).
  * [14] W. Heap., K. Soundararajan, Lower bounds for moments of zeta and L-functions revisited, Mathematika 1, no. 68, 1-14 (2022).
  * [15] J. P. Keating, N. C. Snaith, Random matrix theory and $\zeta(\tfrac{1}{2}+it)$, Comm. Math. Phys. 214, 57–89 (2000).
  * [16] D. Koukoulopoulos,  Pretentious multiplicative functions and the prime number theorem for arithmetic progressions, Compos. Math. 149, no. 7, 1129–1149 (2013).
  * [17] V. Kovaleva. Topics in probabalistic number theory, PhD thesis, University of Oxford (2022).
  * [18] H. Kösters. On the occurrence of the sine kernel in connection with the shifted moments of the Riemann zeta function, J. Number Theory 130, 2596-2609 (2010).
  * [19] Y. Motohashi. An explicit formula for the fourth power mean of the Riemann zeta-function, Acta Math. 170, 181–220 (1993).
  * [20] Y. Motohashi. Spectral theory of the Riemann zeta-function, Cambridge Tracts in Math. 127, Cambridge Univ. Press, Cambridge (1997).
  * [21] N. Ng, Q. Shen, P.-J. Wong. Shifted moments of the Riemann zeta function. arXiv preprint arXiv:2206.03350 (2022).
  * [22] M. Radziwiłł, Large deviations in Selberg’s central limit theorem, arXiv.1108.5092 (2011).
  * [23] M. Radziwiłł, K. Soundararajan, Moments and distribution of central $L$-values of quadratic twists of elliptic curves, Invent. Math. 202 no. 3, 1029–1068 (2015).
  * [24] K. Soundararajan, Moments of the Riemann zeta function, Annals of Math. 170 no. 2, 981–993 (2009).
  * [25] E. C. Titchmarsh, The Theory of the Riemann Zeta-Function, 2nd ed., Oxford University Press, New York, (1986).


# Direct evidence of terahertz emission arising from anomalous Hall effect

V. Mottamchetty  Department of Materials Science and Engineering, Uppsala
University, Box 35, SE-751 03 Uppsala, Sweden  P. Rani  Department of Physics
and Astronomy, Uppsala University, Box 516, SE-75120 Uppsala, Sweden  R.
Brucas  Department of Materials Science and Engineering, Uppsala University,
Box 35, SE-751 03 Uppsala, Sweden  A. Rydberg  Department of Materials Science
and Engineering, Uppsala University, Box 35, SE-751 03 Uppsala, Sweden
Department of Physics and Astronomy, Uppsala University, Box 516, SE-75120
Uppsala, Sweden  P. Svedlindh<EMAIL_ADDRESS>Department of
Materials Science and Engineering, Uppsala University, Box 35, SE-751 03
Uppsala, Sweden  R. Gupta<EMAIL_ADDRESS>Department of Materials
Science and Engineering, Uppsala University, Box 35, SE-751 03 Uppsala, Sweden
Department of Physics and Astronomy, Uppsala University, Box 516, SE-75120
Uppsala, Sweden

###### Abstract

A detailed understanding of the different mechanisms being responsible for
terahertz (THz) emission in ferromagnetic (FM) materials will aid in designing
efficient THz emitters. In this report, we present direct evidence of THz
emission from single layer Co0.4Fe0.4B0.2 (CoFeB) FM thin films. The dominant
mechanism being responsible for the THz emission is the anomalous Hall effect
(AHE), which is an effect of a net backflow current in the FM layer created by
the spin polarized current reflected at the interfaces of the FM layer. The
THz emission from the AHE-based CoFeB emitter is optimized by varying its
thickness, orientation, and pump fluence of the laser beam. Results from
electrical transport measurements show that skew scattering of charge carriers
is responsible for the THz emission in the CoFeB AHE-based THz emitter.

††preprint: APS/123-QED

##  I Introduction

The region of the electromagnetic spectrum that lies between near microwave
and far-infrared radiation is the so-called Terahertz (THz) radiation or THz
gap, i.e. typically frequencies between 100 GHz and 30 THz. Terahertz
radiation finds applications in various fields, such as medicine, security,
etc. Valušis  _et al._ (2021); Federici and Moeller (2010). Photoconductive
switching, optical rectification, transient photo-current in air plasma, and
difference frequency generation constitute techniques that are employed for
the generation of THz radiation Smith  _et al._ (1988); Venkatesh  _et al._
(2014); Lepeshov  _et al._ (2017); Jazbinsek  _et al._ (2019); Vediyappan  _et
al._ (2019); Yang  _et al._ (2007); Kim  _et al._ (2007); Yan  _et al._
(2017); Lu  _et al._ (2017); Bagley  _et al._ (2018); Fülöp  _et al._ (2020).
Moreover, THz emission from magnetic materials, utilizing the spin degree of
freedom, has recently gained popularity as a promising framework for the
generation of broadband radiation without any phonon absorption gaps and with
intensity comparable to the standard zinc telluride THz source Seifert  _et
al._ (2016); Gupta  _et al._ (2021a).

There are several possible mechanisms that can explain THz generation in spin-
based systems. Beaurepaire et al. Beaurepaire  _et al._ (1996) discovered
ultrafast demagnetization (UDM) in 1996, showing that a ferromagnetic (FM) Ni
film when demagnetized on a subpicosecond time scale by a femtosecond (fs)
laser pulse excitation generates THz radiation Zhang  _et al._ (2020). The THz
radiation is in this case proportional to the second time-derivative of the
magnetization Beaurepaire  _et al._ (2004) and shows a linear dependence on
the thickness of the FM layer. Recently, Kampfrath et al. Kampfrath  _et al._
(2013); Seifert  _et al._ (2016) discovered an alternative mechanism for THz
generation, which utilizes the inverse spin Hall effect (iSHE) or inverse
Rasbha Edelstein effect (iREE). Here, the generation mechanism requires a
magnetic heterostructure consisting of an FM layer and a non-magnet (NM) layer
that possesses a high spin-to-charge (S2C) conversion efficiency. In this
mechanism, the amplitude of the THz emission is highly dependent on the S2C
conversion efficiency. Recently, it has been shown that THz emitters can be
designed using a single FM layer, which utilizes the anomalous Hall effect
(AHE) phenomenon Yang  _et al._ (2018); Zhang  _et al._ (2019); Liu  _et al._
(2021); Su  _et al._ (2014). On the one hand, the UDM mechanism relies on bulk
properties of a single FM layer, while on the other hand the AHE mechanism
corresponds to a combined effect of interface and bulk properties, which will
be further discussed below.

A fs laser pulse when incident on a FM layer will excite hot electrons in the
FM layer. The system achieves equilibrium through electron-electron, electron-
phonon and electron-magnon interactions. Before attaining the equilibrium with
respect to electron-electron interactions, the hot electrons acquire a
velocity of order $10^{6}$ m/s and move within the FM layer in a super-
diffusive manner Battiato  _et al._ (2010, 2012). When reaching the
FM/dielectric interfaces as indicated in Fig. 1, electrons reflect back from
the interfaces to form a net backflow current ($j_{bf}$) along the film
thickness direction Battiato  _et al._ (2012). In the presence of the AHE,
$j_{bf}$ is converted to a transient current ($j_{t}$) defined as
$j_{t}=\theta_{AHE}(m\times j_{bf}$), where $\theta_{AHE}$ and $m$ are the
anomalous Hall angle and the magnetization direction, respectively. The net
backflow current depends on the dielectric properties of the interfaces, their
roughness and properties of the FM layer, such as $\theta_{AHE}$ and $m$.

Figure 1: Schematic of THz emission from the AHE-based CoFeB emitter. (a)
pumping from the substrate side and (b) pumping from capping side.

Recently, Zhang et al. Zhang  _et al._ (2019) reported the THz emission in FM
layers with different $\theta_{AHE}$, where (Fe0.8Mn0.2)0.67Pt0.33 (FeMnPt)
with $\theta_{AHE}$=0.0269 was found to exhibit the largest THz emission.
However, in the same study, the THz amplitude was found to be much smaller for
Co0.2Fe0.6B0.2 even though the anomalous Hall angle for Co0.4Fe0.4B0.2 is of
the same order of magnitude ($\theta_{AHE}^{CoFeB}\sim$ -0.023) as that of
FeMnPt Su  _et al._ (2014). Moreover, NiFe was found to exhibit larger THz
emission than Co0.2Fe0.6B0.2, which is in contradiction with results reported
by Seifert et al. Seifert  _et al._ (2016). Note that the latter study
utilized the iSHE mechanism for THz generation. Liu et al. Liu  _et al._
(2021) managed to distinguish and separate the different contributions to the
THz radiation, where they demonstrated that the UDM mechanism (magnetic
dipole) is linearly dependent on the FM layer thickness while the AHE
mechanism is not. In addition, the composition of the FM layer defines the
magnitude of $\theta_{AHE}$, which is a major factor in AHE-based THZ emitters
Yang  _et al._ (2018). However, a systematic study providing direct evidence
of the relationship between the anomalous Hall angle and the terahertz
emission in single FM layers is still missing.

In this study, we used Co0.4Fe0.4B0.2 (CoFeB) as the AHE layer to reveal the
impact of the AHE on the THz emission. We provide direct evidence for the
relationship between the AHE and the THz emission in CoFeB, by presenting
results from systematic studies of the effect of the CoFeB thickness on the
anomalous Hall resistivity, anomalous Hall angle and the anomalous Hall
coefficient. In addition, we also reveal the mechanism behind the AHE in
CoFeB.

##  II Experimental details

Thin films of CoFeB with different thicknesses ($t_{CoFeB}$) were deposited at
room temperature using a DC magnetron sputtering system equipped with a turbo
pump achieving a base pressure of 1$\times$10-10 Torr. The base pressure and
working pressure of the chamber during deposition were $5\times 10^{-10}$ and
$2\times 10^{-3}$ Torr, respectively where 99.999% pure Ar gas was used as a
sputtering gas. Prior to the deposition, the single-side polished Al2O3
substrates (thickness $\sim$ 0.5 mm) were preheated at base pressure to 600∘C
for one hour and then cooled down to room temperature before deposition.
Substrates were rotated at 6 rpm speed for uniform growth. The deposition rate
of CoFeB was calibrated using X-ray reflectivity (XRR) measurements. To
protect the CoFeB layer from oxidation and provide an additional interface for
the backflow current, a 3 nm thick Al cap layer was deposited.

The deposited CoFeB films were investigated by X-ray diffraction as well as
XRR with Cu Kα radiation. It was found that all CoFeB films are amorphous in
nature and that the interface roughness is larger for the cap/CoFeB interface
than for the CoFeB/substrate interface, which provides a net backflow current.
Details are described in section 1 of supplementary materials (SM) SM .

A THz time-domain spectrometer was used to measure the THz emission from the
CoFeB films Gupta  _et al._ (2021a, b); Gupta (2021). A Spectra-Physics
Tsunami (Ti: sapphire) was used as a laser source, which delivers pulses of
$\sim 55$ fs duration (bandwidth $\sim 12$ nm, central wavelength $\sim 800$
nm, and maximum output energy $\sim 10$ nJ) at a repetition rate of 80 MHz. A
low-temperature gallium arsenide photoconductive dipole antenna with $\sim
4\mu$m gap was used as a detector for the THz pulses. A probe beam with
average laser power of 10 mW was used for detection and a static in-plane
magnetic field of $\sim 85$ mT was used to saturate the magnetization of the
CoFeB films. The displayed THz signals in the figures correspond to averages
of 500 detected THz spectra obtained within one minute of measurement time
(cf. Fig. S2 in SM SM ).

Figure 2: (a) Left: THz waveform in time-domain ($t$) for substrate/CoFeB(5
nm)/cap emitter with different magnetization directions ($\pm M$) when pumping
from substrate side. Right: THz waveform pumping from the capping layer and
substrate sides while keeping the same magnetization direction. (b) Fast
Fourier transform of THz electric field and the inset shows THz peak-to-peak
amplitude for substrate/CoFeB($t_{CoFeB}$ nm)/cap emitters as a function of
pump fluence.

The Quantum Design physical property measurement and magnetic properties
measurement systems were employed to measure the AHE and the saturation
magnetization of the CoFeB-based emitters, respectively. For AHE measurements,
Hall bar devices (20$\times$100 $\mu$m2) were patterned using optical
lithography.

##  III Results and discussion

Figure 1 illustrates a schematic of THz emission from AHE-based CoFeB emitters
when the laser pulse is irradiated from the substrate and capping sides. Due
to different interface roughness (cf. section 1, SM SM ), a net backflow
current is expected. According to the relation $j_{t}=\theta_{AHE}$ ($m\times
j_{bf}$), the direction of the net backflow current depends on two factors;
the direction of magnetization ($m$) and the direction of the pumping side
(i.e., laser pumping either from the substrate side or the capping side).

Figure 3: Results from Hall ($\rho_{xy}$) and longitudinal ($\rho_{xx}$)
resistivity measurements on CoFeB films with different thickness. (a)
$\rho_{xy}$ vs. applied magnetic field ($\mu_{0}H_{applied})$, (b)
$\rho_{xy}^{AH}$ vs. $t_{CoFeB}$ where $\rho_{xy}^{AH}$ corresponds to the
saturation value of $\rho_{xy}$, (c) $\rho_{xx}$ vs. $t_{CoFeB}$, and (d) THz
peak-to-peak amplitude (right y-axis) and anomalous Hall angle (left y-axis)
vs. $t_{CoFeB}$.

Terahertz emission due to the iSHE can be neglected, since there is no heavy
metal with large spin-orbit coupling (i.e., high S2C conversion efficiency) in
our films. Instead, the origin of the recorded THz emission can result from a
combination of the UDM and AHE mechanisms. Figure 2(a) shows the generated THz
waveforms from a substrate/CoFeB(5nm)/cap emitter with different magnetization
polarities and different pumping sides. Note that the polarity of the THz
waveform is reversed when reversing the magnetization direction while keeping
the pumping side the same (i.e., the net backflow current direction is the
same). Thus, the THz emission is clearly of magnetic origin. Moreover, the THz
waveform polarity is reversed when the sample is flipped, which is attributed
to the AHE due to a change in direction of the net backflow current. It should
be noted that the waveform polarities will be the same for the UDM mechanism
due to similar magnetization dynamics. Thus, this clearly indicates that the
dominant THz emission mechanism for the CoFeB emitters is the AHE, which is a
combination of bulk and interface effects. The dependence of the THz peak-to-
peak amplitude ($E_{THz}^{P-P}$) on the laser pump fluence is shown in the
inset of Fig. 2(b). Note that $E_{THz}^{P-P}$ shows a linear behaviour up to
0.18 mJ/cm2 pump fluence, thus the pump fluence was fixed at 0.18 mJ/cm2 for
the thickness-dependent study as discussed below.

The CoFeB thickness dependence of the AHE was measured to further investigate
the relationship between the THz emission and the AHE. A schematic of the AHE
measurement using a Hall bar geometry is depicted in the inset of Fig. 3(b). A
longitudinal DC current (4 mA) was applied and the transverse voltage (i.e.,
Hall voltage) was measured while sweeping the out-of-plane magnetic field. The
Hall resistivity ($\rho_{xy}$) contains contributions from the AHE and the
ordinary Hall effect. The anomalous Hall resistivity ($\rho_{xy}^{AH}$),
defined as the saturation value of $\rho_{xy}$, was extracted after
subtracting the contribution from the ordinary Hall effect; the thickness
dependence of $\rho_{xy}^{AH}$ is shown in Fig. 3(b). The CoFeB thickness
dependence of the longitudinal resistivity ($\rho_{xx}$) was also measured,
which follows an exponential behavior as shown in Fig. 3(c). The ratio of the
anomalous Hall resistivity to the longitudinal resistivity defines the
anomalous Hall angle ($\theta_{AHE}=\rho_{xy}^{AH}/\rho_{xx}$). A comparison
of the $\theta_{AHE}$ and the THz peak-to-peak amplitude as a function of
CoFeB thickness is shown in Fig. 3(d). Both $\theta_{AHE}$ and $E_{THz}$
increase and follow a similar trend up to about 5 nm CoFeB thickness. However,
while $\theta_{AHE}$ saturates with a further increase of CoFeB thickness,
$E_{THz}^{P-P}$ decreases. The increase of the THz emission amplitude up to
5nm CoFeB thickness may also be explained by that the applied field ($\sim$ 85
mT) during the THz emission experiment is not sufficient to completely
saturate the CoFeB films as CoFeB may possess an out-of-plane or easy cone
anisotropy in the low thickness regime. Therefore, in-plane magnetization
hysteresis loops were measured for all films, clearly showing that this is not
a valid explanation for the studied films; the magnetization for all CoFeB
films saturates at magnetic fields smaller than 85 mT (cf. Fig. S4 in SM SM ).
The decrease of the THz emission amplitude at larger CoFeB thickness can be
explained by the fact that the spin current generated by the laser pulse
scales as $A/t_{CoFeB}$ Gupta  _et al._ (2021a), where $A$ is the absorbance
of the incident pump pulse by the emitter and $t_{CoFeB}$ is the thickness of
the CoFeB layer. Therefore, the absorbance of the incident laser pulse was
calculated by measuring the incident ($P_{inc}$), reflected ($P_{ref}$) and
transmitted ($P_{trans}$) powers in the optical range for all CoFeB films. The
absorbance was calculated from $A=1-T-R$, where $T=P_{trans}/P_{inc}$ and
$R=P_{ref}/P_{inc}$ are the transmittance and reflectance, respectively. The
absorbance as a function of CoFeB thickness is shown in Fig. 4(a), which
decreases from 60% to 45% as the CoFeB thickness increases. Note that the
change in absorbance ($\sim$33%) is not enough to explain the decrease of the
THz emission ($\sim$2 times). Instead the results show that the decrease of
the THz emission amplitude is dominated by the thickness effect (i.e., the
Fabry-Perot cavity effect Seifert  _et al._ (2016); Gupta  _et al._ (2021a)).

Figure 4: (a) Transmission ($T$), reflectance ($R$) and absorbance ($A$) vs.
$t_{CoFeB}$. Solid lines are shown as guide to the eye. (b) Saturation
magnetization $\mu_{0}M_{s}$ vs. $t_{CoFeB}$. Solid line is shown as guide to
the eye. (c) $\rho_{xy}^{AH}/\mu_{0}M_{s}$ vs. $\rho_{xx}$. Solid line is a
linear fit excluding the result for the 2 nm thick film. (d)
$\mu_{0}M_{s}\cdot t_{CoFeB}$ vs. $t_{CoFeB}$. Solid line is a linear fit to
the experimental data.

The experimental results confirm that the dominant mechanism for THz emission
in CoFeB is the AHE. However, the AHE can have both intrinsic and extrinsic
contributions. The intrinsic contribution for the AHE arises from the
electronic band structure and the Berry phase Nagaosa  _et al._ (2010), which
generally requires growth of epitaxial layers. A dominant intrinsic
contribution to the AHE in our case is unlikley since the CoFeB films are
amorphous in nature (cf. section 1, SM SM ). Moreover, the extrinsic
contribution originates from impurity scatterings such as side jump and skew
scattering Nagaosa  _et al._ (2010). The anomalous Hall resistivity is
expected to be proportional to the saturation magnetization ($M_{s}$) Nagaosa
_et al._ (2010), i.e., $\rho_{xy}^{AH}=R_{s}M_{s}$, where $R_{s}$ is the
anomalous Hall coefficient. $R_{s}$ increases linearly with $\rho_{xx}$ for
skew scattering (i.e., $R_{s}$ $\propto\rho_{xx}$), while the dependence is
parabolic for side jump scattering (i.e., $R_{s}\propto\rho_{xx}^{2}$).
Therefore, the CoFeB thickness dependence of $M_{s}$ was measured to be able
to identify the dominant scattering mechanism for CoFeB; the results are shown
in Fig. 4(b). Figure 4(c) shows $\rho_{xy}^{AH}/M_{s}$ versus $\rho_{xx}$,
indicating a linear relationship. The deviation for the 2 nm thick CoFeB
emitter is explained by the electrical transport in such low thickness is
dominated by surface scattering and that the surface scattering contribution
to the AHE may not be an efﬁcient mechanism for AHE since it is mostly spin-
independent Yang  _et al._ (2018). The argument of the small thickness for
this film is reinforced by considering the effect of the magnetically dead
layer (DL = 0.59 nm) indicated by Fig. 4(d).

##  IV Conclusion

To summarize, we provide direct evidence of THz emission in CoFeB thin films,
which is a consequence of the anomalous Hall effect. The THz emission from
single layer CoFeB films has been studied as a function of CoFeB thickness and
the laser pump fluence. It is shown that the THz emission is a result of the
nonthermal spin-polarized currents created by the laser pulse, which are
reflected at the interfaces of the ferromagnetic layer, forming a net backflow
current that is converted to a transverse charge current via the anomalous
Hall effect. Results from electrical transport measurements show that the
anomalous Hall angle increases with an increase of CoFeB layer thickness up to
about 5 nm of CoFeB, after which it saturates with further increase of the
thickness. This can be contrasted with the thickness dependence of the emitted
THz amplitude, for which the increase up to about 5 nm is superseded by a
decrease due to the Fabry-Perot cavity effect. Lastly, a detailed analysis of
the results obtained from electrical transport measurements indicates that
skew scattering is the dominant mechanism in our anomalous Hall effect-based
CoFeB THz emitter.

##  V acknowledgments

This work is supported by the Swedish Research Council (grant numbers
2021-04658, 2018-04918, and 2017-03725) and Olle Engkvists Stiftelse (grant
number 182–0365).

## References

  * Valušis  _et al._ (2021) G. Valušis, A. Lisauskas, H. Yuan, W. Knap, and H. G. Roskos, Sens. 21, 4092 (2021).
  * Federici and Moeller (2010) J. Federici and L. Moeller, J. Appl. Phys. 107, 6 (2010).
  * Smith  _et al._ (1988) P. R. Smith, D. H. Auston, and M. C. Nuss, IEEE Journal of Quantum Electronics 24, 255 (1988).
  * Venkatesh  _et al._ (2014) M. Venkatesh, K. Rao, T. Abhilash, S. Tewari, and A. Chaudhary, Optical Materials 36, 596 (2014).
  * Lepeshov  _et al._ (2017) S. Lepeshov, A. Gorodetsky, A. Krasnok, E. Rafailov, and P. Belov, Laser & Photonics Reviews 11, 1600199 (2017).
  * Jazbinsek  _et al._ (2019) M. Jazbinsek, U. Puc, A. Abina, and A. Zidansek, Applied Sciences 9, 882 (2019).
  * Vediyappan  _et al._ (2019) S. Vediyappan, A. K. Chaudhary, V. Mottamchetty, R. Arumugam, V. Gandhiraj, M. Senthil Pandian, and R. Perumalsamy, Crystal Growth & Design 19, 6873 (2019).
  * Yang  _et al._ (2007) Z. Yang, L. Mutter, M. Stillhart, B. Ruiz, S. Aravazhi, M. Jazbinsek, A. Schneider, V. Gramlich, and P. Guenter, Advanced Functional Materials 17, 2018 (2007).
  * Kim  _et al._ (2007) K.-Y. Kim, J. H. Glownia, A. J. Taylor, and G. Rodriguez, Optics express 15, 4577 (2007).
  * Yan  _et al._ (2017) D. Yan, Y. Wang, D. Xu, P. Liu, C. Yan, J. Shi, H. Liu, Y. He, L. Tang, J. Feng, _et al._ , Photonics Research 5, 82 (2017).
  * Lu  _et al._ (2017) C. Lu, C. Zhang, L. Zhang, X. Wang, and S. Zhang, Phys. Rev. A 96, 053402 (2017).
  * Bagley  _et al._ (2018) J. D. Bagley, C. D. Moss, S. A. Sorenson, and J. A. Johnson, Journal of Physics B: Atomic, Molecular and Optical Physics 51, 144004 (2018).
  * Fülöp  _et al._ (2020) J. A. Fülöp, S. Tzortzakis, and T. Kampfrath, Advanced Optical Materials 8, 1900681 (2020).
  * Seifert  _et al._ (2016) T. Seifert, S. Jaiswal, U. Martens, J. Hannegan, L. Braun, P. Maldonado, F. Freimuth, A. Kronenberg, J. Henrizi, I. Radu, _et al._ , Nat. Photonics 10, 483 (2016).
  * Gupta  _et al._ (2021a) R. Gupta, S. Husain, A. Kumar, R. Brucas, A. Rydberg, and P. Svedlindh, Adv. Opt. Mater. 9, 2001987 (2021a).
  * Beaurepaire  _et al._ (1996) E. Beaurepaire, J.-C. Merle, A. Daunois, and J.-Y. Bigot, Phys. Rev. Lett. 76, 4250 (1996).
  * Zhang  _et al._ (2020) W. Zhang, P. Maldonado, Z. Jin, T. S. Seifert, J. Arabski, G. Schmerber, E. Beaurepaire, M. Bonn, T. Kampfrath, P. M. Oppeneer, _et al._ , Nature communications 11, 1 (2020).
  * Beaurepaire  _et al._ (2004) E. Beaurepaire, G. Turner, S. Harrel, M. Beard, J.-Y. Bigot, and C. Schmuttenmaer, Applied Physics Letters 84, 3465 (2004).
  * Kampfrath  _et al._ (2013) T. Kampfrath, M. Battiato, P. Maldonado, G. Eilers, J. Nötzold, S. Mährlein, V. Zbarsky, F. Freimuth, Y. Mokrousov, S. Blügel, _et al._ , Nature nanotechnology 8, 256 (2013).
  * Yang  _et al._ (2018) Y. Yang, Z. Luo, H. Wu, Y. Xu, R.-W. Li, S. J. Pennycook, S. Zhang, and Y. Wu, Nature communications 9, 1 (2018).
  * Zhang  _et al._ (2019) Q. Zhang, Z. Luo, H. Li, Y. Yang, X. Zhang, and Y. Wu, Phys. Rev. Applied 12, 054027 (2019).
  * Liu  _et al._ (2021) Y. Liu, H. Cheng, Y. Xu, P. Vallobra, S. Eimer, X. Zhang, X. Wu, T. Nie, and W. Zhao, Phys. Rev. B 104, 064419 (2021).
  * Su  _et al._ (2014) G. Su, Y. Li, D. Hou, X. Jin, H. Liu, and S. Wang, Phys. Rev. B 90, 214410 (2014).
  * Battiato  _et al._ (2010) M. Battiato, K. Carva, and P. M. Oppeneer, Phys. Rev. Lett. 105, 027203 (2010).
  * Battiato  _et al._ (2012) M. Battiato, K. Carva, and P. M. Oppeneer, Phys. Rev. B 86, 024404 (2012).
  * (26) Supplementary Material is available for this article.
  * Gupta  _et al._ (2021b) R. Gupta, E. Bagherikorani, V. Mottamchetty, M. Pavelka, K. Jatkar, D. Dancila, K. Mohammadpour-Aghdam, A. Rydberg, R. Brucas, H. A. Dürr, _et al._ , arXiv preprint arXiv:2110.01547 (2021b).
  * Gupta (2021) R. Gupta,  _Spin Current Generation in Magnetic Heterostructures and its Impact on Terahertz Emission: A Spin Dynamics Perspective_ , Ph.D. thesis, Acta Universitatis Upsaliensis (2021).
  * Nagaosa  _et al._ (2010) N. Nagaosa, J. Sinova, S. Onoda, A. H. MacDonald, and N. P. Ong, Rev. Mod. Phys. 82, 1539 (2010).