# Leveraging Open-Source Models for Legal Language Modeling and Analysis: A
Case Study on the Indian Constitution ††thanks: Citation: Authors. Title.
Pages…. DOI:000000/11111.

VIKHYATH GUPTA
Vidya Jyothi Institute of Technology
Hyderabad
<EMAIL_ADDRESS>
&SRINIVASA RAO P
Curlvee TechnoLabs
Hyderabad
<EMAIL_ADDRESS>

###### Abstract

In recent years, the use of open-source models has gained immense popularity
in various fields, including legal language modelling and analysis. These
models have proven to be highly effective in tasks such as summarizing legal
documents, extracting key information, and even predicting case outcomes. This
has revolutionized the legal industry, enabling lawyers, researchers, and
policymakers to quickly access and analyse vast amounts of legal text, saving
time and resources. This paper presents a novel approach to legal language
modeling (LLM) and analysis using open-source models from Hugging Face. We
leverage Hugging Face embeddings via LangChain and Sentence Transformers to
develop an LLM tailored for legal texts. We then demonstrate the application
of this model by extracting insights from the official Constitution of India.
Our methodology involves preprocessing the data, splitting it into chunks
using ChromaDB and LangChainVectorStores, and employing the Google/Flan-T5-XXL
model for analysis. The trained model is tested on the Indian Constitution,
which is available in PDF format. Our findings suggest that our approach holds
promise for efficient legal language processing and analysis.

_K_ eywords Large language modeling $\cdot$ Hugging Face embeddings $\cdot$
Lang Chain $\cdot$ Sentence Transformers $\cdot$ Chroma DB $\cdot$ Vector
Stores $\cdot$ Google/Flan-T5-XXL model $\cdot$ Legal text analysis.

##  1 Introduction

Legal language processing presents distinct challenges owing to its intricate
nature and highly specialized terminology. The complexity and specificity
inherent in legal texts often confound traditional natural language processing
(NLP) models [9][2], which are typically designed for more general language
understanding tasks. The nuanced vocabulary, syntax, and structure of legal
documents pose obstacles to accurate analysis and interpretation. However,
with the advent of deep learning and the availability of open-source NLP
models, there is newfound potential for overcoming these hurdles and achieving
more precise legal language processing.

Recent advancements in deep learning techniques, particularly the development
of transformer-based architectures, have revolutionized the field of NLP.
Models such as OpenAI’s GPT (Generative Pre-trained Transformer) [11] [13]
series and BERT (Bidirectional Encoder Representations from Transformers) [10]
[12] have demonstrated remarkable capabilities in understanding and generating
human-like text across various domains. Leveraging these advancements,
researchers have begun exploring the creation of specialized legal language
models (LLMs) tailored specifically for legal text comprehension and analysis.
By fine-tuning existing pre-trained models on large corpora of legal
documents, it becomes feasible to develop LLMs that can navigate the
intricacies of legal language with greater accuracy and efficiency.

In this paper, we delve into the development and application of a legal
language model (LLM) [15] utilizing open-source tools and techniques. We
discuss the challenges inherent in legal language processing and explore how
recent advancements in deep learning and NLP can be harnessed to address these
challenges. Furthermore, we examine the process of fine-tuning pre-trained
language models on legal text corpora to create specialized LLMs capable of
handling the unique characteristics of legal documents. Through empirical
evaluation and case studies, we demonstrate the effectiveness of our approach
in enhancing the understanding and processing of legal language, thereby
opening new avenues for automation and efficiency in legal research, document
review, and other legal tasks.

Previous studies have explored various approaches to legal language
processing, including rule-based systems, machine learning algorithms, and
deep learning models. While rule-based systems are often limited by their
reliance on predefined patterns, machine learning and deep learning models
offer more flexibility and scalability. Recent studies have demonstrated the
effectiveness of pre-trained language models for various NLP tasks, including
legal text analysis.

##  2 Methodology

The methodology involves several key steps: Utilizing Hugging Face embeddings
via Lang Chain and Sentence Transformers for LLM development. Preprocessing
the Indian Constitution data extracted from PDF format. Splitting the data
into manageable chunks using Chroma DB and Lang Chain Vector Stores. Training
the Google/Flan-T5-XXL model on the preprocessed data. Evaluating the
performance of the trained model on legal text analysis tasks.

##  3 Mathematical Modeling

The equation that governs the architecture is Transformer Architecture, which
is defined as the fundamental architecture, which forms the basis for models
like GPT and BERT, is represented in the mathematically represented as
follows:

$\text{Attention}(Q,K,V)=\text{softmax}\left(\frac{{QK^{T}}}{{\sqrt{d_{k}}}}\right)V$
(1)

Q,K,V represents matrices for queries, keys and values respectively derived
from the input embeddings is the dimensionality of the keys (and queries),
used for scaling The softmax function is applied to normalize the attention
weights. The optimization of a loss function mathematically with respect to
model parameters using gradient descent. Mathematically is expressed

$L=-\sum_{i=1}^{C}y_{i}\log(p_{i})$ (2)

  * •

L is the loss for a single example

  * •

C is the number of classes.

  * •

yi is a binary indicator of whether class i is the correct classification for
the observation

  * •

pi is the predicted probability of the observation being of class i.

The evaluation of performance of the LLM, various metrics can be used, such as
precision, recall, and F1 score. Mathematically, these metrics are defined as
follows:

$F1=2\times\frac{\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}}$
(3)

where:
TP (True Positive) is the number of correctly predicted positive instances.
FP (False Positive) is the number of incorrectly predicted positive instances.
FN (False Negative) is the number of incorrectly predicted negative instances.

##  4 Data Collection

The cornerstone of any language model development, especially in the legal
domain, lies in the quality and quantity of the data collected for training.
In our research, we focused on gathering a comprehensive dataset of legal
texts, specifically targeting the Indian Constitution as our primary source.
The data collection process involved several steps to ensure the integrity and
relevance of the corpus. We chose the Indian Constitution as the main corpus
for our research due to its significance in legal discourse and its rich and
diverse content. The Constitution encompasses a wide range of legal topics,
including fundamental rights, directive principles, and the structure of
governance, making it an ideal dataset [5][6] for training a legal language
model.

##  5 Extraction of data from various sources

Legal documents, including statutes, regulations, and court opinions, are
frequently stored in Portable Document Format (PDF), presenting challenges for
automated text extraction due to the format’s inherent complexity. In our
research, we encountered [3] the need to extract legal texts from PDF format,
with a primary focus on the Indian Constitution. The extraction process
involved several steps to convert PDF documents into machine-readable text,
ensuring accessibility and compatibility for further analysis.

Figure 1: langchain chromadb vector Database

This figure illustrates the utilization of ChromaDB via Langchain, which
originates from vector stores. Chroma is seamlessly integrated with Langchain.
Additionally, users have the option to create a Chroma client and transmit
information or data accordingly.

##### PDF Parsing Tools :

We utilized sophisticated PDF parsing tools capable of extracting text from
PDF documents while preserving the document’s structure and formatting. These
tools employ algorithms to decipher the intricate layout of PDF files,
identifying text blocks, images, and other elements embedded within the
document.

##### Text Extraction Techniques :

PDF parsing tools employ various text extraction techniques to convert PDF
content into plain text format. These techniques may include optical character
recognition (OCR) for scanned documents, text extraction from embedded fonts,
and handling of special characters and symbols commonly found in legal texts.

##### Handling Document Structure :

Legal documents often contain complex structures, such as chapters, sections,
and subsections, which need to be accurately identified during the extraction
process. PDF parsing tools are designed to recognize and preserve the
hierarchical structure of documents, ensuring that the extracted text retains
its organizational integrity.

##### Dealing with Metadata :

PDF files may contain metadata such as document title, author information, and
creation date, which may not be relevant to the text analysis task. During the
extraction process, it is essential to filter out irrelevant metadata and
focus solely on the textual content of the document.

##### Error Handling :

PDF extraction is prone to errors such as text misalignment, missing
characters, and formatting discrepancies. Robust error handling mechanisms are
employed to detect and correct these errors, ensuring the accuracy and
completeness of the extracted text.

##### Post-extraction Processing :

Once the text is extracted from PDF documents, post-processing techniques may
be applied to clean and refine the text further. This may involve removing
extraneous whitespace, standardizing formatting, and tokenizing the text into
smaller units for analysis.

##  6 Preprocessing

Preprocessing plays a crucial role in preparing raw text data for training a
language model, especially in the legal domain where documents often contain
complex formatting and structural elements. In our research, we implemented a
series of preprocessing steps to clean and standardize the Indian Constitution
dataset before feeding it into our model. These steps aimed to remove noise,
ensure consistency, and optimize the data [10][15] for effective training.

Figure 2: Langchain huggingface Integration

This figure illustrates the seamless integration of data within the Lanchain
framework utilizing Hugging Face technology. It has been sourced from the Lang
Chain documentation

####  6.0.1 Noise Removal : first level

Raw text extracted from legal documents may contain various forms of noise,
including headers, footers, page numbers, and other non-content-related
elements. To enhance the quality of the dataset, we applied techniques such as
regular expressions and pattern matching to identify and remove irrelevant
metadata and formatting artifacts. This process helped isolate the core
textual content from extraneous information.

###  6.1 Formatting Standardization : second level

Legal texts often exhibit diverse formatting styles, including different font
sizes, styles, and alignments. Standardizing the formatting across the dataset
is essential for maintaining consistency and facilitating effective model
training. We employed methods such as text normalization and font detection to
ensure uniformity in the presentation of the text, thereby reducing ambiguity
and enhancing the model’s ability to learn patterns and semantics accurately.

###  6.2 Tokenization : third level

Tokenization involves breaking down the text into individual tokens or words,
forming the basic units of input for the language model. In legal documents,
specialized vocabulary and terminology may pose challenges for traditional
tokenization techniques. To address this, we utilized domain-specific
tokenization libraries tailored to legal text, which can handle complex
linguistic structures and preserve the integrity of legal terms and phrases
during preprocessing.

##  7 An overview of the different types of Flan-T5 models

##### Google/Flan-T5-Small:

This model has the smallest capacity among the Flan-T5 variants. It typically
contains fewer parameters compared to larger models, making it more
lightweight and suitable for tasks where computational resources are limited
or where inference speed is a primary concern. Google/Flan-T5-Base: The Base
variant of the Flan-T5 model offers a moderate capacity, striking a balance
between model size and performance. It contains more parameters than the small
variant, allowing it to capture more complex patterns and relationships in the
data, while still being relatively efficient in terms of memory and
computational resources.

##### Google/Flan-T5-Large:

The Large variant of the Flan-T5 model is characterized by its significantly
larger capacity compared to the Base model. With more parameters, the large
model can capture finer-grained details and nuances in the input data, leading
to improved performance on various natural language processing tasks. However,
training and inference with the large model may require more computational
resources and time. Google/Flan-T5-XL: The XL variant of the Flan-T5 model
represents a further increase in capacity and complexity. With even more
parameters than the large model, the XL model offers enhanced representational
power, enabling it to handle larger datasets and more challenging tasks.
However, this comes at the cost of increased computational requirements and
longer training times.

##### Google/Flan-T5-XXL:

The XXL variant of the Flan-T5 model [7] is the largest and most powerful
among the Flan-T5 models. With an even greater number of parameters, the XXL
model can capture highly intricate patterns and relationships in the input
data, leading to state-of-the-art performance on a wide range of natural
language processing tasks. However, training and inference with the XXL model
are highly resource-intensive and may require specialized hardware and
infrastructure.

Figure 3: Flan-T5 Model

The above Figure illustrates the architecture of the Flant5 model, which can
be extracted from the Hugging Face model repository. It serves as a framework
for training and fine-tuning processes.

##  8 Google/Flan-T5-XXL Model

In our research, the selection of the Google/Flan-T5-XXL model for training
our legal language model (LLM) was based on several key considerations. This
choice was made after careful evaluation of various pre-trained language
models and their suitability for processing legal texts, particularly the
Indian Constitution. The Google/Flan-T5-XXL model stood out due to its
advanced architecture, large parameter size, and proven performance across a
range of natural language processing (NLP) tasks.

##### Model Architecture and Capacity:

The Google/Flan-T5-XXL model is based on the T5 (Text-To-Text Transfer
Transformer) architecture, which has demonstrated state-of-the-art performance
on numerous NLP benchmarks. With an extensive architecture comprising millions
of parameters, including attention heads and layers, the model has a high
capacity to capture intricate linguistic patterns and semantic nuances present
in legal texts. This architectural depth is essential for effectively
representing the complex syntax and vocabulary characteristic of legal
language.

##### Training Data Size and Diversity:

The Google/Flan-T5-XXL model is trained on a vast and diverse corpus of text
data, encompassing a broad range of domains and languages. This extensive pre-
training enables the model to generalize well to new tasks and domains,
including legal language processing. By leveraging the knowledge encoded in
its pre-trained parameters, the model can effectively learn representations of
legal concepts and terminology, facilitating accurate analysis and
interpretation of legal texts.

##### Fine-Tuning Capabilities:

T5 architecture is well-suited for fine-tuning on specific downstream tasks,
allowing researchers to adapt the model to domain-specific datasets and
objectives. In our case, fine-tuning the Google/Flan-T5-XXL model on the
Indian Constitution dataset enables it to specialize in understanding and
processing legal language, improving its performance on tasks such as keyword
extraction, summarization, and sentiment analysis within the legal domain.

Community Support and Documentation: The Google/Flan-T5-XXL model benefits
from strong community support and extensive documentation, providing
researchers with access to resources, tutorials, and pre-trained checkpoints
for efficient model deployment and experimentation. This ecosystem of support
enhances the accessibility and usability of the model, facilitating seamless
integration into our research pipeline and enabling rapid prototyping and
iteration.

Performance and Benchmark Results: Prior research and benchmark evaluations
have demonstrated the superior performance of the Google/Flan-T5-XXL model
across a wide range of NLP tasks, including text generation, question
answering, and summarization. By leveraging a model with proven performance,
we can have confidence in its ability to accurately process and analyze legal
texts, yielding reliable and actionable insights for our research objectives.

##  9 Alternative Models

If alternative models were considered for training our legal language model,
the outcome could vary based on the specific characteristics and capabilities
of each model. Here’s a hypothetical overview of potential outcomes if
alternative models were chosen:

##### BERT (Bidirectional Encoder Representations from Transformers):

BERT is another popular pre-trained language model [14] known for its
bidirectional contextual representation learning. While BERT offers strong
performance on a wide range of NLP tasks, its unidirectional nature may limit
its ability to capture long-range dependencies and syntactic structures
present in legal texts. As a result, the model may struggle to effectively
process complex legal language, potentially leading to suboptimal performance
on tasks requiring deep semantic understanding.

##### GPT (Generative Pre-trained Transformer):

GPT models, including GPT-3, are renowned for their generative capabilities
[13] and contextual understanding of text. However, their autoregressive
nature and lack of bidirectional context may pose challenges for tasks
requiring comprehensive comprehension and analysis of legal documents. While
GPT models excel in text generation and completion tasks, they may not be as
well-suited for tasks such as legal text summarization or sentiment analysis,
where a nuanced understanding of legal concepts is essential.

##### XLNet:

XLNet is a transformer-based language model that leverages permutation-based
training to capture bidirectional context while maintaining the advantages of
autoregressive modeling. XLNet’s innovative training scheme enables it to
achieve strong performance on various NLP benchmarks, including tasks
requiring deep contextual understanding. If XLNet were chosen for training our
legal language model, it could potentially offer comparable performance to the
Google/Flan-T5-XXL model, particularly in tasks involving complex syntactic
and semantic analysis of legal texts.

while alternative models such as BERT, GPT, and XLNet offer unique strengths
and capabilities, the choice of the Google/Flan-T5-XXL model for our research
was informed by its advanced architecture, fine-tuning capabilities, community
support, and proven performance in processing and analyzing natural language
text, including legal documents like the Indian Constitution.

Training the Google Flan-T5 XXL model involves several mathematical concepts
and computations. Here’s an overview of the mathematical aspects involved in
the training process:

The loss function which measures the discrepancy between the model’s
predictions and the ground truth labels. For language modeling tasks, such as
pretraining Flan-T5 XXL, the loss function typically involves calculating the
cross-entropy loss between the predicted probability distribution over the
vocabulary and the actual target tokens, the cross-entropy loss L for a single
training example can be expressed as:

$L=-\sum_{i=1}^{V}Sy_{i}\log(p_{i})$ (4)

#####

Gradient Descent optimization algorithm used to update the parameters of the
model during training. It iteratively adjusts the parameters in the direction
of the steepest descent of the loss function to minimize the loss.

#####

Mathematically ,The parameters of the model are updated according to the
following equation:

$\theta_{t+1}=\theta_{t}-\alpha\Delta\theta L$ (5)

Backpropagation is to compute gradients efficiently in neural networks. It
involves recursively applying the chain rule of calculus to propagate
gradients backward through the network, starting from the output layer to the
input layer. Mathematically, the gradients of the loss function with respect
to the parameters of the model are computed using the chain rule

$\frac{\partial
L}{\partial\theta}=-\frac{1}{|\text{batch}|}\sum_{i=1}^{|\text{batch}|}\nabla_{\theta}\mathcal{L}(\text{output}_{i},\text{hidden
state}_{i})$ (6)

where $\theta$ represents the model parameters, and output and hidden state
represent the output and hidden states of the model, respectively. Training
large models like Flan-T5 XXL often involves processing training data in mini-
batches rather than processing the entire dataset at once. This approach
reduces memory requirements and speeds up training by computing gradients on
smaller subsets of data.

Mathematically, the loss function is computed over a mini-batch of training
examples, and the gradients are averaged over the mini-batch before updating
the model parameters using gradient descent.

These mathematical concepts form the foundation of the training process for
Google Flan-T5 XXL and other neural network models. Understanding these
concepts is essential for effectively training and optimizing the model for
various natural language processing tasks.

##  10 Results

Through rigorous experimentation, we have validated the efficacy of our
approach in processing and analyzing legal texts, particularly focusing on the
intricate domain of the Indian Constitution. Our trained model exhibits a
remarkable capacity to grasp the nuanced semantics and complex structural
elements inherent in constitutional law. This capability empowers a spectrum
of analytical tasks, including but not limited to keyword extraction,
summarization, and sentiment analysis, essential for legal research, policy
formulation, and decision-making processes. By leveraging advanced techniques
in deep learning and fine-tuning methodologies, we have successfully equipped
our model to navigate the intricacies of legal language specific to the Indian
legal system.

Furthermore, our methodology underscores the scalability and efficiency of
utilizing open-source models for the development of specialized legal language
models (LLMs). By harnessing pre-existing language models and fine-tuning them
on domain-specific corpora, we have demonstrated a cost-effective and scalable
approach to LLM development. This approach not only streamlines the process of
model creation but also ensures that the resulting LLMs are tailored to the
unique characteristics of legal texts within a specific jurisdiction or legal
framework. The scalability of our methodology enables the rapid adaptation of
LLMs to evolving legal landscapes and facilitates their integration into
various legal technology applications, thereby fostering innovation and
efficiency within the legal domain.

our experiments serve as a testament to the potential of open-source models
and advanced deep learning techniques in enhancing legal language processing.
By effectively capturing the intricacies of legal texts, our approach lays the
groundwork for a wide range of applications aimed at facilitating legal
research, analysis, and decision-making. Moreover, the scalability and
efficiency demonstrated in our methodology pave the way for the widespread
adoption of specialized legal language models, heralding a new era of
automation and efficiency in the legal profession.

##  11 Discussion

The implications of our findings extend far beyond the confines of our
experimental setup, shedding light on the broader landscape of legal language
processing and analysis. By showcasing the effectiveness of our approach, we
underscore its potential as a viable solution for automating labor-intensive
tasks inherent in legal research and document analysis. The ability of our
model to accurately capture the nuances of legal texts opens doors to
increased efficiency and accuracy in tasks such as case law analysis, contract
review, and statutory interpretation. This not only alleviates the burden on
legal professionals but also reduces the likelihood of errors and oversights,
thereby enhancing the overall quality and reliability of legal work.

Moreover, the open-source nature of our methodology ensures accessibility and
reproducibility, democratizing access to advanced tools and techniques in
legal language processing. By making our methodology openly available to
researchers and practitioners in the legal domain, we foster collaboration and
knowledge sharing, accelerating progress in the field. This accessibility also
encourages the development of diverse applications and extensions, tailored to
specific legal contexts and requirements, thereby enriching the ecosystem of
legal technology solutions.

Looking ahead, we identify several avenues for future research and
development. One such area of exploration involves extending our model to
encompass a broader range of legal documents and jurisdictions. By training
our model on diverse corpora encompassing statutes, regulations, case law, and
legal commentaries from various jurisdictions, we can enhance its versatility
and applicability across different legal systems. Additionally, exploring
techniques for domain adaptation and transfer learning can further improve the
adaptability of our model to new contexts, ensuring its relevance and efficacy
in evolving legal landscapes. Through ongoing research and innovation, we aim
to continuously push the boundaries of legal language processing, advancing
towards a future where technology serves as a powerful ally in the pursuit of
justice and legal excellence.

##  12 Future Directions

As we look ahead, several avenues for future research and development in the
field of legal language modeling and analysis present themselves: While our
study focused on the Indian Constitution, extending our approach to other
legal documents and jurisdictions would enhance its applicability and
relevance. Exploring the adaptation of our model to diverse legal contexts
could provide valuable insights into cross-cultural legal language processing.

Fine-tuning our model for specific legal tasks, such as contract analysis,
case law summarization, or legal opinion mining, could further improve its
performance and utility. Tailoring the model to address the unique
requirements of different legal applications would enhance its effectiveness
and efficiency.

Integrating our LLM into existing legal information retrieval systems could
streamline the process of accessing and analyzing legal documents. Developing
interfaces and APIs that enable seamless integration with legal databases and
platforms would facilitate the adoption of our model in legal research and
practice.

Addressing ethical and bias considerations in legal language modeling is
crucial for ensuring fairness and accountability. Future research should focus
on developing methods for identifying and mitigating biases in legal text
analysis, as well as incorporating ethical principles into the design and
deployment of LLM systems. Collaboration with legal experts and practitioners
is essential for validating and refining our approach. Engaging with legal
professionals to gather feedback, validate model outputs, and identify real-
world use cases would enhance the practical relevance and impact of our
research.

Exploring multimodal approaches that combine text-based analysis with other
modalities, such as images, audio, or structured data, could enrich the
capabilities of our LLM. Investigating the integration of multimodal
information sources for legal text understanding and interpretation could open
new avenues for research and innovation.

By pursuing these future directions, we aim to advance the state-of-the-art in
legal language modeling and analysis, ultimately contributing to more
efficient, accessible, and equitable legal research and practice.

##  13 Privacy Protection

##### Data Access Control:

The code interacts with files stored in Google Drive [18]
[19](drive.mount(’/content/drive’)) and a local directory (pdf-folder-path).
Access to these data sources is controlled through authentication mechanisms,
ensuring that only authorized users can access the files.

##### Minimal Data Exposure:

The code appears to focus on processing and analyzing PDF documents, with no
indication of transmitting or storing personally identifiable information
(PII). This minimizes the risk of exposing sensitive data to unauthorized
parties.

##### Limited Dependencies:

By utilizing only essential libraries and dependencies for PDF processing,
embeddings, and model loading, the code minimizes the attack surface and
potential vulnerabilities that could compromise privacy.

##  14 Conclusion

This case study on the Indian Constitution exemplifies the transformative
impact of open-source models, enabling legal professionals and policymakers to
navigate extensive legal text with ease. The reliability and proper usage of
these language models are essential considerations, and ongoing research is
crucial to ensure their ethical and lawful application. As advancements
continue to unfold, the integration of open-source models in legal language
analysis will undoubtedly shape the future of legal practice and decision-
making processes.
The endeavor culminates in the successful development and validation of a
Legal Language Model (LLM) tailored to the nuances of the Indian Constitution.
Through the utilization of open-source models and meticulous fine-tuning, we
have showcased the model’s robustness and effectiveness in navigating the
complexities of legal language. Our approach represents a significant stride
forward in the realm of legal language processing and analysis, holding
considerable promise for a multitude of applications within the legal domain.

By harnessing the capabilities of open-source tools and techniques, we not
only demonstrate the feasibility of leveraging existing resources for LLM
development but also pave the way for widespread adoption and utilization. The
accessibility and affordability inherent in our methodology democratize access
to advanced language processing technologies, empowering legal professionals
and researchers alike to enhance their workflows and insights.

Looking ahead, our efforts lay the groundwork for further advancements in
legal research and practice. As we continue to refine and expand upon our
model, we anticipate its integration into various legal technology
applications, revolutionizing processes such as contract review, legal
document analysis, and case law research. Through collaborative efforts and
continued innovation, we aspire to foster a future where cutting-edge
technology serves as an invaluable asset in the pursuit of justice and legal
excellence.

[1] Dernoncourt, F., Lee, J. Y., and Szolovits, P. (2020). "Legal BERT: The
Muppets straight out of Law School." arXiv preprint arXiv:1908.10063.

[2] Tissot, A., Strubell, E., Vig, J., Dechter, A., and McCallum, A. (2020).
"Jurassic-1: Pretraining Strategies and Data Splits for Training Large
Language Models for Legal NLP." arXiv preprint arXiv:2012.05426.

[3] Devlin, J., Chang, M. W., Lee, K., and Toutanova, K. (2018). "BERT: Pre-
training of Deep Bidirectional Transformers for Language Understanding." arXiv
preprint arXiv:1810.04805.

[4] Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2018).
"Improving Language Understanding by Generative Pretraining." OpenAI Technical
Report, 8(9), 1-11.

[5] Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., and Le, Q.
V. (2019). "XLNet: Generalized Autoregressive Pretraining for Language
Understanding." arXiv preprint arXiv:1906.08237.

[6] Ashley, K. D., and Walker, V. (2018). "Legal Text Mining: Process and Case
Studies." In Legal Knowledge and Information Systems (pp. 1-18). Springer,
Cham.

[7] Manning, C. D., and Schütze, H. (1999). Foundations of Statistical Natural
Language Processing. MIT Press. [8] Jurafsky, D., and Martin, J. H. (2020).
Speech and Language Processing (3rd ed.). Pearson.

[9] Ramanathan, V., and Bhatia, S. (2020). Legal Language Processing: Corpus-
Based Analysis and Applications.

[10] Katz, D. M., and Bommarito II, M. J. (2019). Research Handbook on Big
Data Law. Edward Elgar Publishing.

[11] Chhabra, S. (2016). Legal Research Methodology. Universal Law Publishing.

[12] Google/Flan-T5-XXL model documentation.

[13] Indian Constitution - Official website of the Government of India
(https://www.india.gov.in/indian-constitution).

[14] Wolf, T., Sanh, V., Chaumond, J., and Delangue, C. (2019). Transformers:
State-of-the-Art Natural Language Processing. arXiv preprint arXiv:1910.03771.

[15] Reimers, N., and Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings
using Siamese BERT-Networks. arXiv preprint arXiv:1908.10084.

[16] Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2018).
Improving Language Understanding by Generative Pretraining. OpenAI.

[17] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.
N., … and Polosukhin, I. (2017). Attention is all you need. Advances in neural
information processing systems, 30, 5998-6008.

[18] OpenAI. (2021). GPT-3: Language Models are Few-Shot Learners.

[19] Devlin, J., Chang, M. W., Lee, K., and Toutanova, K. (2018). BERT: Pre-
training of Deep Bidirectional Transformers for Language Understanding. arXiv
preprint arXiv:1810.04805.

[20] Howard, J., and Ruder, S. (2018). Universal language model fine-tuning
for text classification. arXiv preprint arXiv:1801.06146.

[21] Hugging Face. (2021). Transformers Library Documentation.

[22] A. A. Abokhzam, N. K. Gupta, and D. K. Bose, “Efficient diabetes mellitus
prediction with grid based random forest classifier in association with
natural language processing,”

[23] Arpaci, I., Kilicer, K., and Bardakci, S. (2015). Effects of security and
privacy concerns on educational use of cloud services. Computers in Human
Behavior.

[24] Changchit, C. (2008). Data Protection and Privacy Issue. Journal Of
Information Privacy and Security.


For Henryk Iwaniec
In celebration of his 75th birthday

# Equidistribution for Solutions of $p+m^{2}+n^{2}=N$, and for Châtelet
Surfaces

D.R. Heath-Brown  Mathematical Institute
Radcliffe Observatory Quarter
Woodstock Road
Oxford
OX2 6GG<EMAIL_ADDRESS>
###### Abstract.

We present a general method for handling problems that ask for the
equidistribution of solutions to equations involving $m^{2}+n^{2}$, and
illustrate it by considering $p+m^{2}+n^{2}=N$.

###### 2010 Mathematics Subject Classification:

11D45 (11G35, 11G50, 11P55, 14G05, 14G25)

##  1\. Introduction

It was shown by Linnik [11], [12] that every large integer $N$ may be
represented as $N=p+m^{2}+n^{2}$, with $p$ prime. His work built on a method
of Hooley [7], who had established this result subject to the Generalized
Riemann Hypothesis. In fact there is a positive constant $\delta$ such that

$\\#\\{(p,m,n):p+m^{2}+n^{2}=N\\}=\frac{N}{\log N}C(N)+O(N(\log
N)^{-1-\delta})$

with

$C(N)=\pi\prod_{p>2}\left(1+\frac{\chi_{4}(p)}{p(p-1)}\right)\prod_{p\mid
N}\frac{(p-1)(p-\chi_{4}(p))}{p^{2}-p+\chi_{4}(p)},$

(where $\chi_{4}$ is the non-trivial character modulo 4) — see Hooley [8,
Chapter 5], for example. We remark here that

$(\log\log N)^{-1/2}\ll C(N)\ll(\log\log N)^{1/2},$

so that the asymptotic formula saves a power of $\log N$ over the main term.

In this paper we examine the distribution of the solutions $p,m,n$. There are
various ways to do this, but the approach we describe appears to be very
flexible. It should be stressed however that most of the results we describe
in the context of the equation $N=p+m^{2}+n^{2}$ and its generalizations can
be obtained by a different route, given by Bredikhin and Linnik [2].

There is no difficulty in controlling the size of the variable $p$, and it is
an easy matter to adapt Hooley’s argument to show that

$\\#\\{(p,m,n):p+m^{2}+n^{2}=N,\,aN<p\leq bN\\}\hskip 85.35826pt$ $\hskip
85.35826pt{}=(b-a)\frac{N}{\log N}C(N)+O(N(\log N)^{-1-\delta})$

uniformly for $0\leq a<b\leq 1$. If $aN<p\leq bN$ then $(m,n)$ will lie in the
annulus $(1-b)N\leq||(m,n)||_{2}\leq(1-a)N$. In order to describe the
equidistribution of $(m,n)$ within this annulus it is natural to write
$\alpha=m+in\in\mathbb{Z}[i]$, with $N(\alpha)=m^{2}+n^{2}$, and to consider
$\arg(\alpha)$. We then have the following result.

######  Theorem 1.1.

There is an absolute constant $\delta>0$ as follows. Let real numbers
$a,b,c,d$ be given, with $0\leq a<b\leq 1$ and $0\leq c<d\leq 2\pi$. Let
$N\in\mathbb{N}$ be given. Then

$\\#\\{(p,\alpha):\,p\mbox{
prime},\,\alpha\in\mathbb{Z}[i],\,p+N(\alpha)=N,\,aN<p\leq
bN,\,c<\arg(\alpha)\leq d\\}$ $\hskip
56.9055pt{}=\frac{(b-a)(d-c)}{2\pi}\frac{N}{\log N}C(N)+O_{\varepsilon}(N(\log
N)^{-1-\delta}).$

The natural way to handle the restriction on $\arg(\alpha)$ is via the
Erdős–Turán inequality. For any finite subset $A$ of $\mathbb{Z}[i]$ and any
$H\in\mathbb{N}$ we have

$\\#\\{\alpha\in A:\,c<\arg(\alpha)\leq d\\}\hskip 170.71652pt$ $\hskip
85.35826pt{}=\frac{d-c}{2\pi}\\#A+O(H^{-1}\\#A)+O\left(H^{-1}\sum_{h\leq
H}\left|\sum_{\alpha\in
A}\left(\frac{\alpha}{|\alpha|}\right)^{h}\right|\right).$

In our setting the term $O(H^{-1}\\#A)$ will contribute $O(NC(N)/(H\log N))$
in Theorem 1.1, and so we see that the required result will follow from the
following estimate, on taking $H=\log N$, for example.

######  Theorem 1.2.

Define

$g_{h}(n)=\frac{1}{4}\sum_{N(\alpha)=n}\left(\frac{\alpha}{|\alpha|}\right)^{h}$

and $f_{h}(n)=|g_{h}(n)|$. Then

$\sum_{p<N}f_{h}(N-p)\ll_{\varepsilon}N(\log N)^{-5/4+\varepsilon}$

for any fixed $\varepsilon>0$, uniformly for $1\leq h\leq\log N$.

We remark that the exponent $5/4$ can be improved to $2-2/\pi$, but this has
no qualitative benefit for us.

The reader may be surprised that it is sufficient to consider the average of
$f_{h}(N-p)$, rather than $g_{h}(N-p)$. The fact that this is indeed enough is
the fundamental observation behind this paper. As motivation we point out that
for most primes $p$ the number $N-p$ will not be a sum of two squares, and
$g_{h}(N-p)$ will therefore vanish. However when $N-p$ is a sum of two squares
the sum for $g_{h}(N-p)$ will typically contain many terms $\alpha$, so that
we may expect some cancellation to occur.

The above idea, that we can obtain enough cancellation by considering
$|g_{h}(n)|$ rather than $g_{h}(n)$, has occurred previously in other
contexts. (We are grateful to Valentin Blomer for this observation.) Thus, for
example, Holowinsky’s work [6] on Quantum Unique Ergodicity looks at shifted
convolution sums

$\sum_{n\leq x}|\lambda_{1}(n)\lambda_{2}(n+l)|,$

which can be handled non-trivially using Erdős’ method. In Holowinsky’s
application one takes $\lambda_{1}=\lambda_{2}$ to be the Fourier coefficients
of a holomorphic modular cusp form.

The above ideas can be applied in many other situations. For example, one may
consider the equidistribution of points on Châtelet surfaces, given by
equations

$F(u,v)=x^{2}-ay^{2},$

where $F\in\mathbb{Z}[u,v]$ is a separable quartic polynomial and
$a\in\mathbb{Z}$ is not a square. Investigations into Manin’s Conjecture for
these surfaces have focused on the case $a=-1$, but in this situation the
conjecture has been established for all forms $F$. The most difficult case, in
which $F$ is irreducible over $\mathbb{Q}(i)$, has been handled by de la
Bretèche and Tenenbaum [4]. Since one is interested in rational points it is
natural to consider solutions to

$t^{2}F(u,v)=x^{2}+y^{2}$ (1.1)

with

$t,u,v,x,y\in\mathbb{Z},\;\;\;\gcd(u,v)=\gcd(t,x,y)=1,\;\;\;t>0.$ (1.2)

The counting function considered by de la Bretèche and Tenenbaum, is one half
the number of solutions satisfying the height condition

$t^{1/2}\max(|u|,|v|)\leq B^{1/2}.$ (1.3)

They then obtain an asymptotic formula

$\sigma_{\infty}(F)\mathfrak{S}(F)B\log B+O_{F}(B(\log B)^{99/100}),$

where

$\sigma_{\infty}(F)=\frac{\pi}{2}\mathrm{meas}\\{(u,v)\in[-1,1]^{2}:\,F(u,v)>0\\}$

and $\mathfrak{S}(F)$ is a product of $p$-adic densities. There is no
difficulty in adapting the argument to replace (1.3) with a condition

$t^{1/2}(u,v)\in B^{1/2}\mathscr{R}$ (1.4)

for an arbitrary rectangle $\mathscr{R}\subseteq[-1,1]^{2}$. This produces an
analogous asymptotic formula with $\mathfrak{S}(F)$ replaced by

$\sigma_{\infty}(F;\mathscr{R})=\frac{\pi}{2}\mathrm{meas}\\{(u,v)\in\mathscr{R}:\,F(u,v)>0\\}.$

We then have the following result, which controls the argument of $x+iy$, in
addition to the location of $(u,v)$.

######  Theorem 1.3.

Suppose one is given a rectangle $\mathscr{R}\subseteq[-1,1]^{2}$ and real
numbers $a<b$ in $[0,2\pi]$. Let $N(B)$ be half the number of 5-tuples
$(t,u,v,x,y)\in\mathbb{Z}^{5}$ satisfying (1.1) and (1.2), and with

$t^{1/2}(u,v)\in B^{1/2}\mathscr{R}\;\;\;\mbox{and}\;\;\;\arg(x+iy)\in[a,b].$

Then

$N(B)=\frac{b-a}{2\pi}\sigma_{\infty}(F;\mathscr{R})\mathfrak{S}(F)B\log
B+O_{F}(B(\log B)^{99/100}).$

The key input here is the following estimate, which we will prove in §3

######  Lemma 1.4.

For any fixed $\varepsilon>0$ we have

$\sum_{t^{1/2}\max(|u|,|v|)\leq
B^{1/2}}\left|\sum_{\begin{subarray}{c}x^{2}+y^{2}=t^{2}F(u,v)\\\
\gcd(t,x,y)=1\end{subarray}}\left(\frac{x+iy}{|x+iy|}\right)^{4h}\right|\ll_{F,\varepsilon}B(\log
B)^{3/4+\varepsilon}$

uniformly for $1\leq h\leq\log B$.

These ideas are not restricted to the use of Grössencharacters. For example we
can consider the equation $N=p+m^{2}+n^{2}$ subject to congruence conditions
on $m$ and $n$ by looking at $N=p+N(\alpha)$ subject to
$\alpha\equiv\alpha_{0}($mod $k)$, say, with $\alpha_{0}\in\mathbb{Z}[i]$
coprime to $k\in\mathbb{N}$. This condition can be detected with Dirichlet
characters $\chi($mod $k)$ over $\mathbb{Z}[i]$. With these we have the
following estimate.

######  Theorem 1.5.

Let $k\in\mathbb{N}$ and $\varepsilon>0$ be given. Then

$\sum_{p<N}\left|\frac{1}{4}\sum_{N(\alpha)=N-p}\chi(\alpha)\right|\ll_{k,\varepsilon}N(\log
N)^{-5/4+\varepsilon}$

for every non-principal Dirichlet character $\chi($mod $k)$ over
$\mathbb{Z}[i]$ such that $\chi(i)=1$.

We leave the details of the proof to the reader. In order to apply this to
examine $N=p+N(\alpha)$ with $\alpha\equiv\alpha_{0}($mod $k)$, (for
$\alpha_{0}$ coprime to $k$) we need to know about

$\sum_{N=p+N(\alpha)}\chi_{0}(\alpha)$

where $\chi_{0}$ is the principal character modulo $k$. However $\alpha$ and
$k$ are coprime if and only if $N(\alpha)$ and $k$ are coprime, so that it is
enough to consider

$\\#\\{(p,m,n):\,N=p+m^{2}+n^{2},\,\gcd(N-p,k)=1\\}.$

This may be tackled by a trivial variant of Hooley’s method, in which one puts
congruence conditions on $p$.

One can also handle congruence conditions $\alpha\equiv\alpha_{0}($mod $k)$
when $\alpha$ shares a common factor with $k$. If $\alpha_{0}=\beta\alpha_{1}$
and $k=\beta\gamma$ with $\alpha_{1}$ and $\gamma$ coprime then the congruence
$\alpha\equiv\alpha_{0}($mod $k)$ implies that $\alpha=\beta\alpha^{\prime}$,
say, with

$\alpha^{\prime}\equiv\alpha_{1}(\mathrm{mod}\,\gamma).$ (1.5)

The equation $N=p+N(\alpha)$ then requires $p\equiv N($mod $N(\beta))$, so
that we have to solve

$\frac{N-p}{N(\beta)}=N(\alpha^{\prime})\;\;\;\mbox{with}\;\;\;p\equiv
N(\mathrm{mod}\,N(\beta)),$

subject to the congruence condition (1.5). We can tackle this with the
machinery described above, using a version of Theorem 1.5 in which one
restricts to primes $p\equiv N($mod $N(\beta))$ and replaces the summation
condition $N(\alpha)=N-p$ by $N(\alpha)=(N-p)/N(\beta)$. In particular this is
enough to give an asymptotic formula for the number of solutions to

$N=p+a^{4}m^{2}+b^{4}n^{2}$

for any fixed $a,b\in\mathbb{N}$, and thereby to count solutions of

$N=p+m^{2}+n^{2},\;\;\;\;m,n\mbox{ square-free}.$

This is the problem for which Hooley [9] proves a positive lower bound. The
reader should note though that Hooley [9, page 202] suggests that the
necessary asymptotic formulae “can probably be established …by a different
method involving an elaborate use of the arithmetic of binary quadratic
forms.”

One can also replace $m^{2}+n^{2}$ by other quadratic forms, and this was the
main achievement (in the context of the equation $N=p+m^{2}+n^{2}$) in the
work of Bredikhin and Linnik [2] mentioned earlier. As an example, we consider
the Châtelet surface

$F(u,v)=x^{2}-2y^{2},$

where $F(u,v)\in\mathbb{Z}[u,v]$ is a quartic form, assumed now to be
irreducible over $\mathbb{Q}(\sqrt{2})$. The function

$r_{2}(n)=\sum_{\begin{subarray}{c}d\mid n\\\
d\,\mathrm{odd}\end{subarray}}\left(\frac{2}{d}\right),\;\;\;n\in\mathbb{Z}\setminus\\{0\\},$

counts solutions of $n=x^{2}-2y^{2}$ once from each set of associates of
$x+y\sqrt{2}$. Equivalently, $r_{2}(n)$ counts integral ideals $I$ of
$\mathbb{Q}(\sqrt{2})$ having norm $N(I)=|n|$. (Previously we had used $N(*)$
for the norm function on $\mathbb{Q}(i)$, but here it is the norm on
$\mathbb{Q}(\sqrt{2})$. We trust this will cause no confusion.) A moment’s
reflection shows that if $t^{2}\mid x^{2}-2y^{2}$ then $\gcd(t,x,y)=1$ if and
only if $\gcd(t,\alpha,\alpha^{\sigma})=1$, where $\alpha=x+y\sqrt{2}$ and
$\sigma$ is the non-trivial automorphism of $\mathbb{Q}(\sqrt{2})$. One
therefore wants to count solutions to

$t^{2}|F(u,v)|=N(I)$ (1.6)

subject to $\gcd(u,v)=\gcd(t,I,I^{\sigma})=1$, lying in a region (1.4), these
being the natural analogues of (1.1), (1.2). The method of de la Bretèche and
Tenenbaum [4] can be readily adapted to show that the number of solutions
takes the form

$c(\mathscr{R})B\log B+O_{F}(B(\log B)^{99/100}),$

with a constant $c(\mathscr{R})$ depending on $F$ as well as $\mathscr{R}$.
However one would wish to control the location of the corresponding points
$(x,y)$ for which $I=(x+y\sqrt{2})$. If $\mathscr{R}$ is suitably small one
will know the product

$\left(\frac{|x-y\sqrt{2}|}{t}\right)\left(\frac{|x+y\sqrt{2}|}{t}\right)=t^{-2}N(I)=|F(u,v)|$

to a good degree of approximation, and hence it is enough to control the
quotient

$\left.\left(\frac{|x-y\sqrt{2}|}{t}\right)\middle/\left(\frac{|x+y\sqrt{2}|}{t}\right)=\frac{|x-y\sqrt{2}|}{|x+y\sqrt{2}|}.\right.$

We can do this, and indeed slightly more, by using the Grössencharacter

$\chi(I)=\mathrm{sgn}(x^{2}-2y^{2})\exp\left\\{\pi
i\frac{\log|x-y\sqrt{2}|-\log|x+y\sqrt{2}|}{2\log(1+\sqrt{2})}\right\\},$

where $I=(x+y\sqrt{2})$. (The reader who is unfamiliar with such things may
readily check that this is at least well-defined.) In analogy to Lemma 1.4 we
have the following bound.

######  Lemma 1.6.

For any fixed $\varepsilon>0$ we have

$\sum_{t^{1/2}\max(|u|,|v|)\leq
B^{1/2}}\left|\sum_{\begin{subarray}{c}N(I)=t^{2}F(u,v)\\\
\gcd(t,I,I^{\sigma})=1\end{subarray}}\chi(I)^{h}\right|\ll_{F,\varepsilon}B(\log
B)^{3/4+\varepsilon}$

uniformly for $1\leq h\leq\log B$.

This allows us to count solutions to (1.6) asymptotically, with $\chi(I)$
restricted to any given arc of the unit circle. A moment’s thought shows that
this corresponds to having $I=(x+y\sqrt{2})$ with
$(x-y\sqrt{2})/(x+y\sqrt{2})$ in a given interval
$(a,b)\subset(-\infty,0)\cup(0,\infty)$. We are then able to count solutions
to $t^{2}F(u,v)=x^{2}-2y^{2}$ in which both $(u,v)$ and $(x,y)$ lie in
prescribed regions.

Lastly we mention that one can work analogously with class group characters.
Suppose, for example, that we are interested in the Châtelet surface

$F(u,v)=x^{2}+14y^{2},$

for a quartic form $F\in\mathbb{Z}[u,v]$ which is irreducible over
$\mathbb{Q}(\sqrt{-14})$. 111We are grateful to Evan O’Dorney for pointing out
an error in the discussion of this problem in an earlier version of this
paper. There are four classes of positive definite binary quadratic forms of
discriminant $-56$, with representatives $x^{2}+14y^{2}$, $2x^{2}+7y^{2}$,
$3x^{2}+2xy+5y^{2}$, and $3x^{2}-2xy+5y^{2}$. Since the first two of these lie
in the same genus it is not possible to separate out representations by these
two forms using only congruence conditions. The combined number of
representations of $n\in\mathbb{N}$ by all four classes of forms is twice the
number of integral ideals $I$ of $\mathbb{Q}(\sqrt{-14})$ with norm $N(I)=n$;
and the number of representations by $x^{2}+14y^{2}$ is twice the number of
such $I$ that are principal. As above we will want to consider solutions to
$t^{2}F(u,v)=N(I)$ with $\gcd(u,v)=1$ and $\gcd(t,I,I^{\sigma})=1$, where
$\sigma$ is now complex conjugation. We may obtain an asymptotic formula for
the problem in which we count both principal and non-principal ideals $I$,
based on the formula

$\\#\\{I:\,N(I)=n\\}=\sum_{\begin{subarray}{c}d\mid n\\\
\gcd(d,14)=1\end{subarray}}\left(\frac{-14}{d}\right),$

and following the method laid out by de la Bretèche and Tenenbaum [4]. Then,
to restrict the count to principal ideals we use class group characters.
Following our previous argument we then see that the following lemma suffices

######  Lemma 1.7.

Let $\chi$ be a non-trivial class group character for
$\mathbb{Q}(\sqrt{-14})$, and let $N$ be the norm function for
$\mathbb{Q}(\sqrt{-14})$. Then for any fixed $\varepsilon>0$ we have

$\sum_{t^{1/2}\max(|u|,|v|)\leq
B^{1/2}}\left|\sum_{\begin{subarray}{c}N(I)=t^{2}F(u,v)\\\
\gcd(t,I,I^{\sigma})=1\end{subarray}}\chi(I)\right|\ll_{F,\varepsilon}B(\log
B)^{3/4+\varepsilon},$

where $I$ runs over integral ideals of $\mathbb{Q}(\sqrt{-14})$.

This may be proved following the same reasoning as for Lemma 1.6, the key
information input being the Prime Ideal Theorem for ideal classes.

##  2\. Proof of Theorem 1.2

It is clear that $g_{h}(n)$ vanishes unless $4\mid h$, since each $\alpha$ has
associates $\alpha,i\alpha,-\alpha,-i\alpha$, and
$1+i^{h}+(-1)^{h}+(-i)^{h}=0$ unless $4\mid h$. We therefore assume that
$4\mid h$ henceforth. We now claim that $g_{h}(n)$ is multiplicative, whence
$f_{h}(n)$ is also multiplicative. To establish the claim, suppose that $n=uv$
with coprime factors $u,v$. Then it merely suffices to observe that each
$\alpha$ of norm $n$ can be written as $\alpha=\beta\gamma$ with $N(\beta)=u$
and $N(\gamma)=v$ in exactly four ways, corresponding to the four associate
choices of $\beta=\mathrm{g.c.d.}(\alpha,u)$. (We leave it to the reader to
check this.)

We now call on a general result that gives accurate order of magnitude
estimates for sums of non-negative multiplicative functions. Such bounds go
back to Erdős [5], and were investigated further by Barban and Vehov [1], and
Shiu [14] amongst others. For our application we need a version that applies
to sums of functions over sifted sequences and this has been given by Pollack
[13, Theorem 1.1]. In his notation we take $k=1$, $\beta=\tfrac{1}{4}$ and
$y=x$.

######  Lemma 2.1.

Let $f(n)$ be a multiplicative function satisfying $0\leq f(n)\leq d(n)$ for
all $n\in\mathbb{N}$. Suppose that, for each prime $p\leq x$ the set
$\mathscr{E}_{p}$ is either the empty set or is a non-zero residue class
modulo $p$, and write $\nu(p)=0$ or $\nu(p)=1$ accordingly. Let

$\mathscr{S}=\bigcap_{p\leq x}\mathscr{E}_{p}^{c}.$

Then

$\sum_{\begin{subarray}{c}n\leq x\\\
n\in\mathscr{S}\end{subarray}}f(n)\ll\frac{x}{\log x}\exp\left(\sum_{p\leq
x}\frac{f(p)-\nu(p)}{p}\right).$

The reader should observe that our requirement that $0\leq f(n)\leq d(n)$
implies Pollack’s condition $f\in\mathscr{M}$.

We apply this lemma with $x=N$ and $f=f_{h}$. We take $\mathscr{E}_{p}$ to
consist of the residue class $N$(mod $p$) when $p\leq\sqrt{N}$ does not divide
$N$, and to be the empty set otherwise. Then if $p\geq\sqrt{N}$ is prime we
will have $N-p\in\mathscr{S}$. The lemma then tells us that

$\sum_{\sqrt{N}\leq p\leq N}f_{h}(N-p)\ll\frac{N}{\log N}\exp\left(\sum_{p\leq
N}\frac{f_{h}(p)-\nu(p)}{p}\right).$

Here we note that

$\displaystyle\sum_{p\leq N}p^{-1}\nu(p)$ $\displaystyle\geq$
$\displaystyle\sum_{p\leq\sqrt{N}}p^{-1}-\sum_{p\mid N}p^{-1}$ (2.1)
$\displaystyle\geq$ $\displaystyle\log\log N+O(1)-\sum_{p\leq\omega(N)}p^{-1}$
$\displaystyle\geq$ $\displaystyle\log\log N+O(1)-\sum_{p\leq\log N}p^{-1}$
$\displaystyle=$ $\displaystyle\log\log N-\log\log\log N+O(1).$

so that

$\sum_{\sqrt{N}\leq p\leq N}f_{h}(N-p)\ll\frac{N\log\log N}{(\log
N)^{2}}\exp\left(\sum_{p\leq N}\frac{f_{h}(p)}{p}\right)$

and hence

$\sum_{p\leq N}f_{h}(N-p)\ll\sqrt{N}+\frac{N\log\log N}{(\log
N)^{2}}\exp\left(\sum_{p\leq N}\frac{f_{h}(p)}{p}\right).$ (2.2)

It remains to consider

$\sum_{p\leq N}\frac{f_{h}(p)}{p}.$

Of course $f_{h}(p)=0$ if $p\equiv 3($mod 4$)$, while if $p\equiv 1($mod 4$)$
we may write $p=N(\alpha)$ for some particular $\alpha$, in which case
$f_{h}(p)=2|\cos(h\arg(\alpha))|$. We now use the famous inequality
$3+4\cos\theta+\cos 2\theta\geq 0$, which implies

$2|\cos\theta|\leq\tfrac{3}{2}+\tfrac{1}{2}\cos(2\theta)$

for all real $\theta$, and hence

$f_{h}(p)\leq\tfrac{3}{2}+\tfrac{1}{4}g_{2h}(p),$ (2.3)

for $p\equiv 1($mod 4$)$. Since $f_{h}(p)$ and $g_{2h}(p)$ vanish for $p\equiv
3($mod 4$)$ it follows that

$\sum_{3\leq p\leq
N}\frac{f_{h}(p)}{p}\leq\frac{3}{2}\sum_{\begin{subarray}{c}p\leq N\\\ p\equiv
1(\mathrm{mod}\,4)\end{subarray}}\frac{1}{p}+\frac{1}{4}\sum_{3\leq p\leq
N}\frac{g_{2h}(p)}{p}.$

The first sum on the right is $\tfrac{1}{2}\log\log N+O(1)$, whence

$\sum_{3\leq p\leq N}\frac{f_{h}(p)}{p}\leq\frac{3}{4}\log\log
N+O(1)+\frac{1}{4}\sum_{3\leq p\leq N}\frac{g_{2h}(p)}{p}.$ (2.4)

To estimate the sum on the right we call on the Prime Number Theorem for
primes over $\mathbb{Q}(i)$ with Grössencharacter, in the following form.

######  Lemma 2.2.

There is an absolute constant $c>0$ such that

$\sum_{\begin{subarray}{c}\alpha\in\mathbb{Z}[i]\\\ N(\alpha)\leq
x\end{subarray}}\Lambda(\alpha)\left(\frac{\alpha}{|\alpha|}\right)^{4k}\ll
x\exp\left\\{-c\frac{\log x}{\sqrt{\log x}+\log k}\right\\}(\log xk)^{4}$

uniformly for $x\geq 2$ and $k\in\mathbb{N}$.

This is a special case of Theorem 5.13 in Iwaniec and Kowalski [10], as the
reader may confirm. Using partial summation we may deduce from Lemma 2.2 that
if $N_{0}=\exp\\{(\log 3k)^{2}\\}$ then

$\sum_{\begin{subarray}{c}\alpha\in\mathbb{Z}[i]\\\ N_{0}<N(\alpha)\leq
N\end{subarray}}\frac{\Lambda(\alpha)}{N(\alpha)\log
N(\alpha)}\left(\frac{\alpha}{|\alpha|}\right)^{4k}\ll 1$

uniformly for $N\geq N_{0}$ and $k\in\mathbb{N}$. A trivial bound shows that
we may include terms with $N(\alpha)\leq N_{0}$ at a cost $O(\log\log k)$, and
that we may remove terms in which $N(\alpha)$ is not a prime at a cost $O(1)$.
Taking $4k=2h$ we then deduce that

$\sum_{3\leq p\leq N}\frac{g_{2h}(p)}{p}\ll\log\log h$

whenever $4\mid h$. Substituting this into (2.4) we find that

$\sum_{3\leq p\leq N}\frac{f_{h}(p)}{p}\leq\frac{3}{4}\log\log N+O(\log\log
h).$

Inserting this bound into (2.2) we have

$\sum_{p\leq N}f_{h}(N-p)\ll\sqrt{N}+\frac{N\log\log N}{(\log N)^{5/4}}(\log
h)^{A}$

for some absolute constant $A$. This is sufficient for Theorem 1.2.

##  3\. Proof of Theorem 1.3

In analogy to the argument that leads from Theorem 1.2 to Theorem 1.1 we see
that Lemma 1.4 will suffice for the proof of Theorem 1.3. To establish the
lemma we first note that $t$ is composed entirely of primes $p\equiv 1($mod 4)
whenever $t^{2}\mid x^{2}+y^{2}$ with $\gcd(t,x,y)=1$. Moreover if $p^{e}\mid
t$ with a prime $p=N(\pi)$ over $\mathbb{Z}[i]$, then we must have either
$\pi^{e}\mid x+iy$ or $\pi^{e}\mid x-iy$. Furthermore, if $\pi^{e}\mid x+iy$
with $e\geq 1$ it follows that $x+iy$ must be coprime to $\overline{\pi}$.
Thus $x+iy$ must be divisible by some $\mu$ with $N(\mu)=t$, such that
$\gcd(\mu,\overline{\mu})=1$, and with $\gcd(x+iy,\overline{\mu})=1$. In order
to count each set of associates $i^{n}\mu$ just once we restrict $\mu$ to be
primary; that is to say we require that $\mu\equiv 1($mod $2+2i)$.

We now write $x+iy=\mu\alpha$, so that

$\sum_{\begin{subarray}{c}x^{2}+y^{2}=t^{2}F(u,v)\\\
\gcd(t,x,y)=1\end{subarray}}\left(\frac{x+iy}{|x+iy|}\right)^{4h}=\sum_{\mu}\left(\frac{\mu}{|\mu|}\right)^{4h}\sum_{\begin{subarray}{c}N(\alpha)=F(u,v)\\\
\gcd(\alpha,\overline{\mu})=1\end{subarray}}\left(\frac{\alpha}{|\alpha|}\right)^{4h},$

the sum over $\mu$ being for solutions of $N(\mu)=t$ subject to
$\gcd(\mu,\overline{\mu})=1$ and $\mu\equiv 1($mod $2+2i)$. We therefore set

$f_{k}(n;\mu)=\frac{1}{4}\left|\sum_{\begin{subarray}{c}N(\alpha)=n\\\
\gcd(\alpha,\overline{\mu})=1\end{subarray}}\left(\frac{\alpha}{|\alpha|}\right)^{k}\right|,$

which differs from our previous function $f_{h}(n)$ only in the condition
$\gcd(\alpha,\overline{\mu})=1$. It now follows that

$\sum_{t^{1/2}\max(|u|,|v|)\leq
B^{1/2}}\left|\sum_{\begin{subarray}{c}x^{2}+y^{2}=t^{2}F(u,v)\\\
\gcd(t,x,y)=1\end{subarray}}\left(\frac{x+iy}{|x+iy|}\right)^{4h}\right|\hskip
56.9055pt$ $\hskip 56.9055pt{}\leq
4\sum_{\begin{subarray}{c}\gcd(\mu,\overline{\mu})=1\\\ \mu\equiv
1(\mathrm{mod}\,2+2i)\end{subarray}}\;\;\sum_{\max(|u|,|v|)\leq(B/N(\mu))^{1/2}}f_{4h}(F(u,v);\mu).$
(3.1)

The inner sum above can now be bounded by an Erdős type estimate, a suitable
version of which is given in the next lemma, which is an immediate consequence
of de la Bretèche and Browning [3, Corollary 1].

######  Lemma 3.1.

Let $F(u,v)\in\mathbb{Z}[u,v]$ be an irreducible binary form of degree $d\geq
2$, and let $f(n)$ be a multiplicative function satisfying $0\leq f(n)\leq
d(n)$ for all $n\in\mathbb{N}$. Then

$\sum_{\max(|u|,|v|)\leq X}f(|F(u,v)|)\ll_{F}X^{2}\prod_{d<p\leq
X}\left(1+\frac{\rho(p)(f(p)-1)}{p}\right),$

where

$\rho(p)=\frac{1}{p-1}\\#\\{(u,v)\in(0,p]^{2}:\,p\mid
F(u,v),\,\gcd(u,v,p)=1\\}.$

The reader should note that we have $d_{1}=d_{2}=0$ and $F=G$ in the notation
of [3]. We should also stress that the implied constant in Lemma 3.1 depends
only on $F$, and is uniform over all function $f$ satisfying $0\leq f(n)\leq
d(n)$.

In our situation $\rho(p)$ is the number of zeros modulo $p$ of the polynomial
$F(X,1)$ as soon as $p>|F(1,0)|$. Since $\rho(p)$ and $f_{4h}(*;\mu)$ are
absolutely bounded we see that Lemma 3.1 yields

$\sum_{\max(|u|,|v|)\leq(B/N(\mu))^{1/2}}f_{4h}(F(u,v);\mu)\hskip 85.35826pt$
$\hskip 85.35826pt{}\ll_{F}\frac{B}{N(\mu)}\exp\left\\{\sum_{2<p\leq
B}\frac{\rho(p)(f_{4h}(p;\mu)-1)}{p}\right\\}.$

However

$\sum_{2<p\leq B}\frac{\rho(p)(f_{4h}(p;\mu)-1)}{p}=\sum_{2<p\leq
B}\frac{\rho(p)(f_{4h}(p)-1)}{p}+O\left(\sum_{p\mid
N(\mu)}\frac{1}{p}\right).$

The error term is $O(\log\log\log N(\mu))$, by the argument used for (2.1),
whence

$\sum_{\max(|u|,|v|)\leq(B/N(\mu))^{1/2}}f_{4h}(F(u,v);\mu)\hskip 85.35826pt$
$\hskip 56.9055pt{}\ll_{F}\frac{B(\log\log
N(\mu))^{A}}{N(\mu)}\exp\left\\{\sum_{2<p\leq
B}\frac{\rho(p)(f_{4h}(p)-1)}{p}\right\\}$

for a suitable constant $A$, so that (3.1) yields

$\sum_{t^{1/2}\max(|u|,|v|)\leq
B^{1/2}}\left|\sum_{\begin{subarray}{c}x^{2}+y^{2}=t^{2}F(u,v)\\\
\gcd(t,x,y)=1\end{subarray}}\left(\frac{x+iy}{|x+iy|}\right)^{4h}\right|\hskip
85.35826pt$ $\hskip 28.45274pt{}\ll_{F}B(\log\log B)^{A}(\log
B)\exp\left\\{\sum_{2<p\leq B}\frac{\rho(p)(f_{4h}(p)-1)}{p}\right\\}.$

We now apply the inequality (2.3) to estimate $f_{4h}(p)$ when $p\equiv 1($mod
4), and deduce that

$\sum_{t^{1/2}\max(|u|,|v|)\leq
B^{1/2}}\left|\sum_{\begin{subarray}{c}x^{2}+y^{2}=t^{2}F(u,v)\\\
\gcd(t,x,y)=1\end{subarray}}\left(\frac{x+iy}{|x+iy|}\right)^{4h}\right|\hskip
85.35826pt$ $\hskip 56.9055pt{}\ll_{F}B(\log\log B)^{A}(\log
B)\exp\left\\{-S_{1}+\tfrac{3}{2}S_{2}+\tfrac{1}{4}S_{3}\right\\},$ (3.2)

with

$S_{1}=\sum_{2<p\leq B}\frac{\rho(p)}{p}$
$S_{2}=\sum_{\begin{subarray}{c}2<p\leq B\\\ p\equiv
1(\mathrm{mod}\,4)\end{subarray}}\frac{\rho(p)}{p}$

and

$S_{3}=\sum_{2<p\leq B}\frac{\rho(p)g_{8h}(p)}{p}.$

We observed above that $\rho(p)$ is just the number of solutions of $F(X,1)$
modulo $p$, if $p$ is large enough in terms of $F$. Indeed, if we write
$K=\mathbb{Q}(\theta)$ where $\theta$ is a root of $F(X,1)$, we see from
Dedekind’s Theorem that $\rho(p)$ is the number of first degree prime ideals
$P$ of $K$ above $p$, at least if $p$ is large enough in terms of $F$. It
follows that

$S_{1}=\sum_{2<N(P)\leq B}\frac{1}{N(P)}+O_{F}(1),$

since prime ideals of degree 2 or more contribute $O_{F}(1)$. The Prime Ideal
Theorem then shows that

$S_{1}=\log\log B+O_{F}(1).$ (3.3)

In order to examine $S_{2}$ and $S_{3}$ we will consider the factorization of
rational primes in the field $L=K(i)$. This must be a quadratic extension of
$K$, since if $\mathbb{Q}(i)\subset K$ it would follow that $\theta$ is
quadratic over $\mathbb{Q}(i)$, which is impossible since $F(X,1)$ was assumed
to be irreducible over $\mathbb{Q}(i)$. When $p\equiv 1($mod 4) is a norm
$N(\pi)$ over $\mathbb{Q}(i)$ one sees that each prime ideal $P$ of $K$ lying
above $p$ will split as $(P,\pi)(P,\overline{\pi})$ over $L$, so that
$\rho(p)$ is half the number of first degree prime ideals of $L$ lying over
$p$. On the other hand, there cannot be a first degree prime ideal $P$ of $L$
lying above a rational prime $p\equiv 3($mod 4), since then
$N_{L/\mathbb{Q}(i)}(P)$ would be a first degree prime ideal of
$\mathbb{Q}(i)$ above $p$, which is impossible. It follows that

$S_{2}=\frac{1}{2}\sum_{2<p\leq B}\frac{\rho_{L}(p)}{p},$

where $\rho_{L}(p)$ counts first degree primes of $L$ above $p$. The Prime
Ideal theorem for $L$ then yields

$S_{2}=\tfrac{1}{2}\log\log B+O_{F}(1).$ (3.4)

To handle $S_{3}$ we define a non-trivial Hecke Grössencharacter on prime
ideals of $L$ by setting $\chi_{h}(P)=(\pi/|\pi|)^{4h}$ if
$N_{L/\mathbb{Q}(i)}(P)=(\pi)$. With this definition we will have

$\rho(p)g_{8h}(p)=\sum_{N_{L/\mathbb{Q}}(P)=p}\chi_{2h}(p).$

In analogy with Lemma 2.2 we deduce from the Prime Ideal Theorem with
Grössencharacter that

$\sum_{N(A)\leq x}\Lambda(A)\chi_{h}(A)\ll_{F}x\exp\left\\{-c\frac{\log
x}{\sqrt{\log x}+\log h}\right\\}(\log xh)^{4}$

uniformly for $x\geq 2$ and $k\in\mathbb{N}$, where $A$ runs over non-zero
integral ideals of $L$. Following a similar argument to that used in §2 we now
deduce that

$S_{3}=\sum_{3\leq p\leq N}\frac{\rho(p)g_{8h}(p)}{p}\ll_{F}\log\log h.$

We can now feed this estimate, along with (3.3) and (3.4), into (3.2) to
deduce that

$\sum_{t^{1/2}\max(|u|,|v|)\leq
B^{1/2}}\left|\sum_{\begin{subarray}{c}x^{2}+y^{2}=t^{2}F(u,v)\\\
\gcd(t,x,y)=1\end{subarray}}\left(\frac{x+iy}{|x+iy|}\right)^{4h}\right|\hskip
85.35826pt$ $\hskip 85.35826pt{}\ll_{F}B(\log\log B)^{A}(\log h)^{A}(\log
B)^{3/4},$

for a suitable numerical constant $A$, and Lemma 1.4 follows.

## References

  * [1] M.B. Barban and P.P. Vehov, Summation of multiplicative functions of polynomials. Mat. Zametki, 5 (1969), 669–680.
  * [2] B. M. Bredikhin and Yu. V. Linnik, Asymptotic behaviour and ergodic properties of solutions of the generalized Hardy–Littlewood equation, Mat. Sb. (N.S.), 71 (113) (1966), 145–161.
  * [3] R. de la Bretèche and T.D. Browning, Sums of arithmetic functions over values of binary forms. Acta Arith., 125 (2006), no. 3, 291–304.
  * [4] R. de la Bretèche and G. Tenenbaum, Sur la conjecture de Manin pour certaines surfaces de Châtelet. J. Inst. Math. Jussieu, 12 (2013), no. 4, 759–819.
  * [5] P. Erdős, On the sum $\sum^{x}_{k=1}d(f(k))$. J. London Math. Soc., 27 (1952), 7–15.
  * [6] R. Holowinsky, Sieving for mass equidistribution. Ann. of Math. (2), 172 (2010), no. 2, 1499–1516.
  * [7] C. Hooley, On the representation of a number as the sum of two squares and a prime. Acta Math., 97 (1957), 189–210.
  * [8] C. Hooley, Applications of sieve methods to the theory of numbers. Cambridge Tracts in Mathematics, No. 70. Cambridge University Press, (Cambridge, New York, Melbourne), 1976.
  * [9] C. Hooley, On the representation of a number as the sum of a prime and two squares of square-free numbers. Acta Arith., 182 (2018), no. 3, 201–229.
  * [10] H. Iwaniec and E. Kowalski, Analytic number theory. American Mathematical Society Colloquium Publications, 53. (American Mathematical Society, Providence, RI), 2004.
  * [11] Yu. V. Linnik, All large numbers are sums of a prime and two squares (A problem of Hardy and Littlewood). I. Mat. Sb. (N.S.), 52 (94) (1960), 661–700.
  * [12] Yu. V. Linnik, All large numbers are sums of a prime and two squares (A problem of Hardy and Littlewood). II. Mat. Sb. (N.S.), 53 (95) (1961), 3–38.
  * [13] P. Pollack, Nonnegative multiplicative functions on sifted sets, and the square roots of -1 modulo shifted primes. Glasg. Math. J., 62 (2020), no. 1, 187–199.
  * [14] P. Shiu, A Brun-Titchmarsh theorem for multiplicative functions. J. Reine Angew. Math., 313 (1980), 161–170.


* (48) G. Domènech, S. Pi and M. Sasaki, “Induced gravitational waves as a probe of thermal history of the universe,” JCAP 08, 017 (2020) doi:10.1088/1475-7516/2020/08/017 [arXiv:2005.12314 [gr-qc]]; G. Domènech, “Induced gravitational waves in a general cosmological background,” Int. J. Mod. Phys. D 29, no.03, 2050028 (2020) doi:10.1142/S0218271820500285 [arXiv:1912.05583 [gr-qc]]; L. T. Witkowski, G. Domènech, J. Fumagalli and S. Renaux-Petel, “Expansion history-dependent oscillations in the scalar-induced gravitational wave background,” JCAP 05, no.05, 028 (2022) doi:10.1088/1475-7516/2022/05/028 [arXiv:2110.09480 [astro-ph.CO]]; V. Atal and G. Domènech, “Probing non-Gaussianities with the high frequency tail of induced gravitational waves,” JCAP 06, 001 (2021) doi:10.1088/1475-7516/2021/06/001 [arXiv:2103.01056 [astro-ph.CO]]. for a review see also: G. Domènech, “Scalar Induced Gravitational Waves Review,” Universe 7, no.11, 398 (2021) doi:10.3390/universe7110398 [arXiv:2109.01398 [gr-qc]];
  * (49) K. Inomata, M. Kawasaki, K. Mukaida, Y. Tada and T. T. Yanagida, “Inflationary Primordial Black Holes as All Dark Matter,” Phys. Rev. D 96 (2017) no.4, 043504 doi:10.1103/PhysRevD.96.043504 [arXiv:1701.02544 [astro-ph.CO]].
  * (50) B. J. Carr and S. W. Hawking, “Black holes in the early Universe,” Mon. Not. Roy. Astron. Soc. 168 (1974), 399-415
  * (51) J. C. Niemeyer and K. Jedamzik, “Near-critical gravitational collapse and the initial mass function of primordial black holes,” Phys. Rev. Lett. 80 (1998), 5481-5484 doi:10.1103/PhysRevLett.80.5481 [arXiv:astro-ph/9709072 [astro-ph]]. M. Shibata and M. Sasaki, “Black hole formation in the Friedmann universe: Formulation and computation in numerical relativity,” Phys. Rev. D 60 (1999), 084002 doi:10.1103/PhysRevD.60.084002 [arXiv:gr-qc/9905064 [gr-qc]]. M. Shibata and M. Sasaki, “Black hole formation in the Friedmann universe: Formulation and computation in numerical relativity,” Phys. Rev. D 60 (1999), 084002 doi:10.1103/PhysRevD.60.084002 [arXiv:gr-qc/9905064 [gr-qc]]. I. Musco and J. C. Miller, “Primordial black hole formation in the early universe: critical behaviour and self-similarity,” Class. Quant. Grav. 30 (2013), 145009 doi:10.1088/0264-9381/30/14/145009 [arXiv:1201.2379 [gr-qc]]. T. Harada, C. M. Yoo and K. Kohri, “Threshold of primordial black hole formation,” Phys. Rev. D 88 (2013) no.8, 084051 [erratum: Phys. Rev. D 89 (2014) no.2, 029903] doi:10.1103/PhysRevD.88.084051 [arXiv:1309.4201 [astro-ph.CO]]. C. Germani and I. Musco, “Abundance of Primordial Black Holes Depends on the Shape of the Inflationary Power Spectrum,” Phys. Rev. Lett. 122 (2019) no.14, 141302 doi:10.1103/PhysRevLett.122.141302 [arXiv:1805.04087 [astro-ph.CO]]. A. Escrivà, C. Germani and R. K. Sheth, “Analytical thresholds for black hole formation in general cosmological backgrounds,” JCAP 01 (2021), 030 doi:10.1088/1475-7516/2021/01/030 [arXiv:2007.05564 [gr-qc]]. A. Escrivà, “Simulation of primordial black hole formation using pseudo-spectral methods,” Phys. Dark Univ. 27 (2020), 100466 doi:10.1016/j.dark.2020.100466 [arXiv:1907.13065 [gr-qc]].
  * (52) I. Musco, V. De Luca, G. Franciolini and A. Riotto, “Threshold for primordial black holes. II. A simple analytic prescription,” Phys. Rev. D 103 (2021) no.6, 063538 doi:10.1103/PhysRevD.103.063538 [arXiv:2011.03014 [astro-ph.CO]].
  * (53) B. J. Carr, K. Kohri, Y. Sendouda and J. Yokoyama, “New cosmological constraints on primordial black holes,” Phys. Rev. D 81 (2010), 104019 doi:10.1103/PhysRevD.81.104019 [arXiv:0912.5297 [astro-ph.CO]]. Y. Inoue and A. Kusenko, “New X-ray bound on density of primordial black holes,” JCAP 10 (2017), 034 doi:10.1088/1475-7516/2017/10/034 [arXiv:1705.00791 [astro-ph.CO]]. P. Montero-Camacho, X. Fang, G. Vasquez, M. Silva and C. M. Hirata, “Revisiting constraints on asteroid-mass primordial black holes as dark matter candidates,” JCAP 08 (2019), 031 doi:10.1088/1475-7516/2019/08/031 [arXiv:1906.05950 [astro-ph.CO]]. A. Katz, J. Kopp, S. Sibiryakov and W. Xue, “Femtolensing by Dark Matter Revisited,” JCAP 12 (2018), 005 doi:10.1088/1475-7516/2018/12/005 [arXiv:1807.11495 [astro-ph.CO]]. V. Poulin, P. D. Serpico, F. Calore, S. Clesse and K. Kohri, “CMB bounds on disk-accreting massive primordial black holes,” Phys. Rev. D 96 (2017) no.8, 083524 doi:10.1103/PhysRevD.96.083524 [arXiv:1707.04206 [astro-ph.CO]]. F. Capela, M. Pshirkov and P. Tinyakov, “Constraints on primordial black holes as dark matter candidates from capture by neutron stars,” Phys. Rev. D 87 (2013) no.12, 123524 doi:10.1103/PhysRevD.87.123524 [arXiv:1301.4984 [astro-ph.CO]]. H. Niikura, M. Takada, N. Yasuda, R. H. Lupton, T. Sumi, S. More, T. Kurita, S. Sugiyama, A. More and M. Oguri, et al. “Microlensing constraints on primordial black holes with Subaru/HSC Andromeda observations,” Nature Astron. 3 (2019) no.6, 524-534 doi:10.1038/s41550-019-0723-1 [arXiv:1701.02151 [astro-ph.CO]]. L. Wyrzykowski, J. Skowron, S. Kozlowski, A. Udalski, M. K. Szymanski, M. Kubiak, G. Pietrzynski, I. Soszynski, O. Szewczyk and K. Ulaczyk, et al. “The OGLE View of Microlensing towards the Magellanic Clouds. IV. OGLE-III SMC Data and Final Conclusions on MACHOs,” Mon. Not. Roy. Astron. Soc. 416 (2011), 2949 doi:10.1111/j.1365-2966.2011.19243.x [arXiv:1106.2925 [astro-ph.GA]]. K. Griest, A. M. Cieplak and M. J. Lehner, “New Limits on Primordial Black Hole Dark Matter from an Analysis of Kepler Source Microlensing Data,” Phys. Rev. Lett. 111 (2013) no.18, 181302 doi:10.1103/PhysRevLett.111.181302 P. Tisserand et al. [EROS-2], “Limits on the Macho Content of the Galactic Halo from the EROS-2 Survey of the Magellanic Clouds,” Astron. Astrophys. 469 (2007), 387-404 doi:10.1051/0004-6361:20066017 [arXiv:astro-ph/0607207 [astro-ph]]. Y. Ali-Haïmoud and M. Kamionkowski, “Cosmic microwave background limits on accreting primordial black holes,” Phys. Rev. D 95 (2017) no.4, 043534 doi:10.1103/PhysRevD.95.043534 [arXiv:1612.05644 [astro-ph.CO]]. D. Gaggero, G. Bertone, F. Calore, R. M. T. Connors, M. Lovell, S. Markoff and E. Storm, “Searching for Primordial Black Holes in the radio and X-ray sky,” Phys. Rev. Lett. 118 (2017) no.24, 241101 doi:10.1103/PhysRevLett.118.241101 [arXiv:1612.00457 [astro-ph.HE]]. R. Barbieri and G. F. Giudice, “Upper Bounds on Supersymmetric Particle Masses,” Nucl. Phys. B 306 (1988), 63-76 doi:10.1016/0550-3213(88)90171-X M. Braglia, J. Garcia-Bellido and S. Kuroyanagi, “Testing Primordial Black Holes with multi-band observations of the stochastic gravitational wave background,” JCAP 12 (2021) no.12, 012 doi:10.1088/1475-7516/2021/12/012 [arXiv:2110.07488 [astro-ph.CO]]; M. Y. Khlopov, “Primordial Black Holes,” Res. Astron. Astrophys. 10, 495-528 (2010) doi:10.1088/1674-4527/10/6/001 [arXiv:0801.0116 [astro-ph]]; K. M. Belotsky, V. I. Dokuchaev, Y. N. Eroshenko, E. A. Esipova, M. Y. Khlopov, L. A. Khromykh, A. A. Kirillov, V. V. Nikulin, S. G. Rubin and I. V. Svadkovsky, “Clusters of primordial black holes,” Eur. Phys. J. C 79, no.3, 246 (2019) doi:10.1140/epjc/s10052-019-6741-4 [arXiv:1807.06590 [astro-ph.CO]]; A. Ashoorioon, K. Rezazadeh and A. Rostami, “NANOGrav Signal from the End of Inflation and the LIGO Mass and Heavier Primordial Black Holes,” [arXiv:2202.01131 [astro-ph.CO]]. A. Ashoorioon, A. Rostami and J. T. Firouzjaee, “Examining the end of inflation with primordial black holes mass distribution and gravitational waves,” Phys. Rev. D 103, 123512 (2021) doi:10.1103/PhysRevD.103.123512 [arXiv:2012.02817 [astro-ph.CO]].
  * (54) S. W. Hawking, “Black hole explosions,” Nature 248 (1974), 30-31 doi:10.1038/248030a0
  * (55) A. Arvanitaki, S. Dimopoulos, S. Dubovsky, N. Kaloper and J. March-Russell, “String Axiverse,” Phys. Rev. D 81 (2010), 123530 doi:10.1103/PhysRevD.81.123530 [arXiv:0905.4720 [hep-th]].
  * (56) S. Matarrese, S. Mollerach and M. Bruni, “Second order perturbations of the Einstein-de Sitter universe,” Phys. Rev. D 58 (1998), 043504 doi:10.1103/PhysRevD.58.043504 [arXiv:astro-ph/9707278 [astro-ph]]. V. Acquaviva, N. Bartolo, S. Matarrese and A. Riotto, “Second order cosmological perturbations from inflation,” Nucl. Phys. B 667 (2003), 119-148 doi:10.1016/S0550-3213(03)00550-9 [arXiv:astro-ph/0209156 [astro-ph]]. S. Mollerach, D. Harari and S. Matarrese, “CMB polarization from secondary vector and tensor modes,” Phys. Rev. D 69 (2004), 063002 doi:10.1103/PhysRevD.69.063002 [arXiv:astro-ph/0310711 [astro-ph]].
  * (57) K. N. Ananda, C. Clarkson and D. Wands, “The Cosmological gravitational wave background from primordial density perturbations,” Phys. Rev. D 75 (2007), 123518 doi:10.1103/PhysRevD.75.123518 [arXiv:gr-qc/0612013 [gr-qc]]. D. Baumann, P. J. Steinhardt, K. Takahashi and K. Ichiki, “Gravitational Wave Spectrum Induced by Primordial Scalar Perturbations,” Phys. Rev. D 76 (2007), 084019 doi:10.1103/PhysRevD.76.084019 [arXiv:hep-th/0703290 [hep-th]]. M. Maggiore, “Gravitational wave experiments and early universe cosmology,” Phys. Rept. 331 (2000), 283-367 doi:10.1016/S0370-1573(99)00102-7 [arXiv:gr-qc/9909001 [gr-qc]]. K. Kohri and T. Terada, “Semianalytic calculation of gravitational wave spectrum nonlinearly induced from primordial curvature perturbations,” Phys. Rev. D 97 (2018) no.12, 123532 doi:10.1103/PhysRevD.97.123532 [arXiv:1804.08577 [gr-qc]].
  * (58) J. R. Espinosa, D. Racco and A. Riotto, “A Cosmological Signature of the SM Higgs Instability: Gravitational Waves,” JCAP 09 (2018), 012 doi:10.1088/1475-7516/2018/09/012 [arXiv:1804.07732 [hep-ph]].
  * (59) P. Amaro-Seoane et al. [LISA], “Laser Interferometer Space Antenna,” [arXiv:1702.00786 [astro-ph.IM]].
  * (60) S. Sato, S. Kawamura, M. Ando, T. Nakamura, K. Tsubono, A. Araya, I. Funaki, K. Ioka, N. Kanda and S. Moriwaki, et al. “The status of DECIGO,” J. Phys. Conf. Ser. 840 (2017) no.1, 012010 doi:10.1088/1742-6596/840/1/012010
  * (61) K. Yagi and N. Seto, “Detector configuration of DECIGO/BBO and identification of cosmological neutron-star binaries,” Phys. Rev. D 83 (2011), 044011 [erratum: Phys. Rev. D 95 (2017) no.10, 109901] doi:10.1103/PhysRevD.83.044011 [arXiv:1101.3940 [astro-ph.CO]].
  * (62) B. Carr, K. Dimopoulos, C. Owen and T. Tenkanen, “Primordial Black Hole Formation During Slow Reheating After Inflation,” Phys. Rev. D 97, no.12, 123535 (2018) doi:10.1103/PhysRevD.97.123535 [arXiv:1804.08639 [astro-ph.CO]].
  * (63) N. Bhaumik and R. K. Jain, “Primordial black holes dark matter from inflection point models of inflation and the effects of reheating,” JCAP 01, 037 (2020) doi:10.1088/1475-7516/2020/01/037 [arXiv:1907.04125 [astro-ph.CO]].
  * (64) R. G. Cai, T. B. Liu and S. J. Wang, “Sensitivity of primordial black hole abundance on the reheating phase,” Phys. Rev. D 98, no.4, 043538 (2018) doi:10.1103/PhysRevD.98.043538 [arXiv:1806.05390 [astro-ph.CO]].
  * (65) J. A. S. Lima, S. Basilakos and J. Solà, “Thermodynamical aspects of running vacuum models,” Eur. Phys. J. C 76, no.4, 228 (2016) doi:10.1140/epjc/s10052-016-4060-6 [arXiv:1509.00163 [gr-qc]]; “Nonsingular Decaying Vacuum Cosmology and Entropy Production,” Gen. Rel. Grav. 47, 40 (2015) doi:10.1007/s10714-015-1888-2 [arXiv:1412.5196 [gr-qc]]; J. Solà Peracaula and H. Yu, “Particle and entropy production in the Running Vacuum Universe,” Gen. Rel. Grav. 52, no.2, 17 (2020) doi:10.1007/s10714-020-2657-4 [arXiv:1910.01638 [gr-qc]].
  * (66) E. W. Kolb and M. S. Turner, “The Early Universe,” Front. Phys. 69, 1-547 (1990) doi:10.1201/9780429492860
  * (67) R. Blumenhagen and E. Plauschinn, “Towards Universal Axion Inflation and Reheating in String Theory,” Phys. Lett. B 736, 482-487 (2014) doi:10.1016/j.physletb.2014.08.007 [arXiv:1404.3542 [hep-th]]; for a discussion on reheating in multiaxion string models see, for instance: J. Halverson, C. Long, B. Nelson and G. Salinas, “Axion reheating in the string landscape,” Phys. Rev. D 99, no.8, 086014 (2019) doi:10.1103/PhysRevD.99.086014 [arXiv:1903.04495 [hep-th]].
  * (68) C. Ringeval, “The exact numerical treatment of inflationary models,” Lect. Notes Phys. 738 (2008), 243-273 doi:10.1007/978-3-540-74353-8_7 [arXiv:astro-ph/0703486 [astro-ph]].
  * (69) M. Sasaki, “Large Scale Quantum Fluctuations in the Inflationary Universe,” Prog. Theor. Phys. 76 (1986), 1036 doi:10.1143/PTP.76.1036; V. F. Mukhanov, “Quantum Theory of Gauge Invariant Cosmological Perturbations,” Sov. Phys. JETP 67 (1988), 1297-1302.