# Efficient Document Image Classification Using Region-Based Graph Neural
Network

Jaya Krishna Mandivarapu, Eric Bunch, Qian You, Glenn Fung
American Family Insurance, Machine Learning Research Group
<EMAIL_ADDRESS>, {ebunch, qyou<EMAIL_ADDRESS>

###### Abstract

Document image classification remains a popular research area because it can
be commercialized in many enterprise applications across different industries.
Recent advancements in large pre-trained computer vision and language models
and graph neural networks has lent document image classification many tools.
However using large pre-trained models usually requires substantial computing
resources which could defeat the cost-saving advantages of automatic document
image classification. In the paper we propose an efficient document image
classification framework that uses graph convolution neural networks and
incorporates textual, visual and layout information of the document. Empirical
results on both publicly available and real-world data show that our methods
achieve near SOTA performance yet require much less computing resources and
time for model training and inference. This results in solutions than offer
better cost advantages, especially in scalable deployment for enterprise
applications.

##  1 Introduction

Gartner has estimated 80% of enterprises data is unstructured (emails, PDF and
other documents). These documents contain rich information and knowledge about
internal and external business communication and transactions. And they have
ubiquitous applications in numerous industrial sectors such as finance, health
care, and law etc. Therefore, being able to automatically and efficiently
sort, analyze, and extract structure and content from document images can
improve efficiency and reduce cost for many business workflows. Document image
classification is an import task in these automation solutions, and has been a
popular research area for decades. Early works usually build classifiers that
rely on Optical Character Recognition (OCR) to extract text information, and
employ heuristics to model layout structural features. In light of the
advancement of computer vision and deep learning, VGG-16 (?) pre-trained on
ImageNet (?) reported good classification performance on data sets mixed of
business letters, print advertisement, emails and magazine articles (?). Both
(?) and (?) created document representations by encoding layout coordinates
into positional embeddings as inputs to pre-trained BERT (?) or transformer
architectures. The latest PubLayNet (?) addresses the limited public available
document image data sets by training a Mask R-CNN (?) model on 360k images of
scientific articles, and enables transfer learning to other document domains.
Motivated by the development of graph neural network algorithms (?; ?; ?),
researchers (?) attempted to use graph convolutions to model the interactions
among structural components of a document and between the visual and textual
features, as an alternative to pixel level or token level document modeling.
In contrast to fast moving research progress in document analysis and
classification, few have systematically studied the time and hardware
resources when using different methods and the financial implications of the
model design. However, as document image classifications have been primarily
motivated by its potential in commercialization, it is imperative to study its
model performance with computing resources requirements and financial
implications. In this paper we propose an efficient document image
classification framework as shown in Fig. 1. Semantics regions of a document
is extracted by pre-trained PubLayNet, textual features are extracted by text
embedding models and the image features are extracted by a pre-trained VGG-16
model. Graphs formed for the document, with the document class labels are used
to train a sort pooling graph convolution network (?) which normalizes and
classify arbitrary graphs therefore documents. The major contributions of our
papers are as follows:

  * •

We propose a novel document image classification framework which applies a
graph convolution neural network to a document image graph formed by semantic
regions extracted from a pre-trained document segmentation model. Moreover
both image and text features of the regions are extracted and assigned to the
nodes so that information from both modalities are captured and propagated in
the graph convolutions. To our best knowledge, our framework is the the first
in effectively and economically integrating image, text, and layout
information for document image classification using a graph convolution neural
network.

  * •

We have rigorously bench marked our proposed method against state-of-the-art
pre-trained vision models and transformer language models on document image
data sets. These include an insurance related document image data set
consisted of 11 classes and an open source data set of 10 classes. The results
showed the classification results of our method are comparable to those of
baseline models, if not better.

  * •

We extensively bench marked the computing resources required by all methods.
The results showed our framework needs substantially less computing resources
and less time, further indicating the cost advantages of training, deployment
and hosting at scale. Efficient model also helps accelerate model iterations
and update.

We also discussed a few potential document image classification applications
and the infrastructure to deploy our framework. The potentially large scale
adoption of document image classification further reinforced the need for an
efficient document image classification method.

Figure 1: Eff-GNN Framework overview: textual embedding, segmented regions and
image embeddings of an image are integrated when the graph of document is
formed. Created graph is fed into the Graph Convolution Neural Network for
graph classification as document image classification.

##  2 Related Work

Early document image classification algorithms relied on OCR to extract
content information and exploited the visual structure and layout of a
document image, e.g. using tree-related data structures to model a document
(?; ?; ?). The subsequent decades of research document analysis, including
document image classification, evolves around more sophisticated ways to
leverage the image features, text features and document layout information.

Advancement in Deep Convolutional Neural Networks (DCNN) lend new tools for
document image classification (?), because DCNN could extract salient and
hierarchical visual feature representations which can somewhat reflect
hierarchical nature of document layout. Quite a few DCNN training strategies
for document image classification are proposed and extensively reviewed (?;
?). Variants of VGG-16 (?) achieved the state-of-the art on publicly available
Tobacco data sets (?). Document understanding and analysis community have also
been leveraging word embedding techniques (?) in NLP and large language models
(?) to create contextualized embedding for textual content in an document
image. BERTGrid (?) uses both the contextualized word embeddings and its 2D
layout coordinates to extract information by predicting segmentation masks and
bounding boxes. Assuming syntactic features matter less than content
categories in document classification. DocBERT achieved an economical solution
for document classification task by distilling BERT (?). LayoutLM (?) jointly
models interactions between text and layout information by inputting both text
embeddings along with its 2D layout positional embeddings extracted using OCR
and Region of Interest (ROI) Regressions. Considering both the image and
textual modalities in the document images, multi-modalities methods (?; ?) are
adapted to document classification tasks as well.

##  3 Methodology

In this section, we briefly review the advantages and limitations when using
either CNNs or large language models in document classification. We also
discuss document segmentation and the intuitions of using region based
representations. We then describe our proposed efficient graph neural network,
Eff-GNN

### Deep Convolution Neural Network Learning Approaches

When using deep convolutional neural networks for document image
classification, the document is treated as an image and is ingested as tensor
representing the pixel values of the image. VGG-16 pre-trained on ImageNet (?)
can achieve good results on general business documents (?). Even for the
insurance data set, VGG-16 pre-trained on ImageNet can be a powerful visual
feature extractor.

### Language Model based Approaches

In general, BERT-like pre-trained language models achieve superior performance
on natural language processing tasks by adding self attention mechanism and
positional information to the encoder-decoder architecture. When using BERT-
like models to classify document images, we classify based on the
contextualized embedding of text extracted from image. DocBert (?) assumes
syntax features matter less if only the categories of the document need to be
decided. And DocBert successfully distills trained BERT into a much smaller
LSTM model. This gives us some insight that token level modeling for document
classification may not be necessary.

### Document Segmentation

A business document contains visually salient structural components such as
header, footer, paragraph, table etc. Intuitively one can classify a document
image by its layout and structural components without accessing much of its
content. The regions of structural components are usually pixels or tokens
with similar appearances or groupings. Hence regions are a higher level
abstraction and representation which we can leverage to classify this
document. Research in the computer vision community has provided plenty of
tools of segmenting document images. The latest advancement is PubLayNet which
trained a Mask R-CNN model for 360 thousand document images from scientific
articles. The segmented region results from PubLayNet can be found at Figure 2
(c) & (d).

### Efficient GNN for Document Image Classification

Ideally an effective document classification method need to leverage both
textual, image and layout information. However, training or fine tuning CNNs
or large language models do not only run into resource constraints (e.g. GPUs,
memory ), but also prevent fast model iterations. We attempted to address this
dilemma by graph representations using graph representations to represent
document. We then assign image and text features to the nodes of the graph and
apply a graph convolution neural network. Finally we classify the document as
classifying a graph.

Details of training our proposed Eff-GNN can be found in Algorithm 1. For each
image with class label, we extract its text using OCR PyTesseract and we
extract its semantic regions using PubLayNet. Each image is converted into a
graph where each node is a region. To generate the text feature of the nodes,
we use Word2Vec to create the embeddings for words in the region; to generate
image feature of the nodes, we extract visual features from that region using
VGG-16 pre-trained on ImageNet. Text features and image features of the node
can be concatenated and assigned to the nodes. A graph convolution neural
network classifier with a SortPooling (?) layer is then trained on this data.
We adapted to this specific graph convolution neural network because it
preserve features of individual nodes and also enforces learning from graph
global topology.

Till this end we have integrated textual, image and layout information into a
document classification task using a graph convolution neural network.

Input: $N$ Training documents $\\{(D_{1},y_{1}),\dots,(D_{N},y_{N})\\}$.

Preprocessing

for  _$m=1,\dots,N$_ do

Extract textual information $t_{m}$, detect layout regions $s_{m}$ and visual
features $I_{m}$ present in $D_{m}$

end for

Train word2vec $\mathcal{W}$=$word2vec$(C)  $C=\\{t_{i}\,|\,\mathopen{}1\leq
i\leq N\\}$ for  _$m=1,\dots,N$_ do

Convert each document ($D_{m}$) into graph ($G_{m}$) using $t_{m}$,$s_{m}$,
$I_{m}$ and $\mathcal{W}$ which would be $D_{m}$$\rightarrow$$G_{m}$

end for

end

Training

Training GNN on graph data
$\mathcal{Z}=\left\\{\left(G_{1},y_{1}\right),\left(G_{2},y_{2}\right),\ldots\right\\}$
where $y_{i}\in\mathcal{Y}$ is the label corresponding to graph
$G_{i}\in\mathcal{G},$ with goal of learning a mapping
$f:\mathcal{G}\rightarrow\mathcal{Y}$ that maps graphs to the set of labels

end

Algorithm 1 Efficent Graph Neural Network Training

##  4 Experimental Setup

### Datasets

We use the following two datasets to evaluate our proposed model: Insurance
dataset (Fig.2), Tobacco-3482 dataset(?). Insurance dataset contains 5772
document images which spans across 11 categories and Tobacco-3482 dataset
consists of 3482 images which spans across 10 categories. Categories from the
Tobacco dataset include Advertisements, Emails, Memos, and Scientific Reports.
Categories from the Insurance dataset include Medical Bills, Medical
Authorizations, Medial Records, Police Reports, and Subrogation Letters. See
Fig. 2 for examples of documents from the Insurance dataset.

For the Insurance dataset, we use the splits of 4544 images for train set and
1280 images for test set; for Tobacco-3482 dataset, we use the standard splits
of 2482 images for training, 800 images for testing, and rest 200 images for
validation set.

|  |  |  |
---|---|---|---

(a)

(b)

(c)

(d)

Figure 2: Examples of documents from the Insurance data set. The document
classes (a) Medical Record, (b) Subrogation Letter, (c) Medical Authorization,
and (d) Attorney Correspondence. The two images (c) and (d) include a
visualization of the output of the PubLayNet model.

We also summarized the statistics of graphs that those documents formed in
Table 1. As evidenced in the tables, the graph size of each document is
significantly smaller relative to the number of pixels or the number of words
in a document, which significantly simplifies the subsequent modeling and
computation.

class label |  0 |  1 |  2 |  3 |  4 |  5 |  6 |  7 |  8 |  9 |  10
---|---|---|---|---|---|---|---|---|---|---|---
Insurance |  13 |  5 |  6 |  18 |  5 |  12 |  3 |  11 |  8 |  13 |  11
Tobacco |  5 |  9 |  3 |  11 |  10 |  5 |  2 |  9 |  10 |  2 |  -
Table 1: Median number of nodes per class for the Insurance and Tobacco data
sets.

### Document Pre-processing

To construct a graph for each document image we need to utilize the different
layout regions’ information such as paragraphs, title, list, table present in
each document. We process the scanned document images using pre-trained
PubLayNet to obtain the necessary bounding boxes for each region. To extract
the textual information present in the bounding box of each region in the
document images we apply PyTesseract, an open-source python OCR package. All
the results of the PubLayNet and PyTesseract were then serialized and stored
in tensor format using PyTorch. (?).

### Hyper parameters and infrastructure

We use Hedwig111https://github.com/castorini/hedwig, an open-source deep
learning toolkit with a number of implemented of document classification
models. We use a Tesla K80 GPU for all models requiring GPU for train, and use
amazon EC2-t2.micro and EC2-c type machine when only CPU is needed . We use
PyTorch 1.5 as the backend framework, and gensim (?) package for computing the
node feature vectors using word2vec.

##  5 Results and Discussions

### Comparing classification accuracy

We compare our proposed approach with the state-of-the-art deep learning
models, VGG-16 pre-trained on ImageNet and $\text{BERT}_{base}$ pre-trained on
Wikipedia. When using $\text{BERT}_{base}$ for classification, we extracted
tokens from document images as input and fine tune $\text{BERT}_{base}$ with
class labels. When using pre-trained VGG-16, we used it directly to extract
document image features for classification. No further fine-tuning is used. We
also compared our model with DocBERT which is specially designed for document
level classification by simplifying BERT models with knowledge distillation.

Table 2 shows the model comparison results of classification AUC on the two
data sets. In the insurance data set, our proposed model achieves 90.7% to
91.0 %, very competitive as compared to models in BERT families (91.95 %) and
VGG-16 (90.6 %). In the Tobacco-3482 data set, our models achieved 73.5 % to
77.5 % , comparable to models in BERT families (82.3 %) and VGG-16 (81.5 %).
The fact our proposed model shows more advantages when classifying the
insurance data set could be due to the high intra-class variance and low
inter-class variance in the Tobacco-3482 data set (?).

We also experimented with combining text and image embedding features as node
features in the graph neural network. The combination does provides ample AUC
improvements in our proposed method on Tobacco-3482 dataset i.e. 73.5 % for
Eff-GNN + Word2Vec and for 77.5 % for Eff-GNN + Word2Vec + Image Embedding. In
the insurance data set, Eff-GNN + Word2Vec + Image Embedding shows little
improvement over using Word2Vec text features alone. That could be due to the
fact that both the textual and image content in the insurance data set
provides enough information for classification. This assumption can be further
justified by the similar classification results achieved by models in BERT
family and VGG-16.

In addition to classification performance, we compare the number of trainable
parameters of each model. In both the insurance data set and Tobacco-3482, our
model size is drastically smaller than models in BERT families and VGG-16. We
calculate the parameters of our model as the sum of parameters in the graph
neural network and the parameters in trained word2vec. The small sizes of
graph neural network model (Table 1) results in only 160,000 parameters. The
Word2Vec model is also relatively light weight because each of the data set
contains a very limited vocabulary. Note we did not include the 44.2 million
parameters of PubLayNet, because in our framework we do not train or fine-tune
any parameters of the PubLayNet.

### Comparing computing resources

We also bench marked the time and memory required for training our proposed
Eff-GNN against other models. Table 3 reports the statistics on the insurance
data set, 4544 images for training and 1280 images for inference. Eff-GNN
models take less than 5 minutes to train 50 epochs whereas VGG-16 or models in
BERT Family take hours to train less number of epochs (15 epochs and 28 epochs
respectively). In particular Eff-GNN can run on CPU alone and its model
training time is comparable to its GPU counterpart. This is consistent with
the small size of our model (See Table 2, ”Parameters” column). Eff-GNN can
achieve these advantages because it models the documents using a graph formed
by regions extracted by PubLayNet. The time of using PubLayNet to extract
regions for training images are negligible. The size of the resulting graph
leads to a small model compared to deep models trained on pixel level
information or transformers trained on token level information. Therefore Eff-
GNN only uses 470MB in GPU memory with additional 3.5 GB for using PubLayNet.
Consequently Eff-GNN requires drastically less time for the inference of 1280
images (0.79 seconds) as compared VGG-16 (103 seconds) and BERT (40 seconds).
Note the time of document pre-porcessing steps such as OCR and training
Word2Vec model are not included in the table. Although these two steps are
extra for our proposed framework, we contend that their addition does not
nullify the efficiencies gained through graph neural nets. Even BERT based
models require OCR extraction pre-processing step. Just one Word2Vec needs to
be trained for the entire data sets and OCR can be optimized by e.g. parallel
processing.

Compared with the SOTA pre-trained large models, our proposed Eff-GNN
framework achieved competitive classification results on our insurance
document image data sets, and achieved comparable results on the the open
source Tobacco-3482 data set. We also showed that combining text and image
information as the node features in our graph neural network can be
advantageous when OCR fails to extract text information or when the two
modalities are complimentary. Our proposed method models document
representations using extracted semantic regions, instead of using token level
or pixel level information. Therefore our model size is dramatically less than
other methods, and can be run on CPU machines.

Data Set |  Model | AUC | # Parameters
---|---|---|---
Insurance | DocBert (?) | 91.95 % | 110M
| BERT (?) | 91.95 % | 110M
| VGG-16 (?) | 90.6 % | 130M
| Eff-GNN + Word2Vec (?) | 91.0 % | 124k + 610k
| Eff-GNN + Word2Vec (?) + Image Embedding | 91.0 % | 126k + 610k
Tobacco-3482 | VGG-16 (?) | 81.5% | 130M
| DocBERT (?) | 82.3 % | 110M
| BERT (?) | 79.0 % | 110M
| Eff-GNN+ Word2Vec (?) | 73.5 % | 124k + 610k
| Eff-GNN + Word2Vec (?) + Image Embedding | 77.5 % | 126k + 610k
Table 2: Classification accuracy on the Insurance, Tobacco-3482 dataset. Insurance | Batch Size | Epochs | Training Time | GPU Memory (Training) | Inference Time
---|---|---|---|---|---
VGG | 32 | 28 | 2.30 hours |  7.08GB | 103 seconds
Eff-GNN (GPU) | 32 | 50 | 3.5 mins. |

470MB + 3.5 GB |  0.79 seconds
Eff-GNN (CPU) | 32 | 50 |  4.1 mins. | NA |  0.79 seconds
BERT | 16 | 15 | 6.2 hours | 10.5GB | 40 seconds
DocBERT | 16 | 15 | 6.3 hours | 8.1GB |
”40 times faster
than BERT” (?)
Table 3: Memory, hardware and time required by different models on the
Insurance dataset. The numbers are reported for training 4544 images and
inference for 1280 Images.

##  6 Applications and Deployment

Document image classification can find many enterprise level applications in
insurance companies to reduce manual review and stream line claims processes.
For example, casualty injuries claims usually have multiple correspondences
with hospitals and clinics, each involving documents to be processed by the
insurance company. Personal automobile claims processing can use automatic
document classification of letters of guarantee, purchase receipts, and others
in order to auto-approve a certain percentage of auto claim reimbursements.
Each year, millions of claim related document images sent to insurance
companies belonging to hundreds of categorizes related to financial
institutions, medical providers, or legal organizations. A scalable document
classifier can assign the correct categories as meta data to a document, which
can be used to route to appropriate downstream tasks. Given the scale document
image classification can be deployed in an enterprise, an efficient framework
for model training and iterations, together with an economical model hosting
solutions has clear cost advantages. To support all those applications and
achieve potential scalability , we are in the progress of deploying this
trained model as an API under Container as a Service (CaaS) on AWS using
Fargate managed Elastic Container Service (ECS). The efficient model size
facilitates faster deployment and ease of maintenance. A standard ECS task
with v4CPUs and 30G RAM can be used and the weights of the trained model can
be conveniently packaged in a dockerized image and deployed at scale.

##  7 Conclusion and Future Work

Millions of business document images, such as medical bills, attorney letters,
contracts, bank statements and personal checks are processed in insurance
companies to support a wide range of business workflows and applications. A
scalable and efficient automation that is more intelligent than the brittle
OCR-template based method is desired.

In this paper we proposed a novel document image classification that uses
graph convolution neural network to integrate text, image, and layout
information of a document. We rigorously bench marked our method against the
SOTA computer vision and language models on both the insurance dataset and
Tobacco dataset. We also compared computing time and hardware resources
required for training those models. The results showed our method is not only
competitive on classification performance but also is much smaller in size
therefore requires much less time and resource. This could translate to big
cost advantages of hosting and deployment in real world applications. We are
also working on enabling general document classification that can handle
hundreds of document classes. A few options include training larger models for
domain specific transfer learning, enabling few shot learning and continual
learning when dynamically adding new document classes. In addition, we would
like to further explore more effective document representations including more
sophisticated graph representations or jointly trained layout (?).

## References

  * [Adhikari et al. 2019] Adhikari, A.; Ram, A.; Tang, R.; and Lin, J.  2019\.  Docbert: BERT for document classification.  CoRR abs/1904.08398.
  * [Afzal et al. 2017] Afzal, M. Z.; Kölsch, A.; Ahmed, S.; and Liwicki, M.  2017\.  Cutting the error by half: Investigation of very deep CNN and advanced training strategies for document image classification.  CoRR abs/1704.03557.
  * [Audebert et al. 2019] Audebert, N.; Herold, C.; Slimani, K.; and Vidal, C.  2019\.  Multimodal deep networks for text and image-based document classification.  In Cellier, P., and Driessens, K., eds., Machine Learning and Knowledge Discovery in Databases,International Workshops of ECML PKDD 2019, Wurzburg, Germany, September 16-20, 2019, volume 1167 of Communications in Computer and Information Science, 427–443.  Springer.
  * [Das, Roy, and Bhattacharya 2018] Das, A.; Roy, S.; and Bhattacharya, U.  2018\.  Document image classification with intra-domain transfer learning and stacked generalization of deep convolutional neural networks.  CoRR abs/1801.09321.
  * [Deng et al. 2009] Deng, J.; Dong, W.; Socher, R.; Li, L.; Li, K.; and Li, F.  2009\.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA, 248–255.  IEEE Computer Society.
  * [Dengel 1993] Dengel, A.  1993\.  Initial learning of document structure.  In 2nd International Conference Document Analysis and Recognition, ICDAR ’93, October 20-22, 1993, Tsukuba City, Japan, 86–90.  IEEE Computer Society.
  * [Denk and Reisswig 2019] Denk, T. I., and Reisswig, C.  2019\.  Bertgrid: Contextualized embedding for 2d document representation and understanding.  CoRR abs/1909.04948.
  * [Devlin et al. 2018] Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K.  2018\.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805.
  * [Diligenti, Frasconi, and Gori 2003] Diligenti, M.; Frasconi, P.; and Gori, M.  2003\.  Hidden tree markov models for document image classification.  IEEE Trans. Pattern Anal. Mach. Intell. 25(4):519–523.
  * [He et al. 2017] He, K.; Gkioxari, G.; Dollár, P.; and Girshick, R. B.  2017\.  Mask R-CNN.  CoRR abs/1703.06870.
  * [Kang et al. ] Kang, L.; Kumar, J.; Ye, P.; Li, Y.; and Doermann, D. S.  Convolutional neural networks for document image classification.  In 22nd International Conference on Pattern Recognition, ICPR 2014, Stockholm, Sweden, August 24-28, 2014.
  * [Kölsch et al. 2017] Kölsch, A.; Afzal, M. Z.; Ebbecke, M.; and Liwicki, M.  2017\.  Real-time document image classification using deep CNN and extreme learning machines.  In 14th IAPR International Conference on Document Analysis and Recognition, ICDAR 2017, Kyoto, Japan, November 9-15, 2017, 1318–1323.  IEEE.
  * [Kumar, Ye, and Doermann 2014] Kumar, J.; Ye, P.; and Doermann, D.  2014\.  Structural similarity for document image classification and retrieval.  Pattern Recognition Letters 43:119–126.
  * [Lewis et al. 2006] Lewis, D. D.; Agam, G.; Argamon, S.; Frieder, O.; Grossman, D. A.; and Heard, J.  2006\.  Building a test collection for complex document information processing.  In Efthimiadis, E. N.; Dumais, S. T.; Hawking, D.; and Järvelin, K., eds., Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, Washington, USA, August 6-11, 2006, 665–666.  ACM.
  * [Liu et al. 2019] Liu, X.; Gao, F.; Zhang, Q.; and Zhao, H.  2019\.  Graph convolution for multimodal information extraction from visually rich documents.  In Loukina, A.; Morales, M.; and Kumar, R., eds., Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, 32–39.  Association for Computational Linguistics.
  * [Mikolov et al. 2013] Mikolov, T.; Chen, K.; Corrado, G.; and Dean, J.  2013\.  Efficient estimation of word representations in vector space.  arXiv preprint arXiv:1301.3781.
  * [Paszke et al. 2019] Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; and Lin.  2019\.  Pytorch: An imperative style, high-performance deep learning library.  In Wallach, H.; Larochelle, H.; Beygelzimer, A.; d'Alché-Buc, F.; Fox, E.; and Garnett, R., eds., Advances in Neural Information Processing Systems 32. Curran Associates, Inc.  8024–8035.
  * [Řehůřek and Sojka 2010] Řehůřek, R., and Sojka, P.  2010\.  Software Framework for Topic Modelling with Large Corpora.  In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, 45–50.  Valletta, Malta: ELRA.  http://is.muni.cz/publication/884893/en.
  * [Shin, Doermann, and Rosenfeld 2001] Shin, C. K.; Doermann, D. S.; and Rosenfeld, A.  2001\.  Classification of document pages using structure-based features.  Int. J. Document Anal. Recognit. 3(4):232–247.
  * [Simonyan and Zisserman 2014] Simonyan, K., and Zisserman, A.  2014\.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556.
  * [Wu et al. 2019] Wu, Z.; Pan, S.; Chen, F.; Long, G.; Zhang, C.; and Yu, P. S.  2019\.  A comprehensive survey on graph neural networks.  CoRR abs/1901.00596.
  * [Xu et al. 2019] Xu, Y.; Li, M.; Cui, L.; Huang, S.; Wei, F.; and Zhou, M.  2019\.  Layoutlm: Pre-training of text and layout for document image understanding.  CoRR abs/1912.13318.
  * [Yang et al. 2017] Yang, X.; Yümer, M. E.; Asente, P.; Kraley, M.; Kifer, D.; and Giles, C. L.  2017\.  Learning to extract semantic structure from documents using multimodal fully convolutional neural network.  CoRR abs/1706.02337.
  * [Zhang et al. 2018] Zhang, M.; Cui, Z.; Neumann, M.; and Chen, Y.  2018\.  An end-to-end deep learning architecture for graph classification.  In McIlraith, S. A., and Weinberger, K. Q., eds., Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, 4438–4445.  AAAI Press.
  * [Zhong, Tang, and Jimeno-Yepes 2019] Zhong, X.; Tang, J.; and Jimeno-Yepes, A.  2019\.  Publaynet: Largest dataset ever for document layout analysis.  In 2019 International Conference on Document Analysis and Recognition, ICDAR 2019, Sydney, Australia, September 20-25, 2019, 1015–1022.  IEEE.


# Conditional lower bounds on the distribution of central values in families
of $L$-functions

Maksym Radziwiłł and Kannan Soundararajan  Department of Mathematics
University of Texas at Austin
2515 Speedway, Stop C1200
Austin
TX 78712<EMAIL_ADDRESS>Department of Mathematics
Stanford University
450 Jane Stanford Way, Bldg. 380
Stanford
CA 94305-2125<EMAIL_ADDRESS>To Henryk Iwaniec, with admiration

###### Abstract.

We establish a general principle that any lower bound on the non-vanishing of
central $L$-values obtained through studying the one-level density of low-
lying zeros can be refined to show that most such $L$-values have the typical
size conjectured by Keating and Snaith. We illustrate this technique in the
case of quadratic twists of a given elliptic curve, and similar results would
hold for the many examples studied by Iwaniec, Luo, and Sarnak in their
pioneering work on $1$-level densities [5].

The first author was partially supported by DMS-1902063. The second author is
partially supported by an NSF grant, and a Simons Investigator award from the
Simons Foundation

##  1\. Introduction

Selberg [11, 12] (see [8] for a recent treatment) established that if $t$ is
chosen uniformly from $[0,T]$ then the values $\log|\zeta(\tfrac{1}{2}+it)|$
are distributed approximately like a Gaussian random variable with mean $0$
and variance $\tfrac{1}{2}\log\log T$. More recently, Keating and Snaith [6]
have conjectured that central values in families of $L$-functions have an
analogous log-normal distribution with a prescribed mean and variance
depending on the “symmetry type” of the family. This is a powerful conjecture
which gives more precise versions of conjectures on the non-vanishing of
$L$-values; for example, it refines Goldfeld’s conjecture (towards which
remarkable progress has been made with the work of Smith [13]) that the rank
in families of quadratic twists of an elliptic curve is $0$ for almost all
twists with even sign of the functional equation. In [7] we enunciated a
general principle which shows the upper bound (in a sense to be made precise
below) part of the Keating–Saith conjecture in any family where somewhat more
than the first moment can be computed. In this paper, we consider the
complementary problem of obtaining lower bounds in the Keating-Saith
conjecture, which is intimately tied up with questions on the non-vanishing of
$L$-values. One analytic approach, conditional on the Generalized Riemann
Hypothesis, towards such non-vanishing results is based on computing the
$1$-level density for low lying zeros in families of $L$-functions, and our
goal in this paper is to show how this approach (in the situations where it
succeeds in producing a positive proportion of non-vanishing) may be refined
to give corresponding lower bounds towards the Keating–Snaith conjectures. In
a later paper, we shall consider similar refinements of the mollifier method,
which is another analytic approach that in many cases establishes non-
vanishing results unconditionally. Algebraic approaches such as Smith’s work
[13] on Goldfeld’s conjecture are capable of establishing definitive non-
vanishing results (or, for other examples, see Rohrlich [9, 10] and Chinta
[2]), but we are unable to refine these methods to show that the non-zero
values that are produced in fact have the typical size predicted by the
Keating-Snaith conjectures.

To illustrate our method, we treat the family of quadratic twists of an
elliptic curve $E$ defined over ${\mathbb{Q}}$ with conductor $N$, where the
$1$-level density of low lying zeros has been studied by many authors, notably
Heath-Brown [3]. Let the associated $L$-function be

$L(s,E)=\sum_{n=1}^{\infty}a(n)n^{-s},$

where the coefficients $a(n)$ are normalized such that $|a(n)|\leq d(n)$.
Since elliptic curves are known to be modular, $L(s,E)$ has an analytic
continuation to the entire complex plane and satisfies the functional equation

$\Lambda(s,E)=\epsilon_{E}\Lambda(1-s,E),$

where $\epsilon_{E}$, the root number, is $\pm 1$ and

$\Lambda(s,E)=\Big{(}\frac{\sqrt{N}}{2\pi}\Big{)}^{s}\Gamma(s+\tfrac{1}{2})L(s,E).$

Throughout the paper, let $d$ denote a fundamental discriminant coprime to
$2N$, and let $\chi_{d}=(\frac{d}{\cdot})$ denote the associated primitive
quadratic character. Let $E_{d}$ denote the quadratic twist of $E$ by $d$, and
let its associated $L$-function be

$L(s,E_{d})=\sum_{n=1}^{\infty}a(n)\chi_{d}(n)n^{-s}.$

If $(d,N)=1$ then $E_{d}$ has conductor $Nd^{2}$ and the completed
$L$-function

$\Lambda(s,E_{d})=\Big{(}\frac{\sqrt{N}|d|}{2\pi}\Big{)}^{s}\Gamma(s+\tfrac{1}{2})L(s,E_{d})$

is entire and satisfies the functional equation

$\Lambda(s,E_{d})=\epsilon_{E}(d)\Lambda(1-s,E_{d})$

with

$\epsilon_{E}(d)=\epsilon_{E}\chi_{d}(-N).$

Note that, by Waldspurger’s theorem, $L(\tfrac{1}{2},E_{d})\geq 0$. Of course
$L(\tfrac{1}{2},E_{d})=0$ when $\epsilon_{E}(d)=-1$, and in this paper, we
shall restrict attention to those twists with root number $1$. Put therefore

${\mathcal{E}}=\\{d:\ \ d\text{ a fundamental discriminant with
}(d,2N)=1\text{ and }\epsilon_{E}(d)=1\\}.$

The Keating-Snaith conjectures predict that for $d\in{\mathcal{E}}$, the
quantity $\log L(\tfrac{1}{2},E_{d})$ has an approximately normal distribution
with mean $-\tfrac{1}{2}\log\log|d|$ and variance $\log\log|d|$. To state this
precisely, let $\alpha<\beta$ be real numbers, and for any $X\geq 20$, let us
define

(1) ${\mathcal{N}}(X;\alpha,\beta)=\Big{|}\Big{\\{}d\in{\mathcal{E}},X<|d|\leq
2X:\ \frac{\log
L(\tfrac{1}{2},E_{d})+\tfrac{1}{2}\log\log|d|}{\sqrt{\log\log|d|}}\in(\alpha,\beta)\Big{\\}}\Big{|}.$

Then the Keating-Snaith conjecture states that, for fixed intervals
$(\alpha,\beta)$ and as $X\to\infty$,

(2) ${\mathcal{N}}(X;\alpha,\beta)=|\\{d\in{\mathcal{E}},X\leq|d|\leq
2X\\}|\Big{(}\frac{1}{\sqrt{2\pi}}\int_{\alpha}^{\beta}e^{-\frac{x^{2}}{2}}dx+o(1)\Big{)}.$

Here we interpret $\log L(\tfrac{1}{2},E_{d})$ to be negative infinity if
$L(\tfrac{1}{2},E_{d})=0$, and the conjecture implies in particular that
$L(\tfrac{1}{2},E_{d})\neq 0$ for almost all $d\in{\mathcal{E}}$. Towards this
conjecture, we established in [7] that ${\mathcal{N}}(X;\alpha,\infty)$ is
bounded above by the right hand side of the conjectured relation (2).
Complementing this, we now establish a conditional lower bound for
${\mathcal{N}}(X;\alpha,\beta)$.

######  Theorem 1.

Assume the Generalized Riemann Hypothesis for the family of twisted
$L$-functions $L(s,E\times\chi)$ for all Dirichlet characters $\chi$. Then for
fixed intervals $(\alpha,\beta)$ and as $X\to\infty$ we have

${\mathcal{N}}(X;\alpha,\beta)\geq|\\{d\in{\mathcal{E}},X\leq|d|\leq
2X\\}|\Big{(}\frac{1}{4}\frac{1}{\sqrt{2\pi}}\int_{\alpha}^{\beta}e^{-\frac{x^{2}}{2}}dx+o(1)\Big{)}.$

Above we have assumed GRH for all character twists of $L(s,E)$; this is
largely for convenience, and would allow us to restrict $d$ in progressions.
With more effort one could relax the assumption to GRH for the family of
quadratic twists $L(s,E_{d})$. Note that the factor $\frac{1}{4}$ in our
theorem matches the proportion of quadratic twists with non-zero $L$-value
obtained in Heath-Brown’s work [3].

While we have described results for the family of quadratic twists of an
elliptic curve, the method is very general and applies to many situations
where $1$-level densities of low lying zeros in families have been analyzed
and yield a positive proportion of non-vanishing for the central values. The
work of Iwaniec, Luo, and Sarnak [5] gives many such examples, and the
technique described here refines their non-vanishing corollaries, showing that
the non-zero $L$-values that are produced have the typical size conjectured by
Keating and Snaith. For instance, consider the family of symmetric square
$L$-functions $L(s,\text{sym}^{2}f)$ where $f$ ranges over Hecke eigenforms of
weight $k$ for the full modular group (denote the set of such eigenforms by
$H_{k}$), with $k\leq K$ (thus there are about $K^{2}/48$ such $L$-values).
Assuming GRH in this family, Iwaniec, Luo, and Sarnak (see Corollary 1.8 of
[5]), showed that at least a proportion $\frac{8}{9}$ of these $L$-values are
non-zero. We may refine this to say that for any fixed interval
$(\alpha,\beta)$ and as $K\to\infty$

$\Big{|}\\{f\in H_{k},\ \ k\leq K:\frac{\log
L(\tfrac{1}{2},\text{sym}^{2}f)-\frac{1}{2}\log\log k}{\sqrt{\log\log
k}}\in(\alpha,\beta)\Big{\\}}\Big{|}\geq\Big{(}\frac{8}{9}\frac{1}{\sqrt{2\pi}}\int_{\alpha}^{\beta}e^{-x^{2}/2}dx+o(1)\Big{)}\frac{K^{2}}{48}.$

We end the introduction by mentioning the recent work of Bui, Evans, Lester,
and Pratt [1] who establish “weighted” (where the weight is a mollified
central value) analogues of the Keating-Snaith conjecture. This amounts to a
form of conditioning on non-zero value since central values that are zero are
assigned a weight equal to zero. The use of such a weighted measure allows [1]
to establish a full asymptotic, however as a side effect they have little
control over the nature of the weight.

Acknowledgments. We are grateful to Emmanuel Kowalski for a careful reading of
the paper, and helpful comments. The first author was partially supported by
DMS-1902063. The second author is partially supported by an NSF grant, and a
Simons Investigator award from the Simons Foundation. The paper was completed
while KS was a Senior Fellow at the Institute for Theoretical Studies, ETH
Zürich, whom he thanks for their excellent working conditions, and warm
hospitality.

##  2\. Notation and statements of the key propositions

We begin by introducing some notation, as in our paper [7], and then
describing three key propositions which underlie the proof of the main
theorem. Let $N_{0}$ denote the lcm of $8$ and $N$. Let $\kappa$ be $\pm 1$,
and let $a\bmod{N_{0}}$ denote a residue class with $a\equiv 1$ or $5\bmod 8$.
We assume that $\kappa$ and $a$ are such that for any fundamental discriminant
$d$ with sign $\kappa$ and with $d\equiv a\bmod{N_{0}}$, the root number
$\epsilon_{E}(d)=\epsilon_{E}\chi_{d}(-N)$ equals $1$. Define

${\mathcal{E}}(\kappa,a)=\\{d\in{\mathcal{E}}:\ \ \kappa d>0,\ \ d\equiv
a\bmod{N_{0}}\\},$

so that ${\mathcal{E}}$ is the union of all such sets
${\mathcal{E}}(\kappa,a)$.

We write below

$-\frac{L^{\prime}}{L}(s,E)=\sum_{n=1}^{\infty}\frac{\Lambda_{E}(n)}{n^{s}},$

where $|\Lambda_{E}(n)|\leq 2\Lambda(n)$ so that $\Lambda_{E}(n)=0$ unless
$n=p^{k}$ is a prime power. If $p\nmid N_{0}$, we may write
$a(p)=\alpha_{p}+\overline{\alpha_{p}}$ for a complex number $\alpha_{p}$ of
magnitude $1$ (unique up to complex conjugation), and then

$\Lambda_{E}(p^{k})=(\alpha_{p}^{k}+\overline{\alpha_{p}}^{k})\log p.$

Note that

$-\frac{L^{\prime}}{L}(s,E_{d})=\sum_{n=1}^{\infty}\frac{\Lambda_{E}(n)}{n^{s}}\chi_{d}(n).$

For fundamental discriminants $d\in{\mathcal{E}}$ with $|d|\leq 3X$, and a
parameter $3\leq x$ define

(3) ${\mathcal{P}}(d;x)=\sum_{\begin{subarray}{c}p\leq x\\\ p\nmid
N_{0}\end{subarray}}\frac{a(p)}{\sqrt{p}}\chi_{d}(p).$

Let $h$ denote a smooth function with compactly supported Fourier transform

${\widehat{h}}(\xi)=\int_{-\infty}^{\infty}h(t)e^{-2\pi i\xi t}dt,$

and such that $|h(x)|\ll(1+x^{2})^{-1}$ for all $x\in{\mathbb{R}}$. For
concreteness, one could simply consider $h$ to be the Fejer kernel given by

(4) $h(x)=\Big{(}\frac{\sin(\pi x)}{\pi
x}\Big{)}^{2},\qquad{\widehat{h}}(t)=\max(1-|t|,0).$

Lastly, let $\Phi$ denote a smooth, non-negative function compactly supported
in $[\frac{1}{2},\frac{5}{2}]$ with $\Phi(x)=1$ for $x\in[1,2]$, and we put
${\check{\Phi}}(s)=\int_{0}^{\infty}\Phi(x)x^{s}dx$. Below all implied
constants will be allowed to depend on $N$, $h$, and $\Phi$, which are
considered fixed.

Our first proposition connects $\log L(\frac{1}{2},E_{d})$ with the sum over
primes ${\mathcal{P}}(d;x)$ (for suitable $x$) with an error term given in
terms of the zeros of $L(s,E_{d})$. Such formulae have a long history, going
back to Selberg, and the work here complements an upper bound version that
played a key role in [14].

######  Proposition 1.

Let $d$ be a fundamental discriminant in ${\mathcal{E}}$, and let $3\leq
x\leq|d|$. Assume GRH for $L(s,E_{d})$, and suppose that
$L(\tfrac{1}{2},E_{d})$ is not zero. Let $\gamma_{d}$ run over the ordinates
of the non-trivial zeros of $L(s,E_{d})$. Then

$\log L(\tfrac{1}{2},E_{d})={\mathcal{P}}(d;x)-\tfrac{1}{2}\log\log
x+O\Big{(}\frac{\log|d|}{\log
x}+\sum_{\gamma_{d}}\log\Big{(}1+\frac{1}{(\gamma_{d}\log
x)^{2}}\Big{)}\Big{)}.$

To analyze sums over the zeros we shall use the following proposition, whose
proof is based on the explicit formula. The ideas behind this proposition are
also familiar, and in this setting (and in the case $\ell=1$ below) may be
traced back to the work of Heath-Brown [3].

######  Proposition 2.

Let $h$ be a smooth function with $h(x)\ll(1+x^{2})^{-1}$ and whose Fourier
transform is compactly supported in $[-1,1]$. Let $L\geq 1$ be a real number,
and $\ell$ be a positive integer coprime to $N_{0}$, and assume that
$e^{L}\ell^{2}\leq X^{2}$. If $\ell$ is neither a square, nor a prime times a
square, then

(5)
$\sum_{d\in{\mathcal{E}}(\kappa,a)}\Big{(}\sum_{\gamma_{d}}h\Big{(}\frac{\gamma_{d}L}{2\pi}\Big{)}\Big{)}\chi_{d}(\ell)\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}\ll X^{\frac{1}{2}+\epsilon}\ell^{\frac{1}{2}}e^{\frac{L}{4}}.$

If $\ell$ is a square then

$\displaystyle\sum_{d\in{\mathcal{E}}(\kappa,a)}\Big{(}\sum_{\gamma_{d}}h\Big{(}\frac{\gamma_{d}L}{2\pi}\Big{)}\Big{)}$
$\displaystyle\chi_{d}(\ell)\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}=O(X^{\frac{1}{2}+\epsilon}\ell^{\frac{1}{2}}e^{\frac{L}{4}})$ (6)
$\displaystyle+\frac{X}{N_{0}}\prod_{p|\ell}\Big{(}1+\frac{1}{p}\Big{)}^{-1}\prod_{p\nmid
N_{0}}\Big{(}1-\frac{1}{p^{2}}\Big{)}{\widehat{\Phi}}(0)\Big{(}\frac{2\log
X}{L}{\widehat{h}}(0)+\frac{h(0)}{2}+O\Big{(}\frac{1}{L}\Big{)}\Big{)}.$

Finally if $\ell$ is $q$ times a square, for a prime number $q$, then

(7)
$\sum_{d\in{\mathcal{E}}(\kappa,a)}\Big{(}\sum_{\gamma_{d}}h\Big{(}\frac{\gamma_{d}L}{2\pi}\Big{)}\Big{)}\chi_{d}(\ell)\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}\ll\frac{X}{LN_{0}}\frac{\log
q}{\sqrt{q}}\prod_{p|\ell}\Big{(}1+\frac{1}{p}\Big{)}^{-1}+X^{\frac{1}{2}+\epsilon}\ell^{\frac{1}{2}}e^{L/4}.$

Finally, to understand the distribution of ${\mathcal{P}}(d;x)$ both when $d$
is chosen uniformly over discriminants $d\in{\mathcal{E}}$, and when
$d\in{\mathcal{E}}$ is weighted by contributions from low-lying zeros, we
shall use the method of moments, drawing upon the following proposition.

######  Proposition 3.

Let $k$ be any fixed non-negative integer. Let $X$ be large, and put
$x=X^{1/\log\log\log X}$. Then

(8)
$\sum_{d\in{\mathcal{E}}(\kappa,a)}{\mathcal{P}}(d;x)^{k}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}=\Big{(}\sum_{d\in{\mathcal{E}}(\kappa,a)}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}\Big{)}(\log\log X)^{\frac{k}{2}}(M_{k}+o(1)),$

where $M_{k}$ denotes the $k$-th Gaussian moment:

$M_{k}=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}x^{k}e^{-\frac{x^{2}}{2}}dx=\begin{cases}\frac{k!}{2^{k/2}(k/2)!}&\text{
if }k\text{ is even}\\\ 0&\text{if }k\text{ is odd}.\\\ \end{cases}$

Further, for any parameter $L\geq 1$ with $e^{L}\leq X^{2}$ we have,

$\displaystyle\sum_{d\in{\mathcal{E}}}{\mathcal{P}}(d;x)^{k}$
$\displaystyle\Big{(}\sum_{\gamma_{d}}h\Big{(}\frac{\gamma_{d}L}{2\pi}\Big{)}\Big{)}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}=O(X^{\frac{1}{2}+\epsilon}e^{\frac{L}{4}})$ (9)
$\displaystyle+\frac{X}{N_{0}}\prod_{p\nmid
N_{0}}\Big{(}1-\frac{1}{p^{2}}\Big{)}{\widehat{\Phi}}(0)\Big{(}\frac{2\log
X}{L}{\widehat{h}}(0)+\frac{h(0)}{2}+O\Big{(}\frac{1}{L}\Big{)}\Big{)}(M_{k}+o(1))(\log\log
X)^{\frac{k}{2}}.$

##  3\. Deducing the Theorem from the main propositions

We keep the notations introduced in Section 2. Let $X$ be large, and put
$x=X^{1/\log\log\log X}$.

######  Lemma 1.

Let $\alpha<\beta$ be real numbers. Let $\mathcal{G}_{X}(\alpha,\beta)$ denote
the set of discriminants $d\in{\mathcal{E}}$ with $X\leq|d|\leq 2X$ such that

$\frac{{\mathcal{P}}(d;x)}{\sqrt{\log\log X}}\in(\alpha,\beta),$

and such that there are no zeros $\rho_{d}=\tfrac{1}{2}+i\gamma_{d}$ of
$L(s,E_{d})$ with $|\gamma_{d}|\leq(\log X\log\log X)^{-1}$. Then, for any
$\delta>0$,

$|\mathcal{G}_{X}(\alpha,\beta)|\geq\Big{(}\frac{1}{4}-\delta\Big{)}\Big{(}\frac{1}{\sqrt{2\pi}}\int_{\alpha}^{\beta}e^{-t^{2}/2}dt+o(1)\Big{)}|\\{d\in{\mathcal{E}}:X\leq|d|\leq
2X\\}.$

###### Proof.

Take $\Phi$ to be a smooth approximation to the indicator function of the
interval $[1,2]$, and let $\kappa$ and $a\bmod{N_{0}}$ be as in Section 2. The
first part of Proposition 3 (namely (8)) together with the method of moments
shows that

(10) $\sum_{\begin{subarray}{c}d\in{\mathcal{E}}(\kappa,a)\\\
{\mathcal{P}}(d;x)/\sqrt{\log\log
X}\in(\alpha,\beta)\end{subarray}}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}=\Big{(}\frac{1}{\sqrt{2\pi}}\int_{\alpha}^{\beta}e^{-t^{2}/2}dt+o(1)\Big{)}\Big{(}\sum_{d\in{\mathcal{E}}(\kappa,a)}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}\Big{)}.$

Next, take $h$ to be the Fejer kernel given in (4), and $L=(2-\delta/2)\log
X$. Then the second part of Proposition 3 together with the method of moments
shows that

$\displaystyle\sum_{\begin{subarray}{c}d\in{\mathcal{E}}(\kappa,a)\\\
{\mathcal{P}}(d;x)/\sqrt{\log\log X}\in(\alpha,\beta)\end{subarray}}$
$\displaystyle\sum_{\gamma_{d}}h\Big{(}\frac{\gamma_{d}L}{2\pi}\Big{)}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}=\Big{(}\frac{1}{\sqrt{2\pi}}\int_{\alpha}^{\beta}e^{-t^{2}/2}dt+o(1)\Big{)}\sum_{d\in{\mathcal{E}}(\kappa,a)}\sum_{\gamma_{d}}h\Big{(}\frac{\gamma_{d}L}{2\pi}\Big{)}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}$
$\displaystyle=\Big{(}\frac{1}{\sqrt{2\pi}}\int_{\alpha}^{\beta}e^{-t^{2}/2}dt+o(1)\Big{)}\Big{(}\frac{1}{1-\delta/4}+\frac{1}{2}+o(1)\Big{)}\sum_{d\in{\mathcal{E}}(\kappa,a)}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}.$

Note that the weights $\sum_{\gamma_{d}}h(\gamma_{d}L/(2\pi))$ are always non-
negative, and if $L(s,E_{d})$ has a zero with $|\gamma_{d}|\leq(\log X\log\log
X)^{-1}$ then the weight is $\geq 2+o(1)$ (since there would be a complex
conjugate pair of such zeros, or a double zero at $\frac{1}{2}$). Combining
this with (10), and summing over all the possibilities for $\kappa$ and $a$,
we obtain the lemma. ∎

######  Lemma 2.

The number of discriminants $d\in{\mathcal{E}}$ with $X\leq|d|\leq 2X$ such
that

$\sum_{(\log X\log\log
X)^{-1}\leq|\gamma_{d}|}\log\Big{(}1+\frac{1}{(\gamma_{d}\log
x)^{2}}\Big{)}\geq(\log\log\log X)^{3}$

is $\ll X/\log\log\log X$.

###### Proof.

Applying Proposition 2 with $\ell=1$, $h$ given as in (4), and $1\leq
L\leq(2-\delta)\log X$, we obtain (after summing over the possibilities for
$\kappa$ and $a$)

$\sum_{\begin{subarray}{c}d\in{\mathcal{E}}\\\ X\leq|d|\leq
2X\end{subarray}}\sum_{\gamma_{d}}\Big{(}\frac{\sin(\gamma_{d}L/2)}{\gamma_{d}L/2}\Big{)}^{2}\ll
X\frac{\log X}{L}.$

Integrate both sides of this estimate over $L$ in the range $\log x\leq L\leq
2\log x$. Since, for any $y>0$ and $t\neq 0$,

$\frac{1}{y}\int_{y}^{2y}\Big{(}\frac{\sin(\pi tu)}{\pi
tu}\Big{)}^{2}du\gg\min\Big{(}1,\frac{1}{(ty)^{2}}\Big{)},$

we obtain

$\sum_{\begin{subarray}{c}d\in{\mathcal{E}}\\\ X\leq|d|\leq
2X\end{subarray}}\sum_{\gamma_{d}}\min\Big{(}1,\frac{1}{(\gamma_{d}\log
x)^{2}}\Big{)}\ll X\frac{\log X}{\log x}=X\log\log\log X.$

Now if $|\gamma_{d}|\geq(\log X\log\log X)^{-1}$ then

$\log\Big{(}1+\frac{1}{(\gamma_{d}\log x)^{2}}\Big{)}\ll(\log\log\log
X)\min\Big{(}1,\frac{1}{(\gamma_{d}\log x)^{2}}\Big{)},$

and therefore we may conclude that

$\sum_{\begin{subarray}{c}d\in{\mathcal{E}}\\\ X\leq|d|\leq
2X\end{subarray}}\sum_{(\log X\log\log
X)^{-1}\leq|\gamma_{d}|}\log\Big{(}1+\frac{1}{(\gamma_{d}\log
x)^{2}}\Big{)}\ll X(\log\log\log X)^{2}.$

The lemma follows at once. ∎

With these results in place, it is now a simple matter to deduce the main
theorem. By Proposition 1 1 we know that for $d\in{\mathcal{E}}$ with
$X\leq|d|\leq 2X$

$\log L(\tfrac{1}{2},E_{d})={\mathcal{P}}(d;x)-\tfrac{1}{2}\log\log
X+O(\log\log\log
X)+O\Big{(}\sum_{\gamma_{d}}\log\Big{(}1+\frac{1}{(\gamma_{d}\log
x)^{2}}\Big{)}\Big{)}.$

Lemma 1 tells us that for $d\in\mathcal{G}_{X}(\alpha,\beta)$ we may arrange
for ${\mathcal{P}}(d;x)/\sqrt{\log\log X}$ to lie in the interval
$(\alpha,\beta)$ and for there to be no zeros with $|\gamma_{d}|\leq(\log
X\log\log X)^{-1}$. Lemma 2 now allows us to discard $\ll X/\log\log\log X$
elements of $\mathcal{G}_{X}(\alpha,\beta)$ so as to ensure that the
contribution of zeros with $|\gamma_{d}|\geq(\log X\log\log X)^{-1}$ is
$O((\log\log\log X)^{3})$. Thus there are

$\geq\Big{(}\frac{1}{4}-\delta\Big{)}\Big{(}\frac{1}{\sqrt{2\pi}}\int_{\alpha}^{\beta}e^{-t^{2}/2}dt+o(1)\Big{)}|\\{d\in{\mathcal{E}}:X\leq|d|\leq
2X\\},$

fundamental discriminants $d\in{\mathcal{E}}$ with $X\leq|d|\leq 2X$ for which

$\frac{\log L(\frac{1}{2},E_{d})+\frac{1}{2}\log\log X}{\sqrt{\log\log
X}}+O\Big{(}\frac{(\log\log\log X)^{3}}{\sqrt{\log\log
X}}\Big{)}\in(\alpha,\beta),$

which completes the proof.

##  4\. Proof of Proposition 1

A straight-forward adaptation of Lemma 1 from [14] (itself based on an
identity of Selberg) shows that for any $\sigma\geq\frac{1}{2}$ with
$L(\sigma,E_{d})\neq 0$, and any $x\geq 3$ one has

(11) $-\frac{L^{\prime}}{L}(\sigma,E_{d})=\sum_{n\leq
x}\frac{\Lambda_{E}(n)}{n^{\sigma}}\chi_{d}(n)\frac{\log(x/n)}{\log
x}+\frac{1}{\log
x}\Big{(}\frac{L^{\prime}}{L}\Big{)}^{\prime}(\sigma,E_{d})+\frac{1}{\log
x}\sum_{\rho_{d}}\frac{x^{\rho_{d}-\sigma}}{(\rho_{d}-\sigma)^{2}}+O\Big{(}\frac{1}{x^{\sigma}\log
x}\Big{)}.$

Here $\rho_{d}$ runs over the non-trivial zeros of $L(s,E_{d})$, and this
identity in fact holds unconditionally.

Now assume GRH for $L(s,E_{d})$ and write $\rho_{d}=\tfrac{1}{2}+i\gamma_{d}$.
If $L(\tfrac{1}{2},E_{d})\neq 0$, then integrating both sides of (11) from
$\tfrac{1}{2}$ to $\infty$ yields

$\displaystyle\log L(\tfrac{1}{2},E_{d})=\sum_{n\leq
x}\frac{\Lambda_{E}(n)}{\sqrt{n}\log n}\chi_{d}(n)$
$\displaystyle\frac{\log(x/n)}{\log x}-\frac{1}{\log
x}\frac{L^{\prime}}{L}(\tfrac{1}{2},E_{d})$ (12) $\displaystyle+\frac{1}{\log
x}\sum_{\gamma_{d}}\text{Re}\int_{\frac{1}{2}}^{\infty}\frac{x^{\rho_{d}-\sigma}}{(\rho_{d}-\sigma)^{2}}d\sigma+O\Big{(}\frac{1}{\sqrt{x}(\log
x)^{2}}\Big{)}.$

We may restrict attention to the real part of the integral above since all the
other terms involved are real, or noting that the zeros $\rho_{d}$ appear in
conjugate pairs.

Consider first the sum over $n$ in (4). The contribution from prime powers
$n=p^{k}$ with $k\geq 3$ is plainly $O(1)$. The contribution of the terms
$n=p$ is ${\mathcal{P}}(d;x)+O(1)$, where the error term $O(1)$ arises from
the primes dividing $N_{0}$. Finally, by Rankin–Selberg theory (see for
instance [4]) it follows that

(13) $\sum_{\begin{subarray}{c}p\leq y\\\ p\nmid
N_{0}\end{subarray}}\frac{(\alpha_{p}^{2}+\overline{\alpha_{p}}^{2})\log
p}{p}=\sum_{\begin{subarray}{c}p\leq y\\\ p\nmid
N_{0}\end{subarray}}\frac{(a(p)^{2}-2)\log p}{p}=-\log y+O(1),$

so that, by partial summation, the contribution of the terms $n=p^{2}$ equals

$\sum_{\begin{subarray}{c}p\leq\sqrt{x}\\\ p\nmid
N_{0}\end{subarray}}\frac{(\alpha_{p}^{2}+\overline{\alpha_{p}}^{2})}{2p}\frac{\log(x/p^{2})}{\log
x}+O(1)=\sum_{\begin{subarray}{c}p\leq\sqrt{x}\\\ p\nmid
N_{0}\end{subarray}}\frac{a(p)^{2}-2}{2p}\frac{\log(x/p^{2})}{\log
x}+O(1)=-\frac{1}{2}\log\log x+O(1).$

Thus the contribution of the sum over $n$ in (4) is

(14) ${\mathcal{P}}(d;x)-\tfrac{1}{2}\log\log x+O(1).$

Next we turn to the sum over zeros in (4). If $|\gamma_{d}\log x|\geq 1$, then
note that

$\int_{\frac{1}{2}}^{\infty}\frac{x^{\rho_{d}-\sigma}}{(\rho_{d}-\sigma)^{2}}d\sigma=O\Big{(}\frac{1}{\gamma_{d}^{2}}\int_{\frac{1}{2}}^{\infty}x^{\frac{1}{2}-\sigma}d\sigma\Big{)}=O\Big{(}\frac{1}{\gamma_{d}^{2}\log
x}\Big{)}=O\Big{(}\log x\log\Big{(}1+\frac{1}{\gamma_{d}^{2}(\log
x)^{2}}\Big{)}\Big{)}.$

If $|\gamma_{d}\log x|\leq 1$, then we split into the ranges
$\tfrac{1}{2}\leq\sigma\leq\tfrac{1}{2}+\frac{1}{\log x}$ and larger values of
$\sigma$. The first range contributes

$\displaystyle\int_{\frac{1}{2}}^{\frac{1}{2}+\frac{1}{\log
x}}\text{Re}\frac{x^{\rho_{d}-\sigma}}{(\rho_{d}-\sigma)^{2}}d\sigma$
$\displaystyle=\int_{\frac{1}{2}}^{\frac{1}{2}+\frac{1}{\log
x}}\text{Re}\Big{(}\frac{1}{(\rho_{d}-\sigma)^{2}}+\frac{\log
x}{(\rho_{d}-\sigma)}+O((\log x)^{2})\Big{)}d\sigma$
$\displaystyle=\text{Re}\Big{(}-\frac{1}{i\gamma_{d}}-\frac{1}{1/\log
x-i\gamma_{d}}+\log x\log\frac{-i\gamma_{d}}{1/\log x-i\gamma_{d}}+O(\log
x)\Big{)}$ $\displaystyle=O\Big{(}\log
x\log\Big{(}1+\frac{1}{\gamma_{d}^{2}(\log x)^{2}}\Big{)}\Big{)},$

while the second range contributes

$\ll\int_{\frac{1}{2}+\frac{1}{\log
x}}^{\infty}\frac{x^{\frac{1}{2}-\sigma}}{(\frac{1}{2}-\sigma)^{2}}d\sigma\ll\log
x=O\Big{(}\log x\log\Big{(}1+\frac{1}{\gamma_{d}^{2}(\log
x)^{2}}\Big{)}\Big{)}.$

Thus in all cases the sum over zeros in (4) is

(15) $O\Big{(}\log x\log\Big{(}1+\frac{1}{\gamma_{d}^{2}(\log
x)^{2}}\Big{)}\Big{)}.$

Finally, taking logarithmic derivatives in the functional equation we find
that

$\frac{L^{\prime}}{L}(\tfrac{1}{2},E_{d})=-\log(\sqrt{N}|d|)+O(1).$

The proposition follows upon combining this with (4), (14), and (15).

##  5\. Proof of Proposition 2

The proof of Proposition 2 is based on the explicit formula, which we first
recall in our context.

######  Lemma 3.

Let $h$ be a function with $h(x)\ll(1+x^{2})^{-1}$ and with compactly
supported Fourier transform
${\widehat{h}}(\xi)=\int_{-\infty}^{\infty}h(t)e^{-2\pi i\xi t}dt$. Then, for
any fundamental discriminant $d\in{\mathcal{E}}$

$\displaystyle\sum_{\gamma_{d}}h\Big{(}\frac{\gamma_{d}}{2\pi}\Big{)}=\frac{1}{2\pi}\int_{-\infty}^{\infty}h\Big{(}\frac{t}{2\pi}\Big{)}$
$\displaystyle\Big{(}\log\frac{Nd^{2}}{(2\pi)^{2}}+2\text{Re}\frac{\Gamma^{\prime}}{\Gamma}(1+it)\Big{)}dt$
$\displaystyle-\sum_{n}\frac{\Lambda_{E}(n)}{\sqrt{n}}\chi_{d}(n)\Big{(}{\widehat{h}}(\log
n)+{\widehat{h}}(-\log n)\Big{)},$

where the sum is over all ordinates of non-trivial zeros $1/2+i\gamma_{d}$ of
$L(s,E_{d})$.

Applying the explicit formula to the dilated function $h_{L}(x)=h(xL)$ whose
Fourier transform is $\frac{1}{L}{\widehat{h}}(x/L)$, we obtain

$\displaystyle\sum_{\gamma_{d}}h\Big{(}\frac{\gamma_{d}L}{2\pi}\Big{)}=\frac{1}{2\pi}\int_{-\infty}^{\infty}h\Big{(}\frac{tL}{2\pi}\Big{)}$
$\displaystyle\Big{(}\log\frac{Nd^{2}}{(2\pi)^{2}}+2\text{Re}\frac{\Gamma^{\prime}}{\Gamma}(1+it)\Big{)}dt$
(16)
$\displaystyle-\frac{1}{L}\sum_{n}\frac{\Lambda_{E}(n)}{\sqrt{n}}\chi_{d}(n)\Big{(}{\widehat{h}}\Big{(}\frac{\log
n}{L}\Big{)}+{\widehat{h}}\Big{(}-\frac{\log n}{L}\Big{)}\Big{)}.$

We multiply this expression by $\chi_{d}(\ell)$ and sum over $d$ with suitable
weights. Thus we find

(17)
$\sum_{d\in{\mathcal{E}}(\kappa,a)}\sum_{\gamma_{d}}h\Big{(}\frac{\gamma_{d}L}{2\pi}\Big{)}\chi_{d}(\ell)\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}=S_{1}-S_{2},$

where

(18)
$S_{1}=\frac{1}{2\pi}\int_{-\infty}^{\infty}h\Big{(}\frac{tL}{2\pi}\Big{)}\sum_{d\in{\mathcal{E}}(\kappa,a)}\chi_{d}(\ell)\Big{(}\log\frac{Nd^{2}}{(2\pi)^{2}}+2\text{Re
}\frac{\Gamma^{\prime}}{\Gamma}(1+it)\Big{)}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}dt,$

and

(19)
$S_{2}=\frac{1}{L}\sum_{n}\frac{\Lambda_{E}(n)}{\sqrt{n}}\Big{(}{\widehat{h}}\Big{(}\frac{\log
n}{L}\Big{)}+{\widehat{h}}\Big{(}-\frac{\log
n}{L}\Big{)}\Big{)}\sum_{d\in{\mathcal{E}}(\kappa,a)}\chi_{d}(\ell
n)\Phi\Big{(}\frac{\kappa d}{X}\Big{)}.$

The term $S_{1}$ is relatively easy to handle. If $\ell$ is a square, it
amounts to counting square-free integers $d$ lying in a suitable progression
$\bmod{N_{0}}$ and coprime to $\ell$. While if $\ell$ is not a square, the
resulting sum is a non-trivial character sum, which exhibits substantial
cancellation. A more general term of this type is handled in Proposition 1 of
[7], which we refer to for a detailed proof. Thus when $\ell$ is not a square
we find

(20) $S_{1}=O(X^{\frac{1}{2}+\epsilon}\sqrt{\ell}),$

while if $\ell$ is a square

(21) $\displaystyle S_{1}$
$\displaystyle=\frac{X}{N_{0}}\prod_{p|\ell}\Big{(}1+\frac{1}{p}\Big{)}^{-1}\prod_{p\nmid
N_{0}}\Big{(}1-\frac{1}{p^{2}}\Big{)}{\check{\Phi}}(0)(2\log
X+O(1))\frac{{\widehat{h}}(0)}{L}+O(X^{\frac{1}{2}+\epsilon}\sqrt{\ell}).$

We now turn to the more difficult term $S_{2}$. First we dispose of terms $n$
(which we may suppose is a prime power) that have a common factor with
$N_{0}$. Note that since $d$ is fixed in a residue class $\bmod{N_{0}}$, if
$n$ is the power of a prime dividing $N_{0}$ then $\chi_{d}(n)$ is determined
by the congruence condition on $d$. Thus the contribution of these terms is

(22)
$\ll\frac{1}{L}\sum_{(n,N_{0})>1}\frac{\Lambda(n)}{\sqrt{n}}\Big{|}\sum_{d\in{\mathcal{E}}(\kappa,a)}\chi_{d}(\ell)\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}\Big{|}\ll\delta(\ell=\square)\frac{X}{L}+X^{\frac{1}{2}+\epsilon}\sqrt{\ell},$

where $\delta(\ell=\square)$ denotes $1$ when $\ell$ is a square, and $0$
otherwise.

Henceforth we restrict attention to the terms in $S_{2}$ where $(n,N_{0})=1$.
Note that if $d\equiv a\bmod{N_{0}}$ then $d$ is automatically $1\bmod 4$, and
the condition that $d$ is a fundamental discriminant amounts to $d$ being
square-free. We express the square-free condition by Möbius inversion
$\sum_{\alpha^{2}|d}\mu(\alpha)$, and then split the sum into the cases where
$\alpha>A$ is large, and when $\alpha\leq A$ is small, for a suitable
parameter $A\leq X$. We first handle the case when $\alpha>A$ is large. These
terms give

$\displaystyle\sum_{\alpha>A}\mu(\alpha)\sum_{\begin{subarray}{c}d\equiv
a\bmod N_{0}\\\ \alpha^{2}|d\end{subarray}}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}\frac{1}{L}\sum_{(n,N_{0})=1}\frac{\Lambda_{E}(n)}{\sqrt{n}}\Big{(}{\widehat{h}}\Big{(}\frac{\log
n}{L}\Big{)}+{\widehat{h}}\Big{(}-\frac{\log n}{L}\Big{)}\Big{)}\chi_{d}(\ell
n)$ (23) $\displaystyle\ll\sum_{\alpha>A}\sum_{\begin{subarray}{c}d\equiv
a\bmod N_{0}\\\ \alpha^{2}|d\end{subarray}}\Phi\Big{(}\frac{\kappa
d}{X}\Big{)}(\log X)\ll\frac{X}{N_{0}A}\log X,$

upon using GRH to estimate the sum over $n$ and then estimating the sum over
$d$ trivially.

We are left with the terms with $\alpha\leq A$, and writing $d=k\alpha^{2}$ we
may express these terms as

(24)
$\frac{1}{L}\sum_{(n,N_{0})=1}\frac{\Lambda_{E}(n)}{\sqrt{n}}\Big{(}{\widehat{h}}\Big{(}\frac{\log
n}{L}\Big{)}+{\widehat{h}}\Big{(}-\frac{\log
n}{L}\Big{)}\Big{)}\sum_{\begin{subarray}{c}\alpha\leq A\\\ (\alpha,n\ell
N_{0})=1\end{subarray}}\mu(\alpha)\sum_{\begin{subarray}{c}k\equiv
a\overline{\alpha^{2}}\bmod{N_{0}}\end{subarray}}\Big{(}\frac{k}{\ell
n}\Big{)}\Phi\Big{(}\frac{\kappa k\alpha^{2}}{X}\Big{)}.$

We now apply the Poisson summation formula to the sum over $k$ above, as in
Lemma 7 of [7]. This transforms the sum over $k$ above to

(25) $\frac{X}{n\ell
N_{0}\alpha^{2}}\Big{(}\frac{N_{0}}{n\ell}\Big{)}\sum_{v}e\Big{(}\frac{va\overline{\alpha^{2}n\ell}}{N_{0}}\Big{)}\tau_{v}(n\ell){\widehat{\Phi}}\Big{(}\frac{Xv}{n\ell\alpha^{2}N_{0}}\Big{)},$

where $\tau_{v}(n\ell)$ is a Gauss sum given by

$\tau_{v}(n\ell)=\sum_{b\bmod{n\ell}}\Big{(}\frac{b}{n\ell}\Big{)}e\Big{(}\frac{vb}{n\ell}\Big{)}.$

The Gauss sum $\tau_{v}(n\ell)$ can be described explicitly, see Lemma 6 of
[7] which gives an evaluation of

$G_{v}(n\ell)=\Big{(}\frac{1-i}{2}+\Big{(}\frac{-1}{n\ell}\Big{)}\frac{1+i}{2}\Big{)}\tau_{v}(n\ell),$

from which $\tau_{v}(n\ell)$ may be obtained via

(26)
$\tau_{v}(n\ell)=\Big{(}\frac{1+i}{2}+\Big{(}\frac{-1}{n\ell}\Big{)}\frac{1-i}{2}\Big{)}G_{v}(n\ell).$

The term $v=0$ in (25) leads to a main term; we postpone its treatment, and
first consider the contribution of terms $v\neq 0$. Since ${\widehat{h}}$ is
supported in $[-1,1]$, we may suppose that $n\leq e^{L}$. The rapid decay of
the Fourier transform ${\widehat{\Phi}}(\xi)$ allows us to restrict attention
to the range $|v|\leq\ell e^{L}A^{2}X^{-1+\epsilon}$, with the total
contribution to $S_{2}$ of terms with larger $|v|$ being estimated by $O(1)$.
For the smaller values of $v$, we interchange the sums over $v$, performing
first the sum over $n$ using GRH. Thus these terms contribute

$\displaystyle\frac{X}{\ell LN_{0}}$ $\displaystyle\sum_{0<|v|\leq\ell
e^{L}A^{2}X^{-1+\epsilon}}\sum_{\begin{subarray}{c}\alpha\leq A\\\
(\alpha,\ell N_{0})=1\end{subarray}}\frac{\mu(\alpha)}{\alpha^{2}}$
$\displaystyle\sum_{(n,\alpha
N_{0})=1}\frac{\Lambda_{E}(n)}{n\sqrt{n}}\Big{(}\frac{N_{0}}{n\ell}\Big{)}e\Big{(}\frac{va\overline{\alpha^{2}n\ell}}{N_{0}}\Big{)}\tau_{v}(n\ell)\Big{(}{\widehat{h}}\Big{(}\frac{\log
n}{L}\Big{)}+{\widehat{h}}\Big{(}-\frac{\log
n}{L}\Big{)}\Big{)}{\widehat{\Phi}}\Big{(}\frac{Xv}{n\ell\alpha^{2}N_{0}}\Big{)}.$

We now claim that (on GRH) the sum over $n$ above is

(27) $\ll\frac{\alpha\ell^{\frac{3}{2}}}{\sqrt{X|v|}}X^{\epsilon},$

so that the contribution of the terms with $v\neq 0$ is

(28) $\ll X^{\frac{1}{2}+\epsilon}\ell^{\frac{1}{2}}\sum_{1\leq|v|\leq\ell
e^{L}A^{2}X^{-1+\epsilon}}|v|^{-\frac{1}{2}}\log A\ll\ell
e^{L/2}AX^{\epsilon}.$

To minimize the combined contributions of the error terms in (28) and (5), we
shall choose $A=(X/\ell)^{\frac{1}{2}}e^{-\frac{L}{4}}$, so that the effect of
both these error terms is

(29) $\ll X^{\frac{1}{2}+\epsilon}\ell^{\frac{1}{2}}e^{\frac{L}{4}}.$

To justify the claim (27) we first use (26) to replace $\tau_{v}(n\ell)$ by
$G_{v}(n\ell)$ so that we must bound (for both choices of $\pm$)

$\sum_{(n,\alpha N_{0})=1}\frac{\Lambda_{E}(n)}{n\sqrt{n}}\Big{(}\frac{\pm
N_{0}}{n\ell}\Big{)}e\Big{(}\frac{va\overline{\alpha^{2}n\ell}}{N_{0}}\Big{)}G_{v}(n\ell)\Big{(}{\widehat{h}}\Big{(}\frac{\log
n}{L}\Big{)}+{\widehat{h}}\Big{(}-\frac{\log
n}{L}\Big{)}\Big{)}{\widehat{\Phi}}\Big{(}\frac{Xv}{n\ell\alpha^{2}N_{0}}\Big{)}.$

First consider the generic case when $n$ is a prime power with $(n,v)=1$. Here
(using Lemma 6 of [7]) $G_{v}(n\ell)=0$ unless $n$ is a prime $p$ not dividing
$\ell$ in which case $G_{v}(p\ell)=(\frac{v}{p})\sqrt{p}G_{v}(\ell)$. Thus
such terms contribute to the above

$\Big{(}\frac{\pm N_{0}}{\ell}\Big{)}G_{v}(\ell)\sum_{p\nmid\alpha v\ell
N_{0}}\frac{\Lambda_{E}(p)}{p}\Big{(}\frac{\pm
vN_{0}}{p}\Big{)}e\Big{(}\frac{va\overline{\alpha^{2}p\ell}}{N_{0}}\Big{)}\Big{(}{\widehat{h}}\Big{(}\frac{\log
p}{L}\Big{)}+{\widehat{h}}\Big{(}-\frac{\log
p}{L}\Big{)}\Big{)}{\widehat{\Phi}}\Big{(}\frac{Xv}{p\ell\alpha^{2}N_{0}}\Big{)}$

The rapid decay of ${\widehat{\Phi}}(\xi)$ implies that we may restrict
attention above to the range $p>X^{1-\epsilon}|v|/(\ell\alpha^{2}N_{0})$. Then
splitting $p$ into progressions $\bmod N_{0}$ and using GRH (it is here that
we need GRH for twists of $L(s,E)$ by quadratic characters, as well as all
Dirichlet characters modulo $N_{0}$) we obtain the bound

$\ll|G_{v}(\ell)|\frac{X^{\epsilon}\ell^{\frac{1}{2}}\alpha
N_{0}}{\sqrt{X|v|}}\ll\frac{\ell^{\frac{3}{2}}\alpha
X^{\epsilon}}{\sqrt{X|v|}},$

which is in keeping with (27). Now consider the non-generic case when $n$ is
the power of some prime dividing $v$. We may assume that $n|v^{2}$ (else
$G_{v}(n\ell)=0$ by Lemma 6 of [7]) and also that $n\geq
X^{1-\epsilon}|v|/(\ell\alpha^{2}N_{0})$ else the Fourier transform
$\widehat{\Phi}$ is negligible. Using that
$|G_{v}(n\ell)|\leq(v,n\ell)^{\frac{1}{2}}(n\ell)^{\frac{1}{2}}\leq(|v|n\ell)^{\frac{1}{2}}$
(which again follows from Lemma 6 of [7]) we may bound the contribution of
these terms by

$\ll\sum_{n|v^{2}}\Lambda(n)\frac{(|v|\ell)^{\frac{1}{2}}}{(X^{1-\epsilon}v/(\ell\alpha^{2}N_{0}))}\ll(\log
v)X^{\epsilon}\frac{\ell^{\frac{3}{2}}\alpha^{2}}{X\sqrt{|v|}}\ll\frac{\ell^{\frac{3}{2}}\alpha
X^{\epsilon}}{\sqrt{X|v|}},$

since $\log v\ll\log X\ll X^{\epsilon}$ and $\alpha\leq A\leq\sqrt{X}$. Thus
these terms also satisfy the claimed bound (27).

Now we handle the main term contribution from $v=0$, noting that
$\tau_{0}(n\ell)=0$ unless $n\ell$ is a square, in which case it equals
$\phi(n\ell)$. Thus the main term contribution from $v=0$ is

$\frac{X}{LN_{0}}\sum_{\begin{subarray}{c}(n,N_{0})=1\\\
n\ell=\square\end{subarray}}\frac{\Lambda_{E}(n)}{\sqrt{n}}\frac{\phi(n\ell)}{n\ell}\Big{(}\sum_{\begin{subarray}{c}\alpha\leq
A\\\ (\alpha,\ell
nN_{0})=1\end{subarray}}\frac{\mu(\alpha)}{\alpha^{2}}\Big{)}{\widehat{\Phi}}(0)\Big{(}{\widehat{h}}\Big{(}\frac{\log
n}{L}\Big{)}+{\widehat{h}}\Big{(}-\frac{\log n}{L}\Big{)}\Big{)}.$

Thus this main term only exists if $\ell$ is a square (so that $n$ is a
square), or if $\ell$ is $q$ times a square for a unique prime $q$ (so that
$n$ is an odd power of $q$). In the case $\ell$ is a square, writing $n=m^{2}$
and performing the sum over $\alpha$, we obtain that the main term is

$\frac{X}{LN_{0}}{\widehat{\Phi}}(0)\sum_{(m,N_{0})=1}\frac{\Lambda_{E}(m^{2})}{m}\Big{(}\prod_{p|m\ell}\Big{(}1+\frac{1}{p}\Big{)}^{-1}\prod_{p\nmid
N_{0}}\Big{(}1-\frac{1}{p^{2}}\Big{)}+O\Big{(}\frac{1}{A}\Big{)}\Big{)}\Big{(}{\widehat{h}}\Big{(}\frac{2\log
m}{L}\Big{)}+{\widehat{h}}\Big{(}-\frac{2\log m}{L}\Big{)}\Big{)}.$

Using (13) and partial summation we conclude that the main term when $\ell$ is
a square is

$\displaystyle-\frac{X}{LN_{0}}{\widehat{\Phi}}(0)$
$\displaystyle\Big{(}\prod_{p|\ell}\Big{(}1+\frac{1}{p}\Big{)}^{-1}\prod_{p\nmid
N_{0}}\Big{(}1-\frac{1}{p^{2}}\Big{)}+O\Big{(}\frac{1}{A}\Big{)}\Big{)}\Big{(}\int_{1}^{\infty}\Big{(}\widehat{h}\Big{(}\frac{2\log
y}{L}\Big{)}+{\widehat{h}}\Big{(}-\frac{2\log
y}{L}\Big{)}\Big{)}\frac{dy}{y}+O(1)\Big{)}$ (30) $\displaystyle=$
$\displaystyle-\frac{X}{N_{0}}{\widehat{\Phi}}(0)\frac{{h}(0)}{2}\prod_{p|\ell}\Big{(}1+\frac{1}{p}\Big{)}^{-1}\prod_{p\nmid
N_{0}}\Big{(}1-\frac{1}{p^{2}}\Big{)}+O\Big{(}\frac{X}{A}+\frac{X}{L}\Big{)}.$

Suppose now that $\ell$ is $q$ times a square, for a (unique) prime $q$. Here
the main term may be bounded by

(31) $\ll\frac{X}{LN_{0}}\frac{\log
q}{\sqrt{q}}\prod_{p|\ell}\Big{(}1+\frac{1}{p}\Big{)}^{-1}\prod_{p\nmid
N_{0}}\Big{(}1-\frac{1}{p^{2}}\Big{)};$

naturally we can be more precise here, but this bound suffices.

##  6\. Proof of Proposition 3

The $k$-th moment in (8) is treated in Proposition 6 of [7]. Briefly,
expanding out ${\mathcal{P}}(d;x)^{k}$ we must handle

$\sum_{\begin{subarray}{c}p_{1},\ldots,p_{k}\leq x\\\ p_{i}\nmid
N_{0}\end{subarray}}\frac{a(p_{1})\cdots a(p_{k})}{\sqrt{p_{1}\cdots
p_{k}}}\sum_{d\in{\mathcal{E}}(\kappa,a)}\chi_{d}(p_{1}\cdots
p_{k})\Phi\Big{(}\frac{\kappa d}{X}\Big{)}.$

When $p_{1}\cdots p_{k}$ is not a perfect square, the sum over $d$ exhibits
substantial cancellation (as mentioned earlier in (21)). The main term arises
from terms where $p_{1}\cdots p_{k}$ is a perfect square, which cannot happen
when $k$ is odd. When $k$ is even, the contribution to the main term comes
essentially from the case when there are $k/2$ distinct primes among $p_{1}$,
$\ldots$, $p_{k}$ with each distinct prime appearing twice. The number of such
pairings leads to the coefficient $M_{k}$, and Rankin-Selberg theory is used
to obtain $\sum_{p\leq x}a(p)^{2}/p=\log\log x+O(1)\sim\log\log X$.

To establish (3), once again we expand ${\mathcal{P}}(d;x)^{k}$ and are faced
with evaluating

$\sum_{\begin{subarray}{c}p_{1},\ldots,p_{k}\leq x\\\ p_{i}\nmid
N_{0}\end{subarray}}\frac{a(p_{1})\cdots a(p_{k})}{\sqrt{p_{1}\cdots
p_{k}}}\sum_{d\in{\mathcal{E}}(\kappa,a)}\chi_{d}(p_{1}\cdots
p_{k})\Big{(}\sum_{\gamma_{d}}h\Big{(}\frac{\gamma_{d}L}{2\pi}\Big{)}\Big{)}.$

We now appeal to Proposition 2. The terms where $p_{1}\cdots p_{k}$ is neither
a square nor a prime times a square contribute, using (5),

$\ll X^{\frac{1}{2}+\epsilon}e^{\frac{L}{4}}\sum_{p_{1},\ldots,p_{k}\leq
x}1\ll X^{\frac{1}{2}+\epsilon}e^{\frac{L}{4}}.$

It remains to consider the cases when this product is a square (which can only
happen when $k$ is even) and when it is a prime times a square (which can only
happen for odd $k$). In the first case, we obtain (by (2)) a main term

$\frac{X}{N_{0}}\prod_{p\nmid
N_{0}}\Big{(}1-\frac{1}{p^{2}}\Big{)}{\widehat{\Phi}}(0)\Big{(}\frac{2\log
X}{L}{\widehat{h}}(0)+\frac{h(0)}{2}+O\Big{(}\frac{1}{L}\Big{)}\Big{)}\sum_{\begin{subarray}{c}p_{1},\ldots,p_{k}\leq
x\\\ p_{i}\nmid N_{0}\\\ p_{1}\cdots
p_{k}=\square\end{subarray}}\frac{a(p_{1})\cdots a(p_{k})}{\sqrt{p_{1}\cdots
p_{k}}}\prod_{p|p_{1}\cdots p_{k}}\Big{(}1+\frac{1}{p}\Big{)}^{-1}.$

As before, this main term is dominated by the contribution of terms where
there are $k/2$ distinct primes among $p_{1}$, $\ldots$, $p_{k}$ each
appearing twice, and thus we obtain

$\frac{X}{N_{0}}\prod_{p\nmid
N_{0}}\Big{(}1-\frac{1}{p^{2}}\Big{)}{\widehat{\Phi}}(0)\Big{(}\frac{2\log
X}{L}{\widehat{h}}(0)+\frac{h(0)}{2}+O\Big{(}\frac{1}{L}\Big{)}\Big{)}(M_{k}+o(1))(\log\log
X)^{\frac{k}{2}}.$

This establishes the result (3) for the case $k$ even. When $k$ is odd, the
contribution of the terms when $p_{1}\cdots p_{k}$ is a prime times a square
may be bounded by (using (7) of Proposition 2)

$\ll\frac{X}{LN_{0}}\sum_{q\leq x}\frac{\log
q}{q}\Big{(}\sum_{\begin{subarray}{c}p\leq x\\\ p\nmid
N_{0}\end{subarray}}\frac{a(p)^{2}}{p}\Big{)}^{\frac{k-1}{2}}\ll\frac{X}{N_{0}}\frac{\log
x}{L}(\log\log X)^{\frac{k-1}{2}}\ll\frac{X}{N_{0}}(\log\log
X)^{\frac{k-1}{2}},$

which establishes (3) since $M_{k}=0$ here.

## References

  * [1] H. M. Bui, N. Evans, S. Lester, and K. Pratt.  Weighted central limit theorems for central values of L-functions.  arXiv:2109.06829, 2021.
  * [2] G. Chinta.  Analytic ranks of elliptic curves over cyclotomic fields.  J. Reine Angew. Math., 544:13–24, 2002.
  * [3] D. R. Heath-Brown.  The average analytic rank of elliptic curves.  Duke Math. J., 122(3):591–623, 2004.
  * [4] H. Iwaniec and E. Kowalski.  Analytic number theory, volume 53 of American Mathematical Society Colloquium Publications.  American Mathematical Society, Providence, RI, 2004.
  * [5] H. Iwaniec, W. Luo, and P. Sarnak.  Low lying zeros of families of $L$-functions.  Inst. Hautes Études Sci. Publ. Math., (91):55–131, 2000.
  * [6] J. P. Keating and N. C. Snaith.  Random matrix theory and $L$-functions at $s=1/2$.  Comm. Math. Phys., 214(1):91–110, 2000.
  * [7] M. Radziwiłł and K. Soundararajan.  Moments and distribution of central $L$-values of quadratic twists of elliptic curves.  Invent. math., 202(3):1029–1068, 2015.
  * [8] M. Radziwiłł and K. Soundararajan.  Selberg’s central limit theorem for $\log{|\zeta(1/2+it)|}$.  Enseign. Math., 63(1-2):1–19, 2017.
  * [9] D. E. Rohrlich.  On $L$-functions of elliptic curves and anticyclotomic towers.  Invent. math., 75(3):383–408, 1984.
  * [10] D. E. Rohrlich.  On $L$-functions of elliptic curves and cyclotomic towers.  Invent. math., 75(3):409–423, 1984.
  * [11] A. Selberg.  Contributions to the theory of the Riemann zeta-function.  Arch. Math. Naturvid., 48(5):89–155, 1946.
  * [12] A. Selberg.  Old and new conjectures and results about a class of Dirichlet series.  In Proceedings of the Amalfi Conference on Analytic Number Theory (Maiori, 1989), pages 367–385. Univ. Salerno, Salerno, 1992\.
  * [13] A. Smith.  $2^{\infty}$-Selmer groups, $2^{\infty}$-class groups, and Goldfeld’s conjecture.  arXiv:1702.02325, 2017.
  * [14] K. Soundararajan.  Moments of the Riemann zeta function.  Ann. of Math. (2), 170(2):981–993, 2009.